Actually, you can replace Java with your favorite programming languages such as C or C#.
I was given a task to look after some house-keeping jobs. We realized that the jobs are taking very long to complete and in some occasions, it exceeded the allocated time frame and had to be aborted. Our disk-space quota were getting exhausted quickly. The house-keeping jobs are pretty simple & straight forward. They look into various directories and delete outdated log and temporary files. Some jobs move output files to another directory so that a backup could be done.
Our initial suspicions were that as the system grew, our output were getting larger and hence it took more time to complete the jobs. This was correct after inspecting the log files sizes and quantity. As the system grew in complexity, developers added more logs to it. More output files were also generated.
On our preliminary discussions, we decided to cut down on unnecessary logging. We saw two advantages from this: reduce in log files quantity and size; improvement in system performance as logging requires file I/O and thats expensive. After doing a few rounds of reduction, we hit a plateau. Our house-keeping jobs still failed occasionally.
After more discussions, I looked into the house-keeping jobs codes to see if they could be optimized. Somehow it dawned upon me that why were we using Java for file I/O when we could just use the OS commands for it?! So I picked a job and wrote a shell script for it. The script looked for files older than 90 days and would delete them. The old job was paused. A new job was added to the CRON and tested. It worked fine. I then move on to rewrite jobs for the rest of the house-keeping.
I learnt a couple of lessons from this. Java (or whatever programming languages you’re good at) isn’t the silver bullet for every single situation. Use the right tool for the right problem. File I/O is definitely faster if we could use the native OS commands. In actual fact, we had lesser codes to maintain since the house-keeping jobs became shell scripts. My guess for such an initial approach was: lack of team discussion to come up with the appropriate solution; developer too comfortable with Java and had forgotten that there’s always the OS commands; or maybe unfamiliarity with OS commands.
to*OT 