[TriLUG] bash help
slitt at troubleshooters.com
Thu Oct 28 11:20:01 EDT 2004
On Thursday 28 October 2004 10:11 am, rwshep2000.2725323 at bloglines.com wrote:
> I have a server with a shared repository for files. I plan to devote
> 70GB of an 80GB HD (a single data partition) to the files. The files are
> uploaded and placed in the repository via a web application. Here is what
> I'd like to accomplish:
> When directory size exceeds 70GB, delete files,
> First-In-First-Out, until the repository is pared back to 70GB.
> The best
> case scenario would be to pare back the files each time a new file is
> added. However, I am hoping to do this without adding web application
> logic, which could cause additional latency for the user. Although it
> risks possibly exceeding the size limit, I am thinking of using a bash
> script scheduled with cron. To ensure against exceeding the limit, I'm
> leaving 10GB of the 80GB as buffer. I know this is imperfect but my humble
> intellect can't think of another approach.
> So I'm looking for comments on two things:
> 1. How to make a bash script
> look at total directory size,
> then proceed to delete files FIFO until a
> target size is reached;
Loop: find the oldest, delete it, check disk space
Personally, I'd write this thing in Perl
> 2. Whether there is a better alternative than putting
> this script on cron.
Put the disk space check in your webapp logic. If disk space is OK, your user
has endured one if statement's worth of latency. If disk space is moderately
low (still have 10 GB), have it run the deletion part in background, and your
user has endured latency of one if statement and one background spawn. If
disk space is critically low, throw up a page telling him to wait for disk
maintenance, and run the deletion in the forground. User endures big latency,
but at least his work doesn't get garbled. Presumably this won't happen
often, because it will be caught before it gets critical.
I think I'd build in hysterisis like a thermostat. Run the deletion program at
7GB free, and have it shut off at 14GB free. That way it won't run too often,
and won't oscillate as new files get written during its lifetime. Also, as
the deletion program starts, have it set a flag so that no other deletion
program starts during its lifetime. Upon the deletion program's termination
(all termination points), have it unset the flag so other deletion programs
can be run.
Once again, if it were me, I'd write the program in Perl or Python, whether
the program is spawned as a cron job or from the web app.
Founder and acting president: GoLUG
More information about the TriLUG