Riddle me on this one

[Christopher Browne]

On Sat, Jan 21, 2012 at 10:30 PM, Yanni Chiu <yanni-bJEeYj9oJeDQT0dZR+AlfA at public.gmane.org> wrote:
> On 21/01/12 10:12 PM, Christopher Browne wrote:
>>
>> a) You're actually out of space, there's no place for the rotation
>> process to do its work.
>
> Couldn't the rotation process use file space from a different disk partition
> to accomplish its work?

Sure, but the complexity of the mechanism just went up, and you're now
injuring another filesystem (by chewing up its space).

And the fact that you're needing to switch filesystems because one of
the files is filling it up seems to indicate that the log rotation
system wasn't functioning in the first place.

Any time I have had such problems with log files filling things up,
the trouble has been that the offending file wasn't being rotated due
to some failure of either configuration or of the functioning of the
log rotator.

It doesn't much help to enhance the log rotator to grasp at additional
straws if it's not running...

>> b) If the only remaining file descriptor is held by some live process,
>> and the file is otherwise unlinked, log rotation will also do you no
>> good, as it has no way of getting at the file to suck data out of it.
>
>
> Would this work:
> - attach to live process with a debugger
> - figure out which file descriptor corresponded to the log file
> - poke a program into memory at some innocuous location
> - the program would copy the log file to another file (on another disk
> partition)
> - then the original log file could be truncated

Something like that.  I imagine that some "rooty" process could turn
an inode reference back into a file.  By downloading the sources to
logrotate, I think I've done my system call investigations for the
day; I'll leave that exercise to someone else :-).

But I also step back to the backup analogy...  It's too late to start
worrying about backups *after* your disk fails.  It'll take heroic
efforts to rectify the log file problem after the fact.

Most likely, it's sufficient to kill the offending process, "echo '' >
some_extraordinarily_large_file.log", and make sure that the file gets
added to the rotation regimen for next time.
-- 
When confronted by a difficult problem, solve it by reducing it to the
question, "How would the Lone Ranger handle this?"
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists