Riddle me on this one

[Michael Galea]

On 01/21/12 23:30, D. Hugh Redelmeier wrote:
> | From: Christopher Browne<cbbrowne-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org>
>
> | On Sat, Jan 21, 2012 at 5:48 PM, D. Hugh Redelmeier<hugh-pmF8o41NoarQT0dZR+AlfA at public.gmane.org>  wrote:
> | Decidedly a fuller explanation than I gave...
> |
> |>  The logging system has ways for rolling over logs.  I don't remember
> |>  them, but they are what you want to use if you space is eaten by a log
> |>  file.
> |
> | Actually, I would characterize it in something of an opposite way...
> |
> | Log rotators switch over the files so that the log data gets 'sucked
> | out' into a secondary file, and then truncates it from the "main"
> | file.
>
> Of course you are right: log rotation doesn't generally reduce space
> requirements NOW.  It might in the case where it deletes the oldest logs
> and those logs are large.  Compression would help, but again, not NOW
> (during compression, space is needed for both the original and the
> compressed file).
>
There are a few scenarios in which log rotation is an answer.  You can 
boost the rate of log rotation so that more closely tracks the target 
you wish to enforce for /var/logs.  You can override individual targets 
for maximum log sizes specified in logrotate.d by "forcing" the rotate.
You can make the metric for rotating the size of the directory as 
opposed to the size on any individual file in it.

You should also ensure your daemons are well behaved and log through 
syslog, do not allow them to log directly to files in /var/log.
This allows you to close open files by restarting syslog.

But at this point you have move far away the standard use of logrotate 
in Linux for desktops.  You do these sorts of things on small devices 
such as embedded routers because you value system stability, as judged 
by the logging footprint, to be less important then the useful 
information contained in the logs. The way to get the best of both 
worlds is to export logging using remote syslog, possibly leaving copies 
of a few of the more important logs on the router, for local debugging.

> What I was really thinking (but didn't say) is that any log rotation
> system must solve the problem of getting the log file closed for the
> rotation to take effect.
>
>
> You really have to go back to how programs log things.  I think
> that they are all creatively different.
>
> If they use syslog(3), the actual file writing will be done by
> syslogd(8).  So it is the only program that needs to be kicked to
> close the log file (&  likely reopen a different file with the same
> name).  According to the manpages on my system, a SIGHUP to syslogd
> should do the trick.
>
> So it seems to me (which means: I have made this all up), log rotation
> would be: rename all log file, possibly deleting oldest ones and
> compressing less old ones, and signal syslogd with a SIGHUP to have it
> close and reopen the log files.
>
> This may be optimistic: my suspicion is that logging daemons is where
> there has been some creativity in the distro worlds.  Furthermore, I
> think that many programs think that they know how to log better than
> syslog(8) provides.
>
> The logrotate(8) example /etc/logrotate.conf certainly shows that
> several things (eg.  apache) do their own logging.  How you discover
> this for each application of interest seems like a mess.
>
>
> Sidenote: many programmers don't know how to correctly handle signals
> in C.  In particular, the only safe thing to do in a signal handler is
> to set a flag and return.  The program must then check that flag
> frequently and act accordingly when the flag is found to be set.  For
> example, years ago, I could not convince the main author of PINE that
> doing too much in a signal handler was why PINE was breaking on
> Solaris.  He blamed Solaris.


-- 
Michael Galea
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists