good console bases sys monitors

[Rajinder Yadav]

i guess in reddit case they were dealing with millions of requests, the 
system in going to buckle under that kind of demand. few choices when 
you have low mem, a hung server, shitty code (opps did I say that) or 
errant process, reboot or restart seems the best immediate solution to 
maintain a certain quality of service. this  helps fight the fire while 
you're trying to figure out what is going wrong, if there is a pattern 
is will be easier to figure out than those quirky ones that just show up 
randomly.

you're right it should be fairly easy to write a script to monitor some 
of the things and send out an alert or take some action, i am not sure 
exactly what data can be pulled by those tools for a script to query a 
system. i don't want to run and maintain a lot of scripts that are using 
more cpu to query, parse and poll said services if there is a better 
solution out there, something event driven rather than poll/parse driven.

this is really a good education exercise for me, it's helping me learn 
more linux from a sys op point of view than from a hacker point of view =)

kind regards,
Rajinder




On 10-08-01 06:01 PM, aaron d wrote:
> If you can program it should be fairly trivial to script custom 
> monitoring using the aforementioned tools, that takes the action of 
> your choice. Reboot though, as stated before, is a bad habit to get 
> into and generally NEVER the answer. We aren't running windows here :)
>
>
> On Sun, Aug 1, 2010 at 5:47 PM, Rajinder Yadav <devguy.ca 
> <http://devguy.ca>@gmail.com <http://gmail.com>> wrote:
>
>     i know some of those tools, not an avid user of them, will read up.
>
>     as for rebooting or restarting errant process, i was thinking of
>     this in terms of my own web-server hosting setup, in particular my
>     own rails apps, since i would be logging error, i would know what
>     was causing any kind of abnormal memory, cpu, network load, it
>     would be a stop gap solution till the real issue was rectified in
>     the code or through load balancing, etc.
>
>     i heard a talk from the reddit founder talking about fail often,
>     crash early and they were using some of this monitoring
>     techniques, it was interesting and something to mull over for sure =)
>
>     thanks,
>     Rajinder
>
>
>
>
>     On 10-08-01 08:19 AM, aaron d wrote:
>>     iftop, iostat,  vmstat, free, df ? they aren't monitors in the
>>     sense that you are talking in the second half of your question
>>     (i.e. they don't take action, they simply tell you what is going
>>     on). In my experience with monitoring systems, you generally
>>     would not want something just killing off random PIDs at it's own
>>     discretion, let alone REBOOTING because of a memory leak. We
>>     never reboot :) Usually you would want to receive some form of
>>     alert so that you can use these tools to investigate the issue
>>     and possibly take steps to keep it from happening over and over
>>     again. However there may be a time when you just want to kill a
>>     notoriously bad process (google chrome anyone?) if it gets out of
>>     hand.
>>
>>     conky (http://conky.sourceforge.net/) is not a console-based
>>     monitor, but is very nice, and of course if you are monitoring
>>     remotely you could always forward X to your local machine.
>>
>>     -aaron
>>
>>     On Sat, Jul 31, 2010 at 6:01 PM, Rajinder Yadav <devguy.ca
>>     <http://devguy.ca>@gmail.com <http://gmail.com>> wrote:
>>
>>         Other than top, that's as far as my linux-fu gets me =P, what
>>         other console based monitors are you guys using that has low
>>         cpu load for things like:
>>
>>         network activity, file activity, storage space, low memory
>>
>>         if the system is overloaded, i'm memory leaks, errant process
>>         is there a monitor that will force the server to reboot? i am
>>         asking more for an education purpose and not any kind of
>>         immediate use
>>
>>         Thanks,
>>         Rajinder
>>         --
>>         The Toronto Linux Users Group.      Meetings: http://gtalug.org/
>>         TLUG requests: Linux topics, No HTML, wrap text below 80 columns
>>         How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists
>>
>>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gtalug.org/pipermail/legacy/attachments/20100801/fe69498e/attachment.html>