Efficent handling of sockets

[ted leslie]

I currently work on some chat servers, and other client server apps with threads and sockets,

i always spawn threads, in fact, pre-spawn then, to avoid gang up when a network event has
1000's of socket attempts in seconds (there is some set up on each thread, so nice to get that out of the way on initial app
start up, i.e. like i believe apache does).

I current have a chat app. on a 1GHz pent. 4 server,
with 3000 threads spawned, for 1500 socket connects, 
i spawn two threads for each connect, one for consuming, the other for producing.
I have 5000 cs's per second,
and i notice on a old system like that, things start to get harry
at 60000.
it is also a single cpu

i find for my design, cs's is cheap,

i have run a server with more then 1024 threads, and thus had to ulimit it up,

i would guess i can manage over 10000 chat clients from that old server,
and probably 50K-60K from a quad core 3Ghz modern box.

i think as long as the L1/L2/L3 cache are humming along, cs's isn't all that bad.

for sys% on the 3000 threads on the old cpu, its running sys% at about 1/10 %,
and typically user% is <1% and idle > 99.X

i totally gotta agree with Len,

and i find the thread approach a clean code set up myself.

this is written in mono, so i would guess its using pthreads underneath, not sure.
but years ago in a C implementation, i did the same design but with pthreads.

locks and sems are fun!!! :)   well for other jobs, personally , i really need to know them well,
but yeah there are some rare times they are the reason for a late night debug session :( 



-tl



On Fri, 20 Feb 2009 11:35:22 -0500 (EST)
"D. Hugh Redelmeier" <hugh-pmF8o41NoarQT0dZR+AlfA at public.gmane.org> wrote:

> | From: Lennart Sorensen <lsorense-1wCw9BSqJbv44Nm34jS7GywD8/FfD2ys at public.gmane.org>
> 
> | On Thu, Feb 19, 2009 at 06:24:52PM -0500, D. Hugh Redelmeier wrote:
> | > Threads are a very general purpose mechanism and are implemented quite
> | > expensively compared with a reasonable "get next event" loop.
> | 
> | Threads are not expensive in linux.  I wouldn't be surprised if the
> | overhead of managing bitstrings in select costs more than threads.
> 
> Process switching is expensive in Linux.  Why would you say otherwise?
> Not the fault of Linux, it is the nature of modern hardware.  The
> semantics of "process" could be pared down to allow a cheaper
> implementation but we'd not like that semantics (eg. no process
> isolation).
> 
> I suggested epoll so that bitstrings did not have to be copied in and
> out of kernel space at each "get next event".  So the comparision is
> not relevant.  (I admit that I've never used epoll.)
> 
> Let's assume that we were using select.  How many file descriptors do
> you think we would be considering?  How many threads?  Should they be
> the same order of magnitude?
> 
> How many threads can a Linux kernel handle well?  I have no idea, so
> this isn't rhetorical.
> 
> How about 1000 of each?  Then each socket call needs to copy in and
> copy out 125 contiguous bytes.  Yuck.  But I bet that is a lot less
> than context switching (between user processes (same as between user
> threads)).  How much process context is in registers?
> 8 * 8B plain registers (roughly)
> 8 * 8B FP registers (guess)
> unknown MMU context, some of which causes caching hickups.
> TSC stuff?
> I don't know what else.
> 
> Remember, under load, the select version should handle multiple
> transactions per select.  Not true of the analogous thread version.
> 
> | Because I like software that tries to scale, and single threaded processes
> | using select on a huge number of sockets doesn't scale.  I also think
> | the select interface is awful.
> 
> Locks in userland code (or their lack!) seem to be a great source of
> problems too.
> 
> The existence of thousands of sockets might be a sign of a bad design
> too.  Perhaps the demultiplexing is being done in the wrong place.  It
> depends on what is being attempted.
> 
> | On a multi core system (as most are) you wouldn't even have to context
> | switch (which is very cheap on linux too)
> 
> How so?  Are you saying that you have thousands of cores?
> 
> | > If some (but not all) transactions are expensive to process, they can
> | > then be farmed off to other threads or processes.
> | 
> | Which just means you are writing a scheduler to some extent, which is
> | something the kernel already has, and its scheduler is often better than
> | one thrown together without much thought into the design.
> 
> If you are handling thousands of transaction streams, you had better
> think about scheduling.
> --
> The Toronto Linux Users Group.      Meetings: http://gtalug.org/
> TLUG requests: Linux topics, No HTML, wrap text below 80 columns
> How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists
> 


-- 
ted leslie <tleslie-RBVUpeUoHUc at public.gmane.org>
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists