Efficent handling of sockets

Lennart Sorensen lsorense-1wCw9BSqJbv44Nm34jS7GywD8/FfD2ys at public.gmane.org
Fri Feb 20 15:31:08 UTC 2009


On Thu, Feb 19, 2009 at 06:24:52PM -0500, D. Hugh Redelmeier wrote:
> Threads are a very general purpose mechanism and are implemented quite
> expensively compared with a reasonable "get next event" loop.

Threads are not expensive in linux.  I wouldn't be surprised if the
overhead of managing bitstrings in select costs more than threads.

> | I don't like select very much personally.
> 
> Why?
> 
> But, for high efficiency, with thousands of file descriptors, other
> mechanisms exist that don't have to shovel large bitstrings in and out
> of the kernel at every call.  (I don't do this so I don't remember the
> details. epoll(7), I think, but what horrible man pages.)

Because I like software that tries to scale, and single threaded processes
using select on a huge number of sockets doesn't scale.  I also think
the select interface is awful.

> | Also with a single thread and select, you can only be processing one
> | request at a time, while with threads you could be using multiple CPUs
> | and handling multiple sockets at once.  Locking of course is an issue
> | to deal with for threaded code.
> 
> With select, you can (should) process all the things that come ready at the
> time of the select call returning.  Under load, this is likely to be
> more than one (heck, that is a symptom of heavy load).
> 
> This matters because often it is the process switching that is
> expensive rather than the actual transaction processing.

On a multi core system (as most are) you wouldn't even have to context
switch (which is very cheap on linux too)

> If some (but not all) transactions are expensive to process, they can
> then be farmed off to other threads or processes.

Which just means you are writing a scheduler to some extent, which is
something the kernel already has, and its scheduler is often better than
one thrown together without much thought into the design.

> Anecdote:
> 
> I first realized this when I talked with the designer of a board to
> handle 64 (I think) serial lines.  I always thought interrupts were
> the way to go, but they found polling worked better for the worst
> case.  If you can handle the worst case, the rest is easy.

Sometimes interrupts are good, sometimes they are not.  Sometimes a
hybrid between polling and interrupts is the way to go.

-- 
Len Sorensen
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists





More information about the Legacy mailing list