SIMPL open source project... / Send/Receive/Reply message passing

[bob]

On Wednesday 28 June 2006 05:57 pm, Christopher Browne wrote:
> On 6/28/06, Jason Spiro <jasonspiro4-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org> wrote:
> > On 6/22/06, bob <ican-rZHaEmXdJNJWk0Htik3J/w at public.gmane.org> wrote:
> > > In reality the SIMPL API is really just 5 main functions:
> > >
> > > name_attach() - to register a process name
> > > name_locate() - to open a communication channel to a process found by
> > > name
> > > Send() - throw off a message and wait for a response
> > > Reply() - respond to a message
> > > Receive() - wait for a message to be thrown your way
> >
> > Isn't this similar to the use of TCP sockets or Unix named sockets?
>
> Sure, it's *similar*.
>
> But that's only true in the bland sense that "everything on Unix
> involves manipulating files."  Which may at some level be true, but if
> you're trying to reverse engineer someone's binary data format, it may
> include a whole lot of cursing...
>
> By layering the API on top, you don't need, as a programmer, to pay
> attention to the details of how messages go back and forth.  It's easy
> to work with messages; they look nice and atomic, as opposed to
> figuring out how many bytes to read from a socket.
>
> The docs for Isect outline this nicely...
> http://isectd.sourceforge.net/isect_1.html#SEC1
>
> ---------------------------------------------
> Long before the buzzwords "client-server" and "middleware" were ever
> coined, programmers used middleware to build client-server systems.
> They programmed using character or block IO directly to the network
> devices which may have been proprietary, like Data General's MCA
> cards, or standard like UARTs for serial communications.
>
> Over TCP/IP networks, regardless whether the network interface is
> token-ring, ethernet, or PPP, the socket library developed at the
> University of Berkeley became the standard way for programs running on
> different computers (or even the same computer) to communicate with
> each other.
>
> But writing servers is more complicated than just read()ing requests
> from a client. What if there are multiple clients? What if the client
> dies while you're still processing? What if the server dies? How does
> the client find the server? Should or must the server be multitasking?
> How should semaphores be used? These issues all concern network
> programming and have little (read: nothing) to do with what the
> programmer really needs the server to do. In fact, for simple server
> programs the overhead of the network programming may make it not worth
> it to create the server in the first place.
> ---------------------------------------------

Don't forget the built in synchronization that comes from the S/R/R messaging.   
This is a huge benefit when trying to design and debug complex networked 
systems.

As a rule regardless of how they are deployed (single node or multinode) S/R/R 
systems are forced to behave reproducably.     Nothing makes a testing 
engineer wish for a career change more than a system which exibits chaotic 
and unreproducable behavior.

In fact most S/R/R systems are built and fully tested on a single node and 
then simply redeployed on a network.     If done properly the only thing that 
changes is the apparent execution speed.    The S/R/R sychronization looks 
after insuring that the message sequence is always reproducable ... network 
or not.

The earliest SIMPL implementations (~1997) were done on an AIX system using 
TCP/IP sockets to carry all the messaging.   The current implementation uses 
shared memory blocks to carry the message traffic (locally) and named pipes 
(fifos) to provide a select()'able fd for synchronization.    Along the way 
there have been kernel module implementations of SIMPL ... most of which are 
in severe need of maintainence attention as the Linux kernel internals keep 
evolving.    Although the kernel module implementations offered better 
performance (rivaling QNX in on some hardware) than the shared memory/fifo 
implementation the issues of maintenance overwelmed that advantage in most 
cases.

Actually there still is a TCP/IP implementation of SIMPL which was originally 
developed to allow Tcl/Tk applets to converse,  but has since been extended 
to any inter OS Tcl/Tk communication. (eg.  Windows to Linux,  or Linux to 
deeply embedded devices).

To carry the messages between nodes the SIMPL project provides generic 
surrogates which convert the local shared memory to a TCP/IP stream and then 
back again.   The surrogate framework is general enough that any data 
transport scheme (eg. RS232/485,  replicated memory  etc.) could be used 
transparently and even in parallel (as a failover redundancy scheme).

The surrogates embody all the "knowledge" of the network and simply treat the 
messages as data packets to be transported.     The surrogate abstraction 
saves the programmer a great deal of effort and allows him/her to concentrate 
on building the application code to solve the problem at hand.

bob
--
The Toronto Linux Users Group.      Meetings: http://tlug.ss.org
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://tlug.ss.org/subscribe.shtml