CSCI 373 Computer Networking — Concurrency in servers

The Socket API

Pretty much everything you need to know about the socket for TCP programming was presented last week. However, good network programming requires writing programs that can handles thousands of simultaneous connections.

Server solutions

Pathetic one time server

The run-once-and-halt server appeared in the documentation for Python’s socket module. You should never want to write this.

The serial server

The previous server was worse than you think. In the socket interface, there is no guarantee that a server write of 1000 bytes will result in a client read of 1000 bytes. Both the server and client must use loops to read and write.

Java has an easy way to read entire lines, use the BufferedReader class. Java has a somewhat clumsier way to write entire lines, use the BufferedWriter and PrintWriter classes. The getline method of the C++ istream class and the flush method of ostream can be used similarly.

In Python lines can be read and written with methods of the file class. The socket class has a very useful makefile method that can transform a socket into two files, one for reading and one for writing. This makes it pretty easy to read and write CRLF-terminated lines through a socket.

The one-at-a-time server

By having an outer loop to accept connections and an inner loop to handle the connection, it is possible for a server to handle a series of sequential connections. Putty the connection handler within its own routines simplifies the programming.

Unix-style process creation

Unix (and consequently Linux) has a unusual way to create new processes with the fork system call. The Unix shell depends on four different system calls to run the commands you type:

Python has an os module that implements the Unix process control primitives.

Experienced Unix programmers know all the right patterns for using these calls. For many years, most Unix servers where launched using a super-server implemented with these functions. This approach is easy to program when the client applications run independently; however, there is a lot of overhead in its implementation.

Implementation with threads

Over the last few years, thread creation has begin to replace process creation. Threads are more efficient and generally easier to program. Threads also execute within a shared memory space.

Most thread implementations are based on Posix Threads or Java threads (which are used heavily in Java GUI applications).

The Python threading has several methods used to program servers.

As long as the connections are relatively independent, the thread implementation looks similar to processing implementation.

Implementation by overriding thread classes

In Java most threading applications are created by extending and overriding methods of the threads class. The Python threading library allows the __init__ and run methods to be overridden. In practice the run method code contains the old connection handler code.

Programmers who have experience writing Java threaded applications will like this approach. Others may find it puzzling.

Synchronizing client communication

Sometime clients interact with each other. The best example of this would be the classic chat application. In these situations it is necessary to use Pthread-style locks. The Pthread implementation supports several kinds of locks and most of them are supported in the threading module.

The first rule of programming with locks is: Never hold the lock for a long time. The second rule (which follows from the first) is: Never sleep while holding the lock. Usually, you can write a thread-safe program by adding a few simple pairs of lock and unlock operations; however, it is possible to design homework assignments that require using Pthread-style condition objects.

Python programs can also be interpreted by Jython which translates Python programs to Java byte code. Jython programs start noticeably slower, but they can may perform better for concurrent applications because they does not suffer from having a Global Interpreter Lock.

Because Jython does not support the most recent versions of the Python language, it may be necessary to rewrite your code.

Putting the locks in the objects

It is often a good idea to just override Thread and place the synchronization code into the class.

Improving the BSD model

Although the BSD socket interface uses the same socket call for both client and server, some modern implementations, such as Java’s ServerSocket class, provide an object-oriented interface more appropriate for writing server applications.

Python has its own SocketServer class which can be a bit tricky. You really have to override a connection handler method to use SocketServer. Threading even requires multiple inheritance! Here are some of the methods needed to implement with Python’s SocketServer:

Take a look at this implementation.