Networking, Servers & the Raspberry Pi

Some good references

MAC layer

The MAC layer (Medium Access Control) provides the lowest level the programmer and system administrator can access. The computer receives physical packets encoded in a format defined in the MAC layer definition. Here’s how an Ethernet packet might encapsulate data destined for a higher layer.
802.1Q packet

MAC addresses (also know as physical or hardware addresses) are 48 bits long and typically written as a 12-digit hexadecimal number where colons separate pairs of hexadecimal digits, for example 00:25:64:C0:AC:81.

Generally large Ethernets are managed by spanning trees switches that learn the topology of the network and route packets to the appropriate segment of the net. They look like an ordinary Ethernet to the computer.
spanning tree
Everything you need to know about spanning trees can be learning from the poem Algorithm written by its inventor, Radia Perlman.

Like Ethernet, Wi-Fi is also a MAC-layer protocol.

MAC level administrative programs

IP layer

In the IP layer routers direct packets between physical networks. Thus the physical internet is a collection of local networks joined by routers. The Internet Protocol was specified by RFC 791 in 1981.

Here is the summary of an IP packet header from RFC 791. This is the beginning of the payload for an Ethernet packet.

    0                   1                   2                   3   
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |Version|  IHL  |Type of Service|          Total Length         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         Identification        |Flags|      Fragment Offset    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Time to Live |    Protocol   |         Header Checksum       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Source Address                          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Destination Address                        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Options                    |    Padding    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                    Example Internet Datagram Header

The interface of every computer on the internet is identified by a unique 32-bit address written in dot-decimal notation, four 8-bit numbers separated by periods such as 152.18.69.35 . These addresses are assigned by allocating ranges of 32-bit addresses to organizations using CIDR (classless inter-domain routing) notation. For example, NC-REN manages 152.16.0.0/12, UNC Asheville manages 152.18.0.0/16, and the computer science department manages 152.18.69.0/24 .

IP to MAC address matching

Routers direct data to the target local network using the IP address. On the local network the router determines the MAC address of the target by using ARP (Address Resolution Protocol). This is accomplished by broadcasting the desired IP address to all computers on the local network. The one holding the targeted IP will reply with its MAC address which will then be stored in the ARP table so that the broadcast can be avoided when the next packet arrives.

IP level administrative tools

IPv6

Almost all IP traffic uses IP version 4. IPv6 was defined in 1995 and is specified in RFC 2460. It is a significant change to IPv4. For example, hosts have 128-bit rather than 32-bit addresses. It is claimed that there are some computers using IPv6 out there.

TCP layer

There are two well-used layers above IP, TCP (Transmission Control Protocol) and UDP (User Datagram Protocol). UDP is heavily used for media applications, like video, and for gaming. TCP is used for most everything else, such as email and web page retrieval.

TCP was specified by RFC 793 in 1981 but has had several performance related improvements in the last 32 years. TCP provides reliable stream-oriented connections between applications running on networked computers. These connections are identified by 16-bit port numbers. Most server port numbers are well known. For example, port 80 is for HTTP (web), port 22 is for SSH, port 25 is for SMTP (mail), and port 53 is for domain (IP name lookup). Client port numbers are usually generated by the operating system.

As you can imagine, TCP has a pretty complicated header.

    0                   1                   2                   3   
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Source Port          |       Destination Port        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Sequence Number                        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Acknowledgment Number                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Data |           |U|A|P|R|S|F|                               |
   | Offset| Reserved  |R|C|S|S|Y|I|            Window             |
   |       |           |G|K|H|T|N|N|                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           Checksum            |         Urgent Pointer        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Options                    |    Padding    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             data                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                            TCP Header Format

TCP level administrative tools

Applications

On top of TCP and UDP there are many application protocols. Many of these are specified in very long documents, such as RFC 2821 for mail and RFC 2616 for the World Wide Web.

Network address translation

Network address translation is a scheme in which a router changes the TCP port number and IP address of packets within a TCP session.

For example, I am writing this page while connected, via ssh, from my home Linux system to oteen.cs.unca.edu. My home system thinks its IP address is 192.168.0.217 and that it is using port number 37905 to connect to port number 22 on 152.18.69.35, oteen.cs.unca.edu. However, oteen.cs.unca.edu thinks that its port 22 is connected to port 37905 at IP address 24.196.111.222, which is the IP address that Charter has assigned to my router. My router is changing the IP number in the IP header of packets that pass between my Linux computer and oteen. Sometimes it may even change the TCP port number because two computers in my house may have independently initiated a TCP connection using the same port number.

Domain name lookup

RFC 1035 describes a system for translating domain names, such as oteen.cs.unca.edu, into IP addresses by using a distributed network of servers. Compromising one domain server can disable large portions of the internet as was the case in the recent denial-of-service attack on the New York Times web site.

Domain name system administrative tools

Pi as a server

A very simple server and client with a network debugging tool

Technically any program that accepts connections from the outside is a server.

And any program that initiates a connection is a client.

Try this with pairs of Raspberry Pi’s.

Writing a real client and server

If you want to write your own client and server, I suggest you use Python. Don’t be too concerned about the overhead of using an interpreted language. The network connection will be the bottleneck.

If you want to be true to the Berkeley socket (and Winsock) interface, use Python’s socket module. The Python wiki contains a pretty good example of using the socket module.

Python also provides an object-oriented SocketServer in which you override methods to create the server. Forget the low-level socket interface. Use OOP.

Writing a HTTP server

There is a lot to be said for implementing a server that conforms to the HTTP protocol, especially since Python provides a SimpleHTTPServer module.

Of course, you could install your own LAMP server on the Raspberry Pi, but I suspect all you really want is an application that can read a few buttons, turn on a LED, or get some information out of an I2C device. The overhead of LAMP and CGI is just too much.

A Python based web server for the Pi

So, let’s just roughly follow a tutorial for creating a Raspberry Pi web server from SUNY-Albany’s Information in the 21st Century.

Start by creating a directory on your Pi and downloading a couple of files.

Using nano, open these files and do some appropriate customization.

Now start your server with the following command.

python simpleServer.py 

You should be able to connect to your Pi by directing your browser to lastname-pi.cs.unca.edu:8000 . Be sure to look at the “log” your server is producing when it receives a connection. You can also do a little more specialization if you wish.

Next we’d like for you to get your server doing a bit more than just serving files. Copy in two more files.

Again, open these files, modify them, and store them with more adult names. simpleServerWithXXX.py is a Python file for launching a TCP server. XXXHandler.py is a Python file where you implement the interesting part of the server.

Right now the server will only respond to requests similar to lastname-pi.cs.unca.edu:8000/?XXX=MSG , but your Python program can intercept these requests and do exciting things on your Pi. For example, lastname-pi.cs.unca.edu:8000/?led5=on could turn on an LED and lastname-pi.cs.unca.edu:8000/?gyrox could return a reading from an I2C device.

You could also go a bit further and implement a RESTful server that responds to JavaScript HTML requests. Perhaps your server should be return content of type application/json rather than text/html.