DLIFE distributed architecture reference

DLIFE is distributed, meaning that many people may allocate their spare computing cycles and memory in order to form something which approximates a very large single soup. Actually modelling a very large soup, however, would be very difficult: such a program would have a high communication overhead, would be totally unsuitable to run on the Internet, and adding nodes to this would bring diminishing returns. Therefore DLIFE uses a simpler model: individual computers run small soups, and from time to time, soups exchange a few cells at random with each other. In effect we form a series of islands connected by archipelagoes.

Processes

There are currently three processes involved:

The following diagram shows the interaction between these three programs.

[Diagram] Interaction between DLIFE processes

As shown, there are five client computers and two servers. The second client computer has two CPUs, and runs dlife_soup continuously on both. The third client computer is connected to the Internet by a modem, and so runs dlife_client.pl from /etc/ppp/ip-up.local (this script runs whenever the Internet connection is brought up). All the other computers are permanently connected to the Internet, and dlife_client.pl runs periodically from cron. Clients may connect and exchange cells with more than one server. In the diagram above, the third client computer connects to both servers.

Cells in files

The long-running dlife_soup program does not contain any network code. All network communications are handled by dlife_client.pl. Instead, dlife_soup contains code to save and load cells to files.

dlife_soup saves (by default) six cells per hour, at random, into the directory:

/var/spool/dlife/outgoing/

dlife_soup loads (by default) four cells per hour, at random, from the directory:

/var/spool/dlife/incoming/

The dlife_client.pl program takes some cells at random from the outgoing spool directory and uploads them to one or more servers (see the configuration section below). It then asks the server(s) to send some cells, and it saves these in the incoming spool directory.

The dlife_server.pl program listens for client connections, and uploads or downloads cells. It saves its cells in the directory:

/var/spool/dlife/store/

DLIFE network protocol

The DLIFE network protocol is a very simple ASCII based protocol, similar to (but much more limited than) FTP.

Servers listen for connections on port 5904. When a client connects, they send back the following identifying string:

DLIFE SERVER <version> <protocol> <CR> <LF>

<version> is the server version, for information only.

<protocol> is the protocol version. The protocol described below is version 1.x (for some digit ``x''). If a server reports a different major version number (eg. 2.x) then this is an indication that the protocol has changed dramatically and current clients should give up and abort.

Clients connect to servers on port 5904 and issue a series of simple four letter commands. Each command should take the form:

<command> [ <space> <arguments ...> ] <CR> <LF>

There are currently four commands: HELO, STOR, RETR and QUIT.

Servers respond to commands with a three digit response code and some optional human-readable text:

XYZ [ <space> text ... ] <CR> <LF>

Of the response code, only the first digit (``X'') is meaningful to clients. As for FTP, this digit can have the following meanings:

Response code Meaning
1xx Command OK. Ready to send data, or ready to receive data.
2xx Command OK. Ready for your next command.
4xx Temporary error. The client can retry the last command without modification and it may well succeed next time.
5xx Error. The client should not just retry the same command, but needs to reformulate the command or cancel the operation and report an error back to the user.

The HELO command

The HELO command has the following form:

HELO <space> <client_hostname>
          <space> <client_version> <CR> <LF>

where <client_hostname> is the client hostname (or the client may just send "-" character) and <client_version> is the client version number, used just for information only.

The server should always respond with 2xx to this command.

A typical interaction:

> HELO - 0.0.12
< 200 Hello.

The STOR command

The STOR command is used to upload a single cell to the server.

The format of the command is:

STOR <CR> <LF>

If the server accepts the command and is ready to upload a cell, it will reply with a 1xx response. The client should then send the cell as a series of hex nybbles, terminated by the sequence <CR> <LF> . <CR> <LF>.

A typical interaction:

> STOR
< 100 Send me the cell, terminated by ".".
> 00000000010901010100002227
> 0000000100FFFFFFFFFFFFFFFF
>      :      :
>      :      :
> 00010000002402020202020202022027
> 0001010101
> .
< 200 Cell was uploaded OK.

Servers will typically have limitations on the number and type of cells that they will accept. Generally they will only accept a small number of uploads per session, and each cell will be limited to 8192 bytes or less.

The RETR command

The RETR command is used to download a cell from the cell bank. The format of the command is:

RETR <CR> <LF>

If the server is ready to serve a cell, it will reply with a 1xx response, followed by a series of hex nybbles and terminated by the sequence <CR> <LF> . <CR> <LF> and immediately followed by a 2xx response.

A typical interaction:

> RETR
< 100 OK. Sending you a cell.
< 00000000010901010100002227
< 0000000100FFFFFFFFFFFFFFFF
<      :      :
<      :      :
< 00010000002402020202020202022027
< 0001010101
< .
< 200 Cell was sent OK.

The QUIT command

The QUIT command terminates a session (clients may also just close a session).

The format of the command is:

QUIT <CR> <LF>

The server responds with a 2xx message and then closes its end of the connection.

A typical interaction:

> QUIT
< 200 Goodbye.
(socket is closed)

Configuration files

/etc/dlife/client.conf

This file is used to configure the network client. The file tells the client which server or servers to connect to.

The client can be configured to:

server_url command

The server_url command may be optionally given. It directs the client to download the web page given in the URL. The web page should contain a master list of servers.

server_url http://dlife.annexia.org/servers.txt

server_zone command

The server_zone command is optional. It should be used in conjunction with the server_url command. It directs the client to only consider servers in the particular zone or zones listed.

The general form is:

server_zone <zone> [<zone> [...]]

Each <zone> argument should have the form:

server command

The server command directs the client to connect directly the server or servers listed.

server dlife1.bibliotech.co.uk dlife2.bibliotech.co.uk

max_cells_upload_per_pass command

The max_cells_upload_per_pass directs the client to attempt to upload this many cells in a pass. Note that the server probably imposes its own limit.

max_cells_download_per_pass command

The max_cells_download_per_pass directs the client to attempt to download this many cells in a pass. Note that the server probably imposes its own limit. It is a good idea to ensure that max_cells_upload_per_pass > max_cells_download_per_pass.

/etc/dlife/soup.conf

The soup.conf file configures various internal aspects of the DLIFE virtual machine. For full documentation, please see the example file provided with the source distribution.

Typical configurations

Default configuration


Richard Jones
Last modified: Wed Oct 11 23:19:19 BST 2000