19.1. Ports and Connections

Options used:

• lpd_port=Port for lpd to accept connection

• originate_port=Ports to originate connections on

• reuse_addr FLAG Set SO_REUSEADDR flag on connection

• retry_econnrefused FLAG Retry on connect ECONNREFUSED error

• retry_nolink FLAG Retry on device open or connection ffailure

• socket_linger=socket linger timeout

RFC1179 requires that the lpd server listen for TCP/IP connections on port 515. This port is registered with the Internet Naming Authority, and the /etc/services file or TCP/IP services database usually has an entry:

    printer     515/tcp     spooler     # line printer spooler

RFC1179 explicitly states that all connections to port 515 must originate from ports 721-731. The reason for this restriction is due to the UNIX concept of reserved and privileged ports. By convention, ports in the range 1-1023 can only bound by processes whose Effective User ID (EUID) is 0 (root). This, ordinary users could not originate a connection from the reserved or privileged port range.

In a UNIX environment, this means that the user programs lpr, lpq, lprm, and lpc would have to be SETUID root.

As experience has shown, for security purposes, the fewer programs that need to have privileged status, the better. LPRng uses the lpd_port=printer configuration option to set the actual port to be use. By default, this is port 515, but can be set to other values.

The restriction of originating ports to 721-731 causes another set of problems. Part of the TCP/IP protocol is concerned with avoiding communications problems resulting from the arrival of old or stale packets. When a connection between sourcehost, sourceport and desthost, destport is made, a set of sequence numbers is established and used for sending and acknowledgement of data. When the connection terminates, the TCP/IP protocol restricts the establishment of a new connection between sourcehost, sourceport and desthost, destport for a period long enough for all stale packets to be removed from the system. This is approximately 10 minutes long.

In order to simplify assignments of ports, timing out connections, and other matters, many TCP/IP packages do keep track of explicit connections originating from a port, but simply prevent the port from being reused for either origination or reception of a connection. They do, however, keep track of the active connections to a port, and perform timeouts on these. This is usually much simpler to implement, as it can be done with a list attached to the port.

This implementation method creates some problems when a large number of connections must be originated from a relatively small number of port numbers. Observe what happens when host 1 tries to send a large number of jobs to a server 2. The following connections are established and terminated: host 1, port 721 and host 2, port 515 host 1, port 722 and host 2, port 515 host 1, port 723 and host 2, port 515 host 1, port 724 and host 2, port 515 host 1, port 725 and host 2, port 515 host 1, port 726 and host 2, port 515 host 1, port 727 and host 2, port 515 host 1, port 728 and host 2, port 515 host 1, port 729 and host 2, port 515 host 1, port 730 and host 2, port 515 host 1, port 731 and host 2, port 515

Now according to the RFC1179 rules and the TCP/IP protocol, we will have to wait until one of these connections terminates before we can make another. On the originating system, if the TCP/IP implementation does timeouts on the originating port, we will have to wait for the timeout to elapse before we can make a new connection. Unfortunately, there is no way to find out what the status of the port is, so we will have to try them each in turn until we get a successful connection.

The LPRng code has tried to provide several methods to deal with these problems. Firstly, the originate_port=512 1023 option specifies the range of ports used to originate connections when the software is running either as ROOT or SETUID root. By strict RFC1179 rules, this should be originate_port=721 731, but it turns out that most BSD lpd based implementations only check for a reserved originating port. By using 512 ports we get a greatly reduced rate of errors due to lack of ports due to pending timeouts.

However, on some systems which are acting as servers for a large number of printers even increasing this port range is insufficient, and steps need to be taken use the originating port numbers more efficiently. The Berkeley TCP/IP implementation getsockopt() and setsockopt() allows the user to manipulate some of the underlying timeouts and options of the TCP/IP network. When a TCP/IP connection is established, the setsockopt() facility can be used to set the SO_REUSEADDR flag on the connection. This flag effectively sets the timeout value on the ports and connections to 0, allowing immediate reuse of the ports. When done on an originating end of a connection, this will allow the originating port number to be reused immediately.

It would appear that by setting SO_REUSEADDR on the originating end that we have solved our problems. However, unless the destination end of the connection sets its SO_REUSEADDR flag on the connection, it will still do a timeout. Thus when we try to make a connection from a port that was active within a short period of time to the same host, then it will reject the connection until the timeout is over.

The reuse_addr flag (default off) forces the LPRng software to set the SO_REUSEADDR flag on originating connections. As indicated, this will allow ports to be reused immediately for outgoing connections, rather than waiting for a timeout.

While the reuse_addr flag usually allows us to reuse ports, there is still the problem of dealing with connections failing due to the remote site rejecting the connection due to a pending timeout from a previous connection. A careful study of the original BSD TCP/IP network code and of some others indicates that when a connection fails due to a pending timeout, an ECONNREFUSED error code is returned to a connect() system call. If this happens and we suspect that the remote site is rejecting the connection due to a timeout problem, then we should retry making the connection but from a new port, and continue retrying until all possible ports are used.

The retry_econnrefused (default on) flag is used to specify that we retry connections in this manner. When this is set, a connection refused error causes the connection to be retried using a new port. This will be repeated until all available ports have been tried.

When printing a job and the lpd server connection to a remote site or device open fails, the retry_nolink (default on) will cause the attempt to be retried indefinitely. The combination of retry_econnrefused and retry_nolink will provide robust connection attempts to remote systems.

While the above problems cause difficulties when making connections, there are also problems when terminating connections. After closing a socket, the TCP/IP software will try to flush any pending data to the destination. Unfortunately, on some systems it will only do this while the process is active. This has caused problems on systems which terminate a process it has received an abnormal (signal caused) termination.

The setsockopt() SO_LINGER option allows the user to specify that when a socket is closed normally, that the process should block until pending data is flushed or for the socket_linger period. If socket_linger is 0, then no SO_LINGER operation is done.

In summary, if you experience problems with connection failures due to port exhaustion, first try setting the reuse_port flag, and you should see a reduction. Check to ensure that the retry_econnrefused and retry_nolink flags are set, and the error code in the log and status files. If the failures continue, then the problem is caused by the remote end having timeout limitations and there is little you can do except to set a very long connect_retry interval, say connect_retry=120 (2 minutes).