Whole document tree

11. Examples

11.1 User Filesystems
11.2 Home Directories
11.3 Architecture Sharing
11.4 Wildcard Names & Replicated Servers
11.5 `rwho' servers
11.6 `/vol'
11.7 `/defaults' with selectors
11.8 `/tftpboot' in a chroot-ed environment

11.1 User Filesystems

With more than one fileserver, the directories most frequently cross-mounted are those containing user home directories. A common convention used at Imperial College is to mount the user disks under /home/machine.

Typically, the `/etc/fstab' file contained a long list of entries such as:

machine:/home/machine /home/machine nfs ...

for each fileserver on the network.

There are numerous problems with this system. The mount list can become quite large and some of the machines may be down when a system is booted. When a new fileserver is installed, `/etc/fstab' must be updated on every machine, the mount directory created and the filesystem mounted.

In many environments most people use the same few workstations, but it is convenient to go to a colleague's machine and access your own files. When a server goes down, it can cause a process on a client machine to hang. By minimizing the mounted filesystems to only include those actively being used, there is less chance that a filesystem will be mounted when a server goes down.

The following is a short extract from a map taken from a research fileserver at Imperial College.

Note the entry for `localhost' which is used for users such as the operator (`opr') who have a home directory on most machine as `/home/localhost/opr'.

/defaults opts:=rw,intr,grpid,nosuid charm host!=${key};type:=nfs;rhost:=${key};rfs:=/home/${key} \ host==${key};type:=ufs;dev:=/dev/xd0g # ... # localhost type:=link;fs:=${host} ... # # dylan has two user disks so have a # top directory in which to mount them. # dylan type:=auto;fs:=${map};pref:=${key}/ # dylan/dk2 host!=dylan;type:=nfs;rhost:=dylan;rfs:=/home/${key} \ host==dylan;type:=ufs;dev:=/dev/dsk/2s0 # dylan/dk5 host!=dylan;type:=nfs;rhost:=dylan;rfs:=/home/${key} \ host==dylan;type:=ufs;dev:=/dev/dsk/5s0 ... # toytown host!=${key};type:=nfs;rhost:=${key};rfs:=/home/${key} \ host==${key};type:=ufs;dev:=/dev/xy1g ... # zebedee host!=${key};type:=nfs;rhost:=${key};rfs:=/home/${key} \ host==${key};type:=ufs;dev:=/dev/dsk/1s0 # # Just for access... # gould type:=auto;fs:=${map};pref:=${key}/ gould/staff host!=gould;type:=nfs;rhost:=gould;rfs:=/home/${key} # gummo host!=${key};type:=nfs;rhost:=${key};rfs:=/home/${key} ...

This map is shared by most of the machines listed so on those systems any of the user disks is accessible via a consistent name. Amd is started with the following command

amd /home amd.home

Note that when mounting a remote filesystem, the automounted mount point is referenced, so that the filesystem will be mounted if it is not yet (at the time the remote `mountd' obtains the file handle).

11.2 Home Directories

One convention for home directories is to locate them in `/homes' so user `jsp''s home directory is `/homes/jsp'. With more than a single fileserver it is convenient to spread user files across several machines. All that is required is a mount-map which converts login names to an automounted directory.

Such a map might be started by the command:

amd /homes amd.homes

where the map `amd.homes' contained the entries:

/defaults type:=link # All the entries are of type:=link jsp fs:=/home/charm/jsp njw fs:=/home/dylan/dk5/njw ... phjk fs:=/home/toytown/ai/phjk sjv fs:=/home/ganymede/sjv

Whenever a login name is accessed in `/homes' a symbolic link appears pointing to the real location of that user's home directory. In this example, `/homes/jsp' would appear to be a symbolic link pointing to `/home/charm/jsp'. Of course, `/home' would also be an automount point.

This system causes an extra level of symbolic links to be used. Although that turns out to be relatively inexpensive, an alternative is to directly mount the required filesystems in the `/homes' map. The required map is simple, but long, and its creation is best automated. The entry for `jsp' could be:

jsp -sublink:=${key};rfs:=/home/charm \ host==charm;type:=ufs;dev:=/dev/xd0g \ host!=charm;type:=nfs;rhost:=charm

This map can become quite big if it contains a large number of entries. By combining two other features of Amd it can be greatly simplified.

First the UFS partitions should be mounted under the control of `/etc/fstab', taking care that they are mounted in the same place that Amd would have automounted them. In most cases this would be something like `/a/host/home/host' and `/etc/fstab' on host `charm' would have a line:

/dev/xy0g /a/charm/home/charm 4.2 rw,nosuid,grpid 1 5

The map can then be changed to:

/defaults type:=nfs;sublink:=${key};opts:=rw,intr,nosuid,grpid jsp rhost:=charm;rfs:=/home/charm njw rhost:=dylan;rfs:=/home/dylan/dk5 ... phjk rhost:=toytown;rfs:=/home/toytown;sublink:=ai/${key} sjv rhost:=ganymede;rfs:=/home/ganymede

This map operates as usual on a remote machine (ie ${host} not equal to ${rhost}). On the machine where the filesystem is stored (ie ${host} equal to ${rhost}), Amd will construct a local filesystem mount point which corresponds to the name of the locally mounted UFS partition. If Amd is started with the -r option then instead of attempting an NFS mount, Amd will simply inherit the UFS mount (see section 5.25 Inheritance Filesystem (`inherit')). If -r is not used then a loopback NFS mount will be made. This type of mount is known to cause a deadlock on many systems.

11.3 Architecture Sharing

Often a filesystem will be shared by machines of different architectures. Separate trees can be maintained for the executable images for each architecture, but it may be more convenient to have a shared tree, with distinct subdirectories.

A shared tree might have the following structure on the fileserver (called `fserver' in the example):

local/tex local/tex/fonts local/tex/lib local/tex/bin local/tex/bin/sun3 local/tex/bin/sun4 local/tex/bin/hp9000 ...

In this example, the subdirectories of `local/tex/bin' should be hidden when accessed via the automount point (conventionally `/vol'). A mount-map for `/vol' to achieve this would look like:

/defaults sublink:=${/key};rhost:=fserver;type:=link tex type:=auto;fs:=${map};pref:=${key}/ tex/fonts host!=fserver;type:=nfs;rfs:=/vol/tex \ host==fserver;fs:=/usr/local/tex tex/lib host!=fserver;type:=nfs;rfs:=/vol/tex \ host==fserver;fs:=/usr/local/tex tex/bin -sublink:=${/key}/${arch} \ host!=fserver;type:=nfs;rfs:=/vol/tex \ host:=fserver;fs:=/usr/local/tex

When `/vol/tex/bin' is referenced, the current machine architecture is automatically appended to the path by the ${sublink} variable. This means that users can have `/vol/tex/bin' in their `PATH' without concern for architecture dependencies.

11.4 Wildcard Names & Replicated Servers

By using the wildcard facility, Amd can overlay an existing directory with additional entries. The system files are usually mounted under `/usr'. If instead, Amd is mounted on `/usr', additional names can be overlayed to augment or replace names in the "master" `/usr'. A map to do this would have the form:

local type:=auto;fs:=local-map share type:=auto;fs:=share-map * -type:=nfs;rfs:=/export/exec/${arch};sublink:="${key}" \ rhost:=fserv1 rhost:=fserv2 rhost:=fserv3

Note that the assignment to ${sublink} is surrounded by double quotes to prevent the incoming key from causing the map to be misinterpreted. This map has the effect of directing any access to `/usr/local' or `/usr/share' to another automount point.

In this example, it is assumed that the `/usr' files are replicated on three fileservers: `fserv1', `fserv2' and `fserv3'. For any references other than to `local' and `share' one of the servers is used and a symbolic link to ${autodir}/${rhost}/export/exec/${arch}/whatever is returned once an appropriate filesystem has been mounted.

11.5 `rwho' servers

The `/usr/spool/rwho' directory is a good candidate for automounting. For efficiency reasons it is best to capture the rwho data on a small number of machines and then mount that information onto a large number of clients. The data written into the rwho files is byte order dependent so only servers with the correct byte ordering can be used by a client:

/defaults type:=nfs usr/spool/rwho -byte==little;rfs:=/usr/spool/rwho \ rhost:=vaxA rhost:=vaxB \ || -rfs:=/usr/spool/rwho \ rhost:=sun4 rhost:=hp300

11.6 `/vol'

`/vol' is used as a catch-all for volumes which do not have other conventional names.

Below is part of the `/vol' map for the domain `doc.ic.ac.uk'. The `r+d' tree is used for new or experimental software that needs to be available everywhere without installing it on all the fileservers. Users wishing to try out the new software then simply include `/vol/r+d/{bin,ucb}' in their path.

The main tree resides on one host `gould.doc.ic.ac.uk', which has different `bin', `etc', `lib' and `ucb' sub-directories for each machine architecture. For example, `/vol/r+d/bin' for a Sun-4 would be stored in the sub-directory `bin/sun4' of the filesystem `/usr/r+d'. When it was accessed a symbolic link pointing to `/a/gould/usr/r+d/bin/sun4' would be returned.

/defaults type:=nfs;opts:=rw,grpid,nosuid,intr,soft wp -opts:=rw,grpid,nosuid;rhost:=charm \ host==charm;type:=link;fs:=/usr/local/wp \ host!=charm;type:=nfs;rfs:=/vol/wp ... # src -opts:=rw,grpid,nosuid;rhost:=charm \ host==charm;type:=link;fs:=/usr/src \ host!=charm;type:=nfs;rfs:=/vol/src # r+d type:=auto;fs:=${map};pref:=r+d/ # per architecture bin,etc,lib&ucb... r+d/bin rhost:=gould.doc.ic.ac.uk;rfs:=/usr/r+d;sublink:=${/key}/${arch} r+d/etc rhost:=gould.doc.ic.ac.uk;rfs:=/usr/r+d;sublink:=${/key}/${arch} r+d/include rhost:=gould.doc.ic.ac.uk;rfs:=/usr/r+d;sublink:=${/key} r+d/lib rhost:=gould.doc.ic.ac.uk;rfs:=/usr/r+d;sublink:=${/key}/${arch} r+d/man rhost:=gould.doc.ic.ac.uk;rfs:=/usr/r+d;sublink:=${/key} r+d/src rhost:=gould.doc.ic.ac.uk;rfs:=/usr/r+d;sublink:=${/key} r+d/ucb rhost:=gould.doc.ic.ac.uk;rfs:=/usr/r+d;sublink:=${/key}/${arch} # hades pictures pictures -opts:=rw,grpid,nosuid;rhost:=thpfs \ host==thpfs;type:=link;fs:=/nbsd/pictures \ host!=thpfs;type:=nfs;rfs:=/nbsd;sublink:=pictures # hades tools hades -opts:=rw,grpid,nosuid;rhost:=thpfs \ host==thpfs;type:=link;fs:=/nbsd/hades \ host!=thpfs;type:=nfs;rfs:=/nbsd;sublink:=hades # bsd tools for hp. bsd -opts:=rw,grpid,nosuid;arch==hp9000;rhost:=thpfs \ host==thpfs;type:=link;fs:=/nbsd/bsd \ host!=thpfs;type:=nfs;rfs:=/nbsd;sublink:=bsd

11.7 `/defaults' with selectors

It is sometimes useful to have different defaults for a given map. To achieve this, the `/defaults' entry must be able to process normal selectors. This feature is turned on by setting `selectors_in_defaults = yes' in the `amd.conf' file. See section 6.5.32 selectors_in_defaults Parameter.

In this example, I set different default NFS mount options for hosts which are running over a slower network link. By setting a smaller size for the NFS read and write buffer sizes, you can greatly improve remote file service performance.

/defaults \ wire==slip-net;opts:=rw,intr,rsize=1024,wsize=1024,timeo=20,retrans=10 \ wire!=slip-net;opts:=rw,intr

11.8 `/tftpboot' in a chroot-ed environment

In this complex example, we attempt to run an Amd process inside a chroot-ed environment. `tftpd' (Trivial FTP) is used to trivially retrieve files used to boot X-Terminals, Network Printers, Network routers, diskless workstations, and other such devices. For security reasons, `tftpd' (and also `ftpd') processes are run using the chroot(2) system call. This provides an environment for these processes, where access to any files outside the directory where the chroot-ed process runs is denied.

For example, if you start `tftpd' on your system with

chroot /tftpboot /usr/sbin/tftpd

then the `tftpd' process will not be able to access any files outside `/tftpboot'. This ensures that no one can retrieve files such as `/etc/passwd' and run password crackers on it.

Since the TFTP service works by broadcast, it is necessary to have at least one TFTP server running on each subnet. If you have lots of files that you need to make available for `tftp', and many subnets, it could take significant amounts of disk space on each host serving them.

A solution we implemented at Columbia University was to have every host run `tftpd', but have those servers retrieve the boot files from two replicated servers. Those replicated servers have special partitions dedicated to the many network boot files.

We start Amd as follows:

amd /tftpboot/.amd amd.tftpboot

That is, Amd is serving the directory `/tftpboot/.amd'. The `tftp' server runs inside `/tftpboot' and is chroot-ed in that directory too. The `amd.tftpboot' map looks like:

# # Amd /tftpboot directory -> host map # /defaults opts:=nosuid,ro,intr,soft;fs:=/tftpboot/import;type:=nfs tp host==lol;rfs:=/n/lol/import/tftpboot;type:=lofs \ host==ober;rfs:=/n/ober/misc/win/tftpboot;type:=lofs \ rhost:=ober;rfs:=/n/ober/misc/win/tftpboot \ rhost:=lol;rfs:=/n/lol/import/tftpboot

To help understand this example, I list a few of the file entries that are created inside `/tftpboot':

$ ls -la /tftpboot dr-xr-xr-x 2 root 512 Aug 30 23:11 .amd drwxrwsr-x 12 root 512 Aug 30 08:00 import lrwxrwxrwx 1 root 33 Feb 27 1997 adminpr.cfg -> ./.amd/tp/hplj/adminpr.cfg lrwxrwxrwx 1 root 22 Dec 5 1996 tekxp -> ./.amd/tp/xterms/tekxp lrwxrwxrwx 1 root 1 Dec 5 1996 tftpboot -> .

Here is an explanation of each of the entries listed above:

.amd

This is the Amd mount point. Note that you do not need to run a separate Amd process for the TFTP service. The chroot(2) system call only protects against file access, but the same process can still serve files and directories inside and outside the chroot-ed environment, because Amd itself was not run in chroot-ed mode.

import

This is the mount point where Amd will mount the directories containing the boot files. The map is designed so that remote directories will be NFS mounted (even if they are already mounted elsewhere), and local directories are loopback mounted (since they are not accessible outside the chroot-ed `/tftpboot' directory).

adminpr.cfg

tekxp

Two manually created symbolic links to directories inside the Amd-managed directory. The crossing of the component `tp' will cause Amd to automount one of the remote replicas. Once crossed, access to files inside proceeds as usual. The `adminpr.cfg' is a configuration file for an HP Laser-Jet 4si printer, and the `tekxp' is a directory for Tektronix X-Terminal boot files.

tftpboot

This innocent looking symlink is important. Usually, when devices boot via the TFTP service, they perform the `get file' command to retrieve file. However, some devices assume that `tftpd' does not run in a chroot-ed environment, but rather "unprotected", and thus use a full pathname for files to retrieve, as in `get /tftpboot/file'. This symlink effectively strips out the leading `/tftpboot/'.