Hlfsd is a daemon which implements a filesystem containing a
symbolic link to subdirectory within a user's home directory, depending
on the user which accessed that link. It was primarily designed to
redirect incoming mail to users' home directories, so that it can be read
from anywhere. It was designed and implemented by
Erez Zadok and
Alexander Dupuy, at the
Computer Science Department of
Columbia University. A
paper
on Hlfsd was presented at the Usenix LISA VII conference in 1993.
Hlfsd operates by mounting itself as an NFS server for the directory
containing linkname, which defaults to `/hlfs/home'. Lookups
within that directory are handled by Hlfsd, which uses the
password map to determine how to resolve the lookup. The directory will
be created if it doesn't already exist. The symbolic link will be to
the accessing user's home directory, with subdir appended to it. If
not specified, subdir defaults to `.hlfsdir'. This directory
will also be created if it does not already exist.
A `SIGTERM' sent to Hlfsd will cause it to shutdown. A
`SIGHUP' will flush the internal caches, and reload the password
map. It will also close and reopen the log file, to enable the original
log file to be removed or rotated. A `SIGUSR1' will cause it to
dump its internal table of user IDs and home directories to the file
`/tmp/hlfsddump'.
Electronic mail has become one of the major applications for many
computer networks, and use of this service is expected to increase over
time, as networks proliferate and become faster. Providing a convenient
environment for users to read, compose, and send electronic mail has
become a requirement for systems administrators (SAs).
Widely used methods for handling mail usually require users to be logged
into a designated "home" machine, where their mailbox files reside.
Only on that one machine can they read newly arrived mail. Since users
have to be logged into that system to read their mail, they often find
it convenient to run all of their other processes on that system as
well, including memory and CPU-intensive jobs. For example, in our
department, we have allocated and configured several multi-processor
servers to handle such demanding CPU/memory applications, but these were
underutilized, in large part due to the inconvenience of not being able
to read mail on those machines. (No home directories were located on
these designated CPU-servers, since we did not want NFS service for
users' home directories to have to compete with CPU-intensive jobs. At the
same time, we discouraged users from running demanding applications on
their home machines.)
Many different solutions have been proposed to allow users to read their
mail on any host. However, all of these solutions fail in one or more
of several ways:
they introduce new single points of failure
they require using different mail transfer agents (MTAs) or user agents
(UAs)
they do not solve the problem for all cases, i.e. the solution is only
partially successful for a particular environment.
We have designed a simple filesystem, called the Home-Link File
System, to provide the ability to deliver mail to users' home
directories, without modification to mail-related applications. We have
endeavored to make it as stable as possible. Of great importance to us
was to make sure the HLFS daemon, `hlfsd' , would not hang under
any circumstances, and would take the next-best action when faced with
problems. Compared to alternative methods, Hlfsd is a stable, more
general solution, and easier to install/use. In fact, in some ways, we
have even managed to improve the reliability and security of mail
service.
Our server implements a small filesystem containing a symbolic link
to a subdirectory of the invoking user's home directory, and named symbolic
links to users' mailbox files.
The Hlfsd server finds out the uid of the process that is
accessing its mount point, and resolves the pathname component `home' as a
symbolic link to a subdirectory within the home directory given by the
uid's entry in the password file. If the gid of the process
that attempts to access a mailbox file is a special one (called
HLFS_GID), then the server maps the name of the next pathname
component directly to the user's mailbox. This is necessary so that
access to a mailbox file by users other than the owner can succeed. The
server has safety features in case of failures such as hung filesystems
or home directory filesystems that are inaccessible or full.
On most of our machines, mail gets delivered to the directory
`/var/spool/mail'. Many programs, including UAs, depend on that
path. Hlfsd creates a directory `/mail', and mounts itself on
top of that directory. Hlfsd implements the path name component
called `home', pointing to a subdirectory of the user's home directory.
We have made `/var/spool/mail' a symbolic link to
`/mail/home', so that accessing `/var/spool/mail' actually
causes access to a subdirectory within a user's home directory.
The following table shows an example of how resolving the pathname
`/var/mail/NAME' to `/users/ezk/.mailspool/NAME' proceeds.
This section provides an in-depth discussion of why available methods
for delivering mail to home directories are not as good as the one used
by Hlfsd.
The most common method for mail delivery is for mail to be appended to a
mailbox file in a standard spool directory on the designated "mail
home" machine of the user. The greatest advantage of this method is
that it is the default method most vendors provide with their systems,
thus very little (if any) configuration is required on the SA's part.
All they need to set up are mail aliases directing mail to the host on
which the user's mailbox file is assigned. (Otherwise, mail is
delivered locally, and users find mailboxes on many machines.)
As users become more sophisticated, and aided by windowing systems, they
find themselves logging in on multiple hosts at once, performing several
tasks concurrently. They ask to be able to read their mail on any host
on the network, not just the one designated as their "mail home".
A popular method for providing mail readability from any host is to have
all mail delivered to a mail spool directory on a designated
"mail-server" which is exported via NFS to all of the hosts on the
network. Configuring such a system is relatively easy. On most
systems, the bulk of the work is a one-time addition to one or two
configuration files in `/etc'. The file-server's spool directory
is then hard-mounted across every machine on the local network. In
small environments with only a handful of hosts this can be an
acceptable solution. In our department, with a couple of hundred active
hosts and thousands of mail messages processed daily, this was deemed
completely unacceptable, as it introduced several types of problems:
Scalability and Performance
As more and more machines get added to the network, more mail traffic
has to go over NFS to and from the mail-server. Users like to run
mail-watchers, and read their mail often. The stress on the shared
infrastructure increases with every user and host added; loads on the
mail server would most certainly be high since all mail delivery goes
through that one machine.(2) This
leads to lower reliability and performance. To reduce the number of
concurrent connections between clients and the server host, some SAs
have resorted to automounting the mail-spool directory. But this
solution only makes things worse: since users often run mail watchers,
and many popular applications such as `trn', `emacs',
`csh' or `ksh' check periodically for new mail, the
automounted directory would be effectively permanently mounted. If it
gets unmounted automatically by the automounter program, it is most
likely to get mounted shortly afterwards, consuming more I/O resources
by the constant cycle of mount and umount calls.
Reliability
The mail-server host and its network connectivity must be very reliable.
Worse, since the spool directory has to be hard-mounted,(3) many processes which access the
spool directory (various shells, `login', `emacs', etc.)
would be hung as long as connectivity to the mail-server is severed. To
improve reliability, SAs may choose to backup the mail-server's spool
partition several times a day. This may make things worse since reading
or delivering mail while backups are in progress may cause backups to be
inconsistent; more backups consume more backup-media resources, and
increase the load on the mail-server host.
Despite the existence of a few systems that support delivery to users'
home directories, mail delivery to home directories hasn't caught on.
We believe the main reason is that there are too many programs that
"know" where mailbox files reside. Besides the obvious (the delivery
program `/bin/mail' and mail readers like `/usr/ucb/Mail',
`mush', `mm', etc.), other programs that know mailbox location
are login, from, almost every shell, `xbiff', `xmailbox', and
even some programs not directly related to mail, such as `emacs'
and `trn'. Although some of these programs can be configured to
look in different directories with the use of environment variables and
other resources, many of them cannot. The overall porting work is
significant.
Other methods that have yet to catch on require the use of a special
mail-reading server, such as IMAP or POP. The main disadvantage of
these systems is that UAs need to be modified to use these services ---
a long and involved task. That is why they are not popular at this
time.
Several other ideas have been proposed and even used in various
environments. None of them is robust. They are mostly very
specialized, inflexible, and do not extend to the general case. Some of
the ideas are plain bad, potentially leading to lost or corrupt mail:
automounters
Using an automounter such as Amd to provide a set of symbolic links
from the normal spool directory to user home directories is not
sufficient. UAs rename, unlink, and recreate the mailbox as a regular
file, therefore it must be a real file, not a symbolic link.
Furthermore, it must reside in a real directory which is writable by the
UAs and MTAs. This method may also require populating
`/var/spool/mail' with symbolic links and making sure they are
updated. Making Amd manage that directory directly fails, since
many various lock files need to be managed as well. Also, Amd does
not provide all of the NFS operations which are required to write mail
such as write, create, remove, and unlink.
$MAIL
Setting this variable to an automounted directory pointing to the user's
mail spool host only solves the problem for those programs which know
and use $MAIL. Many programs don't, therefore this solution is partial
and of limited flexibility. Also, it requires the SAs or the users to
set it themselves -- an added level of inconvenience and possible
failures.
/bin/mail
Using a different mail delivery agent could be the solution. One such
example is `hdmail'. However, `hdmail' still requires
modifying all UAs, the MTA's configuration, installing new daemons, and
changing login scripts. This makes the system less upgradable or
compatible with others, and adds one more complicated system for SAs to
deal with. It is not a complete solution because it still requires each
user have their $MAIL variable setup correctly, and that every program
use this variable.
There are several major reasons why SAs might want to deliver mail
directly into the users' home directories:
Location
Many mail readers need to move mail from the spool directory to the
user's home directory. It speeds up this operation if the two are on
the same filesystem. If for some reason the user's home directory is
inaccessible, it isn't that useful to be able to read mail, since there
is no place to move it to. In some cases, trying to move mail to a
non-existent or hung filesystem may result in mail loss.
Distribution
Having all mail spool directories spread among the many more filesystems
minimizes the chances that complete environments will grind to a halt
when a single server is down. It does increase the chance that there
will be someone who is not able to read their mail when a machine is
down, but that is usually preferred to having no one be able to read
their mail because a centralized mail server is down. The problem of
losing some mail due to the (presumably) higher chances that a user's
machine is down is minimized in HLFS.
Security
Delivering mail to users' home directories has another advantage ---
enhanced security and privacy. Since a shared system mail spool
directory has to be world-readable and searchable, any user can see
whether other users have mail, when they last received new mail, or when
they last read their mail. Programs such as `finger' display this
information, which some consider an infringement of privacy. While it
is possible to disable this feature of `finger' so that remote
users cannot see a mailbox file's status, this doesn't prevent local
users from getting the information. Furthermore, there are more
programs which make use of this information. In shared environments,
disabling such programs has to be done on a system-wide basis, but with
mail delivered to users' home directories, users less concerned with
privacy who do want to let others know when they last received or read
mail can easily do so using file protection bits.
In summary, delivering mail to home directories provides users the
functionality sought, and also avoids most of the problems just
discussed.
Much the same way Amd is controlled by `ctl-amd', so does
Hlfsd get controlled by the `ctl-hlfsd' script:
ctl-hlfsd start
Start a new Hlfsd.
ctl-hlfsd stop
Stop a running Hlfsd.
ctl-hlfsd restart
Stop a running Hlfsd, wait for 10 seconds, and then start a new
one. It is hoped that within 10 seconds, the previously running
Hlfsd terminate properly; otherwise, starting a second one could
cause system lockup.
For example, on our systems, we start Hlfsd within `ctl-hlfsd'
as follows on Solaris 2 systems:
hlfsd -a /var/alt_mail -x all -l /var/log/hlfsd /mail/home .mailspool
The directory `/var/alt_mail' is a directory in the root partition
where alternate mail will be delivered into, when it cannot be delivered
into the user's home directory.
Normal mail gets delivered into `/var/mail', but on our systems,
that is a symbolic link to `/mail/home'. `/mail' is managed
by Hlfsd, which creates a dynamic symlink named `home',
pointing to the subdirectory `.mailspool' within the
accessing user's home directory. This results in mail which normally
should go to `/var/mail/$USER', to go to
`$HOME/.mailspool/$USER'.
Hlfsd does not create the `/var/mail' symlink. This needs to
be created (manually) once on each host, by the system administrators,
as follows:
Alternate directory. The name of the directory to which the symbolic
link returned by Hlfsd will point, if it cannot access the home
directory of the user. This defaults to `/var/hlfs'. This
directory will be created if it doesn't exist. It is expected that
either users will read these files, or the system administrators will
run a script to resend this "lost mail" to its owner.
-c cache-interval
Caching interval. Hlfsd will cache the validity of home directories
for this interval, in seconds. Entries which have been verified within
the last cache-interval seconds will not be verified again, since
the operation could be expensive, and the entries are most likely still
valid. After the interval has expired, Hlfsd will re-verify the
validity of the user's home directory, and reset the cache time-counter.
The default value for cache-interval is 300 seconds (5 minutes).
-f
Force fast startup. This option tells Hlfsd to skip startup-time
consistency checks such as existence of mount directory, alternate spool
directory, symlink to be hidden under the mount directory, their
permissions and validity.
-g group
Set the special group HLFS_GID to group. Programs such as
`/usr/ucb/from' or `/usr/sbin/in.comsat', which access the
mailboxes of other users, must be setgid `HLFS_GID' to work properly. The
default group is `hlfs'. If no group is provided, and there is no
group `hlfs', this feature is disabled.
-h
Help. Print a brief help message, and exit.
-i reload-interval
Map-reloading interval. Each reload-interval seconds, Hlfsd
will reload the password map. Hlfsd needs the password map for the
UIDs and home directory pathnames. Hlfsd schedules a `SIGALRM' to
reload the password maps. A `SIGHUP' sent to Hlfsd will force it to
reload the maps immediately. The default value for
reload-interval is 900 seconds (15 minutes.)
-l logfile
Specify a log file to which Hlfsd will record events. If
logfile is the string `syslog' then the log messages will be
sent to the system log daemon by syslog(3), using the `LOG_DAEMON'
facility. This is also the default.
-n
No verify. Hlfsd will not verify the validity of the symbolic link
it will be returning, or that the user's home directory contains
sufficient disk-space for spooling. This can speed up Hlfsd at the
cost of possibly returning symbolic links to home directories which are
not currently accessible or are full. By default, Hlfsd validates
the symbolic-link in the background. The -n option overrides the
meaning of the -c option, since no caching is necessary.
-o mount-options
Mount options which Hlfsd will use to mount itself on top of
dirname. By default, mount-options is set to `ro'. If
the system supports symbolic-link caching, default options are set
to `ro,nocache'.
-p
Print PID. Outputs the process-id of Hlfsd to standard output where
it can be saved into a file.
-v
Version. Displays version information to standard error.
-x log-options
Specify run-time logging options. The options are a comma separated
list chosen from: `fatal', `error', `user', `warn', `info', `map', `stats', `all'.
-C
Force Hlfsd to run on systems that cannot turn off the NFS
attribute-cache. Use of this option on those systems is discouraged, as
it may result in loss or misdelivery of mail. The option is ignored on
systems that can turn off the attribute-cache.
-D log-options
Select from a variety of debugging options. Prefixing an option with
the string `no' reverses the effect of that option. Options are
cumulative. The most useful option is `all'. Since this option is
only used for debugging other options are not documented here. A fuller
description is available in the program source. A `SIGUSR1' sent
to Hlfsd will cause it to dump its internal password map to the file
`/usr/tmp/hlfsd.dump.XXXXXX', where `XXXXXX' will be replaced
by a random string generated by mktemp(3) or (the more secure)
mkstemp(3).
-P password-file
Read the user-name, user-id, and home directory information from the
file password-file. Normally, Hlfsd will use getpwent(3)
to read the password database. This option allows you to override the
default database, and is useful if you want to map users' mail files to
a directory other than their home directory. Only the username, uid,
and home-directory fields of the file password-file are read and
checked. All other fields are ignored. The file password-file
must otherwise be compliant with Unix Version 7 colon-delimited format
passwd(4).