User's Guide To Trapping Directory Lookup Operations in Tmpfs
Version 0.2


1. INSTRUCTIONS FOR THE IMPATIENT

	% modprobe tmpfs
	% mount -t tmpfs -o helper=/path/to/helper/program whatever /mnt
	% ls /mnt/foo
	# Notice that "/path/to/helper/program LOOKUP foo" was executed.


2. OVERVIEW (from the Kconfig help)

	Tmpfs now allows user level programs to implement file systems
that are filled in on demand.  This feature works by invoking a
configurable helper program on attempts to open() or stat() a
nonexistent file.  The access waits until the helper finishes, so the
helper can install the missing file if desired.

	Using this facility, a shell script or small C program can
implement a file system that automatically mounts remote file systems
or creates device files on demand, similar to autofs or devfs,
respectively.  Tmpfs is, however, daemonless and, perhaps
consequently, smaller than either of these, and may avoid some
recursion problems.

	Tmpfs might also be useful for debugging programs where you
want to trap the first access a particular file or perhaps in
automatic installation of missing command or libraries by specifying a
tmpfs file system in certain search paths.

	This access trapping facility is designed to be easily ported
to other file systems.  Kernel developers should examine the following
source files for more information:

	include/linux/fsuserhelper
	include/linux/lookuptrap.h
	fs/lookuptrap.c
	fs/userhelper.c
	mm/shmem.c (for an example of tmpfs using trapping_lookup)


3. GETTING STARTED

3.1 MOUNTING THE FILE SYSTEM

	First, build and boot a kernel with tmpfs either compiled in
or built as a module.  If you compile tmpfs as a module, you may have
to load it, although, since the module name ("tmpfs.ko") and the file
system name ("tmpfs") match, that may be unnecessary if you've
configured modprobe automatically.

	% modprobe tmpfs

	Now let's mount a tmpfs file system on /mnt.

	% mount -t tmpfs blah /mnt

	The file system will behave exactly like a ramfs file system.
In fact, tmpfs is derived from ramfs.  You can create files,
directories, symbolic links and device nodes in it, and they will
exist only in the computer's main memory.  The contents of the file
system will disappear as soon as you unmount it.

	If you mount multiple instances of tmpfs, you will get
separate file systems.


3.2. THE HELPER PROGRAM

	What distinguishes tmpfs from ramfs is that it can invoke a
user level helper program when an attempt is made to open or stat a
nonexistent file for the first time (if the name is not already in the
dcache).  The user level program is set with the "helper" mount
option.  It is possible to set, clear or change the helper command at
any time, so let's go back to our example and put a helper command on
the tmpfs file system that we mounted on /mnt:

	% mount -o remount,helper=/tmp/helper /mnt

	If you use the file system in /mnt now, nothing appears to have
changed.  Now let's put a simple shell script in /tmp/helper:

	% cat > /tmp/helper
	#!/bin/sh
	echo "$*" > /dev/console
	^D
	% chmod a+x /tmp/helper

	Now you should see console messages like "LOOKUP foo" when you
try to access the file /mnt/foo for the first time.

	You can also pass arguments to the helper program by using
spaces in the helper mount option, like so:

	% mount -o remount,helper='/tmp/helper my_argument' /mnt

	If you do this, your console messages will start to look
something like "my_argument LOOKUP foo".  The arguments that
you specify come before "LOOKUP foo" to facilitate the use of
command interpreters, like, say, helper='/usr/bin/perl handler.pl'.
Arguments also make it easy to pass things like the mount point or
configuration files, which should make it easier to write facilities
that work on multiple mount points.

	You can also deactivate the helper at any time, like so:

	mount -o remount,helper='' /mnt

4. PRACTICAL EXAMPLES

4.1 AN NFS AUTOMOUNTER

	% cat > /usr/sbin/tmpfs-automount
	#!/bin/sh
	topdir=$1
	host=${3%/*}
	dir=$topdir/$host
	mkdir $dir
	mount -t nfs $host:/ $dir
	^D
	% chmod a+x /usr/sbin/tmpfs-automount
	% mkdir /auto
	% mount -t tmpfs -o helper="/usr/sbin/tmpfs-automount /auto" x /auto

	Notice how we pass the additional argument "/auto" to the
tmpfs-automount command.

	If you want automatic unmount after a timeout, you'll probably
want to do something a little more elaborate, perhaps with a script that
runs from cron.


4.2 DEMAND LOADING OF DEVICE DRIVERS

	A version of devfs that uses tmpfs is under development and
running on the system I am using to write this document, but I am
still cleaning it up.  Here is how it should work, although I have
not yet actually tried devfs_helper on it.

	The devfs_helper program was originally written for a stripped down
rewrite of devfs, from which tmpfs is derived.  It can read your
/etc/devfs.conf file (the file previously used to configured
devfsd) and load modules specified by "LOOKUP" commands.  Other
devfs.conf command are ignored.

	% ftp ftp.yggdrasil.com
	login: anonymous
	password; guest
	ftp> cd /pub/dist/device_control/devfs
	ftp> get devfs_helper-0.2.tar.gz
	.....
	ftp> quit
	% tar xfpvz devfs_helper-0.2.tar.gz
	% cd devfs_helper-0.2
	% make
	% make install
	% mkdir /tmp/tmpdev
	% mount -t devfs /tmp/tmpdev
	% cp -apRx /dev/* /tmpdev/
	% mount -t devfs -o helper=/sbin/devfs_helper blah /dev
	% mount -t msods /dev/floppy/0 /mnt

	The above example should load the floppy.ko kernel module
if you have a a line in your /etc/devfs.conf file like this:

	LOOKUP	floppy		EXECUTE modprobe floppy


	You should also be able to use execfs in this fashion to get
automatic loading of kernel modules on non-devfs systems, although
you'll need something like udev the larger udev to create the
device files once the device drivers are registered.


4.3 DEBUGGING A PROGRAM TRYING TO ACCESS A FILE

	% cat > /tmp/call-sleep
	#!/bin/sh
	sleep 30
	^D
	% mount -t tmpfs -o helper=/tmp/call-sleep foo /mnt
	% mv .bashrc .bashrc-
	% ln -s /mnt/whatever .bashrc
	% gdb /bin/sh
	GNU gdb 5.2
	[blah blah blah]
	(gdb) run
	[program eventually hangs.  Switch to another terminal session.  You
         cannot control-C out of it, a tmpfs bug from call_usermodehelper.]
	% ps axf
	[Find the process under gdb.  Let say it's pid 1152.]
	% kill -SEGV 1152
	% ps auxww | grep sleep
	[Find the sleeping tmpfs helper.  Let's say it's pid 1120.]
	% kill -9 1120
	[Now back at the first session, running gdb on /bin/sh.]
	Program received signal SIGSEGV, Segmentation fault.
	0xb7f303d4 in __libc_open () at __libc_open:-1
	-1      __libc_open: No such file or directory.
        	in __libc_open
		(gdb) where
	#0  0xb7f303d4 in __libc_open () at __libc_open:-1
	#1  0xb7f8b4c0 in __DTOR_END__ () from /lib/libc.so.6
	#2  0x080921ef in _evalfile (filename=0x80dc788 "/tmp/junk/.bashrc", flags=9)
	    at evalfile.c:85
	#3  0x08092635 in maybe_execute_file (
	    fname=0xfffffffe <Address 0xfffffffe out of bounds>, 
	    force_noninteractive=1) at evalfile.c:218
	#4  0x08059fe8 in run_startup_files () at shell.c:1019
	#5  0x08059849 in main (argc=1, argv=0xbfffebc4, env=0xbfffebcc) at shell.c:581
	#6  0xb7e88e02 in __libc_start_main (main=0x8059380 <main>, argc=1, 
	    ubp_av=0xbfffebc4, init=0x805897c <_init>, 
	    fini=0xb80005ac <_dl_debug_mask>, rtld_fini=0x8000, stack_end=0x0)
	    at ../sysdeps/generic/libc-start.c:129


4.4 AUTOMATIC LOADING OF MISSING PROGRAMS

	% cat > /usr/sbin/missing-program
	#!/bin/sh
	my-automatic-network-downloader $2
	^D
	% chmod a+x /usr/sbin/missing-program
	% mount -t tmpfs -o helper=/usr/sbin/my-automatic-installer /mnt
	% PATH=$PATH:/mnt:$PATH
	# We include $PATH a second time so that the program can be
	# found after it is installed.
	% kdevelop		# Or some other program you don't have...

	...or maybe something like this...

	% cat > /usr/sbin/missing-program
	#!/bin/sh
	export DISPLAY=:0
	konqueror http://www.google.com/search?q="download+$2" &
	^D
	% chmod a+x /usr/sbin/missing-program
	% mount -t tmpfs -o helper=/usr/sbin/missing-program glorp /mnt
	% xhost localhost
	% PATH=$PATH:/mnt
	% kdevelop

4.4.1 AUTOMATIC LOADING OF MISSING LIBRARIES

	Same as above, but with this line:
	% LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/mnt:$LD_LIBRARY_PATH

4.5  ADVANCED EXAMPLE: Generating plain files

	Doing automatic generation of plain files (as opposed to
directories, device files and symbolic links) requires more care,
because the helper program's attempt to open a file for writing
will itself invoke another instance of the user level handler.

	One approach, perhaps the only one, is to define some temporary
file name pattern that your user handler knows to ignore, create the
file with a temporary name that matches that pattern, and then
rename the temporary file to the real file name, since rename
operations are not trapped (and if renames ever are trapped in future,
releases you could filter out renames where the source matched the
tempary file pattern).

	[FIXME. This example is untested!  --Adam Richter, 2004.11.03]


	% cat > /usr/sbin/my-decrypter
	#!/bin/sh
	encrypted_dir=$2
	decrypted_dir=$3
	key=$4
	# $5 is "LOOKUP"
	filename=$6

	case $target in ( tmp/* ) ; exit ;; esac
	tmpfile=$decrypted_dir/$$
	decrypt --key $key < $encrypted_dir/$filename > $tmpfile
	mv $tmpfile $decrypted_dir/$filename
	^D
	% chmod a+x /usr/sbin/my-decrypter
	% mount -t tmpfs -o \
	   helper='/usr/sbin/my-decrypter /cryptdir /mnt key' x /mnt
	% cat /mnt/my-secret-file
	....

	If you want to make the temporary directory completely
inaccessible from the public directory, you can create two mount
points, where the public directory for the file system is actually
a subdirectory of a larger hidden directory, like so:

	% mount -t tmpfs /some/place/hidden
	% chmod go-rwx /someplace/hidden
	% mkdir /some/place/hidden/mirror
	% mount --bind /some/place/hidden/mirror /public

	Now you can set up a helper program that operates in some
other directory of /some/place/hidden, and then renames the
resultant files into /some/place/hidden/mirror.


4.5.1 A WARNING ABOUT SYMBOLIC LINKS AND THE GNU "ln -s" COMMAND

	If your helper program invoke the shell command "ln" to create
symbolic links, you may need to use "filter and rename" technique that
was described above for plain files, or one of several other
workarounds listed below.  If you helper program just uses the
symlink() system call directly, you don't have to worry about this.

	What is the problem, exactly?  The problem is that although
symlink system calls are not trapped, the "ln" command from version
5.2.1 of the GNU coreutils package (latest version as of this writing)
does a stat system call on the target before doing the symlink,
because it is specified to behave differently if the destination path
exists and is a directory.  stat, unlink symlink, is trapped, in order
to support automatic creation of files in case a program stats a file
before deciding whether to open it, and for things like "ls
/auto/fileserver1.mycompany.com/".  Consequently, a user level helper
shell script that simply does "ln -s whatever $target_file_name" will
deadlock (which can be broken by interrupting the child helper
program).

	There are several possible solutions to this problem.

	1. You can use a simpler symlink program, such as ssln,
	   instead of "ln -s".

	2. If the final path element of the symlink's contents is
	   the same as the final path element of the symlink's name,
	   then you can just specify the directory name as the second
	   argument to "ln -s".  In other words, change

			ln -s foo/bar /the/target/bar
			...to...
			ln -s foo/bar /the/target

	3. You can use the same filtering techniques for symlink
	   as discussed for plain files in the previous section
	   (create them with speical temporary names that your helper
	   program knows to ignore and then rename them into place).

	4. Invoke perl to do the symlink, if you know you have it available:

			perl -e 'symlink("contents", "target");'

	5. You can port shell script to C or perl or some other
	   language that gives you direct access to the symlink()
	   system call.


	Perhaps, in the future, the GNU ln command could be changed so
that, when it is called with exactly two file names, it would try to
do the symlink() and then check if the target is a directory only if
the symlink attempt failed.


4.6. OTHER USES?

     I would be interested in hearing about any other uses that you up
with for tmpfs, especially if I can include them in this document.

5. SERIALIZATION

	Note that many instances of the user level helper program
can potentially be running at the same time.  It is up to "you",
the implementor of the helper program to determine what sort of
serialization you need and implement it.  A simple solution to
enforce complete serialization would be to have every instance
of the helper program take an exclusive flock on some common
file.

6. KERNEL DEVELOPER ANSWERS ABOUT IMPLEMENTATION DECISIONS

6.1 Q:  Why doesn't tmpfs provide REGISTER and UNREGISTER events when
        new nodes are created or deleted from the file system, as the
	mini-devfs implementation from which is derived did?

    A:  {,UN}REGISTER in Richard Gooch's implementation of devfs enabled
        things like automatically setting permissions and sound settings
	on your sound device when the driver was loaded, even if
	the loading of the driver had not been caused by devfs.
	For tmpfs-based devfs, I expect to implement that in
	a more complex way by shadowing the real devfs file system
	and creating {,UN}REGISTER events as updates are propagated
	from the real devfs to /dev.  The advantages of this would be
	that module initialization would not be blockable by a user
	level program, and events like a device quickly appearing and
	disappearing could be coalesced (i.e., ignored in this case).

	I'm not convinced that {,UN}REGISTER has to go, but I haven't
	seen any compelling uses for it, and I know it's politically
	easier to add a feature than to remove one, especially if anyone
	has developed a dependence on it and does not want to port.  So,
	I'm starting out with tmpfs not providing {,UN}REGISTER events.

6.2 Q:	Why isn't this facility implemented as an overlay file system?  I'd
	like to be able to apply it to, say, /dev, without having
	to start out with /dev being a devfs file system.  I could
	also demand load certain facilities based on accesses to /proc.

    A:	There are about two dozen routines in inode_operations and
	file_operations that would require pass-through versions, and
	they are not as trivial as you might think because of
	locking issues involved in going through the vfs layer again.
	Also, currently, tmpfs, like ramfs and sysfs, use a struct
	inode and a struct dentry in kernel low memory for every
	existing node in the file system, about half a kilobyte
	per entry.  So, tmpfs would need to be converted to allow
	inode structures to be released if it were to overlay
	potentially large directory trees.  Also there are issues
	related to the underlying file system changing "out from
	under" tmpfs.  Perhaps in the future this an be implemented.


6.3. Q: Why isn't tmpfs a built-in kernel facility that can be
	applied to any file, like dnotify?  That would also have
	the above advantages and could eliminate the mount complexity
	of overlays.

     A:	I thought about defining a separating inode->directory_operations
	from inode->inode_operations. Since all of those operations that
	I want to intercept are called with inode->i_sem held, it
	follows that it would be SMP-safe to change an inode->dir_ops
	pointer dynamically, which would allow stacking of inode
	operations.  This approach could be used to remove the special
	case code that is used to implement dnotify.  But what happens
	when the inode is freed?  It would be necessary to intercept
	the victim superblock's superblock_operations.drop_inode
	routine, which could get pretty messy, especially, if, for
	example, more than one trap was being used on the same file
	system.  Perhaps if drop_inode were moved to struct
	inode_operations this would be easier.


Adam J. Richter (adam@yggdrasil.com)
2004.11.02
