NetBSD-SoC: Running Kernel File Systems in Userspace

What is it?

The main goal of the project is be able to compile the same file system code for running both in userspace and the kernel. This will provide several benefits:

easier file system development. for example adding journaling to ffs, zfs or any other serious file system project will be able to skip the step of first writing test code for a userspace environment and only then migrate the code to the kernel.
faster development cycle time. since a userspace file system will not bring the entire system down in case of a crash, the cycle time for testing modifications is reduced to time required to fsck instead of reboot + fsck.
better possibilities for testing. testing file system error paths for in-kernel file systems is difficult and tedious, since introducing an error to a function called by the file system under test will bring most any file system down. having an isolated environment makes injected faults also isolated. hence, it will enable to do easy regression testing for error paths also.
better componentization and specification of a file system's api. while file systems have a well-defined interface from above (vfs), they are generally free to use any and all functionality provided by the monolithic kernel. running the file system in an environment detached from the rest of the kernel forces some kind of division between the file system bottom end and the rest of the kernel, as it may not be worthwhile to attempt to emulate the entire kernel in userspace.
better debugging tools. kernel development requires a special environment which needs to be built and maintained. even though an emulator makes this relatively easy, the author has, for example, never been able to use kgdb to single-step through code.
stabilizing the puffs api/abi. this project has always loomed on the distant horizon as something that should maybe be enabled by puffs. it has made it difficult to decide if to expose via libpuffs less or more of what normal kernel file systems are exposed to.

A similar idea is running the entire operating system as a process. While it would address the tool-portion of the benefits, it would not address any other points.

Deliverables

Mandatory (must-have) components:

FFS code runnable both in userspace and the kernel + the necessary infrastructure to go with that. Notably, this might not be the same ffs code currently present in the NetBSD kernel. Also as notable, the user- and kernel versions will may (will?) require to be compiled separately. On the never ending list of notables, the NetBSD kernel might not be the one from cvs.

Documentation

Technical Details

See journal below. Reading it probably requires a fairly good understanding on file systems and virtual memory.

Status and Development Journal

August 5th, 2007

The code has been integrated into NetBSD and is being developed there from now on. You can find it here.

Look here for more verbose chattiness on what went on during development.

Antti Kantee <Antti.Kantee@gmail.com>

$Id: index.html,v 1.20 2007/08/06 12:13:50 thepooka Exp $