NetBSD-SoC: In-kernel implementation of posix_spawn

What is it?

A few years ago NetBSD got a real vfork(2) implementation and it was shown at the time, that this offers a measurable performance benefit for fork intensive work loads like build.sh release. vfork avoids the overhead of marking the process as copy on write. Fast forward to the current time. A lot of services are now implemented by multithreaded programs. vfork(2) and multithreading don't play well with each other. Additionally, SMP systems are the norm now and creating and shortly after killing an address space is not cheap. Processes have also grown a lot and a plain fork() is quite expensive from a 8GB JVM.

POSIX added an interface that for this purpose: posix_spawn. This is a combined fork+exec and can be used to implement popen and more advanced things. FreeBSD and some other systems have a userspace implementation, which works, but can't reap the benefits mentioned above.

Implementing a real posix_spawn system calls could do that. Measuring the impact for different work loads makes a nice research paper as side effect. This includes both estimate the temporary memory used and the performance implications.

Deliverables

Status

Aug 18th update

Deadline of the project is approaching. The core function is in an usable state while there are something to do.

The code is available at http://netbsd-soc.cvs.sourceforge.net/viewvc/netbsd-soc/posix_spawn/

Already Done
1. Port FreeBSD's userland implementation of posix_spawn
2. Implement posix_spawn in kernel space and its helper functions in libc. The propotype of some kernel routines is changed in order to be used by posix_spawn. Now posix_spawn can be used to spawn process. You can also pass posix_spawn_file_actions as parameters but posix_spawnattr has some bugs. 3. Write test cases for posix_spawn family

TODO
1. fix the remaining bugs including posix_spawnattr and others
2. performance test between kernel & userland version of posix_spawn
3. tune some programs to use posix_spawn, such as make or /bin/sh

Documentation

1. Single Unix Specification.
2. The Design and Implementation of the 4.4BSD Operating System
3. Design and Implementation of the UVM Virtual Memory System

Technical Details

[Documentation of existing NetBSD interfaces used, new interfaces created, pointers to header-files, section .2/.3/.9 manpages created and used, etc.]

Charles Zhang <$student_email>

$Id: index.html,v 1.3 2011/08/18 16:32:04 zhangchaoyang Exp $