&

NetBSD-SoC: Customizable install tool

What is it?

This project is to provide the NetBSD operating system with a flexible, customizable as well as automatable installation tool. The current utility for instaliing NetBSD on a computer is sysinst. Sysinst is a basic text-based program that is both higly technical and highly interactive. That is, it requires user input at many different points throughout an installation, which requires the user to wait while the installer finishes the various steps. This is unacceptable for both new users and those who would like to make many quick, identical installations, like in a classroom setting. This project will split sysinst into a back end as well as a front end. The idea is that the back end can do a complete installation all at once based on a configuration file either created in advance or made on-the-fly at the time of installation. This updated utility will make NetBSD's installation process much more usable to novices and pros alike.

The project involves two main areas: the creation of an installation configuration file format and the splitting of sysinst into a front end and a back end. The file format must be made so that installations can be described either generally or specifically. It must be able to convey every option that a user can make in the front end, and should be amenable to hand editing for advanced users. The second must dissociate a significant amount of code that is at present tightly integrated, while still maintaining the same functionality. There will be three areas to the code base: the front end, the back end, and the parser (in the form of a library). The front end must be able to parse files, because default installations will be described in the same configuration format, and the front end must disambiguate it to create an exact specification. The front end will also use the parser library in order to write out the files. The project will deliver the file format standard, specified in the form of a man page, the new installation tool, and updated documentation on the tool itself.

Throughout the project period I will be discussing the project on the tech-install mailing list.

You can see the latest changes (as far as they are committed) here, in the CVS browser: http://netbsd-soc.cvs.sourceforge.net/netbsd-soc/install-tool/

Status

Here I will list the milestones in the project as I reach them.

• May 26: Project started. A sample configuration file uploaded to CVS. It's pretty similar to what was envisioned before.
• June 1: The basic parser is imported. It was decided to write the parser first, as it can be complicated and will lead to changes in the syntax of the .conf file (it did). The parser will also define the intermediary data structures used to communicate between the parts of sysinst. As of now, the parser can correctly analyze the entire .conf file, but it only tokenizes it, nothing is done with the data.
• June 7: Some basic data structures have been added to the parser for kernels and sets, and the relevant parts of the parser now generate the appropriate data. A basic framework for writing the .conf file has been implemented as well. One interesting thing is that the data is written in the reverse of the order it was read in, due to the way yacc works (look up left-hand and right-hand recursion).
• July 14: Been a while since I updated this page... The back end is coming along nicely, but along the way the parser has had to be modified some. Due to the way yacc works, there are now two data structures used by the parsing library, one for internal use when reading a .conf file through yacc, and one to interface to externally in the front and back ends. The parser is a complete library now, all of its functionality is wrapped up in a few functions. This is implemented as a shared object for now during development along with a "sysinst.h" header. Once it's ready for full installs it will likely need to be converted into a static object for use on install media.

Basic Calendar

Below you see the rough outline of the project timeline. The bold dates are the important Google-set dates; the others are specific to my project and are likely to be swapped around. The nature of this project involves many separate components that will be completed asynchronously. The main flow of progress should remain the same throughout though.

• May 26: Students begin coding for their GSoC projects; Google begins issuing initial student payments
• May 28: Configuration file standard posted as a rough draft; Public request for suggestions on the standard.
• May 29: Begin splitting sysinst into front end and back end (no change in function, just in organization of source files).
• June 9: Front end and back end totally separated, begin work on back end.
• June 23: Back end completed except for config file parsing. Start to finalize standard.
• June 30: Back end completed, posted for testing. Standard completed and posted as man page(?). "Optional" components of the standard will likely be mentioned in order to be implemented either during the project or afterwards by me or anyone else.
• July 7: Mentors and students can begin submitting mid-term evaluations.
• July 14: Mid-term evaluation deadline; Google begins issuing mid-term student payments provided passing student survey is on file.
• July 14: Front end completed, can be ran to create full conf. file. Public request for translation of menu items. Start integrated front end and back end.
• August 4: Updated sysinst man page completed, all integration between front and back end done with command line options implemented.
• August 18: From August 4 until here all work is on the "optional" components listed in the application.
• August 18: Firm 'pencils down' date. Mentors, students and organization administrators can begin submitting final evaluations to Google.
• September 1: Final evaluation deadline; Google begins issuing student and mentoring organization payments provided forms and evaluations are on file.

Deliverables

Mandatory (must-have) components:

There are a lot of details here, because the configuration file must specify everything that the user can in the current sysinst.

• A complete configuration file standard, published as a man page.
• A "libsysinst" library. This itself will consist of many portions:
  • All of the "glue" between the front end and back end, i.e. all of the common functions.
  • A parsing library, that turns a configuration file into the internal data structures used for communicating between the front or back end and the library itself.
  • A configuration file writer. This takes the above data structures and simply writes them out to a new configuration file.
  • The idea behind the library is that the configuration file format is completely invisible to the front and back end. This way, any of the three components can be radically changed, as long as they keep the above data structures, and it will still work with the other components.
• A front end, very similar to the current one, where the user can specify all the same options. The difference is that they are only written to a file, no other action is taken.
• A back end, that can take a configuration file as input and without intervention do a complete install. There are flexible components of the configuration file that depend on information not known when the file is being written, such as the RAM size. The back end must recognize these abstractions and act appropriately.

Optional (would-be-nice) components:

This is a pretty long list. We'll see how much stuff I can fit in. There is quite a lot of room for further improvement. This is listed roughly in order of importance.

• The configuration file supports many flexible options where an install can be vaguely specified (as described above). Initially, these options will only be available by hand-editing the configuration file. These options should be made available to the front end as well.
• Changing the flow of the front end, to be less serial in nature. The ideal setup would give a list of tasks that the user needs to complete, and the user can do them in any order (or not at all). I.e. the user can jump in and just specify the sets they want to install, and everything else goes by default.
• Retain comments and whitespace when a configuration file is ran through the parser and printed out again. This should be done with very minimal changes to the parser. It's rather difficult, but possible.
• Adding lists of pkgsrc packages to be installed post-installation
• The ability to pull configuration files directly from the net.
• More here later...

Documentation

[Insert use and test cases, instruction for integration into NetBSD, installation, configuration, using; reference existing standards or other existing works where appropriate]

Technical Details

Mostly just existing sysinst stuff! Additionally, the parser uses lex and yacc. More here later as I figure out what exactly the project is going to require (though it's likely not much more than what's here).

[Documentation of existing NetBSD interfaces used, new interfaces created, pointers to header-files, section .2/.3/.9 manpages created and used, etc.]

Zachary Wegner <zwegner@gmail.com> $Id: index.html,v 1.7 2008/07/14 18:56:51 zwegner Exp $