NetBSD-SoC: rewrite pkgsrc wrapper framework in C
What is it?pkgsrc is a package system designed for portability across multiple Unix-like operating systems. It achieves most of its portability, not on a package-by-package basis, but rather through subsystems integrated into its infrastructure. Among these subsystems is the "wrapper framework", a set of scripts that intercept calls to the compiler and linker and perform various transformations before passing the calls through. For compatibility with typical third-party software, the transformations generally act to map GCC options to those of the actual system compiler. Individual packages needn't do anything special to take advantage; the wrappers work automatically behind the scenes. Given an application written for all-the-world's-a-Linux, it often happens that simply adding it to pkgsrc lets it build on foreign platforms such as Solaris or IRIX, thanks in no small part to the wrappers.
The current implementation of the wrapper framework is a complex admixture of sh, awk, sed, and other tools. Its current form evolved out of a Makefile-centric build system to meet an evolving understanding of the domain requirements. The wrapper framework has worked reasonably well for some time, but its internals are difficult to understand and it is a known performance bottleneck. Both faults are largely attributable to the languages of implementation.
Now that the requirements are better understood and the bottleneck has been measured as significant, pkgsrc would benefit greatly from having the wrappers reimplemented in C, following a test-first methodology. Writing tests first will ensure both that the intended behaviors are clearly documented and that the code behaves as intended; writing in C will improve both performance and clarity of the code.
March 29, 2009
The project was not completed for SoC 2008, but it has not been forgotten. The initial tests have been tweaked to run with the new atf-check syntax in atf06. Work will continue as time permits.
July 26, 2008
Blog created. Progress will be described here.
July 12, 2008
pkgsrc branch created ("cwrapper") and the first bits of work have been added to pkgtools/wrapper on the branch.
- April 21, 2008: Community Bonding Period -- Students get to know mentors, read documentation, get up to speed to begin working on their projects.
- May 26, 2008: Students begin coding for their GSoC projects; Google begins issuing initial student payments
- July 7, 2008: Mentors and students can begin submitting mid-term evaluations.
- July 14, 2008: Mid-term evaluation deadline; Google begins issuing mid-term student payments provided passing student survey is on file.
- August 11, 2008: Suggested 'pencils down' date. Take a week to scrub code, write tests, improve documentation, etc.
- August 18, 2008: Firm 'pencils down' date. Mentors, students and organization administrators can begin submitting final evaluations to Google.
- September 1, 2008: Final evaluation deadline; Google begins issuing student and mentoring organization payments provided forms and evaluations are on file.
- Documentation on all transformations, in the form of automated ATF test cases
- Documentation in other forms, where suitable and beneficial
- Drop-in replacement for wrapper framework
- Measurable increase in speed
DocumentationFine-grained automated test coverage (see "Deliverables", above) will provide reliable, easy-to-understand information about the behavior of the code. This is the primary form of documentation to be delivered. Additional verbiage will be conjured where needed.
Technical DetailsYes, definitely.