Tribblix 0m5 - solidification

In Tribblix Milestone 5, there's the dual element of increasing solidity and new development.

First, the new development: ZAP is a simple network package install utility. As in, really simple. Use it like so (as root):

zap install-overlay openexr

or

zap install TRIBpekwm

It should be obvious that it's nowhere near finished, but the necessary first step of having the command exist and the packages be available on the network has been achieved.

As part of that, the funky pkgs.zlib file on the iso that used to be lofi mounted for package installtion has gone. Instead, there's a directory with packages (in zap format) inside it. This is far simpler, and is also much quicker. With a little extra care in package construction, it's also smaller.

Next, a reversion. I've reverted the compiler and toolchain back to gcc3, as in earlier versions and matching OpenIndiana. Migrating to gcc4 is still a target (and is necessary for some newer software) but it has to be done right, and I'm not entirely happy with the gcc4 builds I've been testing. get the system compiler and toolchain wrong, and it's a mistake you have to live with for years.

And there's some polish. Most of this is covered by the change list. Many packages have been rebuilt, which can bring them up to date, optimize their space usage, or build them to my standards rather than importing them from OpenIndiana. Firefox is current, which is important. And there are little things, like including some themes for WindowMaker.

I've said before that there's no real roadmap or release schedule - this is, after all, largely a hobby project. And two months between milestones is rather longer than I would have liked. But to give you a flavour of what might be coming up - gcc4 done right, upgrades, LibreOffice, and working zones are all targets. (Of course, there's significant work in all those areas.)

Zip Archive Packaging

Under the hood, Tribblix uses the traditional SVR4 packaging utilities. There are a number of reasons for this - compatibility, simplicity, and a low footprint are among them. They're also good enough to get the job done. (And my strong belief is that the underlying package tools should become invisible and thus their implementation irrelevant, so the simpler and smaller the better.)

While SVR4 packaging does support installation of packages from networked locations over http, the support isn't great. The native support was almost never used in practice and its implementation is pretty poor (so much so that I would much rather just rip it out to simplify the code).

Allowing package installation from network repositories is expected of any modern system. However, the packaging system itself doesn't need to do so natively. There are any number of utilities and toolkits to do the network retrieval part - curl, wget, and essentially every modern scripting language will do the job.

Which leaves only the question as to what format to use in putting the data on your networked repository. The requirements here are:

• A package is packed up into a single file, to allow easy and efficient transfer using any medium
• The package should be compressed
• The contents of the package should be easily accessible on any platform without special tools
• A file should be able to contain multiple packages

If you look at SVR4 packaging, it has two native formats - filesystem and datastream. The former is simply all the files in the package laid out in a directory hierarchy, the latter is a single-file format. However, package datastream isn't generally suitable - it isn't natively compressed, and it's a private format that can't be easily accessed without the SVR4 tools.

The alternative solution I'm using is to simply zip up the filesystem format into a zip file. Hence, Zip Archive Packaging or zap for short.

This has the following advantages:

• Single file, can contain multiple packages
• Natively compressed
• Widespread support to unpack the archives
• Efficient random access
• Efficient extraction of list of contents
• Widely used in other contexts (eg. jar, war files)
• Some level of data integrity checking
• No need for any additional tools
• Supports extensibility for additional functionality later

Now, the standard widely used versions of zip don't support much compression beyond DEFLATE. Newer versions do, but availability isn't universal. So I limit myself to basic DEFLATE - although you can compress better than regular zip.

So installing a package from a network repo in Tribblix is down to a very simple shell script that runs curl + unzip + pkgadd.