Tuesday, September 29, 2015

Improving the foundations of Tribblix

Mostly due to history, the software packages that make up Tribblix come from 3 places.
In the first category, I'm using an essentially unmodified illumos-gate. The only change to the build is the fix for 5188 so that SVR4 packaging has no external dependencies (or the internal one of wanboot). I then create packages, applying a set of transforms (many of which simply avoid delivering individual files that I see no valid reason to ship - who needs ff or volcopy any more?).

The second category is the historical part. Tribblix was bootstrapped from another distro. Owing to the fact that the amount of time I have is rather limited, not all the bits used in the bootstrapping have yet been removed. These tend to be in the foundations of the OS, which makes them harder to replace (simply due to where they sit in the dependency tree).

In the latest update (the 0m16 prerelease) I've replaced several key components that were previously inherited. Part of this is so that I'm in control of these components (which is a good thing), another is simply that they needed upgrading to a newer version.

One key component here is perl. I've been building my own versions of perl to live separately, but decided it was time to replace the old system perl (I was at 5.10) with something current (5.22). This of itself is easy enough. I then have to rebuild illumos because it's connected to perl, and that's a slightly tricky problem - the build uses the Sun::Solaris module, which comes from the build. (Unfortunately it uses the copy on the system rather than the bits it just built.) So I had to pull the bits out from the failed build, install those on the system, and then the build goes through properly.

Another - more critical - component is libxml2. Critical because SMF uses it, so if you get that wrong you break the system's ability to boot. After much careful study of both the OmniOS and OpenIndiana build recipes, I picked a set of options and everything worked first time. Phew!

(Generally, I'll tend to the OpenIndiana way of doing things, simply because that's where the package I'm trying to replace came from. But I usually look at multiple distros for useful hints.)

A broader area was compression support. I updated zlib along with libxml2, but also went in and built my own p7zip, xz, and bzip2, and then started adding additional compression tools such as lzip and pigz.

The work isn't done yet. Two areas I need to look at are the Netscape Portable Runtime, and the base graphics libraries (tiff, jpeg, png). And then there's the whole X11 stack, which is a whole separate problem - because newer versions start to require KMS (which we don't have) or have gone 64-bit only (which is still an open question, and a leap I'm not yet prepared to take).


UX-admin said...

"(or OpenSXCE for SPARC)"

Wait, does this mean you have a copy of Martin's source code repository, or just that you took some binary packages from OpenSXCE???

If you have a copy, where can I clone his repository???

Another thing I meant to ask is, I looked over your ips2svr.sh (I think that was what it was called), and it looks like you are doing IPS manifest transforms, yet up to snv_130, the build engine produced SVR4 packages. Why didn't you just revert those bits of the build engine, and have it produce SVR4 packages directly?

Peter Tribble said...

For OpenSXCE, no source, just binary packages. Eventually they'll be replaced, it just takes time, which is very limited.

I use the IPS manifests because it's so much easier. They're maintained by someone else and always match the source code. It would be a huge amount of effort to manually maintain separate packaging metadata, there would be errors due to it going out of sync, and given limited time it makes sense to do the simplest thing and simply automate the process away.