Friday, March 22, 2019

Creating illumos packages for Tribblix

In the prior article in this series, I discussed how to build illumos-gate (and it applies to illumos-omnios too), using a Tribblix AMI on AWS.

After that, you'll have two directories of interest under illumos-gate.
  1. The proto area, specifically proto/root_i386, that is a fully installed copy of illumos.
  2. Under packages, an on-disk IPS package respoitory.
If you're using IPS, you could use that repository as is. Tribblix, however, uses SVR4 packages. So the next step is to convert the contents of the IPS repository into a set of SVR4 packages.

For this, there is an additional github repo you'll need to check out, in addition to the ones from the previous article.

cd ${HOME}/Tribblix
git clone https://github.com/tribblix/tribblix-transforms

What is this repo? It's a list of transformations that are applied to the package conversion process. Generally, rather than modify the gate or the build, if there's something I don't want to ship, or something I want to move between packages, or a file I want to change, then it gets transformed at the packaging stage.

The other thing you'll need to use my scripts is a signing certificate. In Tribblix, the illumos ELF objects are digitally signed (yes, I ought to extend this to all ELF objects I ship). Nothing actually uses this yet, but it's all there ready for when verification is required.

To create a signing key and certificate:

mkdir ${HOME}/f
cd ${HOME}/f
openssl req -x509 -newkey rsa:2048 -nodes \
  -subj "/O=MyOrganization/CN=my.domain.name" \
  -keyout elfcert.key -out elfcert.crt -days 3650
Of course, choose the subject to match your own requirements.

Then you can run the scripts in the tribblix-build repo to generate SVR4 packages. These parse the IPS manifests, extract the files from the IPS repo, create the SVR4 prototype file, and automatically generate SVR4 install scripts from the IPS metadata.



So, to create packages, run the following as root:

/illumos/Tribblix/tribblix-build/repo_all.sh \
  -T /illumos/Tribblix \
  -G /illumos/Illumos/illumos-gate \
  -S /illumos/f/elfcert

Of course, replace /illumos with wherever your build user's home directory is. (You can't use $HOME, because this is run as root where $HOME is likely to be different to that of the user who did the build.) The -T flag tells it where the Tribblix tools are checked out, -G where the gate was built, and -S where the signing certificate is. If you want to change the version of the packages you generate, there's a -V flag as well. I normally redirect the output to a log file, as it's quite verbose

Wait a while, and you'll have a set of packages under /var/tmp/illumos-pkgs. Hopefully the pkgs directory there will have your packages, and the build and tmp directories will be empty (if the build directory isn't, it will have a half-created package or packages in it, so you'll be able to see which package or packages failed).

For omnitribblix it's very similar:

/illumos/Tribblix/tribblix-build/omni_all.sh \
  -T /illumos/Tribblix \
  -G /illumos/Illumos/omnitribblix \
  -S /illumos/f/elfcert

And the packages in this case end up in /var/tmp/omni-pkgs.

If you look, you'll see that there are 3 files for each package:

  • A .pkg file, which is in SVR4 datastream format
  • A .zap file, which is the (compressed) zap format Tribblix uses
  • An md5 checksum, which is used for basic validation in the package catalog
I then use those generated SVR4 packages to populate the package repository, and to build the Tribblix ISOs. The details of that will have to wait for another time.

5 comments:

Anonymous said...

I loved zap very much but the dependence on overlays for dependency handling is a very bad thing. zap doesn't handle dependency for normal package. If you can, please implement dependency handling for packages, make zap become a normal package manager like what on Linux. If you don't have time, you could consider to use FreeBSD's pkg. zap would be still there for zone or other system maintainance jobs, it could become a central system managed tool, but the packages handling should devote to FreeBSD's pkg. Ravenports already ported FreeBSD's pkg to and use it on SunOS, so I think we could just took it and integrate to our system.

Thank you very much for Tribblix and sorry for my bad English :)

Peter Tribble said...

The use of overlays is a deliberate design decision.

The truth is that using package dependencies to manage the software on a system turns out to be a terrible idea. The fact that everybody does it that way, and thus gives a horrid experience to the user and administrator, doesn't make it sensible.

Anonymous said...

I respectfully disagree. For example, I only want fluxbox as my wm but I've to install the whole retro-desktop-extras which include many other wms just to have fluxbox. I want to install only XFCE, but the xfce overlay depends on retro-desktop overlay, do I really need it? I don't know how to use wmaker and the also bundled afterstep not even possible to start the X server. WTH going on here? Of course I could install just the fluxbox package, but since zap doesn't care about package dependency who know if it would work or just failed because missing some unknown packages it depends on?

We've a very different view on the problem and I don't expect you to change your mind. I think Tribblix by default bloated so much because of too many unnecessary packages installed by so called overlays, which is packages collection after all.

Anonymous said...

Could you update Firefox to OpenIndiana version? TRIBbrowser is stuck at FF52 although OI has Firefox 60 ESR. Github and Youtube started to complain about unsupported browser :(

Peter Tribble said...

The current firefox situation is less than ideal. Unfortunately, newer versions of Firefox require rather large changes across the whole distribution, are difficult to test, and don't appear to work very well (if at all, half the time). Much as I too want a browser that I don't get a pile of warnings about, it's got to have to wait for a newer Tribblix release.