Creating illumos packages for Tribblix

In the prior article in this series, I discussed how to build illumos-gate (and it applies to illumos-omnios too), using a Tribblix AMI on AWS.

After that, you'll have two directories of interest under illumos-gate.

1. The proto area, specifically proto/root_i386, that is a fully installed copy of illumos.
2. Under packages, an on-disk IPS package respoitory.

If you're using IPS, you could use that repository as is. Tribblix, however, uses SVR4 packages. So the next step is to convert the contents of the IPS repository into a set of SVR4 packages.

For this, there is an additional github repo you'll need to check out, in addition to the ones from the previous article.

cd ${HOME}/Tribblix
git clone https://github.com/tribblix/tribblix-transforms

What is this repo? It's a list of transformations that are applied to the package conversion process. Generally, rather than modify the gate or the build, if there's something I don't want to ship, or something I want to move between packages, or a file I want to change, then it gets transformed at the packaging stage.

The other thing you'll need to use my scripts is a signing certificate. In Tribblix, the illumos ELF objects are digitally signed (yes, I ought to extend this to all ELF objects I ship). Nothing actually uses this yet, but it's all there ready for when verification is required.

To create a signing key and certificate:

mkdir ${HOME}/f
cd ${HOME}/f
openssl req -x509 -newkey rsa:2048 -nodes \
  -subj "/O=MyOrganization/CN=my.domain.name" \
  -keyout elfcert.key -out elfcert.crt -days 3650

Of course, choose the subject to match your own requirements.

Then you can run the scripts in the tribblix-build repo to generate SVR4 packages. These parse the IPS manifests, extract the files from the IPS repo, create the SVR4 prototype file, and automatically generate SVR4 install scripts from the IPS metadata.



So, to create packages, run the following as root:

/illumos/Tribblix/tribblix-build/repo_all.sh \
  -T /illumos/Tribblix \
  -G /illumos/Illumos/illumos-gate \
  -S /illumos/f/elfcert

Of course, replace /illumos with wherever your build user's home directory is. (You can't use $HOME, because this is run as root where $HOME is likely to be different to that of the user who did the build.) The -T flag tells it where the Tribblix tools are checked out, -G where the gate was built, and -S where the signing certificate is. If you want to change the version of the packages you generate, there's a -V flag as well. I normally redirect the output to a log file, as it's quite verbose

Wait a while, and you'll have a set of packages under /var/tmp/illumos-pkgs. Hopefully the pkgs directory there will have your packages, and the build and tmp directories will be empty (if the build directory isn't, it will have a half-created package or packages in it, so you'll be able to see which package or packages failed).

For omnitribblix it's very similar:

/illumos/Tribblix/tribblix-build/omni_all.sh \
  -T /illumos/Tribblix \
  -G /illumos/Illumos/omnitribblix \
  -S /illumos/f/elfcert

And the packages in this case end up in /var/tmp/omni-pkgs.

If you look, you'll see that there are 3 files for each package:

• A .pkg file, which is in SVR4 datastream format
• A .zap file, which is the (compressed) zap format Tribblix uses
• An md5 checksum, which is used for basic validation in the package catalog

I then use those generated SVR4 packages to populate the package repository, and to build the Tribblix ISOs. The details of that will have to wait for another time.

Tweaking the Tribblix installer

In the latest update to Tribblix, I've made a couple of minor tweaks to the installer.

The first is that compression (lz4) is now enabled on the root pool by default. It was possible in the past to give the -C flag to the installer, but now it's always on. There was a time in the past (the distant past) when enabling gzip compression could really hurt performance with some workloads, but the world has moved on, so enabling compression by default is a good thing.

The compression factor you see on something like the AWS image is:

NAME                 PROPERTY       VALUE  SOURCE
rpool                compressratio  1.92x  -
rpool/ROOT           compressratio  1.92x  -
rpool/ROOT/tribblix  compressratio  1.92x  -
rpool/export         compressratio  1.00x  -
rpool/export/home    compressratio  1.00x  -
rpool/swap           compressratio  2.04x  -

It's not bad, getting a factor of almost 2 essentially for free.

The second change is to partitioning. Traditionally, the root pool was created in a partition rather than the whole disk. You made the partition span the whole disk, but the disk was still partitioned in the traditional way.

As of m20.6, if you use -G instead of -B to the installer

./live_install.sh -G c1t0d0 [overlays...]

then it will create a whole disk pool with EFI System partition to support booting system with UEFI firmware. (As it says in the zpool man page.) Good for newer systems, but it brings another key capability.

For EC2 instances, this allows the rpool to be expanded. The base size is still 8G, but you can choose a different (larger) size for the EBS device when creating the instance:

When you initially log in, you still get the original size:

NAME    SIZE  ALLOC   FREE
rpool  7.50G   301M  7.21G

but you can use the following command:

zpool online -e rpool c2t0d0

and it magically expands to use the available space:

NAME    SIZE  ALLOC   FREE
rpool  11.5G   302M  11.2G

At the moment, this has to be done manually, but in future this expansion will happen automatically when the instance boots.

You can, of course, increase the size of an EBS volume while the instance is running. This takes a little while (the State is shown as "in-use - optimizing" while the expansion takes place). Unfortunately it appears at the moment that you need a reboot for the instance to rescan the devices so it knows there's more space.