How I build the Tribblix AMIs

I run Tribblix on AWS, and make some AMIs available. They're only available in London (eu-west-2) by default, because that's the only place where I use them, and it costs money to have them available in other regions. If you want to run them elsewhere, you can copy the AMI.

It's not actually that difficult to create the AMIs, once you've got the hang of it. Certainly some of the instructions you might find can seem a little daunting. So here's how I do it. Some of the details here are very specific to my own workflow, but the overall principles are fairly generic. The same method would work for any of the illumos distributions, and you could customize the install however you wish.

The procedure below assumes you're running Tribblix m29 and have bhyve installed.

The general process is to boot and install an instance into bhyve, then boot that and clean it up, save that disk as an image, upload to S3, and register an AMI from that image.

You need to use the minimal ISO (I actually use a custom, even more minimal ISO, but that's just a convenience for myself). Just launch that as root:

zap create-zone -t bhyve -z bhyve1 \
-x 192.168.0.236  \
-I /var/tmp/tribblix-0m29-minimal.iso \
-V 8G

Note that this creates an 8G zvol, which is the starting size of the AMI.

Then run socat as root to give you a VNC socket to talk to

socat TCP-LISTEN:5905,reuseaddr,fork UNIX-CONNECT:/export/zones/bhyve1/root/tmp/vm.vnc

and as yourself, run the vnc viewer

vncviewer :5

Once it's finished booting, log in as root and install with the ec2-baseline overlay which is what makes sure it's got the pieces necessary to work on EC2.

./live_install.sh -G c1t0d0 ec2-baseline

Back as root on the host, ^C to get out of socat, remove the ISO image and reboot, so it will boot from the newly installed image.

zap remove-cd -z bhyve1 -r

Restart socat and vncviewer, and log in to the guest again.

What I then do is to remove any configuration or other data from the guest that we don't want in the final system. (This is similar to the old sys-unconfig that many of us used to Solaris will be familiar with.)

zap unconfigure -a

I usually also ensure that a functional resolv.conf exists, just in case dhcp doesn't create it correctly.

echo "nameserver    8.8.8.8" > /etc/resolv.conf

Back on the host, shut the instance down by shutting down the bhyve zoned it's running in:

zoneadm -z bhyve1 halt

Now the zfs volume you created contains a suitable image. All you have to do is get it to AWS. First copy the image into a plain file:

dd if=/dev/zvol/rdsk/rpool/bhyve1_bhvol0 of=/var/tmp/tribblix-m29.img bs=1048576

At this point you don't need the zone any more so you can get rid of it:

zap destroy-zone -z bhyve1

The raw image isn't in a form you can use, and needs converting. There's a useful tool - the VMDK stream converter (there's also a download here) - just untar it and run it on the image:

python2 ./VMDK-stream-converter-0.2/VMDKstream.py /var/tmp/tribblix-m29.img /var/tmp/tribblix-m29.vmdk

Now copy that vmdk file (and it's also a lot smaller than the raw img file) up to S3, in the following you need to adjust the bucket name from mybucket to something of yours:

aws s3 cp --cli-connect-timeout 0 --cli-read-timeout 0 \
/var/tmp/tribblix-m29.vmdk s3://mybucket/tribblix-m29.vmdk

Now you can import that image into a snapshot:

aws ec2 import-snapshot --description "Tribblix m29" \
--disk-container file://m29-import.json

where the file m29-import.json looks like this:

{
    "Description": "Tribblix m29 VMDK",
    "Format": "vmdk",
    "UserBucket": {
        "S3Bucket": "mybucket",
        "S3Key": "tribblix-m29.vmdk"
    }
}

The command will give you a snapshot id, that looks like import-snap-081c7e42756d7456b, which you can follow the progress of with

aws ec2 describe-import-snapshot-tasks --import-task-ids import-snap-081c7e42756d7456b

When that's finished it will give you the snapshot id itself, such as snap-0e0a87acc60de5394. From that you can register an AMI, with

aws ec2 register-image --cli-input-json file://m29-ami.json

where the m29-ami.json file looks like:

{
    "Architecture": "x86_64",
    "Description": "Tribblix, the retro illumos distribution, version m29",
    "EnaSupport": false,
    "Name": "Tribblix-m29",
    "RootDeviceName": "/dev/xvda",
    "BlockDeviceMappings": [
        {
            "DeviceName": "/dev/xvda",
            "Ebs": {
                "SnapshotId": "snap-0e0a87acc60de5394"
            }
        }
    ],
    "VirtualizationType": "hvm",
    "BootMode": "legacy-bios"
}

If you want to create a Nitro-enabled AMI, change "EnaSupport" from "false" to "true", and "BootMode" from "legacy-bios" to "uefi".