Running Tribblix on Digital Ocean

A relatively recent feature offered by Digital Ocean is the ability to deploy your own custom image. So, can I deploy a Tribblix image to Digital Ocean?

Short answer: Yes!

For the process, read on.

I'm using Bhyve to create the image. This is a slight variation on the Installing Tribblix in Bhyve on Tribblix procedure.

The basic process looks like this:

• Boot Tribblix in Bhyve
• Install it
• Tweak the installed image for Digital Ocea
• Copy the ZFS volume to Digital Ocean

The first variation on the previous install is that I make the ZFS volume a bit smaller. We can resize it when it's deployed, so we don't need to make it too big. A 4G volume is fine; any smaller and there won't be any space for our swap partition. It doesn't actually matter too much, as we compress the image anyway

zap create-zone -t bhyve -z bhyve1 \
-x 192.168.0.236  \
-I /var/tmp/tribblix-0m24.1.iso \
-V 4G

Install as before,

./live_install.sh -G c1t0d0

remove the cdrom and reboot as before.

We now need to set up networking. We need to do this on a temporary basis, as we don't want any of these network configurations to carry over to the installed image. So I temporarily disable nwam, and manually bring up a working network. When the image boots on Digital Ocean, all this configuration will have been forgotten and it will bring up nwam as normal.

So run the following in the newly installed guest:

svcadm disable -t network/physical:nwam
ifconfig vioif0 plumb
ifconfig vioif0 up
ifconfig vioif0 inet 192.168.0.236/24
route add net default 192.168.0.1
echo "nameserver    8.8.8.8"  > /etc/resolv.conf

Now we need to tweak the image. At some later point this will all be integrated into the installer so it will just work. But for now, we'll start by applying any updates:

zap refresh
zap update-overlay -a

Now for the little tweak. I'm going to add a metadata service that will run at boot on Digital Ocean and do the sort of things that cloud-init would do. Fortunately, there's one for illumos, and it's packaged for Tribblix, so install it:

zap install TRIBmetadata-agent

If you look with svcs you'll see that it's offline. That's not a problem (it's because we've got a temporary manual network setup) - once we boot properly on Digital Ocean we'll have nwam running and the metadata service will run just fine.

We can tidy up and save a bit of space:

zap clean-cache -a

and shut down the zone (and the newly installed instance of Tribblix):

zoneadm -z bhyve1 halt

What we want is a raw image. So all we do is dd the ZFS volume to a file.

dd if=/dev/zvol/rdsk/rpool/bhyve1_bhvol0 \
of=/var/tmp/tribblix-do-m24.1.img bs=1048576

That's a 4G file, the size of the volume. As it's stored on ZFS, and ZFS compression is on, it will actually consume a lot less space as the image is mostly empty. But what we don't want to do is upload 4G of empty space. So we can compress it:

gzip -9 /var/tmp/tribblix-do-m24.1.img

(it ends up as about 300M), or you could use bzip2, I think.

There are two options when you upload the image to Digital Ocean - you can either do a direct upload through the browser, or you can give it a URL where the image can be found and get Digital Ocean to pull it from there. I found it much easier to scp the image up to an existing webserver and get Digital Ocean to grab it, as I don't trust browsers to behave well.

Log in to Digital Ocean.

Select 'Images' from the left hand menu
Tab to 'Custom Images'
Import via URL
[Insert the URL where you've stashed the file]

You then get a dialog

Name - tribblix-do-m24.1.img.gz
Distribution - unknown (it's not on the list)
Region - London

Obviously, for me, London is conveniently local. You get warned there will be a charge, and it shows as pending for a few minutes.

Then it pops up 'your image is ready to use'.

To the right of the image in the list is a 'More' dropdown menu from which you can start a droplet. So off we go.

It selects a pretty hefty instance type by default. Reset that to the very cheapest. Choose the ssh key you're going to use, pick a useful (and shorter) hostname, and Create Droplet.

Don't bother with block storage. That's exposed by virtio-scsi, which illumos doesn't yet support.

It'll take a few moments to create the droplet, and once it's ready you'll see the IP address.

At this point, if everything has worked, you should be able to ssh in as root with the ssh key you chose. Chances are that doesn't quite work yet. If it doesn't, simply ssh in as jack, from where you can su to root.

(Remember, jack is on the live ISO, and we haven't deleted it. During experiments I tend not to, to give myself a way in if things don't work right. A proper production image would have the jack user removed and password login for root disabled.)

If the metadata service hasn't run properly (it will resize the ZFS pool, change the hostname, and add the correct key to ssh in as root) then you can restart the metadata service:

svcadm restart metadata

Now ssh to root works, the hostname is set, and the zfs pool has been expanded to the full size.

None of the above is excessively specific to Tribblix, the same general process will work for any of the illumos distributions. (Although you may have to build and install the metadata service yourself.)