Installing OmniOS on Vultr with iPXE

A while ago, I wrote about booting Tribblix on Vultr using iPXE.

Naturally, the question arises - is this possible for other illumos distributions, like OmniOS?

First, though, a note about why this might be interesting. Imagine I have some servers in a remote datacenter, and I need to install an OS. Most servers have some sort of remote managamenet capbility that includes mounting an ISO image. That's fine, but if you're at home then you're having to transfer the ISO image the wrong way across an asymmetric network connection. It's very slow, quite unreliable, and sometimes plain doesn't work. On the other hand, booting iPXE is very quick and very reliable, because the bootable image is so small. And the same benefits mean you can repeat the process until you've got it right.

Based on that, Vultr is just a very convenient way to test this out.

Because of the way that the OmniOS media are built, a straight copy of the Tribblix procedure won't work. This is what you need to do instead.

First, you need an accessible web server. I'm using the Tribblix repo server, but it really can be any server. It doesn't need to be illumos, as long as it responds to http requests you're good.

In the document root, create a directory. I'm going to call it 'kayak', because that's what the OmniOS network installer is called, and that's what we're going to leverage.

There are 3 files from the OmniOS distribution or media that we need.

You can't use the boot archive from the ISO. You need a custom boot archive for this. Download the miniroot file from the https://downloads.omniosce.org/media/ location. If you look in each release directory, you should find a gzipped miniroot file. Put that in the kayak directory, and gunzip it. I normally rename this to be platform/i86pc/amd64/boot_archive. This is going to be the initrd file in your ipxe configuration.

The next thing you need is the corresponding kernel, aka unix. You need to extract this either from the ISO image or the miniroot. The ISO image might be easier; the miniroot contains a UFS filesystem. You could mount this up using lofi and copy the file out of it. You need the file /platform/i86pc/kernel/amd64/unix, and it needs to be located at platform/i86pc/kernel/amd64/unix.

Another way if you have the iso-read command installed (on Tribblix it's in the TRIBlibcdio package) is

mkdir -p platform/i86pc/kernel/amd64
iso-read -i omniosce-r151030.iso \
  -e platform/i86pc/kernel/amd64/unix \
  -o platform/i86pc/kernel/amd64/unix

The third thing you need is the image to install. This is a compressed zfs send stream. You can download these from https://downloads.omniosce.org/media/ for most releases - look for the .zfs.xz file. Make sure not to use the ones with ngz in the name - they're for zones.

If you're trying r151030, then there isn't a downloadable file, but you can pull this file out of the ISO too. For example

iso-read -i omniosce-r151030.iso \
  -e kayak/kayak_r151030.zfs.xz \
  -o kayak_r151030.zfs.xz

Then you need to create a client configuration script. The instructions at https://omniosce.org/setup/pxe.html are a good start. For example, it might be:

BuildRpool c1t0d0
RootPW '$5$JQkyMDvv$pPzEUsvP/rLwURyrpwz5i1SfVqx2QiEoIdDA9ZrG271'
SetRootPW
SetHostname omnios30
SetTimezone UTC
Postboot '/sbin/ipadm create-if vioif0'
Postboot '/sbin/ipadm create-addr -T dhcp vioif0/v4'

Obviously, you need to use the correct disk device (c1t0d0) and network interface (vioif0) appropriate to your system. These are the right values for Vultr.

The installer will look for this file using its (uppercased) MAC address, and then go for shorter parts of the MAC address. As I don't know what that is yet, I simply create a file and create symlinks 0-9 and A-F pointing to it, as it will eventuially get to the first letter of the MAC address and one of them will match. If you know the actual MAC address of the system, you can create a file with the exact match.

What you end up with is an ipxe.txt (and this is the one I'm using at http://pkgs.tribblix.org/kayak/ipxe.txt) file that looks like this:

#!ipxe
dhcp
kernel /kayak/platform/i86pc/kernel/amd64/unix -B install_media=http://18.130.117.123/kayak/kayak_r151030.zfs.xz,install_config=http://18.130.117.123/kayak
initrd /kayak/platform/i86pc/amd64/boot_archive
boot

To recap:

• The initrd file points at the location of the gunzipped miniroot file you downloaded.
• The kernel line refers to the unix file you extracted.
• The kernel line supplies boot arguments with the -B flag. There are two of them.
• NOTE: the kernel line is just one line, despite how its gets formatted here.
  
• The install_media argument is a URL to the ZFS send stream file you downloaded. Note that I'm using the IP address of the server rather than its DNS name, as you can't guarantee that DNS resolution will work.
• The install_config argument is the URL of the directory that the client configuration scripts are put in. The installer will look for files based on (a substring of) the client's MAC address in this directory

With that all set, you're ready to boot a machine on Vultr.

Go to instances, and click on 'Deploy instance'.

Choose a server. Cloud Compute is the basic starter, I normally use London as it's close to me to minimize latency.

For Server Type, go to the Upload ISO tab, choose iPXE isntaed of My
ISOs, and enter the URL of your ipxe file, ipxe.txt

 

Then select your server size - even the smallest 1024MB instance will work fine, and hit the Deploy Now button.

It should show as Installing for a few moments, and then change to Running. If you click on Running you get to the page for that instance, and the view the console link does exactly that using VNC.

The install is pretty quick:

 

(You can see it trying the variant client configuration scripts.)

And it will then reboot into the installed system.

This is a typical minimalist OmniOS install. You'll probably have to set DNS up (if you didn't do it in your client confiiguration script), add a user, and then you'll be able to ssh in remotely.