Tuesday, January 18, 2022

Inside zone installation

 How do zones actually get put together on Solaris and illumos? Specifically, how does a zone get installed?

 There are various type of zones. The nomenclature here is a brand. A zone's brand defines how it gets installed and managed and its properties. Often, this is mapped to a zone template which is the default configuration for a zone of that type or brand.

 (By the way, this overlap between template and brand can be seen in the create subcommand of zonecfg. You do "create -t SUNWlx" to build a zone from a template, which is where the -t comes from. It's not the create that sets the brand, it's the template.)

 The templates are stored as xml files in /etc/zones. As are the configured zones, which is a bit confusing. So in theory, if you wanted to generate a custom template to save adding so much to your zonecfg each time, you could add your own enhanced template here. The actual zone list is in /etc/zones/index.

 In fact, Tribblix has template zones, which are sparse-root zones built from a different image to the global zone. They are implemented by building an OS image that provides the file systems to be mounted read only, and a template xml file configured appropriately.

 One of the things in the template is the brand. That maps to a directory under /usr/lib/brand. So, for example, the TRIBsparse template in /etc/zones/TRIBsparse.xml sets the brand to be sparse-root, in addition to having the normal lofs mounts for /usr, /lib, and /sbin that you expect for a sparse-root zone. There's a directory /usr/lib/brand/sparse-root that contains everything necessary to manage a sparse-root zone.

 In there you'll find a couple more xml files - platform.xml and config.xml. A lot of what's in those is internal to zones. Of the two, config.xml is the more interesting here, because it has entries that match the zoneadm subcommands. And one of those is the install entry. For TRIBsparse, it is

 /usr/lib/brand/sparse-root/pkgcreatezone -z %z -R %R

 When you invoke zoneadm install, this script gets run, and you get the zone name (-z) and zonepath (-R) passed in automatically. There's not much else that you can specify for a sparse root zone. If you look at the installopts property in config.xml, there's just an h, which means that the user can specify -h (and will get the help).

 For a whole-root zone the install entry is similar, but installopts is now o:O:h - this is like getopts, so it's saying that you can pass the -o and -O flags, and that each must have an argument. These flags are used to define what overlays get installed in a whole-root zone. Having the installopts defined here means that zoneadm can validate the install command.

 So, for a given brand, we've now seen from config.xml what command will be called when you install a zone, and what options it's allowed.

 The point is that there's nothing special here. You can build a custom brand by writing your own install script, and if you need to pass arguments to it you can easily do so as long as you set installopts to match. When building all the zone brands for Tribblix, that's all I did.

 To reiterate, the install script is completely open. For existing ones, you can see exactly what it's going to do. If you want to create one, you can have it do anything you like in order to lay down the files you want in the layout you want.

 As a crazy example, a long time ago I created a brand that built a sparse-root zone on a system using IPS packaging.

 There's a little bit of boilerplate (if you're going to create your own brands, it's probably easier to start with a copy of an existing one so you pick up the common actions that all zone installs do), but after that, the world's your oyster.

 Consider the alien-root zone in Tribblix. If you look at the installer for that, it's just dumping the contents of an iso image, tarball, or zfs send stream into the zone root. It does some cleanup afterwards, but generally it doesn't care what's in the files you give it - you can create an arbitrary software installation, tar it up, and install a zone from it.

 (In fact, I probably won't create more native zone types for Tribblix - the alien-root is sufficiently generic that I would extend that.)

 This generality in scripting goes beyond the install. For example, the prestate and poststate scripts are called before or after the zone transitions from one state to another, and you can therefore get your zone brand to do interesting things triggered by a zone transitioning state. One of the coolest uses here is the way that OmniOS implements on-demand vnics - the prestate script creates a vnic for a zone before a zone boots, and the poststate script tears it down after it halts. (Tribblix uses zap to manage vnics outside of zoneadm, so they're persistent rather than on-demand, it's just a different way of doing things.)

 As you can see, you aren't limited to the zone types supplied by your distribution. With enough imagination, you can extend zones in arbitrary ways.

No comments: