Simple Zone Architecture

I use Solaris zones extensively - the assumption is that everything a user or application sees is a zone, everything is run in zones by default.

(System-level infrastructure doesn't, but that's basically NFS and nameservers. Everything else, just build another zone.)

After a lot of experience building and deploying zones, I've settled on what is basically a standard build. For new builds, that is; legacy replacement is a whole different ballgame.

First, start with a sparse-root zone. Apart from being efficient, this makes the OS read-only. Which basically means that there's no mystery meat, the zone is guaranteed to tbe the same as the host, and all zones are identical. Users in the zone can't change the system at all; which means that the OS administrator (me) can reliably assume that the OS is disposable.

Second, define one place for applications to be. It doesn't really matter what that is. Not being likely to conflict with anything else out there is good. Something in /opt is probably a good idea. We used to have this vary, so that different types of application used different names. But now we insist on /opt/company_name and every system looks the same. (That's the theory - some applications get really fussy and insist on being installed in one specific place, but that's actually fairly rare.)

This one location is a separate zfs filesystem loopback mounted from the global zone. Note that it's just mounted, not delegated - all storage management is done in the global zone.

Then, install everything you need in that one place. And we manage our own stack so that we don't have unnecessary dependencies on what comes with the OS, making the OS installation even more disposable.

We actually have a standard layout we use: install the components at the top-level, such as /opt/company_name/apache, which is root-owned and read-only, and then use /opt/company_name/project_name/apache as the server root. Similar trick works for most applications; languages and interpreters go at the top-level and users can't write to them. This is yet another layer of separation, allowing me to upgrade or replace an application or interpreter safely (and roll it back safely as well).

This means that if we want to back up a system, all we need is /opt/company_name and /var/svc/manifest/site to pick up the SMF manifests (and the SSH keys in /etc/ssh if we want to capture the system identity). That's back up. Restore is just unpacking the archive thus created, including the ssh keys; cloning a system you just unpack a backup of the system you want to reproduce. I have a handful of base backups so I can create a server of a given type from a bare zone in a matter of seconds.

(Because the golden location is its own zfs filesystem, you can use zfs send and receive to do the copy. For normal applications it's probably not worth it; for databases it's pretty valuable. A limitation here is that you can't go to an older zfs version.)

It's so simple there's just a couple of scripts - one to build a zone from a template, another to install the application stack (or restore a backup) that you want, with no need for any fancy automation.