Sunday, March 10, 2019

Tweaking the Tribblix installer

In the latest update to Tribblix, I've made a couple of minor tweaks to the installer.

The first is that compression (lz4) is now enabled on the root pool by default. It was possible in the past to give the -C flag to the installer, but now it's always on. There was a time in the past (the distant past) when enabling gzip compression could really hurt performance with some workloads, but the world has moved on, so enabling compression by default is a good thing.

The compression factor you see on something like the AWS image is:

NAME                 PROPERTY       VALUE  SOURCE
rpool                compressratio  1.92x  -
rpool/ROOT           compressratio  1.92x  -
rpool/ROOT/tribblix  compressratio  1.92x  -
rpool/export         compressratio  1.00x  -
rpool/export/home    compressratio  1.00x  -
rpool/swap           compressratio  2.04x  -

It's not bad, getting a factor of almost 2 essentially for free.

The second change is to partitioning. Traditionally, the root pool was created in a partition rather than the whole disk. You made the partition span the whole disk, but the disk was still partitioned in the traditional way.

As of m20.6, if you use -G instead of -B to the installer

./live_install.sh -G c1t0d0 [overlays...]

then it will create a whole disk pool with EFI System partition to support booting system with UEFI firmware. (As it says in the zpool man page.) Good for newer systems, but it brings another key capability.

For EC2 instances, this allows the rpool to be expanded. The base size is still 8G, but you can choose a different (larger) size for the EBS device when creating the instance:

When you initially log in, you still get the original size:

NAME    SIZE  ALLOC   FREE
rpool  7.50G   301M  7.21G

but you can use the following command:

zpool online -e rpool c2t0d0

and it magically expands to use the available space:

NAME    SIZE  ALLOC   FREE
rpool  11.5G   302M  11.2G

At the moment, this has to be done manually, but in future this expansion will happen automatically when the instance boots.

You can, of course, increase the size of an EBS volume while the instance is running. This takes a little while (the State is shown as "in-use - optimizing" while the expansion takes place). Unfortunately it appears at the moment that you need a reboot for the instance to rescan the devices so it knows there's more space.

7 comments:

Anonymous said...

Does Tribblix support booting on UEFI? It seemed Illumos in general still not support booting on UEFI. OpenIndiana and OmniOS installation iso images could boot but the installed system would not boot at all.

Peter Tribble said...

Tribblix has no UEFI support in the installer, unfortunately. I could attempt to copy changes from the OpenIndiana and OmniOS installers, but I'm not sure they're 100% finished, and I have no way to test it at the moment.

Anonymous said...

How to force ZFS to use 4k blocks on Tribblix? Does the installer script even has an option for it? On OmniOS we have an option to force at least 4k or 8k blocks. It seemed Tribblix always use ashift=9.

Anonymous said...

Could you improve the UFS installer script? I use Tribblix on VirtualBox. My goal is to create a desktop system so I don't care about ZFS or zones or bhyve. Using UFS allowed me to give less RAM for the VM. Since on most of situations it's we're installing to an empty disk or don't care if it being overwritten or not why don't you just clear the disk and partition everything automatically by default like live_install.sh does? If we want to over install or partition manually, we would do it using a seperate ufs_over_install.sh. Please don't let me to mess with the dreadful format program any more, this program is too outdated that it's even not allowing us to use any different units but cylinders!

Another thing I want to suggest is you should increase the default swap size for either ZFS or UFS installations. It should be twice as large as the machine's memory. I've experienced lag for unknown reasons on Illumos until I finally discovered even if I give the system 8G of ram it's still requires an 16G swap volume. I think it's hardcoded and won't change anytime soon.

Peter Tribble said...

Anyone can take the installer scripts and improve them

https://github.com/tribblix/tribblix-build

On zfs, we don't really do partitioning. we just take defaults and let zfs handle things. That's fairly easy. On UFS, that doesn't really work, as the default partition table is invariably useless. It wouldn't be excessively tricky to take format-a-disk.sh and create another swap partition.

You can adjust the swap size using the -s flag to the installer, for example -s 8G. That's for zfs, for ufs you choose the swap during that partition phase. The correct value is massively dependent on workload, the old 2x is mostly a myth - systems have more memory than in the past, and may (especially with SSDs) have smaller disks relative to other capabilities.

Anonymous said...

Thanks for the -s option, I didn't know about it. I've read the shell script files many times but didn't understand anything, as I'm very bad at shell programming. I'm not a sysadmin either, I'm rather a hobbyist, wanted to try new OSes other than Linux.

Please let me know how to install VirtualBox guest addition on Tribblix. I'm badly in need of it for full screen resolution. I know Illumos could serves as a host system for VirtualBox as well but I don't know why you didn't ship the VirtualBox overlay. OpenIndiana and OmniOS both have VirtualBox on the repos. I want to use Tribblix to host VirtualBox machine someday. I've tried such a setup before with OpenIndiana as the host for VirtualBox, everything was fine except Illumos' memory usage was very high compared to Linux, regardless of distributions, so I couldn't run as many VMs as on Linux.

Peter Tribble said...

As for ashift, you're right, there isn't an option to set that.

Having to do this in the installer is a bug, plain and simple. Quite a bit of work has been done to get ZFS to create pools with the correct settings automatically, and that's where fixes should be targeted.