Wednesday, April 12, 2017

Noisy Tribblix

I've had a couple of Tribblix users ask me why audio doesn't work.

This was something I had noticed myself, and the reason was not that audio was in some way broken, but that the permissions on the audio devices were wrong - owned and only writeable by root.

Now I only wanted to actually get any audio out on fairly rare occasions, so a quick chown wasn't that much of an imposition. But it obviously needed fixing properly.

My assumption here is that most desktop users will be logging in through the SLiM login manager. So all I need to do is fix the permissions just before it calls setuid() to the logged in user. And then reset them back once the user is done.

Now, I could have made up a bunch of chowns myself, or written a helper. There's actually code in SLiM to call ConsoleKit - but I don't have ConsoleKit, and don't really see the need to maintain a port of it just for this.

But illumos already has the capability to do this, and the normal login mechanisms use it. There's code in libdevinfo that sets the permissions according to the rules laid out in the /etc/logindevperm file. So the code is really just a call to di_devperm_login() and di_devperm_logout(), and all is well.

This also fixed another irritating bug - I can now eject memory sticks as myself, without needing to be root.

The next thing that happens, of course, is that it doesn't take very long to realise that Twitter has a lot of videos that play automatically. So I'm sitting there and I can hear either the internal loudspeaker or my headphones warbling away.

So the next thing I need is a way to shut the thing up. Historically, I used the old CDE sdtaudiocontrol, which was pretty good. (In general, I detested CDE as a desktop, the mailer and calendar were decent enough for their time, and the audio control was the only other thing I used much.) I use Xfce as my desktop, it used to have xfce4-mixer but that's now unmaintained and deprecated (and I removed that as part of the migration from gstreamer-0.10 to gstreamer1). Which pretty much leaves the command line audio utilities in illumos, specifically audioctl. I've added the package so users who update will automatically get that as well.

The command

audioctl set-control volume 0

silences things, while

audioctl set-control volume 75

puts the volume back to normal. I've created aliases mute and unmute for those. A more sophisticated approach would be to save the volume and restore it afterwards, but this is enough for now.

Sunday, March 12, 2017

How old are illumos man pages?

I've recently been looking at improving the state of the illumos man pages.

One thing you'll notice is that the date on some of the manpages is old - really old, some of them are dated 1990. That presumably means that they haven't been modified in any meaningful sense for a quarter of a century.

(By date here I mean the date displayed by the man command. Which isn't necessarily the date somebody last touched it, but should correspond to the last meaningful change.)

The distribution of dates looks like this:



As you can see, the dates go all the way back to 1990. There's not just the odd one or two either, there are a decent number of man pages that comes from the 1990s.

There are some obvious features in the chart above.

There's a noticeable spike in 1996, which of course significantly predates illumos or OpenSolaris. It's not entirely obvious why there should be a spike, but 100 of those man pages are related to libcurses.

I suspect the dip in 2005 is a result of the launch of Solaris 10, when everyone had a bit of a breather before development kicked into full gear again.

Then there's a drop in 2010. In fact, there's just a single page for 2010. That's when the OpenSolaris project closed, so there was little work done at that point. Also, our man pages were only integrated into illumos-gate in early 2011 (prior to which they were kept separate), and it's taken a while for man page updates to pick up again.

Of course, one reason the man pages are so old is that the software they're documenting is old. That's not necessarily a bad thing, if it ain't broke don't fix it as they say, but there is a certain amount of total rubbish that we ought to clear out.

Thursday, February 02, 2017

Creating a Tribblix package repository

I've previously described how Tribblix packages are built.

The output of that process should be a zap file, which is an SVR4 package in filesystem format, zipped up. The file naming convention is

PKG_NAME.VERSION.zap

Where PKG_NAME is the SVR4 name of the package (you can define aliases to be more user-friendly), VERSION is obvious - but has to match the installed version as shown, for example, by 'pkginfo -x'.

You can install those packages directly. There's a little helper /usr/lib/zap/instzap that will automate the process of unpacking the zap file and running pkgadd on it, or you can do it by hand.

You don't have to use my tooling. If you've got a scheme for building SVR4 packages already, then you could simply convert those.

However, what you would really want to do is stick the packages in a repository somewhere, so they can be accessed using the normal zap commands.

In Tribblix, a repository is just a web-accessible location that contains the zap files and minimal metadata. The metadata is the catalog and a list of aliases.

The aliases file contains lines with two fields separated by a vertical pipe "|". The first field is the friendly alias, the second the real name of the package. If you want multiple aliases, just add multiple lines to the file.

The catalog file contains lines with 5 fields separated by a vertical pipe "|".The first field is the package name, the second the current version, the third a space-separated list of packages this package depends on, the fourth the size of the file in bytes, the fifth is an MD5 checksum of the file. There's a trailing "|" in this case to terminate the line. The size and checksum are used to verify the download was successful and didn't corrupt the file. If you want to be sure that it's actually the package it claims to be then packages can also be signed.

Here's an example line in the catalog:
TRIBabook|0.5.6.0|TRIBreadline|66923|d98f77cfb3e92ee6495e3902cc46486f|
So the TRIBabook package has a current version of 0.5.6.0, depends on the TRIBreadline package, and has the given size and checksum. It's in the file called TRIBabook.0.5.6.0.zap.

The build repo has scripts that create the catalog and aliases files, which I use for convenience. They do make some assumptions about my package build pipeline, so it might be easier to manage the files by hand.

So, having got a nice place on the web that has your packages and a catalog and some aliases, how does Tribblix know to use it?

Assuming your repository is called myrepo, you need to add a file /etc/zap/repositories/myrepo.repo, containing something like the following (which is the configuration for the main tribblix repo in the milestone 18 release)

NAME=tribblix
DESC=Tribblix packages
URL=http://pkgs.tribblix.org/tribblix-m18/
SIGNED=tribblix

If you aren't signing packages (and how to add the keys to the client is an exercise for the reader) then omit the SIGNED line. The NAME in this case would be myrepo, the DESC is whatever you make it, and the URL points at the directory containing the files.

That's almost it. The other thing you need to do is add the repository to the list of repos that zap will search, which is held in the /etc/zap/repo.list file. By default, this contains

100 tribblix
200 illumos
300 oi

That's a simple list, and the number is a priority, lower numbers have higher priority - they're searched first.

(What the priority scheme means is that if you have a package with the same name in multiple repos, the one in the highest priority repo is the one that will be used. For example, the ssh packages used to come from illumos. As we've moved to openssh, all I had to do was put the new openssh packages into the tribblix repo with the same package names they had before and they were installed instead. Of course, you have to be careful that the replacement package delivers the correct functionality, especially if it's delivering shared libraries.)

If your package names are unique to you (for instance, if you name them MYPKGfoo rather than TRIBfoo) then the priority doesn't matter. If you're deliberately trying to override some of the built-in packages, then your repo has to be the highest priority one so it gets searched first.

If you then utter 'zap refresh' it should go and retrieve the catalog and aliases files and then be set up to use them.

And yes, it would be nice if zap had the facility to manage repos for you - that's planned, I just need to implement it.

Monday, January 16, 2017

Package versions in Tribblix

All packages in Tribblix are versioned. If you look at the pages on the package repository you can see the current version of each package in the repo. On an installed system the pkginfo -x command will give you the package description and version.

As Tribblix is created from different sources, the meaning of the package version can vary.

For illumos packages, the version string matches the Tribblix release. For example, "0.18.0" indicate the Milestone 18 (0m18) release.

For packages inherited from another distro, the version matches in some way the distro release I got the packages from. For example, the OpenIndiana packages were (at this time) cut from the oi151a9 release, and have a version string "0.9o".

For packages I build directly from source, the version string is usually the upstream version, with a build number appended. Initially the build number is 0, then increments. If the upstream version is updated, the build number goes back to zero. So it's reasonably obvious what version of a package is installed.

For example, abiword is version 2.8.6 so the first time it was built the package version was 2.8.6.0. Over time the package has needed to be rebuilt, so it's now up to version 2.8.6.4.

The sharp-eyed will notice that the illumos packages have a build number in them. This hasn't yet been used, it's there just in case.

The scheme is reasonably flexible. For example, OpenSSL has letters in its releases - like 1.0.2j - which I could keep verbatim, but in practice I convert the letter to a numeric sub-version, hence 1.0.2.10.

There are some packages for which I originally forgot to add the build number. That OpenSSL package is an example, but there are others. I've tended not to correct those as it disturbs the flow, I will if it ever becomes convenient.

Some releases have a date, this is just converted to numerical form.

One thing that should be obvious is that the scheme doesn't guarantee that package versions are numeric. They're just strings; it just happens that most packages have version numbers that are numeric or can easily be represented as such.

Also, package versions don't necessarily increase, there is no sense of ordering built into versioning. For example (this does happen) there's an upstream version 1.2, which leads to package versions 1.2.0, 1.2.1, 1.2.2, etc. Then there's an upstream 1.2.1, which is packaged as version 1.2.1.0, which is lower than 1.2.2. And sometimes upstreams try a major version bump, then backtrack.

However, package management in Tribblix ascribes no meaning to the version numbers. It's only test for currency is this - does the version installed match the version in the repository catalog? If they're the same, then you're up to date. If not, then apply the version from the repo.

This then makes it easy to roll back errant packages. All I have to do is put the old version back in the catalog. Anyone who has applied the broken version will get a version mismatch and the older version will get installed whenever they update.

(This simplistic approach only works if I haven't built anything against the newer version of the package I want to roll back. But then, all I have to do is roll all those dependent packages back as well.)

Life's a little more complicated if you might want multiple versions of an application installed. In that case you have to have different packages. For example, I have separate packages for Python 2.7 and 3.6, and there might be 2 corresponding packages for any modules. I used to use multiple packages more extensively, sometimes even for minor version updates, but tend to avoid that now when I can.