Wednesday, November 25, 2009

Is package management interesting?

My desktop workstation running Solaris 10 has almost 1600 packages installed. Many of the development systems I use have something close to 1000 packages installed. My Ubuntu (8.04, to be exact) desktop install fresh off the CD has just over 1100.

Given 1600 packages, can a sysadmin manage that, or even concisely describe what is installed and what its function is? I suspect not. It's actually so bad that I suspect most of us don't even bother.

Tools don't really help here. If anything, by giving the illusion of ease of use, they encourage the growth in the number of packages, making the underlying problem worse.

Really, though, is managing packages interesting? I submit that it's not, and that looking at the problem as one of managing packages is completely the wrong question.

Instead, we should be managing applications at a higher level of abstraction. Rather than managing the 250 packages that comprise Gnome, we need to collapse that into a single item, which we can then open up into manageable chunks. A package may be the right level of granularity for a packaging system, but it's the wrong level of granularity for an administrator.

We should be thinking of applications and function, not the detail of how packages deliver that functionality. I want to be able to go to a system and ask "What do you do?" and have it come back and say "I'm a web proxy server and mail relay"; I don't want to sift through 500 packages and try to work out which of them are relevant. If I want to set up a wiki for collaborative document editing that authenticates against my Active Directory infrastructure, then I want to phrase the requirement in those terms rather than try to work out the list of components that I need for that task.

From this, the packaging details become uninteresting. What a package contains, the packaging software, package names - are less important because that's just internal implementation detail.

The old Solaris installer had some of this - it defined clusters and metaclusters. The implementation doesn't really help much - the definition of the contents of clusters and metaclusters was poor, and there was no support for the clusters once you were managing an already installed system. Also, what you really want is a system that allows for clusters to be structured hierarchically (so you could take something like Gnome, and either manage it as a single unit, or have the option of dealing with subunits like games or libraries), and to overlap (for example, you could imagine that the apache web server would be in a whole lot of clusters).

One might be tempted to construct packages that use dependency information to bring in the packages they need. This approach is flawed: it doesn't cleanly separate groups from packages; it doesn't allow you to omit subgroups; and it makes removal of a group excessively difficult. Software clusters need to be cleanly layered above packages in order to allow each layer to best meet its own requirements.

Beyond simply delivering files (via packages) a cluster could also contain the details of how to actually take the various bits and pieces and put them together into a functioning system. In fact, the whole area of application configuration is one desperately in need of more attention.

A quick summary, then: package management shouldn't be interesting, and we need to move forward to managing applications and configuration.


Mike Gerdts said...

We have similar ideas in this area. I blogged about it quite a while back under title Feature Based Meta-Packages. Follow-on work for that which addresses how do you use it? appears in my incomplete proposal for OpenSolaris Personality Profiles that I mentioned as the Use Case Project was being proposed.

Dave said...

Perhaps a better question really is:

Are we even really managing packages at all?

Do Sys Admins really understand what packages are installed and why they are installed?

Are excess packages that will never be used installed and just sitting there?

Livio Ribeiro said...

I've told something like this to Linux users some time ago and they said that package management is very easy for everyone, even to new users.

I wish that Solaris can focus on the user, not the system, like Linux

Anonymous said...

Tools absolutely can help.

This is exactly what I like about both netbsd pkgsrc and gentoo - I don't interact (in an administrative sense) with the vast majority of the pkgs on the system.

What I care about is the leaf pkgs - the ones that represent things I have actually asked to be installed. Other pkgs get pulled in as dependencies (and in some cases I might adjust pkg options before installation to avoid pulling in certain deps). However, pkgs that are pulled in as deps (or build deps) are recorded as such, and can then be treated differently thereafter.

Later, when I uninstall a pkg, or reinstall with different options, or a newer version has different dependencies, the tools in these systems allow me to identify pkgs that are now no longer required, and remove them.

No amount of collecting things into meta-pkgs and bundles will help, if I can't also remove the bundle.

It also helps to have a base system that is truly minimal, rather than a huge WoS, and pull in other things as dependencies require. Solaris probably has a bunch of missing dependencies amongst base pkgs because the pkgs are almost always deployed in a big bundle; this makes working with individual pkgs necessary when trying to minimise a system, because you can't use the tools effectively without good dependency info.

Erik said...

I think you're on the right track by focussing on the application. As it stands, the burden of packaging lies on distribution maintainers, who don't really care or know much about the application.

The distribution should be in the business of providing a basic operating environment that provides a platform for installing your applications. Applications need to be able to be installed and uninstalled atomically, including being delivered and supported by their authors.