Monday, October 31, 2005

Invalid system disk

I was updating Solaris on my W2100z the other night. After applying the latest recommended patches, I rebooted and went off to get a drink.

On my return I was rather startled to find displayed the old message "Invalid system disk. Press any key to continue." Oh bother - what's gone wrong?

It would be a bit of a pain, because I do quite a bit of development on this machine and I really don't want to lose it. The usual suspect in these cases would be a floppy disk, but the W2100z doesn't have one, so I was worried for a second that the main disk had been trashed.

But then I worked out what had happened. I had taken the recommended patch bundle home on a zip disk, and left it connected. So the invalid disk it was complaining about was in fact the one I had left in the USB zip drive. (So clearly the W2100z is capable of booting off a USB device. Scary.)

Panic over...

Thursday, October 20, 2005

Putting Google to work

I don't know how many others use Google as a problem solving tool, but I use it all the time.

I had a problem yesterday, jumpstarting Solaris 10 onto a workstation. Couldn't find the jumpstart directory. So plan A is to drop the error into Google and see what we get.

Bingo! The right answer comes straight back. Yup, simple netmask mismatch.

It's not just that, though. I was intrigued to see that Sun use google for making Solaris development decisions.

Heck, what would we do without Google?

Sunday, October 16, 2005

system() doesn't constitute an API

I like APIs. I like being able to call a function, give it some data, and tell it what to do with it, and have it do it without fuss or bother.

And I like interfaces that are stable and reliable.

However, on a typical unix-like system, certain common operations have no real API. For some operations, the normal way to do things is to run an external program.

Usually the system() call of the title is involved, but it could be popen() or some other variant. I'm using system() here as shorthand terminology for executing another program to do the work.

The two standard examples are mail and printing. You throw the data you want something done with at sendmail or lp.

This is atrocious. This isn't an API - this is merely convention that often works. Frankly, it worries me that there aren't standard programming APIs for such common tasks as sending mail and printing a file.

While mail appears to be standardized sufficiently to be largely hidden by applications, printing is another matter. The variety (and general disfunctionality) of print dialog boxes should be a clue here. Worst, of course, are those that simply have a text field that you type the print command into.

Apart from the inefficiency of launching external programs, the lack of genuine APIs limits the interactions a program can have, in particular the feedback it can get from the application it launched.

It's the 21st century. We deserve to have some decent 21st century APIs.

Friday, October 14, 2005

When did that get added?

I haven't used format to partition a disk by hand under Solaris for ages. Normally I set everything up at install time using jumpstart, and then prtvtoc and fmthard if I need to copy a partition table (if a disk fails and needs to be replaced, for example). So I happened to be working on a machine today, and was presented with:


partition> 4
Part Tag Flag Cylinders Size Blocks
4 unassigned wm 0 0 (0/0/0) 0

Enter partition id tag[unassigned]:
Enter partition permission flags[wm]:
Enter new starting cyl[0]: 14521
Enter partition size[0b, 0c, 14521e, 0.00mb, 0.00gb]:


Say what? What's this e thing? Well, it's what you expect - the end cylinder. Aha! My, this makes putting that last partition in so much easier. No more trying to calculate in your head how many cylinders are left...

(Looking at a couple of other machines, I think this got added in Solaris 9.)

Why don't people upgrade?

Gary mentions a topic I've always found interesting: why do people persist in running old versions of Solaris?

I know of many reasons, some of them are even valid. Most of these are in the general area of supported configurations. If a vendor (or even Sun) won't support a new version of Solaris, then you're pretty well out of luck. It's not just official support, either - sometimes the product plain won't work.

(This last point makes me wonder. Given the very strong binary compatibility guarantees in Solaris, what are some of these vendors doing to make their code stop working? Presumably they think they're being clever, but they must be putting in a lot of effort to break things.)

There are also a couple of burying head in the sand excuses I hear
  • nobody else has upgraded so there must be a problem

or
  • we'll wait for someone else to try it and find all the bug

both of which are bogus. Let's nail the first one for starters. Actually, plenty of people do upgrade, and don't have problems (if they read the instructions, that is). Do you want to be the one left behind? This herd instinct does seem very strong, though.

The "waiting for someone else" mentality is wrong. For one thing, Solaris doesn't go round randomly breaking functionality - compatibility is very strong and the level of bugs is very low. Secondly, the Solaris development model fixes bugs in the new version first and then fixes the older versions later (if at all), so that upgrading to a new version is the best way to reduce the number of bugs that you might be exposed to. Thirdly, it's actually already been tested (to destruction and beyond) very very hard indeed already. I know, I did it. Yes, we found bugs. But a whole lot of people inside and outside of Sun have beaten on Solaris for a long time, and most of the bugs are gone. Finally, because it's been well tested, the remaining bugs tend to be in strange distant areas that nobody has covered yet. In other words, the chances are pretty good that if you're going to get affected by a bug, it may well be specific to you, your environment, and the way you use the product - and these bugs aren't going to be picked up by other people, so it's daft to wait for other people to do the testing because it doesn't mean anything for you.

There's also the argument "it's not broken, so there's no point fixing it". Sure, it is reasonable not to immediately go out and heedlessly upgrade every time a new version comes out. And once something works leaving it alone is generally a good policy. (But I think this really ought to be expressed as "don't fiddle" rather than "never touch".) But regular maintenance is an essential part of the process, and upgrades and replacements need to be planned for. The problem with just leaving things alone is that they start to rot, and if you leave them too long you have a disaster on your hands. For example, if you don't apply any patches for 4 years, and then hit a problem, you have to apply 4 years of patches - a massive change, and a massive risk. Or if you insist on running old hardware too long, you suddenly discover that when you need to replace it you have to change the OS and application by 3 or 4 major versions and it becomes a nightmare. Essentially, planned steady change is a lot better than hanging on too long and hoping you can survive the inevitable wreck. Better many small steps than a desperate leap over a gaping chasm.

I haven't even considered all the cool new features.

So I have this message to all those still hesitating about upgrading to Solaris 10:

Come on in - the water's fine!

Friday, September 30, 2005

Yay! Gnome terminal fixed!

Every so often, there's a Solaris patch that fixes a problem that's bugged me for ages.

Today's example is a big win. After applying patches 120461 and 120289 (x86), or 120460 and 120288 (sparc), the JDS gnome terminal that comes with Solaris 10 no longer needs you to press the stupid shift key to get PageUp, PageDown, Home, and End working.

This is one of those little useability things that makes a huge difference.

To whoever fixed this - Thank You!.

Thursday, September 29, 2005

Being Enterprising

I've been playing around with various bits of the Java Enterprise System.

It comes on a neat DVD in the Solaris 10 media kit. So I had a play with that at home, only to discover that it's missing one thing. Instructions. There is no documentation included. Zilch. Nothing.

The basic instructions can be found on the web, but that's not the point. It's a fairly small pdf file - about 4 megabytes, and there's plenty of space on the DVD to fit that in all the supported languages. It would make it so much easier for people to test.

Another useful document I came across is a useful guide to evaluating JES on a single system. (This would also be absolutely easy to simply copy onto the DVD.) This covers the basic steps you need to get started.

Thursday, September 15, 2005

Google Earth

Surfing away last night, I came across Google Earth.

Wow!

This is a stunner. I haven't shown it to the kids yet, but one of them is doing geography homework at the moment and would find it amazing. (One reason for not showing it to them is so that I can get access to the machine.)

Tuesday, September 13, 2005

New Sun Boxes

So now we know what the new Sun servers are like.

And I'm pleasantly surprised. There are a couple of things that I feel are positive here:
  • The entry level box, the X2100. This makes a nice addition to the range. Yes. it's dumbed down, but you don't always need all the features - sometimes you just want a server.
  • I'm pleased to see the two configurations of the X4100. You can get this in either a 2-disk (plus DVD) or 4-disk variant. I like the 4-disk variant. Often, a local CD is just a waste of space and money, and I like having decnt local storage (read more than 2 disks - so you don't have to worry about quorum in a 2-disk configuration, and you can have some space to put data for a data base or some such locally). It's amazed me that the V210 (and the V1280, even) only allow 2 internal drives.

OK, so the range is still a little thin, but that's effectively twice as many variants as I was expecting, and I like the look of all of them.

Monday, September 05, 2005

All Change

Today's been a pretty big change for the Tribble household.

I started a New Job, while Amanda started a new school. Not just that, but this involves completely different schedules for getting up in the morning, getting ready, and travelling.

I can't tell you much about the new job, because I've largely been drowned in paperwork. I did manage to unpack and install my new workstation (wow!), but that's about all.

Friday, September 02, 2005

Java man pages

As the holidays draw to a close, I've been trying out a few more ideas.

Solaris comes with man pages in nroff or sgml format, that can be preformatted and indexed with catman. (Why the preformatted man pages and index aren't created by default is beyond me.) But there are still a couple of weaknesses with the man system
  • There's no search system
  • There's no way to follow cross-references
The Documentation DVD includes html formatted man pages which have the linking element, which is an improvement. But this doesn't solve the search problem.

However, Solaris comes with Javahelp. So I had a little play today with creating a Javahelp helpset out of the html manpages.

This worked reasonably well, up to a point. It was pretty easy to knock up the helpset file, and the map, TOC, and index files, and to create a full-text database for searching.

The Javahelp hsviewer did a reasonable job of providing a user interface. You can navigate, and the search isn't too bad.

Where it all fell down was that the html display in the viewer uses the swing JEditorPane. Which is - shall we say - a bit primitive. So while the overall solution functions reasonably well, the actual display looks pretty awful. Sufficiently bad, in fact, that it's not going to work in practice.

So the idea that I could produce something from a quick hack came to nothing.

I'm not done yet, though. As I see it, there are 3 ways to go forward:
  • Recreate the html versions of the man pages with an old html syntax that the javahelp display engine is going to cope with.
  • Use a different html renderer component.
  • Integrate this into a web server that does the searching so you do the search and indexing in a servlet but just shove the html pages out to a browser.

At the moment, the first looks the most attractive, as this also solves the problem of what to do with other man pages that you might have.

Galaxy - About Time

According to ZDNet: Sun's 'Galaxy' servers making September debut.

Well, it's about time. And not just for these new servers specifically, but the flow of new product out of Sun has all but dried up this year and Sun need to get some new products out into the marketplace.

I generally like the look of these boxes (although the internal storage is more limiting than I would like).

What does worry me, though, is the lack of choice and diversity. I don't see any tower systems or dirt-cheap single processor boxes (like a server version of the Ultra 20, for example).

Thursday, September 01, 2005

Those ole backup tapes

Some technologies are spectacularly primitive and have been largely replaced by modern equivalents. One such is backup - the idea that you use something like ufsdump to back up a Solaris filesystem has no place in the 21st century. Tools such as Legato Networker (or the Sun equivalent - whatever it may be called this week) or Veritas NetBackup should have relegated ufsdump to museums years ago.

(I have used ufsdump occasionally - usually piped direct into ufsrestore as a quick hack to copy a filesystem.)

But having closed down the systems at my former employer, deleted all the data and sent the machines off to oblivion, and burnt all the backup tapes, of course I needed to get some files back for one particular project.

So: no tapes, no tape libraries, no backup server, no legato license. But I did have a ufsdump tape made several years ago.

Every week we used to drop a copy of critical system files onto a bootstrap tape - jumpstart profiles, copies of systems configurations, basic software. (And we knew the directory trees were stable so we were happy to do the ufsdump live.) The theory was that in the event of a total disaster, we could get any old machine and put enough on it to be able to work well enough to reconstruct everything else. (Having needed to do this, it proved its worth.)

So the files were on this tape. I managed to find a DLT drive in the loft; managed to find a machine to hook it up to; managed to get the files off the tape.

(Finding a suitable machine was more tricky than I expected - I was a little surprised to go round the back of my W2100z - a SCSI based machine - and find no SCSI orifice. Fortunately I'm looking after a SunBlade 2000 for a colleague and that does. The other criterion was that I wanted to drop the files onto a USB connected zip disk, so having SCSI and USB in the same box was a help. I need to get that home network set up!)

So, problem solved. And it just goes to show that despite my reservations about tapes and old backup systems, they do actually come in handy once in a while.

Tuesday, August 30, 2005

The great thing about standards...

You know the old chestnut: The great thing about standards is that there are so many to choose from.

Now SCSI is a great standard. The truth is that SCSI devices do interoperate
incredibly well. At that level, it's been a phenomenal success.

But SCSI really covers a lot of things. The thing that hit me recently was connectors and cables.

I remember old SCSI cables with DB50 and centronics connectors on. I don't remember using them much - they were being phased out when I started playing this game. Then came the small 50-pin connector. Then the HD68, and more recently the VHDCI connector. Did I forget anything?

And then of course there was original SCSI, fast SCSI, fast wide SCSI, FWD, ultraSCSI, and ultra320 SCSI. And single ended plus two variants of differential.

So it's not actually that easy, given a system and a device, to say whether they will in fact interoperate. And then you need to find the right cable. (And then is the cable rated for ultraSCSI speeds?)

Which has bothered me for a while, but recently came to the fore after a bad incident caused a system to fail. So I decide to move the disks from a Netra T1 to a V240 and run the services in a zone. Easy, right? Well, not quite. The T1 has a HD68 orifice as I recall, and I think the V240 has a VHDCI outlet on the back!

So I go for a rummage in the loft and find the right cable. I'll send that across to the machines, and hopefully will be able to set the zone up and have the service back operational later this week.

Monday, August 29, 2005

The good, the bad, and the ugly

Let's start with the good:

We've now got SMBIOS integrated into OpenSolaris. I think it's the systems administrator in me showing, but I find this really exciting!

Now the bad:

I know I don't work for a living at the moment, but still keep an eye on some systems for former colleagues. And one of them suffered a power cut recently. Not just any power cut either, by the sounds of it. I've already alluded to one of the problems, and I had to roll back the SMF repository on another machine, but worse was to come.

Basically, it fried a Netra T1. I managed to persuade it to power on from the lom prompt, but it immediately starts vomiting errors. No system, no ok prompt, just errors. Looks like it can't even run POST. This looks like a dead system to me, beyond hope of repair. That's bad.

And the ugly:

SJVN has been emitting more mindless drivel. I see two choices here - either he's genuinely incapable of understanding licensing, or he's got an axe to grind and is using his journalistic position as a veneer of respect.

The article - at least the snide part about Sun - is simply plain wrong. Using CDDL as the license for OpenSolaris doesn't give Sun control. Exactly the opposite: with CDDL as the license Sun have less control of what people can do with OpenSolaris than they would have had if they had used the GPL.

And as for failing to build a significant programming community, well for one thing they already have gotten a major community, and for another thing getting a handful of part-timers to build Solaris on the cheap was never the driving force - Sun spent a huge amount of cash on building Solaris, and open-sourcing it, and it was done for sound business reasons - such as to open up new markets - not to outsource development. (Or kill the penguin, or introduce submarine patents into Linux, or any of the other stupid reasons naysayers put forward.) Of course, once the open source plan became known, a huge bunch of use started clamouring to get involved so we can modify and improve Solaris in the ways that we want.

Saturday, August 27, 2005

Should I program to Tiger?

Simple question - when writing Java code, should I use the new features in Java 5.0, and thus not allow my code to work with older versions of Java?

There are, after all, some compelling reasons to do so.

In particular, I love Generics - I use Collections a lot, and being able to tell Java that the collection contains objects of a given (often fixed) type makes code so much cleaner. And again working with collections a lot, the enhanced for loop is a big win.

The question still remains - is backwards compatibility more important?

(I never found anything compelling in Java 1.4 to make breaking compatibility with 1.3 worthwhile, but the changes in 5.0 are a different matter.)

Thursday, August 25, 2005

NISminus

I'm a great fan of NIS+. Not only is it very easy to set up and use, it scales well and it's dead easy to manage the data stored in it.

(Much easier than old NIS or LDAP. Sun could be suicidal and ditch it, as they've been threatening too, but replacing it with dramatically inferior solutions like NIS or LDAP is going to make a lot of people miserable.)

Of course, it doesn't always work perfectly. I just had this one case where it stopped working. The server partially booted - the NISplus server was running and serving clients. It's just that the NISplus client on the server wasn't working. So lots of other things wouldn't work. Amongst others, the NFS server. So while I could log in to a client, I couldn't access any files. Bad!

(Some testing indicates an RPC authentication problem. But basically everything goes into the RPC black box and never comes back.)

The only thing I had done was to change the IP address of the server. And yes, I had run nisupdkeys -a to update the server's addresses. So my theory is that somewhere buried deep in the bowels of NISplus is some memory of the machine's old IP address, and I can't spot it right now.

(I managed to get it working again, but I'm not sure which of the half a dozen random kludges was the one that mattered.)

Back to JNI

While still thinking about how to make progress with jkstat - basically, how closely to align it with the existing libjkstat implementation - I started another quick project to use JNI to get at Solaris system information from Java.

And, again, programming in JNI is hard going. The real problem is that it's astonishingly easy to make a mistake. And then it falls over with SIGSEGV and cores on me. Even the stack trace isn't too informative - it tells me that it's gone wrong in the native code (usually when calling back to java). As if I didn't know that! Even knowing where it blew up isn't that much help - it never tells you why. So you have to fix by inspection, and it's not the most obvious syntax.

(For example, it was blowing up at one point. Eventually I realized that I had a typo in the class name, but there was no indication in the failure that that might be the case.)

So what's the application this time? It's a java interface to PICL (the Solaris Platform Information and Control Library). The first aim is to produce a graphical picl browser (like my kstat browser). The prtpicl command is useful, and you get the information with -v, but it's not at all easy to browse through the many pages of output that can result. The picl data is tree structured, so the arrangement used in kstatbrowser works quite well - a Java JTree in the left panel to display the tree structure, and the actual data in a panel on the right.


Give me a day or two to work through the JNI errors I keep making and some code might be ready to release.

Wednesday, August 03, 2005

Keeping Busy

Having finished the last job, I'm at home for a while.

Not entirely idle, though. We've been to a theme park, I've started to sign on, and I've been looking at an odd little NFS problem for some former colleagues.

I've also been going through my CD collection. No, not the musical one! The pile of Sun CDs for Solaris, Java, and all the other bits of software that Sun have made that I've got the media for. After all, how many copies of the StarOffice 5.2 CD do you need? I brought my whole stock home, and have managed to prune it quite considerably.

My next project is to skim through all the sysadmin books I've got, reminding myself of what it used to be like, and what obscure things I'm supposed to know all about but nobody in the 21st century actually uses any more.

(On the first page of the first book, it tells me the key responsibilities of a Systems Administrator. High up on the list is formatting floppy diskettes. Say what?)

Thursday, July 28, 2005

Submerged!

Been a bit busy lately. So much so that I've almost submerged out of sight.

With the closure of my place of work we've been working extremely hard to decommission a pretty significant computing service. It's been tough.

Our kit (aging but still serviceable) has been split up and sent off to a number of other departments - 1, 2, 3, 4, 5, 6 - not to mention some kind people who actually paid us for some of our older kit.

While our service has been shut down, it lives on in spirit.

Actually shutting down a large system cleanly was quite interesting. And, by and large, it did go down cleanly in an organised fashion, while being essentially functional right up to the bitter end. We've been shutting down and decommissioning kit steadily for weeks, and consolidating services onto fewer servers. Shutting a system down is the easy part - we've spend the last two days untangling the spaghetti of ten years worth of patch cables. One of my colleagues was still answering user support queries mid-afternoon yesterday - by early evening the system was down for good. This morning it got put into a lorry and the new owners should be receiving it soon. It just remains to send the last few bits of junk off to the local recycling company, hand in my keys, and put my feet up.