tag:blogger.com,1999:blog-97268332012-01-27T20:28:32.723ZPeter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.comBlogger3881100tag:blogger.com,1999:blog-9726833.post-79215423267871054552011-12-12T22:39:00.000Z2011-12-12T22:40:14.014ZThe <a href="http://www.playframework.org/">Play framework</a> seems to be quite popular around Cambridge.<br /><br />For those not familiar with it, it's a java application framework built for REST. And, like Ruby on Rails, it emphasizes convention over configuration.<br /><br />Rather than creating <i>yet another boring blogging example</i> I decided to use <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> as an example, and see how involved building a RESTful JKstat server was using the Play framework.<br /><br />Creating a project is easy:<br /><br /><div style="background-color: #cccccc; font-family: &quot;Courier New&quot;,Courier,monospace;">play new jkstat</div><br />The one thing I'm not keen on is the way it manages dependencies for you to get you jar files. You can either mess about with yaml files, or drop the jar file into the lib directory of the project. Full details of the complex way are in the play directory of the JKstat source.<br /><br />The next step is to decide how to route requests. The JKstat client only has a few requests, in a fairly fixed form. So I can write the routes file explicitly by hand:<br /><br /><pre style="background-color: #cccccc;"># JKstat queries get sent to JKplay<br />GET /jkstat/getkcid JKplay.getkcid<br />GET /jkstat/list JKplay.list<br />GET /jkstat/get/{module}/{instance}/{name} JKplay.get<br /></pre><br />All this means is that if a client requests <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">/jkstat/list</span>, then the <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">list()</span> method in the <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">JKplay</span> class gets called.<br /><br />Slightly more complex, something of the form <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">/jkstat/get/module/instance/name</span> will invoke a call to <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">get(module, instance, name)</span>.<br /><br />Putting this in a routes file in the application's conf directory is all that's needed to set up the routing. The other thing you need to do is write the <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">JKplay</span> class and put the java source in the application's app/controllers directory. The class just contains public static void methods with the correct signatures. For example:<br /><br /><pre style="background-color: #cccccc;"> public static void list() {<br /> KstatSet kss = new KstatSet(jkstat);<br /> renderJSON(kss.toJSON());<br /> }<br /></pre><br />JKstat knows how to generate JSON output, and the <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">renderJSON()</span>call tells Play that this is JSON data (which just means it won't do anything with it, like format a template, which is the normal mode of operation).<br /><br />And that's basically it. All I then had to do was run the project (with LD_LIBRARY_PATH set to find my jni library) and it was all set. The JKstat client was able to communicate with it just fine.<br /><br /><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-7921542326787105455?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-12604276384926065632011-12-11T11:46:00.001Z2011-12-11T12:43:34.338ZFor the recently released update to <a href="http://www.petertribble.co.uk/Solaris/jangle.html">Jangle</a> I wanted to do a couple of things for the cousin and sibling tabs.<br /><br />First, I wanted to have the list of OIDs and associated data to be slightly better formatted and to update along with the charts. This wasn't too difficult - the chart knows what the data is, so I got it to extend <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">AbstractTableModel</span> and could then simply use <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">JTable</span> to display it, just calling <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">fireTableDataChanged</span> at the end of the loop that updates the data.<br /><br />(As an aside, I'm irritated that the API doesn't include <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">fireTableColumnsChanged</span>. You can do the whole table, or a cell, or by row, but not by column.)<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://www.petertribble.co.uk/Solaris/jangle2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="243" src="http://www.petertribble.co.uk/Solaris/jangle2.png" width="320" /></a></div>The second thing I wanted to do was to allow the user to select the list of items to be shown in the chart. The picture above shows the table with the data, and how I wanted it to look. (And is exactly how I managed to implement it.)<br /><br />Now, the table is already showing the list of available data, so I just added a third column to handle whether it's displayed in the chart or not. I simply implemented <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">getColumnClass</span> and returned <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">Boolean.class</span> for the third column, and the <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">JTable</span> automatically shows it as a checkbox. (With the value of the checkbox simply toggled from the underlying model.)<br /><br />The next step was to be able to edit that field - tick or untick the checkbox - and get it to update the chart. The first step is easy enough - just get <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">isCellEditable</span> to return true for the third column.<br /><br />I then actually got stuck, because the documentation simply isn't clear as to how to connect that all back up. I was looking for all sorts of listeners or event handlers, and couldn't find anything. Searching found a number of threads where someone else clearly didn't understand this either, with responses that were rude, patronising, or unhelpful - from people who regarded the answer as obvious.<br /><br />Anyway, the important thing is that it really is easy and obvious once you've worked it out. Making the cell editable automatically creates a cell editor, and all the plumbing is created for you. All you have to do is implement <span style="font-family: &quot;Courier New&quot;,Courier,monospace;">setValueAt</span> which gets called when you do your edit. So my implementation simply adds or removes the relevant field from the chart. (And because that's what's supplying the model, the value displayed in the checkbox automatically tracks it.)<br /><br />That's the sort of basic thing that ought to be covered in documentation but isn't; this blog entry is there for the next time I forget how to do it.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-1260427638492606563?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-57941724094818398902011-12-03T18:13:00.001Z2011-12-03T18:38:46.005ZWhile Solaris 10 comes with <a href="http://gcc.gnu.org/">gcc</a>, it's quite an old version. For some modern code, you need to use a newer version - this is the case in my <a href="http://www.petertribble.co.uk/Solaris/node.html">Node.js for Solaris</a> builds, for example.<br /><br />Actually building a current gcc on Solaris 10 turns out to be reasonably straightforward but, as in most things, there's a twist. So the follwoing is the procedure I've used successfully to get 4.6.X built.<br /><br />You first need to download the release tarball from <a href="http://gcc.gnu.org/">http://gcc.gnu.org/</a>, and unpack it:<br /><br /><div style="font-family: &quot;Courier New&quot;,Courier,monospace;">bzcat /path/to/gcc-4.6.2.tar.bz2 | gtar xf -</div><br />There are 3 prerequisites: <a href="http://gmplib.org/">gmp</a> (4.3.2), <a href="http://www.mpfr.org/">mpfr</a> (2.4.2), and <a href="http://www.multiprecision.org/">mpc</a> (0.8.1). However, you should use the specific version mentioned, which may not be the current versions. Conveniently, there's a copy of the right versions on <a href="ftp://gcc.gnu.org/pub/gcc/infrastructure/">ftp://gcc.gnu.org/pub/gcc/infrastructure/</a>. Go into the gcc source, unpack them, and rename them to remove the version numbers.<br /><br /><div style="font-family: &quot;Courier New&quot;,Courier,monospace;">cd gcc-4.6.2</div><div style="font-family: &quot;Courier New&quot;,Courier,monospace;">bzcat /path/to/gmp-4.3.2.tar.bz2 | gtar xf -<br />mv gmp-4.3.2 gmp</div><div style="font-family: &quot;Courier New&quot;,Courier,monospace;">zcat /path/to/mpfr-2.4.2.tar.bz2 | gtar xf -<br />mv mpfr-2.4.2 mpfr<br />gzcat /path/to/mpc-0.8.1.tar.gz | gtar xf -<br />mv mpc-0.8.1 mpc</div><div style="font-family: &quot;Courier New&quot;,Courier,monospace;">cd ..</div><br />You have to build from outside the tree. If you followed the above, you'll be in the parent to gcc-4.6.2. Create a build directory and change into it:<br /><div style="font-family: &quot;Courier New&quot;,Courier,monospace;"><br /></div><span style="font-family: &quot;Courier New&quot;,Courier,monospace;">mkdir build</span><br /><span style="font-family: &quot;Courier New&quot;,Courier,monospace;">cd build</span><br /><br />Then you're ready to configure and build:<br /><br /><div style="font-family: &quot;Courier New&quot;,Courier,monospace;">env PATH=/usr/bin:$PATH ../gcc-4.6.2/configure \</div><div style="font-family: &quot;Courier New&quot;,Courier,monospace;">--prefix=/usr/local/versions/gcc-4.6.2 \</div><div style="font-family: &quot;Courier New&quot;,Courier,monospace;">--enable-languages=c,c++,fortran<br /></div><span style="font-family: &quot;Courier New&quot;,Courier,monospace;">env PATH=/usr/bin:$PATH gmake -j 4</span><br /><span style="font-family: &quot;Courier New&quot;,Courier,monospace;">env PATH=/usr/bin:$PATH gmake install</span><br /><br />There are three things to note here.<br /><br />First is that I'm installing it into a location that is specific to this particular version of gcc. You don't have to, but I maintain large numbers of different versions of all sorts of applications, so they always live in their own space. You can put symlinks into a common location if necessary.<br /><br />The second is one of the key tricks: I just build c, c++, and fortran. They're the only languages I actually need, and the build dies spectacularly with other languages enabled.<br /><br />The third is that I force /usr/bin to the front of the PATH. Not ucb, and not xpg4 either.<br /><br />You'll have to wait a while (and then some), but hopefully when that's all finished you'll have a modern compiler installed that will make building modern software such as <a href="http://nodejs.org/">Node.js</a> much easier.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-5794172409481839890?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-35306696286215544882011-11-07T20:51:00.000Z2011-11-07T20:51:14.299ZI spend rather a lot of my time working with images. Scanning and digitizing is one thing we do on a fairly large scale.<br /><br />To allow customers to see image detail at high resolution, we use <a href="http://www.zoomify.com/">Zoomify</a> extensively. We generate the images in advance, using <a href="http://sourceforge.net/projects/zoomifyimage/">ZoomifyImage</a>. OK, so we end up with huge numbers of files to store, but this is the 21st century and we can cope with it.<br /><br />There are other options available that do similar things, of course. Just to mention a few: <a href="http://www.microsoft.com/silverlight/deep-zoom/">Deep Zoom</a>, <a href="http://www.openzoom.org/">OpenZoom</a>, and <a href="http://openlayers.org/">OpenLayers</a>. One snag with some of these zooming capabilities is that they require browser plugins (Flash or Silverlight). Not only does this hurt users who don't have the plugin installed, but some platforms (OK, let's call out the iPad here) don't have any prospect of Flash or Silverlight.<br /><br />However, modern systems do have HTML5 capable browsers, and HTML5 is really powerful.<br /><br />A quick search finds <a href="http://www.akademy.co.uk/software/canvaszoom/canvaszoom.php">CanvasZoom</a>, which is a pretty good start. Given a set of Deep Zoom tiles it just works. I tried it on an iPad and it sort of works, not really doing touch properly. So I <a href="https://github.com/ptribble/CanvasZoom">forked it on github</a> (<a href="https://github.com/ptribble/ImageLoader">with ImageLoader for compatibility</a>) and played with adding touch support.<br /><br />It turns out the adding touch handling is pretty trivial. There's just the touchstart, touchend and touchmove events to handle. You want to call event.preventDefault() so as to stop the normal platform handling of moves in particular. The only tricky bit was working out that while you can get the coordinates for touchstart and touchmove from event.targetTouches[0], for touchend you have to look back at event.changedTouches[0]. So, poke the image and it zooms, poke and move and you can pan the image.<br /><br />One thing I mean to do is to look at whether I can point it at a set of Zoomify tiles. I've already got lots of those, and just having one set of image tiles saves both processing time and storage. If not, I'll have to generate a whole load more images - I'm using <a href="http://code.google.com/p/deepjzoom/">deepjzoom</a> which seems to do a pretty good job.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-3530669628621554488?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-24336749323109230222011-10-14T14:36:00.000+01:002011-10-14T14:36:14.240+01:00I've got a mysql server that's a bit busy, and I idly wondered why. Now, this is mysql 4, so it's a bit old and doesn't have a lot of built-in diagnostics.<br /><br />(In case you're wondering, we have an awful lot of legacy client code that issues queries using a join syntax that isn't supported by later versions of mysql, so simply upgrading mysql isn't an option.)<br /><br />This is running on Solaris, and a quick look with iostat indicates that the server isn't doing any physical I/O. That's good. A quick look with fsstat indicates that we're seeing quite a lot of read activity - all out of memory, as nothing goes to disk. (We're using zfs, which makes it trivially easy to split the storage up to give mysql its own file system, so we can monitor just the mysql traffic using fsstat.)<br /><br />As an aside, when reading MyISAM tables the server relies on the OS to buffer table data in RAM, so you actually see the reads of tables as reads of the underlying files, which gets caught in fsstat and makes the following trivial.<br /><br />But, which tables are actually being read? You might guess, by looking at the queries ("show full processlist" is your friend), but that simply tells you which tables are being accessed, not how much data is being read from each one.<br /><br />Given that we're on Solaris, it's trivial to use DTrace to simply count both the read operations and the bytes read, per file. The following one-liners from the <a href="http://dtracebook.com/index.php/File_Systems">DTrace Book</a> are all we need. First, to count reads per file:<br /><br />dtrace -n 'syscall::read:entry /execname == "mysqld"/ { @[fds[arg0].fi_pathname] = count(); }'<br /><br />and bytes per file:<br /><br />dtrace -n 'fsinfo:::read /execname == "mysqld"/ { @[args[0]-&gt;fi_pathname] = sum(arg1); }'<br /><br />With DTrace, aggregation is built in so there's no need to post-process the data.<br /><br />The latter is what's interesting here. So running that quickly and looking at the last 3 lines which are the most heavily read files:<br /><br />&nbsp; /mysql/data/foobar/subscriptions_table.MYD&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 42585701<br />&nbsp; /mysql/data/foobar/concurrent_accesses.MYD&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 83717726<br />&nbsp; /mysql/data/foobar/databases_table.MYD&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 177066629<br /><br />That last table accounts for well over a third of the total bytes read. A quick look indicates that it's a very small table (39kbytes whereas other tables are quite a bit larger).<br /><br />A quick look in mysql using DESCRIBE TABLE showed that this table had no indexed columns, so the problem is that every query is doing a full table scan. I added a quick index on the most likely looking column and the bytes read drops down to almost nothing, with a little speedup and corresponding reduction of load on the server.<br /><br />I've used this exact technique quite a few times now - fsstat showing huge reads, a DTrace one-liner to identify the errant table. Many times, users and developers set up a simple mysql instance, and it's reasonably quick to start with so they don't bother with thinking about an index. Later (possibly years later) data and usage grows and performance drops as a result. And often they're just doing a very simple SELECT.<br /><br />It's not always that simple. The other two most heavily read tables are also interesting. One of them is quite large, gets a lot of accesses, and ends up being quite sparse. Running OPTIMIZE TABLE to compact out the gaps due to deleted rows helped a lot there. The other one is actually the one used by the funky join query that's blocking the mysql version upgrade. No index I create makes any difference, and I suspect that biting the bullet and rewriting the query (or restructuring the tables somewhat more sanely) is what's going to be necessary to make any improvements.<br /><br /><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-2433674932310923022?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-36368463681030572962011-10-08T17:23:00.000+01:002011-10-08T17:23:20.579+01:00I've just updated <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a>, now up to version 0.60.<br /><br />The key change this time, and the reason for a jump in version number (the previous was 0.53) is that I've changed the way that updates to the kstat chain are handled.<br /><br />Now, libkstat has a kstat_chain_update() function, which you call to synchronize your idea of what kstats exist with the current view held by the kernel. And you can look at the return value to see if anything has changed.<br /><br />This only works if you're running in a single thread, of course. If you have multiple threads, then it's possible that only one will detect a change. Even worse if you have a server with multiple clients.So, the only reliable way for any consumer to detect whether the chain has been updated is to retrieve the current kstat chain ID and compare it with the one it holds.<br /><br />This has largely been hidden because I've usually used the KstatSet class to track updates, and it does the right thing. It checks the kstat ID and doesn't blindly trust the return code from kstat_chain_update(). (And it handles subsets of kstats and will only notify its consumers of any relevant changes.)<br /><br />So what I've finally done is eliminate the notion of calling kstat_chain_update(), which should have been done long ago. The native code still calls this internally, to make sure it's correctly synchronized with the kernel, but all consumers need to track the kstat chain ID themselves.<br /><br />This change actually helps client-server operation, as it means we only need one call to see if anything has changed rather than the two that were needed before.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-3636846368103057296?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-39986705012446974102011-09-30T21:28:00.001+01:002011-09-30T21:28:44.503+01:00By default, a Solaris zone has access to all the resources of the host it's running on. Normally, I've found this works fine - most applications I put in zones aren't all that resource hungry.<br /><br />But if you do want to place some limits on a zone, then the zone configuration offers a couple of options.<br /><br />First, you can simply allocate some CPUs to the zone:<br /><br /><pre>add dedicated-cpu<br />set ncpus=4<br />end<br /></pre><br />Or, you can cap the cpu utilization of the zone:<br /><br /><pre>add capped-cpu<br />set ncpus=4<br />end<br /></pre><br />I normally put all the configuration commands for a zone into a file, and use <code>zonecfg -f</code> to build the zone; if modifying a zone then I create a fragment like the above and load that the same way.<br /><br />In terms of stopping a zone monopolizing a machine, the two are fairly similar. Depending on the need, I've used both.<br /><br />When using dedicated-cpu, it's not just a limit but a guarantee. Those cpus aren't available to other zones. Sometimes that's exactly what you want, but it does mean that those cpus will be idle if the zone they're allocated to doesn't use them.<br /><br />Also, with dedicated-cpu, the zone thinks it's only got the specified number of cpus (just run psrinfo to see). Sometimes this is necessary for licensing, but there was one case where I needed this for something else: consolidating some really old systems running a version of the old Netscape Enterprise Server, and it would crash at startup. I worked out that this was because it collected performance statistics on all the cpus, and someone had decided that hard coding the array size at 100 (or something) would cover all future possibilities. That was, until I ran it one a T5140 with 128 cpus and it segfaulted. Just giving the zone 4 cpus allowed it to run just fine.<br /><br />I use capped-cpu when I just want to stop a zone wiping out the machine. For example, I have a machine that runs application servers and a data build process. The data build process runs only rarely, but launches many parallel processes. When it had its own hardware that was fine: the machine would have occasional overload spikes but was otherwise OK. When shared with other workloads, we didn't want to change the process, but have the build zone capped at 30 or 40 cpus (on a 64-way system) so there's plety of cpu left over for other workloads.<br /><br />One advantage of stopping runaways with capped-cpu is that you can limit each zone to, say, 80% of the system, and you can do that for all zones. It looks like you're overcommitting, but that's not really the case - uncapped is the same as a cap of all the cpus, so you're lower than that. This means that any one zone can't take the system out, but each zone still has most of the machine if it needs it (and the system has the available capacity).<br /><br />The capability to limit memory also exists. I haven't yet had a case where that's been necessary, so have no practical experience to share.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-3998670501244697410?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com2tag:blogger.com,1999:blog-9726833.post-73177396368624343962011-08-08T20:13:00.002+01:002011-08-08T21:37:47.373+01:00Following on from some <a href="http://ptribble.blogspot.com/2011/08/thoughts-on-zfs-compression.html">thoughts on ZFS compression</a>, and nudged by one of the comments, what about ZFS dedup? <br /> <br />There's also a somewhat <a href="http://constantin.glez.de/blog/2011/07/zfs-dedupe-or-not-dedupe">less opinionated</a> article that you should definitely read. <br /> <br />So, my summary: unlike compression, dedup should be avoided unless you have a specific niche use. <br /> <br />Even for a modest storage system, say something in the 25TB range, then you should be aiming for half a terabyte of RAM (or L2ARC). Read the article above. And the point isn't just the cost of an SSD or a memory DIMM, it's the cost of a system that can take enough SSD devices or has enough memory capacity. And then think about a decent size storage system that may scale to 10 times that size. Eventually, the time may come, but my point is that while the typical system you might use today already has cpu power going spare to do compression for you, you're looking at serious engineering to get the capability to do dedup. <br /> <br />We can also see when turning on dedup might make sense. A typical (server) system may have 48G of memory so, scaled from the above, something in the range of 2.5TB of unique data might be a reasonable target. Frankly, that's pretty small, and you actually need to get a pretty high dedup ratio to make the savings worthwhile. <br /> <br />I've actually tested dedup on some data where I expected to get a reasonable benefit: backup images. The idea here is that you're saving similar data multiple times (either multiple backups of the same host, or backups of like data from lots of different hosts). I got a disappointing saving - of order 7% or so. Given the amount of memory we would have needed to put into a box to have 100TB of storage, this simply wasn't going to fly. By comparison, I see 25-50% compression on the same data, and you get that essentially for free. And that's part of the argument behind having compression on all the time, and avoiding dedup entirely. <br /> <br />I have another opinion here as well, which is that using dedup to identify identical data after the fact is the wrong place to do it, and indicates a failure in data management. If you know you have duplicate data (and you pretty much have to know you've got duplicate data to make the decision to enable dedup in the first place) then you ought to have management in place to avoid creating multiple copies of it: snapshots, clones, single-instance storage, or the like. Not generating duplicate data in the first place is a lot cheaper than creating all the multiple copies and then deduplicating them afterwards. <br /> <br />Don't get me wrong: deduplication has its place. But it's very much a niche product and certainly not something that you can just enable by default.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-7317739636862434396?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com2tag:blogger.com,1999:blog-9726833.post-49802583227856249412011-08-07T18:53:00.002+01:002011-08-07T19:18:56.408+01:00Apart from the sort of features that I now take for granted in a filesystem (data integrity, easy management, extreme scalability, unlimited snapshots), ZFS also has built in compression.<br /><br />I've already noted how this can be <a href="http://ptribble.blogspot.com/2010/01/compressing-backup-catalogs.html">used to compress backup catalogs</a>. One important thing here is that it's completely transparent, which isn't true of any scheme that goes around compressing the files themselves.<br /><br />Recently, I've (finally) started to enable compression more widely, as a matter of course. Certainly on new systems there's no excuse, at the default level of compression at any rate.<br /><br />There was a caveat there: at the default compression level. The point here being that the default level of compression can get you decent gains and is essentially free: you gain space and reduce I/O for a negligible CPU cost. The more aggressive compression schemes can compress your data more, but having tried them it's clear that there's a significant performance hit: in some cases when I tried it the machine can freeze completely for a few seconds, which is clearly noticeable to users. Newer more powerful machines shouldn't have that problem, and there have been improvements in Solaris as well that keep the rest of the system more responsive. I still feel, though, that enabling more aggressive compression than the default is something that should only be done selectively when you've actually compared the costs and benefits.<br /><br />So, I'm enabling compression on every filesystem containing regular data from now on.<br /><br />The exception, still, is large image filesystems. Images in TIFF and JPEG format are already compressed so the benefit is pretty negligible. And the old thumpers we still use extensively have relatively little CPU power (both compared to more modern systems, and for the amount of data and I/O these systems do). Compression here is enabled more selectively.<br /><br />Given the continuing growth in cpu power - even our entry-level systems are 24-way now - I'm expecting it won't be long before we get to the point where enabling more aggressive compression all the time is going to be a no-brainer.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-4980258322785624941?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com6tag:blogger.com,1999:blog-9726833.post-88365571428275387352011-07-28T15:10:00.002+01:002011-07-28T15:21:04.421+01:00One advantage of the service management facility (SMF) in Solaris is that you can treat services as single units: you can use <code>svcadm</code> to control all the processes associated with a service in one command. This makes dealing with problems a lot easier than trying to grope through <code>ps</code> output trying to kill the right processes.<br /><br />Sometimes, though, that's not enough. If your system is under <em>real</em> stress (think swapping itself to death with thousands of rogue processes) then you can find that <code>svcadm disable</code> or <code>svcadm restart</code> simply don't take.<br /><br />So, what to to? It's actually quite easy, once you know what you're looking for.<br /><br />The first step is to get details of the service using <code>svcs</code> with the <code>-v</code> switch. For example:<pre><br /># svcs -v server:rogue<br />STATE NSTATE STIME CTID FMRI<br />online - 10:06:05 7226124svc:/network/server:rogue<br /></pre>(you'll notice a presentation bug here). The important thing is the number in the CTID column. This is the contract ID. You can then use <code>pkill</code> with the <code>-c</code> switch to send a signal to every process in that process contract, which defines the boundaries of the service. So:<pre><br />pkill -9 -c 7226124<br /></pre>and they will all go. And then, of course SMF will neatly restart your service automatically for you.<br /><br />(Why use -9 rather than a friendlier signal? In this case, it was because I just wanted the processes to die. Asking then nicely involves swapping them back in, which would take forever.)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-8836557142827538735?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-86438194420363024842011-07-26T20:18:00.003+01:002011-07-26T20:40:40.912+01:00I've recently acquired some new servers - some SPARC <a href="http://www.oracle.com/us/products/servers-storage/servers/sparc-enterprise/t-series/sparc-t3-1-server-170341.html">T3-1s</a> and some x86 based <a href="http://www.oracle.com/us/products/servers-storage/servers/x86/sun-fire-x4170-m2-server-077278.html">X4170M2s</a>.<br /><br />One of the interesting things about these is that the internal drives are multipathed by default - so you get device names like c0t6006016021B02C00F22A3EED6CADE011d0s2 rather than the more traditional c0t0d0s2.<br /><br />This makes building a jumpstart profile a bit more tedious than normal, because you need to have a separate disk configuration section for every box - because the device names are different on each box.<br /><br />However, there's another minor problem. How do you easily map from the WWN-based device names to physical positions in the chassis? You really need this so you're sure you're swapping the right drive. And while a SPARC system really doesn't mind which disk it's booting from, for an x86 system it helps if you install the OS on the first disk in the BIOS boot order.<br /><br />The answer is to install the <a href="http://www.oracle.com/technetwork/server-storage/servermgmt/tech/hardware-management-pack/index.html">Oracle Hardware Management Pack</a>. (Why this isn't even on the preinstalled image I can't explain.) This seems to work on most current and recent Sun server models.<br /><br />Now, actually getting the Hardware Management Pack isn't entirely trivial. So prepare to do battle with the monstrosity called My Oracle Support.<br /><br />So, you're logged in to My Oracle Support. Click the Patches & Updates tab. In the Patch Search area, click the link marked 'Product or Family (Advanced)'. Then scroll down the dropdown list and select the item that says 'Oracle Hardware Management Pack'. Then choose some of the most recent releases (highest version numbers - note that different hardware platforms match different version numbers of the software) and select your desired platform (essentially, SPARC or X86 or both) from the dropdown to the right of where it says 'Platform is'. Then hit the Search button.<br /><br />Assuming the flash gizmo hasn't crashed out on you (again) you should get a list of patches. No, I have no idea why they're called patches when they're not. You can then click on the one you want and download it.<br /><br />What you get is a zip file, so you can unzip that, cd into it and then into the SOFTWARE directory inside it, and then run the install.bin file you find there. (You may have to chmod the install.bin file to make it executable.) I just accept all the defaults and let it get on with it.<br /><br />On a preinstalled system it may claim it's already installed. It probably isn't - just 'pkgrm ipmitool' first. And if you're using your own jumpstart profile, make sure the SUNWCsma cluster is installed. It may be necessary to wait a while and then 'svcadm restart sma' to get things to take the first time.<br /><br />So, once it's installed, what can you do?<br /><br />The first thing is that there's a Storage tab in the ILOM web interface. Go there once you've got the hardware management pack installed and you should be able to see the controllers and disks enumerated.<br /><br />On the system itself, the <code>raidconfig</code> command is very useful. Something like<pre><br />raidconfig list all<br /></pre>will give you a device summary, and<pre><br />raidconfig list disk -c c0 -v<br /></pre><br />will give a verbose listing of the disks on controller c0. (And. just to remind you, the c0 in c0t6006016021B02C00F22A3EED6CADE011d0s2 doesn't refer to physical controller 0.)<br /><br />The hardware management pack is really useful - if you're running current generation Sun T-series or X-series hardware, you ought to get it and use it.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-8643819442036302484?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com5tag:blogger.com,1999:blog-9726833.post-5849931918728545532011-07-08T13:05:00.004+01:002011-07-08T13:32:40.555+01:00Over the years, simple tools like cpustate have allowed you to get a quick visualization of cpu utilization on Solaris. It's sufficiently simple and useful that emulating it was one of the first demos I put together using <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a>.<br /><br />Its presentation starts to suffer as core and thread counts continue to rise. So recently I <a href="http://ptribble.blogspot.com/2011/05/jkstat-044.html">added vertical bars</a> as an option. This allows a more compact representation, and also works better given that most screens are wider that they are tall.<br /><br />Still, even that doesn't work very well on a 128-way machine. And, by treating all cpus equally, you can't see how the load is distributed in terms of the processor topology.<br /><br />However, as of the new release of <a href="http://www.petertribble.co.uk/Solaris/solview.html">SolView</a> there's now a 'vertical' option for the enhanced cpustate demo included there.<br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/-IsXDRe65ICc/Thb0TTXNxBI/AAAAAAAAAOw/kicCUFGOkyY/s1600/cpustate2.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 250px;" src="http://3.bp.blogspot.com/-IsXDRe65ICc/Thb0TTXNxBI/AAAAAAAAAOw/kicCUFGOkyY/s320/cpustate2.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5626953396898350098" /></a><br />So, what's here? This is a 2-chip system (actually an X4170M2), each chip has 6 cores, each of which has 2 threads. The chips are stacked above each other, and within each chip is a smaller display for each core, and within each core are its threads. All grouped together to show how the threads and cores are related, and each core and chip having an aggregate display.<br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/-F_VVc9Rvmg8/Thb25BN62pI/AAAAAAAAAO4/W2UlDi_c6wo/s1600/cpustate3.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 73px;" src="http://3.bp.blogspot.com/-F_VVc9Rvmg8/Thb25BN62pI/AAAAAAAAAO4/W2UlDi_c6wo/s320/cpustate3.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5626956243885808274" /></a><br />Above is a T5140 - 2 chips, each with 8 cores each with 8 threads.<br /><br />I find it interesting to watch these for a while, and you can see how work is scheduled. What you normally see is an even spread across the cores. Normally, if there's an idle core you see a process sent there rather than running on a thread on a busy core and competing for shared resources. (Not always: you can see on the first graph that there's one idle core and another core with both threads busy, which is unusual.) The scheduler is clearly aware of the processor topology and generally distributes the work pretty well to make best use of the available cores.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-584993191872854553?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-33912098411363510952011-06-30T19:34:00.002+01:002011-06-30T19:40:02.873+01:00As a result of a new version of <a href="http://nodejs.org/">Node</a> being released, I've updated my Solaris 10 (x86) package, available <a href="http://www.petertribble.co.uk/Solaris/node.html">here</a>.<br /><br />This updates Node to 0.4.9 and the bundled cURL to 7.21.7.<br /><br />Building this was exactly the same as in my <a href="http://ptribble.blogspot.com/2011/06/building-nodejs-on-solaris-10.html">earlier</a> <a href="http://ptribble.blogspot.com/2011/06/nodejs-on-solaris-part-2.html">posts</a>. I've updated the <a href="http://www.petertribble.co.uk/Solaris/node-v0.4.9.soldiff">patch</a>, although the observant will notice that it's the same patch with the path updated; there's no functional change in the patch.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-3391209841136351095?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-9492811931801579602011-06-28T23:26:00.002+01:002011-06-28T23:51:55.244+01:00One problem I've been looking at recently is how to serve - efficiently - directories containing large numbers of tiny files to http clients. At the moment, we just create the files, put them on a filesystem, and let apache go figure it out.<br /><br />The data is grouped, though, so that each directory is an easily identifiable and self-contained chunk of data. And, if a client accesses one file, chances are that they're going to access many of the other neighbouring files as well.<br /><br />We're a tiny tiny fraction of the way into the project, and we're already up to 250 million files. Anyone who's suffered with traditional backup knows that you can't realistically back this data up, and we don't even try.<br /><br />What I do, though, is generate a zip file of each directory. One file instead of 1000, many orders of magnitude less in your backup catalog (thinking about this sort of data, generating a backup index or catalog can be a killer), and you can stream large files instead of doing random I/O to tiny little files. We save the zip archives, not the original files.<br /><br />So then, I've been thinking, why not serve content out of the zip files directly? We cut the number of files on the filesystem dramatically, improve performance, and make it much more manageable. And the access pattern is in our favour as well - once a client hits a file, they'll probably access many more files in the same zip archive, so we could prefetch the whole archive for even more efficiency.<br /><br />A quick search turned up <a href="http://onteria.wordpress.com/2011/05/31/war-like-application-deployment-in-node-js/">this article</a>. It's not exactly what I wanted to do, but it's pretty close. So I was able to put together a very simple node script using the <a href="http://expressjs.com/">express framework</a> that serves content out of a zip file.<br /><pre><br />// requires express and zipfile<br />// npm install express<br />// npm install zipfile<br />//<br /><br />function isDir(pathname) {<br /> if (!pathname) {<br /> return false;<br /> } else {<br /> return pathname.slice(-1) == '/';<br /> }<br />}<br /><br />var zipfile = require('zipfile').ZipFile;<br />var myzip = new zipfile('/home/peter/test.zip');<br /><br />// hash of filenames and content<br />var ziptree = {};<br />for (var i = 0; i < myzip.names.length ; i++) {<br /> if (!isDir(myzip.names[i])) {<br /> ziptree[myzip.names[i]] = myzip.readFileSync(myzip.names[i])<br /> }<br />}<br /><br />var app = require('express').createServer();<br /><br />app.get('/zip/*', function(req, res){<br /> if(ziptree[req.params[0]]) {<br /> res.send(ziptree[req.params[0]], {'Content-Type': 'text/plain'});<br /> } else {<br /> res.send(404);<br /> }<br />});<br /><br />app.listen(3000);<br />console.log("zip server listening on port %d", app.address().port);<br /></pre><br />So, a quick explanation:<ul><br /><li>I open up a zipfile and, for each entry in it that isn't a directory, shove it into a hash with the name as the key and the data as the value.</li><br /><li>I use express to route any request under /zip/, the filename is everything after the /zip/, and I just grab that path from the hash and return the data.</li></ul><br /><br />See how easy it is to generate something pretty sophisticated using <a href="http://nodejs.org/">Node</a>? And even I can look at the above and understand what it's doing.<br /><br />Now, the above isn't terribly complete.<br /><br />For one thing, I ought to work out the correct content type for each request. This isn't hard but adds quite a lot of lines of code.<br /><br />The other thing that I want to do is to have the application handle multiple zip files. So you get express to split up the request into a zipfile name and the name of a file within the zip archive. And then keep a cache of recently used zipfiles.<br /><br />Which leaves a little work for version 2.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-949281193180157960?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com1tag:blogger.com,1999:blog-9726833.post-87232331383865244182011-06-18T17:12:00.002+01:002011-06-18T17:21:13.100+01:00Now I've got got <a href="http://nodejs.org/">node.js</a> built on Solaris (see <a href="http://ptribble.blogspot.com/2011/06/building-nodejs-on-solaris-10.html">blog entry 1</a> and <a href="http://ptribble.blogspot.com/2011/06/nodejs-on-solaris-part-2.html">blog entry 2</a>) I've been playing with using it as a server.<br /><br />So the next thing I did was to <a href="https://github.com/ptribble/node-kstat">augment node-kstat</a> from Bryan Cantrill's original. There's the odd minor fix, I've essentially completed the list of kstats supported (including most of the raw kstats), and added methods that give the support needed by <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a>, and there's an example jkstat.js script that can run as a server under node that the JKstat client can connect to.<br /><br />A collection of my Node stuff is <a href="http://www.petertribble.co.uk/Solaris/node.html">available here</a>, including a Solaris 10 package for those of you unable to build it yourselves.<br /><br />Of course, in order to use the node-kstat server I've had to add RESTful http client support to <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a>, which has now been updated with the 0.51 release.<br /><br />JKstat also loses the JavaFX demos. It doesn't appear to me that JavaFX is going to be terribly interesting for a while. The 2.0 beta isn't available for Solaris at all (1.x wasn't available for SPARC anyway), and appears to be essentially incompatible anyway.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-8723233138386524418?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-32215673922342709912011-06-17T09:13:00.002+01:002011-06-17T09:24:26.442+01:00I have a number of the old Sun 2530 disk arrays, and a batch that we bought 3 years ago have started to come up with the fault LED lit.<br /><br />These arrays have cache batteries, and the batteries need replacing sometimes to ensure they're operating correctly. Originally, this was a simple 3 year timer. After 3 years, a timer expires and it generates a fault to let you know it's time to put in new batteries.<br /><br />Current Oracle policy is different: rather than relying on a dumb timer and replacing batteries as a precaution, the systems are actually capable of monitoring the health of the batteries (they have built in SMART technology). As a result, they will only send out new batteries if there's actual evidence of a fault.<br /><br />This is actually good, as it means we don't have to take unnecessary downtime to replace the batteries. (And it eliminates the risk of the battery replacement procedure accidentally causing more problems.)<br /><br />Now, the management software version we have (6.xx) doesn't report the SMART status (but will report if a real failure occurs). So you can't see predictive failure, but if CAM just says "near expiration" then it's just the precautionary timer.<br /><br />So, the solution is to check and reset the timer.<br /><br />Go to CAM. (The exact location of the relevant menu item may vary depending on which version of CAM you've got, so you may have to go looking.)<br /><br />Expand the Storage Systems tree<br /><br />Select the array you want to fix<br /><br />Click on service advisor<br /><br />In the new window that pops up, verify that it's actually picked up the correct array. The name should be at the top of the expanded tree in the left-hand panel.<br /><br />Under Array Troubleshooting and recovery, expand the Resetting the Controller Battery Age item.<br /><br />Click on each battery in turn and follow the instructions.<br /><br />This also applies to the 6x80 arrays as well, as I understand it, but I don't have any of those.<br /><br />If you search for "2500 battery" on My Oracle Support, you'll find all this documented.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-3221567392234270991?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com1tag:blogger.com,1999:blog-9726833.post-75177505080905419392011-06-08T22:40:00.002+01:002011-06-08T22:48:57.010+01:00Here's a followup and slight correction to my <a href="http://ptribble.blogspot.com/2011/06/building-nodejs-on-solaris-10.html">notes on building node.js on Solaris 10</a>.<br /><br />If you read carefully, you'll notice that I specified --without-ssl to the configure command. This makes it build, as it looks like there's a dependency on a newer <a href="http://www.openssl.org/">openssl</a> than is shipped with Solaris 10. While this is good enough for some purposes, it turns out the the expresss module wants ssl (it's required, even if you don't use https, although all you have to do is delete the references).<br /><br />So, a better way is to build a current openssl first and then get node to link against that. So for the openssl build:<pre><br />gzcat openssl-1.0.0d.tar.gz | gtar xf -<br />cd openssl-1.0.0d<br />env CC=cc CXX=CC ./Configure --prefix=/opt/Node solaris-x86-cc shared<br />gmake -j 8<br />gmake install<br /></pre>I'm using the Studio compilers here, as I normally do with things like openssl that provide libraries that might be used by other tools, although gcc should work fine. The important parts here are that it matches the architecture of your other components (so 32-bit, ie x86) and you build shared libraries.<br /><br />Then you can unpack and patch node as <a href="http://ptribble.blogspot.com/2011/06/building-nodejs-on-solaris-10.html">before</a>, then configure with<pre><br />env LDFLAGS=-R/opt/Node/lib CFLAGS=-std=gnu99 ./configure --prefix=/opt/Node --openssl-includes=/opt/Node/include --openssl-lib=/opt/Node/lib<br /></pre><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-7517750508090541939?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-41288403464458123122011-06-05T11:34:00.002+01:002011-06-05T12:13:43.877+01:00Constantly on the search for new tools and technologies to play with, I wanted to actually try out <a href="http://nodejs.org/">Node.js</a> on my own machine.<br /><br />I'm running Solaris 10 (x86), and it didn't quite work out first time. But it was pretty trivial to fix. A couple of tweaks to the configure invocation and a simple patch to make things like <code>isnan</code> and <code>signbit</code> work with a vanilla Solaris 10 install.<br /><br />So, to build Node on Solaris 10 x86:<br /><br />First <a href="http://nodejs.org/#download">download node-0.4.8</a> and <a href="http://www.petertribble.co.uk/Solaris/node-v0.4.8.soldiff">my node patch</a>. (Yes, the patch to V8 is ugly, and it's likely to be specific to the V8 version and the Solaris rev and gcc version. And don't expect Node or V8 to work on sparc at all.)<br /><br />Unpack node and apply the patch:<pre><br />gzcat node-v0.4.8.tar.gz | gtar xf -<br />gpatch -p0 -i node-v0.4.8.soldiff<br /></pre><br />Then configure and build:<pre><br />env CFLAGS=-std=gnu99 ./configure --prefix=/opt/Node --without-ssl<br />gmake -j 8<br />gmake install<br /></pre><br />Replacing /opt/Node with wherever you want to install it. (Somewhere you have permission to write to, obviously.)<br /><br />You then want to install <a href="http://npmjs.org/">npm</a>. You will need to have curl for this, although I recommend downloading the install.sh and running it by hand, like so:<pre><br />env PATH=/opt/Node/bin:$PATH TAR=gtar bash install.sh<br /></pre><br />This way, it uses the correct version of tar and uses a compatible shell. (It actually invokes curl in the script, so you still need curl installed. That's not hard, or you can find one on the Companion CD.)<br /><br />To actually use npm you need to continue with the tweaks. For example:<pre><br />env PATH=/opt/Node/bin:$PATH TAR=gtar npm install express<br /></pre>or, if you want it to install express into the Node tree rather than "." you'll need something like:<pre><br />env PATH=/opt/Node/bin:$PATH TAR=gtar npm install -g express<br /></pre><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-4128840346445812312?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-90989424831952035582011-05-31T20:57:00.003+01:002011-05-31T21:31:55.958+01:00I use my web browser a lot. And I mean, <b>a lot</b>. (It's not that I necessarily want to, but the browser seems to have killed off a lot of proper applications, and the world is a poorer place for that.)<br /><br />I've actually switched to Firefox 4. (OK, on Solaris you don't actually have all that much choice.) But normally it takes me very much longer to update to a newly released version of Firefox. Why the move this time? Primarily performance - I've found that some sites (and I suspect Twitter here) can make the whole browser feel sluggish. Certainly since I switched to version 4 the annoying lags I used to have are gone.<br /><br />There are a few changes in Firefox 4 that really can't be described as anything but negative. That's my opinion, of course, but I'm a heavy user and it has really irked me that for the last couple of decades we've matched advances in computing with compensating steps backward.<br /><br />The first thing that didn't actually bother me initially but soon got spectacularly irritating was the new "Tabs on Top" feature. (Ahem, misfeature.) What this really means is that the tabs for a page are separated from the page they apply to by a couple of tool bars (in my case, the Bookmarks toolbar and the Navigation toolbar). This makes the user suffer thrice: the interface appears to put the toolbars into the tabs, causing confusion; separation on screen causes a mental disconnect between the page and its tab; and you have to move the mouse further to get to the tabs, making them harder to use. Fortunately you can turn this off pretty easily by right-clicking (on the home button for example) and unchecking the option.<br /><br />Far worse (because it can't just be turned off) is the switch to tab feature. So you want to open a site, you start typing its address and it appears in the dropdown list. Only if you've already got it open, you don't get to open it, you get to switch to tab - so the browser just goes to the already existing tab. Now, listen people: if I wanted to go to the tab I had already had open, guess what? I would have clicked on the tab! The fact that I'm entering it again means that I absolutely want a new copy. This behaviour is especially annoying when the tab is in another window on another virtual desktop, because it then brings that up (moving the Firefox window to a different virtual desktop in the process). Fortunately there's an <a href="https://addons.mozilla.org/en-US/firefox/addon/switch-to-tab-no-more/">add-on to disable this particular misfeature</a>.<br /><br />Browsing in tabs was a spectacularly useful advance, adding extra scalability to the browsing experience. Firefox 4 attempts to make them less usable; fortunately I've been able to sidestep that (for now).<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-9098942483195203558?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-54440133527134837062011-05-28T22:41:00.002+01:002011-05-28T22:51:01.049+01:00There are a couple of new updates for <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> and <a href="http://www.petertribble.co.uk/Solaris/kar.html">KAR</a>.<br /><br />For <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a>, version 0.45 brings in JSON support, and 0.50 solidifies it.<br /><br />For <a href="http://www.petertribble.co.uk/Solaris/kar.html">KAR</a>, version 0.6.5 likewise introduces JSON support, and 0.7 makes JSON the only supported data format, superseding KAR's private data format.<br /><br />The only point of 0.6.5 is that it can both read and write in both the old KAR format and the new JSON format. So I needed that in order to be able to convert old KAR archives.<br /><br />I've covered some of <a href="http://ptribble.blogspot.com/2011/05/json-meets-jkstat-and-kar.html">the advantages of JSON</a> previously. While those were just early thoughts, subsequent tests confirm the advantages of using JSON for both the client-server communication and the KAR archive format. So these releases represent both a transition and a clean break (which is why there are two new versions of each).<br /><br />Now I've got JSON format data available, I'm learning javascript so as to write a browser-based client. Stay tuned.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-5444013352713483706?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-65494188640233947242011-05-15T22:13:00.003+01:002011-05-15T22:26:57.935+01:00It's about time to modernize Java.<br /><br />I'm not talking fancy things like closures or functional programming. I'm talking some of the basic interfaces in commonly used classes.<br /><br />Exhibit A are the constructors for <code>JTree</code> and <code>JList</code>. These take a <code>Hashtable</code> and a <code>Vector</code>. How long have we had Collections built into Java? So why don't <code>JTree</code> and <code>JList</code> use <code>Map</code> and <code>List</code>?<br /><br />Exhibit B are classes that still return <code>Enumeration</code>. For example, to get the contents of a <code>ZipFile</code> you get an <code>Enumeration</code> and have to work your way through it by hand. Now Java has the enhanced for-loop, there ought to be methods for returning a <code>Collection</code> directly.<br /><br />I could go on, but I think you get the point - many of Java's own classes simply haven't been modernized to bring them in line with improvements elsewhere. And I get really irritated having to continue to write crufty old-fashioned code to deal with those deficiencies.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-6549418864023394724?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-52872306967061742242011-05-14T21:22:00.002+01:002011-05-14T21:51:03.060+01:00I've been looking at <a href="http://www.json.org/">JSON</a> for a long time, wondering whether and how to best make use of it. After considerable experimentation, I'm convinced, and it's going to be an integral part of both <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> and <a href="http://www.petertribble.co.uk/Solaris/kar.html">KAR</a>.<br /><br />So, the first thing I was looking at was the client-server code in <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a>. One of the things that's always irritated me about XML-RPC is it's obvious lack of a long data type. It doesn't seem to be bothered that the world is more than 32-bits. Sure, there are various vendor extensions (and I have to turn them on in the <a href="http://ws.apache.org/xmlrpc/">Apache XML-RPC</a> implementation that I use) but the result is something that's horribly non-portable.<br /><br />The other thing I've been looking at is the archive format used in <a href="http://www.petertribble.co.uk/Solaris/kar.html">KAR</a>. I started out with simple <code>kstat -p</code> but that proved inadequate in that important data (metadata) was missing, so I developed a private format. This was better, but was a horrid hack.<br /><br />So, enter <a href="http://www.json.org/">JSON</a>. It turns out that it makes an excellent serialization format for kstats.<br /><ul><li>Reusing a standard means there's less work for me to do.</li><li>I can use the same code for KAR and the JKstat client-server mode.</li><li>There are JSON bindings in almost all languages, so interoperability is easy to achieve.</li><li>The format is a string, so no messing about with extensions to pass longs.</li><li>For KAR, it's almost twice as quick to parse as my own hacked format.</li></ul><br />As a result, the next versions of <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> and <a href="http://www.petertribble.co.uk/Solaris/kar.html">KAR</a> will support JSON as their archive and interchange format.<br /><br />As an aside, this opens the door to <a href="http://blog.harschsystems.com/uncategorized/remote-cpu-monitoring-using-node-kstat/">wicked cool stuff</a>.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-5287230696706174224?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com1tag:blogger.com,1999:blog-9726833.post-79982752813691344022011-05-03T19:29:00.004+01:002011-05-03T19:36:03.582+01:00Hot on the heels of an <a href="http://ptribble.blogspot.com/2011/05/jkstat-044.html">updated JKstat</a> comes a new release of <a href="http://www.petertribble.co.uk/Solaris/solview.html">SolView</a>.<br /><br />Like with JKstat, there's a lot of tidying up and quite a bit of extra polish. Little things, like changing some of the label text to make it more meaningful, or telling you if a disk or partition is part of a ZFS pool.<br /><br />There's one fairly visible change, to the start of the explorer view, which now looks very different:<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://1.bp.blogspot.com/-0_6RtUZKBo8/TcBKaXTRNMI/AAAAAAAAAOk/5Wqa7aF0MaI/s1600/explorerhome.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 266px;" src="http://1.bp.blogspot.com/-0_6RtUZKBo8/TcBKaXTRNMI/AAAAAAAAAOk/5Wqa7aF0MaI/s320/explorerhome.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5602559753240786114" /></a><br /><br />This is getting closer to what I envisioned early on for this view, a much more graphically rich presentation. It's not finished, and what's shown (and how) will certainly change, but it shows the direction I would like to head in.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-7998275281369134402?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-63172536722681859782011-05-02T11:38:00.003+01:002011-05-02T12:03:57.424+01:00It's been one day off work after another here in the UK recently. In between going to a <a href="http://petertribble.blogspot.com/2011/05/black-bull-sawston.html">beer festival</a> and <a href="http://petertribble.blogspot.com/2011/05/newmarket-races.html">Newmarket races</a>, I've found a little time to work on <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a>.<br /><br />The result is a new version of <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> available for download.<br /><br />There's nothing earth shattering here, but a process of steady improvement. I've gone through the demos, removing some of the cruftier ones, enhancing some of the others, and moving some of those from <a href="http://www.petertribble.co.uk/Solaris/solview.html">SolView</a> back to <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a>. So, the original iostat demo has gone, leaving just the table-based version; the load subcommand has been replaced by an uptime subcommand. The cpustate demo has been merged with the cpuchart demo, and has also had a vertical mode added.<br /><br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://2.bp.blogspot.com/-XpmhLFZV4k0/Tb6LZrKT8-I/AAAAAAAAAOc/ozxYphVcNvo/s1600/cpuvert.png"><img style="float:left; margin:0 10px 10px 0;cursor:pointer; cursor:hand;width: 50px; height: 123px;" src="http://2.bp.blogspot.com/-XpmhLFZV4k0/Tb6LZrKT8-I/AAAAAAAAAOc/ozxYphVcNvo/s320/cpuvert.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5602068259694441442" /></a>OK, so it's not that impressive on my little desktop machine, but it works much better than the old layout as a display arrangement on something like a T5140 that has 128 cpus in it.<br /><br />There's additional polish elsewhere. Apart from fewer typos, I've filtered out kstats from the chartbuilder that you'll never want to chart, so there are fewer kstats to munge through. The charts work a little better too: they now display the statistics in the requested order, and you can set the colors.<br /><br />Another thing I've been playing with is an 'interesting' subcommand that tries to identify kstats with interesting behaviour - as in doing something out of the ordinary. This worked out a whole lot worse than I hoped. The problem, of course, is defining what constitutes normal (let alone anomalous) behaviour. The behaviour of any kstat certainly isn't normally distributed, and in fact many are very spiky as a matter of course. I may revisit this with <a href="http://www.petertribble.co.uk/Solaris/kar.html">KAR</a>, which has historical data from which I can construct a baseline of the expected range of behaviour.<br /><br />Something else I've been looking at is <a href="http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001OR">sparklines</a>. I added a simple line graph to <a href="http://www.petertribble.co.uk/Solaris/jstripchart.html">JStripChart</a> to support this. At the moment they're not really sparklines, just little graphs, but it's a start, and expect to see more of that in <a href="http://www.petertribble.co.uk/Solaris/solview.html">SolView</a>.<br /><br />There's also a stunningly boneheaded bug fixed in KstatFilter. Filtering out things you don't want was matching far too much (it would match any of a module:instance:name specifier rather than the correct behaviour of matching all of them, so unix:0:kstat_types would throw away all unix kstats and all of instance 0, which wasn't what you wanted).<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-6317253672268185978?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-48897777122471851252011-04-29T13:57:00.002+01:002011-04-29T14:38:54.608+01:00As is customary with any <a href="http://www.ubuntu.com/">Ubuntu</a> release, I downloaded the recently released 11.04 and took it for a spin.<br /><br />The installation started off well. I liked the fact that it got started on the install and then asked you the configuration questions, minimizing the overall install time. And it looked like the install was galloping along. Then it bogged down, and got slower and slower. Made a coffee. Drank the coffee. Install going nowhere fast. Checked the Royal wedding pictures, played a few ZX spectrum games. Install going nowhere fast. Updated a slew of other applications on my Solaris box. Install going nowhere fast. Finally, after over an hour, it finishes the install. That was gruesome.<br /><br />After installing the guest additions for <a href="http://www.virtualbox.org/">VirtualBox</a> I got the new Unity interface. And it's not all that bad. It's not that good either, but I didn't get the horribly negative reaction to it that many have reported. In a lot of usage scenarios, I can see it working just fine.<br /><br />So, what does Unity get you? Well, it works pretty well for simple task-based computing. If you just have a handful of applications that you use, and you only use one at a time, then it works great. It allows you to focus on the task at hand. It starts to struggle if you're multitasking or if you need to do a lot of different things because once you're outside the simple task metaphor it really struggles.<br /><br />Is Unity radical? Not really. I've used dozens of desktop interfaces. So this one has panels on the left and top rather than placed elsewhere, but that's no big deal. The window controls have moved, but that's trivial. Seriously, there's nothing much new here.<br /><br />The problems I have with Unity are really that it doesn't fit my normal working style (which tends to involve having huge numbers of applications open, often many windows of the same application). The launcher on the left only really scales up to a dozen applications or so; beyond that it becomes cumbersome. It folds up whatever's at the bottom of the list in the normal view, but expands things out (forcing the launcher to scroll) if you use the folded-up applications. And you can't clean things up by moving your applications across multiple virtual desktops - you get to see all the launcher items all the time, rather than just the ones relevant to the desktop you're on.<br /><br />Dealing with multiple instances of the same application works really badly. There's no good way to organise 100 xterms, for instance. Double-click on the xterm launcher icon and they're sitting there in a 10x10 grid. But which one's which? Again, sorting them by virtual desktop doesn't really help. And right-click on the launcher icon gets you a menu, including quit - which quits all of them.<br /><br />Another major change is the move of the application toolbar to the top of the screen. I hate this for two reasons: for one, it means you have to mouse all the way across the screen, and for two, it interacts really badly with a mouse focus policy of focus-follows-mouse. Fortunately, I tend not to use application menus very much.<br /><br />None of the criticisms are necessarily aimed directly at Unity, of course. It's more that it has changed to a new usage paradigm, but all the rest of the desktop and all the applications haven't, and the mismatch shows pretty clearly. What would be really interesting to see would be what Unity would be like with applications designed around it, rather than with all the (legacy, for it) applications we have today.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-4889777712247185125?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-10212485089876718082011-02-24T22:33:00.002Z2011-02-24T22:37:44.687ZA little more tinkering, and an updated version of <a href="http://www.petertribble.co.uk/Solaris/jproc.html">JProc</a> is now available.<br /><br />The primary addition in this release is that I've implemented access to the <code>prusage</code> struct, both for processes and threads. There's a new usage subcommand that shows prusage data in a table for processes. And, if you right-click in the info or usage display, you can display a breakdown of usage by thread.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-1021248508987671808?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-29490028830928857462011-02-06T22:01:00.002Z2011-02-06T22:16:26.124ZI updated <a href="http://www.petertribble.co.uk/Solaris/kar.html">KAR</a> recently.<br /><br />This follows my recent philosophy that <a href="http://ptribble.blogspot.com/2011/01/code-bugs.html">code and bugs are the same thing</a>, although I hope that each version squishes some bugs as well as introducing them.<br /><br />Most of the bugs fixed this time were as a direct result of trying to use kar for real, generating graphs of system activity to answer question for users, developers, and management. In some cases, kar just gave up or gave the wrong data.<br /><br />(A particularly insidious bug lies in the use of printf in C. You have to specify exactly the type and size of a variable. If the type you print and the actual storage type don't match, bad things can happen. Worse, different processor architectures behave in different ways. Printing a 64-bit value as 32-bit <em>appears</em> to work on an Opteron box, but fails miserably on a SPARC.)<br /><br />One thing I'm actually doing is using kar's print subcommand to output the data, and then munging the data before feeding it into something like <a href="http://ploticus.sourceforge.net/doc/welcome.html">ploticus</a> to do the actual plotting. The inbuilt chart support works well for predefined charts, but I often want something slightly different.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-2949002883092885746?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-72192644025263382802011-01-24T21:56:00.002Z2011-01-24T22:28:02.455ZI've come to the conclusion that all code is buggy, often hideously so.<br /><br />This isn't necessarily an observation on other people's code. My code is perfect. Ha! Only it's not, of course.<br /><br />Something like JKstat is reasonably mature and (I thought) reasonably well tested. But the latest release fixes a number of pretty basic numbskull bugs that had been sitting there for ages.<br /><br />I place a lot of emphasis on code quality. I use jstyle (part of the Solaris ON build environment) to enforce a programming style. Enforcing consistency makes the code more readable, and this style is one that my eyes (and my typing fingers) are comfortable with. I use <a href="http://pmd.sourceforge.net/">PMD</a> to automatically check for basic errors and obvious code deficiencies. When I first started using PMD, it found lots of basic errors. What it doesn't catch are the doofus logic errors that my brain introduces.<br /><br />For Java programming, I found Generics to be a great benefit. Java is a strongly typed language, but a major structural part of the language - the Collections framework - didn't impose any type safety at all. Everything was just an untyped Object. Whilst Generics adds type-safety, and the enhanced for-loop often precisely expresses your intentions (as opposed to looping over an array index), I found that adding Generics made be think about the types of object I was putting in my collections, which was one step in thinking more about the structure of the code itself.<br /><br />Occasionally, I get to look at code written by others. Now, often, open source code isn't actually too bad - there's often a fixed style adopted by a community, and multiple authors can eliminate some of the more bizarre excursions into code obfuscation. But some of the code I get to see in the office truly horrifies me. Invariably without useful comments, with all the old code still in the source file but inactive, and as many different styles as we've had people work on the code (often in a language they're not entirely fluent in), just hacked in to the point where it would work with no attempt (because there's no money in this) to go back and improve maintainability of the code for the next person.<br /><br />One of the problems is the languages themselves. Java is strongly typed (I like this, because it makes simple typing errors into compile failures which are obvious), but requires massive levels of boilerplate. I dislike the use of an IDE, because I want to feel comfortable with every line of code I write. So you're wedged in to a place where just feeding the IDE, or writing boilerplate code, takes far too many brain cycles away from actually implementing real functionality.<br /><br />So change languages, you say! Well, I actually try and learn one new language (or a major area of a language) each year. So I wrote a monitoring application in tcl/tk once, then rewrote it as a webapp using JSP, then rewrote it to do graphics using Java2D, and then rewrote it in PHP, just so I could learn those languages. After several attempts, though, I have completely failed to acquire any significant competence in Python.<br /><br />Something I have been looking at recently (after an excellent talk at the local Cambridge Java Meetup group) is <a href="http://clojure.org/">Clojure</a>. You may find this a bit of an odd choice, but hacking on lisp for emacs many years ago makes it moderately familiar, and being built on the JVM means that I may be able to leverage other skills.<br /><br />Which leads to a question: what programming language (or skill) have you learnt this year?<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-7219264402526338280?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com2tag:blogger.com,1999:blog-9726833.post-61490253517333630652011-01-24T21:43:00.002Z2011-01-24T21:53:41.449ZI've been quietly working on <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a>, and have just released another update, version 0.43.<br /><br />I wasn't actually aiming to cut a release at this point, there are still a couple of things I'm working on. But I needed to get some fixes into <a href="http://www.petertribble.co.uk/Solaris/kar.html">KAR</a>, and didn't want to push out a KAR release with an unreleased version of JKstat in it, hence a synchronization release.<br /><br />Realistically, <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> has largely stabilized. So it's more a case of fixing odd bugs (the network load demo wouldn't display anything if you resized it, and guessed the network speed wrong), adding features like stepping backwards in the browser when reading KAR data, and cleaning up the javadoc. Little features like keeping all the statistics underneath a chart, so that when you add another statistic to be graphed it shows the historical data rather than starting from the current time. Nothing earth-shattering, but (hopefully) making JKstat better.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-6149025351733363065?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-20541603947944904402010-12-03T13:15:00.003Z2010-12-03T13:33:14.596ZA new version of my Java SNMP viewer <a href="http://www.petertribble.co.uk/Solaris/jangle.html">Jangle</a> is now available.<br /><br />One thing I've wanted to do is automatically build relationships. There are two tabs in the Jangle display that do this automatically, showing Siblings and Cousins.<br /><br />The Siblings tab shows attributes that share a common parent. Due to the way that SNMP works, this means different instances of the same thing. For example, if you're interested in ifInOctets, then this will have an entry for each interface - ifInOctets.1, ifInOctets.2, ifInOctets.3 and so on. So the Siblings display will put all these on one graph for you.<br /><br />What if you're just interested in all the attributes of a particular interface? Given ifInOctets.2, you want ifOutOctets.2 and the like? That's where the Cousins tab comes in - it will show (and graph) all these related attributes for you. (Of course, while Cousins is an appropriate term in the sense that the other attributes are children of your parents siblings, it's also only a specific subset of those Cousins. It would be nice to come up with a better term.)<br /><br />It's relatively trivial to build these relationships automatically - the OID tree is structured to make it possible. They also answer the questions I tend to ask most often. Building more complex relationships will likely have to be done by hand, and require domain-specific knowledge.<br /><br />This was relatively easy to code up, with just a little twist. When you select a new OID to display, it starts all the graphs from scratch again. What I also wanted to be sure of was that if the new OID was a sibling or cousin of the one already being shown, then it would keep the existing sibling or cousin graph rather than reset it.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-2054160394794490440?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-51739299748554158172010-11-22T19:23:00.003Z2010-11-22T19:55:31.814ZLast Wednesday I gave a short talk about <a href="http://www.petertribble.co.uk/Solaris/kar.html">KAR</a> at <a href="http://hub.opensolaris.org/bin/view/User+Group+losug/">LOSUG</a>.<br /><br />Apart from the fact that my laptop and the projector refused to get along, it went OK. That disagreement meant that the interactive show and tell (aka the demo) got skipped, but that was always designed as an extra. I hope I got people thinking - the real aim here is to try and make people think about what we do, rather than spoonfeed solutions to them (that's what marketing is for).<br /><br />For the talk I released a new version of KAR, and in preparing the talk and getting some things ready I stumbled across one or two issues I thought worthy of further fixes.<br /><br />Part of this is my implementation, but using <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> to parse the KAR archives has never been exactly quick. Several minutes for a full day's data from a decent server. My original implementation of graph generation involved doing that for every graph - so generating the 100+ graphs took rather a long time.<br /><br />Here follows a completely gratuitous graph: network traffic on my home machine over the course of a day.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://2.bp.blogspot.com/_ZHMLMG5bzhc/TOrIa3Qq_yI/AAAAAAAAANU/xTpQLXvHkjE/s1600/net.mac.png"><img style="display: block; margin: 0px auto 10px; text-align: center; cursor: pointer; width: 320px; height: 192px;" src="http://2.bp.blogspot.com/_ZHMLMG5bzhc/TOrIa3Qq_yI/AAAAAAAAANU/xTpQLXvHkjE/s320/net.mac.png" alt="" id="BLOGGER_PHOTO_ID_5542462655268847394" border="0" /></a><br /><br />The parser in JKstat was only partially finished, so I tidied it up a bit. It now caches parsed data, and allows consumers to step backwards as well as forwards, and rewind, and the cache is shared across multiple instances. This allows you to just parse the data once and then analyze it to your heart's content. What this means is that generating all the graphs takes only a little longer than generating the first one - making it 100 times faster.<br /><br />(Which proves the point: fixing truly stupid implementations can give you huge performance gains. There is no way I can imagine getting a hundred-fold performance improvement by tweaking the parser. That's not to say I'm happy with performance now - far from it - but it's made one particular common task viable, which it wasn't before.)<br /><br />So, a new release of both <a href="http://www.petertribble.co.uk/Solaris/kar.html">KAR</a> and <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> is now available.<br /><br />The next step for KAR is to identify additional measurements that would be generally useful, and then add them to the list of graphs generated (and maybe have specific utilities for specific data).<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-5173929974855415817?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-81934651921864361852010-10-28T22:26:00.002+01:002010-10-28T22:36:53.289+01:00Following on from earlier work, I've made available some <a href="afterstep.html">AfterStep packages</a> for Solaris.<br /><br />Then, I tried some of the twm variants. I started with <a href="http://ctwm.free.lp.se/">ctwm</a> and <a href="http://www.vtwm.org/">vtwm</a>, and then moved on to tvtwm and <a href="http://www.crynwr.com/piewm/index.html~">piewm</a>. Then I hit a problem - piewm didn't work. It was choking somewhere in the lex parser, so refusing to load any config, so no menus or anything. It's based on a older version of tvtwm and, rather than attempt to learn lex and fix that, I had a go at simply adding the pie menu support to the latest version of tvtwm (tvtwm-pl11), and that turned out to be pretty easy.<br /><br />So my merged source, boringly named ptvtwm, is <a href="http://www.petertribble.co.uk/Solaris/ptvtwm.html">available from here</a>, as are the usual Solaris packages. The packages include piewm and tvtwm derived from ptvtwm, as well as ctwm and vtwm, along with a private copy of xli so that tvtwm/piewm can load a background picture.<br /><br />That's quite a lot of Window managers, so I've put together a <a href="http://www.petertribble.co.uk/Solaris/x11-wm.html">list of package window managers</a>.<br /><br />That's essentially it for now. I'm hoping to get <a href="http://www.xfce.org/">Xfce</a> packaged up, but there are a fair number of prerequisites that Solaris doesn't have (or has antiquated versions of) so the packages end up being pretty huge. I'll get there, but haven't yet got a build I'm happy to let out.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-8193465192186436185?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com3tag:blogger.com,1999:blog-9726833.post-85482771603828648802010-10-21T15:17:00.002+01:002010-10-21T15:26:30.486+01:00A couple of window managers packaged up for Solaris this week.<br /><br />First, some <a href="http://www.petertribble.co.uk/Solaris/pekwm.html">solaris packages</a> for the <a href="http://pekwm.org/">pekwm</a> window manager. This pretty much scores a bullseye when it comes to minimalism, and is still being developed.<br /><br />Go back 20 years, and I've built some <a href="http://www.petertribble.co.uk/Solaris/awm.html">awm packages</a>. Now the Ardent Window Manager wasn't the first I ever used, but I spent about 2 years working with it, and it was pretty good. Highly flexible and customizable, in the true style of the time - you open up the config file in your favourite editor and make it do whatever you want.<br /><br />Now, I had to make a tiny number of code changes to get it to compile, but only a tiny number, and all pretty trivial. And it works just fine, even if it does have a bit of a retro look to it.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-8548277160382864880?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-89321089864721824292010-10-14T22:45:00.002+01:002010-10-14T22:55:04.927+01:00As I mentioned earlier when <a href="http://ptribble.blogspot.com/2010/10/window-maker.html">talking about Window Maker</a>, I've been hoping to find time to get some other window managers packaged up for Solaris.<br /><br />OK, so <a href="http://www.petertribble.co.uk/Solaris/e16.html">Enlightenment packages are available</a>. Just to be clear, these are e16, not some cut of the current development branch. (I'm having a look at the EFL beta, trying to get it to work on Solaris.)<br /><br />As with wmaker, these packages should install fine on Solaris 10 (some reasonably current update, anyway) and OpenSolaris/OpenIndiana. They should appear as session options on the login screen, and they should basically work. But this isn't something I use extensively, and there are almost certainly rough edges.<br /><br />The packages include imlib2, e16, libast, Eterm, and themes. They all go together, so there's just the one package. This all installs directly into /usr, which is fine as there aren't any conflicts with anything that ships with Solaris.<br /><br />While I like playing with shiny toys such as enlightenment, I never really got on with it - the behaviour was always slightly interesting, and I found the themes overly fussy, And speaking of themes, a whole bunch of <a href="http://themes.effx.us/e16">themes for e16</a> are available, and a lot of those are quite polished.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-8932108986472182429?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com4tag:blogger.com,1999:blog-9726833.post-83286707488834805272010-10-12T19:13:00.002+01:002010-10-12T19:58:19.943+01:00Recently I went along to the <a href="http://www.meetup.com/Cambridge-Java-Group/">Cambridge Java Group</a>. Enjoyable evening - would be great to get more people along, so if there's anybody else in Cambridge interested in Java and related technologies then do come along!<br /><br />I talked about <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> briefly, and I hope that some people found it interesting. One of the comments was along the lines of "hey, you could expose this with JMX and monitor it remotely". I had looked at <a href="http://java.sun.com/javase/technologies/core/mntr-mgmt/javamanagement/">JMX</a> before, and had largely ignored it because I couldn't see an easy way to use the technology. So I tried again, putting together a bean interface and implementation, and a simple server.<br /><br />I pointed jconsole at it, and it works. I can see the data I exposed there in jconsole. So, that proves that my code works. But is that it? After all, jconsole can do some quite sophisticated monitoring of a JVM, and give pretty graphs. But there seems to be a fair amount of domain-specific knowledge that has gone into that, which is hard-coded into the client. And I really don't want to write (yet another) client interface - the whole point is to leverage what is claimed to be considerable investment in JMX tooling.<br /><br />This investment must be kept locked safely away. I've had a look around, and really can't see much in the way of fully functional general-purpose JMX clients. (<a href="http://java-source.net/open-source/jmx">Here's a list.</a>)<br /><br />Of course, I've been here before. I ended up writing my own <a href="http://www.petertribble.co.uk/Solaris/jangle.html">SNMP client</a> because I couldn't find anything else that did what I wanted. I really don't want to do the same for JMX.<br /><br />So, a question: what do people use for monitoring that uses JMX? How valuable is it?<br /><br />Of course, you wouldn't actually set up a JMX server and client specifically for accessing JKstat. That would be silly. After all, JKstat comes with a fully functional client-server mode that exposes all the data you need in a reasonably efficient manner. The ideal case for using JMX would be when you had an existing system already being monitored with JMX, at which point you could simply add JKstat support to gain access to extra data without adding additional protocols or tools.<br /><br />I've also been looking at other client-server technologies. One that I'll probably do (partly because it shares some framework with JMX, partly because I know how to do it anyway) is RMI. But that, like JMX, is fairly specific to Java. It would be nice to have something that was capable of communicating with, say, python (or pretty much any scripting language). You would have thought that because I had used XML-RPC then I would be able to access it from anything else using XML-RPC, but that's not the case. I needed to use extensions to get sensible type support (like the fact that much of the data is stored as a long not an int), and those aren't commonly deployed. I was, therefore, intrigued by <a href="http://lethargy.org/~jesus/writes/why-json-sucks">Theo's rant</a> which summed up my thoughts exactly, and saved me the trouble of investigating JSON, as it's clearly not up to the task either.<br /><br />So many half-baked answers. Go on, somebody - tell me what I'm missing!<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-8328670748883480527?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-53068154282739955792010-10-02T12:08:00.002+01:002010-10-02T12:25:35.861+01:00I've always experimented with different X11 window managers and desktops. There was a time when there was actually significant choice and variety, and a lot of energy to go with it.<br /><br />I simply never got on with either SunView or OpenWindows, much preferring awm or tvtwm. And while CDE had some useful utilities, the desktop itself drove me to distraction. I appreciate the strengths of KDE, but can't get on with it either. I currently use either the Solaris JDS build of GNOME or a home-built version of XFCE as my primary desktop.<br /><br />I spend a lot of my time living inside VNC sessions, and there I run <a href="http://www.windowmaker.info/">Window Maker</a>. I've always liked Window Maker - fast, lightweight, and there were oodles of themes available. (A theme here is a desktop background and a colour scheme for menus and window decorations.) Unfortunately availability of themes is now limited.<br /><br />I've made available my own <a href="http://www.petertribble.co.uk/Solaris/wmaker.html">Window Maker Packages for Solaris</a>. These include the software and integration with either dtlogin or gdm, and will work on either Solaris 10 or OpenSolaris/OpenIndiana.<br /><br />Included in those packages are a couple of simple themes I've put together based on OpenSolaris and OpenIndiana, which you can also get directly from the above link if you already have Window Maker running.<br /><br />I also run, at times, many of the other older styles of window managers, from the basic twm derivatives up to enlightenment and AfterStep. Hopefully I'll be able to find some time to get those properly packaged up as well.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-5306815428273995579?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com1tag:blogger.com,1999:blog-9726833.post-80066365840284513462010-09-30T21:28:00.003+01:002010-09-30T21:47:04.604+01:00Actually, this was supposed to be about version 0.53 of <a href="http://www.petertribble.co.uk/Solaris/solview.html">SolView</a>, but 0.54 has the <a href="http://ptribble.blogspot.com/2010/09/jproc-05.html">fixed JProc</a> which is quite handy.<br /><br />Apart from that bugfix, <a href="http://www.petertribble.co.uk/Solaris/solview.html">SolView</a> now sports a number of useful refinements.<br /><br />The localprofile subcommand generates a jumpstart profile that will (approximately) match the current system. So on my desktop:<pre><br /># ./solview localprofile<br />cluster SUNWCXall<br />cluster SUNWCapache delete<br />cluster SUNWCapch2 delete<br />cluster SUNWCappserver delete<br />cluster SUNWCbreg delete<br />cluster SUNWCfsmgt delete<br />cluster SUNWCiq delete<br />cluster SUNWCmcdk delete<br />cluster SUNWCmip delete<br />cluster SUNWCnet delete<br />cluster SUNWCpcmc delete<br />cluster SUNWCpgadmin3 delete<br />cluster SUNWCpmgr delete<br />cluster SUNWCpostgr delete<br />cluster SUNWCpostgr-82 delete<br />cluster SUNWCpostgr-82-dev delete<br />cluster SUNWCpostgr-83 delete<br />cluster SUNWCpostgr-83-dev delete<br />cluster SUNWCpostgr-dev delete<br />cluster SUNWCrm delete<br />cluster SUNWCservicetags delete<br />cluster SUNWCsip delete<br />cluster SUNWCsmap delete<br />cluster SUNWCsmc delete<br />cluster SUNWCsppp delete<br />cluster SUNWCswup delete<br />cluster SUNWCtcat delete<br />cluster SUNWCwbem delete<br />cluster SUNWCwsdk delete<br />cluster SUNWCxmft delete<br />cluster SUNWCzebra delete<br />package SK98sol delete<br />package SKfp delete<br />package SUNWast delete<br />package SUNWcnsr delete<br />package SUNWcnsu delete<br />package SUNWfac delete<br />package SUNWfwdc delete<br />package SUNWfwdcu delete<br />package SUNWipplr delete<br />package SUNWipplu delete<br />package SUNWjato delete<br />package SUNWjatodmo delete<br />package SUNWjatodoc delete<br />package SUNWjsnmp delete<br />package SUNWluzone delete<br />package SUNWlvma delete<br />package SUNWlvmg delete<br />package SUNWmcon delete<br />package SUNWmcos delete<br />package SUNWmcosx delete<br />package SUNWmctag delete<br />package SUNWosdem delete<br />package SUNWpdas delete<br />package SUNWrmwbr delete<br />package SUNWrmwbu delete<br />package SUNWsra delete<br />package SUNWsrh delete<br />package SUNWtnamd delete<br />package SUNWtnamr delete<br />package SUNWtnetd delete<br />package SUNWtnetr delete<br />package SUNWwbpro delete<br />package SUNWwebminr delete<br />package SUNWwebminu delete<br />package SUNWxorg-compatlinks delete<br />package SUNWxwslb delete<br />package SUNWzfsgr delete<br />package SUNWzfsgu delete<br /></pre><br />Actually, there's a little bit missing from the above, which is what another new subcommand - missingpackages - will show, namely the list of packages that should be installed to meet dependency requirements but aren't:<pre><br /># ./solview missingpackages<br />missing package SUNW5xmft needed by [SUNW5xfnt, SUNWjxplt, SUNWhkfnt]<br />missing package SUNWcxmft needed by [SUNWcxfnt, SUNWcscgu, SUNWjxplt]<br />missing package SUNWfsmgtr needed by [SUNWdmgtr, SUNWdmgtu]<br />missing package SUNWfsmgtu needed by [SUNWdmgtr, SUNWdmgtu]<br />missing package SUNWjxmft needed by [SUNWjxcft, SUNWjedt, SUNWjxplt]<br />missing package SUNWkxmft needed by [SUNWkuxft, SUNWjxplt]<br />missing package SUNWluzone needed by [SUNWzoneu]<br />missing package SUNWlvma needed by [SUNWlvmr, SUNWmddr]<br />missing package SUNWpostgr-82-client needed by [SUNWpostgr-upgrade]<br />missing package SUNWpostgr-82-libs needed by [SUNWpmdbdpg, SUNWpostgr-upgrade]<br />missing package SUNWpostgr-82-server needed by [SUNWpostgr-upgrade]<br />missing package SUNWwbcor needed by [SUNWdmgtu]<br />missing package SUNWwbcou needed by [SUNWdhcsu, SUNWdmgtu]<br />missing package SUNWwbpro needed by [SUNWdmgtu]<br />missing package SUNWzlibr needed by [openofficeorg-ure]<br /></pre><br />Now, some of those are install errors on my part (I ought to have killed all the postgres packages), but some are just due to wrong dependency data.<br /><br />Continuing a bit of a theme, disks, networks interfaces, processors, packages, and clusters are sorted.<br /><br />In the jumpstart profile builder there's quite a lot more polish. Version 0.52 added automatic recursive removal of packages, but the new version fixes a number of minor display issues - the checkboxes update instantly, clusters properly reflect their status if all their constituent packages are added or removed, and the profile appears automatically when you go to the profile display tab. And if you're fed up, you can start over without exiting. All in all it's now much easier to use.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-8006636584028451346?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-19943071624726707412010-09-30T20:04:00.002+01:002010-09-30T20:47:22.940+01:00I've just released version 0.5 of <a href="http://www.petertribble.co.uk/Solaris/jproc.html">JProc</a>, which allows you to access Solaris data from java applications.<br /><br />I was hoping for a few more features, but encountered a fairly nasty bug, and it was more than worth fixing that straight away.<br /><br />Basically, you get the data from /proc by opening files in the /proc filesystem and reading data from them. I was doing the open(), but there was a distinct lack of close() associated with each one. A massive file descriptor leak ensued.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://www.petertribble.co.uk/Solaris/jproc-info.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 620px; height: 276px;" src="http://www.petertribble.co.uk/Solaris/jproc-info.png" border="0" alt="" /></a><br /><br />The one neat feature that made it was the ability to select which columns are visible in the process viewer. This involved a little fiddling with the Model underlying the JTable. Essentially, all I had to do was add an array of visible columns, each element of which maps to a real column. So I can have only some of the columns visible, and could re-order them if that's desired. The only tricky bit I had trouble with is that I have custom renderers for some of the columns (so that times and sizes are shown in human-friendly terms), and fiddling with the columns completely lost the custom renderers, forcing me to have to re-apply the renderers any time I change the structure.<br /><br />I've hidden a few of the less commonly used columns by default, which helps by making the viewer less cluttered. There's still room for improvement, especially in the area of aggregation and filtering - areas where top and prstat are so-so.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-1994307162472670741?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-26473723247117637062010-09-06T17:06:00.002+01:002010-09-06T17:14:59.877+01:00The latest release of <a href="http://www.petertribble.co.uk/Solaris/kar.html">KAR</a>, the Kstat Activity Reporter, is now available.<br /><br />This version adds a couple of extra features:<br /><br />The graphs subcommand will generate a bunch of useful png graphs for a given input file. Currently, I generate basic graphs for cpu utilization (user/kernel/idle for the whole system and for each cpu), I/O, and network statistics.<br /><br />I've also added a simple browser, which looks for kar output files and lets you load them into either the kstat browser or chart builder from <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a>.<br /><br />Oh, and in line with some of the other work I've been doing recently, most of the output is sorted correctly.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-2647372324711763706?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-36765241648617006992010-09-05T18:05:00.002+01:002010-09-05T18:22:12.834+01:00Updated versions of <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> are coming out more frequently than I would like at the moment. I think I'm done and move on to something else and find I need to add some new feature.<br /><br />Having just finished 0.40, which implemented Comparable to make sorting easier, I went through more of the code - including the demos included, and those in <a href="http://www.petertribble.co.uk/Solaris/solview.html">SolView</a> - to look at how they sorted (or, in some cases, failed to sort where they should have) their output. This led me to make a slight change to the jkstat api, which cleaned up a little of the code.<br /><br />The data stored by a Kstat is a simple key-value hash.The original implementation exposed this internal data structure - you simply got access to the internal Java <code>Map</code>. This always felt slightly ugly, exposing internal implementation details. So that's been changed. There's a new <code>Kstat.statistics()</code> method, which simply returns a <code>Set</code> of the statistic names. This is really what clients want, rather than getting the hash and getting its keys, which is what they did before. Furthermore, in almost every case you want the list of statistic names to be sorted, so rather than everyone getting the list and sorting it themselves, it's backed by a <code>TreeSet</code> and is already sorted. The original <code>getMap()</code> method is gone. Removed. That's why we're not yet at version 1.<br /><br />This version also includes a couple of samples using <a href="http://www.petertribble.co.uk/Solaris/jstripchart.html">JStripChart</a>, as shown in an <a href="http://ptribble.blogspot.com/2010/09/simple-java-strip-charts.html">earlier blog entry</a><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-3676524164861700699?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-72050606136630704272010-09-01T20:10:00.003+01:002010-09-01T20:22:39.644+01:00I've been looking for a very simple Java strip chart for a while, and haven't found one. So in the end I decided to put together something myself, the end result being <a href="http://www.petertribble.co.uk/Solaris/jstripchart.html">JStripChart</a>.<br /><br />(There's already something else out there called jstripchart. Just to confuse you, it's a python implementation.)<br /><br />Using <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> to knock something together I came up with this:<br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/_ZHMLMG5bzhc/TH6mKt4oAXI/AAAAAAAAANM/9jHpEm9UB2I/s1600/cpustrip.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 250px; height: 96px;" src="http://3.bp.blogspot.com/_ZHMLMG5bzhc/TH6mKt4oAXI/AAAAAAAAANM/9jHpEm9UB2I/s320/cpustrip.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5512025696994066802" /></a><br />Of course, you partly miss the point which is that this thing is merrily scrolling away.<br /><br />I expect to be able to use this fairly extensively in both <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> and <a href="http://www.petertribble.co.uk/Solaris/solview.html">SolView</a>, but it's certainly not tied to those in any way - it's designed to be trivially simple to use in any context.<br /><br />The API really is very simple. Just create a chart:<br /><pre><br />JStripChart jsc = new JStripChart();<br /></pre><br />which you can add to your application in the normal way as it's just a <code>JPanel</code>, and then<br /><pre><br />jsc.add(some_data);<br /></pre><br />will add the value <code>some_data</code> to the right of the chart and move everything else along. That's it. (To get it to continuously update, create a Timer loop.)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-7205060613663070427?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-16490499349219203432010-08-30T11:47:00.003+01:002010-08-30T12:01:22.871+01:00I've just released a new version of <a href="http://www.petertribble.co.uk/Solaris/solview.html">SolView</a>. This version cleans up the code used to display SVR4 packaging, and makes a couple of improvements to the <a href="http://www.petertribble.co.uk/Solaris/profile.html">Jumpstart profile builder</a>.<br /><br />Nothing earth-shattering, but I made the list of packages and clusters sorted, which makes it a lot easier to find things. And, to make that less necessary, I implemented recursive removal of packages: if you remove a package, it removes anything that depends on that package as well, keeping the package dependency graph self-consistent.<br /><br />There are various ways to make sorting work, and I had put together a couple of Comparator classes before doing it properly. The proper way is just to <code>implement Comparable</code> in the classes that need sorting. And in most cases the actual comparison is trivial - we're just comparing the name, which is just a String. (For patches it's a 2-stage numerical comparison of the patch id and revision, but still simple.)<br /><br />Once Comparable is implemented, then all I need to do is replace <code>Set</code> with <code>TreeSet</code> and everything sorts. Simple, obvious, and something I should have done much earlier.<br /><br />Of course, there's no point in learning something like this and then only using it the once. So there's a new version of <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> in which the <code>Kstat</code> class <code>implements Comparable</code>, which immediately gets the output from some of the JKstat demos sorted. (And sorted correctly - there is a specific Comparator in use here, to sort names like sd0 which contain numbers correctly: the numerical part is sorted separately and as a number, so that sd2 comes before sd10.)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-1649049934921920343?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com2tag:blogger.com,1999:blog-9726833.post-74219869056852551192010-08-21T12:14:00.002+01:002010-08-21T12:24:10.135+01:00I've just updated <a href="http://www.petertribble.co.uk/Solaris/jangle.html">Jangle</a>, my java based snmp viewer. Just a presentation tweak: the tree and list now show user-friendly names. For example, ifInOctets rather than 1.3.6.1.2.1.2.2.1.10. (Well, I find it easier to read, anyway!)<br /><br />To be useful, you actually need to be running snmp on the machine you wish to monitor. Which leads me to a question: on Solaris, do you use the system supplied snmp daemon?<br /><br />And if you do use the system one, is that just for convenience or do you use any of the extra functionality that it provides?<br /><br />I know that I simply wipe out all the supplied snmp and sma stuff and put my own build of <a href="http://net-snmp.sourceforge.net/">net-snmp</a> in its place.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-7421986905685255119?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com3tag:blogger.com,1999:blog-9726833.post-4044377755480895402010-08-20T10:06:00.003+01:002010-08-20T20:49:26.257+01:00While the world of OpenSolaris has been pretty dark and dismal over the last 6 months, a new light has appeared.<br /><br />The <a href="http://www.illumos.org/">Illumos</a> project, led by Garrett D'Amore, is bringing the OpenSolaris codebase truly into the open. Initially just the base system (ON - kernel and basic utilities), it will be free of encumbered code and capable of self-hosting.<br /><br />Illumos itself is just a foundation, not a fully-fledged system. It can be taken by others as a basis for distributions and, indeed, there are already efforts to do so. While it's still early days, it's already about as close to being capable of being used to create an independent distribution as the original OpenSolaris source was.<br /><br />The initial aim was for Illumos to be fully synchronized with the primary source from Oracle. It seems that Oracle have decided to remove themselves from the community by ceasing to make their code available. Such an ill-considered move hurts only Oracle: it frees the community from doubt, allowing it to move ahead freely, and removes any control that Oracle might have. (And it's clear that one thing that Oracle do want is control, so taking a step that eliminates their ability to control is somewhat strange.)<br /><br />Elsewhere, Oracle's policy of radio silence towards the OpenSolaris community continues. Again, Oracle are harming themselves here - turning supporters into enemies, contributors into competitors. By working with the community rather than antagonizing it, they could have turned it into an asset. Now the outside energy and support that Oracle could have tapped into is being focused into Illumos and its related projects.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-404437775548089540?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-52442343987148488292010-07-14T20:31:00.002+01:002010-07-14T21:05:47.612+01:00Unless you've been living under a rock for a while, it should be clear that the OpenSolaris Community isn't in the healthiest state. Oracle, as the new owners and sponsors of the project, have been spectacularly uncommunicative. The Governing Board have been left completely powerless, reduced to the role of spectators as Oracle withdraws behind its own barriers and what's left of the external community starts to engage in infighting.<br /><br />So, and this hasn't been an easy decision, we have resolved:<br /><br /><span style="font-weight:bold;">The OGB is keen to promote the uptake and open development of OpenSolaris and to work on behalf of the community with Oracle, as such the OGB needs Oracle to appoint a liaison by August 16, 2010, who has the authority to talk about the future of OpenSolaris and its interaction with the OpenSolaris community. Otherwise the OGB will take action at the August 23 meeting to trigger the clause in the OGB charter that will return control of the community to Oracle.</span><br /><br />Now, and sorry Ben, this isn't a "start talking to us or we'll just shot ourselves in the head" ultimatum. We're already on life support, if there are no signs of life then life support will be turned off.<br /><br />Continuing on, or waiting indefinitely, merely perpetuates a lie. All is not well, and we would all be foolish to believe that it is. To do so would simply be delusional.<br /><br />It may be nice for some to think that Oracle will ride up on a white charger to save the day. Even if that were to happen, it wouldn't really change the reality of the situation: that the current state of the community is essentially incompatible with the OpenSolaris charter and constitution, and that we need to move forward to a new organization that has new methods of governance. To get there (wherever that is) implies sweeping the current governance away. Oracle may not be prepared, but the OGB is brave enough to recognize that necessity and take the first steps. We would prefer to do so in concert with Oracle, hence the request that an official liaison be made available.<br /><br />This move should also serve as a reality check and wake up call. Nobody - either within or outside Oracle - should have any excuses for being unaware of how bad things have become.<br /><br />OpenSolaris stands at a crossroads. Many routes are open, there are choices available and decisions to be made. Simply standing still, wringing our hands and waiting indefinitely for Oracle to show up with a map, isn't an option. We need to break the shackles that tie us to this place and move forward.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-5244234398714848829?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com10tag:blogger.com,1999:blog-9726833.post-3704306983750315652010-06-24T22:09:00.002+01:002010-06-24T22:12:37.462+01:00I've been quite pleased with the way that <a href="http://www.petertribble.co.uk/Solaris/kar.html">KAR</a> has worked out, so I've released a new version with some updates. The data collector is unchanged, but I've added emulation of mpstat and fsstat, added a -z flag to iostat, and fixed a number of stupid bugs.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-370430698375031565?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-15422740950142781402010-06-02T22:00:00.005+01:002010-06-02T22:16:07.131+01:00It was my birthday recently (and boy do I feel old). It's increasingly difficult to think of decent gifts I can ask for, so I've started to ask for things to do rather than things to keep. So this year Mel bought me a flight from the <a href="http://duxford.iwm.org.uk/">Imperial War Museum at Duxford</a>, courtesy of <a href="http://www.classic-wings.co.uk/">Classic Wings</a>.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://1.bp.blogspot.com/_ZHMLMG5bzhc/TAbH02YtBCI/AAAAAAAAAMs/0K32wNyzplg/s1600/t1.jpg"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 282px; height: 211px;" src="http://1.bp.blogspot.com/_ZHMLMG5bzhc/TAbH02YtBCI/AAAAAAAAAMs/0K32wNyzplg/s320/t1.jpg" border="0" alt=""id="BLOGGER_PHOTO_ID_5478285707509040162" /></a><br /><br />This was in a 70 year old Dragon Rapide (above). We arrived nice and early, but the weather wasn't clearing as forecast (does it ever?) so we delayed for a while for the cloud to lift and had a look round the museum.<br /><br />Then we (8 of us in all) clambered in, belted up, and set off. Two immediate impressions: for one thing, the seats are very comfortable and the legroom very decent, so much better than a modern airliner in that respect; for another, the thing is loud - you can't hold a conversation, and I imagine that several hours would become quite painful.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/_ZHMLMG5bzhc/TAbI-l9UHQI/AAAAAAAAAM0/sUSeHA-wR0U/s1600/t2.jpg"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 282px; height: 211px;" src="http://3.bp.blogspot.com/_ZHMLMG5bzhc/TAbI-l9UHQI/AAAAAAAAAM0/sUSeHA-wR0U/s320/t2.jpg" border="0" alt=""id="BLOGGER_PHOTO_ID_5478286974409514242" /></a><br /><br />We were just below the clouds (a couple of times we drifted into the cloud briefly) but the view was spectacular. We flew north, round Cambridge (above) then up to Ely, did a circle, then down to Newmarket and back to Duxford (below).<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/_ZHMLMG5bzhc/TAbJr8_XkYI/AAAAAAAAAM8/qXra7Nj4GEo/s1600/t3.jpg"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 282px; height: 211px;" src="http://3.bp.blogspot.com/_ZHMLMG5bzhc/TAbJr8_XkYI/AAAAAAAAAM8/qXra7Nj4GEo/s320/t3.jpg" border="0" alt=""id="BLOGGER_PHOTO_ID_5478287753686258050" /></a><br /><br />Thanks Mel! I hugely enjoyed the experience.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-1542274095014278140?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-89348044056147170102010-05-23T13:01:00.003+01:002010-05-23T18:43:55.398+01:00I've just released an updated version of <a href="http://www.petertribble.co.uk/Solaris/kar.html">KAR</a>, the kstat activity reporter, and a matching <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> update.<br /><br />This version carries on with the original aim of simply saving all the kstats and processing them at point of use, rather than trying to predict which output might be useful at the start.<br /><br />Originally, I was using <em>kstat -p</em> output, and I've had to move on from that. There were a couple of minor issues with the <em>kstat -p</em> output that I ended up having to fix. The major one was that the times from IO kstats were converted from nanoseconds into floating-point seconds. I had a quick look to see if I could supply a modified kstat, but the conversion takes place deep inside perl - it's not just a simple presentation tweak. I could have written special-case parsing code, but it's easier and more reliable to simple generate the correct data in the first place. (Simply printing kstats is pretty trivial to code, so I did.) Having fixed that, I made a couple of other minor changes to make the output more complete (including the kstat type) and smaller and quicker to parse (eliminating all duplicates of the name of the kstat).<br /><br />The overhead of running kar is reduced. The storage requirement is halved from the first version, and the cpu for a data collection is cut from over 0.2s to about 0.01s on my machine. Which is all good - you want to minimise the perturbation on the system caused by any monitoring.<br /><br />I added a little sar output emulator. Just to prove that I could, printing out the cpu utilization like the default sar output. The more interesting one was to generate iostat output, which is what led me to create a custom data collector. Of course, now that I have a complete set of kstats to munge, most of the stat tools in Solaris could be replicated to show what they would have looked like over time (albeit at fairly low time resolution). And, once the CLI tools are exhausted, generation of graphs is next on the list.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-8934804405614717010?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com2tag:blogger.com,1999:blog-9726833.post-29914601671126804762010-05-22T19:01:00.002+01:002010-05-22T19:36:02.298+01:00This last week I've been away on a training course, the Red Hat RHCE rapid track course. It's been a long time since I did formal training (anything beyond the odd hour or so, at least), so I wasn't sure how it was going to go.<br /><br />All in all a success, I think. I certainly learnt a lot, and passed the exam.<br /><br />As anyone who has done the RHCE exam knows, I can't really talk about any of the details. But I think the following would be helpful to others like myself:<ul><br /><li>If you're experienced in unix systems administration, and are used to administering applications, you should cope. Easily.</li><br /><li>With experience on other platforms, the rapid-track course is very useful.</li><br /><li>The RHCE exam covers a lot of application ground. For me, it was really a case of learning how Red Hat has its own tweaks (and how things like the firewall and SELinux interact with applications).</li><br /><li>I found that going away for the course was really helpful. It allowed me to focus on the course without distractions.</li><br /></ul><br />Red Hat do a transition course for Solaris Administrators. I thought about that, and skipped it. I had no problem adjusting to RHEL, so I'm not sure what value the course for Solaris admins would be - I would have thought that if you're a bit unsure, then doing the full RHCE course rather that the Solaris course with the fast-track would be a better bet.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-2991460167112680476?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com6tag:blogger.com,1999:blog-9726833.post-33178265493437282122010-05-09T15:47:00.002+01:002010-05-09T16:10:15.289+01:00So <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> now reaches version 0.37, with the usual spread of minor fixes, enhancements, and new features.<br /><br />One enhancement is the extension to 64-bit. For JKstat itself, there's no need to run it in 64-bit mode, but I need 64-bit libraries to allow JKstat to run inside a 64-bit JVM. One example of this would be <a href="http://www.petertribble.co.uk/Solaris/solview.html">SolView</a>, which would have to run in 64-bit mode to show the sizes of 64-bit processes using <a href="http://www.petertribble.co.uk/Solaris/jproc.html">JProc</a>.<br /><br />A new feature is the ability to generate png images directly from the cli, using input such as the <code>kstat -p</code> archives used by <a href="http://www.petertribble.co.uk/Solaris/kar.html">kar</a>.<br /><br />One of the minor fixes this time is to support the recent 1.3 release of <a href="http://javafx.com/">JavaFX </a>. It looks like JavaFX isn't binary compatible between releases, so code needs to be rebuilt to match whichever version you're using, which is a shame. Also, this version optimized away some of my method calls and I needed to fool it into not doing so.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-3317826549343728212?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-63470024406532493172010-05-05T21:11:00.003+01:002010-05-05T21:30:25.935+01:00I've got a java interface, called <a href="http://www.petertribble.co.uk/Solaris/jproc.html">jproc</a> to process data in Solaris, using the procfs filesystem.<br /><br />In the latest version, apart from starting on a tree-view (currently known to be buggy and woefully incomplete), there are a couple of little technical tricks I had to learn.<br /><br />The first is that accessing the process data, particularly sizes, of a 64-bit application, requires a 64-bit application. That's why tools such as top, ps, and prstat are 64-bit (via isaexec). Now, there's a 64-bit java for Solaris, so I needed to compile my JNI library in 64-bit mode too. Normally you just call<br /><pre><br />cc -G<br /></pre>but for 64-bit mode it's a little more complicated than that. In particular, just adding <code>-m64</code> the way you would expect doesn't work. And it's different on x86 and sparc. So what I've found works, is:<br /><pre><br />amd64:<br /> cc -Kpic -shared -m64 <br /><br />sparc:<br /> cc -xcode=pic13 -shared -m64<br /></pre><br /><br />The other thing I wanted to do was to make the display of items in tables a little more readable. JTable just picks up the type of data and defaults to a fairly basic display. My first attempt was to convert my data to pretty strings, and display those, but that had a couple of snags: by default strings get left-justified, which wasn't what I wanted, and sorting broke because it sorted the strings rather than the underlying numerical data.<br /><br />The answer, of course, is to use a custom TableCellRenderer. This only affects the presentation, so that sorting works correctly against the underlying data. So far all I've done is simply humanize some of the values, but so much more is possible.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-6347002440653249317?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com1tag:blogger.com,1999:blog-9726833.post-91804379806093178222010-03-14T21:57:00.002Z2010-03-14T22:32:46.832ZThe old standby for recording historical system activity is sar - system activity reporter. There are many alternatives, both free and commercial, but sar has the advantage that it comes with the OS, and pretty much any version of any (unix-like) OS.<br /><br />Because it's there, we use sar, saving it's output into a big archive and using tools like <a href="http://www.trickytools.com/php/sar2rrd.php">sar2rrd</a> to produce charts. (It's not the only thing we use, of course.)<br /><br />The problem is that, particularly on Solaris, sar is terrible. The data it collects is woefully incomplete - network data is the worst, being completely absent, but there's much more missing. Some of what is present is aggregated away so that much of the details is lost. And the list of what's present is fixed, so the whole framework is completely non-extensible.<br /><br />So, I'm fed up with that, and need to do better. Note that most tools out there don't help with capturing all the data, as they have their own preconceived notions of what data might be useful (although they are generally far more complete than sar).<br /><br />Enter <a href="http://www.petertribble.co.uk/Solaris/kar.html">kar - the kstat activity reporter</a>. This is really amazingly simple. Given that (almost) all the performance data you want is obtained from kstats, simply save all the kstats on a regular basis. The implementation I have here is to save <code>kstat -p</code> output into files inside zip archives. Now, that's not ideal, but it has some advantages: it's almost zero effort, it gives complete coverage, and it's naturally extensible. If it works out and is found to be useful, more optimal mechanisms could be defined.<br /><br />I've said it a couple of times above, but I'm going to say it again: the key advantage here is that the data is complete and thereby naturally extensible. I don't want to enhance sar by trying to cherry-pick interesting statistics (and we could all argue for months about what might go on the list). By saving everything you automatically pick up anything new that's added. And you let consumers decide which of the statistics are interesting when you get to the post-processing phase. Say I wanted to look at the historical behaviour of the zfs ARC - no problem, it's all there in the kstats.<br /><br />Using <code>kstat -p</code> is a convenient shortcut, but does have other advantages. Because the output is textual, all your favourite analysis tools - awk, sed, perl, grep, python, whatever - can munge the data with no effort. And you can chuck the data into your graphing application of choice.<br /><br />If that wasn't enough, <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">jkstat 0.35</a> has support for reading in the output of kar in both the browser and chart builder.<br /><pre>./jkstat browser -z /var/adm/ka/ka-2010-03-01.zip</pre><br />or<br /><pre>./jkstat chartbuilder -z /var/adm/ka/ka-2010-03-01.zip</pre><br />will do the trick.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-9180437980609317822?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com1tag:blogger.com,1999:blog-9726833.post-78145182989636962672010-03-05T22:14:00.002Z2010-03-05T22:24:12.404ZI've just pushed out a minor update to <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a>.<br /><br />The change here (apart from a couple of minor bugfixes and a jnetloadfx example to remind me what JavaFX looked like) is the addition of a class to update multiple accessories together. Previously, in demos such as iostat and cpustate, each item had its own timer loop and was responsible for handling its own updates. This was especially apparent in the kmemalloc example in SolView - it was obvious that separate widgets weren't being updated simultaneously.<br /><br />Now I can just have a single update loop that updates multiple accessories. Not only does it look neater, but there are noticeable improvements in memory and cpu usage from only having one timer instead of many.<br /><br />Coming up next is more related work. The ability to read historical <code>kstat -p</code> output works fine, but requires some changes so that you step through the data rather than continuously updating in time. (If you think about it for a moment, the class I mentioned above is one example of updating the time and then telling the world to update, so it's - albeit only tangentially - related.) These changes are likely to be a bit complex, so I also decided to cut a version before starting to make more significant changes to the code.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-7814518298963696267?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-20313667253211851652010-02-26T21:23:00.002Z2010-02-26T23:40:10.442ZThe <a href="http://hub.opensolaris.org/bin/view/Main/">OpenSolaris</a> Annual Meeting is under way on IRC. (The meeting itself is one of those odd side-effects of the old constitution, it's never really been used as a proper meeting.)<br /><br />We were fortunate enough today to have Dan Roberts along to answer questions. A <a href="http://www.spcoast.com/irclogs/opensolaris-meeting/index.php?date=2010-02-26">log</a> of the session is available - for the conversation with DanR look just after the noon mark.<br /><br />Positive, simple and straightforward:<br /><br /><blockquote>Oracle will continue to make OpenSolaris available as open source, and Oracle will continue to actively support and participate in the community</blockquote><br /><br />And:<br /><br /><blockquote>Oracle will also continue to deliver OpenSolaris releases, including the upcoming OpenSolaris 2010.03 release.</blockquote><br /><br />In terms of investment:<br /><br /><blockquote>Oracle is investing more in Solaris than Sun did prior to the acquisition, and will continue to contribute technologies to OpenSolaris, as Oracle already does for many other open source projects</blockquote><br /><br />Questioned on open development:<br /><br /><blockquote>Oracle will continue to develop technologies in the open, as we do today</blockquote><br /><br />with the caveat:<br /><br /><blockquote>There may be some things we choose not to open source going forward, similar to how MySQL manages certain value add at the top of the stack.</blockquote><br /><br />(Although the model that came to my mind was the fishworks/analytics add-ons that go into the open storage systems. So that's not a change.)<br /><br />In reply to a question "for 'regular' users and contributor to Open Solaris - do you figure that anything will change in terms of how open solaris is delivered and how developers contribute?":<br /><br /><blockquote>Love to see the tech journal crowd participating! And yes, regular users will find things mostly unchanged. Contributors also.</blockquote><br /><br />On the subject of User Groups, there's already been some outreach (as I've noted before). Part of the story here is that Oracle have a group for community support, so that group will take over the support that has come from Sun in the past. User groups still look after themselves, but wil be centrally supported (if they want and need it) through the new route. I also expect some user groups to form relationships with other organisations such as the independent Oracle User Groups, and all user groups will be free to take whatever steps they wish to in that area.<br /><br />Support is an area that has been contentious recently. On this:<br /><br /><blockquote>And Oracle will ensure customers running OpenSolaris have an option for support on Oracle Sun Systems where it's required, though given the very little sales here this will not be something we expect many customers to deploy going forward. Solaris is our focus, on both SPARC and x86.</blockquote><br /><br />(Read it this way: if anybody wanted support, they should have paid up. I've heard what "very little sales" means. My one concern in this area is support for Solaris/OpenSolaris on 3rd-party systems.)<br /><br />All in all, though, thanks to Dan for sticking his head above the parapet. And - while there are clearly devils in the details - it's clear than Oracle plan to keep pushing OpenSolaris forward, so rumours of its death have been greatly exaggerated.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-2031366725321185165?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com11tag:blogger.com,1999:blog-9726833.post-55057425032820281352010-02-19T20:51:00.003Z2010-02-19T20:59:50.583ZThe OpenSolaris Governing Board is now accepting nominations for candidates to run for the OGB term starting April 2010. Anyone registered with an account on the OpenSolaris.org website is eligible to be nominated, even if not currently recognized as a Core Contributor or Contributor to any existing community group.<br /><br />Nominations should be e-mailed to ogb-discuss@opensolaris.org (please make sure you either use the <a href="http://opensolaris.org/jive/forum.jspa?forumID=206">web forum</a> or subscribe to the mailing list first to avoid getting caught in the spam-filter/moderation queue - use the forums or sign up at <a href="http://mail.opensolaris.org/mailman/listinfo/ogb-discuss">http://mail.opensolaris.org/mailman/listinfo/ogb-discuss</a>)<br /><br />Nominations must submitted by a current Core Contributor - those who are interested in running may nominate themselves if they are a Core Contributor already (see the current lists of Core Contributors on <a href="https://auth.opensolaris.org/">https://auth.opensolaris.org</a>), or send mail to the mailing list asking for a Core Contributor to formally nominate them if they are not a Core Contributor.<br /><br />Those who are nominated by others must send mail to ogb-discuss accepting their nomination.<br /><br />All candidates will be required to submit a statement before the election containing "a list of their commercial affiliation, or other interests related to OpenSolaris, so that a voting member can understand the context from which they would act on the OGB and the likely biases they would bring."<br /><br />The deadline for submitting and accepting nominations will be 23:59 in the US/Pacific timezone on Monday, March 1. Statements from nominees are due before the start of voting, which is scheduled to begin at 00:00 on Monday, March 8.<br /><br />Nominations and acceptances will be tracked during the nomination period using the OpenSolaris bugzilla at <a href="http://defect.opensolaris.org/">http://defect.opensolaris.org/</a> in the "ogb" product under the "nominations" category.<br /><br />More information about the OpenSolaris elections and governance can be found at:<br /><ul><br /><li><a href="http://hub.opensolaris.org/bin/view/Community+Group+ogb/2010">http://hub.opensolaris.org/bin/view/Community+Group+ogb/2010</a><br /><li><a href="http://hub.opensolaris.org/bin/view/Community+Group+ogb/">http://hub.opensolaris.org/bin/view/Community+Group+ogb/</a><br /></ul><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-5505742503282028135?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-19669721720087523192010-02-16T21:43:00.003Z2010-02-16T22:56:00.652ZAfter a prolonged wait, <a href="http://www.oracle.com/">Oracle</a> have now completed their <a href="http://www.oracle.com/us/sun/index.htm">takeover of Sun</a>. Late January, they <a href="http://www.oracle.com/us/sun/044498">presented their plans</a> for taking Sun's products forward.<br /><br /><a href="http://www.opensolaris.org/">OpenSolaris</a> wasn't even mentioned.If you look carefully, it's on a slide, but that's about it.<br /><br />That silence has continued. OpenSolaris has - publicly at least - been completely ignored by Oracle. It's as if we don't exist.<br /><br />Somewhat perturbed by this state of affairs, I asked for a communication channel to be established between Oracle and the <a href="http://hub.opensolaris.org/bin/view/Community+Group+ogb/">OGB</a>. This was ignored. The OGB pointed out (in pretty strong terms) to those Sun staff that we do have communication with that some level of contact was needed. We haven't heard back.<br /><br />It's not as if we're asking for much; the very basic start of a conversation. I've no doubt that Oracle are very busy, but not even bothering to say hello?<br /><br />Recently, Oracle sent a welcome letter to (some of) the OpenSolaris User Group leaders. While any contact is welcome, the manner in which it was done was unfortunate. It ignored OpenSolaris as a whole, ignored all the other groups that make up the OpenSolaris community, and completely bypassed the Governing Board. This is shockingly poor treatment.<br /><br />I've attempted to make contact. Again, this approach has been completely ignored. I have not even been accorded the common courtesy of an acknowledgment.<br /><br />I'm not the only one. It's clear that the OpenSolaris community are very concerned, and they're being left completely in the dark. <a href="http://www.cuddletech.com/">Ben</a> blogged an <a href="http://www.cuddletech.com/blog/pivot/entry.php?id=1108">Open Letter to Oracle</a> and that doesn't seem to have elicited any response either.<br /><br />At a most basic level, this is simply impolite. Even downright rude. Is it a sign of something more sinister? Well, nothing at all has been said that would allow anyone ta make a judgment, but it's fuelling the FUD machine.<br /><br />Needless to say, this is pretty inauspicious start to a relationship. It's about time for Oracle to stop avoiding us and make their intentions known, especially with the OpenSolaris Governing Board Elections just round the corner.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-1966972172008752319?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com12tag:blogger.com,1999:blog-9726833.post-30167444686376734902010-02-14T22:17:00.002Z2010-02-14T22:30:17.060ZWith the end of this year's OGB term coming up, I just sent out the following request for reports to various mailing lists. Historically, we've not been very good at reporting what we're doing in the OpenSolaris community, and really need to do better.<br /><br />There is a lot of work taking place in the <a href="http://www.opensolaris.org/">OpenSolaris</a> community. In order to inform the <a href="http://hub.opensolaris.org/bin/view/Community+Group+ogb/">OGB</a>, the rest of the community, and those outside our community of all the work that's being done, the OGB seeks to collect reports from the communities, projects, and user groups that make up our community.<br /><br />For the 2009-2010 period, we seek brief (presentation style, one-page would be ideal but don't force it if this doesn't fit) reports from all community groups, projects, and user groups, highlighting their achievements. We aim to combine them into a single presentation that could be used by others. If there is additional material - video or images, or full presentations, we would like to know about that too.<br /><br />Some templates you may wish to use are available at:<br /><br /><a href="http://hub.opensolaris.org/bin/view/Community+Group+ogb/Reports">http://hub.opensolaris.org/bin/view/Community+Group+ogb/Reports</a><br /><br />Some suggested details that could be included:<br /><br /><a href-"http://hub.opensolaris.org/bin/view/Main/communities">Communities</a>: projects that have been newly sponsored or completed, and events or meetings that have occurred<br /><br /><a href="http://hub.opensolaris.org/bin/view/Main/projects">Projects</a>: aims, new features, build number if the project has integrated<br /><br /><a href="http://hub.opensolaris.org/bin/view/Main/usergroups">User Groups</a>: number of attendees, frequency of meetings, involvement with other groups or events<br /><br />Please reply directly to me by email.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-3016744468637673490?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-48599354665845157542010-01-31T14:33:00.002Z2010-01-31T15:28:13.240ZThanks for the comments on my <a href="http://ptribble.blogspot.com/2010/01/looking-for-better-terminal.html">search for a better terminal</a>.<br /><br />First. <a href="http://www.eterm.org/">Eterm</a>. So if I build it with gcc, it SEGVs on me, and it won't compile with Studio 12. I used to use it, but don't seem to have a working version available right now.<br /><br />As for providing a binary for <a href="http://www.calno.com/evilvte/">evilvte</a>, I'm not sure how useful that is. The only way to configure it is to build from source. And I'm pretty sure my configuration choices are likely to be different from anyone else's.<br /><br />On to terminator. (Which one? <a href="http://software.jessies.org/terminator/">This one</a>, or <a href="http://www.tenshu.net/terminator/">that one</a>?) The first one fails to build for me - not only does it require a non-standard make, it requires a particular version of that non-standard make, and one my regular Solaris 10 box doesn't have. The second one just gives a python traceback.<br /><br />Then there's the whole <a href="http://sourceforge.net/projects/rxvt">rxvt</a> family. I actually use rxvt quite a bit, when I want to run something (like top) in a throwaway terminal window, due to how lightweight it is. But it has to be said that it's nowhere near as lightweight as it was, and urxvt is much heavier than xterm. There are little irritants too, such urxvt needs its own terminfo entry adding, which is just another barrier. Or not being able to override rxvt's uses of a stipple pattern in the xterm-style scrollbar (I always make it solid).<br /><br />I've tried quite a few others, most of which plain don't build.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-4859935466584515754?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com6tag:blogger.com,1999:blog-9726833.post-83741233165744079012010-01-30T17:51:00.003Z2010-01-30T18:08:47.057ZIt sometimes seems I spend most of my life in xterm, usually logged into a remote server.<br /><br />And while xterm isn't bad, as terminal emulators go, sometimes I want a bit more jazz and reliability. Recently xterm hasn't been as reliable as it should be, and it also doesn't cut it for some use cases - logged into a Sun (oops, Oracle...) ILOM, for instance. Things like color, background images, transparency, all break up the monotony,<br /><br />Locally on my work desktop I run gnome-terminal for some stuff - it's heavy resource consumption is offset by the fact that I'm running a lot of windows. But it doesn't work half as well on remote servers or under other graphical environments. And gnome-terminal is a resource hog.<br /><br />At home I run <a href="http://www.xfce.org/">Xfce</a>, and the <a href="http://www.xfce.org/projects/terminal/">Terminal</a> is just as functional as gnome-terminal but with only half the resource requirements. One showstopper (and this is a real killer for a large number of otherwise decent applications) is its requirement for D-Bus, which makes it far too difficult to run standalone in an arbitrary environment.<br /><br />So I've been looking at some alternatives, and quite like <a href="http://www.calno.com/evilvte/">evilvte</a>. It's functionally equivalent to gnome-terminal and the Xfce Terminal, using the same vte widget, but even lighter weight and faster and with much reduced dependencies. (In particular, no D-Bus.) So it could be used as a replacement for xterm.<br /><br />Building it is somewhat manual. (But no <a href="http://ptribble.blogspot.com/2010/01/stupid-build-systems.html">stupid build system</a> to fight with.) And configuration is a case of editing config.h and running make again to rebuild the binary. But I like it.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-8374123316574407901?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com5tag:blogger.com,1999:blog-9726833.post-63831740303717806902010-01-29T20:26:00.002Z2010-01-29T20:39:08.441ZThere was a time when building software was easy. You typed <code>make</code> and it almost always worked. Sometimes you had to know the secret incantation of <code>xmkmf -a</code>, but usually if make didn't work straight off the software wasn't worth using.<br /><br />In the advanced days of the 21st century, of course, such simplicity - something that actually works - is rarely to be found. We have things like the autotools (the evil behind ./configure) that basically involves making a bunch of unsubstantiated guesses about your system and constructing a random build configuration based on it. And heaven help you if libtool gets its teeth into your software - it's almost guaranteed to miscompile your software in a manner that's undebuggable and unfixable.<br /><br />So cmake promised to be a welcome relief from this madness. Only it's not. I've been having a go at building the <a href="http://awesome.naquadah.org/">awesome window manager</a> on Solaris. When it works I'll post more details, but I almost flipped when having fought it into submission and built it successfully, I did the install and saw:<pre><br />-- Removed runtime path from "/packages/awesome/bin/awesome"<br /></pre><br />And, on checking, it had done exactly that - stripped out the RPATH information so the binary it had carefully built stood no chance whatsoever of actually working.<br /><br />And this is progress?<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-6383174030371780690?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com5tag:blogger.com,1999:blog-9726833.post-65297528503886716592010-01-27T16:05:00.003Z2010-01-27T16:14:42.850ZI've been migrating some old systems from truly antique Sun boxes onto a small number of new Sun T5240s. Lots of zones, one per new physical system.<br /><br />This includes some truly antique versions of oracle.<br /><br />In order to keep the version police happy with the compatibility matrix, they wanted Solaris 8. Despite the fact that it works fine with Solaris 10, they insisted on Solaris 8. Enter Solaris 8 Containers.<br /><br />Now, Solaris 10 has reasonable default shared memory settings. However, the Solaris 8 Container gives a faithful emulation of a Solaris 8 system, including miserly shared memory settings. How to set better values, because the normal Solaris 10 games don't apply?<br /><br />The solution is simple, so simple that it never actually occurred to me. Simply put the settings you want in the Container's /etc/system file in the traditional way, and reboot the Container.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-6529752850388671659?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-44639132898351510962010-01-26T13:25:00.002Z2010-01-26T13:38:23.336ZYou can preconfigure a Solaris Zone by placing a valid sysidcfg file into /etc/sysidcfg in the zone, so it finds all the information when it boots.<br /><br />What's also true, at least for native zones on Solaris 10, is that you can specify a cut-down sysidcfg file. For example:<br /><pre><br />network_interface=PRIMARY {<br /> hostname=mars<br />}<br />system_locale=C<br />timezone=US/Pacific<br />name_service=NIS {<br /> domain_name=foo.com<br /> name_server=bar.foo.com(192.168.1.1)<br />}<br />security_policy=NONE<br />terminal=xterm<br />root_password=0123456789abcd<br />nfs4_domain=foo.com<br /></pre><br />If you try doing this on a Solaris 8 Container, it will go interactive. So you need to supply a bit more information. Such as:<br /><pre><br />network_interface=PRIMARY {<br /> hostname=mars<br /> netmask=255.255.255.0<br /> protocol_ipv6=no<br />}<br />system_locale=C<br />timezone=US/Pacific<br />name_service=NIS {<br /> domain_name=foo.com<br /> name_server=bar.foo.com(192.168.1.1)<br />}<br />timeserver=localhost<br />security_policy=NONE<br />terminal=xterm<br />root_password=0123456789abcd<br /></pre><br />where I've added the netmask and protocol_ipv6 settings to the network_interface section, and told it a timeserver. Clearly the Solaris 10 zone can work these out for itself, but a Solaris 8 branded zone needs to be told explicitly. (Also, it doesn't need the nfs4_domain, as that makes no sense for a Solaris 8 system.)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-4463913289835151096?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-91418957642098711612010-01-23T15:19:00.002Z2010-01-23T15:50:03.177ZBackup solutions such as NetBackup keep a catalog so that you can find a file that you need to get back. Clearly, with a fair amount of data being backed up, the catalog can become very large. I'm just sizing a replacement backup system, and catalog size (and the storage for it) is a significant part of the problem.<br /><br />One way that NetBackup deals with this is to compress old catalogs. On my Solaris server it seems to use the old compress command, which gives you about 3-fold compression by the looks of it: 3.7G goes to 1.2G, for example.<br /><br />However, there's a problem: in order to read an old catalog (in order to find something from an old backup) it has to be uncompressed. There's quite a delay while this happens, and even worse, you need disk space to handle the uncompressed catalog.<br /><br />Playing about the other day, I wondered about using filesystem compression rather than application compression, with ZFS in mind. So, for that 3.7G sample:<br /><br /><table border=2><tr><th>Compression</th><th>Size</th></tr><tr><td>Application</td><td>1.2G</td></tr><tr><td>ZFS default</td><td>1.4G</td></tr><tr><td>ZFS gzip-1</td><td>920M</td></tr><tr><td>ZFS gzip-9</td><td>939M</td></tr></table><br /><br />Even with the ZFS default, we're doing almost as well. With gzip, we do much better. (And it's odd that gzip-9 does worse than gzip-1.)<br /><br />However, even though the default level of compression doesn't compress the data quite as well as the application does, it's still <b>much</b> better to use ZFS to do the compression, as then you can compress all the data: if you leave it to the application then you always leave the recent data uncompressed for easy access, and only compress the old stuff. So assume a catalog twice the size above, and that we used NetBackup to compress half the catalog, then the disk used in the application case would be 3.7G uncompressed and 1.2G compressed. The total disk usage comes out as:<br /><br /><table border=2><tr><th>Compression</th><th>Size</th></tr><tr><td>Application</td><td>4.9G</td></tr><tr><td>ZFS default</td><td>2.8G</td></tr><tr><td>ZFS gzip-1</td><td>1.8G</td></tr><tr><td>ZFS gzip-9</td><td>1.8G</td></tr></table><br /><br />The conclusion is pretty clear: forget about getting NetBackup to compress its catalog, and get ZFS (or any other compressing filesystem) to do the job instead.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-9141895764209871161?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com3tag:blogger.com,1999:blog-9726833.post-45910873791624817432009-12-04T22:36:00.003Z2009-12-04T23:01:26.863ZThere are a number of features that have been suggested for <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> that I've been thinking of including, and I've just added one of the more interesting ones.<br /><br />I think it was Phil Harman suggested this at <a href="http://hub.opensolaris.org/bin/view/User+Group+losug/">LOSUG</a>. Someone may have archived kstat -p output, so why not allow JKstat to read that?<br /><br />So, as of version 0.33, <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> can do exactly that. You can either save kstat -p output into files in a directory, or simply put those files into a zipfile. (The reader is pretty dumb right now - no other files or subdirectories.) It uses the timestamps on the files as the time the measurement was take,<br /><br />The browser can handle the input, but needs a bit more work to be useful - it currently steps through the data in real time, which isn't ideal for what could be days of data sampled at multi-minute intervals. Ideally it would be able to step through the data, pause, and rewind. Next release, perhaps.<br /><br />The chart and areachart subcommands can read in data and display a chart. For example, the cpu behaviour of my home desktop yesterday evening, from kstat -p output saved every 5 minutes, could be shown with:<pre><br />./jkstat areachart -z /var/adm/ka/ka-2009-12-03.zip \<br />cpu_stat:: user kernel idle<br /></pre> which looks like:<br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://4.bp.blogspot.com/_ZHMLMG5bzhc/SxmRy-w5iSI/AAAAAAAAAK0/BT76jfutDzA/s1600-h/Screenshot.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 219px;" src="http://4.bp.blogspot.com/_ZHMLMG5bzhc/SxmRy-w5iSI/AAAAAAAAAK0/BT76jfutDzA/s320/Screenshot.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5411516732289943842" /></a><br /><br />(Yes, there's an anomaly on the first datapoint.)<br /><br />This is a preliminary implementation, so there are plenty of rough edges in need of a bit of polish, but it proves that the idea of reading archival kstat data is viable.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-4591087379162481743?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-66578715775697144862009-11-25T12:21:00.004Z2009-11-25T13:55:44.938ZMy desktop workstation running Solaris 10 has almost 1600 packages installed. Many of the development systems I use have something close to 1000 packages installed. My Ubuntu (8.04, to be exact) desktop install fresh off the CD has just over 1100.<br /><br />Given 1600 packages, can a sysadmin manage that, or even concisely describe what is installed and what its function is? I suspect not. It's actually so bad that I suspect most of us don't even bother.<br /><br />Tools don't really help here. If anything, by giving the illusion of ease of use, they encourage the growth in the number of packages, making the underlying problem worse.<br /><br />Really, though, is managing <em>packages</em> interesting? I submit that it's not, and that looking at the problem as one of managing packages is completely the wrong question.<br /><br />Instead, we should be managing applications at a higher level of abstraction. Rather than managing the 250 packages that comprise Gnome, we need to collapse that into a single item, which we can then open up into manageable chunks. A package may be the right level of granularity for a packaging system, but it's the wrong level of granularity for an administrator.<br /><br />We should be thinking of applications and function, not the detail of how packages deliver that functionality. I want to be able to go to a system and ask "What do you do?" and have it come back and say "I'm a web proxy server and mail relay"; I don't want to sift through 500 packages and try to work out which of them are relevant. If I want to set up a wiki for collaborative document editing that authenticates against my Active Directory infrastructure, then I want to phrase the requirement in those terms rather than try to work out the list of components that I need for that task.<br /><br />From this, the packaging details become uninteresting. What a package contains, the packaging software, package names - are less important because that's just internal implementation detail.<br /><br />The old Solaris installer had some of this - it defined clusters and metaclusters. The implementation doesn't really help much - the definition of the contents of clusters and metaclusters was poor, and there was no support for the clusters once you were managing an already installed system. Also, what you really want is a system that allows for clusters to be structured hierarchically (so you could take something like Gnome, and either manage it as a single unit, or have the option of dealing with subunits like games or libraries), and to overlap (for example, you could imagine that the apache web server would be in a whole lot of clusters).<br /><br />One might be tempted to construct packages that use dependency information to bring in the packages they need. This approach is flawed: it doesn't cleanly separate groups from packages; it doesn't allow you to omit subgroups; and it makes removal of a group excessively difficult. Software clusters need to be cleanly layered above packages in order to allow each layer to best meet its own requirements.<br /><br />Beyond simply delivering files (via packages) a cluster could also contain the details of how to actually take the various bits and pieces and put them together into a functioning system. In fact, the whole area of application configuration is one desperately in need of more attention.<br /><br />A quick summary, then: package management shouldn't be interesting, and we need to move forward to managing applications and configuration.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-6657871577569714486?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com5tag:blogger.com,1999:blog-9726833.post-3870780276678734332009-11-17T16:48:00.002Z2009-11-17T17:00:14.201ZThe <a href="http://www.google.com/">Google home page</a> has seen some changes of late. One of these is the removal of the "I'm feeling lucky" button.<br /><br />One of the things I've noticed over the past few months is that searching online has become dramatically worse. Google is increasingly failing to find useful results, and when it does find something it will rather present you with multiple instances of the same thing (the same news article syndicated to different sources, for example) rather than the more useful list of independent answers. Commonly I end up going to subsequent search pages, and often don't get to anything useful at all.<br /><br />Is Google losing its touch? Has dumb search had its day? Perhaps the "feeling lucky" option was removed because it almost never works any more.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-387078027667873433?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com3tag:blogger.com,1999:blog-9726833.post-45372231237354454942009-11-16T19:52:00.004Z2009-11-16T20:52:18.975ZSo, as might be predicted I suppose, some people wilfully disregarded the thrust of my argument, and turned it into a debate over specific packaging technologies.<br /><br />OK, so that brings us to the question: what is so bad about SVR4 packaging, really?<br /><br />I could go on for pages. One of the reasons I found OpenSolaris attractive was the prospect of being able to fix all the bad things with Installation and Packaging in Solaris. Let's takes some of the comments though:<br /><br /><b>old and clunky</b><br /><br />Guilty as charged. Really, it is old. It is clunky. It's been neglected and unloved. It needs fixing. Those reasons alone aren't enough to dismiss it - the key question should be whether it can actually do the job.<br /><br /><b>missing lots of features</b><br /><br />OK, so what features does dpkg have that SVR4 packaging doesn't? That's really the comparison - versus the dpkg or rpm commands. <br /><br /><b>enabler of dim-sum patching</b><br /><br />Completely and utterly false. The problem with Solaris patching lies fairly and squarely in the domain of patching. This could trivially be solved without any changes to tools - either deliver whole packages, or simply institute a policy saying that you can't deliver changes to a package (or related set of packages) in independent patches. This is a process problem, not something inherent to the underlying package system.<br /><br /><hr /><br /><br />Then there are more material objections:<br /><br /><b>no repository support</b><br /><br />Actually, SVR4 packaging does have the ability to fetch and install packages from remote locations. (A crippling limitation of IPS is that it can't do anything else.) What's wrong with this picture is the lack of repositories - blastwave aside. Wouldn't it have been easier to simply make existing packages available on a web site, without having to retool everything?<br /><br /><b>lack of dependency resolution</b><br /><br />As the success of dpkg/apt demonstrates, having your underlying packager do all the work is neither necessary nor desirable. What that <b>does</b> demonstrate is the requirement for more powerful tools above the base packager. Actually, separating the fancy interface from the tools doing the low-level work is probably a really good thing - it enables compatibility of the system over time even if the higher-level tools change, and it enables innovation by allowing independent components to be developed independently.<br /><br /><b>arbitrary scripting</b><br /><br />Actually, one of the key weaknesses of SVR4 packaging is not that it supports scripting, but that the support it offers is pretty poor. there is no real support - there ought to be a strong scripting framework with a well-defined environment, and predefined functionality for common tasks. Oh, and trigger scripts would be nice. Banning scripting (yet allowing it to sneak in through the back door) fails to solve the problem, and encourages bad workarounds.<br /><br /><b>poor package names</b><br /><br />The actual package names are pretty much immaterial. Assigning real significance to them would be false. What matters is that they're unique and follow a scheme. As a user, you might want to install "php" - all that means is that the software studies the package metadata and works out what packages to really install. Actually having a pcakge with that name isn't necessary, and probably not even desirable (it locks you into current names and prevents evolution).<br /><br /><hr /><br /><br />So, beyond a recognition in the codebase that the 21st century has arrived, and the lack of an apt-get/synaptic style front-0end - all of which could fairly easily be remedied - what is really wrong with SVR4 packaging?<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-4537223123735445494?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com18tag:blogger.com,1999:blog-9726833.post-32531456544109615302009-11-13T20:32:00.003Z2009-11-13T20:39:16.869ZFriday the 13th, and I wonder whether to hold off for a day.<br /><br />But no, another <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> release comes out.<br /><br />Nothing major here, still the continuing cleanup process. The one change here is better exception handling, particularly in Client-Server mode. It's not perfect, by any means, but in the past I just dropped errors and exceptions straight in the bin.<br /><br />The reason for doing so was that I wasn't keen on declaring Exceptions to be thrown - it seriously clutters up the whole API. But Fabrice Bacchella (thanks!) pointed out the obvious, that if I were to throw a RuntimeException then I wouldn't have to declare that I threw it. So the fix is to create a subclass of RuntimeException and throw that, and consumers that need to know can check for that (and can check for the specific failure - RuntimeException is far too generic).<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-3253145654410961530?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-41424707994041536342009-11-10T10:33:00.002Z2009-11-10T10:48:17.797ZThe announcement to EOL Solaris Express Community Edition (SXCE) was telegraphed well in advance, and we're coming to the end of the road with only a handful of planned releases left to look forward to.<br /><br />But, is this just the end of the road for SXCE, or is it something bigger that's at stake here?<br /><br />Read the Sun marketing and you might believe this is a glorious new dawn for the Solaris/OpenSolaris world. The reality may be more like sailing off into the sunset and disappearing from view.<br /><br />The fundamental difference between the old and the new is instalation and packaging, which have been ripped out wholesale and been incompatibly replaced. Even if the replacements had been perfect (and, quite frankly, they fall a huge distance short) this would have been a huge challenge. Organisations (and individuals) are under huge pressure to retrench and consolidate. Adding additional technologies that they're expected to support is an uphill battle. Adding brand new (and essentially untested) technologies that they're going to have to learn from scratch makes it doubly hard.<br /><br />If the next version of Solaris had been based around SXCE, with traditional deployment technologies - traditional packaging and jumpstart - then customers would have been able to start rolling it out tomorrow. Everything a customer knew, all their existing investment in skills and tools, would be preserved. New customers would be able to leverage the skills and expertise of existing customers. All the great features and functionality present in OpenSolaris would be there to be taken advantage of.<br /><br />Contrast that with the planned OpenSolaris transition. You have to retrain all your staff, replace your entire toolset, rebuild your entire systems deployment and administration infrastructure. Most environments are heterogeneous, so this means you now have an entire extra set of infrastructure to support - you aren't going to transition everything to the new scheme immediately, so you're going to have to shoulder the burden of supporting the extra scheme in parallel for years. Isn't the most likely course of action for a cash-strapped IT<br />department with a CIO breathing down their neck to simply reject that and migrate everything over to RHEL?<br /><br />Solaris and OpenSolaris contain fantastic technologies that make them a great choice for IT departments - ZFS, Crossbow, CIFS support, zones (especially sparse root zones), Dtrace, and many more - but by making deployment such an unattractive proposition we're making it far less likely that customers will try or use these technologies, and are giving organizations and managers every excuse to ignore Solaris and OpenSolaris as an option.<br /><br />The best thing Oracle could do for Solaris and OpenSolaris would be to scrap the OpenSolaris distribution (but not the rest of OpenSolaris) and redirect our energies into building a better Solaris based on SXCE. If not, then I fear that Solaris will ride off into the sunset and be consigned to the wastebasket of superior technologies that failed due to bad strategic decisions, and that's a prospect that truly saddens me.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-4142470799404153634?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com14tag:blogger.com,1999:blog-9726833.post-18629326900805767502009-11-08T21:41:00.002Z2009-11-08T22:02:44.223ZTake one look at the <a href="http://hub.opensolaris.org/bin/view/Project+svr4_packaging/">SVR4 packaging</a> <a href="http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/cmd/svr4pkg/">source</a> in Solaris/OpenSolaris and it's clear that it's suffering from serious neglect. It's old. It's fragile. It's had stuff bolted onto the side over and over until it's a wonder that it works at all.<br /><br />Yet, it's survived for the best part of two decades and every Solaris system uses it.<br /><br />I recently had to fix up some packages on Solaris 10 that were in the deadly embrace of webstart. If you tried to remove them with prodreg, it said "<span style="font-style:italic;">oh no, you must use pkgrm</span>", yet when you tried to use pkgrm it said "<span style="font-style:italic;">oh no, you must use prodreg</span>".<br /><br />So, we have access to the source, right? And, while I could have manually gone in and wiped out the errant packages by hand, I had a look at the SVR4 source to see if I could put together a version that actually did what it was told.<br /><br />This was pretty easy. It took a little effort to get enough together that would compile cleanly on Solaris 10 again. And, having done that, and solved the initial problem, I did a bit more tinkering to remove some of the more obviously redundant code and apply some of the performance improvements that are sorely needed.<br /><br />After thinking up a meaningless acronym, I've made the code I have available here: <a href="http://www.petertribble.co.uk/Solaris/sprate.html">SPRATE</a>. You're free to use it; anyone feeling masochistic enough to work on it is free to contact me.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-1862932690080576750?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-13355040707953538602009-10-21T12:48:00.002+01:002009-10-21T13:01:59.796+01:00Buried in a far recess of some organisations you'll find some really old systems. I've just acquired a new playground - not a Solaris 10 box in sight. What I do have is a lot of much older stuff, going back to:<br /><pre><br />SunOS jawdropped 5.5.1 Generic_103640-29 sun4u sparc SUNW,Ultra-1<br />SunOS gobsmacked 5.6 Generic_105181-39 sun4m sparc SUNW,SPARCstation-10<br /></pre><br />Ouch! I thought I had long seen the last of such hardware or Solaris versions.<br /><br />Now, Solaris has <b>excellent</b> binary compatibility. I have some old SunOS 4 binaries from the 1980s that I use on a daily basis. So I'm expecting to be able to take a lot of what's present and simply move it onto new systems.<br /><br />I'm also planning on using <a href="http://www.sun.com/software/solaris/containers/">Solaris Containers</a> to help make the transition easier. I would much prefer to do a proper move of the application to a newer system, but that takes time. The advantage of the Containers approach is that you can lift up a working system lock, stock, and barrel, and migrate the whole lot in one fell swoop - otherwise it can be quite a lot of effort to cleanly extract an application from an old (and usually undocumented) system.<br /><br />Unfortunately there isn't a Containers implementation for Solaris 7, 2.6, or 2.5.1. So those have to be done the hard way.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-1335504070795353860?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com1tag:blogger.com,1999:blog-9726833.post-39323682421046175362009-10-20T10:52:00.002+01:002009-10-20T18:53:28.558+01:00Solaris has PICL - the Platform Information and Control Library - as a mechanism for storing and retrieving platform-specific information. It's structured as a tree, so a natural thought would be to load the information into java and use a swing JTree to present a graphical representation. (And would make a nice add-on to <a href="http://www.petertribble.co.uk/Solaris/solview.html">SolView</a>.)<br /><br />I wrote such a tool several years ago. It uses JNI to interface with the native libpicl, and it never worked properly. It still doesn't work properly - it crashes far more often than not. The crashes are consistent in the sense that if it crashes on a machine it will always crash in the same place (retrieving the same object from the picl tree), but are random in the sense that apparently nonsensical changes to the code will either kill or cure it.<br /><br />Anyway, I'm just tossing it out there. It's called <a href="http://www.petertribble.co.uk/Solaris/piclbrowser.html">piclbrowser</a>, and is <a href="http://www.petertribble.co.uk/Solaris/piclbrowser.html">available here</a>. Anyone's free to take this and play with it, and is welcome to try and fix it.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-3932368242104617536?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-72432365262427697892009-10-19T22:42:00.002+01:002009-10-19T23:20:09.948+01:00I've had a home broadband connection from Virgin - formerly NTL - at home for years. Until recently, the quality of the service was excellent - I could almost always get the full quoted speed, and we only had the occasional minor outage.<br /><br />Recently, though, the connection has been terrible. It's been several weeks, almost a month now. I'm supposed to get 10M/s (that's 10 megabits). Trying the various speed tests that are available - including the ones recommended by Virgin themselves - and I get 5% at best, usually rather less. And I suspect that those aren't telling the full picture, as the actual rate I'm getting in practice is worse. Web pages can take minutes to load. Watching TV programmes or gaming is simply a non-starter. I tried downloading something an hour ago and it's not even managed to get a single byte yet. Packet loss is 10% and up. It's not usage or time dependent.<br /><br />(Fortunately, gmail seems to remain moderately responsive, but pretty much anything else is essentially down. Attempting to actually work from home is a disaster.)<br /><br />Virgin Media's online support is awful. For starters, their web form is completely incapable of working out who you are even if you're logged in - so you have to fill in all sorts of account details. The first one I sent was acknowledged and then ignored. The next one got a standardized answer. I did what was asked, got given some more things to try, and did those too. (My connection is sufficiently poor that some of the tests don't even work.) The no response in over a week. No response to my following up.<br /><br />So I tried the telephone. Oh dear. After the usual rigmarole of giving them every single piece of information they already know we go through the standardized script. Have I rebooted the modem? Tried that, several times. Have I rebooted my computer? Yes. Have I checked for spyware? Yes. Am I running XP or Vista? Well, I have both, and they're slow as well, but they can't even spell Solaris. Can I run a speed test? I try that, and get almost 5% of what I'm paying for, which is actually better than most of the times I've tried.<br /><br />The poor chap then has the temerity to tell me that my broadband connection is working perfectly, and that I need to call the PC helpline who'll help me fix the problem with my computer. I think I managed to stay civilized throughout the resulting exchange.<br /><br />It's possible that there's a problem somewhere in the house, of course. I've tried to exclude that, and can't, because the test required to do that - direct connection of a machine into the modem - simply fails completely. I regard that as an interesting data point, but I'm only a sysadmin not a call-centre automaton reading from a script.<br /><br />Faced with a combination of a terrible broadband connection, and useless customer service, I investigated alternative providers. Which leaves me stuck between a rock and a hard place, because my house must be a long way from the nearest exchange, so that using the BT line (which is the only other connectivity option) would take me down to 3M if I'm lucky. And I really do need the 10M I'm paying for.<br /><br />Grrrr.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-7243236526242769789?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com3tag:blogger.com,1999:blog-9726833.post-5341434480434253652009-10-18T19:20:00.002+01:002009-10-18T19:32:29.627+01:00I've just uploaded <a href="http://www.petertribble.co.uk/Solaris/jangle.html">Jangle 0.05</a>, a minor update to my graphical java snmp client.<br /><br />It's not just graphical now; I've added an snmpwalk subcommand to walk the snmp tree from the command line.<br /><br />There's also the normal clutch of bug fixes and general code improvements.<br /><br />One thing I've been playing with is trying to automate discovery of relationships. The snmp information is stored in a tree, but the last two nodes in the tree are inverted. Consider ifInOctets as an example: it has a child node for each interface, as does ifOutOctets. So just looking at ifInOctets gives you a list of interfaces, whereas what you really want is to generate a list of interfaces and then get all the data for that interface. You can get this by parsing MIB files, but I would like to be able to do this just by looking for patterns in the OID tree, and then look at the MIB afterwards to describe the structure. (One thing I want to avoid is having to ship all the MIBs for every possible device.)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-534143448043425365?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com3tag:blogger.com,1999:blog-9726833.post-47243812129282426892009-10-18T14:52:00.002+01:002009-10-18T15:30:02.265+01:00The <a href="http://www.opensolaris.org/">OpenSolaris</a> <a href="http://www.opensolaris.org/os/community/ogb/governance/">Constitution</a> is known to have a number of flaws. It's not well matched to the way the community actually works; it conflates local functional roles with global governance roles; and it's relatively long and complex.<br /><br />At the last annual election, a revised constitution was proposed that aimed to eliminate those shortcomings - greatly simplified and separating governance from operations. It failed to pass. (Only narrowly, but it failed.)<br /><br />The current <a href="http://www.opensolaris.org/os/community/ogb/">OpenSolaris Governing Board</a> decided early in its term of office that we would again offer a revised constitution to the OpenSolaris community, after taking an opportunity to correct any errors or issues that community members might be concerned about.<br /><br />As <a href="http://mail.opensolaris.org/pipermail/ogb-discuss/2009-October/006659.html">announced</a>, revisions to the constitution are now <a href="http://www.co-ment.net/text/1506/">available for comment</a>, and we invite all interested OpenSolaris community members to send us their comments. (Reviewing the <a href="http://wiki.genunix.org/wiki/index.php/OGB_2009/009_ReleaseNotes">release notes</a> would be a sensible starting point.)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-4724381212928242689?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-83428462347555703392009-10-04T11:42:00.002+01:002009-10-04T11:49:18.902+01:00It looks like my <a href="http://ptribble.blogspot.com/2009/09/solview-047.html">SolView 0.47</a> announcement was a tad premature. Testing was somewhat incomplete.<br /><br />Version 0.48 fixes the problem, and is now available.<br /><br />(The technical goof I made is like this: the solview wrapper script sets things like the LD_LIBRARY_PATH and CLASSPATH. However, if you just run solview with no arguments it pulls the CLASSPATH out of the jar file manifest instead. I had updated the wrapper correctly, but forgotten the manifest. I normally run the individual views in isolation, so hadn't noticed that one of the jar files was missing, resulting in the process view generating a stack trace rather than the useful output it was supposed to.)<br /><br />So, to anyone who grabbed 0.47, you'll want to get 0.48. Sorry about that.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-8342846234755570339?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-35093197066110671162009-09-26T17:34:00.002+01:002009-09-26T17:46:51.884+01:00I've been doing a little more work on <a href="http://www.petertribble.co.uk/Solaris/jproc.html">JProc</a> - a java jni interface to the Solaris procfs filesystem - to make it a little more complete and robust.<br /><br />This has reached a convenient stopping point, so I've released version 0.2.<br /><br />There are a couple of features that have been lifted from <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a>, namely the idea (and parts of the implementation thereof) that it's useful to manage a Set of processes as a unit, and that it's useful to be able to filter on process characteristics.<br /><br />As a start, I've added filtering by user and zone. The demo application now has dropdown lists of users and zones, so you can select processes owned by a user, or running in a specific zone (or both) interactively. (They don't currently update as the list of users and zones changes - that's work left for the future.)<br /><br />I've also added lookups of username, group, and zone name. This is a simple jni interface to <code>getpwnam</code> and friends.<br /><br />One other feature I added was to convert a project id to a name. This proved rather more entertaining, as I discovered that the lookup functions are buried off in libproject rather than libc. While that of itself isn't necessarily an issue, libproject drags in quite a lot of subsidiary libraries. While in normal use they use lazy loading so that you don't drag in the extra libraries until you need them, it looks like jni drags them all in regardless. I'm not sure how useful project name/id lookups are going to be, so I may simply remove that bit of functionality.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-3509319706611067116?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-62032426564621167452009-09-26T17:31:00.003+01:002009-10-03T16:32:35.665+01:00One of the things I've been trying to do with <a href="http://www.petertribble.co.uk/Solaris/solview.html">SolView</a> is to provide a user with multiple views of a Solaris system from all angles. I'm not going to claim I've succeeded, but I've released another SolView update with a couple of new features.<br /><br />The first is to add charts from <a href="jkstat.html">JKstat</a> that show behaviour over time - visible in the processor and network view. I've put quite a bit into the chart support in JKstat, and was pleased that it was so easy to use it to add charts to SolView. (It's not entirely perfect - it would be nice if the colors in the cpu chart matched the cpustate widget, for example. And it's a shame that I don't collect history so that each time you look at a chart it starts from scratch.)<br /><br />The second is a Process view from <a href="jproc.html">JProc</a>. Which is what I wrote JProc for in the first place.<br /><br />Apart from that there's the usual raft of bugfixes, and there's a little more polish. Enjoy!<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-6203242656462116745?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com3tag:blogger.com,1999:blog-9726833.post-38470759989495522522009-09-13T21:03:00.004+01:002009-09-13T21:32:00.625+01:00In the latest update to <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> I've introduced a new piece of functionality I've been wanting to get done for a while.<br /><br /><a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> is pretty good at producing charts, but - until now - you've either had to take what it gives you or delve into the code. No longer. As of version 0.30, if you<pre><br />./jkstat chartbuilder<br /></pre>you'll get a window just like this:<br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/_ZHMLMG5bzhc/Sq1RPKIjIXI/AAAAAAAAAKM/RIA7BOANY6M/s1600-h/jk-cb.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 280px;" src="http://3.bp.blogspot.com/_ZHMLMG5bzhc/Sq1RPKIjIXI/AAAAAAAAAKM/RIA7BOANY6M/s320/jk-cb.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5381046450637709682" /></a><br /><br />A kstat can be selected from the tree; you can then choose the statistic or statistics of interest, and then get a chart of either the rates of change or the values of the chosen statistic(s).<br /><br />You can also choose whether to produce a line chart of the various statistics, or a stacked chart, with statistics in solid color stacked atop each other.<br /><br />You can then choose whether to just show this individual instance of a kstat, or to show all the available instances (so you can, for example, produce a chart showing all the disks or all the cpus, just by picking one).<br /><br />And you can decide whether to show the individual kstats, or whether to show an aggregate over all the kstats.<br /><br />So, the following chart is generated by selecting cpu:0:sys from the tree, checking all instances, and asking for an aggregated, stacked chart of the cpu_nsec_idle, cpu_nsec_user, and cpu_nsec_kernel statistics, showing the relative proportions of user, system, and idle time on my system.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/_ZHMLMG5bzhc/Sq1VxfaqmcI/AAAAAAAAAKU/lf9qNHicfsY/s1600-h/jk-chart.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 219px;" src="http://3.bp.blogspot.com/_ZHMLMG5bzhc/Sq1VxfaqmcI/AAAAAAAAAKU/lf9qNHicfsY/s320/jk-chart.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5381051438512904642" /></a><br /><br />(If you're interested, the spike of user activity at about 21:19 is a build running; the huge splodge of kernel time just after 21:27 is VirtualBox starting up a VM.)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-3847075998949552252?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-8879967010183510342009-09-08T19:28:00.002+01:002009-09-08T19:59:23.339+01:00Virtualization is one of those hot IT topics, but the reality is that - in the general sense - virtualization has been around for what seems like forever.<br /><br />I define virtualization as the abstraction of logical resources away from physical resources.<br /><br />For example, NFS (the Network File System) allows you to access your data from any system, not just the one that happens to be physically connected to the disks on which the data is stored.<br /><br />The X Window System allows you to display applications on a different host to the one where they might be running.<br /><br />VNC (Virtual Network Computing) is one of many technologies that allow you to access a complete desktop on a different system to that where it's running.<br /><br />The various types of Virtual Desktops supported by X11 window managers allow you to manage and display applications independently of the physical screen that you may be sitting in front of.<br /><br />Solaris Zones allow you to construct an operating system instance that, while in any particular instantiation is locked to a piece of hardware, is logically distinct. It's relatively trivial to pack up a Zone, redeploy it on new hardware, and the application layer doesn't notice. Again, a level of abstraction from the physical.<br /><br />Interestingly, the current virtualization fad doesn't quite work the way you would expect based on the definition I started with. Looking at the various virtual machine technologies currently in vogue, while the virtual machines are abstract from the underlying hardware, the abstraction that's presented to the user is that of physical systems. This doesn't introduce anything new, it just gives us the same old IT infrastructure we've always had, just replicated one layer up in the stack. Useful, but not revolutionary. <br /><br />And the cloud doesn't really change that, because all you're getting with a cloud is an extra layer (and a whole new charging model) underneath your virtualization layer. Everything may be virtual, but in many cases all we're doing is finding new ways to implement the physical.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-887996701018351034?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-69283437750334016322009-09-02T16:37:00.002+01:002009-09-02T16:44:03.807+01:00<a href="http://www.xfce.org/">XFCE</a> has a little <a href="http://goodies.xfce.org/projects/applications/xfce4-taskmanager">task manager</a> application. It's not top or prstat, but it's reasonably lightweight and integrated.<br /><br />(It also allows you to click on a column heading to sort. That's one of the things that <a href="http://www.petertribble.co.uk/Solaris/jproc.html">JProc</a> does as well and that you take for granted in a graphical application.)<br /><br />The original task manager was linux-specific, and the way it wandered through /proc was never going to work on Solaris. But it was easy enough to produce a Solaris version that seems to work well enough for me. Source <a href="http://www.petertribble.co.uk/Solaris/xfce4-taskmanager-0.3.2-solaris.tar.bz2">here</a>.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-6928343775033401632?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com1tag:blogger.com,1999:blog-9726833.post-74782226352176816752009-08-17T23:12:00.002+01:002009-08-17T23:24:18.244+01:00As if accessing kstats from Java wasn't enough, I've recently been playing with accessing process information - specifically the /proc filesystem on Solaris - from java. Thus was born <a href="http://www.petertribble.co.uk/Solaris/jproc.html">JProc</a>.<br /><br />The idea, eventually, is to link this with <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> and <a href="http://www.petertribble.co.uk/Solaris/solview.html">SolView</a> to give a more complete view of what's happening on a Solaris system.<br /><br />The obligatory screenshot:<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://www.petertribble.co.uk/Solaris/jproc-info.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 620px; height: 250px;" src="http://www.petertribble.co.uk/Solaris/jproc-info.png" border="0" alt="" /></a><br /><br />I think the correct description here is "work in progress".<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-7478222635217681675?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-43265724310397991462009-07-22T20:47:00.003+01:002009-07-22T21:09:57.154+01:00In gnome-terminal on Solaris 10, if you hit the PgUp key, it scrolls the window up. The PgDn, Home, and End keys similarly move the scrollbar.<br /><br />That's the behaviour I want. It's also what you've traditionally had with xterm, dtterm, and the like.<br /><br />Recently, it's become fashionable to pass PgUp and friends through to the application, so you get the application running inside the terminal emulator to do its own interpretation, and require the shift key to be pressed in order to get the terminal window itself to scroll.<br /><br />I find that incredibly disruptive. I don't run anything inside a terminal that does its own scrolling (other than more, and I don't need to reach as far as the PgUp key to manipulate that). Most of the more advanced applications I run are graphical anyway.<br /><br />So, how to fix? In my <a href="http://ptribble.blogspot.com/2009/07/xfce-on-solaris-10.html">exploration of xfce</a>, the <a href="http://goodies.xfce.org/projects/applications/terminal/">Terminal</a> application uses a newer version of vte which suffers from the above problem. So I set out to fix it, and it was relatively simple.<br /><br />I used vte-0.20.5, and if you look at around line 5055 of vte.c, you'll see some code that looks like:<pre><br /> case GDK_KP_Page_Up:<br /> case GDK_Page_Up:<br /> if (modifiers & GDK_SHIFT_MASK) {<br /> vte_terminal_scroll_pages(terminal, -1);<br /> scrolled = TRUE;<br /> handled = TRUE;<br /> suppress_meta_esc = TRUE;<br /> }<br /> break;<br /></pre><br />simply remove the test for the shift:<pre><br /> case GDK_KP_Page_Up:<br /> case GDK_Page_Up:<br /> vte_terminal_scroll_pages(terminal, -1);<br /> scrolled = TRUE;<br /> handled = TRUE;<br /> suppress_meta_esc = TRUE;<br /> break;<br /></pre><br />for all four keys of interest, and rebuild.<br /><br />My life is so much better as a result.<br /><br />It's a shame that's not the default, but there are others who rely on the opposite behaviour. It would be nice if it was customizable - it's OK for me to build a separate copy for my own use on my own machine, but that's not very flexible. It would be neat if it could autodetect the terminal mode and do the right thing.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-4326572431039799146?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-51369680195617651452009-07-22T20:38:00.002+01:002009-07-22T20:46:44.274+01:00I'm getting somewhat unsatisfied with the various desktop environments available. The desktop is currently in the doldrums, presenting a dull and dreary landscape.<br /><br />So, I thought I would give <a href="http://www.xfce.org/">XFCE</a> another look. OK, so it turns out that it is a little bit of work to get it running on Solaris 10. There's a bunch of prerequisites that you need to build first, and a couple of tweaks. So it was a bit tedious, but I got it to work fairly easily. My <a href="http://www.petertribble.co.uk/Solaris/xfce-install.html">install notes</a> are available to help others down the same path.<br /><br />What I've got is essentially functional. I've not worked out why some of the menu and application icons don't appear, and a few of the window manager themes don't quite work (you get strange visual artifacts) but it's functional, meets my requirements (which may be different from other people) of being easy to use, has a reasonable choice of themes, isn't quite as heavyweight as some of the other desktops, and shows promise for the future.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-5136968019561765145?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com3tag:blogger.com,1999:blog-9726833.post-65423171204712765062009-07-10T10:59:00.004+01:002009-07-10T11:12:42.790+01:00One of my servers (it's a continuous build server) generates a significant amount of logfiles. About 150G so far, growing at over a gigabyte each day.<br /><br />Now, logfiles tend to be compressible, so in preparation for migrating it to a new host (it's a zone, so moving from one physical host to another is trivial) I tested ZFS compression to see how well that would work.<br /><table border="1" align="left"><br /><tbody><tr><th align="left">Type</th><th align="left">Ratio</th></tr><br /><tr><td>lzjb</td><td>5.08x</td></tr><br /><tr><td>gzip</td><td>22.19x</td></tr><br /><tr><td>gzip-1</td><td>16.37x</td></tr><br /></tbody></table><br /><p><br />Now, the default lzjb gives you fair compression. That saves 80% of my storage, so that's pretty good.<br /><br />Going to gzip compression the space saving is spectacular. Unfortunately the machine runs like it's stuck in treacle, so that's not exactly practical.<br /><br />Which is why I tried gzip-1. You save much more space than lzjb, although not as much as the default gzip (which is gzip-6). Unfortunately, while the machine doesn't quite go as comatose as it does with the regular gzip compression, the performance impact is still noticeable.<br /><br />Overall, while I wouldn't mind the extra compression, the performance impact makes it hard, so I'm going to stick with the regular lzjb.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-6542317120471276506?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com6tag:blogger.com,1999:blog-9726833.post-31866414477736730582009-06-14T21:14:00.003+01:002009-06-14T21:38:42.659+01:00One of the announcements out of CommunityOne was the availability of <a href="http://www.javafx.com/">JavaFX</a> for Solaris and OpenSolaris - if you're lucky enough to be running on x86, anyway.<br /><br />So the first thing that occurred to me was - could I use JavaFX to make a graphical front-end to <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a>?<br /><br />(And the same thought occurred to <a href="http://www.cuddletech.com/">Ben Rockwood</a> as well, who asked me how easy it would be and pushed the thought from idle fancy to a must try, especially after I glibly said - "sure, dead easy".)<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/_ZHMLMG5bzhc/SjVcSEIvrDI/AAAAAAAAAI8/OJ8Xo21pznw/s1600-h/jcpustatefx.gif"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 283px; height: 90px;" src="http://3.bp.blogspot.com/_ZHMLMG5bzhc/SjVcSEIvrDI/AAAAAAAAAI8/OJ8Xo21pznw/s320/jcpustatefx.gif" border="0" alt=""id="BLOGGER_PHOTO_ID_5347281597990153266" /></a><br /><br />So the above image is jcpustate using the JKstat core api rendered with JavaFX. Of itself, it's nothing fancy, but it's served its purpose: I've got a rudimentary understanding of JavaFX out of it; I've worked out how to integrate JKstat amd JavaFX; and this validates that the JKstat core api does its job.<br /><br />You can now download <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat 0.29</a>. Look in the jkstatfx directory, the source code for the above example is JCpuStateFX.fx.<br /><br />I had a couple of issues getting this to work. I found I had to put the jni shared library (<code>libkstat_jni.so</code>) directly into the right javafx lib directory - setting LD_LIBRARY_PATH didn't seem to work. The other was that JavaFX doesn't natively support either java Collections or Generics. It clearly understands them - because it tracks the types of the objects in my Collections correctly (and it doesn't understand how to cast objects back toi the right type either). I ended up having to use the toarray(T[] a) method to create a Sequence of the right type.<br /><br />I have to say that this wasn't the feature I had originally planned for this version of JKstat. I've been working on an interactive chart builder so you're not limited to the fixed and limited set of charts that JKstat creates on its own, but that will have to wait a while for the next release.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-3186641447773673058?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com5tag:blogger.com,1999:blog-9726833.post-74334187567924237472009-05-29T20:45:00.002+01:002009-05-29T21:00:09.353+01:00Need to start packing as I'm off to <a href="http://developers.sun.com/events/communityone/">CommunityOne</a>.<br /><br />It's going to be a busy week. Starting off, Sunday I'll be at the <a href="http://wikis.sun.com/display/OpenSolaris/Open+HA+Cluster+Summit+May+2009">Open HA Cluster Summit</a>, on Monday is <a href="http://wikis.sun.com/display/OpenSolaris/OpenSolaris+at+C1+West+-+Schedule">CommunityOne</a>, and the <a href="http://www.opensolaris.org/os/community/ogb/">OGB</a> will be out in force. If you're around, there's an OGB Town Hall at 6pm, before the OpenSolaris party.<br /><br />Tuesday there's more CommunityOne, including the <a href="http://wikis.sun.com/display/OpenSolaris/OpenSolaris+at+C1+West+-+Schedule#OpenSolarisatC1West-Schedule-FREETuesday%2CJune2">Deep Dives</a> and the <a href="http://wikis.sun.com/display/OpenSolaris/Crossbow+BoF+at+CommunityOne+West">Crossbow BOF</a>, then on Wednesday I'm hoping to meet up with the OSUG Bootcamp.<br /><br />And, perhaps the most important thing about getting to events like this, I hope to meet as many people as possible in the few days I'm there.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-7433418756792423747?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-7175138605021335592009-05-15T12:32:00.002+01:002009-05-15T12:44:50.367+01:00The OGB will be at <a href="http://developers.sun.com/events/communityone/">CommunityOne</a> in a few weeks. If you would like to get involved, there's still time (just) to register - <a href="http://www.opensolaris.org/os/">OpenSolaris</a> will have a big presence!<br /><br />On the Monday, June 1st, we're holding a Town Hall at 6pm, and we invite any and all members of the OpenSolaris community to come along, meet us, talk to us, give us feedback about OpenSolaris and how we the OGB can help in making the OpenSolaris community better.<br /><br />(It's somewhat unfortunate that there's something else going on at 6pm that involves free alcohol or something like that. Hard choice, huh?)<br /><br />As part of the OGB presence at CommunityOne, we're looking to <a href="http://www.opensolaris.org/os/community/ogb/Reports/">display reports</a> from communities, projects, and user groups that make up our community. What I'm thinking of here is lots of individual highlight slides we can put on display boards (so they ought to be self-contained on a single page). Bullet points, and pictures make it look better. I'll also put them all together into a single presentation (we might have that running in a permanent loop on a screen, for example) that anyone can take and use.<br /><br />Longer term, I see this as a first run of a regular reporting process. The OpenSolaris community is doing great work, and we ought to all be aware of the great work that's going on and highlighting our achievements so they're more widely known.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-717513860502133559?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-88268472672499067612009-04-18T13:17:00.004+01:002009-04-18T16:26:40.126+01:00It's a great honour to be elected to the <a href="http://www.opensolaris.org/os/community/ogb/">OGB</a>, and I want to thank everyone who took the trouble to vote.<br /><br />So, what is the OGB going to do this year?<br /><br />Let's start by looking at the <a href="http://www.opensolaris.org/os/community/ogb/charter/">Charter</a>. This defines what the OGB has to do. And if you look at that, then the job of the OGB - above all else - is to construct and maintain the OpenSolaris <a href="http://www.opensolaris.org/os/community/ogb/governance/">Constitution</a>. Much of the Charter is about the Constitution, and there's also quite a bit about the structure of the OGB itself (and a lot of that is the initial bootstrapping of the OGB when the OpenSolaris community was created and we started with the CAB).<br /><br />Past that, the OGB then manages the community according to the Constitution. There are clear problems with the current Constitution, which is why a revised version was proposed. Having failed to pass, we're going to have to do something, because the current Constitution is a obstacle that's getting in the way of developing the community, but how we deal with that is another matter.<br /><br />Note that the OGB is quite constrained in its operations. Both the Charter and the Constitution are reasonably specific about what we are supposed to do. In particular, it's pretty explicitly clear that the OpenSolaris we govern is <b>not</b> <a href="http://www.opensolaris.com/">OpenSolaris the Sun distribution</a>, nor is it OpenSolaris the <a href="http://src.opensolaris.org/">codebase</a>, it's just the OpenSolaris community.<br /><br />However, I'm going to interpret the Charter slightly more broadly. At the beginning, it says <em>to manage and direct an OpenSolaris community in its efforts to improve upon and advocate in favor of OpenSolaris</em>. There are two phrases there that are key.<br /><br />The first is <em>manage and direct</em> - and clearly we the OGB cannot in fact direct any member of the community to do anything. What we can do is put into place management processes that will allow the community to do its work more easily; it's up to the community to use them. One of the things we have talked about (it's come up before, but I think we want to push it again) is some level of reporting. We should expect every structural part of the community to produce regular status reports, so that we know what's going on. At the very least, that a project or community is still alive. Beyond that, these reports would promote wider awareness of what each group is doing, and would also give groups an opportunity to make bottlenecks known so that they can be acted upon. There are problems that are well known already, and we need to make sure that progress in those areas is maintained - and is more visible so that people don't think the problems are being ignored.<br /><br />The second key phrase is <em>advocate</em>, and I would like to see the OGB more prominent as advocates and champions of OpenSolaris. On a personal note, this is difficult for a quiet reserved Englishman such as myself, but it needs doing. And at least some of the other OGB members (I'm not going to say all, because I certainly can't presume I can speak for them) feel that the OGB should be taking a more active role. If we don't, then call us to account.<br /><br />Going back to the title - and given the limitations on the real power that the OGB has - what can the OGB do for you?<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-8826847267249906761?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com1tag:blogger.com,1999:blog-9726833.post-91016370843376305972009-04-17T17:14:00.002+01:002009-04-17T17:21:28.892+01:00I'm fed up with NetBackup. It vaguely works most of the time, but not all (which makes it difficult to justify it as a backup solution). And it's sucking up a lot of time - management and configuration takes a lot of work.<br /><br />My experience with Legato has been great. It's incredibly easy to manage, just works, and sits there in the background without needing me to constantly mother it. It's not perfect - the latest java-based GUI is pretty horrid, and my experience of it in non-unix (ie. Windows) environments is that it's a duck out of water.<br /><br />So, I'm looking around. I'll probably give Amanda a whirl. So if anyone has experience of both Amanda (or Zmanda) *and* one or both the above options, then feel free to comment - I'm particularly interested in comparisons.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-9101637084337630597?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com7tag:blogger.com,1999:blog-9726833.post-68506831129223577822009-03-22T21:20:00.002Z2009-03-22T21:39:13.756ZWe all know that just because something is new doesn't mean that it's better than what's gone before. A couple of examples I've had the misfortune to experience first-hand recently emphasize this:<br /><br />Firefox 3 is dramatically inferior to Firefox 2. Not only does it feel much more sluggish, but the URL bar (in particular the drop-down menu) is terrible. Yes, the oldbar extension removes some of the maddening irritations of the look and feel, but the list of URLs presented is completely broken, to the point where it's worse than useless. I've not upgraded every machine I have, and the overall experience on the machines I have upgraded is pretty poor - so much so that I'm tempted to revert.<br /><br />I've used emacs for decades. (And EDT/EVE/TPU before that.) For many years I've stayed with emacs 19.34, because it worked. Recently I've switched to emacs 22, because it's newer, maintained, and is what tends to be found on other systems. Again, the experience is staggeringly poor. Not only is it much slower but some of its features are just plain stupid. One of the more idiotic features I came across recently was that it assumes that a file named with a "zone" extension must be a DNS zone. (As a Solaris sysadmin, a zone is likely to be something else.) And if the file doesn't parse as it thinks a DNS zone file should then you are completely unable to even save the file - it's so smugly superior.<br /><br />The world is full of other examples. Why do we put up with it?<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-6850683112922357782?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com5tag:blogger.com,1999:blog-9726833.post-74206179634192239682009-03-21T15:50:00.002Z2009-03-21T16:10:45.791ZA while back we had a talk at <a href="http://www.opensolaris.org/os/project/losug/">LOSUG</a> about <a href="http://www.agilesysadmin.org/">Agile System Administration</a>. While presenting a particular view, I think Gordon was also trying to get us systems administrators to think more about the theory underlying the work that we do.<br /><br />Development has formal techniques - Agile, XP, Scrum, pair programming, test-first. And note that these are at a different layer from the actual programming skills required to write the software.<br /><br />Does Systems Administration have the same sorts of practices? Does it need them? Do the concepts of Agile Development translate into Systems Administration?<br /><br />Gordon has a <a href="http://blog.agilesysadmin.org/">blog</a> discussing some of his thoughts on the subject. I encourage those of you interested in the subject to read and contribute. (The home page of the blog is just the welcome; you'll need to look at the Archives and Categories to find the actual blog entries.)<br /><br />For what it's worth, I think somewhat differently, preferring flexible and lightweight (preferably zero-touch) minimalist administration over mandated standardization and heavyweight processes.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-7420617963419223968?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com2tag:blogger.com,1999:blog-9726833.post-13598945379525930122009-03-08T22:19:00.003Z2009-03-08T22:26:12.900ZA little bit more work (thanks to all for comments and encouragement!) and a new version of <a href="http://www.petertribble.co.uk/Solaris/jangle.html">Jangle</a> is available.<br /><br />Some highlights:<ul><li>Prototype tree view</li><li>User-defined MIB location (from the File menu)</li><li>Some error handling</li><li>Work done in the background for faster startup and better responsiveness</li><li>The graph has the human-readable name if we can rather than the numeric OID</li></ul><br />Enjoy!<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-1359894537952593012?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-69287387157766026962009-03-02T22:35:00.002Z2009-03-02T22:45:36.758ZOne of the comments I got yesterday was that my new Java SNMP browser didn't work on Ubuntu. That wasn't entirely a surprise - with such a rough first cut I was astonished that it worked at all, even for me.<br /><br />But anyway, that was easy enough to fix and I've released a new version that does run on Ubuntu (verified under 8.10) and that ought to me more sane and portable generally. One thing I need to work out is where the MIB files tend to be found on different systems.<br /><br />I've renamed it to <a href="http://www.petertribble.co.uk/Solaris/jangle.html">Jangle</a>. It's possible to think of this as<br /><br /><b>J</b>ava <b>A</b>ssistant for <b>N</b>etworked <b>G</b>raphs that's, <b>L</b>ike, <b>E</b>asy<br /><br />but I wouldn't want you to read too much into it. Really.<br /><br />Download, and run <code>./jangle browser</code><br /><br />(The new name has more zip, is less generic, and less likely to conflict with other tools.)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-6928738715776602696?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-15850930575085433952009-03-01T21:57:00.002Z2009-03-01T22:06:23.055ZI've been looking at SNMP a bit more closely recently. One thing that has struck me is that while there are a number of java based SNMP clients, none of them seemed to be complete, and I couldn't obviously find one that could produce charts easily.<br /><br />This is easy, right. So a little while later <a href="http://www.petertribble.co.uk/Solaris/snmpgui.html">snmpgui</a> was written.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://www.petertribble.co.uk/Solaris/snmpgui.gif"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 840px; height: 640px;" src="http://www.petertribble.co.uk/Solaris/snmpgui.gif" border="0" alt="" /></a><br /><br />OK, so this <b>is</b> version 0.01. I've been able to retrieve a list of things to monitor (that's the long list of OIDs above). Clearly that needs prettifying a bit - a tree view would be a nice next step. And I can retrieve values and produce pretty charts. That's incoming bytes on my main network interface.<br /><br />It's a simple proof of concept - but it proves that something can be done pretty easily, and that it works.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-1585093057508543395?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com4tag:blogger.com,1999:blog-9726833.post-39121859873385350192009-03-01T17:30:00.002Z2009-03-01T17:45:52.857ZI've uploaded a new version of <a href="http://www.petertribble.co.uk/Solaris/solview.html">SolView</a>. This broadly corresponds to the version I talked about at <a href="http://www.opensolaris.org/os/project/losug/">LOSUG</a>, with one new feature.<br /><br />I've improved the <a href="http://www.petertribble.co.uk/Solaris/profile.html">Jumpstart Profile Builder</a> by adding the ability to import an existing jumpstart profile. The idea is to use this as an initial customization of the set of installed packages that you can then modify further.<br /><br />In practice, this didn't work out anything like as well as I wanted it to. One of the constraints I place upon the construction of a profile is that the package selection be self consistent: package dependencies must be satisfied. (Indeed, it was the desire to be able to construct such profiles that led to writing the profile builder in the first place.) However, most existing profiles have customizations that lead to unsatisfied package dependencies. Often, fixing up such profiles as they are imported ends up simply undoing all your customizations.<br /><br />The current implementation is just a first pass, so I could improve it. For example, if it encounters a dependency error on import, it could ask you whether you want to keep the customization (and add or remove other packages as necessary) or ignore the customization. The problem is that this gets you into a horrible rats nest very quickly.<br /><br />In practice, I find it easier to build a new profile based on an existing one by eye - looking at what I'm trying to add or remove and replicating that in the jumpstart profile builder.<br /><br />So I'm tempted not to invest too much time on this feature. However, one thing I should be able to do relatively easily is to take the import code and use it to build a profile based on a straight list of packages. For example, this could be used to generate a jumpstart profile that would reproduce an existing installed system.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-3912185987338535019?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-28605845027883205742009-02-27T21:15:00.002Z2009-02-27T21:27:45.755ZI pushed out <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat 0.27</a> a few days ago. This is just a minor update to 0.26.<br /><br />I've been playing with the client-server mode a little, and while the basic code hasn't changed it's now a bit cleaner. I added a <code>jkstat.bat</code> and have been running the client on a Vista laptop quite successfully. I also enabled the ability to embed the client inside tomcat in addition to running the basic XML-RPC webserver.<br /><br />I also fixed a regression spotted when I updated SolView. I had changed the name of the shared library in 0.25, but 0.26 had the old name again. The reason is simple forgetfulness. I develop on an x86 machine, which normally isn't a problem, but if I do any work on the native code then I need to copy across to a sparc machine to do a recompile. I did that just before uploading, and must have forgotten to copy the final version back to my development system. Doh!<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-2860584502788320574?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com1tag:blogger.com,1999:blog-9726833.post-33020149674318208582009-02-15T21:39:00.002Z2009-02-15T21:53:29.291ZOne of the things I like about XML-RPC is that is really is astonishingly easy to implement. I like to be able to understand the code I write - and that includes the things I reuse from elsewhere. And with XML-RPC, it's even better - it's normally possible to parse the XML that comes back by eye.<br /><br />Contrast that with the monstrosity that SOAP has evolved into - a huge morass of complexity, massive code size, bloat everywhere, and it's really turned into a right mess. There's been nothing simple about SOAP for a long time now.<br /><br />The snag with XML-RPC is that there are a couple of rather crippling limitations. In particular, you're stuck with 32-bit integers. I've had to enable vendor extensions in order to use 64-bit longs, and while that's fine for my client code, it makes the <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> server far less useful for clients using other frameworks and languages.<br /><br />So I'm stuck in a hole. I'm going to stick with XML-RPC for now, as it will be good enough to test with and is really easy to develop against. (And the way I've implemented the remote access makes it trivial to put in new or replacement access services.)<br /><br />What's the way forward? I'm not at all keen on building a full web-services stack - that defeats the whole object of the exercise. There has been some discussion recently of exposing the kstat hierarchy via SNMP, but that's clunky and depends on SNMP (great if you use SNMP already, less great if you don't). I've been looking at things like JSON-RPC and RESTful web services as alternatives. The simplest approach may just be to encode everything as Strings and convert back.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-3302014967431820858?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com1tag:blogger.com,1999:blog-9726833.post-91296077306838967102009-02-12T20:47:00.002Z2009-02-12T21:13:31.822ZOne of the things I've wanted to add to <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a> for a while is some sort of client-server capability: the ability to run a server process on a machine and connect to it from a remote client which runs the GUI.<br /><br />This separation of responsibility can offload the graphical processing associated with displaying the output off the server onto your desktop device. And that desktop device need not be running Solaris. Although I haven't tested it myself, there's no reason why you can't now run the client on - say - Windows.<br /><br />So, how do you make this work? Having downloaded the latest (0.26) version of <a href="http://www.petertribble.co.uk/Solaris/jkstat.html">JKstat</a>, unpack it, and on the server machine you wish to monitor:<pre><br />./jkstat server -p 7367<br /></pre><br />where you can choose a port number, and on the client machine where you want the kstat browser to run<br /><pre><br />./jkstat remotebrowser -s http://server_name:7367/<br /></pre><br />so it knows where to look for the server.<br /><br />The current implementation is very much an experimental proof of concept. There's no access control implemented at present, for example. And the way it operates is bound to change.<br /><br />At the moment, it's using <a href="http://ws.apache.org/xmlrpc/">Apache XML-RPC</a>. I like XML-RPC because it's very simple to use, pretty fast, and lightweight. There are also - in principle - implementations for other languages so you don't have to use Java.<br /><br />And the Apache XML-RPC implementation can either be standalone (that's the way I'm using it right now) or embedded in a servlet container. The latter would allow you to put pretty well any access control mechanism you like in place.<br /><br />I'm not particularly fixated about either XML-RPC or this specific implementation, though. This was chosen first because I've used it before and know it's so easy to implement - allowing me to concentrate on the JKstat specific part and seeing how well (or badly) client-server works at all before investing a lot of effort.<br /><br />In practice, XML-RPC is a pretty good match. It implements Lists (or arrays) and Maps (or hashes) as native constructs. I don't need any more complex data structures. The weakness is that while all I need to pass in those structures are Strings and Long values, the latter requires Vendor extensions, which I've had to turn on, breaking the portability aspect.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-9129607730683896710?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0tag:blogger.com,1999:blog-9726833.post-68760086291611460222009-01-27T11:11:00.002Z2009-01-27T11:16:55.300ZThe Sun coolthreads boxes are great for throughput, but not all that hot at single-threaded performance. One thing that seems to go particularly badly is compression.<br /><br />Enter <a href="http://compression.ca/pbzip2/">pbzip2</a><br /><pre><br /># pbzip2 foo.tar <br />50187.77u 156.31s 6:39.39 12605.2%<br /></pre><br />It can fill up all 128 threads and something that would otherwise take over half a day can be done in just over 6 minutes.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/9726833-6876008629161146022?l=ptribble.blogspot.com' alt='' /></div>Peter Tribblehttp://www.blogger.com/profile/09363446984245451854noreply@blogger.com0