Monday, December 20, 2021

Keeping Java alive on illumos

Back in 2019, a new JEP (JDK Enhancement Proposal) appeared.

JEP 362: Deprecate the Solaris and SPARC Ports

Of course, for those of us running Solaris or illumos (which is the same platform as far as Java is concerned), this was a big deal. Losing support for a major language on the platform was potentially a problem.

The stated reason for removal was:

Dropping support for these ports will enable contributors in the OpenJDK Community to accelerate the development of new features that will move the platform forward.

Clearly, this reflected a belief that maintaining Solaris and/or SPARC was a millstone dragging Java down. Still, it's their project, they can make whatever decisions they like, despite those of us who thought it was a bad move.

Eventually, despite objections, the ports were removed, towards the end of the JDK15 cycle.

At which point I simply carried on building OpenJDK. All I did was take the patch from the commit that removed Solaris support, applied that backwards, and added on top the pkgsrc patches that Jonathan Perkin had originally developed to support a gcc port on Solaris and illumos - patches we had already been using extensively from JDK11 onwards.

At that point I wasn't quite sure how sustainable this was. My aim was to support it as long as it wasn't proving too onerous or difficult, and my most optimistic hope was that we might be able to get to Java 17 which was planned as the next LTS release.

The modus operandi was really very simple. Every week a new tag is created. Download the tag, apply the patches, fix any errors in the patch set, try a build, hopefully fix any problems breaking the build.

Rinse and repeat, every week. The idea is that by doing it every week, it's a relatively small and manageable set of changes each time. Some weeks, it's just line number noise in the patches. Other weeks, it could be a more significant change. By loitering on the mailing lists, you become aware of what changes are coming up, which gives you a good idea of where to look when the build breaks.

Along the way, I've been cleaning up the patches to eliminate the SPARC code (you could put it back, but it's not a focus of this project) and most of the code to support the Studio toolchain (the version of Studio to build current Java isn't compatible with illumos anyway). So what we're left with is a straightforward Solaris/illumos+gcc port.

Most of the code changes I've needed to make are fairly straightforward procedural changes. Some functions moved namespace. Some function signatures have been changed. There's also been a lot of work to consolidate a number of functions into common posix code, rather than have each OS provide different implementations which might diverge and become hard to maintain.

Most of this was pretty simple. The only one that caused me a significant amount of work was the signal handling rewrite, which took several attempts to get to work at all.

And it's become fairly routine. Java 17 came along, eventually, and the builds were still succeeding and basic smoke-testing worked just fine. So, illumos has Java 17 available, just as I had hoped.

I originally packaged the builds on Tribblix, of course, which is where I'm doing the work. But I've also dropped tarballs of occasional builds so they can be downloaded and used on other illumos distributions.

Actually, the idea of those builds isn't so much that they're useful standalone, but they provide a bootstrap JDK that you can use to build Java yourself. Which, given that bootstrap JDK and my java patches, ought to be fairly straightforward. (There's a separate patch directory for each jdk release - the directory name ought to be obvious.)

Which means that if you want Java 17 on OmniOS, you can have just that - it's built and packaged ready for you. Not only that, Dominik fixed some problems with my signal handling fix so it works properly and without errors, which benefits everyone.

It doesn't stop there. In addition to the stream of quarterly updates (JDK 17 being an LTS release will see these for some time yet) work is continuing on mainline. JDK 18 works just fine, and as it's ramping down for release shouldn't have any breaking changes, so that's another release supported. I'm building JDK 19, although that's only about 1 build in so hasn't really had any significant changes put into it yet.

The fact that a relatively unskilled developer such as myself can maintain an out of tree Java port for a couple of years, tracking all the upstream changes, does make you wonder if supporting Solaris was really that much of a blocker to progress. At the time my belief was that it wasn't Solaris support that was the problem, but the Studio toolchain, and I think that's been borne out by my experience. Not only that, but the consolidation and simplification of the various OS-specific code into common posix code shows that supporting a variety of modern operating systems really isn't that hard.


FenixCorgi said...

So there's no SPARC support in newer Java at all? I run Debian in an OpenBSD LDOM and iirc I'm running JRE17 rn??? Good work on keeping the solaris support though! We need Illumos to become a little more popular for servers and stuff I guess. I'm honestly sad that they dropped SPARC support in illumos, my SPARC T-4 could use a good FOSS OS to run on it.

Polytoximaniac said...

I am maintaining a couple of OpenJDK builds for regular upstream Solaris and can confirm that the Solaris Studio toolchain is a major nuisance. As they (understandably) move to using newer C++ features in hotspot, maintenance will get more difficult, I might have to look into switching to GCC at some point. If Oracle was continuing to develop the Studio compilers or officially supported a GCC for upstream Solaris, I guess they could have kept it around.

Pete said...

I suspect that Oracle's abandoning of Solaris Java was primarily due to SPARC. Not apologizing for them but it was indeed heavy baggage to carry forward, loaded with twenty five years of corner case handling and dwindling institutional and community contributor knowledge for SPARC and its oddball features like register windows. There is a lot of core (and future) JDK functionality that depend on the underlying ISA in great detail. So why not just remove SPARC support in the official JDK distribution, people may ask? Doing so would just cause confusion since Solaris is often thought of as synonymous with SPARC. Node definitely had this issue: ran great on Solaris x86 systems (fueled by Joyent), never ran on SPARC due to the sheer complexity of porting the huge V8 system to SPARC, causing endless churn and dissatisfaction along the way. In the end, some Oracle VP just pulled the trigger to axe the whole Solaris JDK tree. I don't see any reason though why the JDK can't continue on the Illumos x86 train, with the fine work you are doing.

Thomas Stuefe said...

Hi Peter,

keeping this port alive is impressive work for sure.

As a hotspot developer who works in the hotspot runtime, I can tell you that removing the Solaris port was a large work reduction. I wish it were different - having many platforms has many benefits - but we don't have endless cycles and every reduction in maintenance is a relief.

Its not only the code itself. Its testing, keeping test infrastructure, working around compiler weirdnesses and different tool sets (eg dbx instead of gdb)... and all that does not even start touching the JIT compiler itself.

Cheers, Thomas

Peter Tribble said...

Thanks Thomas!

One of the key decisions - and the one that makes it possible for me to maintain the port without it requiring a ridiculous amount of effort - is to use the same compiler (ie gcc) and toolset as other ports. There's really no need to be different, and that choice just makes illumos yet another posix system that can leverage all the other posix ports.