Tuesday, January 27, 2009

Compression going really fast

The Sun coolthreads boxes are great for throughput, but not all that hot at single-threaded performance. One thing that seems to go particularly badly is compression.

Enter pbzip2

# pbzip2 foo.tar
50187.77u 156.31s 6:39.39 12605.2%

It can fill up all 128 threads and something that would otherwise take over half a day can be done in just over 6 minutes.

No comments: