For LZO files to be splittable for Hadoop they require an accompanied index file. One project does this job is the hadoop-lzo. Unfortunately it depends the native libraries via JNI. In other words - pain just around the corner. As creating the index merely needs more than seeking through the LZO file, there really is no reason for not doing this without all these dependencies.
The jars are available on maven central. The source releases you can get in the download section.
If feel adventures or want to help out feel free to get the latest code via git.
git clone git://github.com/tcurdt/lzo-index.git
You can either use the jar directly from the command line
> java -jar lzo-index.jar -v file.lzo
file.lzo -> file.lzo.index
Or you can use the 'LzoIndexer' class directly
LzoIndexer indexer = new LzoIndexer();
indexer.createIndex(
new FileInputStream("file.lzo"),
new FileOutputStream("file.lzo.index"));
mvn clean install
I wish I could release this under ASL 2.0 but unfortunately the LZO code is under GPL which is why one could argue this is derived work ...and therefor also has to be GPL. Feel free to convince Markus to at least change the license to LGPL. Anyway - for now it has to be GPLv2.