Monthly Archives: December 2012

1. install lzo on all nodes sudo apt-get install liblzo2-dev 2. build hadoop-lzo git clone git://github.com/kevinweil/hadoop-lzo.git ant compile-native tar 3. copy jar and libraries into cluster on all nodes cp build/hadoop-lzo-*/hadoop-lzo-*.jar /usr/lib/hadoop-0.20/lib/ cp build/native/Linux-amd64-64/lib/libgplcompression.* /usr/lib/hadoop-0.20/lib/native/Linux-amd64-64/ 4. add to core-site.xml <property> <name>io.compression.codecs</name> <value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.BZip2Codec,com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec</value> </property> … Continue reading →