Hive cannot find LZO codec

428 views Asked by At

Error occurred when execute select * from xxx:

Failed with exception java.io.IOException:java.io.IOException: No LZO codec found, cannot run.

Troubleshooting done:

Checked hadoop-lzo.jar located in $HADOOP_HOME/share/hadoop/common for all hadoop nodes:

# ls $HADOOP_HOME/share/hadoop/common/ | grep lzo
hadoop-lzo-0.4.20.jar

Checked LZO codec configured in $HADOOP_HOME/etc/hadoop/core-site.xml for all hadoop nodes:

<configuration>
    ...
    <property>
        <name>io.compression.codecs</name>
        <value>
            org.apache.hadoop.io.compress.GzipCodec,
            org.apache.hadoop.io.compress.DefaultCodec,
            org.apache.hadoop.io.compress.BZip2Codec,
            org.apache.hadoop.io.compress.SnappyCodec,
            com.hadoop.compression.lzo.LzoCodec,
            com.hadoop.compression.lzo.LzopCodec
        </value>
    </property>
    <property>
        <name>io.compression.codec.lzo.class</name>
        <value>com.hadoop.compression.lzo.LzoCodec</value>
    </property>
</configuration>

I also tested read lzo file in mapreduce, it works correctly, so I think hadoop-lzo configured properly, but it didn't work in hive.

0

There are 0 answers