How to trace the origin of "<init>()V" failures in Avro?

Question

I am using apache crunch and have got a cryptic error message from Avro:

java.lang.NoSuchMethodError: org.apache.avro.mapred.AvroKey: method <init>()V not found
    at org.apache.crunch.types.avro.AvroKeyConverter.getWrapper(AvroKeyConverter.java:57)
    at org.apache.crunch.types.avro.AvroKeyConverter.outputKey(AvroKeyConverter.java:36)
    at org.apache.crunch.types.avro.AvroKeyConverter.outputKey(AvroKeyConverter.java:25)
    at org.apache.crunch.impl.mr.emit.MultipleOutputEmitter.emit(MultipleOutputEmitter.java:41)
    at org.apache.crunch.MapFn.process(MapFn.java:34)
    at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:99)
    at org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56)
    at org.apache.crunch.MapFn.process(MapFn.java:34)
    at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:99)
    at org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56)
    at org.apache.crunch.MapFn.process(MapFn.java:34)
    at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:99)
    at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:110)
    at org.apache.crunch.impl.mr.run.CrunchMapper.map(CrunchMapper.java:60)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)

What is the meaning of the " init()V " error ? Specifically, I'd like to fix this problem in crunch also - it only occurs when using hthe Mapredce pipeline option for a job, but i dont see it occuring using MemPipeline.

Answer 1

<init>()V refers to the 0-parameter constructor. It seems that the class AvroKey does not have such a constructor.

This often happens when you have mismatched versions of the libraries on your classpath. In this case, it is likely that the version of Crunch on your classpath expects a version of Avro that has the no-arg constructor, but the version you are providing does not have that constructor. Hence the runtime NoSuchMethodError.

Answer 2

<init>()V is the internal name of a constructor that takes no parameters.

The error means that the class org.apache.avro.mapred.AvroKey that you are using does not have a no-args constructor.

You might be running your application with a version of Avro that is different from what you compiled it with. If that's the case, make sure you use the same version for compiling and running.

Otherwise, find out why your code is trying to access a constructor that doesn't exist.

Answer 3

To add some color on this:

The OLD AvroKey class only supports a single, ONE argument constructor.

/** The wrapper of keys for jobs configured with {@link AvroJob} . */
public class AvroKey<T> extends AvroWrapper<T> {
  /** Wrap a key. */
  public AvroKey(T datum) { super(datum); }
}

The NEW AvroKey Class (1.4) and up includes an empty constructor.

So it must be an old avro implementation on my org.apache.avro.mapred.AvroKey on my classpath somewhere.