I'm trying to use Giraph on hadoop 2.6.0
with yarn.
I've managed to build it by removing STATIC_SASL_SYMBOL
in <munge.symbols>
in the yarn profile
. with the command :
sudo mvn -Phadoop_yarn -Dhadoop.version=2.6.0 -DskipTests package
Then i've setup a single node cluster with hadoop 2.6.0 and I'm able to perform the basic wordcount example.
I've copied the jar built into the directory share/hadoop/mapreduce
to remove the GiraphMaster
not found execption.
Now I'm trying to run the SimpleShortestPaths
example with the command :
$HADOOP_PREFIX/bin/hadoop jar /usr/local/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.6.0-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip input_tiny2.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hduser/output/shortestpaths445 -w 1 -ca giraph.SplitMasterWorker=false -yj giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.6.0-jar-with-dependencies.jar
The data used is the tiny graph on the giraph quick start page and the job should takes minutes to complete but it doesn't.
The full log is at http://pastebin.com/CW1FnWz2
It's possible to make giraph work on hadoop 2.6.0 or do I have to use an earlier version ? If so which one would you suggest ?