I'm going through the Apache Giraph quick start tutorial: http://giraph.apache.org/quick_start.html and have successfully setup a pseudo-distributed hadoop cluster and have successfully run the example mapreduce jobs.
However when moving to the Giraph section, I've installed Giraph using maven and I'm for some reason unable to run the simplest of Giraph jobs.
I've added the tiny_graph to my hdfs, but running their example command:
hduser@fonz-VirtualBox:~/Giraph$ $HADOOP_HOME/bin/hadoop jar /usr/local/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/hduser/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hduser/output/shortestpaths -w 1
Gives me:
14/12/18 11:19:27 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
14/12/18 11:19:27 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one.
14/12/18 11:19:28 INFO job.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 1, old value = 4)
14/12/18 11:19:31 INFO job.GiraphJob: Tracking URL: http://hdnode01:50030/jobdetails.jsp?jobid=job_201412171202_0010
14/12/18 11:19:31 INFO job.GiraphJob: Waiting for resources... Job will start only when it gets all 2 mappers
14/12/18 11:20:59 INFO job.HaltApplicationUtils$DefaultHaltInstructionsWriter: writeHaltInstructions: To halt after next superstep execute: 'bin/halt-application --zkServer fonz-virtualbox:22181 --zkNode /_hadoopBsp/job_201412171202_0010/_haltComputation'
14/12/18 11:20:59 INFO mapred.JobClient: Running job: job_201412171202_0010
14/12/18 11:21:00 INFO mapred.JobClient: map 100% reduce 0%
14/12/18 11:30:16 INFO mapred.JobClient: map 50% reduce 0%
14/12/18 11:30:24 INFO mapred.JobClient: Job complete: job_201412171202_0010
14/12/18 11:30:24 INFO mapred.JobClient: Counters: 6
14/12/18 11:30:24 INFO mapred.JobClient: Job Counters
14/12/18 11:30:24 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=1244789
14/12/18 11:30:24 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
14/12/18 11:30:24 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
14/12/18 11:30:24 INFO mapred.JobClient: Launched map tasks=2
14/12/18 11:30:24 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
14/12/18 11:30:24 INFO mapred.JobClient: Failed map tasks=1
Any ideas as to why this has failed would be greatly appreciated!
Thanks very much.
Will
The shortest path example does not have a main method and it uses hard coded source ID as 1, to start the algorithm. Therefore there is no vote to halt after 100% Map. Go to https://www.youtube.com/watch?v=yEHeJhMXJy0, and update the sssp.java file and export it to a new Jar. Then run it. I was facing the same issue and it fixed it.