I am trying to setup hadoop 2.7.3 single node cluster on my machine. At one time, I successfully ran all the services and ran my custom jar file which worked fine. After stopping and starting the services again later, I found out that Resource Manager is not running. I start the services using:
start-dfs.sh
start-yarn.sh
Doing jps
gives:
14755 ResourceManager
13396 DataNode
14884 NodeManager
13606 SecondaryNameNode
15099 Jps
13262 NameNode
which is fine, but after sometime when I run my jar file,it stucks at this:
17/09/08 12:54:59 INFO client.RMProxy: Connecting to ResourceManager at /127.0.0.1:8032
17/09/08 12:55:01 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
17/09/08 12:55:02 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
By doing jps
again shows Resource manager not running. I am very confused about this, tries lots of solution but not found a single solution.
Here is my /etc/hosts
127.0.0.1 localhost
127.0.1.1 salman-home
# The following lines are desirable for IPv6 capable hosts
#::1 ip6-localhost ip6-loopback
#fe00::0 ip6-localnet
#ff00::0 ip6-mcastprefix
#ff02::1 ip6-allnodes
#ff02::2 ip6-allrouters
I have disables the firewall by sudo ufw disable
Here is my yarn-site.xml:
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>127.0.0.1:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>127.0.0.1:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>127.0.0.1:8031</value>
</property>
</configuration>
mapred-site.xml:
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
core-site.xml:
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost/</value>
</property>
</configuration>
hdfs-site.xml:
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
Any kind of help will be much appreciated.
It was bit silly of me. That port was be used by
InfluxDB
. Resource manager tried to acquire it but that tug of war always won by Influx. By stop Influxdb service, I solved this problem.