hadaoop + Configuring NameNode Heap Size

96 views Asked by At

in Short - we want to know how to configure the heap size for a name-node

so far we calculate the name-node heap size as the following

first we verified the number of files by - hadoop fs -count / | awk '{print $2}'

and once we get the number from hadoop fs -count / | awk '{print $2}'

then we verify what is the correct name-node heap size from the table - https://www.ibm.com/support/pages/configuring-namenode-heap-size

but we don't sure if we need to consider also the HDFS replication factor ?

for example lets say we have HDFS replica = 3 and number of files is 5000000 ( from hadoop fs -count / | awk '{print $2}' )

so maybe we need to calculate the total number of files as 5000000 X 3 = 15000000

and then take the number 15000000 and verified the name-node heap size according to 15000000 and not 5000000

so am I correct about this?

0

There are 0 answers