Mahout parallel k-means in Hadoop

102 views Asked by At

Is it possible to run Mahout k-means algorithm in parallel (multi-core) using Hadoop? How?

Mahout run using Hadoop but it only uses one CPU:

mahout org.apache.mahout.clustering.syntheticcontrol.kmeans.Job --input testdata --output end1200_50 --numClusters 1200 --t1 1000 --t2 500 --maxIter 50
Running on hadoop, using /usr/local/hadoop/bin/hadoop and HADOOP_CONF_DIR=/usr/local/hadoop/etc/hadoop
MAHOUT-JOB: /usr/local/mahout/mahout-examples-0.10.1-job.jar
[...]





My files are in HDFS hadoop fs -ls /user/root/testdata

Found 12 items
-rw-r--r--   1 root supergroup  373560731 2015-06-26 07:51 /user/root/testdata/16773m.mat.txt
-rw-r--r--   1 root supergroup  373819865 2015-06-26 07:51 /user/root/testdata/16786m.mat.txt
[...]


my mapred-site.xml

<configuration>
 <property>
  <name>mapred.job.tracker</name>
  <value>localhost:54311</value>
 </property>


 <property>
  <name>mapred.tasktracker.map.tasks.maximum</name>
  <value>14</value>
 </property>

 <property>
  <name>mapred.tasktracker.map.tasks.maximum</name>
  <value>4</value>
 </property>

 <property>
  <name>mapred.child.java.opts</name>
  <value>-Xmx7000M</value>
 </property>
</configuration>


0

There are 0 answers