How to process 1.2 million messages per second using apache samza?

Question

Now i made basic setup of samza . After that i process 1,00,000 json records which is from kafka broker topic Each records in size of 100 bytes But per second i can only process 11000 records. How to increase processing speed?

Answer 1

If you need more broker you can increase number partition :

bin/kafka-topics.sh --zookeeper zoo:port/chroot --alter --topic TOPIC_NAME_CHANGE_ME --partitions 10 

So nb partition == nb worker

For more information : Documentation