I have installed Hadoop 3.1.1 which supports GPU resources. And I configured it according to the official document . However, when I run the word count program with a 800M+ file, the GPU finding resource never are used; at the same time the program works well using memory and vcores(CPU).
All the Prerequisites listed in official document are met: GPU device,CUDA,cgoup etc. GPU works well when I run the docker but Hadoop 3.
Follow are the configuration files:
Resource Type:
capacity-scheduler.xml,other property as default
yarn-site.xml
container-executor.cfg