I am a complete beginner in running or using ML workloads. I am just following the ReadMe instructions in the corresponding benchmark git pages.
I want to run the MLPerf Inference benchmark suite on resnet50-v1.5 onnx model, using the ImageNet-2012 validation dataset.
I extract the .tar file downloaded from the link above. The .tar file contains no directory structure; it just contains many images. (I thought it be grouped in a specific directory structure to represent the classes).
$ tar -xf ILSVRC2012_img_val.tar
$ ls
ILSVRC2012_val_00000001.JPEG ILSVRC2012_val_00012501.JPEG ILSVRC2012_val_00025001.JPEG ILSVRC2012_val_00037501.JPEG
ILSVRC2012_val_00000002.JPEG ILSVRC2012_val_00012502.JPEG ILSVRC2012_val_00025002.JPEG ILSVRC2012_val_00037502.JPEG
ILSVRC2012_val_00000003.JPEG ILSVRC2012_val_00012503.JPEG ILSVRC2012_val_00025003.JPEG ILSVRC2012_val_00037503.JPEG
ILSVRC2012_val_00000004.JPEG ILSVRC2012_val_00012504.JPEG ILSVRC2012_val_00025004.JPEG ILSVRC2012_val_00037504.JPEG
...
I exported the data directory and model directory and then decided to use Docker container for experimentation.
$ ./run_and_time.sh onnxruntime resnet50 gpu
The above command successfully builds the docker image and tries to launch the application inside the container, but the application complains about a missing val_map.txt.
[+] Building 0.9s (16/16) FINISHED
=> [internal] load build definition from Dockerfile.gpu
...
TRUNCATED DOCKER BUILD OUTPUT
...
=> => naming to docker.io/library/mlperf-infer-imgclassify-gpu 0.0s
Clearing caches.
3
STARTING RUN AT 2023-07-27 07:23:05 PM
INFO:main:Namespace(accuracy=False, audit_conf='audit.config', backend='onnxruntime', cache=0, cache_dir=None, count=None, data_format=None, dataset='imagenet', dataset_list=None, dataset_path='/home/abhishek/shweta_machine/DGSF_workloads/Image_Classification/ImageNet_2012_validation', debug=False, find_peak_performance=False, inputs=None, max_batchsize=32, max_latency=None, mlperf_conf='./mlperf.conf', model='/home/abhishek/shweta_machine/DGSF_workloads/Image_Classification/resnet50_v1.onnx', model_name='resnet50', output='/output', outputs=['ArgMax:0'], performance_sample_count=None, preprocessed_dir=None, profile='resnet50-onnxruntime', qps=None, samples_per_query=8, scenario='SingleStream', threads=16, time=None, use_preprocessed_dataset=False, user_conf='user.conf')
Traceback (most recent call last):
File "python/main.py", line 624, in <module>
main()
File "python/main.py", line 501, in main
**kwargs)
File "/mlperf/python/imagenet.py", line 53, in __init__
with open(image_list, 'r') as fp:
FileNotFoundError: [Errno 2] No such file or directory: '/home/abhishek/shweta_machine/DGSF_workloads/Image_Classification/ImageNet_2012_validation/val_map.txt'
ENDING RUN AT 2023-07-27 07:23:05 PM
abhishek@abhishek:classification_and_detection$
Looking into the specific section of the codebase I found the following:
...
# input images are in HWC
self.need_transpose = True if image_format == "NCHW" else False
self.not_found = 0
if image_list is None:
# by default look for val_map.txt
image_list = os.path.join(data_path, "val_map.txt")
with open(image_list, 'r') as fp: # <--- line 53
for count, line in enumerate(fp):
pass
count = count + 1
if not self.count:
CNT = count
else:
CNT = count if count <= self.count else self.count
os.makedirs(self.cache_dir, exist_ok=True)
...
I assume the val_map.txt is required for getting the mapping of class, which is kind of mentioned in here.
But I do not have much knowledge regarding this. Please, can anyone help me?
