detectron2 run correctly, but tensormask keep showing RuntimeError: Not compiled with GPU support

59 views Asked by At

Sorry for my bad English first. My OS is WIN11, I created a virtual environment using Anaconda, python version is 3.8.18, and use

conda install pytorch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 cudatoolkit=11.3 -c pytorch -c conda-forge

to install torch and cuda.

Then cd to detectron2 folder, pip install it and run a demo.py, The program runs very smoothly. Although the display of "print(torch.cuda.is_available(), CUDA_HOME)" shows (True,None), it still can run my training code.

After it I cd to tensormask folder and pip install it, it show "Successfully installed tensormask-0.1", but when I trying to run my training code, it keep showing

Y = \_C.swap_align2nat_forward(X, lambda_val, pad_val)
RuntimeError: Not compiled with GPU support

.

The whole error is:

ERROR \[11/17 06:01:47 d2.engine.train_loop\]: Exception during training:
Traceback (most recent call last):
File "c:\\users\\andy5\\downloads\\detectron2-main\\detectron2-main\\detectron2\\engine\\train_loop.py", line 155, in train
self.run_step()
File "c:\\users\\andy5\\downloads\\detectron2-main\\detectron2-main\\detectron2\\engine\\defaults.py", line 496, in run_step
self.\_trainer.run_step()
File "c:\\users\\andy5\\downloads\\detectron2-main\\detectron2-main\\detectron2\\engine\\train_loop.py", line 310, in run_step    loss_dict = self.model(data)
File "C:\\Users\\andy5\\anaconda3\\envs\\detect\\lib\\site-packages\\torch\\nn\\modules\\module.py", line 1102, in \_call_impl
return forward_call(\*input, \*\*kwargs)
File "c:\\users\\andy5\\downloads\\detectron2-main\\detectron2-main\\projects\\tensormask\\tensormask\\arch.py", line 382, in forward
pred_logits, pred_deltas, pred_masks = self.head(features)
File "C:\\Users\\andy5\\anaconda3\\envs\\detect\\lib\\site-packages\\torch\\nn\\modules\\module.py", line 1102, in \_call_impl
return forward_call(\*input, \*\*kwargs)
File "c:\\users\\andy5\\downloads\\detectron2-main\\detectron2-main\\projects\\tensormask\\tensormask\\arch.py", line 908, in forward
cur_mask = cur_mask_module(cur_mask)
File "C:\\Users\\andy5\\anaconda3\\envs\\detect\\lib\\site-packages\\torch\\nn\\modules\\module.py", line 1102, in \_call_impl
return forward_call(\*input, \*\*kwargs)
File "c:\\users\\andy5\\downloads\\detectron2-main\\detectron2-main\\projects\\tensormask\\tensormask\\layers\\swap_align2nat.py", line 54, in forward
return swap_align2nat(X, self.lambda_val, self.pad_val)
File "c:\\users\\andy5\\downloads\\detectron2-main\\detectron2-main\\projects\\tensormask\\tensormask\\layers\\swap_align2nat.py", line 15, in forward
Y = \_C.swap_align2nat_forward(X, lambda_val, pad_val)
RuntimeError: Not compiled with GPU support
\[11/17 06:01:47 d2.engine.hooks\]: Total training time: 0:00:03 (0:00:00 on hooks)
\[11/17 06:01:47 d2.utils.events\]:  iter: 0       lr: N/A  max_mem: 3872M
Traceback (most recent call last):
File "Tensormask_Train.py", line 37, in \<module\>
trainer.train()
File "c:\\users\\andy5\\downloads\\detectron2-main\\detectron2-main\\detectron2\\engine\\defaults.py", line 486, in train
super().train(self.start_iter, self.max_iter)
File "c:\\users\\andy5\\downloads\\detectron2-main\\detectron2-main\\detectron2\\engine\\train_loop.py", line 155, in train
self.run_step()
File "c:\\users\\andy5\\downloads\\detectron2-main\\detectron2-main\\detectron2\\engine\\defaults.py", line 496, in run_step
self.\_trainer.run_step()
File "c:\\users\\andy5\\downloads\\detectron2-main\\detectron2-main\\detectron2\\engine\\train_loop.py", line 310, in run_step    loss_dict = self.model(data)
File "C:\\Users\\andy5\\anaconda3\\envs\\detect\\lib\\site-packages\\torch\\nn\\modules\\module.py", line 1102, in \_call_impl
return forward_call(\*input, \*\*kwargs)
File "c:\\users\\andy5\\downloads\\detectron2-main\\detectron2-main\\projects\\tensormask\\tensormask\\arch.py", line 382, in forward
pred_logits, pred_deltas, pred_masks = self.head(features)
File "C:\\Users\\andy5\\anaconda3\\envs\\detect\\lib\\site-packages\\torch\\nn\\modules\\module.py", line 1102, in \_call_impl
return forward_call(\*input, \*\*kwargs)
File "c:\\users\\andy5\\downloads\\detectron2-main\\detectron2-main\\projects\\tensormask\\tensormask\\arch.py", line 908, in forward
cur_mask = cur_mask_module(cur_mask)
File "C:\\Users\\andy5\\anaconda3\\envs\\detect\\lib\\site-packages\\torch\\nn\\modules\\module.py", line 1102, in \_call_impl
return forward_call(\*input, \*\*kwargs)
File "c:\\users\\andy5\\downloads\\detectron2-main\\detectron2-main\\projects\\tensormask\\tensormask\\layers\\swap_align2nat.py", line 54, in forward
return swap_align2nat(X, self.lambda_val, self.pad_val)
File "c:\\users\\andy5\\downloads\\detectron2-main\\detectron2-main\\projects\\tensormask\\tensormask\\layers\\swap_align2nat.py", line 15, in forward
Y = \_C.swap_align2nat_forward(X, lambda_val, pad_val)
RuntimeError: Not compiled with GPU support

I've searched for many issues, but most of them are related to the "RuntimeError: Not compiled with GPU support error" during training in Detectron2. Also the official suggestion is to confirm with print(torch.cuda.is_available(), CUDA_HOME).

However, the problem I'm facing is that everything works fine with Detectron2 training and downloading TensorMask(with (torch.cuda.is_available, CUDA_HOME) show (True, None)), but the error occurs only when I try to train with TensorMask.

How can I resolve this issue, or are there any keywords can provide for me to search?

0

There are 0 answers