How to resolve "MPS Framework doesn't support float64" on M1 Mac

910 views Asked by At

I am trying to train a YOLO model on a custom dataset using YOLOv8 Ultralytics, using M1 MacBook Pro MPS acceleration which is supported by PyTorch but I am getting the following error while executing the train command.

Command: !yolo task=detect mode=train model=yolov8s.pt data={HOME}/datasets/data.yaml epochs=25 device=mps

Error Message:

Traceback (most recent call last):


File "/opt/homebrew/lib/python3.10/site-packages/ultralytics/yolo/cli.py", line 56, in cli
    func(cfg)
  File "/opt/homebrew/lib/python3.10/site-packages/hydra/main.py", line 79, in decorated_main
    return task_function(cfg_passthrough)
  File "/opt/homebrew/lib/python3.10/site-packages/ultralytics/yolo/v8/detect/train.py", line 207, in train
    model.train(**cfg)
  File "/opt/homebrew/lib/python3.10/site-packages/ultralytics/yolo/engine/model.py", line 193, in train
    self.trainer.train()
  File "/opt/homebrew/lib/python3.10/site-packages/ultralytics/yolo/engine/trainer.py", line 185, in train
    self._do_train(int(os.getenv("RANK", -1)), world_size)
  File "/opt/homebrew/lib/python3.10/site-packages/ultralytics/yolo/engine/trainer.py", line 301, in _do_train
    self.loss, self.loss_items = self.criterion(preds, batch)
  File "/opt/homebrew/lib/python3.10/site-packages/ultralytics/yolo/v8/detect/train.py", line 77, in criterion
    return self.compute_loss(preds, batch)
  File "/opt/homebrew/lib/python3.10/site-packages/ultralytics/yolo/v8/detect/train.py", line 168, in __call__
    targets = self.preprocess(targets.to(self.device), batch_size, scale_tensor=imgsz[[1, 0, 1, 0]])
TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

Is there any way to get around this issue and use MPS for training?

I have tried looking for this issue on PyTorch's GitHub conversation, but got no simple solution.

Edit: I have set HYDRA_FULL_ERROR to 1, and this is the updated error traceback:

Error executing job with overrides: ['task=detect', 'mode=train', 'model=yolov8s.pt', 'data=/Users/aditya_nemiwal/Documents/C++/Python/WiDS_Week3_3/datasets/data.yaml', 'epochs=25', 'device=mps']
Traceback (most recent call last):
  File "/opt/homebrew/bin/yolo", line 8, in <module>
    sys.exit(cli())
  File "/opt/homebrew/lib/python3.10/site-packages/hydra/main.py", line 90, in decorated_main
    _run_hydra(
  File "/opt/homebrew/lib/python3.10/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra
    _run_app(
  File "/opt/homebrew/lib/python3.10/site-packages/hydra/_internal/utils.py", line 457, in _run_app
    run_and_report(
  File "/opt/homebrew/lib/python3.10/site-packages/hydra/_internal/utils.py", line 222, in run_and_report
    raise ex
  File "/opt/homebrew/lib/python3.10/site-packages/hydra/_internal/utils.py", line 219, in run_and_report
    return func()
  File "/opt/homebrew/lib/python3.10/site-packages/hydra/_internal/utils.py", line 458, in <lambda>
    lambda: hydra.run(
  File "/opt/homebrew/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 132, in run
    _ = ret.return_value
  File "/opt/homebrew/lib/python3.10/site-packages/hydra/core/utils.py", line 260, in return_value
    raise self._return_value
  File "/opt/homebrew/lib/python3.10/site-packages/hydra/core/utils.py", line 186, in run_job
    ret.return_value = task_function(task_cfg)
  File "/opt/homebrew/lib/python3.10/site-packages/ultralytics/yolo/cli.py", line 56, in cli
    func(cfg)
  File "/opt/homebrew/lib/python3.10/site-packages/hydra/main.py", line 79, in decorated_main
    return task_function(cfg_passthrough)
  File "/opt/homebrew/lib/python3.10/site-packages/ultralytics/yolo/v8/detect/train.py", line 207, in train
    model.train(**cfg)
  File "/opt/homebrew/lib/python3.10/site-packages/ultralytics/yolo/engine/model.py", line 193, in train
    self.trainer.train()
  File "/opt/homebrew/lib/python3.10/site-packages/ultralytics/yolo/engine/trainer.py", line 185, in train
    self._do_train(int(os.getenv("RANK", -1)), world_size)
  File "/opt/homebrew/lib/python3.10/site-packages/ultralytics/yolo/engine/trainer.py", line 301, in _do_train
    self.loss, self.loss_items = self.criterion(preds, batch)
  File "/opt/homebrew/lib/python3.10/site-packages/ultralytics/yolo/v8/detect/train.py", line 77, in criterion
    return self.compute_loss(preds, batch)
  File "/opt/homebrew/lib/python3.10/site-packages/ultralytics/yolo/v8/detect/train.py", line 168, in __call__
    targets = self.preprocess(targets.to(self.device), batch_size, scale_tensor=imgsz[[1, 0, 1, 0]])
TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.
0

There are 0 answers