layout-parser basic api test code throws weird error

Question

I am trying out layout-parser api from

https://layout-parser.readthedocs.io/en/latest/notes/modelzoo.html#example-usage

My detectron2 installation is quite successful as evident from running python -m detectron2.utils.collect_env

So is my torch installation by running pip3 show torch

I have created a first sample as below

import layoutparser as lp
import cv2

image = cv2.imread("data/test-image.jpeg")
image = image[..., ::-1]
model = lp.models.Detectron2LayoutModel(
            config_path ='lp://PubLayNet/faster_rcnn_R_50_FPN_3x/config', # In model catalog
            label_map   ={0: "Text", 1: "Title", 2: "List", 3:"Table", 4:"Figure"}, # In model`label_map`
            extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.8] # Optional
        )
model.detect(image)

Running the above with python3 sample-first.py throws an error

Traceback (most recent call last):
  File "/Users/_dga/ml-git/layout-parser-trial/sample-first.py", line 6, in <module>
    model = lp.models.Detectron2LayoutModel(
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/layoutparser/models/layoutmodel.py", line 124, in __init__
    self._create_model()
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/layoutparser/models/layoutmodel.py", line 149, in _create_model
    self.model = self._engine.DefaultPredictor(self.cfg)
  File "/Users/_dga/ml-git/detectron2/detectron2/engine/defaults.py", line 288, in __init__
    checkpointer.load(cfg.MODEL.WEIGHTS)
  File "/Users/_dga/ml-git/detectron2/detectron2/checkpoint/detection_checkpoint.py", line 62, in load
    ret = super().load(path, *args, **kwargs)
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/fvcore/common/checkpoint.py", line 155, in load
    checkpoint = self._load_file(path)
  File "/Users/_dga/ml-git/detectron2/detectron2/checkpoint/detection_checkpoint.py", line 99, in _load_file
    loaded = self._torch_load(filename)
  File "/Users/_dga/ml-git/detectron2/detectron2/checkpoint/detection_checkpoint.py", line 114, in _torch_load
    return super()._load_file(f)
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/fvcore/common/checkpoint.py", line 252, in _load_file
    return torch.load(f, map_location=torch.device("cpu"))
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/torch/serialization.py", line 815, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/torch/serialization.py", line 1033, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '<'.

It seems to be coming from torch and I am guessing its unable to load the model in question.

I am using MAC M2 with python3.9 for the whole exercise.

After some research on chatgpt and google, I downgraded my detectron2 to 0.5 from 0.6 and have a new issue

Traceback (most recent call last):
  File "/Users/_dga/ml-git/layout-parser-trial/sample-first.py", line 6, in <module>
    model = lp.models.Detectron2LayoutModel(
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/layoutparser/models/layoutmodel.py", line 52, in __new__
    cls._import_module()
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/layoutparser/models/layoutmodel.py", line 40, in _import_module
    cls, m["import_name"], importlib.import_module(m["module_path"])
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/detectron2/engine/__init__.py", line 11, in <module>
    from .hooks import *
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/detectron2/engine/hooks.py", line 19, in <module>
    from detectron2.evaluation.testing import flatten_results_dict
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/detectron2/evaluation/__init__.py", line 2, in <module>
    from .cityscapes_evaluation import CityscapesInstanceEvaluator, CityscapesSemSegEvaluator
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/detectron2/evaluation/cityscapes_evaluation.py", line 11, in <module>
    from detectron2.data import MetadataCatalog
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/detectron2/data/__init__.py", line 2, in <module>
    from . import transforms  # isort:skip
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/detectron2/data/transforms/__init__.py", line 4, in <module>
    from .transform import *
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/detectron2/data/transforms/transform.py", line 36, in <module>
    class ExtentTransform(Transform):
  File "/Users/_dga/Library/Python/3.9/lib/python/site-packages/detectron2/data/transforms/transform.py", line 46, in ExtentTransform
    def __init__(self, src_rect, output_size, interp=Image.LINEAR, fill=0):
AttributeError: module 'PIL.Image' has no attribute 'LINEAR'

Answer 1

1. I tried your sample script, and no error !!!

python -m detectron2.utils.collect_env

-------------------------------  ---------------------------------------------------------------------------
sys.platform                     win32
Python                           3.9.13 (main, Aug 25 2022, 23:51:50) [MSC v.1916 64 bit (AMD64)]
numpy                            1.23.5
detectron2                       0.6 @c:\travail\stackoverflow\detectron2\detectron2
Compiler                         MSVC 193532217
CUDA compiler                    not available
DETECTRON2_ENV_MODULE            <not set>
PyTorch                          2.0.1+cpu @C:\Travail\Anaconda3\envs\chatgpt\lib\site-packages\torch
PyTorch debug build              False
torch._C._GLIBCXX_USE_CXX11_ABI  False
GPU available                    No: torch.cuda.is_available() == False
Pillow                           10.0.0
torchvision                      0.15.2+cpu @C:\Travail\Anaconda3\envs\chatgpt\lib\site-packages\torchvision
fvcore                           0.1.5.post20221221
iopath                           0.1.9
cv2                              4.8.0

detectron2 in version 0.5 uses LINEAR in PILLOW. But from version 10 of PILLOW, LINEAR is deprecated and replaced by BILINEAR, hence the problem you are having.

Th version detectron2==0.6 use BILINEAR, but 0.5 use LINEAR

Try to downgrade pillow from 10 to 9x

2. I change the model to use mask_rcnn_X_101_32x8d_FPN_3x instead of faster_rcnn_R_50_FPN_3x, and ....

   UnpicklingError: invalid load key, '<'.

i tried to pickle each of two models :

GOOD (faster_rcnn_R_50_FPN_3x)

with open('.torch/iopath_cache\\s/dgy9c10wykk4lq4\\model_final.pth', 'rb') as pickle_file:
    pickle.load(pickle_file)

BAD (mask_rcnn_X_101_32x8d_FPN_3x)

with open('.torch/iopath_cache\\s/57zjbwv6gh3srry\\model_final.pth', 'rb') as pickle_file:
    pickle.load(pickle_file)




Traceback (most recent call last):

  Cell In[25], line 2
    pickle.load(pickle_file)

UnpicklingError: invalid load key, '<'.

An article on https://github.com/ying09/TextFuseNet/issues/55 talks about issue when download the file.