How to Calculate IoU for Polygon Segmentation images in YOLOv8 using Python

Question

Please help me to calculate IoU for Polygon Segmentation of images segmented by yolov8 segment module.

I have a predicted mask that is segmented by yolov8 and a ground truth mask.

The ground truth mask has been obtained after converting json file to mask (using shape_to_mask() utility function). The JSON file is the annotated pixel coordinates file. Annotation has been done using the labelme tool.

Here is what I have tried:

pred_mask=(rs[0].masks.masks[0].numpy()*255).astype("uint8")

#some code ommitted here

    
with open(pt_json,"r",encoding="utf-8") as f:
    dj=json.load(f)
grount_truth_mask=labelme.utils.shape_to_mask((dj['imageHeight'],dj['imageWidth']),dj['shapes'][0]['points'],shape_type=None,line_width=1,point_size=1)

pred_area=st.resize(pred_area,(640,480),order=0,preserve_range=True,anti_aliasing=False)
mask_area=st.resize(mask_area,(640,480),order=0,preserve_range=True,anti_aliasing=False)


pred_area=pred_area.ravel().copy()
mask_area=mask_area.ravel().copy()

pred_area=pred_area==1
mask_area=mask_area==1


intersection = np.sum(pred_area*mask_area)
union=pred_area.sum()+mask_area.sum()-intersection
iou=np.mean(intersection/union)

I am getting IoU=0.0 for all images.

Answer 1

you may want to try provided function from YOLO: ultralytics.utils.metrics.mask_iou(mask1, mask2, eps=1e-07).

Details can be seen here: https://docs.ultralytics.com/reference/utils/metrics/#ultralytics.utils.metrics.mask_iou

Answer 2

I have yet to experiment with yolov8, so I cant provide any specific answer. However, in more general terms, to compute IoU when you have the ground truth and prediction masks, you can simply use numpy.

import numpy as np 

# Create two random masks
y_true = np.greater_equal(np.random.normal(loc=0, scale=1, size=(512,512)), 0.0)    # Random true mask
y_pred = np.greater_equal(np.random.normal(loc=0, scale=1, size=(512,512)), 0.0)    # Random pred mask

# Compute the Intersection over Union
intersection = np.logical_and(y_true, y_pred)                                       # Compute Intersection
union = np.logical_or(y_true, y_pred)                                               # Compute the Union
IoU = np.divide(intersection.sum(), union.sum())                                    # Compute Intersection divided by Union
print(f"Now the intersection over union is computed to be {IoU:.2f}")               # Print results
>>> Now the intersection over union is computed to be 0.33

But again, I wouldn't know if there are any other, better optimized ways to directly use the information from the JSON and thus avoid conversions.