I'm using YOLOv8 to track animals from trail camera footage. The tracking nicely identifies the same animal from frame to frame. However, what type of animal it is works on a per-frame basis and can give different results frame to frame, particularly if there are several similar animals it could be detected as (deer and pig seems a common confusion for some reason!) and the confidence level varies wildly. I'd like to combine all the top 5 detected classes for a track from each individual frame to get an overall classification of the tracked animal.
However, the Boxes result objects only contain the highest detection each frame, in contrast to Probs from the whole image classification that have a top5().
Is there a way to access the top 5 classes in the results of doing detection prediction?
I could extract the contents of the detected box from the original image, and rerun a classification pass to get a top5. However, this would be inefficient as it would be a two-pass which isn't ideal with You Only Look Once!
There is no easy way of doing this. This is why:
You start with a set of detections provided by the model
NMS then:
a. Sorts the detections by their confidences
b. The box with the highest score is selected and all other boxes that have a significant overlap (based on a predefined threshold, like Intersection over Union or IoU) are deleted
The previous step is repeated until no overlaps are achieved and these are the bboxes that the model outputs.
Hence there is no such thing as TOP5 for detections. You would need to modify the NMS function to achieve this.