Object-detection explanations move beyond similarity: masking-induced detection loss becomes a 0-to-1 importance score
Synopsis
Object detectors must explain both what was detected and where, yet the widely used masking-based method D-RISE weights pixels by bounding-box and class similarity and offers no numeric score tied to detection performance. This work instead defines each pixel's importance by how much random masking raises the detection loss, counting classification confidence and localisation quality together, and normalises it to 0 to 1. On COCO and two industrial datasets, the resulting saliency maps concentrate inside predicted boxes, and deleting top-ranked pixels produces a measurable drop in detection confidence.
Interpretation
Explanations for object detection can now carry a numeric score that corresponds directly to detection performance, not just a heatmap. The earlier detection-oriented masking method D-RISE built a similarity from bounding-box overlap, class probability and objectness to weight masks, yielding relative importance without an absolute, loss-linked score. On COCO, a screw-detection dataset and a screen-and-frame detection dataset, the same trained YOLO detector generated explanations with D-RISE and with the new score, and the new method produced saliency maps aligned with predicted boxes under identical masking parameters.
The procedure masks random regions of an image and measures how much the target detection's loss rises; a larger rise marks a more critical region. The detection loss combines the predicted probability of the target class with the intersection-over-union between the original box and the box predicted on the masked image, penalising both misclassification and localisation error. Experiments used lightweight YOLO11 models from Ultralytics, YOLO11s on COCO and YOLO11n on the two industrial datasets, with roughly 2500 to 2800 random masks per dataset depending on object scale.
Scores are normalised to 0 to 1 so they can be compared across pixels and images and aggregated at object, image and model level. Raw importance is the difference in detection loss before and after masking, set to zero when masking adds no loss. At object level, a COCO truck detection reached an importance overlap ratio of 0.461 and a deletion AUC of 0.230, while an industrial screen-frame detection reached 0.454 and 0.331, with removal of the most salient pixels producing a measurable drop in detection confidence.
Perspective
The demonstrated scope is COCO plus the screw-detection and screen-and-frame detection industrial datasets from the RoboSAPIENS project, with lightweight YOLO11 detectors from Ultralytics; quantitative results come from representative detections chosen to illustrate behaviour across object types and scales, not from a large-scale benchmark.
The detection loss used is a simplified proxy rather than the exact training loss of modern detectors such as YOLO; deletion AUC varies with object size and scene complexity, and smaller objects such as screws and persons show lower importance overlap ratios; sensitivity to mask count, resolution and sampling randomness is left for future work.
