Multi-class NMS

[PareshKamble]

🚀 Feature

Can we have multi-class NMS so that we can assign only one class per object?

Motivation

While detecting more than one classes, most of the times, one object is detected as class A as well as class B with same or almost similar bounding box values. Ideally, one object should be labelled as only one class. Due to similar inter-class features, such multiple labelling occur.

[github-actions]

👋 Hello @PareshKamble, thank you for your interest in 🚀 YOLOv5! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://www.ultralytics.com or email Glenn Jocher at glenn.jocher@ultralytics.com.

Requirements

Python 3.8 or later with all requirements.txt dependencies installed, including torch>=1.7. To install run:

$ pip install -r requirements.txt

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), testing (test.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@PareshKamble you are referring to single-label NMS, not single-class NMS. Multi-label NMS is enabled by default as it increases mAP. You can disable it by setting this line to False:

yolov5/utils/general.py

Line 356 in ace3e02

multi_label = nc > 1 # multiple labels per box (adds 0.5ms/img)

You can enable agnostic NMS here, to apply NMS across all classes rather than per class:

yolov5/utils/general.py

Line 340 in ace3e02

def non_max_suppression(prediction, conf_thres=0.25, iou_thres=0.45, classes=None, agnostic=False, labels=()):

[PareshKamble]

Thank you @glenn-jocher
I shall try that out right away.

[glenn-jocher]

BTW agnostic NMS can be used like this:
python detect.py --agnostic

yolov5/detect.py

Line 159 in ace3e02

parser.add_argument('--agnostic-nms', action='store_true', help='class-agnostic NMS')

[iceisfun]

@glenn-jocher We have been having issues with non_max_suppression multi class trying to constrain nms to a group of classes. It seems like there is a bug in the non_max_suppression and we have been trying to hunt it down with only partial success.

I've been trying to get a full test case that exposes the issue

One useful thing here is using pickle to serialize the full pre nms data to help run minimal test cases. We just have not come up with enough concrete data for a issue report.

[glenn-jocher]

@iceisfun yes if you can spot a reproducible issue please let us know. There are two main settings in NMS, agnostic (default False) and multi-label (default True). Agnostic runs NMS on all boxes togethor rather than per class, multi-class can assign multiple classes of detection per box. You can modify these two settings below.

@PareshKamble you are referring to single-label NMS, not single-class NMS. Multi-label NMS is enabled by default as it increases mAP. You can disable it by setting this line to False:

yolov5/utils/general.py

Line 356 in ace3e02

multi_label = nc > 1 # multiple labels per box (adds 0.5ms/img)

You can enable agnostic NMS here, to apply NMS across all classes rather than per class:

yolov5/utils/general.py

Line 340 in ace3e02

def non_max_suppression(prediction, conf_thres=0.25, iou_thres=0.45, classes=None, agnostic=False, labels=()):

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[alucor-it]

@iceisfun yes if you can spot a reproducible issue please let us know. There are two main settings in NMS, agnostic (default False) and multi-label (default True). Agnostic runs NMS on all boxes togethor rather than per class, multi-class can assign multiple classes of detection per box. You can modify these two settings below.

@PareshKamble you are referring to single-label NMS, not single-class NMS. Multi-label NMS is enabled by default as it increases mAP. You can disable it by setting this line to False:

yolov5/utils/general.py

Line 356 in ace3e02

multi_label = nc > 1 # multiple labels per box (adds 0.5ms/img)

You can enable agnostic NMS here, to apply NMS across all classes rather than per class:

yolov5/utils/general.py

Line 340 in ace3e02

def non_max_suppression(prediction, conf_thres=0.25, iou_thres=0.45, classes=None, agnostic=False, labels=()):

Hi,
Should I change "multi_label &= nc > 1" to multi_label &= False ?

Thanks

[abhiagwl4262]

@glenn-jocher We have been having issues with non_max_suppression multi class trying to constrain nms to a group of classes. It seems like there is a bug in the non_max_suppression and we have been trying to hunt it down with only partial success.

I've been trying to get a full test case that exposes the issue

One useful thing here is using pickle to serialize the full pre nms data to help run minimal test cases. We just have not come up with enough concrete data for a issue report.

I am also facing the issues with nms -
[OP_ERROR] <class 'RuntimeError'> : Trying to create tensor with negative dimension -1905798436: [-1905798436]
when I call torchvision.ops.nms

[glenn-jocher]

@abhiagwl4262 👋 hi, thanks for letting us know about this possible problem with YOLOv5 🚀. We've created a few short guidelines below to help users provide what we need in order to get started investigating a possible problem.

How to create a Minimal, Reproducible Example

When asking a question, people will be better able to provide help if you provide code that they can easily understand and use to reproduce the problem. This is referred to by community members as creating a minimum reproducible example. Your code that reproduces the problem should be:

• ✅ Minimal – Use as little code as possible to produce the problem
• ✅ Complete – Provide all parts someone else needs to reproduce the problem
• ✅ Reproducible – Test the code you're about to provide to make sure it reproduces the problem

For Ultralytics to provide assistance your code should also be:

• ✅ Current – Verify that your code is up-to-date with GitHub master, and if necessary git pull or git clone a new copy to ensure your problem has not already been solved in master.
• ✅ Unmodified – Your problem must be reproducible using official YOLOv5 code without changes. Ultralytics does not provide support for custom code ⚠️.

If you believe your problem meets all the above criteria, please close this issue and raise a new one using the 🐛 Bug Report template with a minimum reproducible example to help us better understand and diagnose your problem.

Thank you! 😃

[DoSquared]

@iceisfun yes if you can spot a reproducible issue please let us know. There are two main settings in NMS, agnostic (default False) and multi-label (default True). Agnostic runs NMS on all boxes togethor rather than per class, multi-class can assign multiple classes of detection per box. You can modify these two settings below.

@PareshKamble you are referring to single-label NMS, not single-class NMS. Multi-label NMS is enabled by default as it increases mAP. You can disable it by setting this line to False:

yolov5/utils/general.py

Line 356 in ace3e02

multi_label = nc > 1 # multiple labels per box (adds 0.5ms/img)

You can enable agnostic NMS here, to apply NMS across all classes rather than per class:

yolov5/utils/general.py

Line 340 in ace3e02

def non_max_suppression(prediction, conf_thres=0.25, iou_thres=0.45, classes=None, agnostic=False, labels=()):

Where can i disable the multi label please? in which script?