Inquiry cli: --classes filter option to trackers eval

[aqualiquid]

Hi! I was running trackers track --classes person on MOT17 and noticed that trackers eval doesn't seem to have a --classes option.

When tracking with --classes person, the prediction file only contains person detections, but the GT file still has all classes — so metrics like MOTA/HOTA end up being skewed.

Is there a way to filter by class when running eval that I'm missing? If not, would a --classes option be a addition?

[tstanczyk95]

Hi @aqualiquid, thanks for your interest!
In fact, we follow the standards of the original TrackEval, where in case of MOT17, only the pedestrian class objects are taken into account when evaluating the tracker performance, as established by the MOT17 authors. As it is mentioned in the MOT17 paper (it is called MOT16, yet is the reference for MOT17), section 2.1, p. 3:

In the third group, we annotate all moving vehiclessuch as cars, bicycles, motorbikes and non-motorized vehicles (e.g. strollers), as well as other potential occluders. These annotations will not play any role in the evaluation [...]

However, we might also consider adding a feature for configuring the particular object classes to be evaluated.

[aqualiquid]

Thanks! My concern is about the prediction side, not GT.

When running trackers track without --classes, ByteTrack tracks all possible classes (e.g., person, car, bag, etc.) and pred.txt contains all of them. But GT only has pedestrians.

Those extra non-pedestrian predictions in pred.txt show up as FP and skew evaluation (i.e., MOTA/HOTA). Would a --classes option on trackers eval to filter predictions by class be an addition?