Reinforcing good decisions for global multiple-object tracking

Multiple Object Tracking (MOT) involves the challenge of determining the paths followed by individual objects within a video. Most previous works in MOT achieve tracking using the tracking-by-detection paradigm. The challenge is in resolving ambiguous correspondences between detected objects across the temporal axis. This has lead to learning to extract and utilize more discriminative features. Such distinguishing cues can be extracted either visually, geometrically, spatio-temporally or from velocity. Most relevant to our work are approaches that utilize the global-reasoning capacity of Graph-Neural-Networks (GNNs). However, many networks that take this approach overlook the high number of False Negatives that hinder the tracking performance. In our work, we start with an abundant set of detections. The abundance of detections capture as many False Negatives as possible, but also increases the number of False Positives. We formulate our method with REINFORCE, and a task-specific reward formulation to reason through the vast combinatorial space and find the optimal trajectories. Taking inspiration from two-stage-object-detectors, we develop a Track-of-Interest head to generate track-proposals via a learnable random walk sampler, allowing the network to present the best set of tracks. We show that using a proposal-based approach in this way allows the agent to leverage the global scope of data available to GNNs rather than local, pairwise relations between detected objects. We show an improvement in commonly used tracking metrics on the MOT17 dataset.