Unsupervised multi-object discovery and tracking using memory-augmented slot attention

Learning object-centric representations from static images is a promising research direction in the field of deep learning. However, adapting this approach to videos poses certain challenges due to the necessity of capturing the temporal dynamics of video content. Recent works have made significant progress in object discovery within synthetic video datasets. Nevertheless, these works do not fully exploit the motion of objects in videos and temporal cues. In this thesis, we aim to enhance the performance of object-centric representation learning on video frames by using the temporal information more carefully. To achieve this goal, we propose a new unsupervised learning method that utilizes a memory-augmented slot attention model for multi-object discovery and tracking. The key component of our approach is the integration of memory slots, which store information from past video frames, alongside object slots into the learning architecture. Object slots simultaneously attend to both memory slots for information from past frames and the current image input. Training memory slots requires longer video sequences. However, the most suitable learning structures for this, recurrent neural networks (RNNs), are not very effective in learning long-term temporal data due to the issues of exploding and/or vanishing gradients. To train our memory-augmented model more effectively on long video sequences, we employ truncated back-propagation through time. Experiments conducted on synthetic yet realistic video datasets have yielded promising results, indicating that memory slots significantly improve multi-object tracking and object segmentation performance. Our fully unsupervised learning method contributes to the problem of object-centric representation learning in videos and opens up new possibilities in this field.