Model-based non-prehensile manipulation on a tabletopunder uncertain dynamics

Data-driven planar pushing methods have recently gained attention as they aim to reduce extensive engineering effort and increase generalization compared to purely analytical approaches. Existing work focus narrowly on specific capabilities (e.g. a subset of side switching, precision, model-free training, single task etc), limiting their applicability in broader manipulation scenarios. In this work, we present a model-based framework for non-prehensile tabletop pushing, motivated by the goal of using a single learned model to handle multiple tasks and objectives without retraining. Our approach is built on a novel recurrent architecture that implicitly captures object–environment interaction dynamics through a GRU-based model, enhanced with additional non-linear layers to improve expressivity and stability. To complement this architecture, we design a unique state–action representation tailored to pushing dynamics, which allows the model to generalize effectively across, uncertain dynamics, push lengths, and task requirements. For control, we employ a sampling-based model predictive controller, specifically Model Predictive Path Integral (MPPI), which leverages the learned dynamics model to generate adaptive, taskoriented actions. The proposed framework enables side-switching during pushes, supports variable push lengths, and incorporates diverse objectives such as precise positioning, trajectory following, and obstacle avoidance. The model is trained from simulation using domain randomization to facilitate sim-to-real transfer. We first conduct extensive studies to evaluate the architectural components, demonstrating that our design decisions lead to improved prediction accuracy and more stable rollouts. We then assess the full system in both simulation and real-world experiments, using a Franka Panda robot with markerless visual tracking. Our results show high success rates for precise positioning under strict position and orientation thresholds, and strong performance in trajectory tracking and obstacle-avoidance tasks. Importantly, the versatility of our framework is demonstrated by solving multiple tasks simply by modifying the objective function of the controller,without requiring any model retraining. One drawback of our approach is that we focus on a single type of object. However, we further improve our framework based on this by training a model capable of handling wider push lengths and by designing a balanced controller that selects the most effective action, thereby reducing the number of steps required to achieve longer-horizon objectives.