Decentralized Multi-Agent Reinforcement Learning for Collective Transport

Mukherjee, Apratim

Etd

Decentralized Multi-Agent Reinforcement Learning for Collective Transport

Public Deposited

Deep Reinforcement Learning (DRL) has seen recent success in controlling individual robots, but extending to multi-robot systems presents substantial challenges. Non-stationarity is a critical issue that arises when multiple robots learn concurrently, resulting in an interdependent training process without guaranteed convergence. Addressing non-stationarity often demands unrealistic assumptions such as global information. In this thesis, we investigate four vanilla Reinforcement Learning algorithms applied to a multi-robot system where all of the robots are rigidly connected, commonly referred to as a robot aggregate. We limit the sensing capabilities of the robots to proximity sensors and remove the ability to directly communicate. Our approach is validated by using a collective transport task where robots are pre-attached to an object that must be transported to a predetermined location. We assume that the robots are minimalistic, capable of sensing the target location and nearby obstacles, but without explicit communication abilities, such as message-passing. Instead, they communicate implicitly through the aggregate push-and-pull forces exerted on the object. We apply Centralized Training Decentralized Execution to analyze the coordination capabilities of four prominent deep reinforcement learning algorithms (DQN, DDQN, DDPG, and TD3), investigating the scalability, resilience, and obstacle avoidance capabilities of the robot aggregate in a simulated multi-agent environment. Through a comprehensive study with our experiments, we measure the performance as the successful transport of the object to a goal location within a desired timeframe, highlighting the strengths and weaknesses of each algorithm.

Creator