Comparison of Reinforcement Learning Models for a Testbed for UAV Target Tracking

This research paper compares the performance of curiosity-driven Reinforcement Learning (RL) models on a testbed. More specifically, the Advantage Actor-Critic (A2C), Asynchronous Advantage Actor-Critic (A3C), and Proximal Policy Optimization (PPO) algorithms are employed to improve the performance of Unmanned Aerial Vehicles (UAVs) for target tracking. Experimental simulations were conducted in FlightGear using the JSBSim Flight Dynamics Model (FDM), where one A2C-controlled target UAV was tracked in real-time by three PPO and A3C-based UAV agents. In traditional RL methods, sparse rewards can lead to inefficient exploration and slow convergence, especially in complex and dynamic environments. Our results show that an Intrinsic Curiosity Module (ICM) significantly accelerates the learning process of UAV target tracking compared to standard A3C and PPO methods, as it enables the agent to explore and learn more effectively in the absence of frequent external rewards. Introducing an ICM improves both the efficiency of exploration and the overall accuracy of the target tracking task, leading to faster convergence and better policy generalization. This research paper highlights the potential for curiosity-driven exploration in real-world applications, particularly in scenarios where the reward signal is infrequent or difficult to engineer.