Drone racing trains neural AI for space

In the quest to push the boundaries of spacecraft autonomy, researchers at ESA and TU Delft have turned to an unconventional testing ground: drone racing. By unleashing neural network-controlled drones in a high-speed, unpredictable environment, they aim to evaluate the performance of these cutting-edge AI systems before deploying them in space missions.

The key challenge lies in bridging the gap between simulations and real-world conditions. While neural networks excel in controlled simulations, their true mettle must be tested against the noise, turbulence, and unforeseen factors that arise in dynamic environments like drone racing.

Using the university’s ‘Cyber Zoo’ as a testbed, the researchers pit autonomous drones against human-piloted counterparts, each vying for the fastest time through a tightly choreographed course. The drones are equipped with neural networks trained through a process called “behavioral cloning,” where they learn by observing expert examples over prolonged periods.

As Sebastien Origer, a Young Graduate Trainee at ESA’s Advanced Concepts Team, explains, “Our alternative end-to-end Guidance & Control Networks (G&CNets) approach involves all the work taking place on the spacecraft. Instead of sticking to a single set course, the spacecraft continuously replans its optimal trajectory, starting from its current position, which proves to be much more efficient.”

The performance metrics gathered from these high-stakes drone races provide invaluable insights into the robustness and adaptability of the neural networks under realistic conditions. By overcoming the challenges posed by the Cyber Zoo’s dynamic environment, the researchers can build trust in their AI systems, laying the groundwork for future space exploration missions that demand unparalleled autonomy and resilience.

The autonomous drone racing scenarios at the TU Delft Cyber Zoo push the limits of neural network performance in dynamic, unpredictable environments. Unlike controlled simulations, the drone races introduce real-world variables such as turbulence, sensor noise, and unforeseen obstacles that the AI systems must navigate and adapt to on the fly.

One of the key challenges lies in bridging the “reality gap” between simulations and physical systems. As Christophe De Wagter, principal investigator at TU Delft, explains, “We deal with this by identifying the reality gap while flying and teaching the neural network to deal with it. For example, if the propellers give less thrust than expected, the drone can notice this via its accelerometers. The neural network will then regenerate the commands to follow the new optimal path.”

The races pit autonomous drones, controlled by the G&CNets neural networks, against human-piloted drones in a timed course through the Cyber Zoo. The neural networks must continuously replan the optimal trajectory based on the drone’s current position and environmental conditions, accounting for factors like energy efficiency, obstacle avoidance, and time constraints.

To further challenge the AI systems, the researchers introduce dynamic obstacles, reconfigurable course layouts, and varying lighting conditions during the races. These unpredictable elements force the neural networks to adapt their decision-making in real-time, mimicking the challenges of space exploration scenarios where unexpected events can occur at any moment.

By subjecting the G&CNets to these high-stakes, high-speed races, the researchers can evaluate their performance under extreme conditions, identify potential weaknesses, and refine the neural network architectures and training processes. The insights gained from these autonomous drone racing scenarios are invaluable for building trust in the AI systems and paving the way for future space missions that demand unparalleled autonomy and resilience.