Tools
AERIALIST: UAV Test Bench
Aerialist (unmanned AERIAL vehIcle teST bench) is a novel test bench for UAV software that automates all the necessary UAV testing steps: setting up the test environment, building and running the UAV firmware code, configuring the simulator with the simulated world properties, connecting the simulated UAV to the firmware and applying proper UAV configurations at startup, scheduling and executing runtime commands, monitoring the UAV at runtime for any issues, and extracting the flight log file after the test completion. With Aerialist, we aim to provide researchers with an easy platform to automate tests on both simulated and real UAVs, allowing them to do experiments required to overcome the UAV simulation-based testing challenges. Aerialist is used to organize the UAV Testing Competition.
Sajad Khatiri, Sebastiano Panichella, and Paolo Tonella, "Simulation-based testing of unmanned aerial vehicles with Aerialist"In Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings, 2024.
[PDF/ArXiv]
SURREALIST: Simulation-based Test Case Generation for UAVs in the Neighborhood of Real Flights
Simulation-based testing represents a fundamental testing practice for Unmanned aerial vehicles (UAVs), but the testing scenarios considered in software-in-the-loop testing may not be representative of the actual scenarios experienced in the field. Here, we propose Surrealist (teSting Uavs in the neighboRhood of REAl fLIghtS), a novel search-based approach that analyses logs of real UAV flights and automatically generates simulation-based tests in the neighborhood of such real flights, thereby improving the realism and representativeness of the simulation-based tests. This is done in two steps: first, Surrealist faithfully replicates the given UAV flight in the simulation environment, generating a simulation-based test that mirrors a pre-logged real-world behavior. Then, it smoothly manipulates the replicated flight conditions to discover slightly modified flight scenarios that are challenging or trigger misbehaviors of the UAV under test in simulation. Surrealist internally uses Aerialist, A UAV test bench developed on top of PX4, to evalute the UAV test cases in the simulation environment.
Sajad Khatiri, Francisco Eli Vina Barrientos, Maximilian Wulf, Paolo Tonella, Sebastiano Panichella, "Bridging Research and Practice in Simulation-based Testing of Industrial Robot Navigation Systems", IEEE/ACM International Conference on Automated Software Engineering (ASE), 2025.
[PDF/ArXiv]
UAV Testing Competition
Unmanned Aerial Vehicles (UAVs) equipped with onboard cameras and various sensors have already demonstrated the possibility of autonomous flights in real environments, leading to great interest in various application scenarios: crop monitoring, surveillance, medical and food delivery. Over the years, support for UAV developers has increased with open-access projects for software and hardware, such as the autopilot support provided by PX4 and Ardupilot. However, despite the necessity of systematically testing such complex and automated systems to ensure their safe operation in real-world environments, there has been relatively limited investment in this direction so far. The UAV Testing Competition organized jointly by the International Conference on Software Testing, Verification and Validation (ICST) and Search-Based and Fuzz Testing (SBFT) workshop is an initiative designed to inspire and encourage the Software Testing Community to direct their attention toward UAVs as a rapidly emerging and crucial domain. The joint call is meant to help interested authors/participants reduce travel costs by selecting the most convenient and close venue.
Sajad Khatiri, Tahereh Zohdinasab, Prasun Saurabh, Dmytro Humeniuk, Sebastiano Panichella, "ICST Tool Competition 2025 – UAV Testing Track", International Conference on Software Testing, Verification, and Validation, ICST 2025.
[PDF/ArXiv]
[PDF/ArXiv]