Vision-based swarm tracking of multiple UAVs in air-to-air scenarios

In recent years, drone technology has seen rapid advancement, making multi-object tracking in air-to-air swarm scenarios a critical capability in drone-related applications. With low-altitude airspace gradually opening in countries such as China, effective tracking technologies can be used for aerial route planning, traffic regulation, and the interception of unauthorized drones. These applications have significant implications for public safety and operational efficiency in urban air mobility.

Visual tracking methods offer a unique combination of low deployment cost, high information density, and strong anti-interference capabilities. A key task in swarm tracking is to identify the location of each drone in a continuous video stream and assign consistent identities to each one. However, conventional multi-object tracking algorithms often lack explicit global modeling of trajectory history and struggle when applied to drone swarms, especially in air-to-air perspectives. This leads to two critical challenges:

First, extracting appearance features at the instance level becomes highly difficult due to the homogeneous nature of drones in a swarm, which often belong to the same model. The visual differences between individual drones at a single frame are minimal, rendering most existing appearance extractors ineffective in distinguishing between them. Second, drones often exhibit nonlinear motion in 3D space—sudden accelerations, sharp turns, or unpredictable maneuvers—especially in air-to-air views. These motions make conventional motion estimations unreliable, leading to frequent identity switches and unstable tracking.

Addressing these challenges, a research team led by Dr. Ren Jin and Dr. Tao Song from School of Aerospace Engineering, Beijing Institute of Technology, has proposed a novel algorithmic framework that leverages multi-frame spatio-temporal features for more robust and accurate tracking of swarm UAVs.

The team published their work in the Chinese Journal of Aeronautics on April 30, 2025.

“We found that although drones in a swarm look almost identical at a given moment, their motion postures vary subtly over time,” said Dr. Jin, assistant professor at the School of Aerospace Engineering. “Rotary-wing and fixed-wing drones change positions via rotor angles or wing adjustments. By capturing these multi-frame posture differences, our appearance extraction module improves target discrimination dramatically.”

The team also developed a multi-frame motion difference accumulation strategy to capture variations in drone movement across time. “In air-to-air scenarios, UAVs are typically small and move nonlinearly. Standard distance-based matching fails in such cases. Instead, tracking long-term motion divergence patterns proved to be far more effective,” Dr. Jin explained.

To bring these modules together, the researchers proposed a multi-frame homogeneous association algorithm that combines appearance and motion cues across several frames to associate new detections with existing tracks. They also created AIRMOT, the first visual dataset designed specifically for homogeneous swarm UAV tracking in air-to-air settings.

Experimental results show that the proposed method achieves state-of-the-art performance across several datasets, including AIRMOT, MOT-FLY, and UAVSwarm. “Our algorithm achieved the best results in all current public swarm drone tracking benchmarks,” Dr. Jin noted. “This validates our multi-frame strategy and its suitability for real-world drone swarms.”

However, the team acknowledges that further work is needed to evaluate the system’s robustness in more complex scenarios, such as heterogeneous swarms, harsh weather conditions, or urban air traffic environments. Future directions include multi-view swarm tracking, multi-modal data fusion, and high-resolution swarm detection for dense or long-distance scenarios.

Other contributors include Zhaochen Chu, Tao Song, Defu Lin, from the School of Aerospace Engineering at Beijing Institute of Technology in Beijing, China; Hao Shen from the China Research and Development Academy of Machinery Equipment in Beijing, China; Maolong Lv from the  Air Traffic Control and Navigation College, Air Force Engineering University in Xi'an, China.

Original Source

Zhaochen CHU, Tao SONG, Ren JIN, Defu LIN, Hao SHEN, Maolong LYU. Vision-based swarm tracking of multiple UAVs in air-to-air scenarios [J]. Chinese Journal of Aeronautics, 2025, https://doi.org/10.1016/j.cja.2025.103558.

About Chinese Journal of Aeronautics 

Chinese Journal of Aeronautics (CJA) is an open access, peer-reviewed international journal covering all aspects of aerospace engineering, monthly published by Elsevier. The Journal reports the scientific and technological achievements and frontiers in aeronautic engineering and astronautic engineering, in both theory and practice. CJA is indexed in SCI (IF = 5.7, Q1), EI, IAA, AJ, CSA, Scopus.