Cooperative Active Learning Enables 100-Fold Faster, More Efficient Hazardous-Source Detection

Study conducted by Prof. Wen-Hua CHEN and his research team

Locating an unknown chemical, biological or radiological source is a time-critical task in disaster response, environmental protection and industrial safety. Autonomous ground robots and unmanned aerial vehicles (UAVs) could search areas that are dangerous or inaccessible to people. Yet, these systems face a difficult combination of challenges: the source is unknown and the environment may be turbulent.

Existing autonomous-search methods either move towards the currently estimated source, making use of what is already known, or explore the environment to collect information that may improve the estimate. More advanced dual-control methods attempt to do both, but often depend on computationally intensive Bayesian estimation and large particle filters. This has limited some implementations to remote computing platforms rather than processors carried by the vehicle itself.

To address these limitations, a research team led by Prof. Wen-Hua CHEN, Head and Chair Professor of Robotics and Autonomous Systems of the Department of Aeronautical and Aviation Engineering at The Hong Kong Polytechnic University, proposed a new approach named Cooperative Active Learning-Based Dual Control for Exploration and Exploitation (COAL-DCEE). The study is published in IEEE Transactions on Neural Networks and Learning Systems [1]. Building on the dual-control framework first proposed by Prof. Chen’s team in 2021 [2], the present work introduces a cooperative multi-estimator architecture that substantially reduces computational demand while retaining the dual benefits of exploration and exploitation.

A 100-fold reduction in computation

The central idea is to replace a very large population of Bayesian particles with a much smaller group of cooperating estimators. Each estimator forms a hypothesis about the unknown source, including its location and release rate. The estimators then share information through their collective average. The degree to which their estimates differ provides an indication of uncertainty: if the estimates are widely dispersed, the system has more to learn; if they are clustered, confidence is increasing.

COAL-DCEE combines this estimation process with active path planning. The robot does not simply travel towards the current best estimate. It also selects movements that are expected to reduce uncertainty of the current best estimate and improve future estimates [3]. The system therefore continuously optimally balances:

Exploration: gathering measurements that could make the source estimate more accurate; and

Exploitation: moving towards the believed source location.

The results are notable. COAL-DCEE uses approximately 100 estimators, compared with 10,000 particles in the Bayesian-filtering methods used for comparison. The team reported roughly a 100-fold reduction in computational time, while maintaining comparable or superior search performance. The study also establishes theoretical convergence and performance guarantees under bounded sensor noise and position disturbances caused by factors such as turbulence and control error. This offers a unique feature of the active learning approach proposed in this paper—currently it is almost impossible to have this type of theoretic guarantee for other machine learning or AI algorithms. Numerical simulations and real experimental data were used to validate the approach.

Figure 1. System-environment interaction in autonomous search using COAL-DCEE

Figure 1 presents the central architecture of COAL-DCEE. On the left, the environment estimator receives the UAV’s position and sensor measurements. It then produces a group of estimated source and environmental parameters representing the UAV’s current understanding of the unknown environment.

On the right, the path planner uses these estimates to determine the UAV’s next movement. A candidate movement changes the UAV’s position and, consequently, the measurements it obtains from the physical environment which affects the following estimation/learning. By predicting possible information gain of an action, active learning is realised by optimally selecting an action that not only moves towards the current best estimated location of the source, but more importantly actively probe the unknown environment [3]. The new measurements of the implementation of the selected movement are fed back into the estimator, forming a closed loop between learning, decision-making and the environment.

This arrangement allows the UAV to learn while it searches. For example, if a weak chemical signal suggests that the source may be located to the north but the estimate remains uncertain, the UAV does not necessarily fly directly north. It may first move across the suspected plume to obtain measurements that distinguish between competing possibilities. Once confidence improves, it can direct more of its movement towards the estimated source.

The system therefore avoids two extremes. A purely exploitative strategy may follow an inaccurate estimate, while a purely exploratory strategy may gather information without reaching the source efficiently. COAL-DCEE brings both priorities into a single decision-making process.

The multi-estimator design is especially important. Instead of using thousands of computationally expensive particles, the UAV quantifies uncertainty through the spread of a small number of physically meaningful source estimates. This makes the dual objective practical for onboard processing, particularly for real-time implementation on robots like UAVs with a limited computational power.  The theoretical analysis further shows that the estimators converge to a bounded neighbourhood of the true source parameters, while the UAV converges to a bounded neighbourhood of the source.

29 seconds instead of nearly 45 minutes

One of COAL-DCEE’s most significant practical advantages is its computational efficiency.

The numerical study compared six approaches, including CLEE, Entrotaxis, model predictive control, the original DCEE and COAL-DCEE. The Bayesian approaches used 10,000 particles and restricted movement to a small set of directions and fixed step lengths to maintain computational feasibility. By contrast, COAL-DCEE used only 100 cooperative estimators and allowed movement with arbitrary direction and length.

Figure 2. Comparison of steady-state tracking performance (blue bars) and computational time (red bars) over 200 trials

Figure 2 shows the resulting difference in computational demand. Across 200 trials, COAL-DCEE required only 29.15 seconds of total computation. Entrotaxis required 2,697.66 seconds, while DCEE required 2,827.96 seconds. The proposed method therefore reduced the computational burden by approximately two orders of magnitude.

This distinction matters for UAV operations. Computational time reflects the processing burden of the algorithm, whereas mission time depends on factors such as travel distance, vehicle speed and sensor sampling time. A method that requires extensive processing may need to rely on a remote computation centre, increasing communication requirements and potentially reducing operational independence.

COAL-DCEE’s efficiency makes dual-control search more suitable for portable processors carried by UAVs and other unmanned vehicles. Notably, the computational gain did not come at the expense of active learning or search performance. In the simulation study, the method remained resilient when 40% of sensor readings were assumed to be nondetections.

Real-data validation with fewer samples

The team also evaluated COAL-DCEE using a real experimental dataset. The dataset contains 340 sequential frames of a changing concentration field, providing a demanding test of autonomous search under time-varying environmental conditions.

The original DCEE method was used for comparison. Although DCEE has demonstrated strong search performance, it relies on a much larger population of particles than COAL-DCEE.

Figure 3. Representative search path of COAL-DCEE on real dataset. The subfigures were taken at 1, 10, 60 and 115 sample instances, respectively. The grey shade represents the instantaneous concentration field at the current time step, red lines indicate the paths of the search agent, the green dots are the estimated source position and the black dots are nonzero measurements

Figure 3 illustrates representative COAL-DCEE search behaviour in four panels. The green dots show the evolving estimated source locations, while the red line represents the UAV’s search trajectory. Initially, the green estimates are distributed across several possible source locations. As measurements accumulate, the estimates progressively concentrate. The UAV’s trajectory responds to this changing belief: it first probes the environment, then increasingly moves towards the region in which the source is believed to be located. In the later panels, both the estimated source position and the UAV trajectory converge near the true source.

Across 200 experimental trials, COAL-DCEE achieved a 98% success rate, compared with 99% for DCEE. However, it required only 84.80 samples on average, compared with 119.26 for DCEE. The average mission times were 1,328.7 seconds for COAL-DCEE and 1,560.9 seconds for DCEE.

The computational difference was even more pronounced. COAL-DCEE completed the 200 searches in 15.26 seconds, whereas DCEE required approximately 1,560 seconds. COAL-DCEE therefore achieved essentially comparable search success while using only 100 estimators instead of 10,000 particles and with substantially lower processing demand.

A broader role for dual control

COAL-DCEE demonstrates how a UAV can turn uncertainty into an active decision-making resource. Rather than treating exploration and exploitation as separate strategies, it enables the vehicle to learn about its environment while moving towards its operational objective.

The dual-control framework pioneered by Prof. Chen’s team provides a promising mechanism to implement active learning. Traditional machine learning focuses on how to develop more powerful and/or efficient learning algorithms with provided datasets (e.g. leaning by chance as no good learning outcome if a dataset is of low information value). In contrast, active learning seeks to generate most informative data through carefully planning actions to enable efficient learning. It is particularly suitable for embodied intelligence where the interaction between an agent and its physical environment is at its core. The dual control framework has also been extended beyond autonomous search. Recent work has applied the same principles to maximum power point tracking for solar energy systems under varying weather conditions, efficiency optimisation of electric motors, wave energy converters and automated robotic assembly tasks. Together, these developments demonstrate the broader potential of dual control for goal-oriented systems operating under uncertainty.

Prof. Chen was recognised by Stanford University as one of the top 2% most-cited scientists worldwide (career-long & single-year) in the field of industrial engineering and automation for eight consecutive years, from 2019 to 2026. He was ranked 26th globally and 2nd in the UK in the field of Industrial Engineering and Automation in 2025. An internationally recognised authority in control theory and autonomous systems, he is a Fellow of the Institute of Electrical and Electronics Engineers, the Institution of Mechanical Engineers and the Institution of Engineering and Technology. He was awarded the prestigious EPSRC Established Career Fellowship (2020–2025), the only fellowship of its kind awarded to date in Control Engineering, and was appointed a Changjiang Chair Professor by China’s Ministry of Education in 2013.

As an internationally renowned scholar at the forefront of control and autonomous vehicles, Prof. Chen brings more than 20 years of experience in the research and development of unmanned aircraft systems. He currently leads major initiatives at the University focused on the low-altitude economy, with particular emphasis on autonomous forced landing, which investigates how to safely recover a drone in emergence situations. He also serves as a Co-Principal Investigator for the RGC Collaborative Research Fund project “Coordinated Autonomy and Smart Infrastructure for Low Altitude Economy Systems”, which continues to produce leading results.

[1] Z. Li, W. -H. Chen, J. Yang and C. Liu, Cooperative Active Learning-Based Dual Control for Exploration and Exploitation in Autonomous Search, IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 2, pp. 2221-2233, Feb. 2025, doi: 10.1109/TNNLS.2024.3349467.
[2] W.-H. Chen, C. Rhodes, and C. Liu, Dual control for exploitation and exploration (DCEE) in autonomous search, Automatica, vol. 133, Nov. 2021, Art. no. 109851.
[3] W-H Chen (2022). ‘Perspective View of Autonomous Control in Unknown Environment: Dual Control for Exploitation and Exploration vs Reinforcement Learning’. Neurocomputing. 497(1), 50-63. 

Prof. Wen-Hua CHEN Head and Chair Professor of Robotics and Autonomous Systems, Department of Aeronautical and Aviation Engineering Co-Director,  UM-PolyU Joint Research Centre for Robotics and Embodied Intelligence Director, Research Centre for Low Altitude Economy