Have you ever seen a drone that can hold strong winds？ Caltech teaches drones to fly against the wind with 12 minutes of flight data.

Machine heart report

Editor: Chen Ping, Du Wei

Have you ever seen a drone that can withstand a strong gust? Researchers at the California Institute of Technology use deep neural networks to make drones no longer afraid of the wind.

When the wind is strong enough to blow out the umbrella, the drone is steady, like this:

Flying against the wind is a part of air flight. On a large scale, when pilots land their planes, the wind speed may bring them challenges. On a small scale, gusts will also affect the flight of drones.

At present, drones either fly under controlled conditions and have no wind; Or operated by humans using a remote control. Unmanned aerial vehicles are controlled by researchers to fly in formation in the open sky, but these flights are usually carried out under ideal conditions and environments.

However, in order for UAVs to perform necessary but daily tasks autonomously, such as delivering packages, UAVs must be able to adapt to wind conditions in real time.

In order to make the UAV have better maneuverability when flying in the wind, a group of engineers from California Institute of Technology developed a deep neural network Neural-Fly, an artificial intelligence tool, which can keep the UAV agile in windy conditions, and only need to update some key parameters to help the UAV cope with new and unknown wind conditions in real time.

At the beginning of the article, we have seen the wind resistance of the UAV. The following shows a quadrotor UAV. With the help of the tool developed by this research, it can complete the figure-eight shuttle operation when the wind speed reaches 27 mph:

Watch the drone flying against the wind from another angle;

In order to allow the UAV to shuttle in various wind speeds, data is essential. This research needs little data. After obtaining the flight data in just 12 minutes, the automatic quadrotor UAV equipped with Neural-Fly learned how to respond to strong winds. "The amount of data used is very small," said Michael O’Connell, a graduate student in the Department of Aeronautics and Astronautics at California Institute of Technology and one of the authors of the paper.

The study was published in the journal Science Robotics on Wednesday.

Address: https://www.science.org/doi/10.1126/scirobotics.abm6597.

ArXiv Address: https://arxiv.org/pdf/2205.06908.pdf

In the Neural-Fly experimental training, the UAV flies in the wind tunnel with six different wind speeds, the fastest of which is 13.4 mph.

Based on these data, the team created a deep neural network, and then enabled their aircraft to skillfully maneuver in the same wind tunnel, such as enlarging the gate in a figure-of-eight pattern or cruising through two oval gates. The speed of the drone in the test is faster than it encountered in the training: about 27 miles per hour. This is the maximum wind speed that the wind tunnel can produce, pointed out Shi Guanya, the author of this paper and a graduate student of California Institute of Technology. Except that only a small amount of data is needed, the software only runs on Raspberry Pi, which is a cheap computing device.

Soon-Jo Chung, a professor of aerospace, control and dynamics systems at California Institute of Technology and a correspondent, said that the error rate they saw was 2.5 to 4 times better than that of the existing SOTA technology.

The picture below shows Shi Guanya, Soon-Jo Chung and Michael O’Connell from left to right. Congratulations, Shi Guanya will join the Robotics Institute of Computer College of Carnegie Mellon University as an assistant professor in September 2023.

The researchers said that this technology will be used for drone express delivery or flying cars in the future.

Overview of Neural-Fly technology

If unmanned aerial vehicle (UAV) wants to realize continuous commercialization, it is very important to perform safe and accurate flight maneuvers in dynamic high-speed wind. However, because the relationship between different wind conditions and their influence on the maneuverability of UAV have not been deeply studied, it is challenging to design an efficient robot controller using traditional control design methods.

"Neural-Fly" proposed by researchers of California Institute of Technology is a data-driven method. As a trajectory tracking controller based on deep learning, it can learn to adapt to rapidly changing wind conditions. Figure 3(A) below shows the data collection process; 3(B) When the wind speed is 13.3km/h (3.7m/s), input and label the training data; 3(C) is the distribution of input data and label data under different wind conditions.

On-line Adaptive+Off-line Meta-learning

Figure 2 below is an overview of the Neural-Fly method, which shows that it has made progress in adaptive flight control and robot control based on deep learning. Under the dynamic wind conditions of standard UAV, Neural-Fly can track the centimeter-level position error of flexible and challenging trajectory. Specifically, this method mainly consists of two parts, namely, offline learning stage and online adaptive control stage for real-time online learning.

For the off-line learning stage, the researchers developed Domain Adversarial Invariant Meta-learning (DAML), which learned the expression of deep neural network (DNN) which is independent of wind conditions in a data-efficient way. This representation adapts to different wind conditions by updating the linear coefficients of a set of mixed depth neural network outputs.

DAIML also has data efficiency, and only 12 minutes of flight data under six different wind conditions are used to train the deep neural network. DAIML contains several key features, which can not only improve data efficiency, but also provide information by the downstream online adaptive control stage.

It is worth noting that DAIML uses spectral normalization to control the Lipschitz property of deep neural network, so as to improve the generalization ability of unknown data and provide closed-loop stability and robustness. In addition, DAIML also uses discriminant network to ensure that the learned representation is wind-invariant, and the wind-related information is only included in the linear coefficient that adapts to the online control stage.

For the on-line adaptive control stage, researchers developed a regularized compound adaptive control law, which is based on the basic understanding of how the learned representation interacts with the closed-loop control system and has strict theoretical support.

Specifically, the adaptive law uses the combination of position tracking error term and aerodynamic prediction error term to update the linear coefficient related to wind. This method effectively ensures the stability and rapid adaptation to any wind conditions and the robustness to imperfect learning. Although this adaptive control law can be used with many learned models, the accurate representation learned from DAIML is helpful to further improve the adaptive speed.

experimental result

As far as the effect of Neural-Fly is concerned, it is 66% higher than that of nonlinear tracking controller, 42% higher than that of L_1 adaptive controller and 35% higher than that of incremental nonlinear dynamic inversion (INDI) controller. All these results are achieved by using standard quadrotor UAV hardware, and the default attitude control of PX4 is running at the same time.

The tracking performance of Neural-Fly is competitive even compared with related work without external wind interference and using more complicated hardware (such as on-board optical sensor that needs 10 times control frequency and uses DC motor speed feedback).

Table 1 below shows the error tracking statistics of different methods under different wind conditions.

The researchers compared Neural-Fly with two variants, namely Neural-Fly-Transfer and Neural-Fly-Constant. Among them, NF-Transfer uses representations trained from data of different UAVs, while NF-Constant only uses adaptive control laws with trivial non-learning basis.

Neural-Fly-Transfer shows the robustness to UAV configuration changes and model mismatch. NF-Constant, L_1 and INDI methods all directly adapt to unknown dynamics without assuming the underlying physical structure, and they have similar performance.

Finally, the researcher proves that Neural-Fly can realize a series of functions, allowing the UAV to follow the trajectory and fly over the low-altitude door quickly under the condition of strong wind, as shown in Figure 1 below.

Reference link:

https://www.science.org/doi/10.1126/scirobotics.abm6597

https://www.popsci.com/technology/neural-fly-ai-drone-in-wind/? THE END

Please contact Ben WeChat official account for authorization.

Contribute or seek reports: content@jiqizhixin.com.

Original title: "Have you ever seen a drone that can hold strong winds? Caltech teaches drones to fly against the wind with 12 minutes of flight data.

Read the original text