DeepMind Has Trained an AI to Control Nuclear Fusion

[Hamartia Antidote]

DeepMind Has Trained an AI to Control Nuclear Fusion

The Google-backed AI firm taught a reinforcement learning algorithm to control the fiery plasma inside a tokamak nuclear fusion reactor.

www.wired.com

THE INSIDE OF a tokamak—the doughnut-shaped vessel designed to contain a nuclear fusion reaction—presents a special kind of chaos. Hydrogen atoms are smashed together at unfathomably high temperatures, creating a whirling, roiling plasma that’s hotter than the surface of the sun. Finding smart ways to control and confine that plasma will be key to unlocking the potential of nuclear fusion, which has been mooted as the clean energy source of the future for decades. At this point, the science underlying fusion seems sound, so what remains is an engineering challenge. “We need to be able to heat this matter up and hold it together for long enough for us to take energy out of it,” says Ambrogio Fasoli, director of the Swiss Plasma Center at École Polytechnique Fédérale de Lausanne in Switzerland.

That’s where DeepMind comes in. The artificial intelligence firm, backed by Google parent company Alphabet, has previously turned its hand to video games and protein folding, and has been working on a joint research project with the Swiss Plasma Center to develop an AI for controlling a nuclear fusion reaction.

In stars, which are also powered by fusion, the sheer gravitational mass is enough to pull hydrogen atoms together and overcome their opposing charges. On Earth, scientists instead use powerful magnetic coils to confine the nuclear fusion reaction, nudging it into the desired position and shaping it like a potter manipulating clay on a wheel. The coils have to be carefully controlled to prevent the plasma from touching the sides of the vessel: this can damage the walls and slow down the fusion reaction. (There’s little risk of an explosion as the fusion reaction cannot survive without magnetic confinement).

But every time researchers want to change the configuration of the plasma and try out different shapes that may yield more power or a cleaner plasma, it necessitates a huge amount of engineering and design work. Conventional systems are computer-controlled and based on models and careful simulations, but they are, Fasoli says, “complex and not always necessarily optimized.”

DeepMind has developed an AI that can control the plasma autonomously. A paper published in the journal Nature describes how researchers from the two groups taught a deep reinforcement learning system to control the 19 magnetic coils inside TCV, the variable-configuration tokamak at the Swiss Plasma Center, which is used to carry out research that will inform the design of bigger fusion reactors in the future. “AI, and specifically reinforcement learning, is particularly well suited to the complex problems presented by controlling plasma in a tokamak,” says Martin Riedmiller, control team lead at DeepMind.

The neural network—a type of AI setup designed to mimic the architecture of the human brain—was initially trained in a simulation. It started by observing how changing the settings on each of the 19 coils affected the shape of the plasma inside the vessel. Then it was given different shapes to try to re-create in the plasma. These included a D-shaped cross section close to what will be used inside ITER (formerly the International Thermonuclear Experimental Reactor), the large-scale experimental tokamak under construction in France, and a snowflake configuration that could help dissipate the intense heat of the reaction more evenly around the vessel.

DeepMind’s AI was able to autonomously figure out how to create these shapes by manipulating the magnetic coils in the right way—both in the simulation and when the scientists ran the same experiments for real inside the TCV tokamak to validate the simulation. It represents a “significant step,” says Fasoli, one that could influence the design of future tokamaks or even speed up the path to viable fusion reactors. “It’s a very positive result,” says Yasmin Andrew, a fusion specialist at Imperial College London, who was not involved in the research. “It will be interesting to see if they can transfer the technology to a larger tokamak.”


Fusion offered a particular challenge to DeepMind’s scientists because the process is both complex and continuous. Unlike a turn-based game like Go, which the company has famously conquered with its AlphaGo AI, the state of a plasma constantly changes. And to make things even harder, it can’t be continuously measured. It is what AI researchers call an “under–observed system.”

“Sometimes algorithms which are good at these discrete problems struggle with such continuous problems,” says Jonas Buchli, a research scientist at DeepMind. “This was a really big step forward for our algorithm, because we could show that this is doable. And we think this is definitely a very, very complex problem to be solved. It is a different kind of complexity than what you have in games.”

This isn’t the first time artificial intelligence has been used to try to control nuclear fusion. Since 2014, Google has been working with California-based fusion company TAE Technologies to apply machine learning to a different type of fusion reactor—speeding up the analysis of experimental data. Research at the Joint European Torus (JET) fusion project in the UK has used AI to try to predict the behavior of plasma. The concept even appears in fiction: In 2004’s Spider-Man 2, villain Doc Ock creates an AI-powered, brain-controlled exoskeleton to control his experimental fusion reactor, which works great until the AI takes over his mind and starts killing people.

All in all, the collaboration with DeepMind could prove most pivotal as fusion reactors get bigger. Although physicists have a good handle on how to control the plasma in smaller-scale tokamaks through conventional methods, the challenge will only increase as scientists try to make power-plant-sized versions viable. Progress has been slow but steady. Last week the JET project made a breakthrough, setting a new record for the amount of energy extracted from a fusion project, and construction is ongoing at France’s ITER, an international collaboration which will become the world’s largest experimental fusion reactor when it fires up in 2025.

“The more complex and high performance the tokamak, the greater the need to control more quantities with higher and higher reliability and accuracy,” says Dmitri Orlov, an associate research scientist at the Center for Energy Research in San Diego. An AI-controlled tokamak could be optimized to control the transfer of heat out of the reaction to the walls of the vessel and prevent damaging “plasma instabilities.” The reactors themselves could be redesigned to take advantage of the tighter control offered by reinforcement learning.

Ultimately, Fasoli says, the collaboration with DeepMind could allow researchers to push the boundaries and accelerate the long journey toward fusion power. “AI would enable us to explore things that we wouldn’t explore otherwise, because we can take risks with this kind of control system we wouldn’t dare take otherwise,” he says. “If we are sure that we have a control system that can take us close to the limit but not beyond the limit, we can actually explore possibilities that wouldn’t otherwise be there for exploring.”