Deep neural electronic wave functions
Quantum Monte Carlo allows for exact numerical solutions of the electronic Schrödinger equation, but so far has been limited by the flexibility of available wave-function ansatzes. This can change with deep learning
We dreamed up the project on Friday, June 15, 2018, on the bus ride from the IPAM reunion meeting at Lake Arrowhead to the LAX airport. It turned out that both Frank and I had been thinking about taking the seminal proof-of-principle work by Carleo and Troyer on representing wave functions of lattice systems with neural networks,1 and pushing it to practical applications in quantum chemistry. A perfect research project was born.
Variational quantum Monte Carlo (QMC) is a seductively simple method to calculate the ground-state properties of molecules and solids. One only has to provide a so-called wave function ansatz, an arbitrary parametric prescription for the wave function, evaluate the energy by sampling electron positions from this wave function, and adjust the parameters in the ansatz to lower the energy. The crux of the matter is to have a sufficiently powerful ansatz, capable of expressing the intricate many-body correlations in electronic wave functions. Traditional ansatzes are relatively simple mathematical functions that aren’t sufficiently flexible, and QMC typically requires additional, more sophisticated methods beyond the variational approach to achieve high accuracy.
In the last decade, machine learning made massive inroads into artificial intelligence, signal processing, and natural sciences. In particular deep learning—the use of deep neural networks to learn patterns in data—claimed the most visible achievements, such as beating the world champion in Go, winning the protein folding competition, or generating texts that fool many to have been written by a human. In essence, deep neural networks are highly expressive parametric functions that can be fitted by minimizing a given loss function. And this is precisely the focal point where variational QMC and deep learning meet—the former provides the loss function in the form of the variational principle, while the latter supplies a powerful wave function ansatz in the form of a deep neural network.
Our initial attempts at representing wave functions with neural networks were met with frustration when we couldn’t reach the accuracy of even the standard Hartree–Fock method of quantum chemistry. Not even on the smallest molecules, where such a calculation takes only a few seconds on a modern computer. This eventually motivated us to build the Hartree–Fock baseline and other components enforcing the correct physics of wavefunctions into our architecture, which we dubbed PauliNet—a neural network that obeys the Pauli exclusion principle.2 We were then quite surprised in September 2019 when we found out that researchers at DeepMind, who pursued the same idea in parallel, were able to reach impressive accuracy without building any physics into their architecture, called FermiNet, although at the cost of using much larger networks and hence requiring more computational resources.3
This brings us to the question of the tradeoff between accuracy and computational cost, which is omnipresent in computational chemistry. The full configuration-interaction (FCI) method can solve the electronic Schrödinger equation exactly in principle—with the accuracy limited only by the basis-set size—but at the price of being so computationally demanding that calculations on molecules with more than 20 electrons become unfeasible. In contrast, the Hartree–Fock method is relatively cheap, but also inaccurate, sometimes catastrophically so. The most successful methods are the outliers in this trend, density functional theory (DFT) being perhaps the best known example. Of all the potentially exact methods, variational QMC offers very slow scaling of computational cost with system size, and now with the help of deep learning, it can also reach superb accuracy. We believe that PauliNet, FermiNet, some combination of the two, or rather deep QMC in general, will become another outlier in terms of computational efficiency next to DFT.
- Carleo, G. & Troyer, M. Solving the Quantum Many-Body Problem with Artificial Neural Networks. Science 355, 602–606 (2017).
- Hermann, J., Schätzle, Z. & Noé, F. Deep-neural-network solution of the electronic Schrödinger equation. Nat. Chem. (2020).
- Pfau, D., Spencer, J. S., Matthews, A. G. de G. & Foulkes, W. M. C. Ab initio solution of the many-electron Schrödinger equation with deep neural networks. Phys. Rev. Research 2, 033429 (2020).