Toward a simple yet efficient cost function for the optimization of Gaussian process regression model hyperparameters
Source abstract
Abstract
FFLUX is a novel machine-learnt force field using pre-trained Gaussian process regression (GPR) models to predict energies and multipole moments of quantum atoms in molecular dynamic simulations. At the heart of FFLUX lies the program FEREBUS, a Fortran90 and OpenMP-parallelized regression engine, which trains and validates GPR models of chemical accuracy. Training a GPR model is about finding an optimal set of model hyperparameters (θ). This time-consuming task is usually accomplished by maximizing the marginal/concentrated log-likelihood function LLy|x,θ, known as the type-II maximum likelihood approach. Unfortunately, this widespread approach can suffer from the propagation of numerical errors, especially in the noise-free regime, where the expected correlation betweenLLy|x,θ̂ [maximized value of theLLy|x,θfunction] and the models’ performance may no longer be valid. In this scenario, the LLy|x,θ function is no longer a reliable guide for model selection. While one could still rely on a pre-conditioner to improve the condition number of the covariance matrix, this choice is never unique and often comes with increased computational cost. Therefore, we have equipped FEREBUS with an alternatively simple, intuitive, viable, and less error-prone protocol called “iterative hold-out cross-validation” for the optimization of θ values. This protocol involves (1) a stratified random sampling of both training and validation sets, followed by (2) an iterative minimization of the predictive RMSE(θ) of intermediary models over a sufficiently large validation set. Its greatest asset is the assurance that the optimization process keeps reducing the generalization error of intermediary GPR models on unseen datasets, something that maximizing LLy|x,θ does not guarantee.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning in Materials Science · Mass Spectrometry Techniques and Applications · Computational Drug Discovery Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Chemistry, The University of Manchester , Oxford Road, Manchester M13 9PL, United Kingdom
Funding
Engineering and Physical Sciences Research Council
References (47)
Show source-provided references
1. Transferability and additivity of dihedral parameters in polarizable and nonpolarizable empirical force fields10.1002/jcc.24012 ↗
2. Long-timescale molecular dynamics simulations of protein structure and function10.1016/j.sbi.2009.03.004 ↗
3. Developing accurate intramolecular force fields for conjugated systems through explicit coupling terms10.1007/s00214-018-2254-8 ↗
4. Structural ensembles of intrinsically disordered proteins depend strongly on force field: A comparison to experiment10.1021/acs.jctc.5b00736 ↗
5. Multipolar electrostatics10.1039/c3cp54829e ↗
6. Escaping atom types in force fields using direct chemical perception10.1021/acs.jctc.8b00640 ↗
7. Development and testing of a general amber force field10.1002/jcc.20035 ↗
8. Combining machine learning and computational chemistry for predictive insights into chemical systems10.1021/acs.chemrev.1c00107 ↗
9. Machine learning force fields10.1021/acs.chemrev.0c01111 ↗
10. Optimal construction of a fast and accurate polarisable water potential based on multipole moments trained by machine learning10.1039/b905748j ↗
11. Creating Gaussian process regression models for molecular simulations using adaptive sampling10.1063/5.0017887 ↗
12. Producing chemically accurate atomic Gaussian process regression models by active learning for molecular simulation10.1002/jcc.27006 ↗
13. Gaussian process regression models for predicting atomic energies and multipole moments10.1021/acs.jctc.2c00731 ↗
14. ICHOR: A modern pipeline for producing Gaussian process regression models for atomistic simulations10.1039/d2ma00673a ↗
15. DL_FFLUX: A parallel, quantum chemical topology force field10.1021/acs.jctc.1c00595 ↗
16. FEREBUS: A high-performance modern Gaussian process regression engine10.1039/d2dd00082b ↗
17. CP2K: An electronic structure and molecular dynamics software package–Quickstep: Efficient and accurate electronic structure calculations10.1063/5.0007045 ↗
18. 2018
19. 2009
20. 2019
21. T. L. Fletcher , Ph.D. thesis, School of Chemistry, University of Manchester, Great Britain, 2014.
22. 2013
23. FEREBUS: Highly parallelized engine for kriging training10.1002/jcc.24486 ↗
24. Gaussian process regression for geometry optimization10.1063/1.5017103 ↗
25. 2018
26. J. Mol. Graphics Modell. · 2006 · 25 · 24710.1016/j.jmgm.2005.12.005 ↗
27. Electrostatic forces: Formulas for the first derivatives of a polarizable, anisotropic electrostatic potential energy function based on machine learning10.1021/ct500565g ↗
28. Interacting quantum atoms: A correlated energy decomposition scheme based on the quantum theory of atoms in molecules10.1021/ct0501093 ↗
29. Passive sampling in reproducing kernel Hilbert spaces using leverage scores10.1016/j.sigpro.2022.108603 ↗
30. Interpolation and extrapolation of global potential energy surfaces for polyatomic systems by Gaussian processes with composite kernels10.1021/acs.jctc.9b00700 ↗
31. Gaussian process regression with Student-t likelihood
32. Robust Gaussian process regression with a student-t likelihood
33. Gaussian process modelling with Gaussian mixture likelihood10.1016/j.jprocont.2019.06.007 ↗
34. Predictive approaches for choosing hyper-parameters in Gaussian process10.1162/08997660151134343 ↗
35. How cross-validation can go wrong and what to do about it10.1017/pan.2018.39 ↗
36. Improved barebones particle swarm optimization with neighborhood search and its application on ship design10.1155/2013/175848 ↗
37. Grey wolf optimizer10.1016/j.advengsoft.2013.12.007 ↗
38. A novel random walk grey wolf optimizer10.1016/j.swevo.2018.01.001 ↗
39. Bat algorithm: A novel approach for global engineering optimization10.1108/02644401211235834 ↗
40. Firefly algorithm: Recent advances and applications10.1504/ijsi.2013.055801 ↗
41. Firefly algorithm with chaos10.1016/j.cnsns.2012.06.009 ↗
42. The problem with Sturges’ rule for constructing histograms
43. On the histogram as a density estimator: L2 theory10.1007/bf01025868 ↗
44. Accuracy and tractability of a kriging model of intramolecular polarizable multipolar electrostatics and its application to histidine10.1002/jcc.23333 ↗
45. Comparison of iterative pressure solvers for turbulent flow over hills10.1002/asl.145 ↗
46. Prediction of conformationally dependent atomic multipole moments in carbohydrates10.1002/jcc.24215 ↗
47. GPyTorch: Blackbox matrix-matrix Gaussian process inference with GPU acceleration