Inversion of geophysical data supported by Reinforcement Learning

Local optimisation algorithms allow exploring a given region of the search space and getting close to, or finding exactly, the extrema of the function in that region (local optima). However, we are more frequently interested in finding global optima of a given objective function. For this purpose, there are many global, or quasi-global, exploration strategies that allow performing an expanded exploration of the model space. Unfortunately, their effectiveness is counterbalanced by their slow convergence towards the solution and, consequently, by long or prohibitive computation times. In order to improve their effectiveness, it is possible to combine global optimisation techniques with local search methods. Based on such hybrid approach, in this paper, we introduce a novel geophysical inversion strategy aimed at optimising the exploration of the model space with the support of Reinforcement Learning (RL) techniques. These include a suite of Machine Learning methods concerned with how intelligent agents ought to take actions by interacting with their environment with the objective to maximise the notion of Cumulative Reward (CR). Following the Bellman equation, this is given by the contribution of a Short-Term and a Long-Term reward, balanced by a trade-off term called discount factor. Adopting these RL concepts, our goal is to teach an artificial agent to search for the global optimum of the cost function, limiting the risk of being trapped in local optima. This can be done by relating the CR to various indicators of the inversion performance, including the trend of the objective function over a limited number of iterations. Using multiple inversion tests on synthetic as well as real geo-electric data, we verified that our approach fits the purpose of expanding the exploration of the model space, finding optimal solutions (global optima) even in complex 3D inversion scenarios.