Multi-fidelity Surrogates

---

Multi-fidelity (MF) surrogates combine somehow the information of different fidelity models by correcting low-fidelity (LF) models using information from high-fidelity (HF) models. Modern science and engineering use simulations based on numerical models to predict various phenomena ranging from the weather to missile trajectories. These simulations often need to be repeated for a range of input parameters, such as ocean temperatures in the case of the weather, or wind velocities in the case of missile trajectories. When simulations are very expensive and many are required, as in optimization and uncertainty quantification, a way to reduce cost is using surrogate models. The data used to construct surrogates come from simulations based on physical models, which are used to describe the physics of the studied phenomenon as in the case of computational fluid dynamics (CFD), Timoshenko beam theory, experiments, etc. In the surrogate model construction, the quantity of interest is approximated using algebraic functions of the variables of interest. These are obtained by fitting a number of simulations outputs spread over variable space using a design of experiments technique. Some of the more common surrogates include linear regression, Kriging, and support vector regression. MF surrogates become necessary when multiple numerical models to predict the quantity of interest are available and only a few HF simulations and substantial LF simulations can be afforded. HF models usually represent the behavior of a system to acceptable accuracy for the application intended. These models are usually expensive, and their multiple realizations often cannot be afforded, for example in fluid mechanics a highly refined grid Reynolds-average Navier-Stokes (RANS) (e.g.) or direct numerical simulation (DNS) (e.g.). On the other hand, low-fidelity (LF) models are cheaper but less accurate. Examples of LF models in the context of MF are dimensionality reduction (e.g.), simpler physics models (e.g.), coarser discretization (e.g.), partially converged results (e.g.), etc. A surrogate model constructed using data from an LF model is usually called LF surrogate. Similarly, a surrogate built using HF data is called HF surrogate. MF surrogates combine the information of multiple models, with different cost and accuracy (fidelity). MF surrogates are attractive because, in multiple cases, they have proven to achieve the desired accuracy at a lower cost.

Multi-fidelity Surrogates Equations

Given a vector of parameters $\mathbf {x}$, an LF model, $y_{LF}(\mathbf {x} )$, and an HF model, $y_{HF}(\mathbf {x} )$, their surrogates are denoted as ${\hat {y}}_{LF}(\mathbf {x} )$ and ${\hat {y}}_{HF}(\mathbf {x} )$, respectively. In general, ${\hat {\cdot }}$ is used to denote a surrogate model. The additive correction approach assumes that the relation between $y_{LF}$ and $y_{HF}$ is

${\hat {y}}_{MF}(\mathbf {x} )={\hat {y}}_{LF}(\mathbf {x} )+{\hat {\delta }}(\mathbf {x} ),$

(1)

where ${\hat {\delta }}(\mathbf {x} )$ is the surrogate constructed using the difference between the $y_{HF}$ and $y_{LF}$ models at data points where both LF and HF model computations are considered and ${\hat {y}}_{LF}(\mathbf {x} )$ is the LF surrogate constructed with samples from the LF model, $y_{LF}$. The multiplicative approach is

${\hat {y}}_{MF}(\mathbf {x} )={\hat {\mu }}(\mathbf {x} ){\hat {y}}_{LF}(\mathbf {x} ),$

(2)

where ${\hat {\mu }}(\mathbf {x} )$ is the surrogate constructed using the quotient between $y_{HF}$ and $y_{LF}$ models at common data points. A substantial improvement in accuracy was achieved by the introduction of a scalar multiplier to the LF function. The following form of MF surrogates is called the comprehensive approach. For the comprehensive approach, additive and multiplicative corrections are combined,

${\hat {y}}_{MF}(\mathbf {x} )=\rho {\hat {y}}_{LF}(\mathbf {x} )+{\hat {\delta }}(\mathbf {x} ),$

(3)

where $\rho$ is a constant. The decision as to whether to use directly the LF function ($y_{LF}$) or replace it with a surrogate (${\hat {y}}_{LF}$) in Eqs. (1), (2) and (3) depends on its cost, complexity, and the number of needed surrogate evaluations. In some applications (e.g.), the LF function is very cheap so that replacing it with a surrogate leads to unnecessary loss of accuracy. In others (e.g.), the number of simulations needed for constructing an accurate surrogate to the LF function is very high. If the available number of LF data is higher than the number of needed evaluations of the surrogate then it should be considered not to replace the LF function by a surrogate.

Example

The ideal situation for an MF surrogate is when the difference between the HF function and a scaled LF function has a simple behavior that can be captured by a surrogate fitted to a small number of samples. Consider for example an HF model, $y_{HF}$,

${\hat {y}}_{HF}(x)=\sin(20x)+x,\quad 0<x<1$

(4)

and an LF model, $y_{LF}(x)$,

${\hat {y}}_{HF}(x)=\sin(20x)+0.5x+0.5,\quad 0<x<1$

(5)

where $x\in \mathbb {R}$ .

If more than two or three HF samples cannot be afforded but a very large LF number of samples are available, the MF surrogate with additive correction, ${\hat {\delta }}(x)$, will give a perfect fit,

${\hat {\delta }}(x)=0.5x-0.5.$

(6)

Often, however, the complex part requires some scaling. For example, consider that instead of Eq.(1) an LF model, $y_{LF}$, is available as follows

${\hat {y}}_{LF}(x)=0.8\sin(20x)+0.5x+0.5,\quad 0<x<1$

(7)

where $x\in \mathbb {R}$. Then an additive correction will not have the desired performance, because the difference between the two functions is ${\hat {y}}_{LF}(x)=0.2\sin(20x)-0.5x-0.5$, and this cannot be fitted by an accurate surrogate with only two or three samples of the difference. In this case a comprehensive MF surrogate is desired, by using $\rho =1.25$ and ${\hat {\delta }}(x)=0.375x-0.625$. If only 3 HF samples can be afforded at $x_{1}=0.2$, $x_{2}=0.6$ and $x_{3}=1.0$ and 9 LF samples (7 uniformly distributed between 0 and 1 including the endpoints, plus $x=0.2$ and $0.6$), using linear regression with monomial basis functions the results shown in the figure and the table below are obtained.

HF and LF functions (continuous line) along with the three approximations constructed (dashed line). MF comprehensive surrogate predicts the HF function exactly.

Correction functions and root mean square error (RMSE) obtained for each surrogate constructed. The RMSE is calculated between the surrogate and the HF function at 100 equally spaced points.

Surrogate	$\rho$	${\hat {\delta }}(x)$	RMSE
HF	-	$-0.406-1.523x-3.482x^{2}$	$0.9565$
MF additive	-	$-0.581-0.005x+0.768x^{2}$	$0.1913$
MF comprehensive	$1.25$	$-0.625+0.375x$	$0$

The table shows the correction functions for each surrogate and their corresponding root mean square error (RMSE). The error is calculated using 100 validation points uniformly distributed in the interval [0,1]. Note that for the MF comprehensive correction a linear additive correction is enough to have zero RMSE. This is a clear example where MF surrogates work very well. Note also that even the MF additive correction performs well and greatly improves the HF surrogate performance. For this example, a nested design of experiments was used, i.e. HF samples were taken in the same location than LF samples.

The success of the combination of Gaussian process or Kriging surrogates with a Bayesian identification of $\rho$ and $\delta (x)$ by using the maximum likelihood estimation has been studied concluding that the Bayesian approach tends to minimize the bumpiness of ${\hat {\delta }}(x)$ so that it can be fitted accurately with a small number of available HF samples. In other words, the correlation between HF and LF models is improved by minimizing the bumpiness. In one dimension the bumpiness $b$ of a function $f(x)$ is defined as the integral of the square of the second derivative as

$b=\int |f''(x)|^{2}dx.$

(8)

In higher dimensions, the bumpiness over a large number of lines in random directions can be averaged. For the examples examined, the minimum error was very close in value to the one that minimizes the bumpiness

References

This article "Multi-fidelity Surrogates" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:Multi-fidelity Surrogates. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.

---