1Blurb::
2Correction approaches for surrogate models
3Description::
4
5Some of the surrogate model types support the use of correction
6factors that improve the local accuracy of the surrogate models.
7
8The \c correction specification specifies that the approximation will
9be corrected to match truth data, either matching truth values in the
10case of \c zeroth_order matching, matching truth values and gradients
11in the case of \c first_order matching, or matching truth values,
12gradients, and Hessians in the case of \c second_order matching. For
13\c additive and \c multiplicative corrections, the correction is local
14in that the truth data is matched at a single point, typically the
15center of the approximation region. The \c additive correction adds a
16scalar offset (\c zeroth_order), a linear function (\c first_order),
17or a quadratic function (\c second_order) to the approximation to
18match the truth data at the point, and the \c multiplicative
19correction multiplies the approximation by a scalar (\c zeroth_order),
20a linear function (\c first_order), or a quadratic function (\c
21second_order) to match the truth data at the point. The \c additive
22\c first_order case is due to \cite Lew00
23and the \c multiplicative \c first_order case is commonly known as
24beta correction \cite Haftka1991. For the \c combined
25correction, the use of both additive and multiplicative corrections
26allows the satisfaction of an additional matching condition, typically
27the truth function values at the previous correction point (e.g., the
28center of the previous trust region). The \c combined correction is
29then a multipoint correction, as opposed to the local \c additive and
30\c multiplicative corrections. Each of these correction capabilities
31is described in detail in \cite Eld04.
32
33
34The
35correction factors force the surrogate models to match the true
36function values and possibly true function derivatives at the center
37point of each trust region.
38Currently, Dakota supports either zeroth-,
39first-, or second-order accurate correction methods, each of which can
40be applied using either an additive, multiplicative, or combined
41correction function. For each of these correction approaches, the
42correction is applied to the surrogate model and the corrected model
43is then interfaced with whatever algorithm is being employed. The
44default behavior is that no correction factor is applied.
45
46The simplest correction approaches are those that enforce consistency
47in function values between the surrogate and original models at a
48single point in parameter space through use of a simple scalar offset
49or scaling applied to the surrogate model. First-order corrections
50such as the first-order multiplicative correction (also known as beta
51correction \cite Cha93) and the first-order additive
52correction \cite Lew00 also enforce consistency in the gradients and
53provide a much more substantial correction capability that is
54sufficient for ensuring provable convergence in SBO algorithms.
55SBO convergence rates can be further
56accelerated through the use of second-order corrections which also
57enforce consistency in the Hessians \cite Eld04, where the
58second-order information may involve analytic, finite-difference, or
59quasi-Newton Hessians.
60
61Correcting surrogate models with additive corrections involves
62<!-- cannot reuse anchor in a DUPLICATE: \anchor eq-correct_val_add -->
63\f{equation}
64\hat{f_{hi_{\alpha}}}({\bf x}) = f_{lo}({\bf x}) + \alpha({\bf x})
65\f}
66where multifidelity notation has been adopted for clarity. For
67multiplicative approaches, corrections take the form
68<!-- cannot reuse anchor in a DUPLICATE: \anchor eq-correct_val_mult -->
69\f{equation}
70\hat{f_{hi_{\beta}}}({\bf x}) = f_{lo}({\bf x}) \beta({\bf x})
71\f}
72where, for local corrections, \f$\alpha({\bf x})\f$ and \f$\beta({\bf x})\f$
73are first or second-order Taylor series approximations to the exact
74correction functions:
75<!-- cannot reuse anchor in a DUPLICATE: \anchor eq-taylor_a -->
76<!-- cannot reuse anchor in a DUPLICATE: \anchor eq-taylor_b -->
77\f{eqnarray}
78\alpha({\bf x}) & = & A({\bf x_c}) + \nabla A({\bf x_c})^T
79({\bf x} - {\bf x_c}) + \frac{1}{2} ({\bf x} - {\bf x_c})^T
80\nabla^2 A({\bf x_c}) ({\bf x} - {\bf x_c})  \\
81\beta({\bf x}) & = & B({\bf x_c}) + \nabla B({\bf x_c})^T
82({\bf x} - {\bf x_c}) + \frac{1}{2} ({\bf x} - {\bf x_c})^T \nabla^2
83B({\bf x_c}) ({\bf x} - {\bf x_c})
84\f}
85where the exact correction functions are
86<!-- cannot reuse anchor in a DUPLICATE: \anchor eq-exact_B -->
87<!-- cannot reuse anchor in a DUPLICATE: \anchor eq-exact_A -->
88\f{eqnarray}
89A({\bf x}) & = & f_{hi}({\bf x}) - f_{lo}({\bf x})    \\
90B({\bf x}) & = & \frac{f_{hi}({\bf x})}{f_{lo}({\bf x})}
91\f}
92Refer to \cite Eld04 for additional details on the derivations.
93
94A combination of additive and multiplicative corrections can provide
95for additional flexibility in minimizing the impact of the correction
96away from the trust region center. In other words, both additive and
97multiplicative corrections can satisfy local consistency, but through
98the combination, global accuracy can be addressed as well. This
99involves a convex combination of the additive and multiplicative
100corrections:
101\f[ \hat{f_{hi_{\gamma}}}({\bf x}) = \gamma \hat{f_{hi_{\alpha}}}({\bf x}) +
102(1 - \gamma) \hat{f_{hi_{\beta}}}({\bf x})  \f]
103where \f$\gamma\f$ is calculated to satisfy an additional matching
104condition, such as matching values at the previous design iterate.
105
106It should be noted that in both first order correction methods, the
107function \f$\hat{f}(x)\f$ matches the function value and gradients of
108\f$f_{t}(x)\f$ at \f$x=x_{c}\f$. This property is necessary in proving that
109the first order-corrected SBO algorithms are provably convergent to a
110local minimum of \f$f_{t}(x)\f$. However, the first order correction
111methods are significantly more expensive than the zeroth order
112correction methods, since the first order methods require computing
113both \f$\nabla f_{t}(x_{c})\f$ and \f$\nabla f_{s}(x_{c})\f$. When the SBO
114strategy is used with either of the zeroth order correction methods,
115or with no correction method, convergence is not guaranteed to a local
116minimum of \f$f_{t}(x)\f$. That is, the SBO strategy becomes a heuristic
117optimization algorithm. From a mathematical point of view this is
118undesirable, but as a practical matter, the heuristic variants of SBO
119are often effective in finding local minima.
120
121<b> Usage guidelines </b>
122
123\li Both the \c additive zeroth_order and
124 \c multiplicative zeroth_order correction methods are
125 "free" since they use values of \f$f_{t}(x_{c})\f$ that are normally
126 computed by the SBO strategy.
127
128\li The use of either the \c additive first_order method or
129 the \c multiplicative first_order method does not necessarily
130 improve the rate of convergence of the SBO algorithm.
131
132\li When using the first order correction methods, the
133 gradient-related response keywords must be modified to allow either analytic or
134 numerical gradients to be computed. This provides the gradient data
135 needed to compute the correction function.
136
137\li For many computationally expensive engineering optimization
138 problems, gradients often are too expensive to obtain or are
139 discontinuous (or may not exist at all). In such cases the heuristic
140 SBO algorithm has been an effective approach at identifying optimal
141 designs \cite Giu02.
142
143Topics::
144Examples::
145Theory::
146Faq::
147See_Also::
148