Dakota Reference Manual  Version 6.4
Large-Scale Engineering Optimization and Uncertainty Analysis
 All Pages
metrics


Compute surrogate quality metrics

Topics

This keyword is related to the topics:

Specification

Alias: diagnostics

Argument(s): STRINGLIST

Default: No diagnostics

Required/Optional Description of Group Dakota Keyword Dakota Keyword Description
Optional cross_validation

Perform k-fold cross validation

Optional press

Leave-one-out cross validation

Description

A variety of diagnostic metrics are available to assess the goodness of fit of a global surrogate to its training data.

The default diagnostics are:

  • root_mean_squared
  • mean_abs
  • rsquared

Additional available diagnostics include

  • sum_squared
  • mean_squared
  • sum_abs
  • max_abs

The keywords press and cross_validation further specify leave-one-out or k-fold cross validation, respectively, for all of the active metrics from above.

Theory

Most of these diagnostics refer to some operation on the residuals (the difference between the surrogate model and the truth model at the data points upon which the surrogate is built).

For example, sum_squared refers to the sum of the squared residuals, and mean_abs refers to the mean of the absolute value of the residuals. rsquared refers to the R-squared value typically used in regression analysis (the proportion of the variability in the response that can be accounted for by the surrogate model). Care should be taken when interpreting metrics, for example, errors may be near zero for interpolatory models or rsquared may not be applicable for non-polynomial models.