In optimization, algorithms included in the program search for the model parameters that yield the best value of an index, also known as objective function. HEC-HMS includes 17 objective functions. Search methods function by iteratively adjusting parameter values to increase or decrease the objective function, if the goal is maximization or minimization, respectively. For example, a smaller value of Peak-weighted RMSE yields more favorable results meaning that the goal is to minimize the Peak-weighted RMSE objective function value. HEC-HMS includes response objective functions (Peak Elevation, Peak Discharge, and Discharge Volume). Response objective functions are used to describe the hydrologic response of the system (e.g. maximize the peak discharge of a hydrologic element), rather than quantify the difference between observed and simulated time series. The following Objective Functions are available in the Optimization Trial when the Goal is set to Minimization:

  • The Mean of Absolute Residuals is used to minimize the average distance between the observed and simulated values. 
  • The Mean of Squared Residuals is used to minimize the average distance between the observed and simulated values, with larger weight to larger errors.
  • The Root Mean Square Error (RMSE) is used to minimize the average distance between the observed and simulated, with larger weights given to larger errors. Mean Squared Error (MSE) is the most widely used performance metric in statistics. 
  • The Peak-Weighted Root Mean Square Error (USACE, 1998) is identical to the calibration objective function included in computer program HEC-1 (USACE, 1998). It compares all ordinates, squaring differences, and it weights the squared differences. The weight assigned to each ordinate is proportional to the magnitude of the ordinate. Ordinates greater than the mean of the observed hydrograph are assigned a weight greater than 1.00, and those smaller, a weight less than 1.00. The peak observed ordinate is assigned the maximum weight. The sum of the weighted, squared differences is divided by the number of computed hydrograph ordinates; thus, yielding the mean squared error. Taking the square root yields the root mean squared error. This function is an implicit measure of comparison of the magnitudes of the peaks, volumes, and times of peak of the two hydrographs.
  • The Peak-Weighted Variable Power is similar to the Peak-Weighted Root Mean Square Error in that the function assigns more weight to larger data values. The absolute residuals are raised to an exponent of the range normalized value plus 1. 
  • The Percent Error in Peaks Discharge measures only the goodness-of-fit of the simulatedh peak to the observed peak. It quantifies the fit as the absolute value of the difference, expressed as a percentage, thus treating overestimates and underestimates as equally undesirable. It does not reflect errors in volume or peak timing. This objective function is a logical choice if the information needed for designing or planning is limited to peak flow or peak stages. This might be the case for a floodplain management study that seeks to limit development in areas subject to inundation, with flow and stage uniquely related.
  • The Sum of Absolute Residuals (Stephenson, 1979) compares each ordinate of the computed hydrograph with the observed, weighting each equally. The index of comparison, in this case, is the difference in the ordinates. However, as differences may be positive or negative, a simple sum would allow positive and negative differences to offset each other. In hydrologic modeling, both positive and negative differences are undesirable, as overestimates and underestimates as equally undesirable. To reflect this, the function sums the absolute differences. Thus, this function implicitly is a measure of fit of the magnitudes of the peaks, volumes, and times of peak of the two hydrographs. If the value of this function equals zero, the fit is perfect: all computed hydrograph ordinates equal exactly the observed values. Of course, this is seldom the case.
  • The Sum of Squared Residuals (Diskin and Simon, 1977) is commonly-used objective function for model calibration. It too compares all ordinates, but uses the squared differences as the measure of fit. Thus a difference of 10 m3/sec "scores" 100 times worse than a difference of 1 m3/sec. Squaring the differences also treats overestimates and underestimates as undesirable. This function too is implicitly a measure of the comparison of the magnitudes of the peaks, volumes, and times of peak of the two hydrographs.
  • The Time-Weighted Root Mean Square Error is used to minimize the average distance between the observed and simulated values, with larger weight given to data near the end of the time window.

Minimization Goal Objective Functions For Optimization

Objective Function

Equation

Mean of Absolute Residuals

Z=\frac{\sum_{i=1}^{N} |q_s(i) - q_o(i)|}{N}

Mean of Squared Residuals

Z=\frac{\sum_{i=1}^{N} [q_s(i) - q_o(i)]^2}{N}

Root Mean Square Error (RMSE)

Z=\sqrt\frac{\left[\sum_{i=1}^{N}\left(q_{o}(i)-q_{s}(i)\right)^{2}\right]}{N}

Peak-Weighted RMSE (USACE, 1998)

Z=\left\{\frac{1}{N}\left[\sum_{i=1}^{N}\left(q_{o}(i)-q_{s}(i)\right)^{2}\left(\frac{q_{o}(i)+q_{o}(m e a n)}{2 q_{o}(m e a n)}\right)\right]\right\}^{1/2}

Peak-Weighted Variable Power

Z=\sum_{i=1}^{N} |q_s(i) - q_o(i)|^{\frac{q_o(i) - q_o(min)}{q_o(peak) - q_o(min)} + 1}

Percent Error in Peak Discharge

Z=100 \frac{q_{s}(p e a k)-q_{o}(p e a k)}{q_{o}(p e a k)}

Sum of Absolute Residuals (Stephenson, 1979)

Z=\sum_{i=1}^{N}\left|q_{o}(i)-q_{s}(i)\right|

Sum of Squared Residuals (Diskin and Simon, 1977)

Z=\sum_{i=1}^{N}\left[q_{o}(i)-q_{s}(i)\right]^{2}

Time-Weighted RMSE

Z=\left\{\frac{1}{N}\left[\sum_{i=1}^{N}\left(q_{o}(i)-q_{s}(i)\right)^{2}\left(\frac{i}{N - 1}\right)\right]\right\}^{1/2}

The following Objective Functions are available in the Optimization Trial when the Goal is set to Maximization:

  • The Coefficient of Determination, also known as R2, describes the proportion of the variation in the dependent variable that is described by the independent variable. 
  • The Index of Agreement (Willmott, 1981) is a measure of the degree to which a model's predictions are error free and was proposed as an alternative to Pearson's correlation coefficient and the Coefficient of Determination.
  • The Modified Kling Gupta Efficiency is a multi-objective alternative to mean squared error and Nash Sutcliffe Efficiency (NSE). MKGE can be decomposed into three terms: (1) correlation coefficient r, (2) bias ratio β, and (3) variability ratio γ. The value of MKGE gives the lower limit of the three components (r, βγ). The variability ratio represents the variability of prediction errors. In the MKGE equation (shown in the table below), the radicand is the Euclidean distance of the effects of the mean, variance, and correlation of the time series.
  • The Normalized Nash Sutcliffe is a modified version of the Nash Sutcliffe Efficiency. The Nash Sutcliffe efficiency is a normalized variation of MSE. Normalization is applied to NSE so that the objective function value only varies between (0, 1].

Maximization Goal Objective Functions For Optimization

Objective Function

Equation

Coefficient of Determination

Z= r^2
r = \frac{\sum_{i=1}^{N}([q_o(i) - q_o(mean)] *[q_s(i) - q_s(mean)]}{\sqrt{\sum_{i=1}^{N}[q_o(i) - q_o(mean)]^2 *[q_s(i) - q_s(mean)]^2}}

Index of Agreement (Willmott, 1981)

Z=1 - \frac{\sum_{i=1}^{N}[q_s(i) - q_o(i)]^2}{\sum_{i=1}^{N}[|q_s(i) - q_o(mean)| + |q_o(i) - q_o(mean)|]^2}

Modified Kling-Gupta

Z=1 - \sqrt{(r - 1)^2 + (\beta - 1)^2 + (\gamma - 1)^2}

See above equation for r.

\beta = \frac{\mu_s}{\mu_o}
\gamma= \frac{CV_s}{CV_o} = \frac{\sigma_s/\mu_s}{\sigma_o/\mu_o}

Normalized Nash Sutcliffe

Z=1 - \frac{1}{2 - NSE}
NSE=1 - \frac{\sum_{i=1}^{N}[q_o(i) - q_s(i)]^2}{\sum_{i=1}^{N}[q_o(i) - q_o(mean)]^2}
  • Z = objective function
  • N = number of computed hydrograph ordinates
  • q_o(i) = observed flows
  • q_s(i) = calculated flows, computed with a selected set of model parameters
  • q_o(peak) = observed peak
  • q_o(mean) = mean of observed flows
  • q_o(min) = observed minimum
  • q_s(peak) = simulated peak
  • q_s(mean) = mean of simulated flows
  • \sigma_o = standard deviation of observed flows
  • \sigma_s = standard deviation of simulated flows
  • e = absolute residual
  • r = correlation coefficient
  • \beta = bias ratio
  • \gamma = variability ratio
  • NSE = Nash Sutcliffe

In addition to the numerical measures of fit, the program also provides graphical comparisons that permit visualization of the fit of the model to the observations of the hydrologic system. A comparison of computed hydrographs can be displayed, much like that shown in Figure 46. In addition, the program displays a scatter plot, as shown in Figure 47. This is a plot of the calculated value for each time step against the observed flow for the same step. Inspection of this plot can assist in identifying model bias as a consequence of the parameters selected. The straight line on the plot represents equality of calculated and observed flows: If plotted points fall on the line, this indicates that the model with specified parameters has predicted exactly the observed ordinate. Points plotted above the line represents ordinates that are over-predicted by the model. Points below represent under-predictions. If all of the plotted values fall above the equality line, the model is biased; it always over-predicts. Similarly, if all points fall below the line, the model has consistently under-predicted. If points fall in equal numbers above and below the line, this indicates that the calibrated model is no more likely to over-predict than to under-predict.

The spread of points about the equality line also provides an indication of the fit of the model. If the spread is great, the model does not match well with the observations – random errors in the prediction are large relative to the magnitude of the flows. If the spread is small, the model and parameters fit better.

The program also computes and plots a time series of residuals—differences between computed and observed flows. Figure 48 is an example of this. This plot indicates how prediction errors are distributed throughout the duration of the simulation. Inspection of the plot may help focus attention on parameters that require additional effort for estimation. For example, if the greatest residuals are grouped at the start of a runoff event, the initial loss parameter may have been poorly chosen.