Viewing Results for the Current Trial

A variety of Graphical and Tabular results are available after an Optimization Trial is computed. The same results are also available for the current trial, so long as no data used in the simulation has changed. The program tracks all of the data in the Optimization Trial and the selected Basin Model and selected Meteorologic Model. It also tracks any Time-Series Data, Paired Data, or Grid Data used in the Basin and Meteorologic Models. Results for an Optimization Trial can only be accessed when none of this data has changed since the last time the trial was computed. If any of the data has changed, you will need to recompute the Optimization Trial before you will be allowed to start accessing results.

Once a result is open for viewing it will remain open until it is closed by the user. It is possible that data used during the simulation that produced the result could change while the result is open for viewing. In this case, the open result will immediately show an indication that data has changed and the Optimization Trial needs to be recomputed. After the trial is recomputed the open results are automatically updated with the new results and the indication is updated with the date and time of the most-recent compute.

Objective Function Table

The Objective Function Table provides summary information about the Objective Function at the evaluation location. It includes summary information such as the Basin Model and Meteorologic Model, Start Time, and End Time. It indicates which Objective Function Method was used, the Start and End of the Evaluation Period for the function, and the Final Value of the function when the search method finished. It gives the Volume, Peak Flow, Time of peak flow, and Time to the Center of Mass of the computed and observed hydrographs. Finally, it provides Volume and Peak Flow Differences between the computed and observed hydrographs. The Objective Function Table is accessed from the Results menu. When the Basin Map is open and a trial is selected, choose the Objective Function Summary command to view the results. Sample results are shown in the following figure.

Optimized Parameters Table

The Optimized Parameters Table provides summary information including the Basin Model, Meteorologic Model, Start Time, and End Time. It lists in tabular form the parameters that were selected for optimization with one row for each parameter. The table has columns for Element Name, Parameter Name, Units, Initial Value, Optimized Value, and Sensitivity. The sensitivity is computed after the search method has finished; it indicates the percentage change in the objective function value resulting from a one percent change in the parameter value. The Optimized  Parameters Table is accessed from the Results menu. When the Basin Map is open and a trial is selected, choose the Optimized Parameters command to view the results. A sample set of optimized parameters is shown in the following figure.

Parameter by Iteration Graph

The Iterative Evolution of a Parameter can be displayed by selecting the Parameter Graph in the results tree view. For Deterministic Optimization, a single series is displayed, showing the parameter evolution throughout the simulation. The decreasing variation in parameter values by iteration indicates that convergence is occurring.

For MCMC simulations, each chain in the pool is displayed on a single Plot of Iteration vs. Value (shown in the following figure). Convergence to equilibrium is generally not visible on this plot. Instead, a special Convergence Statistic Plot by Iteration is used instead (shown in the subsequent figure). Both of these plots show a thinned version of the pre-equilibrium samples taken by the MCMC algorithm. Once the chains have achieved equilibrium, then the sampler writes the specified number of equilibrium samples to a DSS record for analysis or use later.

Observed Data Graphs

The Observed Data Graphs shows three different pieces of information: Computed Results and Observed data at the Objective Function evaluation location, an Observed vs. Computed Scatter Plot, and a Residual Time Series. The three graphs can be toggled between by selecting their respective tabs on the Observed Data Graph window. These graphs allow visual comparison of how well the computed and observed hydrographs match. Vertical Lines are drawn to show the Start and End of the Objective Function Time Window. The Time-Series Graph can indicate the quality of the parameter estimation if good visual agreement is achieved. If the computed results are exactly equal to the observed data, then the data will plot exactly on a 45-degree line within the comparison graph. However, in virtually all cases the match is not exact and there will be scatter in the data around the 45-degree line. Data points before the time of peak are shown with Red Circles and Points and after the time of peak are shown with Blue Triangles. The amount of scatter helps to indicate the quality of the parameter estimation. The Flow Residuals Graph shows the difference between computed and observed for each time step. It is determined as the computed results minus the observed data and may be positive or negative. Vertical lines are drawn to show the start and end of the objective function time window. The magnitude of the Residuals helps to indicate the quality of the parameter estimation. The residuals also help to indicate if there are Biases in the agreement between the computed results and observed data. The Observed Data Graphs are accessed from the Results menu. When the Basin Map is open and a trial is selected, choose the Observed Data command to view the results. Sample Observed Data Graphs are shown in the following figures.

Objective Function Graph

The Objective Function Graph shows the value of the Objective Function at Each Iteration of the search method. The graph gives some indication of how fast the search method was able to converge to optimal parameter values. In some cases it may show that the search method was not converging at all. The Objective Function Graph is accessed from the Results menu. When the Basin Map is open and a trial is selected, choose the Objective Function command to view the results. A sample Objective Function Graph is shown in the following figure.

Individual Elements

There are three ways to access results for Individual Elements in the current Optimization Trial. All methods show the same results. You must select one or more elements in the Basin Map before you can view results for that element. However, not all elements in the Basin Map may have results. The Optimization Trial is based on a Simulation Run that uses a specific Basin Model. The Optimization Trial works with the elements upstream of the location where observed data is used to evaluate the Objective Function. The trial does not perform simulations at any elements downstream of the element with observed data. Therefore, results will only be available at elements upstream of the location with observed data. Any downstream elements will have results shown as unavailable.

The first method for viewing individual element results is to use the Results menu. You may need to first open the Basin Model by clicking on it in the Watershed Explorer, on the Components tab. The current Optimization Trial is shown in brackets in the Basin Map Title Bar. Select one or more elements in the Basin Map by clicking with the Arrow Tool. With an element selected in the Basin Map, click the Results menu and select the Element Graph command, the Element Summary Table command, or the Element Time-Series Table command. The appropriate result will automatically be shown in the Desktop area. The information included in the graph (shown in the following figure) varies by element type, but always includes Outflow. Optional items such as Observed Flow, Computed Snow Water Equivalent (SWE), and Observed SWE are also included.

The information included in the Summary Table (Figure 11) also varies by element type but always includes the Peak Flow, Time of Peak Flow, and Outflow Volume. 

The Time-Series Table includes the same information as the graph but in numerical format. If you selected more than one element, then one result for each element will open. The menu commands on the Results menu will only be enabled if the results for the current Optimization Trial do not need to be recomputed, and there is an element selection in the Basin Map, and the selected element is upstream of the objective function evaluation element with observed data.

The second method for viewing Individual Element Results is to use the Element icon in the Basin Map. Again you may need to open the Basin Model before attempting to view results. Click on an element to highlight it. Keep the mouse over the element and press the right mouse button. A context menu is displayed that contains several choices including View Results. The name of the current Optimization Trial is shown in brackets after the menu command. Move the mouse into the View Results submenu and select any of the result commands: Graph, Summary Table, or Time-Series Table. The appropriate result will open. If you selected more than one element, then one result for each element will open. The View Results menu command will only be enabled if the results for the current Optimization Trial do not need to be recomputed, and the selected element is upstream of the Objective Function evaluation element with observed flow.

The third method for viewing Individual Element Results is to use the Buttons on the Toolbar. First you must open the Basin Model and select one or more elements in the Basin Map by clicking with the Arrow Tool. Once you have a selected element, click the Graph, Summary Table, or Time-Series Table buttons. The button for selecting a Graph shows a Line Plot. The button for the Summary Table shows a Plain Table. The button for the Time-Series Table shows a Table Plus a Clock. The appropriate result will automatically open. If you selected more than one element, then one result for each element will be added to the Desktop. The toolbar buttons for viewing element results will only be enabled if the results for the current Optimization Trial do not need to be recomputed, and there is an element selection in the Basin Map, and the selected element is upstream of the Objective Function evaluation element with observed flow.