2.1 - Validation and Verification (V&V)
Background
The purpose of this page is to describe the validation and verification testing undertaken by HEC staff to ensure proper functioning of the HEC-FDA Version 2.1 (beta) software and to compare and explain results where they differ from Version 2.0.2.
Case Study Models
For this effort, nine case study models were utilized for V&V testing consistent with previous version releases. A base year, without-project condition was modeled for each case study and results were recorded for Version 2.0.2 and Version 2.1 to compare expected annual damage (EAD) and performance results. These models were also utilized to ensure functional forward migration of studies.
Additional modeling was conducted and recorded for Version 2.1 for future and with-project conditions. These results were utilized to confirm equivalent annual damage (EqAD) results against an independent spreadsheet model, confirm damages reduced calculations, ensure proper functioning of with-project inputs, and to ensure general proper functioning of the standard user interface (UI).
The table below describes the input components of each model.
| Model | Hydraulic Data Type | Terrain Data Type | Ground Elevations Source | Frequency Function |
|---|---|---|---|---|
Glendive | Gridded | N/A | Inventory | Stage-Only |
| Greenbrook | Unsteady | N/A | Inventory | Stage-Only |
| London Orleans | Gridded | N/A | Inventory | Stage-Only |
| Muncie | Gridded | HDF | Terrain | Analytical Flow |
North DeSoto | Gridded | N/A | Inventory | Stage-Only |
| River Des Peres | Gridded | TIF | Terrain | Stage-Only |
| Tafuna | Gridded | TIF | Terrain | Graphical Flow |
| Watertown | Gridded | TIF | Terrain | Graphical Flow |
| West Sacramento | Unsteady | N/A | Inventory | Analytical Flow |
Results
Equivalent Annual Damages
Equivalent annual damages (EqAD) results in version 2.1 were compared to an independent spreadsheet model. Results in version 2.1 were exactly equal to the results in the spreadsheet model. The spreadsheet model and EqAD results from version 2.1 are included at the shared drive location listed below.
Performance
Annual exceedance probability (AEP) performance results in version 2.1 were compared to version 2.0.2. Results are shown below for the base-year, without-project condition. More detailed results are available at the shared drive location listed below.
| Glendive | Threshold Value | Mean | Median | 10 | 30 | 50 | 0.1 | 0.04 | 0.02 | 0.01 | 0.004 | 0.002 |
| 2.0.2 | 13.65 | 0.0349 | 0.0325 | 0.2988 | 0.6552 | 0.8305 | 0.9969 | 0.6293 | 0.2585 | 0.1084 | 0.0298 | 0.0164 |
| 2.1 | 13.65 | 0.0351 | 0.0325 | 0.3001 | 0.6572 | 0.8321 | 0.9969 | 0.6495 | 0.2601 | 0.1098 | 0.0299 | 0.0167 |
| Difference | 0 | 0.0002 | 0 | 0.0013 | 0.002 | 0.0016 | 0 | 0.0202 | 0.0016 | 0.0014 | 1E-04 | 0.0003 |
| Greenbrook | Threshold Value | Mean | Median | 10 | 30 | 50 | 0.1 | 0.04 | 0.02 | 0.01 | 0.004 | 0.002 |
| 2.0.2 | 100.11 | 0.999 | 0.9991 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| 2.1 | 100.58 | 0.999 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| Difference | 0.47 | 0 | 0.0009 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| London Orleans | Threshold Value | Mean | Median | 10 | 30 | 50 | 0.1 | 0.04 | 0.02 | 0.01 | 0.004 | 0.002 |
| 2.0.2 | 5.82 | 0.1716 | 0.1728 | 0.8478 | 0.9965 | 0.9999 | 0.0734 | 0.02 | 0.0191 | 0.0148 | 0.0027 | 0.0009 |
| 2.1 | 5.82 | 0.177178 | 0.172832 | 0.8577 | 0.9971 | 0.9999 | 0.075 | 0.0047 | 0.0048 | 0.0047 | 0.0027 | 0.0009 |
| Difference | 0 | 0.005578 | 3.2E-05 | 0.0099 | 0.0006 | 0 | 0.0016 | -0.0153 | -0.0143 | -0.0101 | 0 | 0 |
| Muncie (IA1) | Threshold Value | Mean | Median | 10 | 30 | 50 | 0.1 | 0.04 | 0.02 | 0.01 | 0.004 | 0.002 |
| 2.0.2 | 945.78 | 0.0537 | 0.0496 | 0.4243 | 0.8092 | 0.9367 | 0.9022 | 0.3906 | 0.1663 | 0.0845 | 0.0468 | 0.0338 |
| 2.1 | 945.78 | 0.0539 | 0.0497 | 0.4251 | 0.81 | 0.9372 | 0.9003 | 0.3879 | 0.1662 | 0.0837 | 0.0443 | 0.0325 |
| Difference | 0 | 0.0002 | 0.0001 | 0.0008 | 0.0008 | 0.0005 | -0.0019 | -0.0027 | -0.0001 | -0.0008 | -0.0025 | -0.0013 |
| North DeSoto | Threshold Value | Mean | Median | 10 | 30 | 50 | 0.1 | 0.04 | 0.02 | 0.01 | 0.004 | 0.002 |
| 2.0.2 | 9.12 | 0.5418 | 0.5433 | 0.9996 | 1 | 1 | 0.0001 | 0.0001 | 0.0001 | 0 | 0 | 0 |
| 2.1 | 9.12 | 0.5447 | 0.5433 | 0.9996 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| Difference | 0 | 0.0029 | 0 | 0 | 0 | 0 | -0.0001 | -0.0001 | -0.0001 | 0 | 0 | 0 |
| River Des Peres | Threshold Value | Mean | Median | 10 | 30 | 50 | 0.1 | 0.04 | 0.02 | 0.01 | 0.004 | 0.002 |
| 2.0.2 | 543.71 | 0.3191 | 0.332 | 0.9786 | 1 | 1 | 0.0077 | 0 | 0 | 0 | 0 | 0 |
| 2.1 | 543.71 | 0.3302 | 0.332 | 0.9818 | 1 | 1 | 0.0059 | 0 | 0 | 0 | 0 | 0 |
| Difference | 0 | 0.0111 | 0 | 0.0032 | 0 | 0 | -0.0018 | 0 | 0 | 0 | 0 | 0 |
| Tafuna | Threshold Value | Mean | Median | 10 | 30 | 50 | 0.1 | 0.04 | 0.02 | 0.01 | 0.004 | 0.002 |
| 2.0.2 | 42.33 | 0.4627 | 0.5119 | 0.998 | 1 | 1 | 0.0001 | 0 | 0 | 0 | 0 | 0 |
| 2.1 | 42.33 | 0.5338 | 0.5117 | 0.9995 | 1 | 1 | 0.0001 | 0 | 0 | 0 | 0 | 0 |
| Difference | 0 | 0.0711 | -0.0002 | 0.0015 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Watertown | Threshold Value | Mean | Median | 10 | 30 | 50 | 0.1 | 0.04 | 0.02 | 0.01 | 0.004 | 0.002 |
| 2.0.2 | 1704.73 | 0.0378 | 0.0345 | 0.3199 | 0.6855 | 0.8545 | 0.9877 | 0.5846 | 0.2574 | 0.0748 | 0.0304 | 0.0108 |
| 2.1 | 1704.73 | 0.0376 | 0.0345 | 0.3181 | 0.6829 | 0.8526 | 0.9877 | 0.5845 | 0.2577 | 0.0746 | 0.0301 | 0.0107 |
| Difference | 0 | -0.0002 | 0 | -0.0018 | -0.0026 | -0.0019 | 0 | -1E-04 | 0.0003 | -0.0002 | -0.0003 | -0.0001 |
| West Sacramento | Threshold Value | Mean | Median | 10 | 30 | 50 | 0.1 | 0.04 | 0.02 | 0.01 | 0.004 | 0.002 |
| 2.0.2 | 21.51 | 0.7174 | 0.7237 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| 2.1 | 21.51 | 0.7174 | 0.7237 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| Difference | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Expected Annual Damages
Most of the results show negligible change in EAD. Tafuna demonstrates relatively larger change in EAD due to changes in how uncertainty is estimated for graphical flow frequency functions. Specifically, in Impact Area 11, the upper bound on flow uncertainty for a given frequency decreases by several thousand CFS for many frequencies between .5 AEP and .02 AEP, meaning the flows sampled in the flow-frequency function are significantly lower in version 2.1 relative to version 2.0.2, which will correspond to significantly lower stages and damage values.
The table below shows results by case study broken down by damage category and software version. More detailed results are available at the shared drive location listed below.
| Study Name | Damage Category | Version 2.0.2 Mean EAD | Version 2.1 Mean EAD | Percent Change |
|---|---|---|---|---|
| Glendive | RES | $231.28 | $225.6 | -2.46% |
| COM | $1,084.65 | $1,067.6 | -1.57% | |
| Total | $1,315.92 | $1,293.27 | -1.72% | |
| Greenbrook | RES | $50,336.46 | $50,135.10 | -0.40% |
| IND | $12,267.77 | $12,199.33 | -0.56% | |
| COM | $14,836.33 | $14,747.31 | -0.60% | |
| UTL | $13.59 | $13.05 | -3.97% | |
| APT | $16,301.84 | $16,340.11 | 0.23% | |
| MUN | $3,830.87 | $3,867.56 | 0.96% | |
| Total | $97,586.86 | $97,302.46 | -0.29% | |
| London Orleans | COM | $9,659.69 | $9,651.11 | -0.09% |
| IND | $2,378.43 | $2,368.5 | -0.42% | |
| PUBL | $1,292.98 | $1,292.26 | -0.06% | |
| RES | $34,832.81 | $34,734.13 | -0.28% | |
| Total | $48,163.91 | $48,046.00 | -0.24% | |
| Muncie (IA1) | RES | $391,451.82 | $392,574.71 | 0.29% |
| IND | $46,956.59 | $47,135.29 | 0.38% | |
| COM | $83,706.71 | $84,054.13 | 0.42% | |
| PUB | $397,351.69 | $398,732.80 | 0.35% | |
| Total | $919,466.80 | $922,496.9 | 0.33% | |
| North DeSoto | Residential | $704.80 | $704.08 | -0.10% |
| Commercial | $2,598.89 | $2,599.34 | 0.02% | |
| Industrial | $138.50 | $137.55 | -0.69% | |
| Public | $35.16 | $34.77 | -1.11% | |
| Agricultural | $1.27 | $1.19 | -6.30% | |
| Total | $3,478.63 | $3,476.93 | -0.05% | |
| River Des Peres | COM | $2,593.88 | $2,574.41 | -0.75% |
| PUB | $438.97 | $439.55 | 0.13% | |
| RES | $4,013.10 | $4,008.18 | -0.12% | |
| IND | $181.51 | $179.53 | -1.09% | |
| Total | $7,227.46 | $7,201.67 | -0.36% | |
| Tafuna (IA11) | IND | $112.02 | $93.38 | -16.64% |
| RES | $21.95 | $6.58 | -70.02% | |
| Total | $133.97 | $99.95 | -25.39% | |
| Watertown | RES | $2,479.82 | $2,518.59 | 1.56% |
| COM | $2,263.48 | $2,305.26 | 1.85% | |
| IND | $874.71 | $890.45 | 1.80% | |
| PUB | $108.61 | $111.45 | 2.61% | |
| Total | $5,726.62 | $5,825.74 | 1.73% | |
| West Sacramento | RES | $189,002.06 | $189,002.06 | 0.00% |
| COM | $1,001,089.69 | $1,001,089.69 | 0.00% | |
| IND | $430,818.36 | $430,818.42 | 0.00% | |
| PUB | $22,620.99 | $22,621.00 | 0.00% | |
| Total | $1,643,531.10 | $1,643,531.10 | 0.00% |
Life Safety Risk
Due to a lack of prior modeling and lack of available case studies, a single case study for life safety risk was utilized to test the UI for functionality and for reasonable results. That case study, Alaiwai, is provided among the supporting data at the shared drive location provided below.
Supporting Documentation and Data
The supporting models and detailed results tables can be found at this shared drive location. The EqAD spreadsheet calculator used to independently verify EqAD results is also located in the folder.
The release notes can be found here.
HEC-FDA Version 2.1 (beta) can be found here. Download the portable package to a location where you have write access, such as C:\Programs and extract the contents of the zip file. Open the HEC-FDA 2.0 folder that contains the extracted contents to find the .exe application HEC.FDA.View with the green house icon. Double-click on HEC.FDA.View to launch the software.