Data-Gap Filler

The Data-Gap Filler estimates missing values in gridded time-series data. Four methods are available: Focal Mean, Linear Interpolation, ML Regression, and Kriging. Focal Mean uses spatial information, Linear Interpolation uses temporal information, and the two new methods combine spatial and temporal information. An additional option inserts missing time steps.

In HEC-HMS, select Tools | Data | Gap Filler. A project does not need to be open. The utility can also be launched from the standalone Vortex distribution.

A project does not have to be open to access the Data-Gap Filler.

The latest Vortex distribution can be downloaded as a standalone and run outside of HEC-HMS. 

Step 1: Select Source Variables

Browse to the Source Grid File (1). Choose the variables to process from Available Variables (2) and move them to Selected Variables (3) using the arrow buttons or by double-clicking. Select at least one variable, then click Next.

Use a gridded time series whose records have consistent grid dimensions, cell locations, coordinate system, units, and missing-data value. Cells to be filled must be identified by the source grid’s missing-data value. A valid zero, such as zero precipitation, is retained.

At least one variable must be selected to proceed.

Step 2: Select an Interpolation Method

Select one interpolation method, then click Next. The selected method fills missing cells within gridded records that already exist. Existing valid cell values are retained.

Spatial Fill: Focal Mean (1)

Focal Mean fills a missing cell with the mean of valid values in a surrounding neighborhood within the same grid. The neighborhood expands if no valid cells are found. Evaluate this method for spatial gaps, including locations that are missing throughout the time series, when surrounding cells provide a reasonable estimate.

Temporal Fill: Linear Interpolation (2)

Linear Interpolation estimates a missing value from the nearest valid values before and after the gap at the same cell location. The estimate is weighted by time. Evaluate this method for gaps bracketed by valid observations; it cannot fill a cell that has no valid value on one side of the gap.

Spatio-temporal Fill: ML Regression (3)

ML Regression fits a regression tree using valid cells in the current grid and predicts values for its missing cells. The model uses cell position, statistics from surrounding cells, and values at the same location in up to two preceding and two following gridded records. The utility trains the model automatically; no separate training dataset or model file is required.

Evaluate ML Regression when spatial patterns and changes between nearby records can help explain missing values. Enough valid cells must remain in the current grid to train the model. Model fitting adds processing work, so consider both accuracy and processing time when comparing methods.

Spatio-temporal Fill: Kriging (4)

Kriging estimates missing values using ordinary kriging with nearby valid observations from the current grid and up to two preceding and two following records. A variogram describes how differences between observed values vary with separation in space and time. The utility fits a spherical variogram automatically and uses it to calculate interpolation weights.

Evaluate Kriging when nearby observations may provide useful estimates of a gap. The utility selects the neighborhood and variogram parameters; the wizard does not require these settings. If the kriging equations cannot be solved, the utility uses inverse-distance weighting for that estimate. If a usable variogram cannot be formed, it fills gaps with the mean of valid cells in the current grid.

Availability of spatial and temporal information

Both spatiotemporal methods use up to five existing records: the current record and its two neighbors on either side. At the beginning or end of the series, fewer temporal neighbors are available. Missing temporal values do not by themselves prevent processing when the current grid contains enough valid information.

The temporal neighborhood is based on record position, rather than a fixed duration in hours. Use a regular time interval so that neighboring records represent comparable time separations.

When the current grid contains fewer than ten valid cells, ML Regression and Kriging use Focal Mean instead. An entirely missing grid cannot be filled by this fallback, even if neighboring records contain valid data. Linear Interpolation may be appropriate for an existing all-missing grid bracketed by valid records. Use the option in Step 3 when a record is absent altogether.

At least one interpolation method must be selected to proceed.

Step 3: Select Insert Missing Time Steps

Select Insert Missing Time Steps (1) when the time series contains absent gridded records. This option creates records by linear interpolation between existing time steps. It runs after the method selected in Step 2 fills cell gaps. Leave the option cleared if only cells within existing records need to be filled, then click Next.

Step 4: Select Destination

The File Browse button (1) can be used to browse to a Destination file. If the destination file is HEC-DSS format, additional options are presented for specifying pathname parts (2). Some pathname parts cannot be edited and will be populated automatically. There are also options for changing the DSS Units (3) and DSS Data Type (4).

Step 5: Processing

After Next has been selected on Step 4, a progress bar will display, indicating the progress of the Data-Gap Filler. Messages indicating status will be presented in the message panel.

Step 6: Data-Gap Filler Complete

After the Data-Gap Filler is complete, the wizard will automatically progress to step 6, indicating that the conversion is complete. There are buttons to Restart or Close the wizard. The gap-filled data will be saved to the user selected HEC-DSS file (set in step 4).

If time-step fill is selected in addition to data-gap fill, the gap fill is processed in two steps. The progress bar will indicate data-gap fill progress first, and then re-set for time-step fill progress.