The SDI plugin expects input data as HEC-DSS time series stored as a "collection" and for the data to be organized into a "rectangular" array of data by events and lifecycles.  A time series collection in HEC-DSS represents an "ensemble" of input data, with members of the ensemble being distinguished by a collection ID as the prefix to the DSS pathname "F part".  These collection members for SDI may represent anything the user desires to input to the model, as long as the input data can be organized by the following constraints:


Requirements:

  • SDI data must be stored with all collection members for a single location and parameter to be stored in the same HEC-DSS file.  Other location/parameters can be mapped to the same DSS file or stored in other DSS files, as long as each  uses a single file.
  • DSS Record pathname:
    • Data for a given location and parameter combination must have consistent DSS record pathnames – that is identical A, B, C, and E parts. 
    • The time-step for the data must be consistent across all collection members, that is having the same E-part in the DSS record pathname.
    • SDI data collection members are mapped into lifecycles for the compute, represented as the collection ID.  That is, data for the first lifecycle must be stored in the first collection member – the time series with an F part that has the collection ID of C:000001| as a prefix.  The second lifecycle must use the second collection member, with a prefix of C:000002|  and so on. 
    • Note: The DSS record "D" part represents the start-date of the given block of data, typically these are ignored for the purpose of mapping input data to the SDI alternatives.
  • Presence or absence of data:
    • Depending on settings within the SDI alternative and the presence or absence of data in the time series, each event will be generated from the data within each lifecycle.  SDI is typically configured to divide these into one event per year, but can also run a single continuous event across the entire lifecycle.   The start date and end date for all lifecycles will be constrained by the Analysis Period selected for the given simulation.
  • The time series data's type and units will be determined by the DSS records provided.
  • The time-step, data type, and units do not have to be consistent across all records of the same parameter type, however this is advised.


Additional metadata that SDI may properly handle

The SDI plugin reads and writes the DSS time series records "as-is" from the input DSS files to the FRA compute's "lifecycle DSS file".  HEC-DSS time series data may contain information about the following metadata, which should not cause an issue for the SDI plugin, but has not been substantially tested:

  • Data quality flags
  • Data annotations
  • Time-zone information
  • Vertical datum information or datum shifts.

If input data from the SDI plugin is failing to copy into the lifecycle DSS correctly, or subsequent models are failing to compute, we recommend doing additional preparation of the input data to ensure consistency in these fields and then to remove them from the input data via the tools available in the HEC-DSSVue software.


Creating data for use within an SDI Alternative:

The process to create input data for an SDI alternative may vary substantially depending on the study purpose and the type of data being used.  Below is an example of a quick method to create input data from an existing period-of-record though is to use the HEC-DSSVue Collections menu to generate a collection from cleaned input data.  This may be useful if the SDI plugin is being used with the HEC-WAT Distributed Compute plugin to run individual years of a model in parallel.


Within the HEC-DSSVue software's main window, with the desired time series data selected, selecting New Collection from Period of Record or New Collection from POR Peaks both allow splitting a continuous record into individual "events" either by year or limited to a window around annual peaks, and shifts the resulting data to a single year.  The resulting collection is usable within the SDI plugin.



This example data is a continuous sequence of daily data pulled from a USGS stream gage record, and cleaned up to ensure there are no missing data points.


When using the New Collection from Period of Record, one can choose whole years or seasons bound by dates.  To split the data by water year, enter a date starting with 01Oct and an end date of 30Sep.


The starting sequence number should be entered as 000001 to set the first collection ID correctly.


The date for the first data point of the collection may be left as the selected start date of 01Oct combined with the year 3000.  Using the year 3000 helps indicates the data was shifted from the original year and is not the year the data is stored in.


Clicking OK on this last dialog will cause the tool to create the collections and report the number of collection members created.  These can then be plotted within DSSVue to see the results – below is a plot of the first three collection members showing the first three years from the source data.


The new collection members can be seen in the DSSVue catalog with the F part prefixes and shifted time range in the D part.  This data can now be mapped into an SDI alternative to run a single year starting on 01Oct3000.