Introduction


The Hydrologic Engineering Center's (HEC) Data Storage System Visual Utility Engine (HEC-DSSVue) is a graphical user interface program for viewing, editing, and manipulating data in the HEC Data Storage System (HEC-DSS) database files. With HEC-DSSVue, you may plot, tabulate, and edit data, as well as manipulate data with over fifty mathematical functions. Along with these functions, HEC-DSSVue provides several utility functions that allow you to enter datasets into a database, rename dataset names, copy data sets to other HEC-DSS database files, and delete datasets.


This version of HEC-DSSVue incorporates HEC-DSS version 7, as well as maintains combability with HEC-DSS version 6 database files.  You can open and work with either version 7 or version 6 files.  You can convert version 6 files to version 7 and visa-versa (excluding new data types), as well as copy data between different versions.


Typically, you will select datasets from a sorted/filtered list of names in a HEC-DSS database file. HEC-DSSVue also incorporates the "Jython" standard scripting language that allows you to specify a routine sequence of steps in a text format and then execute the sequence from a user-defined button or a "batch" process.


This version of HEC-DSSVue incorporates HEC-DSS version 7, as well as maintains combability with HEC-DSS version 6 database files.  You can open and work with either version 7 or version 6 files.  You can convert version 6 files to version 7 and visa-versa (excluding new data types), as well as copy data between different versions.


HEC-DSSVue was written using the Java programming language that allows it to be run under a variety of different operating systems.  HEC-DSSVue is supported on Microsoft Windows.  

What's New in HEC-DSS version 7?

HEC-DSS Version 7 incorporates a new 64-bit file addressing system (independent of the computer file system, which can be either 32-bit or 64-bit).  This addressing system essentially allows for an unlimited database file size (it has been tested up to 500 GB), and has been optimized to be fast with large database files.

Some of the new features in version 7 are:

  • 64-bit file addressing - Unlimited database size.  A data set’s pathname is used to compute a “double hash”, which is used to determine where the data set is located within the file.  Internal table sizes are adjusted based on the number of data sets within the file
  • Pathnames - Pathnames retain upper- and lower-case characters, but are case insensitive.  For example, “FLOW” is viewed the same as “Flow”, but will be displayed using the original character cases.
  • Catalog - Separate catalog files are no longer used (although they can be made, if desired.)  The list of pathnames (the catalog) is read directly from the internal tables inside the database file.  Catalog sort orders are retained within the file, so that only changes to the catalog will be sorted.
  • Aliases - You can have multiple pathnames point to the same data set.  Useful in CWMS and HEC-RTS, where “grid sets” have observed grids and forecasted grids.  The observed grids are duplicated and occupy a large amount of space.  Aliases will allow different name sets to point to the same observed data.
  • Recycling and Space Recovery - HEC-DSS Version 7 reuses unused file space from deleted or expanded records.  In version 6, a deleted record was marked as deleted, but the space was not removed until a squeeze was performed on the file.  In version 7, the record is still marked as deleted, but reused when the next write (that fits within that space) is done.
  • Time Series - There are many enhancements to the time series data type, including time Series Profiles.
  • Automatic Compression - Removes repeating values and is lossless.  For example, hourly precipitation data contains many zero values.  A sequence of zeros will be reduced to one zero, along with a flag of how many zeroes it represents.  Often precipitation data sets are reduced to 3 to 10 % of their original size.  This compression makes HEC-DSS database files smaller, and data set sizes smaller (on disk) so that they are faster to read and write.
  • Resolution to 1 second - The time resolution for time series data has been extended to one second from one minute.  For regular-interval data, you may now have E parts such as “30Seconds”, “5Seconds”, “1Second”, as well as others.  Data with a time resolution of one second use an internal data record length of one day.  Irregular-interval time series data can have a time resolution of one second, which is set when saving the data set.
  • Missing Data stored as -FLT_MAX (not -901.) Missing data, usually within a regular-interval data set is stored with a -FLT_MAX flag, not -901., which was used in version 6.  That flag can still be converted to -901.0 for legacy programs.  (HEC programs written in Java use -FLT_MAX to represent missing data values.
  • Extended Dates - Time series data stored in HEC-DSS version 7 can have dates with years –5,873,711 to 5,879,610 (that is 01Jan-5873711 to 01Jan5879610), to represent over 11 million years.  Such dates are mainly used in statistical analysis and are day-count based; i.e., they do not include any Gregorian calendar changes.
  • Variable length data values and quality flags - In version 6, data was generally stored in a 32-bit cell, along with an optional 32-bit quality flag.  In version 7, you can specify the length of data and quality cells for a record.  For example, you may wish to store data in 64-bit cells and the quality value in 128-bit cells.
  • “Notes” for each value (e.g., comments about values) - In version 7, time series data values can contain “notes”.  Notes are usually a string of characters (for example, “the datum was off by 1.23 feet for this measurement”), but also can be a set of integers (with a variable length.)
  • Pseudo Intervals (~1Day) - Internally stored as irregular, but expected to be similar to data with a regular interval.
  • Ensemble – Collections
  • Profile Data - This is a combination of Time Series data and paired data.
  • Paired Data - no size limits, range updates.
  • Text and Text Tables
  • Generic Arrays - such as an array of integers

Internals:

  • Multi-thread Functionality
  • Easier programming
  • Better Messaging, Diagnostic and Error Output
  • Better Error Detection