General Concepts for HEC-DSS
HEC-DSS uses a block of sequential data as the basic unit of storage. This concept
results in a more efficient access of time series or other sequentially related data.
Each block contains a series of values of a single variable over a time span appropriate
for most applications. The basic concept underlying HEC-DSS is the organization of
data into records of continuous, applications-related elements as opposed to individually
addressable data items. This approach is more efficient for scientific applications
than a conventional database system because it avoids the processing and storage overhead
required to assemble an equivalent record from a conventional system.
Data is stored in blocks, or records, within a file and each record is identified by a
unique name called a "pathname.” Each time data is stored or retrieved from the file,
its pathname must be given. The data and
information about it (e.g., units) is stored in a "header array.” HEC-DSS
automatically stores the name of the program writing the data, the number of times the data
has been written to, and the last written date and time. HEC-DSS documents stored
data completely via information contained in the pathname and stored in the header so no
additional information is required to identify it. The self-documenting nature of the
database allows information to be recognized and understood months or years after it was
stored.
The pathname is the key to the data’s location in the database. HEC-DSS analyzes each
pathname to determine a “hash” index number. This index determines where the data set is
stored within the database. The design ensures that very few disk accesses are made to
retrieve or store data sets. One data set is not directly related to another so there is no
need to update other areas of the database when a new data set is stored.
Because of the self-documenting nature of the pathname and the conventions adopted, there
is no need for a data dictionary or data definition file as required with other database
systems. In fact, there are no database creation tasks or any database setup.
Both HEC-DSS utility programs and applications that use HEC-DSS will generate and
configure HEC-DSS database files automatically.
There is no pre-allocation of space; the software automatically expands the file size as
needed.
A HEC-DSS database file has a user-specified conventional name with an extension of
“.dss.” As many database files as desired may be generated and there are no size
limitations, apart from available disk space. Corps offices have HEC-DSS files that
range from a few data sets to thousands. HEC-DSS adjusts internal tables and hash
algorithms to match the database size so as to access both small and very large databases
efficiently.
HEC-DSS database files are “direct-access” binary files with no published format. Only programs linked with the HEC-DSS software library can be used to access HEC-DSS files. Direct access files allow efficient retrieval and storage of blocks of data compared to sequential files.
A principal feature of HEC-DSS is that many users can read and write data to a single
database at the same time. This multi-user access capability is implemented with
system record locking and flushing functions. There is no daemon or other background
program managing accesses to a database. A database may exist on a Windows or Unix
server machine, which can be accessed by users on PC’s or other computers via NFS or the
Microsoft network, as long as locking and flushing functions are implemented.