Retrieving and storing data from a HEC-DSS file is done through the class HecDss in the hec.heclib.dss package. The first call that should be made is the static member HecDss.open(string filename), which will return a HecDss object to retrieve and store data, as well as perform math and other functions.

You need to import the classes that you use as well as the hec.script package. To do this, include a subset of the following lines at the beginning of your script:

import example

from hec.script import Plot, MessageBox
from hec.io import TimeSeriesContainer
from hec.io import PairedDataContainer
from hec.hecmath import TimeSeriesMath
from hec.hecmath import PairedDataMath
from hec.heclib.dss import HecDss, DSSPathname, HecDataManager
PY



HecDss Class

from hec.heclib.dss import HecDss, DSSPathname


HecDss.open(string filename)
HecDss.open(string filename, string startTime, string endTime)
HecDss.open(string filename, string timeWindow)
HecDss.open(string filename, Boolean fileMustExist)


PY



The HecDss class is used to gain access to an HEC-DSS file, as illustrated below. You must import HecDss from hec.heclib.dss.


Example 15: Opening and Releasing a DSS File

from hec.heclib.dss import HecDss

theFile = HecDss.open("MyFile.dss") 
#or 
theFile = HecDss.open("C:/temp/sample.dss", True) 


#Finished using the file – release it 
theFile.done()
PY


Using one of the methods that specifies a time window affects the operation of the get() and read() methods as described in the Pattern Strings section below.


Note: The preferred manner of releasing access to an HEC-DSS file in most cases is to use the done() method and not the close() method. This is because HEC-DSS maintains information about file access and can grant access to a file released with done() much more quickly than it can to one released with close(). HEC-DSS will close files when a script exits or when they have not been accessed for a while. The only time that the close() method should be called explicitly is to perform some operating system operation on it, such as renaming or deleting the file.

HecDss Retrieve and Store Functions


HecDss objects are used to retrieve and store data sets in a DSS file, as well as several utility functions for DSS files. Data is retrieved and stored using DataContainers. DataContainers are simple classes that are database independent and are how most Hec classes exchange data. DataContainers are a base class for several data types, such as time series data, paired data, gridded data, etc. DataContainers and their related classes are described following the HecDss class. If the data set you are retrieving does not exist, the DataContainer will have the numberValues set to 0 (zero).


HecDss, and most classes accessed through scripts, use Java Exceptions for error handling. Use a try block to catch and process errors. If data cannot be stored, HecDss will throw an exception. Refer to Handling Exceptions in Scripting Basics for more information.


Tip: From the HEC-DSSVue main screen, select a pathname, right click and select Copy Pathnames to Clipboard. You can then paste the pathname into your script and avoid having to type it. You cannot use condensed pathnames for these functions.


The HecDss methods for retrieval, storage, and utility methods are described in the tables below.  Time Series Data Storage Constants (Regular and Irregular) are also shown below.



HecDss Retrieval and Storage Methods

Method

Returns

Description

get(String pathname)

DataContainer

Retrieves data for the single pathname and returns in a DataContainer. DataContainer will be a TimeSeriesContainer, PairedDataContainter, etc., depending on the data type. If a default time window has been set via one of the open() or setTimeWindow() methods, the "D" part of a time series pathname is ignored and the data for the time window is retrieved.

get(string pathname, Boolean getEntireDataSet)

DataContainer

Ignores the "D" part of time series data and retrieves data for the entire date span in the DSS file.

get(DSSIdentifier dssid)

DataContainer

Retrieves data for the pathname and time window given in the dssid

put(DataContainer dataContainer)

None

Store the data in DataContainer in DSS. The DataContainer must be a TimeSeriesContainer or PairedDataContainter, etc. Throws an exception if the data cannot be stored


HecDss Additional Primary Methods

Method

Returns

Description

done()

None

Tells HEC-DSS that you are finished using it and releases the file. Use this instead of close().

getPathnameList()

list

Returns a list of the pathnames in the file.

getTimeSeriesExtents(String pathname, HecTime start, HecTime end)

Boolean

Sets the contents of the start and end parameters to the date/time of the first and last piece of data in the entire data span. Returns whether the pathname exists.

isOpened ()

Boolean

Returns if the DSS file is accessible.

open(String dssFileName)

HecDss

Opens the DSS file and returns a HecDss object for further access. Creates the file if it does not exist. Throws an exception if the file cannot be created or opened.

open(String dssFileName, Boolean fileMustExist)

HecDss

Opens the DSS file and returns a HecDss object for further access. Throws an exception if you indicate the file must exist and it does not.

recordExists(String pathname)

Boolean

Tests to see if the single record exists.



Example 16: Reading from DSS

from javax.swing import JOptionPane
from hec.heclib.dss import HecDss
from hec.script import Plot
import java

try : 
  try :
    myDss = HecDss.open("C:/temp/sample.dss")
    flow = myDss.get("/RUSSIAN/NR UKIAH/FLOW/01MAR2006/1HOUR//")
    plot = Plot.newPlot("Russian River at Ukiah")
    plot.addData(flow)
    plot.showPlot()
    plot.stayOpen()
  except Exception, e :
    JOptionPane.showMessageDialog(None, ' '.join(e.args), "Python Error", JOptionPane.ERROR_MESSAGE)
  except java.lang.Exception, e :
    MessageBox.showError(e.getMessage(), "Java Error")
finally :
  myDss.done()

PY

The following plot is displayed when running the script above.  The plot includes Marcy of 2016 because there is a date '01MAR2016' in the path.



Example 17 : Reading a complete data set from DSS

from hec.script import Plot,MessageBox
from hec.heclib.dss import HecDss
import java

try :
  myDss = HecDss.open("C:/temp/sample.dss")
  flow = myDss.get("/MISSISSIPPI/ST. LOUIS/FLOW//1DAY/OBS/", 1)

  if flow.numberValues == 0 :
    MessageBox.showError("No Data", "Error")
  else :
    plot = Plot.newPlot("Mississippi")
    plot.addData(flow)
    plot.showPlot()

except Exception, e :
    MessageBox.showError(' '.join(e.args), "Python Error")
except java.lang.Exception, e :
    MessageBox.showError(e.getMessage(), "Error")
finally :
  myDss.done()
PY


Example 17 results in a plot of the entire data set because the D part (date) of the path was left blank.



Example 18: Writing Time series data to DSS

from hec.script import Plot, MessageBox
from hec.heclib.dss import HecDss, HecTimeSeries
from hec.io import TimeSeriesContainer
from hec.heclib.util import HecTime
import java

try:
    myDss = HecDss.open("myFile.dss")
    tsc = TimeSeriesContainer()
    tsc.fullName = "/Basin/loc/FLOW/01NOV2002/1Hour//"
    start = HecTime("01Nov2022","0800")
    tsc.interval = 60
    flows = [0.0,2.0,1.0,4.0,3.0,6.0,5.0,8.0,7.0,9.0]
    times =[]

    for value in flows:
        times.append(start.value())
        start.add(tsc.interval)
      
    tsc.times = times
    tsc.values = flows
    tsc.numberValues = len(flows)
    tsc.units ="CFS"
    tsc.type = "PER-AVER"
    myDss.put(tsc)
 
except Exception, e:
    MessageBox.showError(''.join(e.args),"Python Error")

finally:
    myDss.close()


PY

Output from the Example above

11:57:20.724      -----DSS---zopen   Existing file opened,  File: c:\programs\HEC-DSSVue-v3.2.1.130\myFile.dss
11:57:20.724                         Handle 3;  Process: 2424;  DSS Versions - Software: 7-HV, File:  7-HV
11:57:20.724                         Single-user advisory access mode
11:57:20.725 -----DSS--- zwrite  Handle 3;  Version 2:  /Basin/loc/FLOW/01Nov2022/1Hour//
11:57:20.725      -----DSS---zclose  Handle 3;  Process: 2424;  File: c:\programs\HEC-DSSVue-v3.2.1.130\myFile.dss
11:57:20.726                         Number records:         1
11:57:20.726                         File size:              15826  64-bit words
11:57:20.726                         File size:              123 Kb;  0 Mb
11:57:20.726                         Dead space:             0
11:57:20.726                         Hash range:             8192
11:57:20.726                         Number hash used:       1
11:57:20.726                         Max paths for hash:     1
11:57:20.727                         Corresponding hash:     2734
11:57:20.727                         Number non unique hash: 0
11:57:20.727                         Number bins used:       1
11:57:20.727                         Number overflow bins:   0
11:57:20.727                         Number physical reads:  9
11:57:20.727                         Number physical writes: 8
11:57:20.727                         Number denied locks:    0
TEXT



Method

Returns

Description

close()

None

Close the DSS file. Only call this when you need the file closed (e.g. to rename it); otherwise call done()

copyRecordsFrom(String toDSSFilename, list of strings pathnameList)

integer
Success = 0
Fail < 0

Copy records from the open DSS file to the DSS file name specified in the call.

copyRecordsFrom(String toDSSFilename, Vector pathnameList)

integer
Success = 0
Fail < 0

Copy records from the open DSS file to the DSS file name specified in the call.

copyRecordsInto(String fromDSSFilename, list of strings pathnameList)

integer
Success = 0
Fail < 0

Copy records from the file specified in the call into the currently open DSS file



HecDss Secondary Methods


Method

Returns

Description

copyRecordsInto(string fromDSSFilename, Vector pathnameList)

integer
Success = 0
Fail < 0

Copy records from the file specified in the call into the currently open DSS file

delete(list of strings pathnameList)

integer
Success = 0
Fail < 0

Deletes the records corresponding the list of pathnames

delete(Vector pathnameList)

integer
Success = 0
Fail < 0

Deletes the records corresponding the list of pathnames

duplicateRecords(list of strings pathnameList, list of strings newPathnameList)

integer
Success > 0
Fail < 0

Duplicates the records in the first list giving the names in the second list (same DSS file)

duplicateRecords(Vector pathnameList, Vector newPathnameList)

integer
Success > 0
Fail < 0

Duplicates the records in the first list giving the names in the second list (same DSS file)

getCatalogedPathnames()

list of strings

Returns a list of all cataloged pathnames without generating a new catalog.

getCatalogedPathnames(Boolean forceNew)

list of strings

Returns a list of all cataloged pathnames, optionally generating a new catalog first.

getCatalogedPathnames(string pattern)

list of strings

Returns a list of all cataloged pathnames that match the specified pattern without generating a new catalog.

getCatalogedPathnames(string pattern, Boolean forceNew)

list of strings

Returns a list of all cataloged pathnames that match the specified pattern, optionally generating a new catalog first.

getDataManager()

CombinedDataManager

Gets the data manager for this DSS file

getEndTime()

string

Returns the end time set

getFilename()

string

Returns the name of the DSS file

getStartTime()

String

Returns the start time set

isRemote()

Boolean

Returns if the DSS file is being served on a remote machine (client/server mode)

read(string pathname)1

HecMath

Returns an HecMath object that holds the data set specified by pathname. If a default time window has been set via one of the open() or setTimeWindow() methods, the "D" part of a time series pathname is ignored and the data for the time window is retrieved.

read(string pathname, string timeWindow) 1

HecMath

Returns an HecMath object that holds the data set specified by pathname with the specified time window.

read(string pathname, string startTime, string endTime) 1

HecMath

Returns an HecMath object that holds the data set specified by pathname with the specified time window.

renameRecords (list of strings pathnameList, list of strings newPathnameList)

integer
Success > 0
Fail < 0

Renames the records in the first list to those in the second list.

renameRecords (Vector pathnameList, Vector newPathnameList)

integer
Success > 0
Fail < 0

Renames the records in the first list to those in the second list.

setIrregularStoreMethod (integer storeMethod)

None

See table following

setRegularStoreMethod (integer storeMethod)

None

See table following

setTimeWindow(string timeWindow)

None

The default time window for this DSSFile.

setTimeWindow(string startTime, string endTime)

None

The default time window for this DSSFile.

setTrimMissing(Boolean trim)

None

Sets whether TimeSeriesMath objects retrieved via calls to read(…) will have missing data trimmed from the beginning and end of the time window. 2

write(HecMath dataset) 1

integer

Write the data set to the DSS file. A return value of zero indicates success.

write (TimeSeriesMath timeSeriesData, string storeMethod)

integer

See table following


1 Currently, the read(…) and write(…) methods can operate only on time-series data, paired data, stream rating data, and text data  represented by TimeSeriesMath, PairedDataMath, and StreamRatingMath TextMath objects, respectively.  Other record types, such as gridded data are not yet supported by these methods.

2 By default, TimeSeriesMath objects retrieved via calls to read(…) contain data only between the first non-missing value and the last non-missing value within the specified time window.  Calling setTrimMissing(False) causes the data retrieved to include all data for the specified time window, including blocks of missing values at the beginning and end of the time window.


Time Series Data Storage Constants (Regular and Irregular)


Integer

String

Description

Regular

0

REPLACE_ALL

Overwrite every value in the existing data


1

REPLACE_MISSING_VALUES_ONLY

Don't overwrite non-missing values in the existing data


2

REPLACE_ALL_CREATE

REPLACE_ALL + if new data has entire records of missing data, create empty records in existing data


3

REPLACE_ALL_DELETE

REPLACE_ALL + if new data has entire records of missing data, delete records from existing data


4

REPLACE_WITH_NON_MISSING

Overwrite every value in the existing data only if replacement is non-missing

Irregular

0

MERGE

Result for time window will be combination of existing and specified data


1

DELETE_INSERT

Result for time window will be specified data only




Pattern Strings


The pattern strings used with the getCatalogedPathnames(…) methods filter the list of returned pathnames in a user-specified manner. Pattern strings can be specified in two modes: pathname mode and pathname part mode. Both modes make use of pathname part filters: 

  • Pathname mode = /Afilter/Bfilter/Cfilter/Dfilter/Efilter/Ffilter/
  • Pathname part mode = [A=Afilter] [B=Bfilter] [C=Cfilter] [D=Dfilter] [E=Afilter] [F=Ffilter]


In pathname mode, filters must be supplied for all pathname parts.  In pathname part mode, only those parts that will not match everything must be specified.


Filters are comprised of the following components:


Normal text characters.  These characters are interpreted as they appear.  For example, a pattern string of "B=XYZ" specifies matching every pathname that has a B-part of "XYZ".

Special characters.  The special characters are comprised of the following list:

  • '@' or '*' (used interchangeably).  This character can be specified as the first and/or last character of a filter and specifies matching a string of zero or more characters.  For example, a pattern string of "B=XYZ@" specifies matching every pathname that as a B-part that begins with "XYZ".  A pattern string of "B=*XYZ" specifies matching every pathname that has a B-part that ends with "XYZ".  A pattern string of “B=*XYZ*” specifies matching every pathname that "XYZ" anywhere in the B-part.  Note that the @ or * character must be the first and/or last character of the filter (e.g. a pattern string of "B=ABC*XYZ" is invalid).
  • '#' or '!' (used interchangeably).  This character must be specified as the first character of a filter and specifies matching of every string except the remainder of the filter (e.g. it negates the remainder of the filter).  A pattern string of "B=!XYZ*" specifies matching every pathname that does not have a B-part that begins with "XYZ".
  • No character. The absence of any character specifies an empty filter, which matches a blank pathname part.  Both of the following pattern strings match every pathname that has a blank A-part

    • "//*/*/*/*/*/"
          ●  "A="

Using getCatalogedPathnames(…) with a pattern string utilizes the pathname matching in the underlying DSS library. To accomplish more sophisticated pathname filtering, use one of the getCatalogedPathnames(…) methods in conjunction with python text parsing and matching utilities.

Time Windows


Dates and times used to specify time windows should not contain blank characters and should include the 4 digit year. An example date and time is "04MAR2003 1400". If a single string is used to specify the time window, the starting date and time must precede the ending date and time, for example dssFile.setTimeWindow ("04MAR2003 1400 06APR2004 0900"). A relative time may be used in the single string command, where the letter "T" represents the current time, and the days or hours can be subtracted or added to that. For example dssFile.setTimeWindow("T-14D T-2H") would specify the starting time as current time minus fourteen days and the ending time as the current time minus two hours.


Time Windows (specified by starting time and ending time) effect how the DSSFile.read(…) method operates for the various data types.
Time-series data. The time window supplied to DSS.open(…) or dssFile.setTimeWindow(…) specifies the default time window for all subsequent dssFile.read(…) operations. If no time window is supplied to DSS.open(…) then the default time window is undefined until dssFile.setTimeWindow(…) is called. If the default time window is undefined, then all dssFile.read(…) operations involving time-series data must specify a time window either implicitly via the D-part of the specified pathname or explicitly via the startTime and endTime parameters. The order of precedence for time windows is as follows:
Explicit time window. Specified in dssFile.read(pathname, startTime, endTime) or dssFile.read(pathname, timeWindow). The D-part of the pathname is ignored and may be empty.
Default time window. Specified in DSS.open(filename, startTime, endTime) or DSS.open(filename, timeWindow) or dssFile.setTimeWindow(startTime, endTime) or dssFile.setTimeWindow(timeWindow). The D-part of the pathname is ignored and may be empty. The default time window can be set to undefined by calling dssFile.setTimeWindow("", "").
Implicit time window. Specified as the D-part of the pathname supplied to dssFile.read(pathname) when the default time window is undefined. The D-part of the pathname must not be empty.
Paired data. Time windows have no effect on reading paired data records.
Stream rating data. Stream rating data are comprised of a time-series of individual rating records for a common location. The reading of stream rating data is not affected by default time windows. The implicit time window for reading stream rating data is the entire time span covered by the rating records. An explicit time window may be supplied by using the dssFile.read(pathname, startTime, endTime) method. If an explicit time window is specified, a (possible) subset of the rating records is retrieved that
cover the specified time window. The explicit time window is interpreted as the time window containing all time-series data that is to be rated via the stream rating data. The set of rating records retrieved for an explicit time window meets the following criteria.
● The earliest record retrieved is the latest rating that is on or before the start of the time window. If no such record exists, the earliest record in the rating time-series is retrieved.
● The latest record retrieved is the earliest rating that is on or after the end of the time window. If no such record exists, the latest record in the rating time-series is retrieved.
● All rating records between the earliest and latest retrieved records are also retrieved.
Text data. Time windows have no effect on reading text records.