Working with DataContainers
The HecDss get() method returns a DataContainer object and the put() method takes a DataContainer object as a parameter. DataContainers are a base for different types of data. The TimeSeriesContainer and PairedDataContainer classes discussed below are both types of DataContainer classes. DataContainer objects have no methods that can be called by the user, but all data fields of DataContainer objects are directly accessible. DataContainer data fields are described in the table below.
You can also exchange DataContainers with HecMath objects. Both TimeSeriesContainer and PairedDataContainer classes are extracted from HecMath objects using the getData() method as documented in Math Functions. HecMath objects can be updated from DataContainer objects using the setData() method as documented in Math Functions.
DataContainer Field | Type | Description |
---|---|---|
fullName | string | The full name associated with the data in the data store (DSS pathname, if the DataContainer is associated with a DSS file) |
location | string | The location associated with the data (DSS pathname B-part if the DataContainer is associated with a DSS file). |
subVersion | string | The sub-version associated with the data. |
version | string | The version associated with the data (DSS pathname F-part if the DataContainer is associated with a DSS file). |
watershed | string | The watershed associated with the data (DSS pathname A-part if the DataContainer is associated with a DSS file). |
Example 19: Extracting a DataContainer from HecMath
from hec.script import HecDss
theFile = HecDss.open("c:/temp/myFile.dss")
flowDataSet = theFile.read("/Basin/loc/FLOW/01NOV2002/1Hour//")
theFile.done()
flowData = flowDataSet.getData()
thisWatershed = flowDataSet.getData().watershed
print(thisWatershed)
TimeSeriesContainer Class
TimeSeriesContainer is a type of DataContainer that contains information about time series data. TimeSeriesContainer objects are returned by the get () method and are required in the put() method of HecDss objects. TimeSeriesContainer objects have all the data fields described above for DataContainerClass, in addition to those described in the table below. New TimeSeriesContainer objects can be created by a script if the TimeSeriesContainer class has been imported from the hec.io module with: "from hec.io import TimeSeriesContainer". The new object can be created by calling TimeSeriesContainer () (e.g. "myTSC = TimeSeriesContainer ()").
TimeSeriesContainer Data Field | Type | Description |
---|---|---|
endTime | Integer1 | The end time of the time window. If the data were retrieved, the end time may be later than the last time in the times list. |
interval | Integer | The interval, in minutes, between each set of consecutive times in the times list. For irregular-interval times, the interval is 0. |
numberValues | Integer | The length of values and times lists. |
parameter | string | The parameter associated with the data. |
quality | list of integers2 | The optional list of quality flags. If this list is present, there must be a quality for each value in the values array. If this list is not present, the quality field is set to None. |
startTime | integer2 | The start time of the time window. If the data were retrieved, the start time may be earlier than the first time in the times list. |
subLocation | string | The sub-location associated with the data. |
subParameter | string | The sub-parameter associated with the data. |
times | list of integers1 | The list of times. There must be a time for each value in the values list, and times must increase from one index to the next. |
timeZoneID | string | The time-zone for times in the times list. If unknown, the timeZoneID field is set to None |
timeZoneRawOffset | integer | The offset, in milliseconds, from UTC to the time zone for the times in the times list. |
type | string | The type of the data (e.g. "INST-VAL", "INST-CUM", ""PER-AVER", "PER-CUM"). |
units | string | The units of the data. |
values | list of floating-point | The data values, each of which has a corresponding time in the times list and optionally a corresponding quality in the quality list. All lists must be the same length. |
1Integer times from this field can be converted to string representations by using the set() and dateAndTime() methods of HecTime objects and can be generated by the HecTime value() method.
2Quality values are interpreted according to the table below.
Data Quality Bit(s) | Description |
---|---|
1 | Set when the datum has been tested (screened). |
2 | Set when the datum passed all tests. |
3 | Set when the datum is missing (either originally missing or set to missing by a test). |
4 | Set when at least one test classified the datum as questionable. |
5 | Set when at least one test classified the datum as rejected. |
6-7 | Set by the RANGE test. Interpreted as a 2-bit unsigned integer with values having the following meanings: |
0 | value of datum < than 1st limit or no range test applied |
1 | 1st limit <= value of datum < 2nd limit |
2 | 2nd limit <= value of datum < 3rd limit |
3 | 3rd limit <= value of datum |
8 | Set when the datum has been changed from the original value. |
9-11 | Datum replacement indicator. Interpreted as a 3-bit unsigned integer with values having the following meanings: |
0 | datum was not replaced (original value) |
1 | datum was replaced by an automated tool |
2 | datum was replaced by an interactive tool (e.g., fill operaton) |
3 | datum was replaced by manual edit in an interactive tool |
4 | original value was accepted or restored in an interactive tool |
5-7 | reserved for future use |
12-15 | Datum replacement value computation method. Interpreted as a 4-bit unsigned integer with values having the following meanings: |
0 | datum was not replaced (original value) |
1 | datum value computed by linear interpolation |
2 | datum value was entered manually |
3 | datum value was replaced with a missing value |
4 | datuma value was entered graphically |
5-15 | reserved for future use |
16 | set when datum failed an absolute magnitude test |
17 | set when datum failed a constant value test |
Bit(s) | Description |
18 | set when datum failed a rate of change test |
19 | set when datum failed a relative magnitude test |
20 | set when datum failed a duration magnitude test |
21 | set when datum failed a negative incremental value test |
22 | reserved for future use |
23 | set when datum is excluded from testing (e.g. DATCHK GAGEFILE entry) |
24 | reserved for future use |
25 | set when datum failed a user-defined test |
26 | set when datum failed a distribution test |
27-31 | reserved for future use |
32 | set when datum is protected from being replaced |
Example 20: Using a TimeSeriesContainer Object
from hec.heclib.dss import HecDss, HecTimeSeries
from hec.io import TimeSeriesContainer
from hec.heclib.util import HecTime
watershed = "Green River"
loc = "OAKVILLE"
param = "STAGE"
ver = "OBS"
startTime = "12Oct2003 0100"
values = [12.36, 12.37, 12.42, 12.55, 12.51, 12.47, 12.43, 12.39]
hecTime = HecTime()
tsc = TimeSeriesContainer()
tsc.watershed = watershed
tsc.location = loc
tsc.parameter = param
tsc.version = ver
tsc.fullName = "/%s/%s/%s//1HOUR/%s/" % (watershed, loc, param, ver)
tsc.interval = 60
hecTime.set(startTime)
times=[]
for value in values:
times.append(hecTime.value())
hecTime.add(tsc.interval)
tsc.values = values
tsc.times = times
tsc.startTime = times[0]
tsc.endTime = times[-1]
tsc.numberValues = len(values)
tsc.units = "FEET"
tsc.type = "INST-VAL"
dssFile = HecDss.open("myFile.dss")
dssFile.put(tsc)
dssFile.done()
PairedDataContainer Class
PairedDataContainer is a type of DataContainer that contains information about paired data. PairedDataContainer objects are returned by the get () method. PairedDataContainer objects have all the data fields described previously for the DataContainer Class, plus those in the table below. New PairedDataContainer objects can be created by a script if the PairedDataContainer class has been imported from the hec.io module (e.g. "from hec.io import PairedDataContainer"). The new object can be created by calling *PairedDataContainer () (e.g. "myPDC = PairedDataContainer ()").
PairedContainer Data Field | Type | Description |
---|---|---|
date | string | The date associated with the paired-data. |
datum | floating-point | The zero-stage elevation of a stream gauge. |
labels | list of strings | The list of labels used to identify each of the y-ordinates lists. If there is only 1 y-ordinates list (e.g. numberCurves == 1), the labels field is set to None. |
labelsUsed | Boolean | A flag specifying whether labels are used. This field is set to True if there is more than 1 y-ordinates list and False otherwise. |
numberCurves | integer | The number of y-ordinates lists. |
numberOrdinates | integer | The length of the x-ordinates list and each of the y-ordinates lists. |
offset | floating-point | The offset value of a stream rating. |
shift | floating-point | The shift value for a stream rating. |
transformType | integer | Type of transformation to use (1 = "LINLIN", 2 = "LOGLOG") |
xOrdinates | list of floating-point | The x-ordinate values. Each y-ordinate values list must be of the same length as this field. |
xparameter | string | The parameter of the x-ordinates. |
xtype | string | The type of the x-ordinates ("UNT" for unitary or "LOG" for logarithmic). |
xunits | string | The units of the x-ordinates. |
yOrdinates | list of lists of floating-point | The y-ordinate values. Each list of y-ordinate values must be of the same length as the list of x-ordinate values. The nth value of each y-ordinates list is associated with the nth value of the x-ordinates list. |
yparameter | string | The parameter of the y-ordinates. |
ytype | string | The type of the y-ordinates ("UNT" for unitary or "LOG" for logarithmic). |
yunits | string | The units of the y-ordinates. |
Example 21: Using a PairedDataContainer Object
from hec.io import PairedDataContainer
from hec.heclib.dss import HecDss
watershed = "GREEN RIVER"
loc = "OAKVILLE"
xParam = "STAGE"
yParam = "FLOW"
date = "12Oct2003"
stages = [0.4, 0.5, 1.0, 2.0, 5.0, 10.0, 12.0]
flows = [0.1, 3, 11, 57, 235, 1150, 3700]
pdc = PairedDataContainer()
pdc.watershed = watershed
pdc.location = loc
pdc.xparameter = xParam
pdc.yparameter = yParam
pdc.version = "v1"
pdc.fullName = "/%s/%s/%s-%s///%s/" % \
(watershed, loc, xParam, yParam, date)
pdc.xOrdinates = stages
pdc.yOrdinates = [flows]
pdc.numberCurves = 1
pdc.numberOrdinates = len(stages)
pdc.labelsUsed = False
pdc.xunits = "FEET"
pdc.yunits = "CFS"
pdc.xtype = "LOG"
pdc.ytype = "LOG"
pdc.xparameter = xParam
pdc.yparameter = yParam
pdc.date = date
pdc.transformType = 2
dssFile = HecDss.open("c:/temp/myFile.dss")
dssFile.put(pdc)
dssFile.done()
Other Types of Data
The HecDss class is focused on supporting Scripting for TimeSeriesContainers and PairedDataContainers. Other data types are also supported by DSS such as GriddedData, Arrays, Text, Locations, Images. However, writing some types of data may require using the appropirate 'Lower' level java DSS API instead of using the HecDss class. For example in the code below the HecDssArray class is used to write an array container to a DSS file. Other examples can be derived from the Java Programmers Documentation.
Example 22: Array Container
from hec.io import ArrayContainer
from hec.heclib.dss import HecDssArray
from jarray import array
ac = ArrayContainer()
ac.setFloatArray([1.0, 2.0, 3.14])
dss = HecDssArray("c:/temp/myFile.dss")
dss.setPathname("//script/data///test/")
dss.write(ac)