GetUSGS Script using CDA

Instructions on setting up the new getUSGS script that utilizes CDA and cwms-python can be found in the following webinar

\\wpc-netapp3.eis.ds.usace.army.mil\RMCSTORAGE5\MMC_Training_Library\2024.04-30-CWMS-Scripting

The code for this can be found at: https://github.com/HydrologicEngineeringCenter/CWMS-data-acquisition-python/blob/main/src/getUSGS/getUSGS_CDA.py

Step 1 Setup Location Group

Setup a location group to link the location to a USGS station number

Goto the "Location Group" tab in CWMS-Vue
Expand "Agency Aliases", then expand "USGS Station Number"
While "USGS Station Number is selected, hover mouse and right click to bring up menu
Select "Edit Location"
A pop up window titles "Edit Location Assignments" will open
Click in the "Location Column and sort the Locations
Navigate to the Location and check the box "assigned" column. Click OK
Add the USGS number to the location in the "Alias" column. Double click into the box and enter the USGS number. Once entered, clock out of the text box. Push the save button (bottom right of screen) to save all changed.

Notes: If you have multiple locations (i.e base and sub locations) that reference the same USGS station number, assign the USGS station number to the Base location. An alias can only be assigned to a single location and must be unique similar to the Location ID. The USGS script will propagate the USGS station ID to all sublocations if one has not already been assigned to that sublocation. For example in the below scenario the USGS ID assigned to the base location Barker will propagate to the sublocations Barker-Gate 1, Barker-Gate 2, Barker-Gate 3, and Barker-Gate_Total.

Step 3 Setup Time Series Group

Setup the time series groups to link the individual CWMS time series to the USGS parameter and gage (Flow, Stage, etc)

Go to the "Time Series Groups" tb
Expand "Data Acquisition" the expand "USGS TS Data Acquisition"
While "USGS TS Data Acquisition" is selected, hover mouse and right lcik to brgin up menu
Select "Assign Time Series"
A pop up window titles "Assign Time Series" will open
Navigate to and select the Time Series to link to the USGS

Determine the USGS parameter code associated with that timeseries. If the parameter code and CWMS parameter are found in the following table then no further action is needed. However, if the parameter code and associated CWMS parameter are not found in the standard table below then under the "Attribute" column, enter the USGS parameter code associate to the type of data being captured.

CWMS_PARAMETER	USGS_PARAMETER	USGS_Alias
Temp-Water	00010	Water Temp
Temp-Air	00021	Air Temp
Speed-Wind	00035	Wind Speed
Dir-Wind	00036	Wind Dir
Precip-Inc	00045	Precip
Precip	00045	Precip
%-Humidity	00052	RelHumidity
Flow	00060	Flow
Stage	00065	Stage
Cond	00095	Sp Cond
Conc-Salinity	00096	Salinity
Stor	72036	Res Storage
Irrad-Solar	62608	Sol Rad
Elev	63160	Elev-NAVD88

When Looking the new USGS website for the specific parameter you can see the parameter code in the url when the parameter you want to grab is selected. In the case below the parameter for flow is 00060 which is in the table above. Therefore the Attribute field can be left blank or as 0.
Example below of how the Attribute field might be filled in. The first 4 time series are left as 0. There fore according to the table above the Stage timeseries will pull from usgs parameter 00065 and the flow timeseries will pull from 00060. For the Elevation we want to pull the USGS parameter 30211 → Elevation in NGVD 1929 instead of the standard parameter 63160 that pull elevation in NAVD88. Therefore in the Attribute field we enter a value of 30211 to pull the correct USGS parameter.
IF the USGS has multiple readings associated with a single parameter, the USGS method/tsid needs to be added to the Alias column. To get the USGS TSID view the parameter on the USGS site. If multiple method/Timeseries are assigned to the parameter, select the one that you want to grab. The timeseries ID will then show in the URL for that specific method/sub-location.
If the information you are looking for is not on the standard external USGS site you can check the internal site for parameters and tsids, just update the site_no in the following URL. (https://waterdata.usgs.gov/nwis/uv?format=rdb&site_no=08072500&period=PT7D). An example of how the TSID would look in CWMS-Vue is given below
when you entries are complete make sure to hit the save button in the bottom right hand corner to save all of your changes

STEP 3 Setup getUSGS script on T7 server

setup CDA, python3, and cwms-python on your server.
https://www.hec.usace.army.mil/confluence/display/CCM/Setting+up+CDA+writes+and+cwms-python+on+T7s
create and API key for your local CDA instance.
Grab getUSGS_CDA.py script from github and save to your server. https://github.com/HydrologicEngineeringCenter/CWMS-data-acquisition-python/tree/main/src/getUSGS

run getUSGS_CDA.py script.

~/scripts/getUSGS$ python3 getUSGS_CDA.py --help

usage: getUSGS_CDA.py [-h] [-d DAYS_BACK] -o OFFICE -a API_ROOT [-k API_KEY] [-kl API_KEY_LOC]

optional arguments:
  -h, --help            show this help message and exit
  -d DAYS_BACK, --days_back DAYS_BACK
                        Days back from current time to get data. Can be decimal and integer values (default: 1)
  -o OFFICE, --office OFFICE
                        Office to grab data for (Required). (default: None)
  -a API_ROOT, --api_root API_ROOT
                        Api Root for CDA (Required). (default: None)
  -k API_KEY, --api_key API_KEY
                        api key. one of api_key or api_key_loc are required (default: None)
  -kl API_KEY_LOC, --api_key_loc API_KEY_LOC
                        file storing Api Key. One of api_key or api_key_loc are required (default: None)


~/scripts/getUSGS$ python3 getUSGS_CDA.py -d 2 -o NWDM -a $CDA_API_ROOT -kl $CWMS_HOME/.cwms/cda_api_key