SHEF Exporter Tutorial (CWMS Database → SHEF file)

This tutorial describes how to export time series data from a CWMS database to a SHEF file using the shefParser export workflow, including how to set up the required Time Series Group → SHEF Export mapping and how to validate the exported file. It also includes an example pattern for sending the resulting file onward via Batch → LDM.

Overview

What you’re doing

  • Select CWMS time series (either by Time Series Group or explicit TSIDs)
  • Export their values over a specified time window
  • Write the result as a SHEF file (or to stdout)

Workflow

  • CWMS database → (CDA API) → shefParser export → .SHEF file → (optional) Batch script → LDM product

1) Prerequisite: Create a SHEF Export Time Series Group

To export using a group, you first define a group under SHEF Export and populate it with the time series you want to export.

1.1 Create the group

In your CWMS UI / client tools:

  • Navigate to Time Series Groups
  • Under the category SHEF Export, create a new group (custom name)

1.2 Add time series to the group

Add the desired TSIDs to the group (e.g., stage, precip, pool elevation).

1.3 Use ln file import tool in cwms-cli to create the timeseries group and add timeseries to it

If you are using the existing exportSHEF script you can utilize a cwms-cli tool to process you .in file, create a timeseries group and add all of the timeseries in the .in file to that group.

cwms-cli shef import_infile -f infile.in -g ts_group_name -a $API_ROOT -o MVP -k $API_KEY


Why this matters

This provides the mapping between:

  • Time Series IDs in CWMS
  • The set of series you intend to export as a coherent SHEF product

2) Install / verify shefParser export command

Make sure shef-parser is installed (same package used for parsing/ingest):


python3 -m pip install shef-parser
shefParser export --help

  

3) Export via command line (shefParser export)

3.1 Required inputs

From the CLI help, the key parameters are:

  • -a, --api-root: CDA base URL (can also come from environment variable like CDA_API_ROOT depending on your setup)
  • -o, --office: CWMS office (required)
  • Either:
    • -tsg, --timeseries-group: time series group id to export, or
    • -tsids, --timeseries-ids: comma-separated list of TSIDs
  • -f, --export-file: output SHEF filename (defaults to stdout)
  • -st, --start-time: start time (ISO or HEC “T-time”)
  • -et, --end-time: end time (ISO or HEC “T-time”)
  • -v, --loglevel: logging verbosity (optional)

3.2 Time window formats (HEC time vs ISO)

Start/end time may be specified using either:

  • HEC “T-time” like: T, T-1D, T-1Hr, T-2Hr
  • Date tokens like: 23Feb2026
  • ISO timestamps like: 2026-02-26T00:00:00

3.3 Example: Export using a Time Series Group

Example pattern (shown in the “Exporting via command line” figure on page 7):


shefParser export -a <api_root> -o <office> -f <filename.shef> -st <start_time> -et <end_time> -tsg <timeseries_group>

Concrete example:

shefParser export -a https://cwms-data-test.cwbi.us/cwms-data/ -o SWT -tsg KEYS -st T-2Hr -et T -f test.SHEF

3.4 What “success” looks like

A typical successful run shows:

  • CDA loader initialized
  • Export time window printed
  • “Saving data to file …”
  • A summary similar to:
    • values output in ... messages from ... time series

4) Validate the exported SHEF file

Open the resulting .SHEF file and confirm it contains expected messages/values.

Example content shows repeated blocks like:

  • Station identifier and timestamp line, then data line(s), e.g.:
    • .E KEY02 20260210 Z DH100000/...
    • .E1 .../.../.../

Suggested checks:

  • Station / location identifiers are correct
  • Time stamps match the export window
  • Parameter codes/values appear reasonable (no empty sections, obvious missing data)

5) Export via Python (example)

If you need to drive export programmatically, you can call the export function directly.

Example pattern shown in the code figure (page 11), cleaned up:

import os 
from datetime import datetime, timedelta 
from shef import shef_parser 

cda_url = os.getenv("CDA_API_ROOT") 
office_id = os.getenv("CDA_OFFICE_ID") 
tsids = [ # "A.B.C.D.E.F", # ... ] 

export_file = "/path/to/output_file" 
shef_parser.export( api_root=cda_url, 
					office=office_id, 
					export_file=export_file, 
					timeseries_ids=tsids, 
					start_time=datetime.now() - timedelta(days=1), 
					end_time=datetime.now(), )

Notes:

  • Use timeseries_ids when you want explicit control over exactly what is exported.
  • Use CLI + -tsg when you want the database-managed group to define the export set.

6) (Optional) Send the exported SHEF file via Batch → LDM

If your workflow includes publishing the exported SHEF product to LDM:

  • Prepare a batch/job script that:
    • runs the export
    • posts the resulting file to an LDM endpoint/product

An example reference is provided here:

And an example product page is shown here:

Quick checklist

  • Create Time Series Group under SHEF Export and add TSIDs
  • Confirm shefParser export --help works
  • Run shefParser export with:
    • -a api root
    • -o office
    • -tsg group (or -tsids list)
    • -st / -et time window
    • -f output filename
  • Validate the .SHEF output content