System Overview



Python Libraries

  1. CWMS-Python: A wrapper for CDA. A way for the district to read/write into the CWMS database.
  2. DSS-Python: Package to interact with DSS in python.
  3. CWMS-CLI: Command line utility to run cwms functions (i.e. run USGS scripts, save file to BLOB, setup scripts for SHEF imports and USGS ratings script).
  4. SHEF-Parser: Package to interact with SHEF files including parsing SHEF files and storing into CWMS database or DSS.  Can also extract data and store into a SHEF file.
  5. HEC-Python: This package provides many of the capabilities of using Jython to access HEC's Java class libraries. The library is focused on working with time series objects and associated infrastructure:

OpenDCS/CCP

The Open Data Collection System (OpenDCS) is an open-source software is used primarily for collecting, decoding, storing, and distributing environmental time-series data. 

Cumulus

Cumulus serves as the centralized, national gridded data service for CWMS. It collects, archives, and delivers real-time and forecasted meteorological data—primarily precipitation, temperature, and snow—from the National Weather Service and other trusted sources. Users can access and download data on demand via the Cumulus website or through the extract process integrated within HEC-RTS.

A2W

X

CWMS Database

The CWMS database is for Corps offices to store time series and relational data used in modeling CWMS watersheds. Oracle was chosen as the database facilitator because it is a widely used database and follows the corporate guidelines for USACE. The CWMS system provides a variety of ways to interact with the CWMS database, including PL/SQL, CDA, and various application programs. The CWMS database schema also stores procedures used to write and receive data to the database.

CWMS Data API

The CWMS Data API (CDA) is a Representational State Transfer (REST) API that enables secure data exchange between computer systems over the internet. CDA serves as the critical interface for transferring Water Management data to and from cloud environment. It allows users to send and receive information through various methods over a secure internet connection. The API supports authentication, granting write access only to authorized users while allowing the public to view publicly available data. 

CWMS-Batch

CWMS-Batch is a tool for districts to run scheduled custom python or Java scripts. This tool is triggered by Airflow to run Hourly/Daily. Districts can add/update/remove scripts through a district specific GitHub repo. CWMS-Batch uses a standard Image to run scripts with python modules and java modules.

CWMS-Batch-Event/Jobs

An API was added for CWMS-Batch that allows it to be accessed and run from the workstation. District staff can individual scripts in a manual way outside of the Airflow - CWMS-Batch workflow. 

Airflow

Pure python platform to programmatically author, schedule, and monitor workflows. Airflow is used to ...

LDM

Local Data Model, this service connects to NWS River Forecast Centers to send and receive data files, that include forecast datasets, to Cumulus and the CWMS Database. The SHEF-Parser code is used to format data, and scripts are run from CWMS-Batch and scheduled by Airflow. 

CWMS Monitoring

X

Web Data Entry:

X