District Cloud Migration Phases

This roadmap outlines a generalized approach to cloud migration intended to provide a consistent framework across all offices. While it serves as a guiding structure for planning and execution, the specific activities, timelines, and sequencing within each phase may vary based on the unique technical and operational requirements of each individual office. 

Phase 1: Pre-Migration (CWBI-DEV)

Step 1: Convert/Test All Data Processes to use Cloud Ready Technologies on T7

  • Task 1.1 – Converting all custom scripts running from the server to use cwms-python/CDA/OpenDCS using the script decision flow chartas a guide (Tasks 1-5 in Cloud MigrationChecklist)
  • Task 1.2 – actively using the new getUSGS scripts in cwms-cli to get timeseries data/ratings from the USGS on T7 server if applicable 
  • Task 1.3 – Run the critimportfunction to import critical information into the database for processing SHEF files on local T7 server 
  • Task 1.4 – Testing the python shef-parserpackage to process SHEF files on local T7 server 

Step 2: Test/Implement Custom Data Processes in CWBI-Dev and Prepare for Migration

  • Task 2.1 – Load database objects into CWBI-dev database using cwms-cli. Locations, timeseries ids….
  • Task 2.2 – have all custom scripts (data processing and reporting) running in the CWBI-DEV environment batch process.  Have fully tested that they are working as intended. 
  • Task 2.3 – compiled a list of all data products using LDM either receiving or sending including file names or regex expressions for grabbing the correct data. 
  • Task 2.4 – Identify any custom database schema functions/processes that will be needed in CWBI 

Step 3: Test modeling operations in CAVI/CWMS-Vue using CWBI-dev or CWBI-test

  • Task 3.1 – Connect to cwbi-test and/or CWBI-dev database using client tools and test your districts modeling operations. 


Phase 2: Migration/Testing (CWBI-TEST)

Step 4 - Migrate All Processes to CWBI-TEST

HEC will work with the District to complete all tasks in Step 4, which may include:

  • Migrating any required custom database Schema functions into the CWMS Database Schema
  • Adding all LDM data products to CWBI-TEST
  • Configuring offices for USGS data processes in CWBI-TEST
  • Migrating CWMS-Batch scripts to CWBI-TEST branch
  • Importing OpenDCS configurations

Step 5 - Fully Test All Data Processes and Models in CWBI-TEST

The District is responsible for completing all tasks in Step 5. HEC will work with District staff to determine the appropriate duration for testing. This includes:

  • Confirming all data processes are running successfully
  • Running HEC-RTS Forecasts using processes set up in CWBI-TEST

Step 6 - Fix Any Issues Found During Step 5

HEC will work with the District to complete all tasks in Step 6.

Phase 3: Complete Migration (CWBI-PROD)

Step 7 - Migrate all data processes to CWBI-PROD and verify all components are working

HEC will work with the District to complete all tasks in Step 7, which may include:

  • Adding all LDM data products to CWBI-PROD
  • Configuring offices for USGS data processes in CWBI-PROD
  • Migrating CWMS-Batch scripts to CWBI-PROD branch
  • Importing OpenDCS configurations

Step 8 - Turn off Goldengate and Migrate Historical Data to National Database

HEC is responsible for completing all tasks in Step 8, which may include:

  • Turning off Goldengate connection from T7 to the CWBI-TEST database
  • Importing time series data for the past 2 years into the CWBI-PROD database
  • Migrating all historical data into the CWBI-PROD database

Step 9 - Decommission T7 Server

District staff is responsible for completing Step 9.