CSEP1 Status

CSEP Working Group Home Page

Introduction

The CSEP1 system has been in production for over a decade with dozens of models running prospectively during this time period. Currently, we do not have an understanding of the status of the operational system with respect to missing forecasts and their evaluations. This page provides the status of the CSEP1 system and outlines our work toward understanding the status of the CSEP1 operational system.

Data Model

CSEP1 simulation meta data represented as SQL schema. Extraction scripts can be found on GitHub at https://github.com/wsavran/csep_db/.

   CREATE TABLE IF NOT EXISTS ScheduledForecasts (
       scheduled_forecast_id INTEGER PRIMARY KEY,
       date_time TEXT NOT NULL
   );

   CREATE TABLE IF NOT EXISTS ScheduledEvaluations (
       scheduled_evaluation_id INTEGER PRIMARY KEY,
       date_time TEXT NOT NULL
   );

   CREATE TABLE IF NOT EXISTS Dispatchers (
       dispatcher_id INTEGER PRIMARY KEY,
       script_name TEXT NOT NULL,
       config_file_name TEXT NOT NULL
   );

   CREATE TABLE IF NOT EXISTS ForecastGroups (
       forecastgroup_id INTEGER PRIMARY KEY,
       group_name TEXT NOT NULL,
       config_filepath TEXT NOT NULL,
       dispatcher_id INTEGER NOT NULL,
       FOREIGN KEY(dispatcher_id) REFERENCES Dispatchers
   );

   CREATE TABLE IF NOT EXISTS Forecasts (
       forecast_id INTEGER PRIMARY KEY,
       schedule_id INTEGER NOT NULL,
       group_id INTEGER NOT NULL,
       name TEXT NOT NULL,
       filepath TEXT,
       meta_filepath TEXT,
       waiting_period TEXT,
       logfile TEXT,
       status TEXT,
       FOREIGN KEY(schedule_id) REFERENCES ScheduledForecasts,
       FOREIGN KEY(group_id) REFERENCES ForecastGroups
   );

   CREATE TABLE IF NOT EXISTS Evaluations (
       evaluation_id INTEGER PRIMARY KEY,
       scheduled_id INTEGER NOT NULL,
       forecast_id INTEGER NOT NULL,
       compute_datetime TEXT,
       filepath TEXT,
       name TEXT,
       status TEXT,
       FOREIGN KEY(scheduled_id) REFERENCES ScheduledEvaluations,
       FOREIGN KEY(forecast_id) REFERENCES Forecasts
   );

Verifying Database

In this section, we manually verify the integrity of the csep_db contents to ensure that the database is accurately representing the information present on the CSEP operational system. We show the verification results for a single dispatcher script.

1) Dispatcher Scripts

  sqlite> select * from Dispatchers;
  1|/usr/local/csep/cronjobs/dispatcher_ANSS1985_one_day.tcsh|/usr/local/csep/cronjobs/dispatcher_ANSS1985_one_day.init.xml

# Crontab file for CSEP periodic processes
  
  # Usage:
  # crontab $CSEP/cronjobs/csep_cron_table - to install cronjobs
  # crontab -e - to edit cronjobs
  # crontab -l - to view installed cronjobs
  # crontab -r - to remove cronjobs
  
  # Schedule format as follows:
  # Minute	Hour	Day of month	Month	Day of week	Command
  # (0-59)	(0-23)	(1-31)		(1-12)	(0-6)		command to run
  
  00 00  *   *   *   /bin/tcsh /usr/local/csep/cronjobs/dispatcher_ANSS1985_forecasts.tcsh >& /home/csep/crontab_logs/daily_ANSS1985_forecasts_log.txt;

  # Invoke Dispatcher for "today" (with 31 day delay for the testing date):
  nohup python3 $CENTERCODE/src/generic/Dispatcher.py --year="$year" --month="$month" --day="$day" 
  --configFile=/usr/local/csep/cronjobs/dispatcher_ANSS1985_one_day.init.xml --waitingPeriod=31 --enableForecastXMLTemplate -- 
  enableForecastCompression --enableForecastMap --publishServer=csep-usc@cseppublishing.usc.edu -- 
  publishDirectory=/export/cseppublishing/csep/us/results/data/us/usc/california --logFile="$logfile" >& "$logfile" &

2) Forecast Groups

3) Forecasts

4) Evaluations