CSEP 1 STEPCoulombQuickFix
Introduction
RFC002 discusses the issues observed with the STEPCoulomb model in the CSEP New Zealand testing center. CSEP provides a computational infrastructure to perform and evaluate both prospective and retrospective earthquake forecasting experiments. In regards to a retrospective forecasting experiment, it was observed that the STEPCoulomb model installed in CSEP did not produce the expected output for the Darfield Experiment -- a forecasting experiment of the 2010 Christchurch Earthquake sequence.
The STEPCoulomb model represents a hybrid model consisting of a statistical STEP model and a physics-based Coulomb stress model. The executable that computes the STEP forecast obtains a Coulomb mask based on available slip models, and uses the information contained from the Coulomb stress model to provide physical constraints to the statistical STEP (short-term earthquake prediction) forecast. The STEPCoulomb forecast should produce two forecast outputs: (1) STEP forecast, and (2) STEPCoulomb forecast, which can be compared to determine the pairwise efficacy of the two forecasting models.
This document is laid out as follows: Problem Statement provides a description of the problems as communicated by the science groups to SCEC IT; and Proposed Solution provides an explanation of the proposed solution to the problem outlined in the problem statement. Implemented Solution outlines the specific steps taken to solve the problem.
Problem Statement
1. Overview
It was suspected the STEPCoulomb forecast, computed by the New Zealand CSEP testing center, did not produce the expected outputs. This issue affects an on-going manuscript review, so an urgent response is needed. The problem was confirmed by comparing the STEP forecast with the supposed STEPCoulomb forecast and determining that there were no differences between the forecasts. Computing differences between the two forecasts indicated that both files were identical. It is expected that the physics-based Coulomb stress model should influence the rates forecasted by the statistical STEP model.
2. Detailed Explanation
When invoking the STEPCoulomb model, CSEP makes sequential, external calls to two executables: (1) STEPCoulomb and (2) step-aftershock. In model (1), finite-fault slip models are used to produce bitwise masks based on Coulomb stress calculations. The masks produced by model (1) are meant to be read by model (2), and used to update the forecasted rates. To facilitate the interchange of these executables, CSEP creates 'CSEP-specific' input files to define the file paths for forecast files and associated metadata on the CSEP filesystem. For example, the file located at
/home/csep.canterbury/DarfieldExperiment/bestAvailableTC/batch/runs/csep/2015_11/20151109214739/pid_3368/scec.csep.STEPCOULOMBOneDay.CoulombPart_STEPCOULOMBOneDay_control.dat.1447165313.806906.1
provides the CSEP-specific inputs to forecast model (1). An example input file for (2) can be found at:
/home/csep.canterbury/DarfieldExperiment/bestAvailableTC/batch/runs/csep/2015_11/20151109214739/pid_3368/scec.csep.STEPCOULOMBOneDay.STEPCOULOMBOneDay_control.dat.1447165313.743799.1
If we compare the contents of this input file with the source code of executable (2), we find that there is a bug which would produce the issue described in the overview section. The contents of the linked input file for exectuable (2) are shown below:
1 1 2010 0 0 0 29 11 2010 16 35 42 1 /home/csep/DarfieldExperiment/bestAvailableTC/batch/runs/csep/2015_11/20151109214739/pid_3368/OneDayModelInputCatalog.dat /home/csep/DarfieldExperiment/bestAvailableTC/one-day-forecasts/forecasts/STEPCOULOMBOneDay_11_29_2010.16_35_42.dat /home/csep/csep-install/src/SCECModels/NewZealand/src/STEPCoulombModel/OpenSHA/NZZeroRate05.dat /home/csep/DarfieldExperiment/bestAvailableTC/one-day-forecasts/forecasts/CoulombPart_STEPCOULOMBOneDay_11_29_2010.16_35_42.dat
The code below shows the RegionDefaults.setCsepParams(...) function that is responsible for reading the CSEP generate input file and overwriting the default parameters of the model.
public static void setCsepParams(String filePath)
throws FileNotFoundException, IOException
{
logger.info(">> setCsepParams filePath " + filePath);
ArrayList<String> fileLines = FileUtils.loadFile(filePath);
String line = (String)fileLines.get(0);
logger.info("catalog start date " + line);
EVENT_START_TIME = parseCsepTime2Cal(line);
logger.info("EVENT_START_TIME " + dateformater.format(EVENT_START_TIME.getTime()));
line = (String)fileLines.get(1);
logger.info("catalog end date " + line);
forecastStartTime = parseCsepTime2Cal(line);
logger.info("forecastStartTime " + dateformater.format(forecastStartTime.getTime()));
line = (String)fileLines.get(2);
logger.info("forecast length " + line);
forecastLengthDays = Integer.parseInt(line.trim());
line = (String)fileLines.get(3);
logger.info("catalog input file " + line);
cubeFilePath = line;
line = (String)fileLines.get(4);
logger.info("forecast output file " + line);
outputAftershockRatePath = line;
line = (String)fileLines.get(5);
logger.info("background rate input file " + line);
BACKGROUND_RATES_FILE_NAME = line;
line = (String)fileLines.get(6);
logger.info("Coulomb input file " + line);
coulombFilterPath = line;
}
We see that the output file name is assigned to the variable outputAftershockRatePath which defines the output for the pure STEP forecast. For more details on this aspect, view the method STEP_main.saveRatesFile(...). It shows that outputAftershockRatePath defines the output file path of the pure STEP forecast. The Coulomb portion of the STEPCoulomb model is implemented in "ApplyCoulombFilter.applyFilter(...)". The IO portion of this method is shown below:
try
{
FileWriter fr = new FileWriter(RegionDefaults.outputCoulombRatePath);
bgRegionSize = bgLocList.size();
for (int k = 0; k < bgRegionSize; k++)
{
Location bgLoc = bgLocList.getLocationAt(k);
HypoMagFreqDistAtLoc bgDistAtLoc = bggrid.getHypoMagFreqDistAtLoc(k);
if (bgDistAtLoc != null)
{
int numForecastMags = 1 + (int)((RegionDefaults.maxForecastMag - RegionDefaults.minForecastMag) / RegionDefaults.deltaForecastMag);
for (int index = 0; index < numForecastMags; index++)
{
double mag = RegionDefaults.minForecastMag + index * RegionDefaults.deltaForecastMag;
double wrate = bgDistAtLoc.getFirstMagFreqDist().getIncrRate(mag);
if (wrate == 0.0D) {
wrate = 1.0E-11D;
}
fr.write(locFormat.format(bgLoc.getLongitude() - RegionDefaults.gridSpacing / 2.0D) + " " + locFormat.format(bgLoc.getLongitude() + RegionDefaults.gridSpacing / 2.0D) + " " + locFormat.format(bgLoc.getLatitude() - RegionDefaults.gridSpacing / 2.0D) + " " + locFormat.format(bgLoc.getLatitude() + RegionDefaults.gridSpacing / 2.0D) + " " + RegionDefaults.MIN_Z + " " + RegionDefaults.MAX_Z + " " + magFormat.format(mag - RegionDefaults.deltaForecastMag / 2.0D) + " " + magFormat.format(mag + RegionDefaults.deltaForecastMag / 2.0D) + " " + wrate + " 1\n");
}
}
}
fr.close();
}
catch (IOException ee)
{
ee.printStackTrace();
}
We see that a FileWriter is created using the filepath of "RegionDefaults.outputCoulombRatePath" which is hard-coded to be "./coulombRates.txt." As seen by the "RegionDefaults.setCsepParams(...)" code listen above, this location is not overwritten by the CSEP generate input files. This indicates that the Coulomb Rates are not being correctly stored to the CSEP file system and are being overwritten with each subsequent run.
Proposed Solution
There are two potential ways to solve this problem. The decision depends on how we want CSEP to treat the individual forecasting models, and considerations with modifying the step-aftershock.jar executable.
1. Output two forecast models from step-aftershock
This solution would require modifying STEPCoulombModel.writeSTEPFile(...) in the CSEP generic source code to accept the additional argument required to write out the STEPCoulomb. This would also require modifying RegionDefaults.setCsepParams(...) in the step-aftershock source code to accept this argument. It's currently uncertain if CSEP will respond well to a model producing two independent forecast files. This is not a concern for computing the forecasts, but could cause issues if trying to run evalulations within CSEP. This solution seems to be the most straightforward.
[Favored] 2. Have STEPCoulomb output one forecast and run STEP to output the STEP only forecast
This solution would only require changing the RegionDefaults.setCsepParams(...) method to read the CSEP command line argument to the correct variable "outputCoulombRatePath". We would also need to supply configuring the pure STEP model as an additional model in the forecast group configuration. This would require likely more than twice the computational time, because the currently installed STEP model is written in Matlab. I am unsure how long the forecasts take to run in Matlab. However, this solution seems more in line with the CSEP philosophy.
To implement this solution elegantly in CSEP we will register a new model that represents the true STEPCoulomb model. This model will use a separate executable containing the proposed solution outlined in this document. The CSEP system will treat the two models independently, and allow evaluations to be computed using the standard CSEP methodology.
Implemented Solution
All changes made on Git branch wsavran/step-coulomb-rfc002. Branch has not been pushed to remote. Waiting for confirmation on proper way to proceed to uphold CSEP values of transparency and reproducibility.
1. Modifications to step-aftershock
git diff HEAD^ HEAD
diff --git a/src/SCECModels/NewZealand/src/STEPCoulombModel/OpenSHA/classes/.gitignore b/src/SCECModels/NewZealand/src/STEPCoulombModel/OpenSHA/classes/.gitignore
new file mode 100644
index 0000000..3f20248
--- /dev/null
+++ b/src/SCECModels/NewZealand/src/STEPCoulombModel/OpenSHA/classes/.gitignore
@@ -0,0 +1,6 @@
+/junk/
+/scratch/
+/Makefile
+/log4j.properties
+/org/
+/resources/
diff --git a/src/SCECModels/NewZealand/src/STEPCoulombModel/OpenSHA/src/org/opensha/step/calc/RegionDefaults.java b/src/SCECModels/NewZealand/src/STEPCoulombModel/OpenSHA/src/org/opensha/step/calc/RegionDefaults.java
index 8943db8..37a3304 100644
--- a/src/SCECModels/NewZealand/src/STEPCoulombModel/OpenSHA/src/org/opensha/step/calc/RegionDefaults.java
+++ b/src/SCECModels/NewZealand/src/STEPCoulombModel/OpenSHA/src/org/opensha/step/calc/RegionDefaults.java
@@ -401,7 +401,7 @@ public class RegionDefaults {
cubeFilePath = line;
line = fileLines.get(4);
logger.info("forecast output file " + line);
- outputAftershockRatePath = line;//OUTPUT_DIR + "/" + props.getProperty("output.file.time.dep.rates", "TimeDepRates.txt");
+ outputCoulombRatePath = line;//OUTPUT_DIR + "/" + props.getProperty("output.file.time.dep.rates", "TimeDepRates.txt");
line = fileLines.get(5);
logger.info("background rate input file " + line);
2. Modifications to CSEP system
3. Reprocessing Specifics
