CSEP 2.0 Developments: Difference between revisions

From SCECpedia
Jump to navigationJump to search
← Older edit
Wsavran (talk | contribs)
659 edits
Wsavran (talk | contribs)
659 edits
No edit summary
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
[[CSEP_Working_Group|CSEP Working Group Home Page]]<br>
[[CSEP_Working_Group|CSEP Working Group Home Page]]<br>
__TOC__
__TOC__
== Proposed software developments needed to test UCERF3 -- CSEP2.0 as a workflow ==
* abstract goes here.
* summary statement of proposal
* clearly state proposed plans for putting ucerf3 under test
* [under development until remainder of proposal is written]


== Introduction ==
== Introduction ==
Retrospective experiments of earthquake forecasts represent an important aspect of the CSEP software infrastructure. They provide significant insight into the performance of earthquake forecasting models and allow scientists to evaluate the performance of forecasts on historic earthquake sequences. The CSEP1 system is designed to support retrospective forecast experiments in addition to prospective forecasting. CSEP operates entirely in serial; meaning that both retrospective and prospective processing capabilities must happen sequentially. This sequential behavior is enforced within CSEP using locks that are placed on active directories during computation. While the serial nature of the system is not strictly enforced, working around the CSEPLocks require non-elegant hacks that violate the core concept of forecast groups within CSEP. This document explores the possibility of repurposing the current CSEP framework into a module to exclusively facilitate retrospective forecasting experiments.
* state reasons for transitioning from CSEP1 -> CSEP2
* introduce technical/scientific challenges of testing UCERF3 etas
* introduce the csep problem as a workflow
* describe remaining software infrastructure
* 'on-demand' prospective/retrospective experiments


; Guiding Principals
; Guiding Principals
Line 11: Line 21:
# Reproducibility
# Reproducibility


The locks are used to prevent data contamination, and to prevent overwriting data by a different process. This is required because CSEP uses the same directory for all forecast models before moving forecast files to archive directories
== Testing the Next Generation of Earthquake Forecasts ==
* cover technical/scientific challenges of testing UCERF3 etas
* fault and stochastic event sets
* potential models under forecast
* make point that software developments are required to test CSEP2.0
** software developments
** first step to evaluation forecast is to compute forecasts
 
== CSEP2.0 as a Workflow ==
* proposed software and hardware to test CSEP2.0
* describe tech used to achieve guiding principals
** controlled environment -> containers
** transparency -> web application (results viewer)
** comparability -> (open sourced evaluations)
** reproducibility -> containers and workflow
* decoupling of forecasts and evaluations
* proposal to open-source experiments to community
* versioning of catalog data
* introduce the minimum viable product to run Open-SHA from container in pegasus
 
== Proposed scientific questions for testing new Forecasting models ==
* do faults provide improved forecasting skill?
* retrospective experiements for past earthquakes
** el mayor cucupah
** landers
** napa valley
** northridge (unknown fault... hinterlands)
* simple versus complex models
* comparison against csep1 simulations (use case for csep1 dataset)
** need method for comparing rate based forecasts and fault-based simulation based forecasts
 
== Roadmap toward testing UCERF3 ==
* describe specific steps required for testing UCERF3
* essentially, previous sections written as actionable statements from previous section
* include timeline
 
== Conclusion ==
* wrap up proposal with brief paragraph reiterating main points

Latest revision as of 23:37, 28 June 2018

CSEP Working Group Home Page

Proposed software developments needed to test UCERF3 -- CSEP2.0 as a workflow

  • abstract goes here.
  • summary statement of proposal
  • clearly state proposed plans for putting ucerf3 under test
  • [under development until remainder of proposal is written]

Introduction

  • state reasons for transitioning from CSEP1 -> CSEP2
  • introduce technical/scientific challenges of testing UCERF3 etas
  • introduce the csep problem as a workflow
  • describe remaining software infrastructure
  • 'on-demand' prospective/retrospective experiments
Guiding Principals
  1. Controlled Environment
  2. Transparency
  3. Comparability
  4. Reproducibility

Testing the Next Generation of Earthquake Forecasts

  • cover technical/scientific challenges of testing UCERF3 etas
  • fault and stochastic event sets
  • potential models under forecast
  • make point that software developments are required to test CSEP2.0
    • software developments
    • first step to evaluation forecast is to compute forecasts

CSEP2.0 as a Workflow

  • proposed software and hardware to test CSEP2.0
  • describe tech used to achieve guiding principals
    • controlled environment -> containers
    • transparency -> web application (results viewer)
    • comparability -> (open sourced evaluations)
    • reproducibility -> containers and workflow
  • decoupling of forecasts and evaluations
  • proposal to open-source experiments to community
  • versioning of catalog data
  • introduce the minimum viable product to run Open-SHA from container in pegasus

Proposed scientific questions for testing new Forecasting models

  • do faults provide improved forecasting skill?
  • retrospective experiements for past earthquakes
    • el mayor cucupah
    • landers
    • napa valley
    • northridge (unknown fault... hinterlands)
  • simple versus complex models
  • comparison against csep1 simulations (use case for csep1 dataset)
    • need method for comparing rate based forecasts and fault-based simulation based forecasts

Roadmap toward testing UCERF3

  • describe specific steps required for testing UCERF3
  • essentially, previous sections written as actionable statements from previous section
  • include timeline

Conclusion

  • wrap up proposal with brief paragraph reiterating main points
Retrieved from "https://strike.scec.org/index.php?title=CSEP_2.0_Developments&oldid=21182"