CSEP Powell Center 2018 - Revision history

Wsavran at 22:54, 14 March 2018

2018-03-14T22:54:39Z

Wsavran: /* Day 3 */

2018-03-14T22:53:55Z

Day 3

Wsavran: /* Day 3 */

2018-03-14T22:47:12Z

Day 3

Wsavran at 22:46, 14 March 2018

2018-03-14T22:46:55Z

Wsavran at 22:46, 14 March 2018

2018-03-14T22:46:17Z

Wsavran at 22:41, 14 March 2018

2018-03-14T22:41:35Z

Show changes

Wsavran at 22:32, 14 March 2018

2018-03-14T22:32:21Z

Wsavran at 22:26, 14 March 2018

2018-03-14T22:26:39Z

Wsavran at 22:25, 14 March 2018

2018-03-14T22:25:36Z

Wsavran at 22:21, 14 March 2018

2018-03-14T22:21:49Z

@@ Line 359: / Line 359: @@
 * [All] Web based interface would be valuable.
-; Notes: Targeting publication for July/August issue of SRL. Have some time to develop the webpage.
+; Notes about SRL special issue:
 * Expect Press material surrounding the SRL release, so need to be prepared with digestible figures.

@@ Line 304: / Line 304: @@
 #*	Validate
 #*	Valuate
-.	Milestones
+#	Milestones
-"	Develop infrastructure
+#*	Develop infrastructure
-"	Retrospective testing of UCERF3
+#*	Retrospective testing of UCERF3
-"	Prospective testing of UCERF3
+#*	Prospective testing of UCERF3
-"	Comparatively evalulate U3 against empirical models and physics-based models
+#*	Comparatively evalulate U3 against empirical models and physics-based models
-.	5 types of testing
+#	5 types of testing
-o	Exploratory testing: Turing
+#*	Exploratory testing: Turing
-o	Comparatively: T and W tests
+#*	Comparatively: T and W tests
-o	Mean-Hazard testing: null hypothesis significance testing
+#*	Mean-Hazard testing: null hypothesis significance testing
-o	Ontological testing: including epistemic uncertainty
+#*	Ontological testing: including epistemic uncertainty
-o	Sequence-specific testing: testing U3-ETAS against observed aftershock sequences
+#*	Sequence-specific testing: testing U3-ETAS against observed aftershock sequences
-"	Guiding principal: all OEF models should be under continual prospective testing, put ETAS under operational testing. Need to find the value for UCERF3-ETAS testing.
+*	Guiding principal: all OEF models should be under continual prospective testing, put ETAS under operational testing. Need to find the value for UCERF3-ETAS testing.
-"	Could possibly find out what aspects of U3 are superfluous and could simplify the model for using in other locales.
+*	Could possibly find out what aspects of U3 are superfluous and could simplify the model for using in other locales.
-"	Slip-rate data are special for California data sets
+*	Slip-rate data are special for California data sets
-"	Important to compare against physics-based earthquake simulations such as RSQsim to evaluate certain assumptions in the model.
+*	Important to compare against physics-based earthquake simulations such as RSQsim to evaluate certain assumptions in the model.
-"	U3 drivers of CSEP2
+*	UCERF3 drivers of CSEP2
-o	Datasets used in prospective testing must be versioned and archived. Should include analyzed datasets and raw catalogs
+**	Datasets used in prospective testing must be versioned and archived. Should include analyzed datasets and raw catalogs
-o	Testing on simulated event catalogs
+**	Testing on simulated event catalogs
-"	Benefits of U3-TP to the USGS
+*	Benefits of UCERF3-TP to the USGS
-o	Scientific value
+**	Scientific value
-o	Software infrastructure
+**	Software infrastructure
-List of Possible milestones
+;List of Possible milestones
-"	CSEP1.0. What products would be useful?
+*	CSEP1.0. What products would be useful?
-o	Do we need to keep operationalizing CSEP1.0?
+**	Do we need to keep operationalizing CSEP1.0?
-"	[Matt] Mistake to shutdown CSEP1.0
+**	New models necessary?
-o	New models necessary?
+***	[Max] CSEP1 big achievement/success was incorporating new models.
-"	[Max] CSEP1 big achievement/success was incorporating new models.
+*** [Ned] Should support new models but need to be more selective.
-"	[Ned] Should support new models but need to be more selective.
+*** [Max] Wants to published the CA 1-day forecast results.
-o	[Max] Wants to published the CA 1-day forecast results.
+*** [Morgan] Value in providing CSEP1.0 data set publicly.
-o	[Morgan] Value in providing CSEP1.0 data set publicly.
+*** [Mike B.] Need some clear scientific findings and be provocative about this. Need to find models that can be rejected and not worth pursuing. Need to curated in such a way that allows scientists to access and work. Would build community.
-o	[Mike B.] Need some clear scientific findings and provocative about this. Need to find models that can be rejected and not worth pursuing. Need to curated in such a way that allows scientists to access and work. Would build community.
+*** [Peter] Need products that are digestible by the public.
-o	[Peter] Need products that are digestible by the public.
+* Simulation based testing
-"	Simulation based testing
+** Methods:
-o	Methods:
+*** ETAS, U3-ETAS,
-"	ETAS, U3-ETAS,
+** Need: Process for defining timelines, and which models we would be evaluating.
-o	Need: Process for defining timelines, and which models we would be evaluating.
+* Event-triggered/sequence specific testing
-"	Event-triggered/sequence specific testing
+* Comparative valuations
-"	Comparative valuations
+** Turing tests
-o	Turing tests
+** Verification
-o	Verification
+** Inter-comparison of models
-o	Inter-comparison of models
+* Modeling catalog completeness
-"	Modeling catalog completeness
+* Modeling epistemic uncertainties: Important, but IT challenges.
-"	Modeling epistemic uncertainties: Important, but IT challenges.
+* Fault/cell participation
-"	Fault/cell participation
+* [Ned] Testing usefulness!
-"	[Ned] Testing usefulness!
+* Rupture association problem
-"	Rupture association problem
+* Fault characteristics
-"	Fault characteristics
+* UCERF3-TD elastic-rebound testing
-"	U3-TD elastic-rebound testing
+* [Peter] Declustering work could help with the USGS. Lots of ambiguity on different declustering models.
-"	[Peter] Declustering work could help with the USGS. Lots of ambiguity on different declustering models.
+* [Phil] CSEP Should expose its methods so users could leverage the algorithm
-"	[Phil] CSEP Should expose its methods so users could leverage the algorithm
+* [Mike B.] Valuation question most important. Should physics-based simulations be used for Hazard?
-"	[Mike B.] Valuation question most important. Should physics-based simulations be used for Hazard?
+* [Kevin] Errors need to be propagated moving forward!
-"	[Kevin] Errors need to be propagated moving forward!
+* [Phil] Need to figure out how the data will be provided to the scientist.
-"	[Phil] Need to figure out how the data will be provided to the scientist.
+* [All] Web based interface would be valuable.
+; Notes: Targeting publication for July/August issue of SRL. Have some time to develop the webpage.
-Notes: Targeting publication for July/August issue of SRL. Have some time to develop the webpage.
+* Expect Press material surrounding the SRL release, so need to be prepared with digestible figures.

@@ Line 301: / Line 301: @@
 ; Milestones for UCERF3 Testing Program (U3-TP)
 #	Goals:
-**	Verify
+#*	Verify
-**	Validate
+#*	Validate
-**	Valuate
+#*	Valuate
 .	Milestones
 "	Develop infrastructure

@@ Line 297: / Line 297: @@
 * Might want to update forecast when event happens and when event doesn't happen.
-Milestones for UCERF3 Testing Program (U3-TP)
+== Day 3 ==
-.	Goals:
-o	Verify
+; Milestones for UCERF3 Testing Program (U3-TP)
-o	Validate
+#	Goals:
-o	Valuate
+**	Verify
 .	Milestones
 "	Develop infrastructure

@@ Line 249: / Line 249: @@
 *** Critical for real-time forecast and predictions that are generated externally to CSEP platform.
-Testing Fault Based Models
+; Testing Fault Based Models
-"	Association problem: mapping an eq to the ucerf3 fault model
+* Association problem: mapping an eq to the ucerf3 fault model
-"	Need to understand the stopping probabilities associated with stopping between fault segments
+* Need to understand the stopping probabilities associated with stopping between fault segments
-"	Proposed Procedure:
+* Proposed Procedure:
-.	Separate linear fault into sections
+:# Separate linear fault into sections
-.	For each section: estimate nucleation rate for eqs of interest
+:# For each section: estimate nucleation rate for eqs of interest
-.	Estimate conditional probabilities of earthquake stopping
+:# Estimate conditional probabilities of earthquake stopping
-.	Evaluate frequency of eqs for each pair of section
+:# Evaluate frequency of eqs for each pair of section
-"	Could rely on aftershocks to determine the extent of the rupture plane or maybe a finite-fault inversion
+* Could rely on aftershocks to determine the extent of the rupture plane or maybe a finite-fault inversion
-"	Fault participation is the most important this to test for fault-based models.
+* Fault participation is the most important this to test for fault-based models.
-"	Null-hypothesis can be established using the following assumptions
+* Null-hypothesis can be established using the following assumptions
-o	Known magnitude distribution
+** Known magnitude distribution
-o	Known scaling between mw and length
+** Known scaling between mw and length
-o	Uniform distribution of rupture locations on fault
+** Uniform distribution of rupture locations on fault
-Considering Epistemic Uncertainty
+; Considering Epistemic Uncertainty
-"	Aleatory variability: inherent complexity or randomness in some physical process
+* Aleatory variability: inherent complexity or randomness in some physical process
-"	Epistemic uncertainty: comes from our lack of knowledge about the process
+* Epistemic uncertainty: comes from our lack of knowledge about the process
-"	An exchangeable event allow testing of Bayesian models in frequentist framework
+* An exchangeable event allow testing of Bayesian models in frequentist framework
-"	Modifying experimental concept allows for ontological testing of exchangeable sequences
+* Modifying experimental concept allows for ontological testing of exchangeable sequences
-"	Hierarchy of uncert. Necessary for testing
+* Hierarchy of uncert. Necessary for testing
-o	Aleatory variability -> frequentist
+** Aleatory variability -> frequentist
-o	Epistemic uncertainty -> Bayesian methods
+** Epistemic uncertainty -> Bayesian methods
-o	Ontological error -> rejection of 'ontological' null hypothesis
+** Ontological error -> rejection of 'ontological' null hypothesis
-"	States that the true hazard is a realization of the extended experts distribution (EED)
+*** States that the true hazard is a realization of the extended experts distribution (EED)
-"	Rejection of this null hypothesis implies ontological error
+*** Rejection of this null hypothesis implies ontological error
-o	Ontological tests requires 'experimental concept' that conditions the aleatory variability of the natural system.
+** Ontological tests requires 'experimental concept' that conditions the aleatory variability of the natural system.
-Ensemble Modeling and Hybrid model
+; Ensemble Modeling and Hybrid model
-"	Definition: inferring the extended expert's distribution from the sample provided by any set of models that sample the epistemic uncertainty
+* Definition: inferring the extended expert's distribution from the sample provided by any set of models that sample the epistemic uncertainty
-"	Combining models allows the ensemble to perform only slightly worse than the best performing model. Useful when not sure what is 'correct' model.
+* Combining models allows the ensemble to perform only slightly worse than the best performing model. Useful when not sure what is 'correct' model.
-"	Two main features:
+* Two main features:
-o	Describe epistemic uncertainty
+** Describe epistemic uncertainty
-o	Significantly increases the skill of forecast
+** Significantly increases the skill of forecast
-"	Hybrid models to increase information gain
+* Hybrid models to increase information gain
-o	Additive hybrid
+** Additive hybrid
-"	Best fitting linear combination of models
+*** Best fitting linear combination of models
-o	Maximum hybrid
+** Maximum hybrid
-o	Multiplicative hybrids
+** Multiplicative hybrids
-"	Exploit independent information ie., GPS and smoothed seismicity
+*** Exploit independent information ie., GPS and smoothed seismicity
-"	Form hybrids for better information gain
+* Form hybrids for better information gain
-"	Does not require a choice of best model but leverages all models
+* Does not require a choice of best model but leverages all models
-"	Could be a target for CSEP to help gain hybrid models -> improve collaborations
+* Could be a target for CSEP to help gain hybrid models -> improve collaborations
-Event based testing
+; Event based testing
-"	Could move to solution where you can make updates to forecast during the forecast period. Likely more important for long-term models
+* Could move to solution where you can make updates to forecast during the forecast period. Likely more important for long-term models
-"	Might want to update forecast when event happens and when event doesn't happen.
+* Might want to update forecast when event happens and when event doesn't happen.
 Milestones for UCERF3 Testing Program (U3-TP)

@@ Line 1: / Line 1: @@
 ==  CSEP2 Challenges ==
 ; Testing fault and simulation-based models
-; Care about low prob. events
+;* Testing fault and simulation-based models
-; Should we be testing something other nucleation?
+;* Care about low prob. events
-; Is UCERF3-ETAS more valuable given the alternatives?
+;* Should we be testing something other nucleation?
-; Epistemic Uncertainties
+;* Is UCERF3-ETAS more valuable given the alternatives?
 == Day 1 ==
@@ Line 109: / Line 110: @@
 * CSEP2 needs to be able to compare synthetic catalogs coming from RSQSim, ie., the ability to handle non-standard catalog sources.
-Objectives & Challenges of Model Testing
+; Objectives & Challenges of Model Testing
-"	Not enough data
+* Not enough data
-"	Expand data sources in space and time
+* Expand data sources in space and time
-o	i.e., incorp. South America
+** i.e., incorp. South America
-o	retrospective testing experiments
+** retrospective testing experiments
-o	time-dependence
+** time-dependence
-o	extend authoritative data sets
+** extend authoritative data sets
-"	issue 2: primitive models based on point-based models (ie, hypocentral/nucleation)
+* issue 2: primitive models based on point-based models (ie, hypocentral/nucleation)
-o	need: simulation- , fault-, and physics-based models
+** need: simulation- , fault-, and physics-based models
-o	3d models
+** 3d models
-o	Need to account for non-poissonian & correlation structures
+** Need to account for non-poissonian & correlation structures
-o	DJ: expand ucerf3 approach to new locations to understand principal components
+** [DJ] expand ucerf3 approach to new locations to understand principal components
-"	Issue 3: need to build complete prob. models accounting for ontological error (unknown unknowns)
+* Issue 3: need to build complete prob. models accounting for ontological error (unknown unknowns)
-o	How to go from logic tree to continuous pdf?
+** How to go from logic tree to continuous pdf?
-o	Need to consider correlations within tree
+** Need to consider correlations within tree
-"	Issue 4: testing fault-based models
+* Issue 4: testing fault-based models
-o	Lots of work to be done
+** Lots of work to be done
-o	'Turing' style testing can help… Page (2018)
+** 'Turing' style testing can help… Page (2018)
-Turing Tests of UCERF3
+; Turing Tests of UCERF3
-"	properly accounting for spatial diffusivity
+* properly accounting for spatial diffusivity
-"	Inter-sequence aftershock productivity
+* Inter-sequence aftershock productivity
-"	Foreshock and aftershock productivity as function of differential magnitude
+* Foreshock and aftershock productivity as function of differential magnitude
-"	Nearest neighbor separations
+* Nearest neighbor separations
-"	Analysis of clusters
+* Analysis of clusters
-"	Paleo hiatus
+* Paleo hiatus
-"	NF: what are the possible explanations of hiatus?
+* [NF] what are the possible explanations of hiatus?
-"	Super cycle: extreme clustering over extreme period
+* Super cycle: extreme clustering over extreme period
-"	CSEP testing should be more visual and include into CSEP2
+* CSEP testing should be more visual and include into CSEP2
-Comparing R&J with ETAS
+; Comparing R&J with ETAS
-"	R&J -> ETAS
+* R&J -> ETAS
-o	Secondard sequences
+** Secondary sequences
-o	Faster adaptation
+** Faster adaptation
-o	Spatial forecasts
+** Spatial forecasts
-o	Better estimates of the range of outcomes
+** Better estimates of the range of outcomes
-"	CSEP 1day forecasts begin at start of day; lose some power
+* CSEP 1day forecasts begin at start of day; lose some power
-"	Challenges for USGS testing:
+* Challenges for USGS testing:
-o	Overlapping windows
+** Overlapping windows
-o	Update forecasts within window
+** Update forecasts within window
-o	New RJ89 method no longer Poissonian
+** New RJ89 method no longer Poissonian
-o	ETAS forecasts are not Poissonian
+** ETAS forecasts are not Poissonian
-o	All violate CSEP testing methods
+** All violate CSEP testing methods
-"	Likelihood based on Poisson distribution using standard statistical test
+* Likelihood based on Poisson distribution using standard statistical test
-"	CSEP strategy:
+* CSEP strategy:
-o	Poisson numbers based on RJ forecasts
+** Poisson numbers based on RJ forecasts
-"	Strategy:
+* Strategy:
-.	Want dist. Of events in window that starts at t with duration d
+:# Want dist. Of events in window that starts at t with duration d
-.	Using R&J and ETAS to simulate "real" observations
+:# Using R&J and ETAS to simulate "real" observations
-.	Fit R&J to ETAS model: fit is the mode of the individual ETAS runs
+:# Fit R&J to ETAS model: fit is the mode of the individual ETAS runs
 "	Using overlapping windows CSEP assumptions on independence fail; solved by incorporating R&J simulations, forecast, and observations
 "	ETAS observations fails bc they are not Poisson distributed

@@ Line 93: / Line 93: @@
 **** Aleatory variability and sequence specific etas parameters
-RSQSim Rate-State earthquake Simulator
+; RSQSim Rate-State earthquake Simulator
-"	Physics-based forecasting model based on R&S statistics
+* Physics-based forecasting model based on R&S statistics
-"	Using RSQsim ruptures in hazard assessments
+* Using RSQsim ruptures in hazard assessments
-o	Need to create ucerf3 style ruptures
+** Need to create ucerf3 style ruptures
-o	Do RSQSim ruptures pass ucerf3 plausibility criteria?
+** Do RSQSim ruptures pass ucerf3 plausibility criteria?
-"	Surprisingly, most RSQsim ruptures didn't pass the coulomb criterion. ~17.5% did not pass.
+*** Surprisingly, most RSQsim ruptures didn't pass the coulomb criterion. ~17.5% did not pass.
-"	Multi-fault ruptures tend to agree between UCERF3 and RSQsim
+*** Multi-fault ruptures tend to agree between UCERF3 and RSQsim
-"	RSQsim agrees well with UCERF3 without specific tuning of recurrence intervals
+* RSQsim agrees well with UCERF3 without specific tuning of recurrence intervals
-o	Also: repeat times and short period ground motions
+** Also: repeat times and short period ground motions
-o	but starts to disagree at longer spectral periods
+** but starts to disagree at longer spectral periods
-"	interesting conditional probs:
+* interesting conditional probs:
-o	what is the prob of having 2 mw 7 on the Mojave within 1 week?
+** what is the prob of having 2 mw 7 on the Mojave within 1 week?
-"	4.5% in UCERF3 and 5.6% in RSQSim
+*** 4.5% in UCERF3 and 5.6% in RSQSim
-"	Could look at two-point statistics pairwise difference between centroids
+* Could look at two-point statistics pairwise difference between centroids
-"	CSEP2 needs to be able to compare synthetic catalogs coming from RSQSim, ie., the ability to handle non-standard catalog sources.
+* CSEP2 needs to be able to compare synthetic catalogs coming from RSQSim, ie., the ability to handle non-standard catalog sources.
 Objectives & Challenges of Model Testing

@@ Line 5: / Line 5: @@
 * Is UCERF3-ETAS more valuable given the alternatives?
 * Epistemic Uncertainties
 ; Reasenberg & Jones for USGS OAF:
@@ Line 55: / Line 57: @@
 * Testing strategy:
 ** Given a swarm; how long should we provide forecasts?
 ; UCERF3