CSEP1 Status - Revision history

Wsavran: /* Results */

2018-10-26T22:33:16Z

Results

Wsavran: /* Introduction */

2018-10-26T22:32:41Z

Introduction

Wsavran: /* Introduction */

2018-07-26T18:29:48Z

Introduction

Wsavran: /* Introduction */

2018-07-03T21:30:43Z

Introduction

Wsavran: /* Evaluations */

2018-06-26T23:15:32Z

Evaluations

Wsavran: /* Evaluations */

2018-06-26T21:49:13Z

Evaluations

Wsavran: /* Evaluations */

2018-06-26T21:32:13Z

Evaluations

Wsavran: /* Evaluations */

2018-06-26T21:11:16Z

Evaluations

Wsavran: /* Evaluations */

2018-06-26T02:38:30Z

Evaluations

Wsavran: /* Evaluations */

2018-06-26T02:37:28Z

Evaluations

← Older revision		Revision as of 22:33, 26 October 2018
Line 6:		Line 6:

	== Results ==		== Results ==
		+
		+	The link below shows the results for the CSEP1 SCEC testing center.

	http://hypocenter.usc.edu/research/csep1/CSEP1.0_Status_v0.9.html		http://hypocenter.usc.edu/research/csep1/CSEP1.0_Status_v0.9.html

@@ Line 4: / Line 4: @@
 == Introduction ==
 The CSEP1 system has been in production for over a decade with dozens of models running prospectively during this time period. Currently, we do not have an understanding of the status of the operational system with respect to missing forecasts and their evaluations. This page provides the status of the CSEP1 system and outlines our work toward understanding the status of the CSEP1 operational system with respect to one-day rate-based forecasting models.
 http://hypocenter.usc.edu/research/csep1/CSEP1.0_Status_v0.9.html

@@ Line 4: / Line 4: @@
 == Introduction ==
 The CSEP1 system has been in production for over a decade with dozens of models running prospectively during this time period. Currently, we do not have an understanding of the status of the operational system with respect to missing forecasts and their evaluations. This page provides the status of the CSEP1 system and outlines our work toward understanding the status of the CSEP1 operational system with respect to one-day rate-based forecasting models.
 == Data Model ==

@@ Line 3: / Line 3: @@
 == Introduction ==
-The CSEP1 system has been in production for over a decade with dozens of models running prospectively during this time period. Currently, we do not have an understanding of the status of the operational system with respect to missing forecasts and their evaluations. This page provides the status of the CSEP1 system and outlines our work toward understanding the status of the CSEP1 operational system.
+The CSEP1 system has been in production for over a decade with dozens of models running prospectively during this time period. Currently, we do not have an understanding of the status of the operational system with respect to missing forecasts and their evaluations. This page provides the status of the CSEP1 system and outlines our work toward understanding the status of the CSEP1 operational system with respect to one-day rate-based forecasting models.
 == Data Model ==

← Older revision		Revision as of 23:15, 26 June 2018
Line 767:		Line 767:
	./2012-12-13/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_12_13_2012-fromXML.xml.1407265079.436862.2		./2012-12-13/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_12_13_2012-fromXML.xml.1407265079.436862.2
	./2012-12-18/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_12_18_2012-fromXML.xml.1407284266.034296.2		./2012-12-18/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_12_18_2012-fromXML.xml.1407284266.034296.2
		+	</pre>
		+
		+	Since the evaluations might be written in different directories, here are the results from the ''find'' command on the operational system.
		+
		+	For group ''one-day-models-V10.10''
		+	<pre>
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results/2012-09-25/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_9_25_2012-fromXML.xml.1407034485.064768.3
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results/2012-09-26/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_9_26_2012-fromXML.xml.1407044193.731013.3
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results/2012-09-27/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_9_27_2012-fromXML.xml.1407053484.363481.3
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results/2012-09-28/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_9_28_2012-fromXML.xml.1407066029.124641.3
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results/2012-09-29/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_9_29_2012-fromXML.xml.1407094241.989658.3
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results/2012-10-18/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_10_18_2012-fromXML.xml.1407103322.669393.3
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results/2012-11-04/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_4_2012-fromXML.xml.1407122141.539472.3
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results/2012-11-15/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_15_2012-fromXML.xml.1407144499.858849.3
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results/2012-11-22/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_22_2012-fromXML.xml.1407182120.590843.3
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results/2012-11-24/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_24_2012-fromXML.xml.1407200704.854854.3
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results/2012-11-26/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_26_2012-fromXML.xml.1407219350.492590.3
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results/2012-12-13/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_12_13_2012-fromXML.xml.1407267242.612174.3
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results/2012-12-18/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_12_18_2012-fromXML.xml.1407286428.449104.3
		+	</pre>
		+
		+	For group ''one-day-models-V12.7''
		+	<pre>
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results/2012-09-25/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_9_25_2012-fromXML.xml.1407037982.879646.4
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results/2012-09-26/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_9_26_2012-fromXML.xml.1407046972.990417.4
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results/2012-09-27/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_9_27_2012-fromXML.xml.1407056315.647794.4
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results/2012-09-28/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_9_28_2012-fromXML.xml.1407068908.781897.4
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results/2012-09-29/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_9_29_2012-fromXML.xml.1407097022.806162.4
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results/2012-10-18/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_10_18_2012-fromXML.xml.1407106120.147435.4
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results/2012-11-04/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_4_2012-fromXML.xml.1407124959.788090.4
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results/2012-11-15/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_15_2012-fromXML.xml.1407147236.313998.4
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results/2012-11-22/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_22_2012-fromXML.xml.1407184916.610078.4
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results/2012-11-24/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_24_2012-fromXML.xml.1407203478.408629.4
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results/2012-11-26/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_26_2012-fromXML.xml.1407222190.625967.4
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results/2012-12-13/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_12_13_2012-fromXML.xml.1407270131.911237.4
		+	/home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results/2012-12-18/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_12_18_2012-fromXML.xml.1407289342.475311.4
		+	</pre>

← Older revision		Revision as of 21:49, 26 June 2018
Line 720:		Line 720:
	ls: cannot access /home/csep/operations/SCEC-natural-laboratory/one-day-models-V16.10/results/2015-02-23: No such file or directory		ls: cannot access /home/csep/operations/SCEC-natural-laboratory/one-day-models-V16.10/results/2015-02-23: No such file or directory
	</pre>		</pre>
		+
		+	Next, we verify the first 20 "Completed" simulations for an evaluation where the majority are missing.
		+	<pre>
		+	sqlite> select Forecasts.name, ScheduledEvaluations.date_time from Evaluations join Forecasts on Forecasts.forecast_id=Evaluations.forecast_id join ScheduledEvaluations on Evaluations.schedule_id=ScheduledEvaluations.scheduled_evaluation_id where Evaluations.status="Complete" and Forecasts.name="ETAS" and Evaluations.name="LW" limit 20;
		+	ETAS\|2009-11-29 00:00:00
		+	ETAS\|2009-11-30 00:00:00
		+	ETAS\|2009-12-01 00:00:00
		+	ETAS\|2009-12-02 00:00:00
		+	ETAS\|2010-08-23 00:00:00
		+	ETAS\|2010-08-25 00:00:00
		+	ETAS\|2010-08-26 00:00:00
		+	ETAS\|2012-09-25 00:00:00
		+	ETAS\|2012-09-26 00:00:00
		+	ETAS\|2012-09-27 00:00:00
		+	ETAS\|2012-09-28 00:00:00
		+	ETAS\|2012-09-29 00:00:00
		+	ETAS\|2012-10-18 00:00:00
		+	ETAS\|2012-11-04 00:00:00
		+	ETAS\|2012-11-15 00:00:00
		+	ETAS\|2012-11-22 00:00:00
		+	ETAS\|2012-11-24 00:00:00
		+	ETAS\|2012-11-26 00:00:00
		+	ETAS\|2012-12-13 00:00:00
		+	ETAS\|2012-12-18 00:00:00
		+	</pre>
		+	Here are the results from running ''find . -name "LW-Test_ETASxml*" -print > ~/wsavran/LW-Test_ETAS-one-day-models-V9.1.txt ; sort ~/wsavran/LW-Test_ETAS-one-day-models-V9.1.txt > ~/wsavran/LW-Test_ETAS-one-day-models-V9.1-sorted.txt''
		+	<pre>
		+	./2009-11-29/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_29_2009-fromXML.xml.1407008088.841302.2
		+	./2009-11-30/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_30_2009-fromXML.xml.1407011267.496842.2
		+	./2009-12-01/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_12_1_2009-fromXML.xml.1407015914.721951.2
		+	./2009-12-02/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_12_2_2009-fromXML.xml.1407018997.827789.2
		+	./2010-08-23/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_8_23_2010-fromXML.xml.1407022122.738802.2
		+	./2010-08-25/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_8_25_2010-fromXML.xml.1407025187.764500.2
		+	./2010-08-26/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_8_26_2010-fromXML.xml.1407028336.569310.2
		+	./2012-09-25/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_9_25_2012-fromXML.xml.1407031969.083919.2
		+	./2012-09-26/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_9_26_2012-fromXML.xml.1407042083.107250.2
		+	./2012-09-27/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_9_27_2012-fromXML.xml.1407051331.716230.2
		+	./2012-09-28/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_9_28_2012-fromXML.xml.1407063925.933241.2
		+	./2012-09-29/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_9_29_2012-fromXML.xml.1407092175.741659.2
		+	./2012-10-18/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_10_18_2012-fromXML.xml.1407101243.736082.2
		+	./2012-11-04/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_4_2012-fromXML.xml.1407120086.484133.2
		+	./2012-11-15/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_15_2012-fromXML.xml.1407142495.333533.2
		+	./2012-11-22/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_22_2012-fromXML.xml.1407180048.513000.2
		+	./2012-11-24/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_24_2012-fromXML.xml.1407198649.649907.2
		+	./2012-11-26/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_11_26_2012-fromXML.xml.1407217302.796640.2
		+	./2012-12-13/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_12_13_2012-fromXML.xml.1407265079.436862.2
		+	./2012-12-18/scec.csep.DiagnosticsTest.dTest_LW-Test_ETAS_12_18_2012-fromXML.xml.1407284266.034296.2

← Older revision		Revision as of 21:32, 26 June 2018
Line 704:		Line 704:
	</pre>		</pre>
	We will manually search these directories and show this model is not present.		We will manually search these directories and show this model is not present.
		+	<pre>
		+	[csep@csep-op results]$ ls /home/csep/operations/SCEC-natural-laboratory/one-day-models/results/2015-02-23
		+	ls: cannot access /home/csep/operations/SCEC-natural-laboratory/one-day-models/results/2015-02-23: No such file or directory
		+
		+	[csep@csep-op results]$ ls /home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results/2015-02-23
		+	ls: cannot access /home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results/2015-02-23: No such file or directory
		+
		+	[csep@csep-op results]$ ls /home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results/2015-02-23
		+	ls: cannot access /home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results/2015-02-23: No such file or directory
		+
		+	[csep@csep-op results]$ ls /home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.10/results/2015-02-23
		+	ls: cannot access /home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.10/results/2015-02-23: No such file or directory
		+
		+	[csep@csep-op results]$ ls /home/csep/operations/SCEC-natural-laboratory/one-day-models-V16.10/results/2015-02-23
		+	ls: cannot access /home/csep/operations/SCEC-natural-laboratory/one-day-models-V16.10/results/2015-02-23: No such file or directory
		+	</pre>

← Older revision		Revision as of 21:11, 26 June 2018
Line 694:		Line 694:
	ETAS\|N\|2018-05-26 00:00:00		ETAS\|N\|2018-05-26 00:00:00
	</pre>		</pre>
		+	Determining which evaluations are missing is challenging because the evaluations could be saved in any forecast group where ETAS is listed. For example the ETAS forecast is listed in the following forecast groups meaning that these evaluations could be present in any of these directories. (However, in this particular case the first missing date could not occur in the V16.10 directory due to timing).
		+	<pre>
		+	one-day-models \| /home/csep/operations/SCEC-natural-laboratory/one-day-models/results
		+	one-day-models-V9.1 \| /home/csep/operations/SCEC-natural-laboratory/one-day-models-V9.1/results
		+	one-day-models-V10.10 \| /home/csep/operations/SCEC-natural-laboratory/one-day-models-V10.10/results
		+	one-day-models-V12.7 \| /home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.7/results
		+	one-day-models-V12.10 \| /home/csep/operations/SCEC-natural-laboratory/one-day-models-V12.10/results
		+	one-day-models-V16.10 \| /home/csep/operations/SCEC-natural-laboratory/one-day-models-V16.10/results
		+	</pre>
		+	We will manually search these directories and show this model is not present.

@@ Line 667: / Line 667: @@
 <pre>
 sqlite> select Forecasts.name, Evaluations.name, ScheduledEvaluations.date_time from Evaluations join ScheduledEvaluations on Evaluations.schedule_id=ScheduledEvaluations.scheduled_evaluation_id join Forecasts on Evaluations.forecast_id=Forecasts.forecast_id where Evaluations.status="Missing" and Evaluations.name="N" and Forecasts.name="ETAS";
 ETAS|N|2015-02-23 00:00:00
 ETAS|N|2015-08-18 00:00:00
 ETAS|N|2015-11-06 00:00:00
 ETAS|N|2016-01-19 00:00:00
 ETAS|N|2016-03-30 00:00:00
 ETAS|N|2016-03-31 00:00:00
@@ Line 704: / Line 692: @@
 ETAS|N|2017-11-19 00:00:00
 ETAS|N|2017-11-20 00:00:00
-ETAS|N|2018-05-14 00:00:00
+ETAS|N|2018-05-26 00:00:00
 </pre>

@@ Line 396: / Line 396: @@
 Next we enumerate the number of missing evaluation for each forecast and evaluation type.
 <pre>
 ETAS|CL|1147
 ETAS|L|113