Difference between revisions of "GetFile"
(Create GetFile entry) |
(Rewrite GetFile entry in terms of OpenSHA problem) |
||
| Line 1: | Line 1: | ||
| − | == | + | == OpenSHA Problem == |
| − | + | Fault section data and fault system rupture sets are loaded by OpenSHA to compute earthquake rupture forecasts. The geospatial data for several models is stored directly with the OpenSHA code that operates on it. Models for seismic hazard analysis under the OpenSHA framework are becoming progressively larger. Unfortunately, there are file size constraints of 100MB on GitHub, which can't fit the new 2023 US National Seismic Hazard Model (NSHM23). | |
| − | |||
| − | |||
| − | |||
| − | == | + | == Current Solution == |
| − | + | Smaller models can continue to be hosted on GitHub with the OpenSHA code, but UCERF3 has been moved to a server on USC campus. Currently, OpenSHA downloads the model from the ASB "cheesegrater" server. This solution is not scalable and has the potential to partially download or otherwise download a corrupted version of the UCERF3 model. These older servers are going to be decommissioned soon and we need to transition to a better long-term solution. | |
| − | |||
| − | |||
| − | == | + | == Proposed Solution == |
| − | OpenSHA | + | GetFile is a more robust solution to hosting hazard models for use in OpenSHA. It will be used to download and validate the UCERF3 and NSHM23 models. It may see use in other models and several projects across SCEC that need to download and validate files, such as UCVM. Scientific models can be stored on USC CARC and downloaded via the GetFile framework. GetFile provides a sophisticated feature-set for data validation, rolling back to older model snapshots, and enabling automatic updates of the GetFile framework for seamless deployment of new features and bug fixes. |
| − | |||
== Docs and Code == | == Docs and Code == | ||
The source code and detailed usage and setup documentation: https://github.com/abhatthal/getfile | The source code and detailed usage and setup documentation: https://github.com/abhatthal/getfile | ||
| − | Demo applications using the GetFile | + | Demo applications using the GetFile library: https://github.com/abhatthal/getfile-demo |
Latest revision as of 21:51, 21 November 2024
OpenSHA Problem
Fault section data and fault system rupture sets are loaded by OpenSHA to compute earthquake rupture forecasts. The geospatial data for several models is stored directly with the OpenSHA code that operates on it. Models for seismic hazard analysis under the OpenSHA framework are becoming progressively larger. Unfortunately, there are file size constraints of 100MB on GitHub, which can't fit the new 2023 US National Seismic Hazard Model (NSHM23).
Current Solution
Smaller models can continue to be hosted on GitHub with the OpenSHA code, but UCERF3 has been moved to a server on USC campus. Currently, OpenSHA downloads the model from the ASB "cheesegrater" server. This solution is not scalable and has the potential to partially download or otherwise download a corrupted version of the UCERF3 model. These older servers are going to be decommissioned soon and we need to transition to a better long-term solution.
Proposed Solution
GetFile is a more robust solution to hosting hazard models for use in OpenSHA. It will be used to download and validate the UCERF3 and NSHM23 models. It may see use in other models and several projects across SCEC that need to download and validate files, such as UCVM. Scientific models can be stored on USC CARC and downloaded via the GetFile framework. GetFile provides a sophisticated feature-set for data validation, rolling back to older model snapshots, and enabling automatic updates of the GetFile framework for seamless deployment of new features and bug fixes.
Docs and Code
The source code and detailed usage and setup documentation: https://github.com/abhatthal/getfile
Demo applications using the GetFile library: https://github.com/abhatthal/getfile-demo
