Difference between revisions of "CVM Release Process"
| (18 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| + | [[Image:CVM-T-Release-Process-Diagram.png|center]] | ||
| + | |||
== Finalizing the Release == | == Finalizing the Release == | ||
| + | |||
| + | === Target Dates === | ||
| + | |||
| + | New versions of a CVM will be issued on a quarterly basis, beginning Dec 1, 2010 (10.12.0). This is a nominal schedule in that there will be no new releases if no changes have been made, and releases may be delayed to fix problems or perform additional testing. | ||
| + | |||
=== Testing === | === Testing === | ||
| − | === | + | The CVM release candidate will be submitted to the Automated Test Framework where it will be evaluated in a number of unit and acceptance tests. At this point the software will be feature complete; no additional features or enhancements are expected to be developed. The software distribution will be assigned a release tag of format YY.MM.RR_RC, where: |
| + | |||
| + | * YY.MM.RR: Prospective release identifier according to above format | ||
| + | * RC : Release candidate flag | ||
| + | |||
| + | |||
| + | The primary acceptance test involves a goodness of fit (GoF) evaluation for the 2008 Chino Hills earthquake. The release candidate passes the testing when the following criteria are met: | ||
| + | |||
| + | * CVM and cvm2mesh are successfully built | ||
| + | * All CVM unit tests pass | ||
| + | * cvm2mesh is successfully able to extract a Chino Hills 100m mesh | ||
| + | * The SCEC Broadband GoF bias for all periods in the range 1s-10s is less than or equal to the GoF bias computed in the previous release. In other words, the goodness of fit of Chino Hills synthetics versus observed is expected to improve from release to release. | ||
| + | |||
| + | |||
| + | If tests fail or problems are found, they will be corrected and the tests re-run until all tests pass. | ||
| + | |||
| + | |||
| + | === Scientific/Technical Review === | ||
| + | |||
| + | Scientific review of ATF simulation results and horizontal/profile plots for correctness, along with technical review of code and software usage. A period of one week following a code freeze is allocated to this review activity. The review team makes a recommendation of either to proceed with the release, or hold in order to perform more tests or changes. At the end of the review, a full code freeze is enacted for any additional changes (other than release ID assignment). | ||
| − | |||
=== Release Authority === | === Release Authority === | ||
| + | |||
| + | The release authority examines the test results and the recommendation of the scientific and technical reviewers and makes the decision whether or not to proceed with the new release. A one week period is allocated for this activity. | ||
| + | |||
=== Assign Release Tag === | === Assign Release Tag === | ||
| − | |||
| − | where: | + | An official release tag is assigned to the release candidate. Official release tags will have the format YY.MM.RR, where: |
* YY: Two digit year | * YY: Two digit year | ||
* MM: Two digit month | * MM: Two digit month | ||
| − | * RR: Mid-month revision id, starting at | + | * RR: Mid-month revision id, starting at 0 |
| + | |||
| + | |||
| + | The appropriate file(s) in version control are updated with this tag. The software is then recompiled and re-tested with unit tests to ensure that the tag update was successful. | ||
| + | |||
=== Package Products === | === Package Products === | ||
| − | + | The following packages will be posted for download at hypocenter: | |
| + | |||
| + | *cvm<name>-YY.MM.RR.tar.gz: Installation package containing CVM model and query software | ||
| + | *cvm<name>-YY.MM.RR.tar.gz.md5: md5 checksum | ||
| − | |||
=== User Community Notification === | === User Community Notification === | ||
| − | + | Communication with the user community is accomplished with an email list (CVM-T-ANNOUNCE@usc.edu) with public subscription. Once the product packages have been publicly posted, a notification is sent to this email lists with release notes and information where to get the new version. | |
== Software Defects == | == Software Defects == | ||
| + | |||
| + | === Defect Tracking === | ||
| + | |||
| + | Trac systems will be established for svn repositories cvmh, cvm2mesh, cvmtest, and viz-cvm. Users, scientists, and developers are all encouraged to open new Trac tickets to report bugs, or request new features. | ||
| + | |||
=== Defects in a Release Candidate === | === Defects in a Release Candidate === | ||
| + | |||
| + | When a defect is found in a release candidate, it will be tracked with a Trac trouble ticket. The developers (and possibly scientific reviewers) will need to determine the severity of the defect and if it is necessary to delay the upcoming release. Early in the release process there is still time to fix minor bugs and retest the system. Serious defects in either the s/w or the underlying model may require significant time to fix and can cause a release to be postponed. | ||
| + | |||
=== Defects in an Existing Release === | === Defects in an Existing Release === | ||
| + | |||
| + | When a defect is found in the most recent existing release, it will be tracked with a Trac trouble ticket. If a defect is found in an older release, but it is not present in the most recent release, the user will be advised to upgrade to the latest version. | ||
| + | |||
| + | A notification will also be sent to the email subscription list announcing that a defect has been found and assigned a Trac ticket number. The developers (and possibly scientific reviewers) will need to determine if a bug fix release is warranted to fix the defect. If the problem is deemed minor, its fix may be rolled into the next quarterly release. If the fix is deemed a priority or directly impacts s/w usability, a bug fix release will be issued that directly addresses the defect. This bug fix release will follow the standard release process except there will be no time requirements for each step of the process in order to get the fix into the hands of users as quickly as possible. | ||
| + | |||
| + | |||
| + | == Scientific/Technical Review Team == | ||
| + | |||
| + | {| class="wikitable" | border="1" | ||
| + | |- | ||
| + | ! Name | ||
| + | ! Organization | ||
| + | |- | ||
| + | | John Shaw | ||
| + | | Harvard University | ||
| + | |- | ||
| + | | Andreas Plesch | ||
| + | | Harvard University | ||
| + | |- | ||
| + | | Geoff Ely | ||
| + | | USC | ||
| + | |- | ||
| + | | Kim Olsen | ||
| + | | UCSD | ||
| + | |- | ||
| + | | Brad Aagaard | ||
| + | | USGS | ||
| + | |- | ||
| + | | Scott Callaghan | ||
| + | | USC | ||
| + | |- | ||
| + | | Kevin Milner | ||
| + | | USC | ||
| + | |- | ||
| + | | Patrick Small | ||
| + | | USC | ||
| + | |} | ||
| + | |||
| + | |||
| + | == Release Authority == | ||
| + | |||
| + | {| class="wikitable" | border="1" | ||
| + | |- | ||
| + | ! Name | ||
| + | ! Organization | ||
| + | |- | ||
| + | | Phil Maechling | ||
| + | | USC | ||
| + | |} | ||
| + | |||
| + | |||
| + | == End User License == | ||
| + | |||
| + | TBD. End user license agreement to be inserted into LICENSE.TXT in root directory of software distribution. | ||
Latest revision as of 17:08, 2 September 2011
Contents
Finalizing the Release
Target Dates
New versions of a CVM will be issued on a quarterly basis, beginning Dec 1, 2010 (10.12.0). This is a nominal schedule in that there will be no new releases if no changes have been made, and releases may be delayed to fix problems or perform additional testing.
Testing
The CVM release candidate will be submitted to the Automated Test Framework where it will be evaluated in a number of unit and acceptance tests. At this point the software will be feature complete; no additional features or enhancements are expected to be developed. The software distribution will be assigned a release tag of format YY.MM.RR_RC, where:
- YY.MM.RR: Prospective release identifier according to above format
- RC : Release candidate flag
The primary acceptance test involves a goodness of fit (GoF) evaluation for the 2008 Chino Hills earthquake. The release candidate passes the testing when the following criteria are met:
- CVM and cvm2mesh are successfully built
- All CVM unit tests pass
- cvm2mesh is successfully able to extract a Chino Hills 100m mesh
- The SCEC Broadband GoF bias for all periods in the range 1s-10s is less than or equal to the GoF bias computed in the previous release. In other words, the goodness of fit of Chino Hills synthetics versus observed is expected to improve from release to release.
If tests fail or problems are found, they will be corrected and the tests re-run until all tests pass.
Scientific/Technical Review
Scientific review of ATF simulation results and horizontal/profile plots for correctness, along with technical review of code and software usage. A period of one week following a code freeze is allocated to this review activity. The review team makes a recommendation of either to proceed with the release, or hold in order to perform more tests or changes. At the end of the review, a full code freeze is enacted for any additional changes (other than release ID assignment).
Release Authority
The release authority examines the test results and the recommendation of the scientific and technical reviewers and makes the decision whether or not to proceed with the new release. A one week period is allocated for this activity.
Assign Release Tag
An official release tag is assigned to the release candidate. Official release tags will have the format YY.MM.RR, where:
- YY: Two digit year
- MM: Two digit month
- RR: Mid-month revision id, starting at 0
The appropriate file(s) in version control are updated with this tag. The software is then recompiled and re-tested with unit tests to ensure that the tag update was successful.
Package Products
The following packages will be posted for download at hypocenter:
- cvm<name>-YY.MM.RR.tar.gz: Installation package containing CVM model and query software
- cvm<name>-YY.MM.RR.tar.gz.md5: md5 checksum
User Community Notification
Communication with the user community is accomplished with an email list (CVM-T-ANNOUNCE@usc.edu) with public subscription. Once the product packages have been publicly posted, a notification is sent to this email lists with release notes and information where to get the new version.
Software Defects
Defect Tracking
Trac systems will be established for svn repositories cvmh, cvm2mesh, cvmtest, and viz-cvm. Users, scientists, and developers are all encouraged to open new Trac tickets to report bugs, or request new features.
Defects in a Release Candidate
When a defect is found in a release candidate, it will be tracked with a Trac trouble ticket. The developers (and possibly scientific reviewers) will need to determine the severity of the defect and if it is necessary to delay the upcoming release. Early in the release process there is still time to fix minor bugs and retest the system. Serious defects in either the s/w or the underlying model may require significant time to fix and can cause a release to be postponed.
Defects in an Existing Release
When a defect is found in the most recent existing release, it will be tracked with a Trac trouble ticket. If a defect is found in an older release, but it is not present in the most recent release, the user will be advised to upgrade to the latest version.
A notification will also be sent to the email subscription list announcing that a defect has been found and assigned a Trac ticket number. The developers (and possibly scientific reviewers) will need to determine if a bug fix release is warranted to fix the defect. If the problem is deemed minor, its fix may be rolled into the next quarterly release. If the fix is deemed a priority or directly impacts s/w usability, a bug fix release will be issued that directly addresses the defect. This bug fix release will follow the standard release process except there will be no time requirements for each step of the process in order to get the fix into the hands of users as quickly as possible.
Scientific/Technical Review Team
| Name | Organization |
|---|---|
| John Shaw | Harvard University |
| Andreas Plesch | Harvard University |
| Geoff Ely | USC |
| Kim Olsen | UCSD |
| Brad Aagaard | USGS |
| Scott Callaghan | USC |
| Kevin Milner | USC |
| Patrick Small | USC |
Release Authority
| Name | Organization |
|---|---|
| Phil Maechling | USC |
End User License
TBD. End user license agreement to be inserted into LICENSE.TXT in root directory of software distribution.
