Difference between revisions of "UCVM Release Planning"

From SCECpedia
Jump to navigationJump to search
 
(24 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Fork and Pull Development ==
+
== Updated UCVM Documentation ==
Using the fork and pull method, start with fork of sceccode/ucvm.git into personal repo. Then, to do development on CARC, plan to clone pjmaechling/ucvm.git for development. Set git upstream to original repo sceccode/ucvm.git to keep in sync with that repo. Follow Instruction for setting upstream here:
+
*[https://github.com/sceccode/ucvm/wiki UCVM Wiki]
*[https://github.com/billtron/pycsep_git_tutorial Savran Git Tutorial]
+
**[UCVM README.md]
 +
*[https://github.com/sceccode/ucvm_docker/wiki UCVM Docker Wiki]
 +
**[UCVM Docker README.md]
 +
*[https://github.com/sceccode/ucvm_plotting/wiki UCVM Plotting Wiki]
 +
**[UCVM Plotting READM.md]
  
== Development on CARC ==
+
== Citable Code ==
* ssh -o "ServerAliveInterval 60" discovery.usc.edu
+
*[https://guides.lib.berkeley.edu/citeyourcode Make UCVM Citable]
* cd /project/scec_608/maechlin/dev
 
* git clone https://github.com/pjmaechling/ucvm.git
 
* adding codemeta.json file.
 
* add Contributors.MD file.
 
* add MODULES.md file to explain how CARC load modules are configured to work
 
* Follow UCVM installation instructions: [https://github.com/SCECcode/UCVMC/wiki/Installation-From-Source-Code Install Wiki]
 
*[https://strike.scec.org/scecpedia/UCVM_Install Install Notes on Discovery]
 
  
== Compiling ==
+
== UCVM Tests Organization ==
* the source directory /project/scec_608/maechlin/dev/ucvm
+
A set of C language tests are defined as Unittests:
* the installation directory /project/scec_608/maechlin/ucvm_bin
+
*[[UCVM Testing]]
  
== UCVM Tests Organized ==
+
UCVM Programs found in the ucvm source code directory contains numerous example programs that are not included in the ucvm_bin installation directory. A list of programs in the UCVM Source directory is given here:
*[[UCVM Testing]]
+
*[[UCVM Examples]]
 +
 
 +
UCVM Plotting routines have been moved into a separate repo. Information on the plotting routines.
 +
*[[UCVM Plotting]]
  
== Stage Large files on public S3 ==
 
* get_large_files.py script in release points to hypocenter.usc.edu/research/ucvmc/V19_4
 
* copy this directory from hypocenter
 
* create v21.10 directory
 
* move large files into that
 
* move v21.10 to ResearchComputing AWS account under maechlin@usc.edu
 
 
== ToDo ==
 
== ToDo ==
* Change the CVM names presented to users during install: Use science names.
+
 
* Move the run_ucvm.sh and run_ucvm_query.sh script to utilities
 
 
* Document the scripts in ucvm/utilities
 
* Document the scripts in ucvm/utilities
 
* Update the version numbers from 21.7 in get_largefiles.py and check_large_files.py
 
* Update the version numbers from 21.7 in get_largefiles.py and check_large_files.py
 
* fix version in setup.list
 
* fix version in setup.list
 
* confirm that models are correctly un-commented in ucvm.conf file (seems like cvm-H and others were not enabled)
 
* confirm that models are correctly un-commented in ucvm.conf file (seems like cvm-H and others were not enabled)
* Check UCVM_INSTALL path that is set correctly
+
* Check references to proj-4 in documentation and code error messages and update if necessary to proj-5
* change any reference to proj-4 in documentation and code error messages
 
* add comments to MODULES.md file to explain what was done to make it work on CARC with their modules system
 
* Review install warnings sent via slack
 
 
* Check ucvm/tests/Makefile.am setting, but never using, AM_CFLAGS
 
* Check ucvm/tests/Makefile.am setting, but never using, AM_CFLAGS
 
* change, or explain to users, two different ucvm.conf in the system
 
* change, or explain to users, two different ucvm.conf in the system
* make largefiles usage consistent in install scripts
 
  
Friday Due Date
 
* SAM Poster
 
* SAM Presentation Video
 
  
== Key UCVM Improvements: ==
+
== Key UCVM Improvements in 21.10 release: ==
#Convergence of versions
+
* Move the run_ucvm.sh and run_ucvm_query.sh script to utilities
#Large files next release stored on S3
+
*Convergence of versions
#CI setup
+
** Used updated version used UCVM website
#Documentation Updated into new structure
+
** Combined source codebase web version and distributed version/command line version
#Tests output
+
** reduce number of UCVM versions that we need to maintain
#Code Metadata included in repo
+
*Large files next release stored on S3
#Tags from USGS Thesarus
+
** Should improve download speed and reliability of install
#Post DOI badge on UCVM
+
*CI setup
#Test with singularity on an XSEDE system
+
** Review the CI for sceccode/ucvm before release
#UCVM Communitee
+
*Documentation Updated into new structure
#Code of Conduct
+
** Re-organized to match recommended types of docs
#Open Source Metrics setup
+
*Tests updated
#Code coverage statements
+
** combined c-language tests into unittest.
#Identification of sub-licenses in distribution
+
* Code Metadata included in repo
#Authorship contributions noted
+
** added metadata to repo
 
+
* Used Tags from USGS Thesarus for metadata
== Standard Contents of Git repo: ==
+
* Post DOI badge on UCVM
#A README with pictures/gifs of the product in action and a nice logo.
+
* UCVM Community developments encouraged
#Documentation.
+
** Add Code of Conduct document into repo
#Code QA (Static Code Analysis).
+
** Added "contributor instructions" into README.md
#Contributing instructions.
+
* Open Source Metrics setup (registered users, downloads, tags, visits)
#A well-defined setup section.
+
* Identification of sub-licenses in distribution
#Support (Respond to Issues/PR)
+
* Velocity models stored in standalone repositories
#Publish software news in every possible way.
+
* Authorship contributions noted
 
+
* add cv
== Contents of README.md ==
+
* add iv
When someone is looking at your project, they want to know:
+
* add albacore
# what is it?
+
* add utah wasatch front
# how good the code is?
+
* projection code fix
# how much support is available?
+
* Install using Dockerhub
# what’s included?
+
* Review issues that were done.
# what does it look like?
+
* Added UCVM Website data plotting tools
# how set it up?
 
 
 
== Science Code Manifesto Elements: ==
 
# Code
 
# Copyright
 
# Citation
 
# Credit
 
# Curation
 
 
 
== Steps To Software Product: ==
 
# Create citable, definitive version of software with doi, license, and repository.
 
# Define reference publication used to cite software.
 
# Define software as reference implementation of a method, and define a set of approved software acceptance/regression tests that can be used to establish a software implements that “method”.
 
# Create software maintenance organization with commit authority for pull requests and approval process for change requests, and process of approving new releases.
 
# Establish software community through registrations, newsletters, activity, regular calls, regular meetings, define community and roles.
 
 
 
== Adoption of Fork and Pull Git Repo Model ==
 
* Use the model used by the majority of open-source projects (including pyCSEP).
 
* The “maintainer” of the shared repo assigns rights to “Collaborators”
 
* Collaborators do not have push access to main (upstream) repo
 
* Core development teams accepts (PRs) from collaborators, reviews them, then merges them into main repo
 
 
 
== Contributor Process: ==
 
Working with shared projects on GitHub
 
# Fork the repository
 
# Clone your forked copy
 
# Sync your personal repo with shared repo
 
# Git merge/git rebase
 
# Make a contribution
 
# Pull request
 
 
 
== How we want it Cited: ==
 
* Example Citation:
 
* Example Acknowledgements:
 
* Example Reference:
 
 
 
== Basic Recommendations: ==
 
# Make source code publicly accessible
 
# Make software easy to discover by providing software metadata via a popular community registry (Examples of community registries of software metadata are bio.tools (Ison et al., 2016), (Ison et al., 2016) biojs.io (Corpas et al.,2014; Gómez et al., 2013) and Omic Tools (Henry et al., 2014) in the life sciences and DataCite (Brase, n.d.) as a generic metadata registry for software as well as data.
 
# Adopt a license and comply with the license of third party dependencies
 
# Define clear and transparent contribution, governance and communications processes (For instance the Galaxy project’s website describes the team’s structure, how to be part of the community, and their communication channels.)
 
 
 
== Types of Documentation with axis: ==
 
* help learning – help working
 
* theoretical knowledge – practical knowledge
 
# tutorials  - learning oriented
 
# how-to guides – task-oriented
 
# Background/Concept explanations – understanding-oriented
 
# technical reference – information-oriented
 
 
 
== DOCUMENTATION TYPES ==
 
* CODE DOCUMENTATION - Semantic identifiers, comments, API, engineering, dependencies, requirements
 
* USER DOCUMENTATION - How to get, run, use the software; parameters, data model, etc.; license
 
* MAINTENANCE DOCUMENTATION - How to build, release, review code, publish
 
* DEVELOPER DOCUMENTATION - How to contribute, contribution templates (issues, pull/merge requests)
 
* METADATA - Software metadata (CodeMeta), Citation File (CFF), "references" (dependencies)
 
* PROJECT DOCUMENTATION - Rationale, teams, governance, community (contact, code of conduct)
 
 
 
== Where Documentation Lives ==
 
Documentation lives where the source code lives! (This is never in an email, chat, or similar!)
 
 
 
Conceptual Documentation:
 
* Requirements
 
* Projects
 
Hands-on documentation
 
* How-tos, getting started
 
* Templates for issues, pull/merge
 
* Contribution guidelines
 
Reference documentation
 
* API
 
* Tests
 
* Metadata
 
 
 
== Toolbox Documentation: ==
 
Toolbox documentation should describe the steps off analysis in a pedagogical, narrative
 
fashion, with example data that users can load to follow along with and understand the documentation.
 
 
 
== Implement Multiple Test Types: ==
 
# Functional tests – Unit Tests essential core ucvm functions
 
# Integration Tests – Test utilities including meshing, layer searches, gtls, and performance
 
# Model Tests - Each velocity model has tests showing expected results for some points
 
# Acceptance tests – Confirm results on users system. Maybe union of funcational, integration, and model tests.
 
  
== Recommended Basic Practices: ==
+
TBD
# Training on Software Practices
+
* Test with both Docker and singularity on an XSEDE system
# Code in a Code Repo
+
* Code coverage statements
# Automated Testing
 
# Persistent ID for software versions
 
  
 
== UCVM Versus CIG Standards: ==
 
== UCVM Versus CIG Standards: ==

Latest revision as of 02:22, 4 November 2021

Updated UCVM Documentation

Citable Code

UCVM Tests Organization

A set of C language tests are defined as Unittests:

UCVM Programs found in the ucvm source code directory contains numerous example programs that are not included in the ucvm_bin installation directory. A list of programs in the UCVM Source directory is given here:

UCVM Plotting routines have been moved into a separate repo. Information on the plotting routines.

ToDo

  • Document the scripts in ucvm/utilities
  • Update the version numbers from 21.7 in get_largefiles.py and check_large_files.py
  • fix version in setup.list
  • confirm that models are correctly un-commented in ucvm.conf file (seems like cvm-H and others were not enabled)
  • Check references to proj-4 in documentation and code error messages and update if necessary to proj-5
  • Check ucvm/tests/Makefile.am setting, but never using, AM_CFLAGS
  • change, or explain to users, two different ucvm.conf in the system


Key UCVM Improvements in 21.10 release:

  • Move the run_ucvm.sh and run_ucvm_query.sh script to utilities
  • Convergence of versions
    • Used updated version used UCVM website
    • Combined source codebase web version and distributed version/command line version
    • reduce number of UCVM versions that we need to maintain
  • Large files next release stored on S3
    • Should improve download speed and reliability of install
  • CI setup
    • Review the CI for sceccode/ucvm before release
  • Documentation Updated into new structure
    • Re-organized to match recommended types of docs
  • Tests updated
    • combined c-language tests into unittest.
  • Code Metadata included in repo
    • added metadata to repo
  • Used Tags from USGS Thesarus for metadata
  • Post DOI badge on UCVM
  • UCVM Community developments encouraged
    • Add Code of Conduct document into repo
    • Added "contributor instructions" into README.md
  • Open Source Metrics setup (registered users, downloads, tags, visits)
  • Identification of sub-licenses in distribution
  • Velocity models stored in standalone repositories
  • Authorship contributions noted
  • add cv
  • add iv
  • add albacore
  • add utah wasatch front
  • projection code fix
  • Install using Dockerhub
  • Review issues that were done.
  • Added UCVM Website data plotting tools

TBD

  • Test with both Docker and singularity on an XSEDE system
  • Code coverage statements

UCVM Versus CIG Standards:

Minimum:

  • Version control – ok
  • Code – ok
  • Portable – ok
  • Testing – (a) tests that verify it runs properly (b) accuracy or benchmark tests
  • Documentation – (a) install (b) parameters (c) physics (d) example inputs cookbooks (e) citable pub
  • Userworkflow – ok

Standard:

  • Version control -ok
  • Coding – (a) params at runtime (b) development plan (c) code comments (d) add features without modify main branch (e) useful error reports
  • Portability: (a) dependency cheking (b) automake (c) output configuration and build options
  • Testing – pass fail tests
  • Documentation: (a) workflow for research (b) how to extend code
  • Userworkflow: (a) easy to change sim params (b) user specific directories/filenames for i/o (c) standard binary formats (d) citation for code version.