Difference between revisions of "UCVM Release Planning"

From SCECpedia
Jump to navigationJump to search
 
(15 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
== Fork and Pull Development ==
+
== Updated UCVM Documentation ==
−
Using the fork and pull method, start with fork of sceccode/ucvm.git into personal repo. Then, to do development on CARC, plan to clone pjmaechling/ucvm.git for development. Set git upstream to original repo sceccode/ucvm.git to keep in sync with that repo. Follow Instruction for setting upstream here:
+
*[https://github.com/sceccode/ucvm/wiki UCVM Wiki]
−
*[https://github.com/billtron/pycsep_git_tutorial Savran Git Tutorial]
+
**[UCVM README.md]
 +
*[https://github.com/sceccode/ucvm_docker/wiki UCVM Docker Wiki]
 +
**[UCVM Docker README.md]
 +
*[https://github.com/sceccode/ucvm_plotting/wiki UCVM Plotting Wiki]
 +
**[UCVM Plotting READM.md]
  
−
== Development on CARC ==
+
== Citable Code ==
−
* ssh -o "ServerAliveInterval 60" discovery.usc.edu
+
*[https://guides.lib.berkeley.edu/citeyourcode Make UCVM Citable]
−
* cd /project/scec_608/maechlin/dev
 
−
* git clone https://github.com/pjmaechling/ucvm.git
 
−
* adding codemeta.json file.
 
−
* add Contributors.MD file.
 
−
* add MODULES.md file to explain how CARC load modules are configured to work
 
−
* Follow UCVM installation instructions: [https://github.com/SCECcode/UCVMC/wiki/Installation-From-Source-Code Install Wiki]
 
−
*[https://strike.scec.org/scecpedia/UCVM_Install Install Notes on Discovery]
 
−
 
 
−
== Compiling ==
 
−
* the source directory /project/scec_608/maechlin/dev/ucvm
 
−
* the installation directory /project/scec_608/maechlin/ucvm_bin
 
  
 
== UCVM Tests Organization ==
 
== UCVM Tests Organization ==
Line 27: Line 20:
 
*[[UCVM Plotting]]
 
*[[UCVM Plotting]]
  
−
== Stage Large files on public S3 ==
+
== ToDo ==
−
* get_large_files.py script in release points to hypocenter.usc.edu/research/ucvmc/V19_4
 
−
* copy this directory from hypocenter
 
−
* create v21.10 directory
 
−
* move large files into that
 
−
* move v21.10 to ResearchComputing AWS account under maechlin@usc.edu
 
  
−
== ToDo ==
 
−
* Change the CVM names presented to users during install: Use science names.
 
−
* Move the run_ucvm.sh and run_ucvm_query.sh script to utilities
 
 
* Document the scripts in ucvm/utilities
 
* Document the scripts in ucvm/utilities
 
* Update the version numbers from 21.7 in get_largefiles.py and check_large_files.py
 
* Update the version numbers from 21.7 in get_largefiles.py and check_large_files.py
 
* fix version in setup.list
 
* fix version in setup.list
 
* confirm that models are correctly un-commented in ucvm.conf file (seems like cvm-H and others were not enabled)
 
* confirm that models are correctly un-commented in ucvm.conf file (seems like cvm-H and others were not enabled)
−
* Check UCVM_INSTALL path that is set correctly
+
* Check references to proj-4 in documentation and code error messages and update if necessary to proj-5
−
* change any reference to proj-4 in documentation and code error messages
 
−
* add comments to MODULES.md file to explain what was done to make it work on CARC with their modules system
 
−
* Review install warnings sent via slack
 
 
* Check ucvm/tests/Makefile.am setting, but never using, AM_CFLAGS
 
* Check ucvm/tests/Makefile.am setting, but never using, AM_CFLAGS
 
* change, or explain to users, two different ucvm.conf in the system
 
* change, or explain to users, two different ucvm.conf in the system
−
* make largefiles usage consistent in install scripts
 
−
* add programs/ucvm/run_ucvm_query_albacore.sh to src/examples programs/ucvm/run_ucvm_query_wfcvm.sh
 
−
* updated kml showing albacore, iv, and cv model regions
 
  
−
Friday Due Date
 
−
* SAM Poster
 
−
* SAM Presentation Video
 
  
−
== Key UCVM Improvements: ==
+
== Key UCVM Improvements in 21.10 release: ==
 +
* Move the run_ucvm.sh and run_ucvm_query.sh script to utilities
 
*Convergence of versions
 
*Convergence of versions
 
** Used updated version used UCVM website
 
** Used updated version used UCVM website
Line 86: Line 63:
 
* Install using Dockerhub
 
* Install using Dockerhub
 
* Review issues that were done.
 
* Review issues that were done.
 +
* Added UCVM Website data plotting tools
  
 
TBD
 
TBD
 
* Test with both Docker and singularity on an XSEDE system
 
* Test with both Docker and singularity on an XSEDE system
 
* Code coverage statements
 
* Code coverage statements
−
 
−
== Standard Contents of Git repo: ==
 
−
#A README with pictures/gifs of the product in action and a nice logo.
 
−
#Documentation.
 
−
#Code QA (Static Code Analysis).
 
−
#Contributing instructions.
 
−
#A well-defined setup section.
 
−
#Support (Respond to Issues/PR)
 
−
#Publish software news in every possible way.
 
−
 
−
== Contents of README.md ==
 
−
When someone is looking at your project, they want to know:
 
−
# what is it?
 
−
# how good the code is?
 
−
# how much support is available?
 
−
# what’s included?
 
−
# what does it look like?
 
−
# how set it up?
 
−
 
−
== Science Code Manifesto Elements: ==
 
−
# Code
 
−
# Copyright
 
−
# Citation
 
−
# Credit
 
−
# Curation
 
−
 
−
== Steps To Software Product: ==
 
−
# Create citable, definitive version of software with doi, license, and repository.
 
−
# Define reference publication used to cite software.
 
−
# Define software as reference implementation of a method, and define a set of approved software acceptance/regression tests that can be used to establish a software implements that “method”.
 
−
# Create software maintenance organization with commit authority for pull requests and approval process for change requests, and process of approving new releases.
 
−
# Establish software community through registrations, newsletters, activity, regular calls, regular meetings, define community and roles.
 
−
 
−
== Adoption of Fork and Pull Git Repo Model ==
 
−
* Use the model used by the majority of open-source projects (including pyCSEP).
 
−
* The “maintainer” of the shared repo assigns rights to “Collaborators”
 
−
* Collaborators do not have push access to main (upstream) repo
 
−
* Core development teams accepts (PRs) from collaborators, reviews them, then merges them into main repo
 
−
 
−
== Contributor Process: ==
 
−
Working with shared projects on GitHub
 
−
# Fork the repository
 
−
# Clone your forked copy
 
−
# Sync your personal repo with shared repo
 
−
# Git merge/git rebase
 
−
# Make a contribution
 
−
# Pull request
 
−
 
−
== How we want it Cited: ==
 
−
* Example Citation:
 
−
* Example Acknowledgements:
 
−
* Example Reference:
 
−
 
−
== Basic Recommendations: ==
 
−
# Make source code publicly accessible
 
−
# Make software easy to discover by providing software metadata via a popular community registry (Examples of community registries of software metadata are bio.tools (Ison et al., 2016), (Ison et al., 2016) biojs.io (Corpas et al.,2014; Gómez et al., 2013) and Omic Tools (Henry et al., 2014) in the life sciences and DataCite (Brase, n.d.) as a generic metadata registry for software as well as data.
 
−
# Adopt a license and comply with the license of third party dependencies
 
−
# Define clear and transparent contribution, governance and communications processes (For instance the Galaxy project’s website describes the team’s structure, how to be part of the community, and their communication channels.)
 
−
 
−
== Types of Documentation with axis: ==
 
−
* help learning – help working
 
−
* theoretical knowledge – practical knowledge
 
−
# tutorials  - learning oriented
 
−
# how-to guides – task-oriented
 
−
# Background/Concept explanations – understanding-oriented
 
−
# technical reference – information-oriented
 
−
 
−
== DOCUMENTATION TYPES ==
 
−
* CODE DOCUMENTATION - Semantic identifiers, comments, API, engineering, dependencies, requirements
 
−
* USER DOCUMENTATION - How to get, run, use the software; parameters, data model, etc.; license
 
−
* MAINTENANCE DOCUMENTATION - How to build, release, review code, publish
 
−
* DEVELOPER DOCUMENTATION - How to contribute, contribution templates (issues, pull/merge requests)
 
−
* METADATA - Software metadata (CodeMeta), Citation File (CFF), "references" (dependencies)
 
−
* PROJECT DOCUMENTATION - Rationale, teams, governance, community (contact, code of conduct)
 
−
 
−
== Where Documentation Lives ==
 
−
Documentation lives where the source code lives! (This is never in an email, chat, or similar!)
 
−
 
−
Conceptual Documentation:
 
−
* Requirements
 
−
* Projects
 
−
Hands-on documentation
 
−
* How-tos, getting started
 
−
* Templates for issues, pull/merge
 
−
* Contribution guidelines
 
−
Reference documentation
 
−
* API
 
−
* Tests
 
−
* Metadata
 
−
 
−
== Toolbox Documentation: ==
 
−
Toolbox documentation should describe the steps off analysis in a pedagogical, narrative
 
−
fashion, with example data that users can load to follow along with and understand the documentation.
 
−
 
−
== UCVM Implements Multiple Test Types: ==
 
−
# Functional tests – Unit Tests essential core ucvm functions
 
−
# Integration Tests – Test utilities including meshing, layer searches, gtls, and performance
 
−
# Model Tests - Each velocity model has tests showing expected results for some points
 
−
# Example program - Examples programs and scripts showing working examples of ucvm capabilities.
 
−
# Acceptance tests – Confirm results on users system. Maybe union of funcational, integration, and model tests.
 
−
 
−
== Recommended Basic Software Practices: ==
 
−
# Training on Software Practices
 
−
# Code in a Code Repo
 
−
# Automated Testing
 
−
# Persistent ID for software versions
 
  
 
== UCVM Versus CIG Standards: ==
 
== UCVM Versus CIG Standards: ==

Latest revision as of 02:22, 4 November 2021

Updated UCVM Documentation

Citable Code

UCVM Tests Organization

A set of C language tests are defined as Unittests:

UCVM Programs found in the ucvm source code directory contains numerous example programs that are not included in the ucvm_bin installation directory. A list of programs in the UCVM Source directory is given here:

UCVM Plotting routines have been moved into a separate repo. Information on the plotting routines.

ToDo

  • Document the scripts in ucvm/utilities
  • Update the version numbers from 21.7 in get_largefiles.py and check_large_files.py
  • fix version in setup.list
  • confirm that models are correctly un-commented in ucvm.conf file (seems like cvm-H and others were not enabled)
  • Check references to proj-4 in documentation and code error messages and update if necessary to proj-5
  • Check ucvm/tests/Makefile.am setting, but never using, AM_CFLAGS
  • change, or explain to users, two different ucvm.conf in the system


Key UCVM Improvements in 21.10 release:

  • Move the run_ucvm.sh and run_ucvm_query.sh script to utilities
  • Convergence of versions
    • Used updated version used UCVM website
    • Combined source codebase web version and distributed version/command line version
    • reduce number of UCVM versions that we need to maintain
  • Large files next release stored on S3
    • Should improve download speed and reliability of install
  • CI setup
    • Review the CI for sceccode/ucvm before release
  • Documentation Updated into new structure
    • Re-organized to match recommended types of docs
  • Tests updated
    • combined c-language tests into unittest.
  • Code Metadata included in repo
    • added metadata to repo
  • Used Tags from USGS Thesarus for metadata
  • Post DOI badge on UCVM
  • UCVM Community developments encouraged
    • Add Code of Conduct document into repo
    • Added "contributor instructions" into README.md
  • Open Source Metrics setup (registered users, downloads, tags, visits)
  • Identification of sub-licenses in distribution
  • Velocity models stored in standalone repositories
  • Authorship contributions noted
  • add cv
  • add iv
  • add albacore
  • add utah wasatch front
  • projection code fix
  • Install using Dockerhub
  • Review issues that were done.
  • Added UCVM Website data plotting tools

TBD

  • Test with both Docker and singularity on an XSEDE system
  • Code coverage statements

UCVM Versus CIG Standards:

Minimum:

  • Version control – ok
  • Code – ok
  • Portable – ok
  • Testing – (a) tests that verify it runs properly (b) accuracy or benchmark tests
  • Documentation – (a) install (b) parameters (c) physics (d) example inputs cookbooks (e) citable pub
  • Userworkflow – ok

Standard:

  • Version control -ok
  • Coding – (a) params at runtime (b) development plan (c) code comments (d) add features without modify main branch (e) useful error reports
  • Portability: (a) dependency cheking (b) automake (c) output configuration and build options
  • Testing – pass fail tests
  • Documentation: (a) workflow for research (b) how to extend code
  • Userworkflow: (a) easy to change sim params (b) user specific directories/filenames for i/o (c) standard binary formats (d) citation for code version.
Retrieved from "https://strike.scec.org/scecwiki/index.php?title=UCVM_Release_Planning&oldid=26164"