UCVM Release Planning: Difference between revisions

From SCECpedia
Jump to navigationJump to search
← Older edit
Maechlin (talk | contribs)
11,776 edits
Maechlin (talk | contribs)
11,776 edits
No edit summary
 
(35 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Fork and Pull Development ==
== Updated UCVM Documentation ==
Using the fork and pull method, start with fork of sceccode/ucvm.git into personal repo. Then, to do development on CARC, plan to clone pjmaechling/ucvm.git for development. Set git upstream to original repo sceccode/ucvm.git to keep in sync with that repo. Follow Instruction for setting upstream here:
*[https://github.com/sceccode/ucvm/wiki UCVM Wiki]
*[https://github.com/billtron/pycsep_git_tutorial Savran Git Tutorial]
**[UCVM README.md]
*[https://github.com/sceccode/ucvm_docker/wiki UCVM Docker Wiki]
**[UCVM Docker README.md]
*[https://github.com/sceccode/ucvm_plotting/wiki UCVM Plotting Wiki]
**[UCVM Plotting READM.md]


== Development on CARC ==
== Citable Code ==
* ssh -o "ServerAliveInterval 60" discovery.usc.edu
*[https://guides.lib.berkeley.edu/citeyourcode Make UCVM Citable]
* cd /project/scec_608/maechlin/dev
* git clone https://github.com/pjmaechling/ucvm.git
* adding codemeta.json file.
* add Contributors.MD file.
* Follow UCVM installation instructions: [https://github.com/SCECcode/UCVMC/wiki/Installation-From-Source-Code Install Wiki]
*[https://strike.scec.org/scecpedia/UCVM_Install Install Notes on Discovery]


== Compiling ==
== UCVM Tests Organization ==
* the source directory /project/scec_608/maechlin/dev/ucvm
A set of C language tests are defined as Unittests:
* the installation directory /project/scec_608/maechlin/ucvm_bin
*[[UCVM Testing]]


== UCVM Tests Organized ==
UCVM Programs found in the ucvm source code directory contains numerous example programs that are not included in the ucvm_bin installation directory. A list of programs in the UCVM Source directory is given here:
*[[UCVM Tests]]
*[[UCVM Examples]]
 
UCVM Plotting routines have been moved into a separate repo. Information on the plotting routines.
*[[UCVM Plotting]]


== Stage Large files on public S3 ==
* get_large_files.py script in release points to hypocenter.usc.edu/research/ucvmc/V19_4
* copy this directory from hypocenter
* create v21.10 directory
* move large files into that
* move v21.10 to ResearchComputing AWS account under maechlin@usc.edu
== ToDo ==
== ToDo ==
* Change the CVM names presented to users during install: Use science names.
 
* Move the run_ucvm.sh and run_ucvm_query.sh script to utilities
* Document the scripts in ucvm/utilities
* Document the scripts in ucvm/utilities
* Update the version numbers from 21.7 in get_largefiles.py and check_large_files.py
* Update the version numbers from 21.7 in get_largefiles.py and check_large_files.py
* fix version in setup.list
* fix version in setup.list
* confirm that models are correctly un-commented in ucvm.conf file (seems like cvm-H and others were not enabled)
* Check references to proj-4 in documentation and code error messages and update if necessary to proj-5
* Check ucvm/tests/Makefile.am setting, but never using, AM_CFLAGS
* change, or explain to users, two different ucvm.conf in the system




Friday Due Date
== Key UCVM Improvements in 21.10 release: ==
* SAM Poster
* Move the run_ucvm.sh and run_ucvm_query.sh script to utilities
* SAM Presentation Video
*Convergence of versions
 
** Used updated version used UCVM website
== Key UCVM Improvements: ==
** Combined source codebase web version and distributed version/command line version
#Convergence of versions
** reduce number of UCVM versions that we need to maintain
#Large files next release stored on S3
*Large files next release stored on S3
#CI setup
** Should improve download speed and reliability of install
#Documentation Updated into new structure
*CI setup
#Tests output
** Review the CI for sceccode/ucvm before release
#Code Metadata included in repo
*Documentation Updated into new structure
#Tags from USGS Thesarus
** Re-organized to match recommended types of docs
#Post DOI badge on UCVM
*Tests updated
#Test with singularity on an XSEDE system
** combined c-language tests into unittest.
#UCVM Communitee
* Code Metadata included in repo
#Code of Conduct
** added metadata to repo
#Open Source Metrics setup
* Used Tags from USGS Thesarus for metadata
#Code coverage statements
* Post DOI badge on UCVM
#Identification of sub-licenses in distribution
* UCVM Community developments encouraged
#Authorship contributions noted
** Add Code of Conduct document into repo
 
** Added "contributor instructions" into README.md
== Standard Contents of Git repo: ==
* Open Source Metrics setup (registered users, downloads, tags, visits)
#A README with pictures/gifs of the product in action and a nice logo.
* Identification of sub-licenses in distribution
#Documentation.
* Velocity models stored in standalone repositories
#Code QA (Static Code Analysis).
* Authorship contributions noted
#Contributing instructions.
* add cv
#A well-defined setup section.
* add iv
#Support (Respond to Issues/PR)
* add albacore
#Publish software news in every possible way.
* add utah wasatch front
 
* projection code fix
== Contents of README.md ==
* Install using Dockerhub
When someone is looking at your project, they want to know:
* Review issues that were done.
# what is it?
* Added UCVM Website data plotting tools
# how good the code is?
# how much support is available?
# what’s included?
# what does it look like?
# how set it up?
 
== Science Code Manifesto Elements: ==
# Code
# Copyright
# Citation
# Credit
# Curation
 
== Steps To Software Product: ==
# Create citable, definitive version of software with doi, license, and repository.
# Define reference publication used to cite software.
# Define software as reference implementation of a method, and define a set of approved software acceptance/regression tests that can be used to establish a software implements that “method”.
# Create software maintenance organization with commit authority for pull requests and approval process for change requests, and process of approving new releases.
# Establish software community through registrations, newsletters, activity, regular calls, regular meetings, define community and roles.
 
== Adoption of Fork and Pull Git Repo Model ==
* Use the model used by the majority of open-source projects (including pyCSEP).
* The “maintainer” of the shared repo assigns rights to “Collaborators”
* Collaborators do not have push access to main (upstream) repo
* Core development teams accepts (PRs) from collaborators, reviews them, then merges them into main repo
 
== Contributor Process: ==
Working with shared projects on GitHub
# Fork the repository
# Clone your forked copy
# Sync your personal repo with shared repo
# Git merge/git rebase
# Make a contribution
# Pull request
 
== How we want it Cited: ==
* Example Citation:
* Example Acknowledgements:
* Example Reference:
 
== Basic Recommendations: ==
# Make source code publicly accessible
# Make software easy to discover by providing software metadata via a popular community registry (Examples of community registries of software metadata are bio.tools (Ison et al., 2016), (Ison et al., 2016) biojs.io (Corpas et al.,2014; Gómez et al., 2013) and Omic Tools (Henry et al., 2014) in the life sciences and DataCite (Brase, n.d.) as a generic metadata registry for software as well as data.
# Adopt a license and comply with the license of third party dependencies
# Define clear and transparent contribution, governance and communications processes (For instance the Galaxy project’s website describes the team’s structure, how to be part of the community, and their communication channels.)
 
== Types of Documentation with axis: ==
* help learning – help working
* theoretical knowledge – practical knowledge
# tutorials  - learning oriented
# how-to guides – task-oriented
# Background/Concept explanations – understanding-oriented
# technical reference – information-oriented
 
== DOCUMENTATION TYPES ==
* CODE DOCUMENTATION - Semantic identifiers, comments, API, engineering, dependencies, requirements
* USER DOCUMENTATION - How to get, run, use the software; parameters, data model, etc.; license
* MAINTENANCE DOCUMENTATION - How to build, release, review code, publish
* DEVELOPER DOCUMENTATION - How to contribute, contribution templates (issues, pull/merge requests)
* METADATA - Software metadata (CodeMeta), Citation File (CFF), "references" (dependencies)
* PROJECT DOCUMENTATION - Rationale, teams, governance, community (contact, code of conduct)
 
== Where Documentation Lives ==
Documentation lives where the source code lives! (This is never in an email, chat, or similar!)
Conceptual Documentation:
* Requirements
* Projects
Hands-on documentation
* How-tos, getting started
* Templates for issues, pull/merge
* Contribution guidelines
Reference documentation
* API
* Tests
* Metadata
 
== Toolbox Documentation: ==
Toolbox documentation should describe the steps off analysis in a pedagogical, narrative
fashion, with example data that users can load to follow along with and understand the documentation.
 
== Implement Multiple Test Types: ==
# Functional tests – Unit Tests essential core ucvm functions
# Integration Tests – Test utilities including meshing, layer searches, gtls, and performance
# Model Tests - Each velocity model has tests showing expected results for some points
# Acceptance tests – Confirm results on users system. Maybe union of funcational, integration, and model tests.


== Recommended Basic Practices: ==
TBD
# Training on Software Practices
* Test with both Docker and singularity on an XSEDE system
# Code in a Code Repo
* Code coverage statements
# Automated Testing
# Persistent ID for software versions


== UCVM Versus CIG Standards: ==
== UCVM Versus CIG Standards: ==

Latest revision as of 02:22, 4 November 2021

Updated UCVM Documentation

Citable Code

UCVM Tests Organization

A set of C language tests are defined as Unittests:

UCVM Programs found in the ucvm source code directory contains numerous example programs that are not included in the ucvm_bin installation directory. A list of programs in the UCVM Source directory is given here:

UCVM Plotting routines have been moved into a separate repo. Information on the plotting routines.

ToDo

  • Document the scripts in ucvm/utilities
  • Update the version numbers from 21.7 in get_largefiles.py and check_large_files.py
  • fix version in setup.list
  • confirm that models are correctly un-commented in ucvm.conf file (seems like cvm-H and others were not enabled)
  • Check references to proj-4 in documentation and code error messages and update if necessary to proj-5
  • Check ucvm/tests/Makefile.am setting, but never using, AM_CFLAGS
  • change, or explain to users, two different ucvm.conf in the system


Key UCVM Improvements in 21.10 release:

  • Move the run_ucvm.sh and run_ucvm_query.sh script to utilities
  • Convergence of versions
    • Used updated version used UCVM website
    • Combined source codebase web version and distributed version/command line version
    • reduce number of UCVM versions that we need to maintain
  • Large files next release stored on S3
    • Should improve download speed and reliability of install
  • CI setup
    • Review the CI for sceccode/ucvm before release
  • Documentation Updated into new structure
    • Re-organized to match recommended types of docs
  • Tests updated
    • combined c-language tests into unittest.
  • Code Metadata included in repo
    • added metadata to repo
  • Used Tags from USGS Thesarus for metadata
  • Post DOI badge on UCVM
  • UCVM Community developments encouraged
    • Add Code of Conduct document into repo
    • Added "contributor instructions" into README.md
  • Open Source Metrics setup (registered users, downloads, tags, visits)
  • Identification of sub-licenses in distribution
  • Velocity models stored in standalone repositories
  • Authorship contributions noted
  • add cv
  • add iv
  • add albacore
  • add utah wasatch front
  • projection code fix
  • Install using Dockerhub
  • Review issues that were done.
  • Added UCVM Website data plotting tools

TBD

  • Test with both Docker and singularity on an XSEDE system
  • Code coverage statements

UCVM Versus CIG Standards:

Minimum:

  • Version control – ok
  • Code – ok
  • Portable – ok
  • Testing – (a) tests that verify it runs properly (b) accuracy or benchmark tests
  • Documentation – (a) install (b) parameters (c) physics (d) example inputs cookbooks (e) citable pub
  • Userworkflow – ok

Standard:

  • Version control -ok
  • Coding – (a) params at runtime (b) development plan (c) code comments (d) add features without modify main branch (e) useful error reports
  • Portability: (a) dependency cheking (b) automake (c) output configuration and build options
  • Testing – pass fail tests
  • Documentation: (a) workflow for research (b) how to extend code
  • Userworkflow: (a) easy to change sim params (b) user specific directories/filenames for i/o (c) standard binary formats (d) citation for code version.
Retrieved from "https://strike.scec.org/index.php?title=UCVM_Release_Planning&oldid=26164"