Software Testing
New versions of publicly distributed scientific software codes released by SCEC software engineering group undergo rigorous software testing during development and prior to release.
Overview
CME Project software developers integrate complex scientific codes together into larger computational systems that we call Computational Platforms. Each SCEC computational platform is designed to reliably perform some useful or valuable research calculation. In some cases, SCEC provides source code distributions of computational platforms for use by the SCEC community. To ensure these platforms meet the design specifications and user requirements, rigorous software testing is employed. The two key forms of testing, Unit and Acceptance testing are employed to ensure the quality of the released software.
Types of Testing
Unit Testing
Unit testing, also referred to as White-box testing is employed to test the internal structure of individual code modules or sub-systems in the platform. This ensures these modules were 'built right'. Unit testing allows us to test parts of the program work correctly and reduces the time required to integrate these modules.
Unit testing also reduces uncertainty in units as these modules are re-factored into different platforms.
Acceptance Testing
Acceptance testing is also referred to as Black-box testing or validation testing. It consists of a suite of tests that are run to validate the end-to-end functionality of the integrated system. The test environment is designed to emulate the anticipated user environment. Unlike unit testing which tests the internal code structure of individual modules, Acceptance testing is more high level and is designed to test the integration of the underlying modules. Acceptance testing validates the overall functionality of the built system and ensures 'we built the right thing'.
The tests can be run with supplied input data for various use cases or using scripts that generate work-flows of exhaustive test cases to include every possible combination of sub-modules. The results of these tests give confidence that the integrated system is functioning as expected and is fit to be released.
