Test Built Dakota

The following tests can be run after compiling Dakota (NOTE: the examples shown below assume that the binary tree is named build)

Simple version test:


$ build/src/dakota -version

Dakota version 6.4+ (stable) released Jun  1 2016.
Subversion revision 4268 built Jun  1 2016 10:41:13.

Developers can run the test suite labelled FastTest or (optional) the test suite labelled AcceptanceTest.  The FastTest is typically used for developer's pre-commit, "sanity" check and will usually run in just a few minutes on a SNL COE RHEL6/GCC 4.4.7 development host. The AcceptanceTest suite is useful for assessing Dakota quality for atypical platforms/configurations.  Unfortunately ctest reports that a test showing a DIFF to be a FAIL. To avoid executing tests that are known to sporadically DIFF (e.g. a test exercising a stochastic algorithm), ctest's exclude feature can be used to avoid the frustration of needing to perform further diagnostics to determine whether a test FAIL is indeed genuine.

cd build/test
# "pre-commit" set of fast-running tests that will PASS on typical developer's local host (i.e. RHEL6/GCC 4.4.7)
ctest -j 4 -L FastTest -LE Diff
# test suite that should PASS, even for non-development, but supported hosts (e.g. MS Windows)
ctest -j 4 -L AcceptanceTest
# run all Dakota tests active in this build (should PASS or DIFF)
ctest -j 4 -R dakota
# Show all available ctest labels
ctest  --print-labels
# Show all relevant tests but do not run; can be used with various labels (-L) or regular expressions (-R)
ctest -N -R dakota_

Note that ctest represents a coarse level of testing in that each ctest test may actually contain multiple variations of a base test. If any of these variations report DIFF or FAIL, the entire ctest test reports a FAIL. To obtain more granular test information, a dakota_diffs.out summary can be generated via:


# Unix only: generate difference files dakota_diffs.out, dakota_pdiffs.out
make dakota-diffs

The resulting dakota_[p]diffs.out file can then be inspected to ascertain which test variation(s) failed, eg


dakota_experiment_read.in
PASS test 0
PASS test 1
FAIL test 2 (*** new ***)
FAIL test 3 (*** new ***)

A successful build should include PASS and DIFF, with associated small numerical differences.  If these files contain DIFF with large abberations, or FAIL, there may be an issue with the build.