Diagnosing non-PASSing tests

If any credible test differences are found, a starting point for determing the cause of failure involves the following steps:

  1. Determine which test variation(s) is (are) failing by invoking "make dakota-diffs" from within the test subdirectory.  Examine the resulting dakota_[p]diffs.out file to see which failures are new.
  2. Change into the subdirectory corresponding to the failing test, eg "cd dakota_experiment_read".
  3. Extract the test variation that is failing using the dakota_test.perl utility, eg for test #3 of dakota_experiment_read

# From within the test subdirectory of the build directory
cd dakota_experiment_read
./dakota_test.perl --extract --file-extract test3.in dakota_experiment_read.in 3

Note that documentation for using the dakota_test.perl script can be obtained by invoking it with either "--help" or "--man". At this point, there will be a Dakota input file, test3.in. You can then run this test variation directly, eg "../../src/dakota -i test3.in". The console output and/or log files can then provide details about what why the test did not pass.