helpTextVerbose
auto helpTextVerbose =
q"EOS
Synopsis: diff-test-result-dirs [options] <test-dir>
This tool runs diff comparisons of test result files. The exit status gives the diff
status. Zero indicates success (no differences), one indicates failure (differences).
This tool was developed for TSV Utilities command line tests. Command line tests work
by running a tool against a set of command line test inputs. Results are written to
files and compared to a "gold" set of correct results. Tests generate one or more
output files; all written to a single directory. The resulting comparison is a "test"
directory vs a "gold" directory.
A directory level 'diff' is sufficient in many cases. This is the default behavior of
this tool. In some cases the corrent results depend on the compiler version. The main
case is error tests, where the output message may differ between runtime library
versions. (TSV Utilities often include exception message text in error output.)
The latter case is what this tool was developed for. It allows for multiple versions
of an output file in the gold set. The files used in the test are read from a JSON
config file. The config file also contains the set of version files available. The
effect is to run a modified form of directory diff, comparing the "test" directory
against the "gold" directory, allowing for the presence of version files.
The presence of the config file triggers the version-aware diff. A plain directory
diff is run if there is no config file.
An example JSON config file is shown below. Each output file has an entry with one
required element, "name", and one optional element, "versions". If present,
"versions" contains a list of alternate test files.
==== test-config.json ====
{
"output_files" : [
{
"name" : "test_1.txt"
},
{
"name" : "test_2.txt"
},
{
"name" : "test_3.txt",
"versions" : [
"test_3.2081.txt",
"test_3.2079.txt"
]
}
]
}
Options:
EOS";