ichthyo/LUMIERA.clone
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

documentation of my internal testcase runner

Browse source
This commit is contained in:
Fischlurch 2007-08-23 19:13:28 +02:00
parent 5575a7679d
commit e3d1c35ca3
2 changed files with 85 additions and 9 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

49
tests/50components.tests
View file

@ -1,16 +1,15 @@
TESTING "Component Test Suite: ALL" ./test-components
TESTING "Component Test Suite" ./test-components
TEST "Fac test" Factory_test <<END
return: 0
END
TEST "Hello test" HelloWorld_test <<END
out: This is how the world ends...
return: 0
END
TEST "Cmdline Wrapper" CmdlineWrapper_test <<END
TEST "CmdlineWrapper_test" CmdlineWrapper_test <<END
out: wrapping cmdline:...
out: -->
out: wrapping cmdline:
@ -37,11 +36,45 @@ out: 4|€|
out: -->oo _O()O_ ä + €
out: wrapping cmdline:...
out: -->
out: Standard Cmdlineformat: one two
out: Standard Cmdlineformat:one two
END
TEST "Parseoption" TestOption_test <<END
TEST "TestOption_test" TestOption_test <<END
out: Testing invocation with cmdline: ...
return: 0
out: --> Testgroup=ALL
out: --> Test-ID =--missing--
out: --> remaining=
out: Testing invocation with cmdline: --help...
out: --> Testgroup=ALL
out: --> Test-ID =--missing--
out: --> remaining=
out: Testing invocation with cmdline: --group TestGroupID...
out: --> Testgroup=TestGroupID
out: --> Test-ID =--missing--
out: --> remaining=
out: Testing invocation with cmdline: SingleTestID...
out: --> Testgroup=ALL
out: --> Test-ID =SingleTestID
out: --> remaining=SingleTestID
out: Testing invocation with cmdline: SingleTestID --group TestGroupID...
out: --> Testgroup=TestGroupID
out: --> Test-ID =SingleTestID
out: --> remaining=SingleTestID
out: Testing invocation with cmdline: --group TestGroupID SingleTestID ...
out: --> Testgroup=TestGroupID
out: --> Test-ID =SingleTestID
out: --> remaining=SingleTestID
out: Testing invocation with cmdline: --group TestGroupID SingleTestID spam eggs...
out: --> Testgroup=TestGroupID
out: --> Test-ID =SingleTestID
out: --> remaining=SingleTestID spam eggs
out: Testing invocation with cmdline: SingleTestID spam --group TestGroupID --eggs...
out: --> Testgroup=TestGroupID
out: --> Test-ID =SingleTestID
out: --> remaining=SingleTestID spam --eggs
END
TEST "Factory_test" Factory_test <<END
END

45
wiki/index.html
View file

@ -3552,7 +3552,7 @@ Here is the order suggested:
</pre>
</div>
<div title="TestSuite" modifier="Ichthyostega" modified="200708180218" created="200708120254" tags="buildsys organization testsuite" changecount="11">
<div title="TestSuite" modifier="Ichthyostega" modified="200708231706" created="200708120254" tags="buildsys organization testsuite" changecount="16">
<pre>For running the automatic Tests, we use Cehteh's simple [[test.sh|TestSh]].
This page is a proposal (by Ichthyo) how the various tests could be organized.
@ -3564,6 +3564,49 @@ This page is a proposal (by Ichthyo) how the various tests could be organized.
* the Testsuite executable provides some command line magic to select individual tests.
* Top-level Testsuites or ''~Test-Collections'' for [[test.sh|TestSh]] contain calls to the different (sub)-Suites, together with the expected results/output
!internal Testsuite runner
The class {{{test::Suite}}} (common/test/suite.hpp) helps building an executable which will run all //registered// test case objects, or some group of such testcases. Each test case implements a simple interface and thus provides a {{{run (args)}}} function, moreover, it registers itself immediately alongside with his definition; this works by the usual trick of defining a static class object and calling some registration function from the constructor of this static var. See the following __hello-world-Example__:
{{{
#include &lt;iostream&gt;
#include &quot;common/test/run.hpp&quot;
class HelloWorld_test : public Test
{
virtual void run(Arg arg)
{
greeting();
}
void greeting()
{
std::cout &lt;&lt; &quot;This is how the world ends...\n&quot;;
}
};
/** Register this test class to be invoked in some test groups (suites) */
LAUNCHER (HelloWorld_test, &quot;unit function common&quot;);
}}}
Notes:
* type Arg is {{{typedef std::vector&lt;string&gt; &amp; Arg;}}}
* this vector may be {{{size()==0}}}, which means no comandline args available.
* otherwise arg[0] is always the ID (normally the classname) of the test
* the following args may contain further arguments passed from system commandline.
* the test can/should produce output that can be checked with Cehteh's [[./test.sh|TestSh]].
* the macro &quot;LAUNCHER&quot; expands to {{{Launch&lt;HelloWorld_test&gt; run_HelloWorld_test(&quot;HelloWorld_test&quot;,&quot;unit function common&quot;);}}}
* note the second parameter to the macro (or the Laucher-ctor) is a space-delimited list of group names
* thus any test can declare itself as belonging to some groups, and we can create a {{{test::Suite}}} for each group if we want.
!!!invoking the testrunner
The class {{{test::TestOption}}} predefines a boost-commandlineparser to support the following optons:
|&gt;|!{{{./test-components --group &lt;groupID&gt; [testID [arguments...]]}}}|
|{{{--help}}}| options summary|
|{{{--group|-g &lt;groupID&gt;}}}| build a Testsuite out of all tests from this group. If missing, ALL tests will be included |
|{{{[testID]}}}| (optional) one single testcase. If missing, all testcases of the group will be invoked |
|{{{--describe}}}| print all registered tests to stdout in a format suited for use with test.sh |
Further commandline arguments are deliverd to a single testcase only if you specify a {{{testID}}}. Otherwise, all commandline arguments remaining after options parsing will be discarded and all tests of the suite will be run with an commandline vector of size()==0
!conventions for the Buildsystem
to help with automating the build, ichthyo would appreciate to have the following conventions.
* in the {{{tests}}} directory are