lumiera_/tests
Ichthyostega 92bc044e9e Library: consider how to handle randomness in tests
Using random or pseudo-random numbers as input for tests
can be a very effective tool to spot unintended behaviour in
corner cases, and also helps writing more principled test verifications.
However, investigating failures in randomised tests can be challenging.

A well-proven solution is to exploit the **determinism** of pseudo-random-numbers
by documenting a randomly generated seed, that can be re-injected for investigation.

Up to now, most tests rely on the old library function `rand()`, while
at some places already the C++ standard framework for random number generation
is used, packaged into a custom wrapper. Adding adequate support for
documented seed values seems to be easy to achieve, after switching
existing usages of `rand()` to a suitable drop-in replacement.

After some consideration, I decided ''against'' wiring random generator instances
explicitly, while allowing to do so on occasion, when necessary. Thus
the planned seeding mechanism will rather re-seed a ''implicit default''
generator, which could then be used to construct explicit generator instances
when required (e.g. for multithreaded tests)

As a starting point, this changeset replaces the `randomise()` API call
by a direct access to the ''reseeding functionality'' exposed by the
C++ framework and all default generators. Since we already provide a
dedicated static instance of the plattform entropy source, re-randomisation
can be achieved by seeding from there.

NOTE: there was extended debate in the net, questioning the viability
of the `std::random_seq` -- these arguments, while valid from a theoretical
point of view, seem rather moot when placed into a practical context,
where even 2^32 different generation-paths(cycles) are more than enough
to provide sufficient diffusion of results (unless the goal is really to
engage into Monte-Carlo simulations for scientific research or large model
simulations).

Notable most of the more catchy reprovals raised by Melissa O'Neill
have been refuted by experts of the field, even while being still propagated
at various places in the net, often combined with promoting PCG-Random.
2024-11-10 03:25:45 +01:00
..
basics Library: uniform definition scheme for error-IDs 2024-03-21 19:57:34 +01:00
core Invocation: add some test coverage for the basic genrator function 2024-11-06 04:13:49 +01:00
include MERGE Doxygen clean-up done during the last months 2017-04-01 23:59:00 +02:00
library Library: consider how to handle randomness in tests 2024-11-10 03:25:45 +01:00
operation Doxygen: magically insert a reference to the test class 2017-02-22 03:17:18 +01:00
plugin Fix notorious warning 2017-04-01 23:59:37 +02:00
stage Library: uniform definition scheme for error-IDs 2024-03-21 19:57:34 +01:00
tool Commands: some unit test coverage for unbinding arguments 2017-04-16 19:38:56 +02:00
vault Library: reorganise test helpers and cover logging tracker object 2024-06-16 04:22:29 +02:00
00helloworld.tests link tests with stringent application scope dependencies (closes #938) 2014-10-17 21:15:59 +02:00
00support.tests Library: reorganise test helpers and cover logging tracker object 2024-06-16 04:22:29 +02:00
00test.tests
10errorhandling.tests re-arrange tests according to layer structure 2013-01-07 05:43:01 +01:00
10exception.tests Library: discontinue setting error flag from Exceptions (see #1341) 2023-10-01 20:11:45 +02:00
11concurrency.tests LibraryApplication: tie DispatcherLoop to thread lifecycle 2023-10-12 20:23:59 +02:00
11list.tests reorganise test suite compartments 2014-10-17 20:02:25 +02:00
11locking.tests Library/Application: complete technology switch (closes #1279) 2023-10-16 01:44:04 +02:00
11luid.tests reorganise test suite compartments 2014-10-17 20:02:25 +02:00
11safeclib.tests reorganise test suite compartments 2014-10-17 20:02:25 +02:00
11time.tests test-suite: fix lots of missing return value checks 2016-01-28 22:30:24 +01:00
11vault-memory.tests Block-Flow: completed implementation of low-level cyclic extent storage 2023-07-12 21:55:50 +02:00
12metaprogramming.tests Library: clarify name of index-based iterator 2024-11-09 22:43:05 +01:00
15library.tests Invocation: consider storage and allocation of fan-in/fan-out 2024-05-13 18:34:42 +02:00
16calculation.tests Library: complete coverage of CSV data table including storage 2024-03-15 02:45:45 +01:00
20config_lowlevel.tests clean-up: sourcefile layout and spell checking 2014-10-23 23:04:35 +02:00
20threads.tests
22config_highlevel.tests re-arrange tests according to layer structure 2013-01-07 05:43:01 +01:00
25fundamental.tests Library: prepare switch of Thread-wrapper to C++17 2023-09-21 23:23:55 +02:00
30backend-file.tests
30interfaces.tests
31plugin.tests integrate priority queue: lumiera namespace prefix; unit test pass 2013-09-13 05:44:58 +02:00
32scheduler.tests Scheduler-test: complete and document stress testing effort (closes #1344) 2024-04-20 01:56:54 +02:00
40core.tests Global-Layer-Renaming: adapt lots of documentation 2018-11-15 21:13:52 +01:00
41asset.tests Assets: investigating the unclear distinction between asset::Struct and asset::Meta (#1156) 2018-09-14 21:06:12 +02:00
42query.tests Global-Layer-Renaming: adapt lots of documentation 2018-11-15 21:13:52 +01:00
43session.tests Dispatcher-Pipeline: prepare test for the new NodeGraphAttachment 2023-06-07 04:03:00 +02:00
44controller.tests Dispatcher-Pipeline: prepare test for the new NodeGraphAttachment 2023-06-07 04:03:00 +02:00
45builder.tests Dispatcher-Pipeline: prepare test for the new NodeGraphAttachment 2023-06-07 04:03:00 +02:00
46fixture.tests Dispatcher-Pipeline: create hook for self-validation 2023-06-12 01:18:59 +02:00
46node.tests Invocation: forge a path for integration 2024-04-23 01:13:40 +02:00
47engine.tests Invocation: start with some rename-refactorings 2024-05-11 16:39:58 +02:00
48playout.tests Dispatcher-Pipeline: prepare test for the new NodeGraphAttachment 2023-06-07 04:03:00 +02:00
49core-integration.tests Dispatcher-Pipeline: prepare test for the new NodeGraphAttachment 2023-06-07 04:03:00 +02:00
51-gui-model.tests Segmentation: structure analysis for splitSplice operation 2023-05-02 04:29:34 +02:00
52-gui-control.tests Global-Layer-Renaming: fix remaining textual usages and IDs in the code 2018-12-10 00:09:56 +01:00
80-regression.tests test-suite: fix lots of missing return value checks 2016-01-28 22:30:24 +01:00
DIR_INFO
SConscript Global-Layer-Renaming: fix remaining textual usages and IDs in the code 2018-12-10 00:09:56 +01:00
test.conf Testsuite: increase virtual memory limit 2023-10-04 22:42:37 +02:00
test.sh test.sh - reorganise how limits are set 2015-08-16 01:17:39 +02:00
testrunner.cpp Doxygen: supply missing file level comments for test support helpers 2017-02-22 01:58:49 +01:00