- Result found in typically 6-7 steps;
- running 20 instead of 30 samples seems sufficient
Breaking point in this example at stress-Factor 0.47 with run-time 39ms
This statistical criterion defines when to count observed Scheduler performance
as loosing control. The test is comprised of three observations, which
all must be confirmed:
- an individual run counts as accidentally failed when the execution slips
away by more than 2ms with respect to the defined overall schedule.
When more than 55% of all observed runs are considered as failed,
the first condition is met
- moreover, the observed standard derivation must also surpass the
same limit of > 2ms, which indicates that the Scheduling mechanism
is under substantial strain (on average, the slip is ~ 200µs)
- the third condition is that the ''averaged delta'' has surpassed
4ms, which is 2 times the basic failure indicator.
These conditions are based on watching the Scheduler in operation;
typically all three conditions slip away by large margin after a
very narrow yet critical increase in the stress level.
Using three conditions together should improve robustness; often
the problems to keep up with the schedule build up over some parameter
range, yet the actual decision should be based on complete loss of control.
adapt the code written yesterday explicitly for the test case
into the new framework for performing a stress-test run.
Notable difference: times converted to millisecond immediately
Elaborate the draft to include all the elements used directly in the test case thus far;
the goal is to introduce some structuring and leave room for flexible confguration,
while implementing the actual binary search as library function over Lambdas.
My expectation is to write a series of individual test instances with varying parameters;
while it seems possible to add further performance test variations into that scheme later on.
- the goal is to run a binary search
- the search condition should be factored out
- thus some kind of framework or DSL is required,
to separate the technicalities of the measurement
from the specifics of the actual test case.
