Commit graph

7 commits

Author SHA1 Message Date
ef91088fb7 Documentation clean-up and fixes
more clean-up and polishing
after some further test regarding the topic of timing measurements

Improved handling: filter test cases to be performed
2024-03-10 23:36:38 +01:00
48d74f261f heuristics to establish a tolerance band for watching global trends
After the individual tests, we calculate the averaged delta over the
whole test suite, to detect changes to the overall timings. As it turned out,
using the error propagation for the calculation of the averaged delta
yields the right tolerance band to ignore random fluctuations but
trigger alarm on real changes.

Moreover, add several further timing test cases
to verify the calibration via "platform model" works as intended
2024-03-10 23:36:38 +01:00
f2adfc6406 calculate statistics and trend for the complete testsuite 2024-03-10 23:36:38 +01:00
e9eae205f0 use regression to monitor short term and long term trends
Since the platform calibration inevitably incurs some additional error band,
a linear regresssion over the time series of measurements can additionally be used
to spot ongoing systematic changes below this general error band, while
leveling out local statistical fluctuations.
2024-03-10 23:36:38 +01:00
8f9e54b7c7 implement fitting the platform model by linear regression
* triggered by --calibrate
 * normalise away any known expense factors, but use them as weight
 * calculate simple linear regression from statistic data
2024-03-10 23:36:38 +01:00
015a6ed6f2 add global storage and apply existing platform model
...this is largely just wiring of components built thus far

...TODO build platform model by linear regression
2024-03-10 23:36:38 +01:00
75767a3a97 capture and store individual timings as time series
Note: work-in-progress...
TODO: derive the expense factor and delta
2024-03-10 23:36:38 +01:00