benn/lumiera_
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Scheduler-test: investigate extended load patterns

Browse source 
...this seems to be the last topic for this investigation of Scheduler behaviour;
the goal is to demonstrate readiness for stable-state operation over an extended period of time
This commit is contained in:
Fischlurch 2024-04-15 16:54:42 +02:00
parent 1d4f6afd18
commit 7c2b9a8ba5
3 changed files with 12 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
tests/vault/gear/scheduler-stress-test.cpp
View file

@ -481,12 +481,13 @@ cout << "time="<<runTime/1000
struct Setup : StressRig
{
double UPPER_STRESS = 12;
// double UPPER_STRESS = 12;
//
double FAIL_LIMIT = 1.0; //0.7;
// double FAIL_LIMIT = 1.0; //0.7;
double TRIGGER_SDEV = 1.0; //0.25;
double TRIGGER_DELTA = 2.0; //0.5;
// uint CONCURRENCY = 4;
uint CONCURRENCY = 8;
// bool SCHED_DEPENDS = true;
bool showRuns = true;
@ -504,8 +505,8 @@ cout << "time="<<runTime/1000
auto testSetup (TestLoad& testLoad)
{
return StressRig::testSetup(testLoad)
.withBaseExpense(200us)
.withLoadTimeBase(500us);
// .withBaseExpense(200us)
.withLoadTimeBase(4ms);
}
};
auto [stress,delta,time] = StressRig::with<Setup>()

2
tests/vault/gear/stress-test-rig.hpp
View file

@ -207,7 +207,7 @@ namespace test {
uint CONCURRENCY = work::Config::getDefaultComputationCapacity();
bool INSTRUMENTATION = true;
double EPSILON = 0.01; ///< error bound to abort binary search
double UPPER_STRESS = 0.6; ///< starting point for the upper limit, likely to fail
double UPPER_STRESS = 1.2; ///< starting point for the upper limit, likely to fail
double FAIL_LIMIT = 2.0; ///< delta-limit when to count a run as failure
double TRIGGER_FAIL = 0.55; ///< %-fact: criterion-1 failures above this rate
double TRIGGER_SDEV = FAIL_LIMIT; ///< in ms : criterion-2 standard derivation

8
wiki/renderengine.html
View file

@ -7317,7 +7317,7 @@ The primary scaling effects exploited to achieve this level of performance are t
&amp;rarr; [[Scheduler performance testing|SchedulerTest]]
</pre>
</div>
<div title="SchedulerProcessing" creator="Ichthyostega" modifier="Ichthyostega" created="202312281750" modified="202404111651" tags="Rendering operational draft" changecount="7">
<div title="SchedulerProcessing" creator="Ichthyostega" modifier="Ichthyostega" created="202312281750" modified="202404150019" tags="Rendering operational draft" changecount="9">
<pre>At first sight, the internals of [[Activity|RenderActivity]] processing may seem overwhelmingly complex -- especially since there is no active »processing loop« which might serve as a starting point for the understanding. It is thus necessary to restate the working mode of the Scheduler: it is an //accounting and direction service// for the //active// [[render workers|SchedulerWorker]]. Any processing happens stochastically and is driven by various kinds of events --
* a //worker// becoming ready to perform further tasks
* an external //IO event// {{red{12/23 only planned yet}}}
@ -7325,9 +7325,13 @@ The primary scaling effects exploited to achieve this level of performance are t
The last point highlights a //circular structure:// the planning job itself was part of the schedule, picked up by a worker and activated; this way, the system feeds itself.
!Participants and Realms
From the outside, the Scheduler appears as a service component, exposing two points-of-access: Jobs can be added to the schedule (planned), and a worker can retrieve the next instruction, which is either to perform an (opaque) computation, or to sleep for some given short period of time. Jobs are further distinguished into processing tasks, IO activities and meta jobs, which are related to the self-regulation of the scheduling process. On a closer look however, several distinct realms can be identified, each offering an unique perspective of operation.
From the outside, the Scheduler appears as a service component, exposing two points-of-access: Jobs can be added to the schedule (planned), and a worker can retrieve the next instruction, which implies either to perform an (opaque) computation, or to sleep for some given short period of time. Jobs are further distinguished into processing tasks, IO activities and meta jobs, which are related to the self-regulation of the scheduling process. On a closer look however, several distinct realms can be identified, each offering an unique perspective of operation.
!!!The Activity Realm
[[Activities|SchedulerActivity]] are part of an //activity language// to describe patterns of processing, and will be arranged into activity terms; what is scheduled is actually the entrance point to such a term. Within this scope it is assumed abstractly that there is some setup and arrangement to accomplish the //activation// of activities. At the level of the activity language, this is conceptualised and represented as an {{{ExecutionCtx}}}, providing a set of functions with defined signature and abstractly defined semantics. This allows to define and even implement the entirety of activity processing without any recurrence to implementation structures of the scheduler. Rather, the functions in the execution-context will absorb any concern not directly expressed in relations of activities (i.e. anything not directly expressed as language term).
While the ability to reason about activities and verify their behaviour in isolation is crucial to allow for the //openness// of the activity language (able to accommodate future requirements not foreseeable at this time), it also creates some kind of »double bottom« -- which can be challenging when it comes to reasoning about the processing steps within the scheduler. While all functions of the execution-context are in fact wired into internals of the scheduler for the actual usage, it is indispensable to keep both levels separate and to ensure each level is logically consistent in itself.
!!!Queues and Activation Events
!!!The Worker Aspect
!!!Process Control