ichthyo/LUMIERA.clone
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Scheduler: plot steps towards integration

Browse source 
(1) SchedulerInvocation_test
    »Layer-1« : Queue operation

(2) SchedulerCommutator_test
    »Layer-2« : Activity execution

(3) SchedulerUsage_test
    Component End-to-End
This commit is contained in:
Fischlurch 2023-10-16 22:46:31 +02:00
parent 3af6a54219
commit c76e5488bd
5 changed files with 220 additions and 26 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

41
doc/technical/overview.txt
View file

@ -303,8 +303,25 @@ invoke and control the dependency and time based execution of the _Render Jobs._
Regarding architecture, the Scheduler is located in the Vault-Layer and running the Scheduler
is equivalent to activating the »Vault Subsystem«. An EngineFaçade acts as entrance point,
providing high-level render services to other parts of the application: render jobs can be
activated under various timing and dependency constraints.
Internally, the implementation is organised into two layers:
activated under various timing and dependency constraints, which can be grouped
into three basic modes of control:
Deadline::
Higher priority jobs ordered by a deadline time plus some (negative) hysteresis.
Jobs are started when they approach their deadline. When it is foreseeable that a
jobs will miss the deadline, it will be skipped, preferring to use resources on
calculations expected to be successful and on time.
Best Effort::
Background calculations will use excess resources of the system;
a typical example would be the calculation of preview images and sound waveform
outlines for the GUI.
Freewheeling::
Calculation for final rendering will use all available resources and never
skip any processing, irrespective of the time it takes to complete.
Internally, the implementation of the Scheduler is organised in two layers:
Layer-2: Coordination::
Maintain a network of interconnected activities, track dependencies and observe
@ -313,28 +330,10 @@ Layer-2: Coordination::
Layer-1: Invocation::
Operates a low-level priority scheduling mechanism for time-bound execution of activities.
Generally speaking, a render calculation can be organised by different modes of control:
.Deadline
Higher priority jobs ordered by a deadline time plus some (negative) hysteresis.
Jobs are started when they approach their deadline. When it is foreseeable that a
jobs will miss the deadline, it will be skipped, preferring to use resources on
calculations expected to be successful and in time.
.Background
Background calculations will use excess resources of the system;
a typical example would be the calculation of preview images and sound waveform
outlines for the GUI.
.Best Effort
Calculation for final rendering will use all available resources and never skip any
processing, irrespecive of the time it takes to complete.
Job
^^^
A _Render Job_ is prepared as a self-contained functor, with a well defined time window
of execution. A job, once triggered, will run to completion without and can not be aborted.
of execution. A job, once triggered, will run to completion unattended and can not be aborted.
The scheduler is able to observe and maintain dependencies between render jobs, allowing
to break down an extended calculation into a chain of small steps.

4
src/vault/gear/scheduler-invocation.hpp
View file

@ -29,7 +29,7 @@
** @see SchedulerUsage_test integrated usage
** @see scheduler.cpp implementation details
**
** @todo WIP-WIP-WIP 6/2023 »Playback Vertical Slice«
** @todo WIP-WIP-WIP 10/2023 »Playback Vertical Slice«
**
*/
@ -60,7 +60,7 @@ namespace gear {
/**
* Scheduler Layer-2 : invocation.
* Scheduler Layer-1 : invocation.
*
* @see Scheduler
* @see SchedulerInvocation_test

2
tests/vault/gear/scheduler-activity-test.cpp
View file

@ -279,7 +279,7 @@ namespace test {
* - when receiving a notification, the latch is decremented
* - if this causes the Gate to open, the chain is immediately
* scheduled for activation, but the Gate also locked forever
* - neither a further activation, or a further notification
* - neither a further activation, nor a further notification
* has any effect after this point...
*/
void

16
tests/vault/gear/scheduler-invocation-test.cpp
View file

@ -29,13 +29,13 @@
#include "vault/gear/scheduler-invocation.hpp"
//#include "lib/time/timevalue.hpp"
//#include "lib/format-cout.hpp"
//#include "lib/util.hpp"
#include "lib/util.hpp"
//#include <utility>
using test::Test;
//using std::move;
//using util::isSameObject;
using util::isSameObject;
namespace vault{
@ -72,6 +72,18 @@ namespace test {
void
simpleUsage()
{
SchedulerInvocation sched;
Activity activity;
CHECK (not sched.peekHead());
sched.instruct (activity);
sched.feedPriorisation();
CHECK (sched.peekHead());
Activity* head = sched.pullHead();
CHECK (not sched.peekHead());
CHECK (isSameObject (*head, activity));
}

183
wiki/thinkPad.ichthyo.mm
View file

@ -81671,6 +81671,46 @@ Date:&#160;&#160;&#160;Thu Apr 20 18:53:17 2023 +0200<br/>
<node COLOR="#338800" CREATED="1687825916348" ID="ID_254177626" MODIFIED="1687826402411" TEXT="feedPriorisation">
<linktarget COLOR="#31c960" DESTINATION="ID_254177626" ENDARROW="Default" ENDINCLINATION="169;9;" ID="Arrow_ID_1506119350" SOURCE="ID_167939453" STARTARROW="None" STARTINCLINATION="-2;9;"/>
<icon BUILTIN="button_ok"/>
<node COLOR="#435e98" CREATED="1697474990041" HGAP="39" ID="ID_1899510010" MODIFIED="1697483151657" TEXT="wie wird der Bypass der Instruct-Queue realisiert?" VSHIFT="39">
<icon BUILTIN="help"/>
<node CREATED="1697475018687" ID="ID_1627247776" MODIFIED="1697475050844" TEXT="wenn wir uns in interner Verarbeitung befinden..."/>
<node CREATED="1697475051688" ID="ID_285432698" MODIFIED="1697475071594" TEXT="dann kann der POST einer Activity direkt in die Priority-Queue gehen"/>
<node COLOR="#435e98" CREATED="1697475074049" ID="ID_948956152" MODIFIED="1697483089574">
<richcontent TYPE="NODE"><html>
<head>
</head>
<body>
<p>
wird das hier auf Layer-1 realisiert &#8212;
</p>
<p>
oder bietet Layer-1 einen Direkt-Eingang?
</p>
</body>
</html></richcontent>
<icon BUILTIN="help"/>
<node CREATED="1697483009149" ID="ID_857193449" MODIFIED="1697483086272" TEXT="Konsistenz und Balance spielen eine Rolle">
<richcontent TYPE="NOTE"><html>
<head>
</head>
<body>
<p>
single level of abstraction / do only one thing and do it well
</p>
</body>
</html></richcontent>
</node>
<node CREATED="1697483030738" ID="ID_924466619" MODIFIED="1697483050956" TEXT="deshalb sollte das GroomingToken auf Layer-2 angesiedelt werden">
<icon BUILTIN="yes"/>
</node>
</node>
<node CREATED="1697483094233" ID="ID_410099903" MODIFIED="1697483115370" TEXT="&#x27f9; Layer-1 mu&#xdf; einen Direkteingang bereitstellen"/>
<node CREATED="1697483129772" ID="ID_22524648" MODIFIED="1697483148605" TEXT="die eigentliche Entscheidung ist der Kern des &#xbb;commutators&#xab;">
<icon BUILTIN="idea"/>
</node>
</node>
</node>
<node COLOR="#338800" CREATED="1687737502608" ID="ID_191302870" MODIFIED="1687826469913" TEXT="pullHead">
<icon BUILTIN="button_ok"/>
@ -84812,6 +84852,43 @@ Date:&#160;&#160;&#160;Thu Apr 20 18:53:17 2023 +0200<br/>
</node>
<node CREATED="1694015344557" ID="ID_571276241" LINK="https://en.cppreference.com/w/cpp/atomic/memory_order#Release-Acquire_ordering" MODIFIED="1694015377170" TEXT="auch CPP-Reference ist v&#xf6;llig klar in diesem Punkt (&#xbb;Release-Acquire-Ordering&#xab;)"/>
</node>
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1697473042062" ID="ID_1462757404" MODIFIED="1697473067073" TEXT="Platz im Scheduler und Zugriff">
<icon BUILTIN="flag-yellow"/>
</node>
</node>
</node>
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1697475170816" HGAP="36" ID="ID_1920465683" MODIFIED="1697475181832" TEXT="API festlegen" VSHIFT="7">
<icon BUILTIN="flag-yellow"/>
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1697475184438" ID="ID_140675790" MODIFIED="1697475203063" TEXT="bisher treten zwei verschiedene Perspektiven auf">
<icon BUILTIN="messagebox_warning"/>
<node CREATED="1697475204580" ID="ID_17372466" MODIFIED="1697475216748" TEXT="Layer-2 als Ausf&#xfc;hrungs-Kette">
<node CREATED="1697482695414" ID="ID_1741555904" MODIFIED="1697482816448" TEXT="und zwar insofern stets einige Schritte strukturiert vollzogen werden m&#xfc;ssen">
<linktarget COLOR="#63707f" DESTINATION="ID_1741555904" ENDARROW="Default" ENDINCLINATION="122;4;" ID="Arrow_ID_1523984047" SOURCE="ID_1818028653" STARTARROW="None" STARTINCLINATION="149;-4;"/>
</node>
<node CREATED="1697482829701" ID="ID_1600142035" MODIFIED="1697482859597" TEXT="...welche letztlich eine Activity f&#xfc;r den Dispatch liefern"/>
</node>
<node CREATED="1697475217562" ID="ID_1975524605" MODIFIED="1697475237819" TEXT="Layer-2 als Execution-Environment (f&#xfc;r ActivityLang)">
<node CREATED="1697482864748" ID="ID_696380968" MODIFIED="1697482909359" TEXT="konkret m&#xfc;ssen die ExecutionCtx-&#x3bb; implementiert werden"/>
<node CREATED="1697482924537" ID="ID_1521505068" MODIFIED="1697482960792" TEXT="jeweils mit sehr spezifischer Logik hinterlegt &#x27f6; GroomingToken"/>
</node>
<node BACKGROUND_COLOR="#fdfdcf" COLOR="#ff0000" CREATED="1697475301927" ID="ID_538072997" MODIFIED="1697475325940" TEXT="diese Perspektiven scheinen sich zu widersprechen">
<icon BUILTIN="broken-line"/>
<node CREATED="1697482590548" ID="ID_751862810" MODIFIED="1697482639262" TEXT="die erste Betrachtungsweise ist wohl inzwischen &#xfc;berholt">
<richcontent TYPE="NOTE"><html>
<head>
</head>
<body>
<p>
...sie ist entstanden aus den ersten Analysen, als die Bedeutung der &#187;Activity-Language&#171; noch nicht gekl&#228;rt war
</p>
</body>
</html></richcontent>
</node>
<node CREATED="1697482670795" ID="ID_1818028653" MODIFIED="1697482816448" TEXT="sie enth&#xe4;llt jedoch einen strukturell bedeutsamen Kern">
<arrowlink COLOR="#63707f" DESTINATION="ID_1741555904" ENDARROW="Default" ENDINCLINATION="122;4;" ID="Arrow_ID_1523984047" STARTARROW="None" STARTINCLINATION="149;-4;"/>
</node>
</node>
</node>
</node>
<node CREATED="1687788623248" ID="ID_1335631068" MODIFIED="1687788627335" TEXT="Ausf&#xfc;hrungs-Struktur">
@ -85682,18 +85759,124 @@ Date:&#160;&#160;&#160;Thu Apr 20 18:53:17 2023 +0200<br/>
<node CREATED="1688337123318" ID="ID_1686181795" MODIFIED="1688337233916" TEXT="&#xbb;Layer-1&#xab; : Queue operation">
<icon BUILTIN="info"/>
</node>
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1697473080944" ID="ID_1787410167" MODIFIED="1697493162214" TEXT="Aufgabe: die low-Level Operationen durchspielen">
<icon BUILTIN="yes"/>
</node>
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1697483171239" ID="ID_1311717572" MODIFIED="1697483179605" TEXT="scaffolding">
<icon BUILTIN="flag-yellow"/>
<node CREATED="1697483188828" ID="ID_1400224430" MODIFIED="1697483199834" TEXT="direkt konstruierte Activity-Records"/>
<node CREATED="1697483303661" ID="ID_1179779450" MODIFIED="1697483324629" TEXT="eingeben und query-API pr&#xfc;fen"/>
</node>
</node>
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1688336794355" ID="ID_1705923970" MODIFIED="1688337246572" TEXT="SchedulerCommutator_test">
<icon BUILTIN="flag-yellow"/>
<node CREATED="1688337141875" ID="ID_1442397948" MODIFIED="1688337233917" TEXT="&#xbb;Layer-2&#xab; : dependent Activities">
<icon BUILTIN="info"/>
</node>
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1697473097086" ID="ID_236233579" MODIFIED="1697493162215" TEXT="Aufgaben">
<icon BUILTIN="yes"/>
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1697483439491" ID="ID_20743819" MODIFIED="1697483452170" TEXT="die Bestandteile des API abdecken">
<icon BUILTIN="flag-yellow"/>
<node CREATED="1697483458376" ID="ID_1335346824" MODIFIED="1697483469019" TEXT="das GroomingToken"/>
<node CREATED="1697483469919" ID="ID_1121885008" MODIFIED="1697483490272" TEXT="die ExecutionCtx-&#x3bb;"/>
</node>
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1697473111557" ID="ID_750298563" MODIFIED="1697484608763" TEXT="Bestandteile des Work-Funktors direkt verbinden">
<icon BUILTIN="flag-yellow"/>
<node CREATED="1697484212735" ID="ID_853921255" MODIFIED="1697484238920" TEXT="Ma&#xdf;gabe: korrekte Verschaltung der ExecCtx-&#x3bb;">
<icon BUILTIN="yes"/>
</node>
<node CREATED="1697484623469" ID="ID_1364042662" MODIFIED="1697484634775" TEXT="hier noch kein geschlossenes Ende-zu-Ende"/>
<node CREATED="1697484240696" ID="ID_1718523921" MODIFIED="1697484965787" TEXT="also einen hinreichend komplexen Term in die Instruct-Queue stellen"/>
<node CREATED="1697484725655" ID="ID_246292556" MODIFIED="1697484730138" TEXT="das Grooming-Token erlangen"/>
<node CREATED="1697484656288" ID="ID_1076701414" MODIFIED="1697484744545" TEXT="Activity aus der prepareWork-Funktion herausholen"/>
<node CREATED="1697484919260" ID="ID_256219082" MODIFIED="1697484975618" TEXT="diese in den POST-Eingang geben"/>
<node CREATED="1697485021536" ID="ID_473846960" MODIFIED="1697485038425" TEXT="&#x27f9; Kette mu&#xdf; komplett durchlaufen"/>
<node CREATED="1697485039349" ID="ID_393821740" MODIFIED="1697485047472" TEXT="&#x27f9; GroomingToken mu&#xdf; gedropped werden"/>
</node>
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1697473133474" ID="ID_1452561509" MODIFIED="1697473150323" TEXT="high-level-Schnittstelle zur Planung kl&#xe4;ren">
<icon BUILTIN="flag-yellow"/>
<node CREATED="1697483606913" ID="ID_315948316" MODIFIED="1697483617085" TEXT="geh&#xf6;rt eigentlich nicht hierher"/>
<node CREATED="1697483617691" ID="ID_1269032096" MODIFIED="1697483624678" TEXT="es handelt sich um den POST-Eingang"/>
<node CREATED="1697484847247" ID="ID_441483633" MODIFIED="1697484871392" TEXT="zusammen mit dem (bereits konstruierten) ActivityLang-Term-Builer"/>
<node BACKGROUND_COLOR="#f0d5c5" COLOR="#990033" CREATED="1697485132481" ID="ID_1157256654" MODIFIED="1697485162006">
<richcontent TYPE="NODE"><html>
<head>
</head>
<body>
<p>
offen: wie erfolgt hier der <i>&#187;Seed&#171; </i>?
</p>
</body>
</html>
</richcontent>
<icon BUILTIN="help"/>
<node CREATED="1697485169820" ID="ID_372957466" MODIFIED="1697485263308" TEXT="wenigstens eine Activity mu&#xdf; von au&#xdf;en kommen">
<richcontent TYPE="NOTE"><html>
<head>
</head>
<body>
<p>
typischerweise ist das eine Planungs-Activity, also ein activity::Term::META_JOB
</p>
</body>
</html>
</richcontent>
</node>
<node BACKGROUND_COLOR="#f0d5c5" COLOR="#990033" CREATED="1697485275366" ID="ID_1311095701" MODIFIED="1697485322572" TEXT="spielt es &#xfc;berhaupt eine Rolle, da&#xdf; sie &#x201e;von au&#xdf;en&#x201c; kommt?">
<icon BUILTIN="help"/>
<node CREATED="1697485325431" ID="ID_9815483" MODIFIED="1697485338017" TEXT="im Extermfall k&#xf6;nnte das in Job-Ausf&#xfc;hrung m&#xfc;nden"/>
<node CREATED="1697485346020" ID="ID_657727930" MODIFIED="1697485349319" TEXT="und zwar wenn....">
<node CREATED="1697485350683" ID="ID_264150224" MODIFIED="1697485356918" TEXT="das GroomingToken erlangt werden kann"/>
<node CREATED="1697485382455" ID="ID_1769287316" MODIFIED="1697485401537" TEXT="die Job-Planung etwas f&#xfc;r &#x201e;sofort jetzt&#x201c; einplant"/>
<node CREATED="1697485403636" ID="ID_762107196" MODIFIED="1697485428517" TEXT="&#x27f9; Konsequenz: stets R&#xfc;ckkehr am Ende der Kette"/>
<node CREATED="1697485511390" ID="ID_1983862774" MODIFIED="1697485584481" TEXT="dagegen ein externer Aktor beim Start des Playback macht nur einen Aufruf">
<richcontent TYPE="NOTE"><html>
<head>
</head>
<body>
<p>
...was <i>schlimmstenfalls</i>&#160;dazu f&#252;hrt, da&#223; er direkt den ersten Planungs-Chunk ausf&#252;hrt
</p>
</body>
</html>
</richcontent>
</node>
<node CREATED="1697485465076" ID="ID_1682905763" MODIFIED="1697485510363" TEXT="ein Worker w&#xfc;rde dann halt gleich wieder auf der Matte stehen">
<richcontent TYPE="NOTE"><html>
<head>
</head>
<body>
<p>
...und w&#252;rde dadurch dann nach dem Meta-Job auch gleich noch den ersten Frame-Job ausf&#252;hren, was aber genau so auch w&#252;nschenswert ist
</p>
</body>
</html>
</richcontent>
</node>
</node>
</node>
</node>
</node>
</node>
</node>
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1688336835085" ID="ID_1753236898" MODIFIED="1688337246573" TEXT="SchedulerUsage_test">
<icon BUILTIN="flag-yellow"/>
<node CREATED="1688337200875" ID="ID_1269476607" MODIFIED="1688337233917" TEXT="Scheduler Component End-to-End ">
<icon BUILTIN="info"/>
</node>
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1697473158929" ID="ID_505069585" MODIFIED="1697493162215" TEXT="Aufgaben">
<icon BUILTIN="yes"/>
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1697484506891" ID="ID_1292738036" MODIFIED="1697484528227" TEXT="den kompletten Work-Funktor durchspielen">
<icon BUILTIN="flag-yellow"/>
</node>
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1697473158929" ID="ID_1723058065" MODIFIED="1697484548083" TEXT="vollst&#xe4;ndiger Ende-zu-Ende Test incl. WorkForce">
<icon BUILTIN="flag-yellow"/>
</node>
</node>
</node>
</node>
</node>