LUMIERA.clone/doc/devel/rfc/SchedulerRequirements.txt

SchedulerRequirements
=====================

// please don't remove the //word: comments

[grid="all"]
`------------`-----------------------
*State*         _Idea_
*Date*          _Mi 09 Jan 2013 12:04:03 CET_
*Proposed by*   Ichthyostega <prg@ichthyostega.de>
-------------------------------------

********************************************************************************
.Abstract
The rendering and playback subsystem relies on a Scheduler component to invoke
individual frame rendering and resource fetching jobs. This RfC summarises the
general assumptions and requirements other parts of the application are relying
on
********************************************************************************

Define the core properties and requirements of the Scheduler.

Description
-----------
//description: add a detailed description:

The *Scheduler* is responsible for getting the individual _render jobs_ to run.
The basic idea is that individual render jobs _should never block_ -- and thus
the calculation of a single frame might be split into several atomic jobs,
including resource fetching. This expected usage should be considered together
with the data exchange protocol defined for data output through the `OutputSlot`
instances; moreover the extended data of the low-level model can be hot-swapped
while rendering continues to go on, necessitating to release blocks of superseded
model data at well defined points. Combining all these known usage constraints
leads to the following requirements for the scheduler:

ordering of jobs::
  the scheduler has to ensure all prerequisites of a given job are met
  
job time window::
  when it is not possible to run a job within the defined target time window, it
  must not be run any more but rather be marked as failure

failure propagation::
  when a job fails, either due to an job internal error, or by timing glitch,
  the effect of this failure needs to propagate reliably; we need a mechanism
  for dependent jobs to receive a notification of such a failure state

conditional scheduling::
  we need to provide some way to tie the activity of jobs to external conditions,
  notable examples being the availability of cached data, or the arrival of data
  loaded from storage

superseding of planned jobs::
  changes in playback modes require us to ``change the plan on-the-fly'' --
  essentially this means we need to 'supersede' a group of already planned jobs.
  Moreover, we need certain ordering guarantees to ensure the resulting switch
  in the effective output data happens once and without glitches.

The scheduler interface and specification establishes some kind of micro-language
to encode the patterns of behaviour prompted by the playback control and the
interpretation of the render node model. Together these basic requirements
help to address some relevant themes

dependency on prerequisites
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Render tasks don't exist in isolation; they depend on prerequisites, both preceding
calculations and the availability of data. Since our primary decision is to avoid
blocking waits, these prerequisites need to be modelled as other jobs, which leads
to dependencies and conditional scheduling.


detecting termination
~~~~~~~~~~~~~~~~~~~~~
The way other parts of the system are built, requires us to obtain a guaranteed
knowledge of some specific job's termination. More precisely, we need to find out
when a ``stream of calculations'' has left a well defined domain -- and this can
be modelled by passing of some marker jobs. It is possible to obtain that knowledge
with some timing leeway, but in the end, this information needs to arrive with
absolutely reliability (violations leading to segfault).



Tasks
~~~~~
// List what needs to be done to implement this Proposal:
// * first step ([green]#✔ done#)
 * define the job interface [yellow-background]#WIP#
 * define a protocol for job state handling [red yellow-background]#TBD#
 * define the representation of dependencies and the notifications in practice [red yellow-background]#TBD#
 * verify the proposed requirements by an scheduler implementation sketch [red yellow-background]#TBD#


///////

Discussion
~~~~~~~~~~

Pros
^^^^
// add a fact list/enumeration which make this suitable:
//  * foo
//  * bar ...



Cons
^^^^
// fact list of the known/considered bad implications:



Alternatives
^^^^^^^^^^^^
//alternatives: explain alternatives and tell why they are not viable:

/////////


Rationale
---------
//rationale: Give a concise summary why it should be done *this* way:
[red yellow-background]#to be written#


//Conclusion
//----------
//conclusion: When approbate (this proposal becomes a Final)
//            write some conclusions about its process:




Comments
--------
//comments: append below


//endof_comments:

''''
Back to link:/documentation/devel/rfc.html[Lumiera Design Process overview]