lumiera_/doc/devel/rfc/EngineInterfaceOverview.txt

Engine Interface Overview
=========================
:Date: 2010

// please don't remove the //word: comments

[grid="all"]
`------------`-----------------------
*State*         _Draft_
*Date*          _2010-04-16_
*Proposed by*   link:Ichthyostega[]
-------------------------------------


Overview Engine Interface(s)
----------------------------

********************************************************************************
At the Engine Interfaces, Lumiera's Backend and Session get connected and work
together to produce rendered output. This design proposal intends to give an
overview of the connection points and facilities involved, to define some terms
and concepts and to provide a foundation for discussion and working out the
APIs in detail.
********************************************************************************



Participants
~~~~~~~~~~~~
 *Render Process*:: represents an ongoing calculation as a whole
 *Engine Model*:: encloses the details of the current engine configuration and
  wiring
 *Dispatcher*:: translates a render process into the (planned) invocation of
  individual nodes
 *Scheduler*:: cares for calculations actually to happen, in the right order
  and just in time, if at all
 *Node*:: abstraction of an processing unit, supports planning by the
  dispatcher, allows to pull data, thereby driving the actual calculation.

Render Process
~~~~~~~~~~~~~~
The render process brackets an ongoing calculation as a whole. It is not to be
confused with a operating system process or thread; rather it is a point of
reference for the relevant entities in the GUI and Steam-Layer in need to
connect to such a "rendering", and it holds the specific definitions for this
calculation series. A render process
_corresponds to a single data stream_ to be rendered. Thus, when the play
 controller of some timeline in the model is
in _playing_ or _paused_ state, typically multiple corresponding render
processes exist.

* there is an displayer- or output slot, which got allocated on creation
  of the process
* the process disposes calculated data frames "into" this slot
* the process can be paused/started and stopped (aborted, halted).
* some processes allow for changing parameters dynamically (e.g. speed,
  direction)
* each process has to ensure that the output/display slot gets closed or
  released finally

.Process parameters
A process is linked to a single stream data format (a -> 
link:StreamTypeSystem.html[stream implementation type]). +
It is configured with _frame quantisation_ and _timings_, and a _model port_
identifier and _channel selector_.

 quantisation::
   translates time values into frame numbers. (In the most general
   case this is a function, connected to the session)
   
 timings::
   a definition to translate global model time units in real clock time,
   including _alignment_ to an external _time grid_.
   
 model port::
   a point in the (high level) model where output can be produced. +
   This might be a global pipe in one of the model's timelines, or
   it might be a _probe point_.
   
 channel::
   within the session and high level model, details of the stream
   implementation are abstracted. Typically, a global pipe (master bus
   or subgroup) corresponds to a multichannel stream, and each of these
   channels might be hooked up to an individual render process
   (we have to work out if that's _always the case_ or just under
   _some circumstances_)


[NOTE]
===================
While certainly the port and channel definition is fixed, unfortunately the
quantisation and the timings are'nt. The timings may be changed in the middle
of an ongoing render process, due to changed playback speed, shuffling or
requirements forwarded from chase-and-lock synchronisation to an external
source. We still need to discuss if Lumiera is going to support variable
framerates (several media professionals I've talked to were rather positive we
need to support that -- personally I'm still in doubt we do). Variable
framerates force us to determine the frame numbers by an integration over time
from a start position up to the time position in question. The relevant data to
be integrated is located in the session / high-level model; probably we'll then
create an excerpt of this data, but still the less quantisation will be a
function of time. Anyway, it is the render processes job to translate all kinds
of parameter changes into relevant internal API calls to reconfigure the
calculation process to fit.
===================



Engine Model (low-level Model)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The low level model is a network of interconnected render nodes. It is created
by the build process to embody any configuration, setup and further
parametrisation derived from the high-level description within the session. But
the data structure of this node network is _opaque_ and considered an
implementation detail. It is not intended to be inspected and processed by
outward entities (contrast this to the high-level model within the session,
which provides an extensive discovery API and can be manipulated by model
mutating commands). We just provide a set of _query and information retrieval
functions_ to suit the needs of the calculation process. The engine model is
_not persisted._

* the engine model is partitioned by a _segmentation_ of the time axis.
  Individual segments can be hot-swapped.
* the engine has _exit nodes,_ corresponding to the model ports mentioned above
* each exit node provides a stream type definition plus quantisation and
  alignment constraints.

Thus, for any pair (port, time) it is possible to figure out a segment and an
exit node to serve this position. The segmentation(s) for multiple ports might
differ. To allow for effective dispatching, the model should provide
convenience functions to translate these informations into frame number ranges.
The mentioned quantisation and alignment constraints stem from the fact that
the underlying media source(s) are typically themselves quantised and the
timings might be manipulated within the processing chain. We might or might not
be able to shift the underlying media source
(it might be a live input or it might be tied to a fixed timecode)



Processing Node
~~~~~~~~~~~~~~~
In this context, a node is a conceptual entity: it is an elementary unit of
processing. It might indeed be a single invocation of a _processor_ (plugin or
similar processing function), or it might be a chain of nodes, a complete
subtree, it might _represent_ a data source (file, external input or peer in
case of distributed rendering), or it might stand for a pipeline implemented in
hardware. The actual decision about these possibilities happened during the
build process and can be configured by rules. Information about these decisions
is retained only insofar it is required for the processing, most of the
detailed type information is discarded after the wiring and configuration step.
As mentioned above, each node serves two distinct purposes, namely to assist
with the planning and dispatching, and to pull data by performing the
calculations.

Nodes can be considered _stateless_ -- pulling a node has no effect outside the
invocation context. While a node _might_ actually be configured to drive a
whole chain or subtree and propagate the pull request
_within_ this tree or chain internally, the node _never propagates a pull
 request beyond its realm._ The pull()
call expects to be provided with all prerequisite data, intermediary and output
buffers.


Dispatching Step
~~~~~~~~~~~~~~~~
The dispatcher translates a render process (actually a _calculation stream_ as
part of a render process) into sequences of node invocations,
which then can be analysed further (including planning the invocation of
prerequisites) and scheduled. This mapping is assisted by the engine model API
(to find the right exit node in the right segment), the render process (for
quantisation) and the involved node's invocation API (to find the
prerequisites)


Node Invocation API
~~~~~~~~~~~~~~~~~~~
As nodes are stateless, they need to be embedded into an invocation context in
order to be of any use. The node invocation has two distinct stages and thus
the invocation API can be partitioned in two groups

Planning
^^^^^^^^
During the planning phase, the dispatcher retrieves various informations
necessary to _schedule_ the following pull call. These informations include

 * reproducible invocation identifier, usable to label frames for caching
 * opaque source identifier (owned by the backed) in case this node
   represents a source
 * prerequisite nodes
 * index (channel) of the prerequisite's output to be fed as input buffer(s)
 * number and size of the output buffers required
 * additional memory required
 * control data frame(s)


Node pull
^^^^^^^^^
 * the pull call expects to be provided with all the resources announced during
   the planning step
 * moreover, the pull call needs to know (or some way to figure out) the time
   coordinates
 * after retrieving automation, the control flow forwards to the actual
   processing function
 * there is an result/error code (assuming the scheduler prefers error codes
   over exceptions)


''''

Tasks
~~~~~
 * find out if we need to support variable framerate
   ([green]#->  yes, implementation deferred#)
 * find out about the exact handling of multichannel data streams ([green]#✔ done#)
 * design and prototypical implementation of frame quantisation ([green]#✔ done#)
 * design a buffer descriptor ([green]#✔ done#)
 * design a buffer designation scheme [red]#TODO#
 * expand on the node identification scheme [red]#TODO#
 * clarify how control data frames can be addressed [red]#TODO#


Discussion
~~~~~~~~~~

Pros/Cons/Alternatives
^^^^^^^^^^^^^^^^^^^^^^
Currently we're focussing on how to implement _this_ concept, not on
evaluating alternatives. Especially the idea of scheduling individual frame jobs
is a core concept of Lumiera. This RfC tries to bridge from the session model to
an engine based on these concepts. It's the attempt to link two concepts already
defined and decided on....


Rationale
^^^^^^^^^
* allow for optimal resource use and avoid blocking of threads
* shift away complexity from the engine into the builder, which is by far not
  so performance critical
* allow to adjust the actual behaviour of the engine in a wide range, based on
  actual measurements
* create a code structure able to support the foreseeable extensions (hardware
  and distributed rendering) without killing maintainability




//Conclusion
//----------
//conclusion: When approbate (this proposal becomes a Final)
//            write some conclusions about its process:




Comments
--------
//comments: append below

.State -> Draft
Requirements and details of the design are sufficiently clear meanwhile.
Ther seems to be not much room for alternative approaches, given our
general planning for the application

    Mi 11 Mai 2011 19:27:12 CEST Ichthyostega <prg@ichthyostega.de>


//endof_comments:

''''
Back to link:/documentation/devel/rfc.html[Lumiera Design Process overview]