first draft spec for the OutputSlot
This commit is contained in:
parent
a379476414
commit
ce6a917b59
2 changed files with 46 additions and 17 deletions
|
|
@ -34,6 +34,8 @@ extern "C" {
|
|||
|
||||
namespace mobject {
|
||||
|
||||
namespace mp = lumiera::typelist;
|
||||
|
||||
class MObject;
|
||||
|
||||
template<class MX>
|
||||
|
|
@ -99,8 +101,8 @@ namespace mobject {
|
|||
|
||||
private:
|
||||
enum {
|
||||
SPEC_SIZ = lumiera::typelist::maxSize<
|
||||
lumiera::typelist::Types<PID,lumiera_uid,uint>::List>::value
|
||||
SPEC_SIZ = mp::maxSize<
|
||||
mp::Types< PID, lumiera_uid, uint>::List>::value
|
||||
};
|
||||
typedef lib::OpaqueHolder<TargetSpec, SPEC_SIZ> SpecBuff;
|
||||
|
||||
|
|
|
|||
|
|
@ -3159,7 +3159,7 @@ The operation point is provided by the current BuilderMould and used by the [[pr
|
|||
This is possible because the operation point has been provided (by the mould) with informations about the media stream type to be wired, which, together with information accessible at the [[render node interface|ProcNode]] and from the [[referred processing assets|ProcAsset]], with the help of the [[connection manager|ConManager]] allows to figure out what's possible and how to do the desired connections. Additionally, in the course of deciding about possible connections, the PathManager is consulted to guide strategic decisions regarding the [[render node configuration|NodeConfiguration]], possible type conversions and the rendering technology to employ.
|
||||
</pre>
|
||||
</div>
|
||||
<div title="OutputDesignation" modifier="Ichthyostega" modified="201011190258" created="201006220126" tags="Model draft design discuss" changecount="50">
|
||||
<div title="OutputDesignation" modifier="Ichthyostega" modified="201106162303" created="201006220126" tags="Model draft design discuss" changecount="59">
|
||||
<pre>An ever recurring problem in the design of Luimiera's ~Proc-Layer is how to refer to output destinations, and how to organise them.
|
||||
Wiring the flexible interconnections between the [[pipes|Pipe]] should take into account both the StreamType and the specific usage context ([[scope|PlacementScope]]) -- and the challenge is to avoid hard-linking of connections and tangling with the specifics of the target to be addressed and connected. This page, started __6/2010__ by collecting observations to work out the relations, arrives at defining a //key abstraction// of output management.
|
||||
|
||||
|
|
@ -3183,21 +3183,21 @@ Wiring the flexible interconnections between the [[pipes|Pipe]] should take into
|
|||
* //indirect// means to derive the destination transitively (looking at the destination's output designation and so on)
|
||||
* //relative// in this context means that we refer to "the N^^th^^ of this kind" (e.g. the second video out)
|
||||
* we need to attach some metadata with an output; especially we need an associated StreamType
|
||||
* output designation is structured into several //levels://
|
||||
* the referral to an output designation can be observed on and is structured into several //levels://
|
||||
** within the body of the model, mostly we address output destinations relatively
|
||||
** at some point, we'll address a //subgroup// within the global pipes, which acts like a summation sink
|
||||
** there might be //master pipes//
|
||||
** finally, there is the hardware output or the distinct channel within the rendered result &mdash; never to be referred explicitly
|
||||
** there might be //master pipe(s),// collecting the output of various subgroups
|
||||
** finally, there is the hardware output or the distinct channel within the rendered result &mdash; never to be referenced explicitly
|
||||
!!!relation to Pipes
|
||||
in almost every case mentioned above, the output designation is identical with the starting point of a [[Pipe]]. This might be a global pipe or a ClipSourcePort. Thus it sounds reasonable to use pipe-~IDs directly as output designation. Pipes, as an accountable entity (=asset) just //leap into existence by being referred.// On the other hand, the //actual//&nbsp; pipe is a semantic concept, a specific structural arrangement of objects. Basically it means that some object acts as attachment point and thereby //claims//&nbsp; to be the entrance side of a pipe, while other processor objects chain up in sequence.
|
||||
!!!system outputs
|
||||
System level output connections seem to be an exception to the above rule. There is no pipe at an external port, and these externally visible connection points can appear and disappear, driven by external conditions. But the question is if system outputs shall be directly addressable at all as output designation? Generally speaking, Lumiera is not intended to be a system wide connection manager or a real time performance software. Thus it's advisable to refrain from direct referrals to system level connections from within the model. Rather, there should be a separate OutputManagement to handle external outputs and displays, both in windows or full screen. So these external outputs become rather a matter of application configuration &mdash; and for all the other purposes we're free to ''use pipe-~IDs as output designation''.
|
||||
System level output connections seem to be an exception to the above rule. There is no pipe at an external port, and these externally visible connection points can appear and disappear, driven by external conditions. Yet the question is if system outputs shall be directly addressable at all as output designation? Generally speaking, Lumiera is not intended to be a system wide connection manager or a real time performance software. Thus it's advisable to refrain from direct referrals to system level connections from within the model. Rather, there should be a separate OutputManagement to handle external outputs and displays, both in windows or full screen. So these external outputs become rather a matter of application configuration &mdash; and for all the other purposes we're free to ''use pipe-~IDs as output designation''.
|
||||
!!!consequences of mentioning an output designation
|
||||
The immediate consequence is that a [[Pipe]] with the given ID exists as an accountable entity. Only if &mdash; additionally &mdash; a suitable object within the model //claims to root this pipe,// a connection to this designation is wired, using an appropriate [[processing pattern|ProcPatt]]. A further consequence then is for the mentioned output designation to become a possible candidate for a ModelPort, an exit node of the built render nodes network. By default, only those designations without further outgoing connections actually become active model ports (but an existing and wired pipe exit node can be promoted to a model port explicitly).
|
||||
&rarr; OutputManagement
|
||||
|
||||
!!Challenge: mapping of output designations
|
||||
An entire sequence can be embedded within another sequence as a VirtualClip. While for a simple clip there is a Clip-~MObject placed into the model, holding an reference to a media asset, in case of a virtual clip an BindingMO takes on the role of the clip object. Note that BindingMO also acts as [[Timeline]] implementation &mdash; indeed the same sequence might be bound simultaneously as a timeline and into a virtual clip. This flexibility creates a special twist, as the contents of this sequence have no way of finding out if they're used as timeline or embedded virtual clip. So parts of this sequence might mention a OutputDesignation which, when using the sequence as virtual clip, needs to be translated into a virtual media channel, which can be treated in the same fashion as any channel (video, audio,...) found within real media. Especially, a new output designation has to be derived in a sensible way, which largely depends on how the original output designation has been specified.
|
||||
An entire sequence can be embedded within another sequence as a VirtualClip. While for a simple clip there is a Clip-~MObject placed into the model, holding an reference to a media asset, in case of a virtual clip an BindingMO takes on the role of the clip object. Note that, within another context, BindingMO also acts as [[Timeline]] implementation &mdash; indeed even the same sequence might be bound simultaneously as a timeline and into a virtual clip. This flexibility creates a special twist, as the contents of this sequence have no way of finding out if they're used as timeline or embedded virtual clip. So parts of this sequence might mention a OutputDesignation which, when using the sequence as virtual clip, needs to be translated into a virtual media channel, which can be treated in the same manner like any channel (video, audio,...) found within real media. Especially, a new output designation has to be derived in a sensible way, which largely depends on how the original output designation has been specified.
|
||||
&rarr; see OutputMapping regarding details and implementation of this mapping mechanism
|
||||
|
||||
|
||||
|
|
@ -3205,7 +3205,7 @@ An entire sequence can be embedded within another sequence as a VirtualClip. Whi
|
|||
|
||||
!Output designation definition
|
||||
OutputDesignation is a handle, denoting a target [[Pipe]] to connect.
|
||||
It exposes a function to resolve to a Pipe, and to retrieve the StreamType of that resolved output. It can be ''defined'' either explicitly by ~Pipe-ID, or by an indirect or relative specification. The later cases are resolved on demand only (which may be later and might change the meaning depending on the context). It's done this way intentionally to gain flexibility and avoid hard wiring (=explicit ~Pipe-ID)
|
||||
It exposes a function to resolve to a Pipe, and to retrieve the StreamType of that resolved output. It can be ''defined'' either explicitly by ~Pipe-ID, or by an indirect or relative specification. The later cases are resolved on demand only (which may be later and might even change the meaning, depending on the context). It's done this way intentionally to gain flexibility and avoid hard wiring (=explicit ~Pipe-ID)
|
||||
|
||||
!!!Implementation notes
|
||||
Thus the output designation needs to be a copyable value object, but opaque beyond that. Mandated by the various ways to specify an output designation, a hidden state arises regarding the partial resolution. The implementation solves that dilemma by relying on the [[State|http://en.wikipedia.org/wiki/State_pattern]] pattern in combination with an opaque in-place buffer.
|
||||
|
|
@ -3216,10 +3216,10 @@ While actually data frames are //pulled,// on a conceptual level data is assumed
|
|||
* first of all, we need to know //what to route// -- kind of the traits of the data. This is given by the //current pipe.//
|
||||
* then we'll need to determine an output designation //suitable for this data.// This is given by a "Plug" (WiringRequest) in the placement, and may be derived.
|
||||
* finally, this output designation will be //resolved// -- at least partially, resulting in a target pipe to be used for the wiring
|
||||
As both of these specifications are given by [[Pipe]]-~IDs, the actual designation information may be reduced. Much can be infered from the circumstances, because any pipe includes a StreamType, and an output designation for an incompatible stream type (e.g. and audio output when the pipe currently in question deals with video) is irrelevant.
|
||||
As both of these specifications are given by [[Pipe]]-~IDs, the actual designation information may be reduced. Much can be infered from the circumstances, because any pipe includes a StreamType, and an output designation for an incompatible stream type is irrelevant. (e.g. and audio output when the pipe currently in question deals with video)
|
||||
</pre>
|
||||
</div>
|
||||
<div title="OutputManagement" modifier="Ichthyostega" modified="201106130008" created="201007090155" tags="Model Rendering Player spec draft" changecount="21">
|
||||
<div title="OutputManagement" modifier="Ichthyostega" modified="201106162128" created="201007090155" tags="Model Rendering Player spec draft" changecount="23">
|
||||
<pre>//writing down some thoughts//
|
||||
|
||||
* ruled out the system outputs as OutputDesignation.
|
||||
|
|
@ -3236,9 +3236,9 @@ From the implementation side, the only interesting exit nodes are the ones to be
|
|||
* __playback__ always happens at a viewer element
|
||||
|
||||
!Attaching and mapping of exit nodes
|
||||
[[Output designations|OutputDesignation]] are created by using a [[Pipe]]-ID and &mdash; at the same time &mdash; by some object //claiming to root this pipe.// The applicability of this pattern is figured out dynamically while building the render network, resulting in a collection of model ports as part of the current [[Fixture]]. A RenderProcess can be started to pull from these active exit points of a given timeline. Besides, when the timeline enclosing these model ports is [[connected to a viewer|ViewerPlayConnection]], an [[output network|OutputNetwork]] //is built to allow hooking exit points to the viewer component.// Both cases encompass a mapping of exit nodes to actual output channels. Usually, this mapping relies on relative addressing of the output sinks, starting to allocate connections with the "first of each kind".
|
||||
[[Output designations|OutputDesignation]] are created using a [[Pipe]]-ID and &mdash; they become real by some object //claiming to root this pipe.// The applicability of this pattern is figured out dynamically while building the render network, resulting in a collection of model ports as part of the current [[Fixture]]. A RenderProcess can be started to pull from these active exit points of a given timeline. Besides, when the timeline enclosing these model ports is [[connected to a viewer|ViewerPlayConnection]], an [[output network|OutputNetwork]] //is built to allow hooking exit points to the viewer component.// Both cases encompass a mapping of exit nodes to actual output channels. Usually, this mapping relies on relative addressing of the output sinks, starting to allocate connections with the "first of each kind".
|
||||
|
||||
We should note that in both cases this mapping operation is controlled and driven by the output side of the connection: A viewer has fixed output capabilities, and rendering targets a specific container format, again with fixed and pre-settled channel configuration (when configurting a render process, it might be necessary to account for //possible kinds of output streams,// so to provide a sensible pre-selection of possible output container formats for the user to select from). Thus, as a starting point, we'll create a default configured mapping, assigning channels in order. This mapping then should be exposed to modification and tweaking by the user. For rendering, this is part of the render options dialog, while in case of a viwer connection, a switch board is created to allow modifying the default mapping.
|
||||
We should note that in both cases this [[mapping operation|OutputMapping]] is controlled and driven by the output side of the connection: A viewer has fixed output capabilities, and rendering targets a specific container format, again with fixed and pre-settled channel configuration (when configurting a render process, it might be necessary to account for //possible kinds of output streams,// so to provide a sensible pre-selection of possible output container formats for the user to select from). Thus, as a starting point, we'll create a default configured mapping, assigning channels in order. This mapping then should be exposed to modification and tweaking by the user. For rendering, this is part of the render options dialog, while in case of a viwer connection, a switch board is created to allow modifying the default mapping.
|
||||
|
||||
!Connection to external outputs
|
||||
External output destinations are never addressed directly from within the model. This is an design decision. Rather, model parts connect to an OutputDesignation, and these in turn may be [[connected to a viewer element|ViewerPlayConnection]]. At this point, related to the viewer element, there is a mapping to external destination(s): for images, a viewer typically has an implicit, natural destination (read, there is a corresponding viewer window or widget), while for sound we use an mapping rule, which could be overridden locally in the viewer.
|
||||
|
|
@ -3250,14 +3250,14 @@ Any external output sink is managed as a [[slot|DisplayerSlot]] in the ~OutputMa
|
|||
While mostly the model just denotes the destination for output as OutputDesignation, at some point we need to map these abstract designations to real output capabilities. This OutputManager interface exposes mapping and the ability to control and manage it. Several elements within the application, most notably the [[viewers|ViewerAsset]], provide an implementation of this interface -- yet there is one primary implementation, the ''global output manager''. It can be accessed through the {{{Output}}} façade interface and is the final authority when it comes to allocating an mapping of real output possibilities.
|
||||
</pre>
|
||||
</div>
|
||||
<div title="OutputManager" modifier="Ichthyostega" modified="201106130000" created="201106122359" tags="Player Model def" changecount="2">
|
||||
<pre>The term &raquo;''Output Manager''&laquo; might deonte two things: first, there is an {{{proc::play::OutputManager}}} interface, which can be exposed by several components within the application, most notably the [[viewer elements|ViewerAsset]]. And then, there is "the" global OutputManager, which finally tracks all registered OutputSlot elements and thus is the gatekeeper for any output leaving the application.
|
||||
<div title="OutputManager" modifier="Ichthyostega" modified="201106162346" created="201106122359" tags="Player Model def" changecount="3">
|
||||
<pre>The term &raquo;''Output Manager''&laquo; might denote two things: first, there is an {{{proc::play::OutputManager}}} interface, which can be exposed by several components within the application, most notably the [[viewer elements|ViewerAsset]]. And then, there is "the" global output manager, which finally tracks all registered OutputSlot elements and thus is the gatekeeper for any output leaving the application.
|
||||
|
||||
&rarr; see [[output management overview|OutputManagement]]
|
||||
&rarr; see OutputSlot
|
||||
&rarr; see ViewerPlayConnection</pre>
|
||||
</div>
|
||||
<div title="OutputMapping" modifier="Ichthyostega" modified="201105221814" created="201011080238" tags="Model spec draft" changecount="25">
|
||||
<div title="OutputMapping" modifier="Ichthyostega" modified="201106162204" created="201011080238" tags="Model spec draft" changecount="28">
|
||||
<pre>An output mapping serves to //resolve//&nbsp; [[output designations|OutputDesignation]].
|
||||
|
||||
!Mapping situations
|
||||
|
|
@ -3284,9 +3284,36 @@ All these mapping steps are listed here, because they exhibit a common pattern.
|
|||
* there is an //unconnected//&nbsp; state.
|
||||
|
||||
!Implementation notes
|
||||
Thus the mapping is a copyable value object, based on a associative array. It may be attached to a model object and persisted alongside. The mapping is assumed to run a defaults query when necessary. To allow for that, it should be configured with a query template (string). Frequently, special //default pipe// markers will be used at places, where no distinct pipe-ID is explicitly specified. Besides that, invocations might supply additional predicates (e.g. {{{ord(2)}}} to denote "the second stream of this kind") to hint the defaults resolution. Moreover, the mapping needs a way to retrieve the set of possible results, allowing to filter the results of the rules based default. Mappings might be defined explicitly. Instead of storing a //bottom value,// an {{{isDefined()}}} predicate might be preferable.
|
||||
Thus the mapping is a copyable value object, based on a associative array. It may be attached to a model object and persisted alongside. The mapping is assumed to run a defaults query when necessary. To allow for that, it should be configured with a query template (string). Frequently, special //default pipe// markers will be used at places where no distinct pipe-ID is specified explicitly. Besides that, invocations might supply additional predicates (e.g. {{{ord(2)}}} to point at "the second stream of this kind") thereby hinting the defaults resolution. Moreover, the mapping needs a way to retrieve the set of possible results, allowing to filter the results of the rules based default. Mappings might be defined explicitly. Instead of storing a //bottom value,// an {{{isDefined()}}} predicate might be preferable.
|
||||
|
||||
First and foremost, mapping can be seen as a //functional abstraction.// As it's used at implementation level, encapsulation of detail types in't the primary concern, so it's a candidate for generic programming: For each of those use cases outlined above, a distinct mapping type is created by instantiating the {{{OutputMapping<DEF>}}} template with a specifically tailored definition context ({{{DEF}}}), which takes on the role of a strategy. Individual instances of this concrete mapping type may be default created and copied freely. This instantiation process includes picking up the concrete result type and building a functor object for resolving on the fly. Thus, in the way typical for generic programming, the more involved special details are moved out of sight, while being still in scope for the purpose of inlining. But there //is// a concern better to be encapsulated and concealed at the usage site, namely accessing the rules system. Thus mapping leads itself to the frequently used implementation pattern where there is a generic frontend as header, calling into opaque functions embedded within a separate compilation unit.
|
||||
</pre>
|
||||
</div>
|
||||
<div title="OutputSlot" modifier="Ichthyostega" modified="201106170206" created="201106162339" tags="def Concepts Player spec" changecount="4">
|
||||
<pre>Within the Lumiera player and output subsystem, actually sending data to an external output requires to allocate an ''output slot''
|
||||
This is the central metaphor for the organisation of actual (system level) outputs; using this concept allows to separate and abstract the data calculation and the organisation of playback and rendering from the specifics of the actual output sink. Actual output possibilities can be added and removed dynamically from various components (backend, GUI), all using the same resolution and mapping mechanisms (&rarr; OutputManagement)
|
||||
|
||||
!Properties of an output slot
|
||||
Each OutputSlot is an unique and distinguishable entity. It corresponds explicitly to an external output, output file or similar capability accepting media content. First off, an output slot needs to be provided, configured and registered, using an implementation for the kind of media data to be output (sound, video) and the special circumstances of the output capability (render a file, display video in a GUI widget, send video to a full screen display, establish a Jack port, just use some kind of "sound out"). In some cases, this explicit registration may be extened to a //factory for additional output slots// -- to be used on demand. An output slot is always limited to a single kind of media, and to a single connection unit, but this connection may still be comprised of multiple channels (stereoscopic video, multichannel sound).
|
||||
|
||||
In order to be usable as //output sink,// an output slot needs to be allocated and locked. At any time, there may be only a single client using a given output slot this way. To stress this point: output slots don't provide any kind of inherent mixing capability; any adaptation, mixing, overlaying and sharing needs to be done within the nodes network producing the output data fed to the slot. (yet some special kinds of external output capabilities -- e.g. the Jack audio connection system -- may still provide additional mixing capabilities, but that's beyond the scope of the Lumiera application)
|
||||
|
||||
Once allocated, the output slot returns a set of concrete ''sink handles'' (one for each physical channel expecting data). The size and other characteristics of the data frames is assumed to be suitable. Typically this won't be verified at that level anymore (but the sink handle provides a hook for assertions). Besides that, the allocation of an output slot reveals detailed ''timing expectations''. The client is required to comply to these timings when ''emitting'' data -- he's even required to provide a //current time specification,// alongside with the data. Yet the output slot has the ability to handle timing failures gracefully; the concrete output slot implementation is expected to provide some kind of de-click or de-flicker facility, which kicks in automatically when a timing failure is detected.
|
||||
|
||||
!!!data exchange models
|
||||
Data is handed over by the client invoking an {{{emit(time,...)}}} function on the sink handle. There are two different models how this data hand-over might be performed. On allocation of a slot, the client has to commit to one of them, allowing the output slot to adapt accordingly
|
||||
;buffer handover model
|
||||
:the client owns the data buffer and cares for allocation and de-allocation. The {{{emit()}}}-call just propagates a pointer to the buffer holding the data ready for output. The output slot implementation in turn has the liability to copy or otherwise use this data within a given time limit.
|
||||
;shared buffer model
|
||||
:here the output mechanism owns the buffer. Within a certain time window prior to the expected time of the {{{emit()}}}-call, the client may obtain this buffer (pointer) to fill in the data. The slot implementation won't touch this buffer until the {{{emit()}}} handover, which in this case just provides the time and states that the client is done with that buffer. If the data emitting handshake doesn't happen at all, it counts as late and superseded by the next handshake.
|
||||
|
||||
!!!timing expectations
|
||||
Besides the sink handles, allocation of an output slot defines some timing constraints, which are binding for the client. These timings are detailed and explicit, including a grid of deadlines for each frame to deliver, plus a fixed //latency.// Within this context, latency means the requirement to be ahead of the nominal time by a certain amount, to compensate for the processing time necessary to propagate the media to the physical output pin. The output slot implementation itself is bound by external constraints to deliver data at a fixed framerate and aligned to an externally defined timing grid, plus the data needs to be handed over ahead of these time points by an time amount given by the latency. Depending on the data exchange model, there is an additional time window limiting the buffer management.
|
||||
|
||||
The assumption is for the client to have elaborate timing capabilities at his disposal. More specifically, the client is a job running within the engine scheduler and thus can be configured to run within certain limits. Thus the client is able to provide a //current nominal time// -- which is suitably close to the actual wall clock time. The output slot implementation can be written such as to work out from this time specification if the call is timely or overdue -- and react accordingly.
|
||||
|
||||
{{red{TODO 6/11}}}in this spec, both data exchange models exhibit a weakness regarding the releasing of buffers. At which time is it safe to release a buffer, when the handover didn't happen? Do we need an explicit callback, and how could this callback be triggered? This is similar to the problem of closing a network connection, i.e. the problem is generally unsolvable, but can be handled pragmatically within certain limits.
|
||||
|
||||
</pre>
|
||||
</div>
|
||||
<div title="Overview" modifier="Ichthyostega" modified="200906071810" created="200706190300" tags="overview img" changecount="13">
|
||||
|
|
|
|||
Loading…
Reference in a new issue