benn/lumiera_
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Invocation: painstaking analysis of calculation requirements

Browse source 
...the complexity of details is a nightmare
...still fighting to grasp a generic structure allowing to ''fold down''
   the details into the specific ''domain ontologies'' for the media libraries
This commit is contained in:
Fischlurch 2024-07-02 23:33:23 +02:00
parent 8c536fc637
commit 604c4b580b
4 changed files with 1217 additions and 113 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

25
src/steam/engine/node-wiring-builder.hpp
View file

@ -45,11 +45,14 @@
#include "steam/engine/proc-node.hpp"
#include "lib/nocopy.hpp"
#include <utility>
namespace steam {
namespace engine {
using std::move;
/////////////////////////////////////////////////////////////////////////////////////////////////////////////TICKET #1367 : Rebuild the Node Invocation
class NodeWiringBuilder
@ -57,6 +60,28 @@ namespace engine {
{
public:
NodeWiringBuilder
inSlots (uint s)
{
UNIMPLEMENTED ("define number of predecessor-source slots");
return move(*this);
}
NodeWiringBuilder
outSlots (uint r)
{
UNIMPLEMENTED ("define number of result slots");
return move(*this);
}
template<class ILA, typename...ARGS>
NodeWiringBuilder
createBuffers (ARGS&& ...args)
{
UNIMPLEMENTED ("define builder for all buffers to use");
return move(*this);
}
/****************************************************//**
* Terminal: complete the Connectivity defined thus far.
*/

2
src/steam/streamtype.hpp
View file

@ -67,7 +67,7 @@ namespace steam {
RAW,
SOURCE,
TARGET,
INTERMEDIARY
TRANSIENT
};
struct Prototype;

48
wiki/renderengine.html
View file

@ -1250,20 +1250,20 @@ Beyond that, it can be necessary to associate at least a state flag with //indiv
__Note__: while the API to access this service is uniform, conceptually there is a difference between just using the (shared) type information and associating individual metadata, like the buffer state. Type-~IDs, once allocated, will never be discarded (within the lifetime of an Lumiera application instance -- buffer associations aren't persistent). To the contrary, individual metadata //will be discarded,// when releasing the corresponding buffer. According to the ''prototype pattern'', individual metadata is treated as a one-way-off specialisation.
</pre>
</div>
<div title="BufferProvider" modifier="Ichthyostega" created="201107082330" modified="202406241656" tags="Rendering spec draft" changecount="3">
<div title="BufferProvider" modifier="Ichthyostega" created="201107082330" modified="202406282147" tags="Rendering spec draft" changecount="6">
<pre>It turns out that -- throughout the render engine implementation -- we never need direct access to the buffers holding actual media data. Buffers are just some entity to be //managed,// i.e. &quot;allocated&quot;, &quot;locked&quot; and &quot;released&quot;; the //actual meaning of these operations can be left to the implementation.// The code within the render engine just pushes around ''smart-prt like handles''. These [[buffer handles|BuffHandle]] act as a front-end, being created by and linked to a buffer provider implementation. There is no need to manage the lifecycle of buffers automatically, because the use of buffers is embedded into the render calculation cycle, which follows a rather strict protocol anyway. Relying on the [[capabilities of the scheduler|SchedulerRequirements]], the sequence of individual jobs in the engine ensures...
* that the availability of a buffer was ensured prior to planning a job (&quot;buffer allocation&quot;)
* that a buffer handle was obtained (&quot;locked&quot;) prior to any operation requiring a buffer
* that buffers are marked as free (&quot;released&quot;) after doing the actual calculations.
!operations
While BufferProvider is an interface meant to be backed by various different kinds of buffer and memory management approaches, there is a common set of operations to be supported by any of them
While BufferProvider is an interface meant to be backed by various and diverse kinds of buffer and memory management approaches, there is a common set of operations to be supported by any of them
;announcing
:client code may announce beforehand that it expects to get a certain amount of buffers. Usually this causes some allocations to happen right away, or it might trigger similar mechanisms to ensure availability; the BufferProvider will then return the actual number of buffers guaranteed to be available. This announcing step is optional an can happen any time before or even after using the buffers and it can be repeated with different values to adjust to changing requirements. Thus the announced amount of buffers always denotes //additional buffers,// on top of what is actively used at the moment. This safety margin of available buffers usually is accounted separately for each distinct kind of buffer (buffer type). There is no tracking as to which specific client requested buffers, beyond the buffer type.
;locking
:this operation actually makes a buffer available for a specific client and returns a [[buffer handle|BuffHandle]]. The corresponding buffer is marked as used and can't be locked again unless released. If necessary, at that point the BufferProvider might allocate memory to accommodate (especially when the buffers weren't announced beforehand). The locking may fail and raise an exception. You may expect failure to be unlikely when buffers have been //announced beforehand.// To support additional sanity checks, the client may provide a token-ID with the lock-operation. This token may be retrieved later and it may be used to ensure the buffer is actually locked for //this token.//
:this operation actually makes a buffer available for a specific client and returns a [[buffer handle|BuffHandle]]. The corresponding buffer is marked as used and can't be locked again unless released. If necessary, at that point the BufferProvider might allocate memory to accommodate (especially when the buffers weren't announced beforehand). The locking may fail and raise an exception. Such a failure will be unlikely when buffers have been //announced beforehand.// To support additional sanity checks, the client may provide a token-ID with the lock-operation. This token may be retrieved later and it may be used to ensure the buffer is actually locked for //this token.//
;attaching
:optionally the client may attach an object to a locked buffer. This object is placement-constructed into the buffer and will be destroyed automatically when releasing the buffer. Alternatively, the client may provide a pair of constructor- / destructor-functors, to be invoked in a similar way. This allows e.g. to install descriptor structures within the buffer, as required by an external library.
:optionally the client may attach an object to a locked buffer. This object is placement-constructed into the buffer and will be destroyed automatically when releasing the buffer. Alternatively, the client may provide a pair of constructor- / destructor-functors, to be invoked in a similar way. This allows e.g. to install descriptor structures within the buffer, as required by an external media handling library.
;emitting
:the client //may optionally mark a state transition// -- whose precise meaning remains implicit and implementation dependent. From the client's perspective, emitting and releasing may seem equivalent, since the buffer should not be touched after that point. However, conceivably there are usages where it matters for //the consumer// to be sure an expected result was actually achieved, since the producer may well acquire the buffer and then fail to complete the required work, prompting some clean-up safety mechanism to merely release the resources.
;releasing
@ -1279,8 +1279,10 @@ __see also__
&amp;rarr; RenderMechanics for details on the buffer management within the node invocation for a single render step
</pre>
</div>
<div title="BufferTable" modifier="Ichthyostega" created="201109172253" modified="201201192303" tags="def spec Rendering draft">
<pre>The invocation of individual [[render nodes|ProcNode]] uses an ''buffer table'' internal helper data structure to encapsulate technical details of the allocation, use, re-use and feeing of data buffers for the media calculations. Here, the management of the physical data buffers is delegated through a BufferProvider, which typically is implemented relying on the ''frame cache'' in the Vault. Yet some partially quite involved technical details need to be settled for each invocation: We need input buffers, maybe provided as external input, while in other cases to be filled by a recursive call. We need storage to prepare the (possibly automated) parameters, and finally we need a set of output buffers. All of these buffers and parameters need to be rearranged for invoking the (external) processing function, followed by releasing the input buffers and commiting the output buffers to be used as result.
<div title="BufferTable" modifier="Ichthyostega" created="201109172253" modified="202407030218" tags="def spec Rendering draft" changecount="1">
<pre>{{red{⚠ In-depth rework underway as of 7/2024...}}}
^^┅┅┅┅┅┅the following text is ''superseded''┅┅┅┅┅┅┅┅┅^^
The invocation of individual [[render nodes|ProcNode]] uses an ''buffer table'' internal helper data structure to encapsulate technical details of the allocation, use, re-use and feeing of data buffers for the media calculations. Here, the management of the physical data buffers is delegated through a BufferProvider, which typically is implemented relying on the ''frame cache'' in the Vault. Yet some partially quite involved technical details need to be settled for each invocation: We need input buffers, maybe provided as external input, while in other cases to be filled by a recursive call. We need storage to prepare the (possibly automated) parameters, and finally we need a set of output buffers. All of these buffers and parameters need to be rearranged for invoking the (external) processing function, followed by releasing the input buffers and commiting the output buffers to be used as result.
Because there are several flavours of node wiring, the building blocks comprising such a node invocation will be combined depending on the circumstances. Performing all these various steps is indeed the core concern of the render node -- with the help of BufferTable to deal with the repetitive, tedious and technical details.
@ -4808,8 +4810,10 @@ Moreover, the design of coordinate matching and resolving incurs a structure sim
In the most general case the render network may be just a DAG (not just a tree). Especially, multiple exit points may lead down to the same node, and following each of this possible paths the node may be at a different depth on each. This rules out a simple counter starting from the exit level, leaving us with the possibility of either employing a rather convoluted addressing scheme or using arbitrary ID numbers.{{red{...which is what we do for now}}}
</pre>
</div>
<div title="NodeOperationProtocol" modifier="Ichthyostega" created="200806010251" modified="201308100351" tags="Rendering operational" changecount="1">
<pre>The [[nodes|ProcNode]] are wired to form a &quot;Directed Acyclic Graph&quot;; each node knows its predecessor(s), but not its successor(s). The RenderProcess is organized according to the ''pull principle'', thus we find an operation {{{pull()}}} at the core of this process. Meaning that there isn't a central entity to invoke nodes consecutively. Rather, the nodes themselves contain the detailed knowledge regarding prerequisites, so the calculation plan is worked out recursively. Yet still there are some prerequisite resources to be made available for any calculation to happen. So the actual calculation is broken down into atomic chunks of work, resulting in a 2-phase invocation whenever &quot;pulling&quot; a node. For this to work, we need the nodes to adhere to a specific protocol:
<div title="NodeOperationProtocol" modifier="Ichthyostega" created="200806010251" modified="202407030216" tags="Rendering operational" changecount="2">
<pre>{{red{⚠ In-depth rework underway as of 7/2024...}}}
^^┅┅┅┅┅┅the following text is ''superseded''┅┅┅┅┅┅┅┅┅^^
The [[nodes|ProcNode]] are wired to form a &quot;Directed Acyclic Graph&quot;; each node knows its predecessor(s), but not its successor(s). The RenderProcess is organized according to the ''pull principle'', thus we find an operation {{{pull()}}} at the core of this process. Meaning that there isn't a central entity to invoke nodes consecutively. Rather, the nodes themselves contain the detailed knowledge regarding prerequisites, so the calculation plan is worked out recursively. Yet still there are some prerequisite resources to be made available for any calculation to happen. So the actual calculation is broken down into atomic chunks of work, resulting in a 2-phase invocation whenever &quot;pulling&quot; a node. For this to work, we need the nodes to adhere to a specific protocol:
;planning phase
:when a node invocation is foreseeable to be required for getting a specific frame for a specific nominal and actual time, the engine has to find out the actual operations to happen
:# the planning is initiated by issuing an &quot;get me output&quot; request, finally resulting in a JobTicket
@ -6387,8 +6391,10 @@ In 2018, the middle Layer was renamed into &amp;rarr; Steam-Layer
</pre>
</div>
<div title="ProcNode" modifier="Ichthyostega" created="200706220409" modified="200806211606" tags="def spec">
<div title="ProcNode" modifier="Ichthyostega" created="200706220409" modified="202407030215" tags="def spec" changecount="5">
<pre>A data processing node within the Render Engine. Its key feature is the possibility to pull from it one (freely addressable) [[Frame]] of calculated data. Further, each ~ProcNode has the ability to be wired with other nodes and [[Parameter Providers|ParamProvider]]
{{red{⚠ In-depth rework underway as of 7/2024...}}}
^^┅┅┅┅┅┅the following text is ''superseded''┅┅┅┅┅┅┅┅┅^^
!! {{red{open questions}}}
* how to address a node
@ -6939,8 +6945,10 @@ Currently the Render/Playback is beeing targetted for implementation; almost eve
[img[Entities comprising the Render Engine|uml/fig128389.png]]
</pre>
</div>
<div title="RenderImplDetails" modifier="Ichthyostega" created="200806220211" modified="201109151533" tags="Rendering impl img">
<pre>Below are some notes regarding details of the actual implementation of the render process and processing node operation. In the description of the [[render node operation protocol|NodeOperationProtocol]] and the [[mechanics of the render process|RenderMechanics]], these details were left out deliberately.
<div title="RenderImplDetails" modifier="Ichthyostega" created="200806220211" modified="202407030217" tags="Rendering impl img" changecount="1">
<pre>{{red{⚠ In-depth rework underway as of 7/2024...}}}
^^┅┅┅┅┅┅the following text is ''superseded''┅┅┅┅┅┅┅┅┅^^
Below are some notes regarding details of the actual implementation of the render process and processing node operation. In the description of the [[render node operation protocol|NodeOperationProtocol]] and the [[mechanics of the render process|RenderMechanics]], these details were left out deliberately.
{{red{WIP as of 9/11 -- need to mention the planning phase more explicitly}}}
!Layered structure of State
@ -7002,8 +7010,10 @@ Prerequisite data for the media calculations can be considered just part of that
* all it needs to know is the ''effective nominal time'' and an ''invocation instance ID''
</pre>
</div>
<div title="RenderMechanics" modifier="Ichthyostega" created="200806030230" modified="201109172221" tags="Rendering operational impl img">
<pre>While the render process, with respect to the dependencies, the builder and the processing function is sufficiently characterized by referring to the ''pull principle'' and by defining a [[protocol|NodeOperationProtocol]] each node has to adhere to &amp;mdash; for actually get it coded we have to care for some important details, especially //how to manage the buffers.// It may well be that the length of the code path necessary to invoke the individual processing functions is finally not so important, compared with the time spent at the inner pixel loop within these functions. But my guess is (as of 5/08), that the overall number of data moving and copying operations //will be//&amp;nbsp; of importance.
<div title="RenderMechanics" modifier="Ichthyostega" created="200806030230" modified="202407030217" tags="Rendering operational impl img" changecount="1">
<pre>{{red{⚠ In-depth rework underway as of 7/2024...}}}
^^┅┅┅┅┅┅the following text is ''superseded''┅┅┅┅┅┅┅┅┅^^
While the render process, with respect to the dependencies, the builder and the processing function is sufficiently characterized by referring to the ''pull principle'' and by defining a [[protocol|NodeOperationProtocol]] each node has to adhere to &amp;mdash; for actually get it coded we have to care for some important details, especially //how to manage the buffers.// It may well be that the length of the code path necessary to invoke the individual processing functions is finally not so important, compared with the time spent at the inner pixel loop within these functions. But my guess is (as of 5/08), that the overall number of data moving and copying operations //will be//&amp;nbsp; of importance.
{{red{WIP as of 9/11 -- need to mention the planning phase more explicitly}}}
!requirements
@ -7144,8 +7154,10 @@ An Activity is //performed// by invoking its {{{activate(now, ctx)}}} function -
In a similar vein, also ''dependency notifications'' need to happen decoupled from the activity chain from which they originate; thus the Post-mechanism is also used for dispatching notifications. Yet notifications are to be treated specially, since they are directed towards a receiver, which in the standard case is a {{{GATE}}}-Activity and will respond by //decrementing its internal latch.// Consequently, notifications will be sent through the ''λ-post'' -- which operationally re-schedules a continuation as a follow-up job. Receiving such a notification may cause the Gate to become opened; in this case the trigger leads to //activation of the chain// hooked behind the Gate, which at some point typically enters into another calculation job. Otherwise, if the latch (in the Gate) is already zero (or the deadline has passed), nothing happens. Thus the implementation of state transition logic ensures the chain behind a Gate can only be //activated once.//
</pre>
</div>
<div title="RenderProcess" modifier="Ichthyostega" created="200706190705" modified="201112162056" tags="Rendering operational">
<pre>For each segment (of the effective timeline), there is a Processor holding the exit node(s) of a processing network, which is a &quot;Directed Acyclic Graph&quot; of small, preconfigured, stateless [[processing nodes|ProcNode]]. This network is operated according to the ''pull principle'', meaning that the rendering is just initiated by &quot;pulling&quot; output from the exit node, causing a cascade of recursive downcalls or prerequisite calculations to be scheduled as individual [[jobs|RenderJob]]. Each node knows its predecessor(s), thus the necessary input can be pulled from there. Consequently, there is no centralized &quot;engine object&quot; which may invoke nodes iteratively or table driven &amp;mdash; rather, the rendering can be seen as a passive service provided for the Vault, which may pull from the exit nodes at any time, in any order (?), and possibly multithreaded.
<div title="RenderProcess" modifier="Ichthyostega" created="200706190705" modified="202407030216" tags="Rendering operational" changecount="1">
<pre>{{red{⚠ In-depth rework underway as of 7/2024...}}}
^^┅┅┅┅┅┅the following text is ''superseded''┅┅┅┅┅┅┅┅┅^^
For each segment (of the effective timeline), there is a Processor holding the exit node(s) of a processing network, which is a &quot;Directed Acyclic Graph&quot; of small, preconfigured, stateless [[processing nodes|ProcNode]]. This network is operated according to the ''pull principle'', meaning that the rendering is just initiated by &quot;pulling&quot; output from the exit node, causing a cascade of recursive downcalls or prerequisite calculations to be scheduled as individual [[jobs|RenderJob]]. Each node knows its predecessor(s), thus the necessary input can be pulled from there. Consequently, there is no centralized &quot;engine object&quot; which may invoke nodes iteratively or table driven &amp;mdash; rather, the rendering can be seen as a passive service provided for the Vault, which may pull from the exit nodes at any time, in any order (?), and possibly multithreaded.
All State necessary for a given calculation process is encapsulated and accessible by a StateProxy object, which can be seen as the representation of &quot;the process&quot;. At the same time, this proxy provides the buffers holding data to be processed and acts as a gateway to the Vault to handle the communication with the Cache. In addition to this //top-level State,// each calculation step includes a small [[state adapter object|StateAdapter]] (stack allocated), which is pre-configured by the builder and serves the purpose to isolate the processing function from the detals of buffer management.
@ -11180,8 +11192,10 @@ On a more global level, this LowLevelModel within the engine exposes a number of
!Control connections
</pre>
</div>
<div title="WiringDescriptor" modifier="Ichthyostega" created="200807132338" modified="200807132352" tags="Rendering operational impl spec">
<pre>Each [[processing node|ProcNode]] contains a stateless ({{{const}}}) descriptor detailing the inputs, outputs and predecessors. Moreover, this descriptor contains the configuration of the call sequence yielding the &amp;raquo;data pulled from predecessor(s)&amp;laquo;. The actual type of this object is composed out of several building blocks (policy classes) and placed by the builder as a template parameter on the WiringDescriptor of the individual ProcNode. This happens in the WiringFactory in file {{{nodewiring.cpp}}}, which consequently contains all the possible combinations (pre)generated at compile time.
<div title="WiringDescriptor" modifier="Ichthyostega" created="200807132338" modified="202407030217" tags="Rendering operational impl spec" changecount="1">
<pre>{{red{⚠ In-depth rework underway as of 7/2024...}}}
^^┅┅┅┅┅┅the following text is ''superseded''┅┅┅┅┅┅┅┅┅^^
Each [[processing node|ProcNode]] contains a stateless ({{{const}}}) descriptor detailing the inputs, outputs and predecessors. Moreover, this descriptor contains the configuration of the call sequence yielding the &amp;raquo;data pulled from predecessor(s)&amp;laquo;. The actual type of this object is composed out of several building blocks (policy classes) and placed by the builder as a template parameter on the WiringDescriptor of the individual ProcNode. This happens in the WiringFactory in file {{{nodewiring.cpp}}}, which consequently contains all the possible combinations (pre)generated at compile time.
!building blocks
* ''Caching'': whether the result frames of this processing step will be communicated to the Cache and thus could be fetched from there instead of actually calculating them.

1255
wiki/thinkPad.ichthyo.mm
View file

File diff suppressed because it is too large Load diff