From 3406b6abf5b141a572241189e9136cb1066f2015 Mon Sep 17 00:00:00 2001 From: Ichthyostega Date: Sat, 28 Dec 2024 23:16:55 +0100 Subject: [PATCH] Invocation: anchor data frame in `TurnoutSystem` So this is a design sketch how a `ParamBuildSpec` descriptor could be created, which in turn would provide the foundation to implement a ''Parameter Weaving Pattern...'' __Note__: since this is an extension for advanced usage, yet relies on a storage layout defined to allow for extensions like this use case here, the anchor type is now defined to reside in the `TurnoutSystem` in the form of a ''standard parameter block''. Those standard invocation parameters are fixed and thus can be hard coded. --- src/steam/engine/param-weaving-pattern.hpp | 165 +++++---------------- src/steam/engine/turnout-system.hpp | 17 +-- tests/core/steam/engine/node-feed-test.cpp | 4 +- tests/library/hetero-data-test.cpp | 2 +- wiki/thinkPad.ichthyo.mm | 161 ++++++++++++++++---- 5 files changed, 185 insertions(+), 164 deletions(-) diff --git a/src/steam/engine/param-weaving-pattern.hpp b/src/steam/engine/param-weaving-pattern.hpp index 66fc49175..75e6a3b43 100644 --- a/src/steam/engine/param-weaving-pattern.hpp +++ b/src/steam/engine/param-weaving-pattern.hpp @@ -1,8 +1,7 @@ /* - PARAM-WEAVING-PATTERN.hpp - Base patterns to organise data connections and render processing invocation + PARAM-WEAVING-PATTERN.hpp - Pattern to carry out special parameter computations Copyright (C) - 2008, Hermann Vosseler 2024, Hermann Vosseler   **Lumiera** is free software; you can redistribute it and/or modify it @@ -14,116 +13,22 @@ /** @file param-weaving-pattern.hpp - ** Construction set to assemble and operate a data processing scheme within a Render Node. - ** Together with turnout.hpp, this header provides the "glue" which holds together the - ** typical setup of a Render Node network for processing media data. A MediaWeavingPattern - ** implements the sequence of steps — as driven by the Turnout — to combine the invocation - ** of media processing operations from external Libraries with the buffer- and parameter - ** management provided by the Lumiera Render Engine. Since these operations are conducted - ** concurrently, all invocation state has to be maintained in local storage on the stack. + ** Construction kit to establish a set of parameters pre-computed prior to invocation + ** of nested nodes. This arrangement is also known as »Parameter Agent Node« (while actually + ** it is a Weaving Patter residing within some Node's Port). The use-case is to provide a set + ** of additional parameter values, beyond what can be derived directly by a parameter-functor + ** based on the _absolute-nominal-Time_ of the invocation. The necessity for such a setup may + ** arise when additional context or external state must be combined with the nominal time into + ** a tuple of data values, which shall then be consumed by several follow-up evaluations further + ** down into a recursive invocation tree _for one single render job._ The solution provided by + ** the Parameter Agent Node relies on placing those additional data values into a tuple stored + ** directly in the render invocation stack frame, prior to descending into further recursive + ** Node evaluations. Notably, parameter-functors within the scope of this evaluation tree can + ** then access these additional parameters through the TurnoutSystem of the overall invocation. ** - ** # Integration with media handling Libraries - ** - ** A Render invocation originates from a [Render Job](\ref render-invocation.hpp), which first - ** establishes a TurnoutSystem and then enters into the recursive Render Node activation by - ** invoking Port::weave() for the »Exit Node«, as defined by the job's invocation parameters. - ** The first step in the processing cycle, as established by the Port implementation (\ref Turnout), - ** is to build a »Feed instance«, from the invocation of `mount(TurnoutSystem&)`. - ** - ** Generally speaking, a `Feed` fulfils the role of an _Invocation Adapter_ and a _Manifold_ of - ** data connections. The standard implementation, as given by MediaWeavingPattern, relies on a - ** combination of both into a \ref FeedManifold. This is a flexibly configured data adapter, - ** directly combined with an embedded _adapter functor_ to wrap the invocation of processing - ** operations provided by an external library. - ** - ** Usually some kind of internal systematics is assumed and applied within such a library. - ** Operations can be exposed as plain function to invoke, or through some configuration and - ** builder notion. Function arguments tend to follow a common arrangement and naming scheme, - ** also assuming specific arrangement and data layout for input and output data. This kind of - ** schematism is rooted in something deeper: exposing useful operations as a library collection - ** requires a common ground, an understanding about the _order of things_ to be treated — at least - ** for those kind of things, which fall into a specific _domain,_ when tasks related to such a - ** domain shall be supported by the Library. Such an (implicit or explicit) framework of structuring - ** is usually designated as a **Domain Ontology** (in contrast to the questions pertaining Ontology - ** in general, which are the subject of philosophy proper). Even seemingly practical matters like - ** processing media data do rely on fundamental assumptions and basic premises regarding what is - ** at stake and what shall be subject to treatment, what fundamental entities and relationships - ** to consider within the domain. Incidentally, many of these assumptions are positive in nature - ** and not necessarily a given — which is the root of essential incompatibilities between Libraries - ** targeting a similar domain: due to such fundamental differences, they just can not totally agree - ** upon what kinds of things to expect and where to draw the line of distinction. - ** - ** The Lumiera Render Engine and media handling framework is built in a way _fundamentally agnostic_ - ** to the specific presuppositions of this or that media handling library. By and large, decisions, - ** distinctions and qualifications are redirected back into the scope of the respective library, by - ** means of a media-library adapter plug-in. Assuming that the user _in fact understands_ the meaning - ** of and reasoning behind employing a given library, the _mere handling_ of the related processing - ** can be reduced to a small set of organisational traits. For sake of consistency, you may label - ** these as a »Render Engine Ontology«. In all brevity, - ** - We assume that the library provides distinguishable processing operations - ** that can be structured and classified and managed as _processing assets,_ - ** - we assume that processing is applied to sources or »media« and that - ** the result of processing is again a source that can be processed further; - ** - specific operations can thus be conceptualised as processing-stages or _Nodes,_ - ** interconnected by _media streams,_ which can be tagged with a _stream type._ - ** - At implementation level, such streams can be represented in entirety as data buffers - ** of a specific layout, filled with some »frame« or chunk of data - ** - and the single processing step or operation can be completely encapsulated - ** as a pure function (referentially transparent, without side effects); - ** - all state and parametrisation can be represented either as some further data stream in/out, - ** or as parameters-of-processing, which can be passed as a set of values to the function - ** prior of invocation, thereby completely determining the observable behaviour. - ** - ** # composition of the Invocation State - ** - ** By means of this positing, the attachment point to an external library can be reduced into a small - ** number of connector links. Handling the capabilities of a library within the Session and high-level Model - ** will require some kind of _registration,_ which is beyond the scope of this discussion here. As far as - ** the Render Engine and the low-level-Model is concerned, any usage of the external libraries capabilities - ** can be reduced into... - ** - Preparing an adapter functor, designated as **processing-functor**. This functor takes three kinds of - ** arguments, each packaged as a single function call argument, which may either be a single item, or - ** possibly be structured as a tuple of heterogeneous elements, or an array of homogeneous items. - ** + an output data buffer or several such buffers are always required - ** + (optionally) an input buffer or several such buffers need to be supplied - ** + (optionally) also a parameter value, tuple or array can be specified - ** - Supplying actual parameter values (if necessary); these are drawn from the invocation - ** of a further functor, designated as **parameter-functor**, and provided from within - ** the internal framework of the Lumiera application, either to deliver fixed parameter - ** settings configured by the user in the Session, or by evaluating _Parameter Automation,_ - ** or simply to supply some technically necessary context information, most notably the - ** frame number of a source to retrieve. - ** - Preparing buffers filled with input data, in a suitable format, one for each distinct - ** item expected in the input data section of the processing-functor; filling these - ** input buffers requires the _recursive invocation_ of further Render Nodes... - ** - Allocating buffers for output data, sized and typed accordingly, likewise one for - ** each distinct item detailed in the output data argument of the processing-functor. - ** - ** The FeedManifold template, which (as mentioned above) is used by this standard implementation - ** of media processing in the form of the MediaWeavingPattern, is configured specifically for each - ** distinct signature of a processing-functor to match the implied structural requirements. If a - ** functor is output-only, no input buffer section is present; if it expects processing parameters, - ** storage for an appropriate data tuple is provided and a parameter-functor can be configured. - ** A clone-copy of the processing-functor itself is also stored as clone-copy alongside within - ** the FeedManifold, and thus placed into stack memory, where it is safe even during deeply nested - ** recursive invocation sequences, while rendering in general is performed massively in parallel. - ** - ** In the end, the actual implementation code of the weaving pattern has to perform the connection - ** and integration between the »recursive weaving scheme« and the invocation structure implied by - ** the FeedManifold. It has to set off the recursive pull-invocation of predecessor ports, retrieve - ** the result data buffers from these and configure the FeedManifold with the \ref BuffHandle entries - ** retrieved from these recursive calls. Buffer handling in general is abstracted and codified thorough - ** the [Buffer Provider framework](\ref buffer-provider.hpp), which offers the means to allocate further - ** buffers and configure them into the FeedManifold for the output data. The »buffer handling protocol« - ** also requires to invoke BuffHandle::emit() at the point when result data can be assumed to be placed - ** into the buffer, and to release buffers not further required through a BuffHandle::release() call; - ** Notably this applies to the input buffers are completion of the processing-functor invocation, and - ** is required also for secondary (and in a way, superfluous) result buffers, which are sometimes - ** generated as a by-product of the processing function invocation, but are actually not passed - ** as output up the node invocation chain. - ** - ** @see feed-manifold.hpp + ** @see node-builder.hpp ** @see weaving-pattern-builder.hpp + ** @see NodeFeed_test::feedParamNode() ** @see \ref proc-node.hpp "Overview of Render Node structures" ** ** @warning WIP as of 12/2024 first complete integration round of the Render engine ////////////////////////////TICKET #1367 @@ -137,7 +42,7 @@ #include "steam/common.hpp" #include "steam/engine/turnout.hpp" #include "steam/engine/turnout-system.hpp" -#include "steam/engine/feed-manifold.hpp" +#include "steam/engine/feed-manifold.hpp" ////////////TODO wegdamit /////////////////////////////////////////////////////////////////////////////////////////////////////////////TICKET #1367 : Rebuild the Node Invocation //#include "vault/gear/job.h" //#include "steam/engine/exit-node.hpp" @@ -151,27 +56,39 @@ //#include "lib/util.hpp" ////////OOO wegen manifoldSiz() //#include +#include +#include namespace steam { namespace engine { - using lib::Several; + using std::move; + using std::tuple; + using lib::Several;////TODO RLY? + + + template + struct ParamBuildSpec + { + using Functors = tuple; + + Functors functors_; + + ParamBuildSpec (Functors&& funz) + : functors_{move (funz)} + { } + }; + + auto + buildParamSpec() + { + return ParamBuildSpec{tuple<>{}}; + } /** - * Standard implementation for a _Weaving Pattern_ to connect - * the input and output data feeds (buffers) into a processing function. - * @tparam INVO a configuration / policy base class to _adapt for invocation_ - * @note assumptions made regarding the overall structure - * - `INVO::Feed` defines an _invocation adapter_ for the processing function - * - `INVO::buildFeed()` is a functor to (repeatedly) build `Feed` instances - * - the _invocation adapter_ in turn embeds a `FeedManifold` to hold - * + a setup of output buffer pointers (single, tuple or array) - * + (optionally) a similar setup for input buffer pointers - * + (optionally) a parameter or parameter tuple - * + storage to configure BuffHandle entries for each «slot» - * + storage to hold the actual processing functor + * Implementation for a _Weaving Pattern_ to conduct extended parameter evaluation. */ template struct ParamWeavingPattern diff --git a/src/steam/engine/turnout-system.hpp b/src/steam/engine/turnout-system.hpp index 0312dcc05..705aeaf80 100644 --- a/src/steam/engine/turnout-system.hpp +++ b/src/steam/engine/turnout-system.hpp @@ -50,13 +50,6 @@ namespace engine { using lib::time::Time; using ProcessKey = uint64_t; - template - class ParamStorageFrame - : util::NonCopyable - { - - }; - /** * Communication hub to coordinate and activate the »Render Node Network« performance. @@ -65,12 +58,18 @@ namespace engine { * and initiates the recursive pull()-call into the render node network as attached for this call. */ class TurnoutSystem + : util::NonCopyable { - /////////////////////////////////////////////////////////OOO Storage: ich brauche Overflow-Buckets. KISS ==> erst mal intrusive linked List + public: + using FrontBlock = lib::HeteroData; + + private: + FrontBlock invoParam_; + public: TurnoutSystem (Time absoluteNominalTime, ProcessKey procKey =0) + : invoParam_{FrontBlock::build (absoluteNominalTime,procKey)} { } - // this is a copyable front-end object }; diff --git a/tests/core/steam/engine/node-feed-test.cpp b/tests/core/steam/engine/node-feed-test.cpp index bbbb78dfb..f594cbb3d 100644 --- a/tests/core/steam/engine/node-feed-test.cpp +++ b/tests/core/steam/engine/node-feed-test.cpp @@ -20,6 +20,7 @@ #include "steam/engine/proc-node.hpp" #include "steam/engine/node-builder.hpp" #include "steam/engine/media-weaving-pattern.hpp" +#include "steam/engine/param-weaving-pattern.hpp" #include "steam/engine/turnout-system.hpp" #include "steam/engine/turnout.hpp" #include "steam/engine/diagnostic-buffer-provider.hpp" @@ -104,7 +105,8 @@ namespace test { void feedParamNode() { - TODO ("implement the logic for the TurnoutSystem --> node-base-test.cpp"); + auto spec = buildParamSpec(); +SHOW_TYPE(decltype(spec)) TODO ("implement a simple Builder for ParamAgent-Node"); TODO ("then use both together to demonstrate a param data feed here"); } diff --git a/tests/library/hetero-data-test.cpp b/tests/library/hetero-data-test.cpp index c780ceb6b..0dd4de274 100644 --- a/tests/library/hetero-data-test.cpp +++ b/tests/library/hetero-data-test.cpp @@ -178,7 +178,7 @@ namespace test{ /** @test demonstrate elaborate storage layout with several chain frames * - follow-up frames shall be built using constructor types * - these can be defined prior to any data allocation - * - individual data elements can be accessed type-safe through accessor functors + * - individual data elements can be accessed through type-safe accessor functors * @warning as demonstrated, this is a dangerous bare-bone memory layout without runtime checks! */ void diff --git a/wiki/thinkPad.ichthyo.mm b/wiki/thinkPad.ichthyo.mm index d5c3e886f..2a0e2741b 100644 --- a/wiki/thinkPad.ichthyo.mm +++ b/wiki/thinkPad.ichthyo.mm @@ -29309,9 +29309,7 @@ - - - +
  1. @@ -30903,9 +30901,7 @@ - - - +

    Konvention: Clips mit start==Time::NEVER werden verborgen @@ -31325,9 +31321,7 @@ - - - +

     void set_path(const WidgetPath& path); @@ -31728,9 +31722,7 @@ - - - +

    denn was passiert, wenn sich durch das Setzen einer neuen Größe der sichtbare Bereich ändert? Löst das dann nicht erneut einen draw()-Aufruf aus?? @@ -82274,8 +82266,7 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
    Beachte: es handelt sich um einen Erweiterungspunkt von theoretischer Relevanz...

    - -     + @@ -82312,8 +82303,7 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
    ⟹ folglich braucht es eine vorgelagerte ParamBuildSpec

    - -     +     @@ -88258,8 +88248,7 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
    der PortBuilder kann einen Adapter für klassische Parameter-Automation bauen; dieser muß intern auf die absolute-nominal-Time  zugreifen können

    - - +     @@ -88295,8 +88284,7 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
    die konkreten Accessoren werden als spezielle Typisierung in einen privaten Kontext in ein invocation-λ eingebettet, so daß sie nur aus dem Scope der tatsächlichen Invocation heraus sichtbar sind; solange dieser Scope lebt, ist per Definitionem auch zumindest dieser Chain-Block valide und zugreifbar.

    - -     +     @@ -91307,8 +91295,8 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
    - + @@ -94171,12 +94159,130 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
         + - + + + + + + +

    + Zur Erinnerung: HeteroData ist ein low-level Daten-Layout, das ich speziell für den Anwendungsfall im Turnout mir ausgedacht habe; es beruht auf einer single-linked-List von Datenblöcken, in denen jeweils ein Datentupel sitzt. Und hier liegt das Problem: diese Chain-Blöcke sollen nun schrittweise über eine Builder-Notation aufgebaut werden — aber die Konstruktor-Funktion in HeteroData erwartet die Angabe des ganzen Tupel-Typs auf einmal. +

    + + +     + +     + + + + +

    + daher scheidet ein Typ-Konstruktor als Einstiegspunkt aus; man kann nämlich i.d.R. vom User nicht erwerten, daß der jeweilige Funktor-Typ explizit anggebbar ist +

    + +     +     + + + + +

    + man baut ja schrittweise diese ParamBuildSpec auf ⟹ also muß es nicht gleich der final korrekte Tupel-Typ sein; erst wenn man damit tatsächlich alloziert, legen wir uns fest. +

    + +     +     + + + + + + + + + + +     + + + + + + + + + + + + + +

    + also ein Instanz-Typ allein legt alles fest; keine Indirektion oder Virtualisierung +

    + + +     +     + + + + +

    + damit liegt das unumgängliche Minimum an Information bereits fest; insofern wir explizit getypte Funktoren durch das System grundsätzlich durchreichen können wollen (nicht bloß std::function-Instanzen), bleibt keine Alternative, als diese Typen in ein Tupel zu packen.... +

    + +     +     + + + + +

    + ...weil wir allein mit einer solchen ParamBuildSpec auf ein TurnoutSystem losgehen wollen; eine Hetero-Data prefix-Chain ist damit notwendig, als Anker-Punkt um einen konkreten Konstruktor für einen Chain-Block zu generieren, den wir in den aktuellen StackFrame legen wollen +

    + + +     +     + + + + + + + +

    + also ParamBuildSpec<ANK, FUNZ...> +

    + +     +     +     + + + + + + + + +

    + ⟹ Ankerpunkt muß ein nested type TurnoutSystem::FrontBlock sein +

    + + +     + +     +     +     @@ -97610,8 +97716,8 @@ StM_bind(Builder<R1> b1, Extension<R1,R2> extension)     - + @@ -97624,8 +97730,7 @@ StM_bind(Builder<R1> b1, Extension<R1,R2> extension) NodeFeed_test::feedParamNode()

    - -     +     @@ -97701,8 +97806,7 @@ StM_bind(Builder<R1> b1, Extension<R1,R2> extension) Über das Prototyping hat eine Art Normierung stattgefunden; ich bin jetzt sicher, daß die allermeisten Aufgaben zur Medien-Berechnung in dieses Standard-Pattern abgebildet werden können, vor allem durch die stark erweiterte Flexibilität des Buffer- und Parameter-Bindings

    - - +     @@ -100010,8 +100114,7 @@ StM_bind(Builder<R1> b1, Extension<R1,R2> extension) Für die Parameter-Behandlung sind komplexe Spezialfälle vorhersehbar, welche die Pasis-Parameter, den Kontext-Zugriff über einen Prozeß-Key, sowie weitere, kontextuell zugängliche Zustandsdaten in einen erweiterten Satz konkreter Steuerparameter synthetisieren, um diese dann an anderer Stelle, rekursiv tiefer im Invocation-Tree, zur Parametrisierung der Medien-Berechnung heranzuziehen. Ein solches Muster ist in der Tat „speziell“ und ohne im Moment für mich erkennbaren konkreten Bezug; ich leite diese Möglichkeit lediglich aus allgemeinen Überlegungen zur Struktur her, und ziehe den Schluß, zumindest die Ansatzpunkte hierfür schaffen zu müssen....

    - - +