What I'm about to do amounts to a massive generalisation, which is tricky.
Instead of having a fixed array-style layout, we want to accept arbitrary and mixed arguments.
Notably, we want to give the ''actual Library Plug-in'' a lot of leeway for binding:
- optionally, the library might want to require **Parameters** (which is the reason for this change)
- moreover, accepting input-buffers shall now be optional, since many generation functions do not need them
- and on top of all this, we want to accept an arbitrary mix of types for each kind.
So conceptually we are switching from C-style arrays to tuples with full type safety
''this going to become quite nasty and technical, I'm afraid...''
This is a first step towards the goal to introduce a ''parameter tuple'' into the `FeedManifold`.
Doing so invalidates some of the previously taken decisions regarding the `FeedManifold`;
at that time I was still under the impression of the old design from 2012, which called for a ''Buffer Table''.
Now we are forced to allow for more leeway in the function definition; even more so, since the limitation
to one single input and output Buffer type can be deemed unrealistic anyway.
So why sticking to an array at all? ''Buffers could also be a tuple...''
Seemingly another reason why I used an array was the idea to somehow limit the number of template instances,
by grouping them into a few number of array sizes, like 1,2,5 and 10.
This idea falls short, since in reality it can not be avoided to have the processing function on the type signature anyway.
Thus, the only point where the number of templates could be limited lies in the library plug-in,
where this »processing function« is actually defined as an adapter.
Starting from a prototypical implementation,
where each »slot« in the function is directly connected to the corresponding lead / port,
the implementation of the `SimpleWeavingPattern` (as it was called previously) could be
augmented and adapted gradually — and seems well suited to cover most standard cases of ''media processing''
So a name change is mandated, and the code is also extracted and relocated, possibly even
to be combined with the code of the `InvocationAdapter`, thereby hopefully making the implementation more accessible
Generally speaking, ''weaving patterns'' take on the role of the prime extension point regarding `Port` implementation.
This is an attempt to take aim at the next step,
which is to fill in the missing part for an actual node invocation...
''...still fighting to get ahead, due to complexity of involced concerns...''
This was an extended digression into architecture planning,
which became necessary in order to suitably map out the role
for the `TurnoutSystem` — which can now be defined as ''mediator''
to connect and forward control- and parameter data while specific
render invocation proceeds through the render node network.
* Lumiera source code always was copyrighted by individual contributors
* there is no entity "Lumiera.org" which holds any copyrights
* Lumiera source code is provided under the GPL Version 2+
== Explanations ==
Lumiera as a whole is distributed under Copyleft, GNU General Public License Version 2 or above.
For this to become legally effective, the ''File COPYING in the root directory is sufficient.''
The licensing header in each file is not strictly necessary, yet considered good practice;
attaching a licence notice increases the likeliness that this information is retained
in case someone extracts individual code files. However, it is not by the presence of some
text, that legally binding licensing terms become effective; rather the fact matters that a
given piece of code was provably copyrighted and published under a license. Even reformatting
the code, renaming some variables or deleting parts of the code will not alter this legal
situation, but rather creates a derivative work, which is likewise covered by the GPL!
The most relevant information in the file header is the notice regarding the
time of the first individual copyright claim. By virtue of this initial copyright,
the first author is entitled to choose the terms of licensing. All further
modifications are permitted and covered by the License. The specific wording
or format of the copyright header is not legally relevant, as long as the
intention to publish under the GPL remains clear. The extended wording was
based on a recommendation by the FSF. It can be shortened, because the full terms
of the license are provided alongside the distribution, in the file COPYING.
...taking into account the prospecive usage context
where the builder expressions will be invoked from within
a media-library plug-in, using std::string_view to pass
the symbolic information seems like a good fit, because
the given spec will typically be assembled from some
building blocks, and thus in itself not be literal data.
Building a precise Frame Cache is a tough job, and is doomed to fail
when attempting to tie cache invalidation to state changes. The only
viable path is to create a system of systematic tagging of processing
steps, and use this as foundation for chained hash values, linked
in accordance to the actual processing structure.
This is complicated by the secondary concern of maintaining memory efficacy
for the render node model, which can be expected to grow to massive scale.
And even while this invocation can not be fully devised right now,
an attempt can be made to build a foundation that is not outright
wasteful, by detaching the logical information from the specific
weaving pattern used for implementation, and by minimising the
representation in memory and computing the compound information
on-demand....
The immediate next goal is to verify properties of render nodes
generated by the builder framework; two kinds of validations
can be distinguished
* structural aspects of the wiring
* the fact that processing functionality is invoked in proper order
Looking into the structural aspects brings about the necessity
to identify the actual processing function bound into some functor.
Some recapitulation of goals and requirements revealed, that this
can not be a merely technical identity record — because the intention
is to base the ''cache key'' on chained processing node identities,
so that the key is stable as long as the user-visible results will be
equivalent. And while structural data can be aggregated, at the
core this information must be provided by the scheme embedded
into the domain ontology, which is tasked with invoking the
builder in order to implement a ''specific processing-asset''
Since it would in fact be possible to access and write beyond the configured storage,
simply by using the builder API without considering consistency,
it seems advisable to use explicit runtime checks here, instead of
only assertions, and to throw an exception when violating bounds.
Moreover, unsuccessfully attempted to better arrange the functionality
between PortBuilder and WeavingBuilder; seemingly we have an rather tight
coupling here, and also the expectations regarding the processing function
seem to be too tight (but that's the reason why it's an prototype...)
...which then also allow to fill in the missing parts for the
default 1:1 wiring scheme, which connects each »input slot«
of the processing function with the corresponding ''lead node''
The intention is to offer an automatic 1:1 association
between the »input parameter slots« of the processing function
and the ''lead nodes,'' thereby always using the same default
port, corresponding to the current port number under construction.
Unfortunately, the preceding refactoring removed the information
necessary for a simple implementation, as the port array is now
built up late, in the final build() function...
The next step is to round out the first prototypical implementation,
which requires access to ''lead node ports'' and thereby generally
places focus on the interplay of ''data builders'' within the ongoing
build process. While the prototype still uses the fall-back to simple
heap allocation, notably the intended usage will require to wire-through
the connection to a single `AllocationCluster`. This poses some
challenge, since further ''data builders'' will be added step-wise,
implying that this wiring can not be completed at construction time.
Thus it seems indicated to slightly open-up the internal allocator
policy base template used by `lib::SeveralBuilder` to allow for some
kind of ''cross building'' based on a shared compatible base allocator
type, so that the allocation policy wiring can be passed-on from an
existing `SeveralBuilder`
- the chaining constructor is picked reliably when the
slicing is done by a direct static_cast
- the function definition can be passed reliably in all cases
after it has been ''decayed,'' which is done here simply by
taking it by-value. This is adequate, since the function
definition must be copied / inlined for each invocation.
With these fixes, the simplest test case now for the first time
**runs through without failure**
This change allows to disentangle the usages of `lib::SeveralBuilder`,
so that at any time during the build process only a single instance is
actively populated, all in one row — and thus the required storage can
either be pre-allocated, or dynamically extended and shrinked (when
filling elements into the last `SeveralBuilder` currently activated)
By packaging into a λ-closure, the building of the actual `Port`
implementation objects (≙ `Turnout` instances) is delayed until the
very end of the build process, and then unloaded into yet another
`lib::Several` in one strike. Temporarily, those building functor
objects are „hidden“ in the current stack frame, as a new `NodeBuilder`
instance is dropped off with an adapted type parameter (embedding the
λ-type produced by the last nested `PortBuilder` invocation, while
inheriting from previous ones.
However, defining a special constructor to cause this »chaining«
poses some challenge (regarding overload resolution). Moreover,
since the actual processing function shall be embedded directly
(as opposed to wrapping it into a `std::function`), further problems
can arise when this function is given as a ''function reference''
Conduct in-depth analysis to handle a secondary, implementation-related
(and frankly quite challenging) concern regarding the placement of node
and port connectivity data in memory. The intention is for the low-level
model to use a custom data structure based on `lib::Several`, allowing for
flexible and compact arrangement of the connectivity descriptors within
tiled memory blocks, which can then later be discarded in bulk, whenever
a segment of the render graph is superseded. Yet since the generated
descriptors are heterogeneous and, due to virtual functions, can not be
trivially copied, the corresponding placement invocations on the
data builder API must not be mixed, but rather given in ordered strikes
and preceded by a dimensioning call to pre-reserve a bulk of storage
However, doing so directly would jeopardise the open and flexible nature
of the node builder API, thereby creating a dangerous coupling between
the implementation levels of the node graph and of prospective library
wrapper plug-ins in charge of controlling details of the graph layout.
The solution devised here entails a functional helper data structure
created temporarily within the builder API stack frames; the detailed
and local type information provided from within the library plug-in
can thereby be embedded into opaque builder functors, allowing to
delay the actual data generation up until the final builder step,
at which point the complete number and size requirements of
connectivity data is known and can be used for dimensioning.
...and as expected, this turns up quite some inconsistencies,
especially regarding usage of the »buffer types«.
Basically, the `PortBuilder` is responsible for the high-level functionality
and thus must ensure the nested `WiringBuilder` is addressed and parameterised
properly to connect all »slots« of the processing function.
- can use a helper function in the WiringBuilder to fill in connections
- but the actual buffer types passed over these connectinos are totally
unchecked at that level, and can not see yet how this danger can be
mitigated one level above, where the PortBuilder is used.
- it is still unclear what a »buffer type« actually means; it could
be the pointer type, but it could also imply a class or struct type
to be emplaced into the buffer, which is a special extension to the
`BufferProvider` protocol, yet seems to be used here rather to transport
specific data types required by the actual media handling library (e.g. FFmpeg)
__Analysis__: what kind of verifications are sensible to employ
to cover building, wiring and invocation of render nodes?
Notably, a test should cover requirements and observable functionality,
while ''avoiding direct hard coupling to implementation internals...''
__Draft__: the most simple node builder invocation conceivable...
* decision how to provide a default service for tests
while also allow for configuration of more specific services
* as starting point for the prototype: use the `TrackingHeapBlockProvider`
(simply because this is the only implementation available and tested)
...need to pass a binding for the actual processing function
in a way that it acts as a ''prototype'' — since the `Feed`,
i.e. the ''Invocation Adapter'' must be generated for each
invocation anew within the current stack frame
(so to avoid spurious heap allocations)
...seems that the former is well suited to serve as detail builder
used internally by the latter to provide a simplified standard adaptation
for a given processing function.
The integration can be achieved to layer a specialised detail builder class
on top, which can be entered only by specifying the concrete function or lambda
to wrap for the processing; the further builder-API-functions to control
the wiring in detail become thus only accessible after the function type
is known; this allows to place the detail builder as member into the
enclosing port builder and thus to allocate everything within the current
stack frame invoking the builder chain.
...by relying on DI for some effectively global services, notably
the cache provider, the API for building and wiring render nodes
can be simplified to cover only the actual node connectivity
Doing so directly seems to be a better solution than to inject an OutputBufferProvider;
the latter will still be needed, yet will not be part of the regular weaving pattern,
but used directly at top-level to obtain the output `BuffHandle`, which is then
passed to the `Port::weave()` call
...still not convinced that this is a good design,
since it seems to subvert the general design to treat one special case.
However, I can't see a good way to address this special case directly
Requirement analysis shows that the ''actual buffer provider'' to use
constitutes yet another independent degree of freedom, which conceivably
must be handled by the Builder internals rather than by the Domain Ontology.
Thus the simple solution to use a `BuffDescr` to mark the type must be augmented
to also allow configuration of the underlying `BufferProvider`, which generates
the descriptor and can later be invoked with this descriptor to ''lock an actual Buffer.''
In some cases, setup of the buffer types could even be more complicated and require
access to the actual (runtime) invocaton context; such extreme cases however
could be rendered as an extension of the scheme established here,
by storing the (up to now transient) constructor functors persistently.
Which leads to the decision not to care for those extremely complicated
corner cases right now, and thus to construct all buffer descriptors
in the `build()` call
...still fighting to find a suitable API to define
how inputs and outputs are connected and mapped to function parameters.
The solution drafted here uses the reshaped `DataBuilder` (≙`lib::SeveralBuilder`)
to add up connections for each »slot«, disregarding the possibility of permutations.
Similar to `NodeBuilder`, a policy template is used to pass down the setup
for an actual custom allocator.
