A lot of repetitive, pre C++11 metaprogramming code can now be removed,
and several helper constructs, which were needed to handle generic
function application and passing a tuple of values to create a binder.
Note however, the highly complex and technical core of this header
still remains intact; which is to create a ''partial closure'' over
some arguments of a function, while keeping the remaining arguments
open as parameters for invocation.
TODO: Even in the remaining code there is a lot of redundancy
and helper construct which are no longer necessary
Most of the type-list and type-sequence related eccosystem can be
just switched over, after having added the conversion variants for
the new-style variadic type sequences
Again this was used as opportunity to improve readability of related tests
As expected, these work on the new-style variadic type sequences
equally well than on the old ones (tail-filled with `Nil` markers).
On that occasion, a complete makeover of the huge test case was carried out,
now relying on `ExpectString` instead of printing to STDOUT. This has the
benefit of showing the expectation immediately next to the code to be tested,
and thus makes it much easier to ''actually see'' how these meta-functions
operate on their parameters (which in fact are types in a type list)
- provide complete conversion paths old-style ⟷ new-style
- switch the basic tests to the new variadic sequences
- modernise the code; replace typedefs by `using`
- change some struct-style ''meta-functions'' into
constexpr or compile-time constants
During the early stage of the Project, at some point I attempted
to »attack« the topic of Engine and Render Nodes following a ''top down path.''
This effort went into a dead end eventually — due to the total lack
of tangible reference points to relate to. However, the implementation
at that time prompted the development of several supporting facilities,
which remain relevant until today. And it resulted in a ''free wheeling''
compound of implementation structures, which could even be operated
through some highly convoluted unit test.
This piece of implementation code was valuable as starting point for th
»Playback Vertical Slice« in 2024 — resulting in a new design which was
''re-oriented'' towards a new degree of freedom (the »Domain Ontology«)
while handling the configuration and connectivity of Render Nodes in
a rather fixed and finite way. This new approach seems to be much more
successful, as we're now able to build, connect and invoke Render Nodes,
thereby mapping the processing through a functor binding into some
arbitrary, external processing function (which will later be supplied
by a media processing library — and thus be part of some »Domain Ontology«)
* now able to demonstrate close-front, close-back and close-argument
* can also apply the same cases to `std::array`, with input and
output type seamlessly adapted to `std::array`
Tuples and the ''C++ tuple protocol'' build upon variadic arguments
and are thus rather tedious to handle, especially in this situation here,
where the argument can ''sometimes be a tuple...''
Several years ago I made the observation that processing by explicit ''type sequences''
(Loki-style) is much simpler to handle and easier to lift to a generic level of processing.
Thus I'll attempt now to extract the ''iteration and extraction part'' of the logic into a new helper.
`lib::meta::ElmTypes<TUP>` allows to process all ''tuple-like types'' and generic ''type sequences'' uniformely
and enables to use both styles interchangably (btw, it is quite common to ''abuse'' `std::tuple` as a type sequence).
With this helper, we can now
- build a ''type sequence'' from any ''tuple-like'' object (and vice-versa)
- re-bind (i.e. transfer the template parameters to another template)
- apply some wrapper
- create AND / OR evaluations over the types
For sake of completeness, since the `IterExplorer` supports building extended
search- and evaluation patterns, a tuple-zipping adapter can be expected
to handle these extended usages transparently.
While the idea is simple, making this actually happen had several ramifications
and required to introduce additional flexibility within the adaptor-framework
to cope better with those cases were some iterator must return a value, not a ref.
In the end, this could be solved with a bit of metaprogramming based on `std::common_type`
...and indeed, this is all quite nasty stuff — in hindsight, my initial intuition
to shy away from this topic was spot-on....
and yes ... this revealed a **long standing bug**
The `Filter::pullFilter()` invocation in the ctor may produce dangling refs,
whenever an underlying source-iterator generates a reference that points
into the iterator itself.
The reason is: due to the »onion shell« design of the iterator pipeline,
we are bound to move a source iterator into the next layer constructor.
Basically I am sick of writing for-loops in those cases
where the actual iteration is based on one or several data sources,
and I just need some damn index counter. Nothing against for-loops
in general — they have their valid uses — sometimes a for-loop is KISS
But in these typical cases, an iterator-based solution would be a
one-liner, when also exploiting the structured bindings of C++17
''I must admit that I want this for a loooooong time —''
...but always got intimidated again when thinking through the fine points.
Basically it „should be dead simple“ — as they say
Well — — it ''is'' simple, after getting the nasty aspects of tuple binding
and reference data types out of the way. Yesterday, while writing those
`TestFrame` test cases (which are again an example where you want to iterate
over two word sequences simultaneously and just compare them), I noticed that
last year I learned about the `std::apply`-to-fold-expression trick, and
that this solution pattern could be adapted to construct a tuple directly,
thereby circumventing most of the problems related to ''perfect forwarding''
So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`)
and I have learned how to make this application completely generic.
As a second step, I implemented a proof-of-concept in `IterZip_test`,
which indeed was not really challenging, because the `IterExplorer`
is so very sophisticated by now and handles most cases with transparent
type-driven adaptors. A lot of work went into `IterExplorer` over the years,
and this pays off now.
The solution works as follows:
* apply the `lib::explore()` constructor function to the varargs
* package the resulting `IterExplorer` instantiations into a tuple
* build a »state core« implementation which just lifts out the three
iterator primitives onto this ''product type'' (i.e. the tuple)
* wrap it in yet another `IterExplorer`
* add a transformer function on top to extract a value-tuple for each ''yield'
As expected, works out-of-the-box, with all conceivable variants and wild
mixes of iterators, const, pointers, references, you name it....
PS: I changed the rendering of unsigned types in diagnostic output
to use the short notation, e.g. `uint` instead of `unsigned int`.
This dramatically improves the legibility of verification strings.
Originally, this helper was called `IterIndex`, thereby following a
common naming scheme of iteration-related facilities in Lumiera, e.g.
* `IterAdapter`
* `IterExplorer`
* `IterSource`
However, I myself was not able to recall this name, and found myself
now for the second time unable to find this piece of code, even while
still able to recall vaguely that I had written something of this kind.
(and unable to find it by a text search for "index", for obvious reasons)
So, on a second thought, the original name is confusing: we do not create
an index of / for iterators; rather we are iterating an index. So this
is what it should be called...
Phew... this was a tough one — and not sure yet if this even remotely works...
Anyway, the `lib::SeveralBuilder` is already prepared for collaboration with a
custom allocator, since it delegates all memory handling through a base policy,
which in turn relies on std::allocator_traits.
The challenge however is to find a way...
* to make this clear and easy to use
* to expose an extension point for specific tweaks
* and to make all this work without excessive header cross dependencies
...extract and improve the tuple-rewriting function
...improve instance tracking test dummy objects
...complete test coverage and verify proper memory handling
The second design from 2017, based on a pipeline builder,
is now renamed `TreeExplorer` ⟼ `IterExplorer` and uses
the memorable entrance point `lib::explore(<seq>)`
✔
decided to add a very specific preprocessing here, to make the DSL notation more natural.
My guess is that most people won't spot the presence of this tiny bit of magic,
and it would be way more surprising to have rules like
UICoord::currentWindow().panel("viewer").create()
fail in most cases, simply because there is a wildcard on the perspective
and the panel viewer does not (yet) exist. In such a case, we now turn the
perspective into a "existential quantified" wildcard, which is treated as if
the actually existing element was written explicitly into the pattern.
several extensions and convenience features are conceivable,
but I'll postpone all of them for later, when actual need arises
Note especially there is one recurring design challenge, when creating
such a demand-driven tree evaluation: more often than not it turns out
that "downstream" will need some information about the nested tree structure,
even while, on the surfice, it looks as if the evaluation could be working
completely "linearised". Often, such a need arises from diagnostic features,
and sometimes we want to invoke another API, which in turn could benefit
from knowing something about the original tree structure, even if just
abstracted.
I have no real solution for this problem, but implementing this pipeline builder
leads to a pragmatic workaround: since the iterator already exposes a expandChildren(),
it may as well expose a depth() call, even while keeping anything beyond that
opaque. This is not the clean solution you'd like, but it comes without any
overhead and does not really break the abstraction.
It seems more adequate to push the somewhat intricate mechanics
for the "fall back" onto generic commands down into the implementation
level of CommandInstanceManager. The point is, we know the standard
usage situation is to rely on the instance manager, and thus we want
to avoid redundant table lookups, only to support the rare case of
fallback to global commands. The latter is currently used only from
unit-tests, but might in future also be used by scripts.
Due to thread safety considerations, I have refrained from handing
out a direct reference to the command token sitting in the registry,
even while not doing so incurs a small runtime penalty (accessing
the shared ref-count for creating a copy of the smart-handle).
This is the typical situation where you'd be tempted to sacrifice
sanity for the sake of an imaginary performance benefit, which
in fact is dwarfed by all the machinery of UI-Bus and argument
passing via GenNode.
obsoleted by C++11
* in most cases, it can be replaced by an explicit conversion operator
* especially for the Lumiera Forward Iterators, we need an implicit conversion
...since there is not any test coverage for this trait, which
turned out to be quite deeply rooted in the system by now and
handles several rather subtle special cases
This is a little bit of functionality needed again and again;
first I thought to use the TypedCounter, but this would be overkill,
since we do not actually need different instances, and we do not need
to select by type when incrementing the counter. In fact, we do not
even need anything beyond just allocating a number.
So I made a new class, which can be used RAII style
incidentally, this uncovered yet another unwanted narrowing conversion,
namely from double via gavl_time_t to TimeValue or alternatively
from double via FSecs (= rational<long>) to Duration.
As in all the previos cases, actually the compiler is to blame,
and GCC-5 is known to get that one right, i.e. let the SFINAE fail
instead of passing it with a "narrowing conversion" warning.
Note: the real test for command binding with immutable types
can be found in BusTerm_test
very similar to boost::irange, but without heavyweight boost
includes, and moreover based on our Lumiera Forward Iterator concept
Such a inline-range construct makes writing simple tests easy
without that check, in theory our test runner will tolerate
a non-zero return value, like throwing or failing an assert,
which is not what we want....
guess these happenend to get in by forgetting to
add this check when switching a test from PLANNED to TEST
due to the new automatic string conversion in operator<<
the representation of objects has changed occasionally.
I've investigated and verified all those incidents.
- the tests covering threadind support and object monitors
are located in the backend test-library and linked against liblumierabackend.so
- some fundamental facilities of proc-layer moved from the library tree
into the basic components tree, since *testing* them requires at least
to link against liblumieracommon.so
