After the leftovers of the first Render-Engine implementation attempt were removed,
only one further usage of `RefArray` remains to be sorted out: the ''Session Element Tracker''.
Luckily, this one did not actually make any use of the abstraction abilities
of the `RefArray` — rather it basically stated that ''the interface is a data structure...''
After considering ''what kind of data'' can be expected to live in this structure,
it became clear that ''this will be a symbolic representation''
And thus the container can be simply switched to a `std::vector`.
This change allows to retain the existing placeholder-implementation unaffected,
while it would be possible to maintain algebraic terms here, in future.
__As an asside__: in order to decide about a suitable replacement in the Session,
I had to consier a first draft regarding the intended usage
and the prospective way of content representation
Investigated this topic again...
* these were initially created before C++11
* at that time, ''non-copyable'' objects were not common place
* but we embraced that concept already, and thus had quite some pain
when attempting to use such objects in STL containers
* with C++11 and ''move semantics'' these problems basically evaporated
* most usages were already upgraded and resolved
* another use case is to handle a state variable, which is based on
an immutable entity (like Time entities); `ItemWrapper` can be used
as a remedy in such a situation
Now looking into largely obsolete library facilities...
Starting from `ScopedHolder`, I found a surprising problem with ''perfect forwarding....''
...which however turned out to be the result of ''sloppy programming'' on my side.
At that time, in 2017, seemingly I was under pressure to define a Session-Command,
which captures a Time-entity as »State Memento«. Which turned out to be impossible,
since the Time entities were conceived as being immutable -- a questionable design
decision (see #1261), which already generated quite some additional complexities.
In the course of this »exercise«, I could again clarify that the implementation
of perfect forwarding works as expected on modern compilers — confusion may arrise
sometimes due to ''copy elision'' (which the compiler is required to perform
since C++17, even when the elided constructor has observable side effects).
And it can be derailed when (as was the case here) a »grab everything« constructor
accidentally ends up generating a copy- or move-constructor for the container class
itself. This is an instance of the problem documented with #963 ...
.... and the best solution is to abide by the rule-of-five (and a half)
and to put that `ReplacableItem` to where it belongs...
...to be more compliant to the »Lumiera Forward Iterator« concept.
This can be easily achieved by inheriting from util::RegexSearchIter,
similar to the example in CSV.hpp
Regarding #896, I changed the string rendering to use fs::path::generic_string
where appropriate, which means that we're using the normalised path rendering
* the implementation of this ''Sum Type'' got quite technical and complicated;
thus better to be extracted as separate library component
* use this as base for the `AltModel`
* make a usage sketch, invoking only the model interactions required
Unfortunately, there are some common syntactic structures, which can not easily be dissected by regular expressions alone, since they entail nested subexpressions. While it is possible to get beyond those fundamental limitations with some trickery, doing so remains precisely that, ''trickery.''
After fighting some inner conflicts, since ''I do know how to write a parser'' —
in the end I have brought myself to just do it.
And indeed, as you'd might expect, I have looked into existing library solutions,
and I would not like to have any one of them as part of the project.
* I do not want a ''parser engine'' or ''parser generator''
* I want the directness of recursive-descent, but combined with Regular Expressions as terminal
* I want to see the structure of the used grammar at the definition site of the custom parser function
* I want deep integration of ''model bindings'' into the parse process, i.e. binding-λ
* I do not want to write model-dissecting or pattern-matching code after the parse
* I do not want to expose ''Monads'' as an interface, since they tend to spread unhealthy structure to surrounding code
* I do not want to leak technicalities of the parse mechanics into the using code
* I do not want to impose hard to remember specific conventions onto the user
Thus I've set the following aims:
* The usage should require only a single header include (ideally header-only)
* The entrance point should be a small number of DSL-starter functions
* The parser shall be implemented by recursive-descent, using the parser-combinator technique
* But I want that wrapped into a DSL, to be able to control what is (not) provided or exposed.
* I want a stateful, applicative logic, since parsing, by its very nature, is stateful!
* I want complete compile-time typing, visible to the optimiser, without a virtual »Parser« interface
And last but not least, ''I do not want to create a ticket, since I do not know if those goals can be achieved...''
The choice to rely on strictly typed functor bindings for the Node operation
bears the danger to produce ''template bloat'' — it would be dangerous to add
further functions to the Port-API naïvely; espeically simple information functions
will likely not depend on the full type information.
A remedy to explore would be to exploit properties marked into the Port's `ProcID`
as key for a dispatcher hashtable; assuming that the `NodeBuilder` will be responsible
for registering the corresponding implementation functions, such a solution could even
be somewhat type-safe, as long as the semantics of the ProcID are maintained correctly.
...as part of the rendering process, executed on top of the
low-level-model (Render Node network) as conceived thus far.
Parameter handling could be ''encoded'' into render nodes altogether,
or, at the other extreme, an explicit parameter handling could be specified
as part of the Render Node execution. As both extremes will lead to some
unfavourable consequences, I am aiming at a middle ground: largely, the
''automation computation'' will be encoded and hidden within the network,
implying that this topic remains to be addressed as part of defining
the Builder semantics and implementation. Yet in part the required
processing structure can be foreseen at an abstract level, and thus
the essential primitive operations are specified explicitly as part
of the Render Node definition. Notably the ''standard Weaving Pattern''
will include a ''parameter tuple'' into each `FeedManifold` and require
a binding function, which accepts this tuple as first argument.
Moreover — at implementation level, a library facility must be provided
to support handling of ''arbitrary heterogeneous data values'' embedded
directly into stack frame memory, together with a type-safe compile-time
overlay, which allows the builder to embed specific ''accessor handles''
into functor bindings, even while the actual storage location is not
yet known at that time (obviously, as being located on the stack).
__Note__: a recurring topic is how to return descriptor objects from builder functions; for this purpose, I am adjusting the semantics of `lib/nocopy.hpp` to be more specific...
I changed the rendering of unsigned types in diagnostic output
to use the short notation, e.g. `uint` instead of `unsigned int`.
This dramatically improves the legibility of verification strings.
Moreover, I took the opportunigy to look through the existing page
with codeing style guides to explicitly write down some conventions
formed over years of usage.
I did not just »make up« those light heartedly, rather these conventions
are the result of a craftsman's ''attentive observation and self-reflection.''
Since each `TestFrame` now has a metadata header,
we can store an additional data checksum there,
so that it is now possible both to detect if data
is in pristine state, or if it matches a changed state
recorded in the additional checksum.
So we have now three different levels of verification
isSane:: consistent metadata header found
isValid:: metadata header found and checksum there matches data
isPristine:: in addition, the data is exactly as generated from the `(frameNr,family)`
We use the memory address to detect reference to ''the same language object.''
While primarily a testing tool, this predicate is also used in the
core application at places, especially to prevent self-assignment
and to handle custom allocations.
It turns out that actually we need two flavours for convenient usage
- `isSameObject` uses strict comparison of address and accepts only references
- `isSameAdr` can also accept pointers and even void*, but will dereference pointers
This leads to some further improvements of helper utilities related to memory addresses...
At the time of the initial design attempts, I naively created a
classic interface to describe an fixed container allocated ''elsewhere.''
Meanwhile the C++ language has evolved and this whole idea looks
much more as if it could be a ''Concept'' (C++20). Moreover, having
several implementations of such a container interface is deemed inadequate,
since it would necessitate ''at least two indirections'' — while
going the Concept + Template route would allow to work without any
indirection, given our current understanding that the `ProcNode` itself
is ''not an interface'' — rather a building block.
the metafunction `is_basically<X>` performed only an equality match,
while, given it's current usage, it should also include a subtype-interface-match.
This changes especially the `is_StringLike<S>` metafunction,
both on const references and on classes built on top of string
or string_view.
A minimalist `TextTemplate` engine is available for in-project use.
* supports only the bare minimum of features (no programming language)
* substitution of `${placeholder}` by key-name data access
* conditional section `${if key}...${end if}`
* iteration over a data sequence
* other then most solutions available as library,
this implementation does **not require** a specific data type,
nor does it invent a dynamic object system or JSON backend;
rather, a generic ''Data Source Adapter'' is used, which can
be specialised to access any kind of ''structured data''
* the following `DataSource` specialisations are provided
* `std::map<string,string>`
* Lumiera »External Tree Description« (based on `GenNode`)
* a string-based spec for testing
Conducted an extended investigation regarding text templating
and the library solutions available and still maintained today.
The conclusion is
* there are some mature and widely used solutions available for C++
* all of these are considered a mismatch for the task at hand,
which is to generate Gnuplot scripts for test data visualisation
Points of contention
* all solutions offer a massive feature set, oriented towards web content generation
* all solutions provide their own structured data type or custom property-tree framework
**Decision** 🠲 better to write a minimalistic templating engine from scratch rather
...using the same method for sake of uniformity
Also move the permissions helpers to the file.hpp support functions
and setup a separate unit test for these
...to sort out the interpretation of measurement results,
the actual duration and concurrency of ComputationLoad invocations
should be recorded, allowing to draw conclusions regarding the
Scheduler's performance as opposed to further system and thread
management effects due to concurrent operation under pressure.
...so this was yet another digression, caused by the desire
somehow to salvage this problematic component design. Using a
DSL token fluently, while internally maintaining a complex and
totally open function based configuration is a bit of a stretch.
the RandomDraw rules developed last days are meant to be used
with user-provided λ-adapters; employing these in a context
of a DSL runs danger of producing dangling references.
Attempting to resolve this fundamental problem through
late-initialisation, and then locking the component into
a fixed memory location prior to actual usage. Driven by
the goal of a self-contained component, some advanced
trickery is required -- which again indicates better
to write a library component with adequate test coverage.
The idea is to use some source of randomness to pick a
limited parameter value with controllable probability.
While the core of the implementation is nothing more
than some simple numeric adjustments, these turn out
to be rather intricate and obscure; the desire to
package these technicalities into a component
however necessitates to make invocations
at usage site self explanatory.
on second thought, the ability to transport an exception still seems
worthwhile, and can be achieved by some rearrangements in the design.
As preparation, reorganise the design of the Either-wrapper (lib::Result)
- relocate some code into a dedicated translation unit to reduce #includes
- actually set the thread-ID (the old implementation had only a TODO at that point)
Using the same building blocks, this operation can be generalised even more,
leading to a much cleaner implementation (also with better type deduction).
The feature actually used here, namely summing up all values,
can then be provided as a convenience shortcut, filling in std::plus
as a default reduction operator.
after completing the recent clean-up and refactoring work,
the monad based framework for recursive tree expansion
can be abandoned and retracted.
This approach from functional programming leads to code,
which is ''cool to write'' yet ''hard to understand.''
A second design attempt was based on the pipeline and decorator pattern
and integrates the monadic expansion as a special case, used here to
discover the prerequisites for a render job. This turned out to be
more effective and prolific and became standard for several exploring
and backtracking algorithms in Lumiera.
...this is something I should have done since YEARS, really...
Whenever working with symbolically represented data, tests
typically involve checking *hundreds* of expected results,
and thus it can be really hard to find out where the
failure actually happens; it is better for readability
to have the expected result string immediately in the
test code; now this expected result can be marked
with a user-defined literal, and then on mismatch
the expected and the real value will be printed.
The header "format-cout.hpp" offers a convenience function
to print pretty much any object or data in human readable form.
However, the formatter for pointers used within this framework
switched std::cout into hexadecimal display of numbers and failed
to clean-up this state.
Since the "stickyness" of IOS stream manipulators is generally a problem,
we now provide a RAII helper to capture the previous stream state and
automatically restore it when leaving the scope.
Complete the investigation and turn the solution into a generic
mix-in-template, which can be used in flexible ways to support
this qualifier notation.
Moreover, recapitulate requirements for the ElementBoxWidget
All of the existing "simple" tests for the »Diff Framework« are way to much low-level;
they might indeed be elementary, but not introductory and simple to grasp.
We need a very simplistic example to show off the idea of mutation by diff,
and this simple example can then be used to build further usage test cases.
My actual goal for #1206 to have such a very basic usage demonstration and then
to attach a listener to this setup, and verify it is actually triggered.
PS: the name "GenNodeBasic_test" is somewhat pathetic, this test covers a lot
of ground and is anything but "basic". GenNode in fact became a widely used
fundamental data structure within Lumiera, and -- admittedly -- the existing
implementation might be somewhat simplistic, while the whole concept as such
is demanding, and we should accept that as the state of affairs
...these magical strings are already spreading dangerously throughout the code base
PS: also fixup for c6b8811af0 (broken whitespace in test definition)
The boost::hash documentation does not mention a significant change in that area,
yet the frequent collisions on identifiers with number suffix do not occur anymore
in Boost 1.65
all these tests are ported by drop-in replacement
and should work afterwards exactly as before (and they do indeed)
A minor twist was spotted though (nice to have more unit tests indeed!):
Sometimes we want to pass a custom constructor *not* as modern-style lambda,
but rather as direct function reference, function pointer or even member
function pointer. However, we can not store those types into the closure
for later lazy invocation. This is basically the same twist I run into
yesterday, when modernising the thread-wrapper. And the solution is
similar. Our traits class _Fun<FUN> has a new typedef Functor
with a suitable functor type to be instantiated and copied. In case of
the Lambda this is the (anonymous) lamda class itself, but in case of
a function reference or pointer it is a std::function.
...written as byproduct from the reimplementation draft.
NOTE there is a quite similar test from 2013, DependencyFactory_test
For now I prefer to retain both, since the old one should just continue
to work with minor API adjustments (and thus prove this rewrite is a
drop-in replacement).
On the long run those two tests could be merged eventually...
