Especially for the BlockFlow allocator, sanity checks are elided
for performance reasons; yet, generally speaking, it can be a very bad idea
to "optimise" away sanity checks. Thus an additional adaptor is provided
to layer such checks on top of an existing core; and IterEplorer now
always wires in this additional adaptor, and so the original behaviour
is now restored in this respect (and for the largest part of the code base)
While at first sight just a superficial variation of the existing IterStateWrapper,
it became clear with the evolution of the IterExplorer framework that
this setup represents a distinct concept, and especially lends itself
for complex and cohesive collaboration in a layered pipeline. Which
may, or may not be a good idea, depending on the circumstances.
Now, for the implementation of the scheduler memory allocation scheme,
another twist is added to the picture: we can not effort the sanity checks
on each access, even more so when layering / adapting iterators, where
it is essential that the optimiser can remove all unnecessary warts.
Library: add "obvious" utility to the IterExplorer, allowing to
materialise all contents of the Pipeline into a container
...use this to take a snapshot of all currently active Extent addresses
The second design from 2017, based on a pipeline builder,
is now renamed `TreeExplorer` ⟼ `IterExplorer` and uses
the memorable entrance point `lib::explore(<seq>)`
✔
after completing the recent clean-up and refactoring work,
the monad based framework for recursive tree expansion
can be abandoned and retracted.
This approach from functional programming leads to code,
which is ''cool to write'' yet ''hard to understand.''
A second design attempt was based on the pipeline and decorator pattern
and integrates the monadic expansion as a special case, used here to
discover the prerequisites for a render job. This turned out to be
more effective and prolific and became standard for several exploring
and backtracking algorithms in Lumiera.
- decision: the Monad-style iteration framework will be abandoned
- the job-planning will be recast in terms of the iter-tree-explorer
- job-planning and frame dispatch will be disentangled
- the Scheduler will deliberately offer a high-level interface
- on this high-level, Scheduler will support dependency management
- the low-level implementation of the Scheduler will be based on Activity verbs
Considering the fact that we are bound to introduce yet another iteration control function,
because there is literally no other way to cause a refresh within the IterTreeExplorer-Layers,
it is indicated to reconsider the way how IterStateWrapper attaches to the
iteration control API.
As it turns out, we'll never need an ADL-free function here;
and it seems fully adequate to require all "state core" objects to expose
the API as argument less member function. Because these reflect precisely
the contract of a "state core", so why not have them as member functions.
And as a nice extra, the implementation becomes way more concise in
all the cases refactored with this changeset!
Yet still, we stick to the basic design, *not* relying on virtual functions.
So this is a typical example of a Type Class (or "Concept" in C++ terminology)
This can be seen as a side track, but the hope is
by relying on some kind of monadic evaluation pattern, we'll be
able to to reconcile the IterExplorer draft from 2012 with the requirement
to keep the implementation of "tree position" entirely opaque.
The latter is mandatory in the use case here, since we must not intermingle
the algorithm to resolve UI-coordinates in any way with the code actually
navigating and accessing GTK widgets. Thus, we're forced to build some kind
of abstraction barrier, and this turns out to be surprisingly difficult.
after looking into our various iterator tools,
it seems obvious that our filtering iterator implementation
has almost all of the required behaviour; we only need to
add a hook to rewrite and extend the filtering functor,
which can now nicely done with a lambda closure.
This means all memory management, if necessary, is
pushed into std::function and the automated memory
management for closures provided by the runtime.
Since C++ is not a real functional programming language and
has unsafe unmanaged pointers, it is not difficult to produce
dangling references within an extended evaluation pipeline
involving transient objects and pass-by-reference.
In the initial implementation, I built in a safeguard copy
into the signature of the Explorer function, to make sure even
a transiently dressed-up input value gets materialised before
proceeding with the source sequence. Unfortunately this safeguard
turns out as a roadblock now; we might as well take the input
by reference and return an "expanded" state by value. We might
even want to do the full "expansion" on referred state, when
we're able to ensure the source values remain in memory
until consumption.
Thus now the full power of decision is placed on the signature
of the explorer function. The expansion strategies of IterExplorer
will no longer attempt to "sanitise" the signature of the passed-in
function to prevent desaster; I've added some warnings into the
documentation to highlight that danger. Basically, if you want
to be clever, then you're bound to read and understand inticacies
of the implementation.
If in doubt, use values and copying. C++ is optimised for that.
remembered that some years ago I had to deal with a very similar problem
for planning the frame rendering jobs. It turned out, that the
iterator monad developed for this looks promising for our task at hand
...for the very specific situation when we want
to explore an existing data structure, and the
exploration assumes value semantics.
The workaround then is to use pointers as values.
...attempt to build it based on the monadic iterator primitives.
Only problem is: need to find out relation between nodes
after the fact. In the real usage situation, this
is not a problem, since we have a state object
there, which can track the relation as it is established
especially the exploration stack is pushed down
first successful definition of all the JobPlanning classes
just the framework of classes necessary to pass the compiler;
all implementation is still stubbed
brainstorming how to implement the job planning stage
the idea is to built on top of the IterExplorer,
but have the "stack" of re-evaluation integrated
into a custom type, which exploits the static
node network structure to avoid heap allocations
solution idea: again use a builder function?
the template _Fun started as an internal helper
for function-closure, but seems to be of
general use. Thus move it into meta/function.hpp
(function-closure.hpp is heavyweight)
this enables expansion of a (functional) data structure
until exhaustion -- which is what we need to
build job functors by traversing and expanding
an arbitrarily nested job definition structure
the intention is to use this to simplify
generating render jobs based on the elaborated
dependency network of the render nodes. The key
challenge is to overcome the necessity to
store partially done evaluations as
continuation
the tricky part seems to be how to combine the
source iterators into a new monad instance, while
keeping this "Combinator" Strategy configurable
...just passes the compiler, while still lacking
even the generic implementation of joining
together the source iterators
The idea is to avoid building a data structure
for intermediary results, while still being able
to process a variably sized and arbitrary shaped
set of source data
