The intention is to augment the iterator based (linear) search
used in EventLog to allow for real backtracking, based on a evaluation tree.
This should be rather staight forward to implement, relying on the
exploreChildren() functionality of TreeExplorer. The trick is to package
the chained search step as a monadic flatMap operation
we did an unnecessary copy of the argument, which was uncovered
by the test case manipulating the state core.
Whew.
Now we have a beautiful new overengineered solution
outift the Filter base class with the most generic form of the Functor
wrapper, and rather wrap each functor argument individually. This allows
then to combine various kinds of functors
...this solution works, but has a shortcoming:
the type of the passed lambdas is effectively pinned to conform
with the signature of the first lambda used initially when building the filter.
Well, this is the standard use case, but it kind of turns all the
tricky warpping and re-binding into a nonsense excercise; in this form
the filter can only be used in the monadic case (value -> bool).
Especially this rules out all the advanced usages, where the filter
collaborates with the internals of the source.
while this is basically just code code cosmetics,
at least it marks this as a very distinct special case,
and keeps the API for the standard Filter layer clean.
a quite convoluted construct built from several nested generic lambdas.
When investigated in the debugger, the observed addresses and the
invoked code looks sane and as expected.
The intention is to switch from the itertools-based filter
to the filter available in the TreeExplorer framework.
Thus "basically" we just need to copy the solution over,
since both are conceptually equivalent.
However...... :-(
The TreeExplorer framework is designed to be way more generic
and accepts basically everything as argument and tries to adapt apropriately.
This means we have to use a lot of intricate boilerplate code,
just to get the same effect that was possible in Itertools with
a simple and elegant in-place lambda assignment
Fillter needs to be re-evaluated, when an downstream entity requests
expandChildren() onto an upstream source. And obviously the ordering
of the chained calls was wrong here.
As it turns out, I had discovered that necessity to re-evaluate with
the Transformer layer. There is a dedicated test case for that, but
I cut short on verifying the filter in that situation as well, so
that piece of broken copy-n-paste code went through undetected.
This is in fact a rather esoteric corner case, because it is only
triggered when the expandChildren() call is passed through the filter.
When otoh the filter sits /after/ the entity generating the expandChildren()
calls, everything works as intended. And the latter is the typical standard
usage situation of an recursive evalutation algorithm: the filter is here
used as final part to drive the evaluation ahead and pick the solutions.
There is a bug or shortcoming in the existing ErrorLog matcher implementation.
It is not really difficult to fix, however doing so would require us to intersperse
yet another helper facility into the log matcher. And it occurred to me, that
this helper would effectively re-implement the stack based backtracking ability,
which is already present in TreeExplorer (and was created precisely to support
this kind of recursive evaluation strategies).
Thus I intend to switch the implementation of the EventLog matcher from the
old IterTool framework to the newer TreeExplorer framework. And this intention
made me re-read the code, fixing several comments and re-thinking the design
seemingly my quick-n-dirty implementation was to naiive.
We need real backtracking, if we want to support switches
in the search direction (match("y").after("x").before("z")
Up to now, I have cheated myself around this obvious problem :-/
The boost::hash documentation does not mention a significant change in that area,
yet the frequent collisions on identifiers with number suffix do not occur anymore
in Boost 1.65
- state-of-the-art implementation of access with Double Checked Locking + Atomics
- improved design for configuration of dependencies. Now at the provider, not the consumer
- support for exposing services with a lifecycle through the lib::Depend<SRV> front-end
...which declare DependencyFactory as friend.
Yes, we want to encourrage that usage pattern.
Problem is, std::is_constructible<X> gives a misleading result in that case.
We need to do the instantiation check within the scope of DependencyFactory
ideally we want
- just a plain unique_ptr
- but with custom deleter delegating to lib::Depend
- Depend can be made fried to support private ctor/dtor
- reset the instance-ptr on deletion
- always kill any instance
all these tests are ported by drop-in replacement
and should work afterwards exactly as before (and they do indeed)
A minor twist was spotted though (nice to have more unit tests indeed!):
Sometimes we want to pass a custom constructor *not* as modern-style lambda,
but rather as direct function reference, function pointer or even member
function pointer. However, we can not store those types into the closure
for later lazy invocation. This is basically the same twist I run into
yesterday, when modernising the thread-wrapper. And the solution is
similar. Our traits class _Fun<FUN> has a new typedef Functor
with a suitable functor type to be instantiated and copied. In case of
the Lambda this is the (anonymous) lamda class itself, but in case of
a function reference or pointer it is a std::function.
...written as byproduct from the reimplementation draft.
NOTE there is a quite similar test from 2013, DependencyFactory_test
For now I prefer to retain both, since the old one should just continue
to work with minor API adjustments (and thus prove this rewrite is a
drop-in replacement).
On the long run those two tests could be merged eventually...
...but not yet switched into the main LocationQuery interface,
because that would also break the existing implementation;
recasting this implementation is the next step to do....
...which basically allows us to return any suitable implementation
for the child iterator, even to switch the concrete iteration on each level.
We need this flexibility when implementing navigation through a concrete UI
...at least when using a wrapped Lumiera Iterator as source.
Generally speaking, this is a tricky problem, since real mix-in interfaces
would require the base interface (IterSource) to be declared virtual.
Which incurres a performance penalty on each and every user of IterSource,
even without any mix-in additions. The tricky part with this is to quantify
the relevance of such a performance penalty, since IterSource is meant
to be a generic library facility and is a fundamental building block
on several component interfaces within the architecture.
This is just a temporary solution, until IterSource is properly refactored (#1125)
After that, IterSource is /basically a state core/ and the adaptor will be more or less trivial
- as it stands currently, IterSource has a design problem, (see #1125)
- and due to common problems in C++ with mix-ins and extended super interfaces,
it is surprisingly tricky to build on an extension of IterSource
- thus the idea is to draft a new solution "in green field"
by allowing TreeExplorer to adapt IterSource automatically
- the new sholution should be templated on the concrete sub interface
and ideally even resolve the mix-in-problem by re-linearising the
inheritance line, i.e. replace WrappedLumieraIter by something
able to wrap its source, in a similar vein as TreeExplorer does
several extensions and convenience features are conceivable,
but I'll postpone all of them for later, when actual need arises
Note especially there is one recurring design challenge, when creating
such a demand-driven tree evaluation: more often than not it turns out
that "downstream" will need some information about the nested tree structure,
even while, on the surfice, it looks as if the evaluation could be working
completely "linearised". Often, such a need arises from diagnostic features,
and sometimes we want to invoke another API, which in turn could benefit
from knowing something about the original tree structure, even if just
abstracted.
I have no real solution for this problem, but implementing this pipeline builder
leads to a pragmatic workaround: since the iterator already exposes a expandChildren(),
it may as well expose a depth() call, even while keeping anything beyond that
opaque. This is not the clean solution you'd like, but it comes without any
overhead and does not really break the abstraction.
...so sad.
The existing implementation was way more elegant,
just it discarded an exahusted parent element right while in expansion,
so effectively the child sequence took its place. Resolved that by
decomposing the iterNext() operation. And to keep it still readable,
I make the invariant of this class explicit and check it (which
caught yet another undsicovered bug. Yay!)
instead of building a very specific collaboration,
rather just pass the tree depth information over the extended iterator API.
This way, "downstream" clients *can* possibly react on nested scope exploration
We get conflicting goals here:
- either the child expansion happens within the opaque source data
and is thus abstracted away
- or the actual algorithm evaluation becomes aware of the tree structure
and is thus able to work with nested evaluation contexts and a local stack
...build on top of the core features of TreeExplorer
- completely encapsulate and abstract the source data structure
- build an backtracking evaluation based on layered evaluation
of this abstracted expandable data source
NOTE: test passes compilation, but doesn't work yet
...and there is a point where to stop with the mere technicalities,
and return to a design in accordance with the inner nature of things.
Monads are a mere technology, without explicatory power as a concept or pattern
For that reason
- discard the second expansion pattern implemented yesterday,
since it just raises the complexity level for no given reason
- write a summary of my findings while investigating the abilities
of Monads during this design excercise.
- the goal remains to abandon IterExplorer and use the now complete
IterTreeEplorer in its place. Which also defines roughly the extent
to wich monadic techniques can be useful for real world applications
...it can sensibly only be done within the Expander itself.
Question: is this nice-to-have-feature worth the additional complexity
of essentially loading two quite distinct code paths into a single
implementation object?
As it stands, this looks totally confusing to me...
At that time, our home-made Tuple type was replaced by std::tuple,
and then the command framework was extended to also allow command invocation
with arguments packaged as lib::diff::Record<GenNode>
With changeset 0e10ef09ec
A rebinding from std::tuple<ARGS...> to Types<ARGS> was introduced,
but unfortunately this was patched-in on top of the existing Types<ARGS...>
just as a partial specialisation.
Doing it this way is especially silly, since now this rebinding also kicks
in when std::tuple appears as regular payload type within Types<....>
This is what happened here: We have a Lambda taking a std::tuple<int, int>
as argument, yet when extracting the argument type, this rebinding kicks in
and transforms this argument into Types<int, int>
Oh well.
this leads to either unfolding the full tree depth-first,
or, when expanding eagerly, to delve into each sub-branch down to the leaf nodes
Both patterns should be simple to implement on top of what we've built already...
IterSource should be refactored to have an iteration control API similar to IterStateWrapper.
This would resolve the need to pass that pos-pointer over the abstraction barrier,
which is the root cause for all the problems and complexities incurred here
turns out that -- again -- we miss some kind of refresh after expanding children.
But this case is more tricky; it indicates a design mismatch in IterSource:
we (ab)use the pos-pointer to communicate iteration state. While this might be
a clever trick for iterating a real container, it is more than dangerous when
applied to an opaque source state as in this case. After expanding children,
the pos-pointer still points into the cache buffer of the last transformer.
In fact, we miss an actualisation call, but the IterSource interface does not
support such a call (since it tries to get away with state hidden in the pos pointer)
this was a design decision, but now I myself run into that obvious mistake;
thus not sure if this is a good design, or if we need a dedicated operation
to finish the builder and retrieve the iterable result.
as it turned out, when "inheriting" ctors, C++14 removes the base classes' copy ctors.
C++17 will rectify that. Thus for now we need to define explicitly that
we'll accept the base for initialising the derived. But we need do so
only on one location, namely the most down in the chain.
Since this now requires to import iter-adapter-stl.hpp and iter-source.hpp
at the same time, I decided to drop the convenience imports of the STL adapters
into namespace lib. There is no reason to prefer the IterSource-based adapters
over the iter-adapter-stl.hpp variants of the same functionality.
Thus better always import them explicitly at usage site.
...actual implementation of the planned IterSource packaging is only stubbed.
But I needed to redeclare a lot of ctors, which doesn't seem logical
And I get a bad function invocation from another test case which worked correct beforehand.
due to switching from ADL extension points to member functions,
we now need to detect a "state core" type in a different fashion.
The specific twist is that we can not spell out the full signature
in all cases, since the result type will be formed as a consequence
of this type detection. Thus there are now additional detectors to
probe for the presence of a specific function name only, and the
distinction between members and member functions has been sharpened.
Considering the fact that we are bound to introduce yet another iteration control function,
because there is literally no other way to cause a refresh within the IterTreeExplorer-Layers,
it is indicated to reconsider the way how IterStateWrapper attaches to the
iteration control API.
As it turns out, we'll never need an ADL-free function here;
and it seems fully adequate to require all "state core" objects to expose
the API as argument less member function. Because these reflect precisely
the contract of a "state core", so why not have them as member functions.
And as a nice extra, the implementation becomes way more concise in
all the cases refactored with this changeset!
Yet still, we stick to the basic design, *not* relying on virtual functions.
So this is a typical example of a Type Class (or "Concept" in C++ terminology)
good news: it (almost) works out-of-the-box as expected.
There is only one problem: expandChildren() alters the content of the
data source, yet downstream decorators aren't aware of that fact and
continue to present cached evaluations, until the next iterate() call
is issued. Yet unfortunately this iterate already consumes the first
of the expanded children, which thus gets shadowed by the cached
outcome of parent node already consumed and expanded at that point
See the first example:
"10-8-expand-8-4-2-6-4-2"
should be 6 ^^^
...which happens to be supported out of the box,
due to the generic adaptor magic shared with the explore-operation
Exploiting this feature, some functor could even subvert the layering order
- always layer the TreeExplorer (builder) on top of the stack
- always intersperse an IterableDecorator in between adjacent layers
- consequently...
* each layer implementation is now a "state core"
* and the source is now always a Lumiera Iterator
This greatly simplifies all the type rebindings and avoids the
ambiguities in argument converison. Basically now we can always convert
down, and we just need to pick the result type of the bound functor.
Downside is we have now always an adaptation wrapper in between,
but we can assume the compiler is able to optimise such inline
accessors away without overhead.
...yet this seems like a rather bad idea,
it breeds various problems and requires arcane trickery to make it fly
==> abandon this design
==> always intersperse an IterableDecorator between each pair of Layers
...especially relevant in the context of TreeExplorer,
where the general understanding is that the "Data Source" (whatever it is)
will be piggy-backed into the pipeline builder, and this wrapping is
conceived as being essentially a no-op.
It is quite possible we'll even start using such pipeline builders
in concert with move-only types. Just consider a UI-navigator state
hooked up with a massive implementation internal pointer tree attached
to all of the major widgets in the UI. Nothing you want to copy in passing by.
As it turned out, we had two bugs luring in the code base,
with the happy result of one cancelling out the adverse effects of the other
:-D
- a mistake in the invocation of the Itertools (transform, filter,...)
caused them to move and consume any input passed by forwarding, instead
of consuming only the RValue references.
- but util::join did an extraneous copy on its data source, meaning that
in all relevant cases where a *copy* got passed into the Itertools,
only that spurious temporary was consumed by Bug #1.
(Note that most usages of Itertools rely on RValues anyway, since the whole
point of Itertools is to write concise in-line transformation pipelines...)
*** Added additional testcode to prove util::stringify() behaves correct
now in all cases.
- we do strip references
- we delegate to nested typedefs
Hoever, we do *not* treat const or pointers in any way special --
if the user want to strip or level these, he has to do so explicitly.
Initially it seemed like a good idea to do something clever here, but
on the long run, such "special treatment" is just good for surprises
...automatically whenever those are present.
Up to now, we hat that as base case, which limited usage to those cases
where we already know such nested definitions are actually present
attempt to re-use the same traits as much as possible
NOTE: new code not passing compiler yet, but refactored old code
does, and still passes unit test
...which uncovered an error in the test fixture
plus helped to spot the spurious copy when passing the argument to the expand functor
And my GDB crashed when loading the executable, YAY!
so we'll need to coment out some code from now on,
until we're able to switch to a more recent toolchain (#1118)
but possible only for the iterator -> iterator case
Since we can not "probe" a generic lambda, we get only one shot:
we can try to bind it into a std::function with the assumed signature
This is a consequence of the experiments with generic lambdas.
Up to now, lib::meta::_Fun<F> failed with a compilation error
when passing the decltype of such a generic lambda.
The new behaviour is to pick the empty specialisation (std::false_type) in such cases,
allowing to guard explicit specialisations when no suitable functor type
is passed
this solution makes me feel somewhat queasy..
stacking several adaptors and wrappers and traits on top of each other.
Well, it type checks and passes the test, so let's trust functional programming
The plan is to use a monad-like scheme, but allow for a lot of leeway
with respect to the src and value types of the expand functor.
A key idea is to allow for a *different* state core than used in the source
The key trick is to form an expression with the free function, using a declval of the type to probe.
What is somewhat tricky is the fact that functions can be void, so we need just to pick up
the type and use it in another type expression
Here, the tricky question remains, how to relate this evalutaion scheme
to the well known monadic handling of collections and iterators.
It seems, we can not yet decide upon that question, rather we should
first try to build a concrete implementation of the envisioned algorithm
and then reconsider the question later, to what extent this is "monadic"
This can be seen as a side track, but the hope is
by relying on some kind of monadic evaluation pattern, we'll be
able to to reconcile the IterExplorer draft from 2012 with the requirement
to keep the implementation of "tree position" entirely opaque.
The latter is mandatory in the use case here, since we must not intermingle
the algorithm to resolve UI-coordinates in any way with the code actually
navigating and accessing GTK widgets. Thus, we're forced to build some kind
of abstraction barrier, and this turns out to be surprisingly difficult.
...which can be helpful when a function usually returns a somewhat dressed-up iterator,
but needs to return a specific fixed value under some circumstances
- fix some warnings due to uninitialised members
(no real problem, since these members get assigned anyway)
- use a lambda as example function right in the test
- use move initialisation and the new util::join
Explicitly assuming that those functions are called solely from IterAdapter
and that they are implemented in a typical standard style, we're able to elide
two redundant calls to the checkPoint() function. Since checkPoint typically performs
some non-trivial checks, this has the potential of a significant performance improvement
- we check (and throw ITER_EXHAUST) anyway from operator++, so we know that pos is valid
- the iterate() function ensures checkPoint is invoked right after iterNext,
and thus the typical standard implementation of iterNext need not do the same
...under the assumption that the content is normalised,
which means
- leading NULL is changed to Symbol::EMPTY
- missing elements in the middle are marked as "*"
- trailing NULL in extension storage is handled by adjusting nominal extension size
as it turned out, the solution from yesterday works only with uniform argument lists,
but not with arbitrarily mixed types. Moreover the whole trickery with the
indices was shitty -- better use a predicate decision on template argument level.
This simple solution somehow just didn't occur to me...
exploring the idea of a configuration DSL.
As a first step, this could be a simple internal DSL,
implemented as a bunch of static functor objects, which are internally bound
and thus implemented by the ViewLocator within InteractionDirector
again surprising how such fundamental bugs can hide for years...
Here the reason is that IterAdapter leaves the representation of "NIL" to
its instantiation / users; some users (here in for example the ScopedCollection)
can choose to allow for different representations of "NIL", but the comparison
provided by IterAdapter just compares the embedded pos by face value.
...while tests in the library subdirectory are linked only against
liblumierasupport, which does not provide the multithreading support
In this special case here the actual facility to be tested does not rely
on thread support, only on locking. But the stress test obviously needs
to create several threads. Simple workaround is to move the test into
a test collection linked against all of the application core...
for the simplistic implementation we're using right now this effort might look
exaggerated, but we should consider using a lock-free implementation at some
point in the future, at which point it is good to have a stress test in place
- concept for a first preliminary implementation of dispatch into the UI thread
- define an integration effort to build a complete working communication chain
Up to now, we tolerated null pointers in Literal instances.
But we can not tolerate passing a null cString to Symbol initialisation.
Rather, hereby we introduce a dedicated "bottom" Symbol, a valid "null object"
...which means, from now on identical input strings
will produce the same Symbol object (embedded pointer).
TODO: does not handle null pointers passed in as c-String properly
obsoleted by C++11
* in most cases, it can be replaced by an explicit conversion operator
* especially for the Lumiera Forward Iterators, we need an implicit conversion
This changeset fixes a huge pile of problems, as indicated in the
error log of the Doxygen run after merging all the recent Doxygen improvements
unfortunately, auto-linking does still not work at various places.
There is no clear indication what might be the problem.
Possibly the rather unstable Sqlite support in this Doxygen version
is the cause. Anyway, needs to be investigated further.
...since there is not any test coverage for this trait, which
turned out to be quite deeply rooted in the system by now and
handles several rather subtle special cases
this bit of Sed magic relies on the fact that we happen to write
the almost correct class name of a test into the header comment.
HOWTO:
for F in $(find tests -type f \( -name '*.cpp' \) -exec egrep -q '§§TODO§§' {} \; -print);
do sed -r -i -e'
2 {h;x;s/\s+(.+)\(Test\).*$/\\ref \1_test/;x};
/§§TODO§§/ {s/§§TODO§§//;G;s/\n//}'
$F;
done
