This change demonstrates how to deal properly with possible duplicate entities
with similar symbolic ID: define a RandomID (to guarantee a distinct hash on each instance).
In the actual implementation, this should happen already within the domain model,
not when constructing the diff (obviously of course...)
This change also adds a mutation sequence to inject the actual track name
same pattern as the existing EntryID, i.e. a human readable symbol plus a hash
but the hash is just random, instead of deriving it from the symbol text.
Use case is when we explicitly need a distinct identity, even when the
human readable symbolic name is the same. Actual example: the fork root in the timeline
...it should have been explicit from start, since there is no point
in converting an EntryID into a plain flat string without further notice
this became evident, when the compiler picked the string overload on
MakeRec().genNode(specialID)
...which is in compliance to the rules, since string is a direct match,
while BareEntryID would be an (slicing) upcast. However, obviously we
want the BareEntryID here, and not an implicit string conversion,
thereby discarding the special hash value hidden within the ID
so this seems to be the better approach for dealing with this insidious problem.
In some cases -- as here most prominently with the root track within the timeline --
we have to care within the domain model to prepare unique ids even for sub objects
treated as attributes. In the actual case, without that special attention,
all timelines would hold onto an attribute "fork" with the same ID, based
on the type of the nested object plus the string "fork". Thus all root track
representations in the GUI would end up listening to the same ID on the UI-Bus...
...rather extend the "object builder" DSL notation to allow passing in a given EntryID literally.
Rationale is, we should handle the problem of unique IDs on the level of the domain model.
If we attempt to "fix" this within GenNode, the price would be to make the ETD creation stateful
this is not a problem, strictly sepaking, locally.
But it becomes a problem once the GUI uses those attribute IDs
as Element-IDs for tangible UI entities, which need to be uniquely
addressable via the UI-Bus.
An obvious solution is to inject randomness into the Attribute ID hash
other than the regular way of building an object,
we do expect a minimal structure to be sent right within the INS message.
Rationale: the standard way would allow for too much leeway and created
unwanted intermediary states. The non-standard way decided upon here
is well within the limits of our diff language
As it turns out, using the functional-notation form conversion
with *parentheses* will fall back on a C-style (wild, re-interpret) cast
when the target type is *not* a class. As in the case in question here, where
it is a const& to a class. To the contrary, using *curly braces* will always
attempt to go through a constructor, and thus fail as expected, when there is
no conversion path available.
I wasn't aware of that pitfall. I noticed it since the recently introduced
class TimelineGui lacked a conversion operator to BareEntryID const& and just
happily used the TimelineGui object itself and did a reinterpret_cast into BareEntryID
Problem is, the InteractionDirector, being the representation of the model root,
needs to manage and maintain the collection of "timelines". However, these
can not be widgets, rather, they need to attach to widgets living within
the GUI widget structure proper, i.e. within the TimelinePanel
proposed solution is to build a smart handle based on WLink,
but also delegating the DiffMutable interface
...these magical strings are already spreading dangerously throughout the code base
PS: also fixup for c6b8811af0 (broken whitespace in test definition)
As it turns out, several problems reinforce each other
- lumiera error does not properly propagate the cause message
- our test/dummy code does not check the ExecResult
- thus the exception is detected rather accidentally, when entring the next sync/wait state
- emergency shutdown is chaotic in its very nature (this is well known...)
- but especially triggerShutdown is not airtight and might die...
- causing the shutdown to hang....
And last but not least, a ZombieCheck tripwire got triggered,
but unfortunately I was unable to get hold of the zombie iteself
test_meta_markAction always produces a state mark with payload type string.
However, the model::Tangible expects a bool payload when handling the "expand" mark.
- add diagnostics to lib::variant to indicate expected and actual payload type
- attempt to fix with boost::lexical_cast; this is insufficient, since
you'd expect such a function to understand "true" and "false" etc.
Moreover, raising this exception causes emergency shutdown, which
flounders due to triggering a ZombieCheck. Interesting.
well... reduction in size of the debug build objects
turns out not to be so large as I hoped. But it is significant anyway,
about 3-4MB on the most affected test classes. Plus from now on we
do not repeat that code on other tests using the same features.
up to now, EventLog was header only, which seems to cause
a significant bloat in terms of generated code size, especially
in debug builds. One major source for this kind of "template bloat"
is the IterChainSearch, rsp. the filter and transformer iterators.
And since EventLog is not meant for performance critical application code,
but rather serves as helper for writing unit tests, an obvious remedy is
to move that problematic part of the code down into a dedicate translation
unit, instead of using inline functions. To prepare this refactoring,
some var arg (templated) API funcitons need to be segregated.
For the before / after chaining search functions,
we now do one single step in the respective direction before evaluating
the new (next) filter condition. However, we also need to *deactivate* the
previous condition, otherwise that single "step" might cause us to jump
or even exhaust the underlying filter, due to the old filter condition
still being applied.
due to the lack of real backtracking, the existing solution
relied on a quirk, and started the before / after chained search
conditions /at/ the current element, not after / before it.
Now we're able to remove this somewhat surprising behaviour, yet to do so
we also need to introduce basic "just search" variations of all search
operations, in order to define the initial condition for a chained search.
Without that, the first condition in a chain would never be able to
match on the header entry of the log
- need to use dedicated steps in the chain for every added condition now
- seems to break the logic on tests on non-match.
This doesn't come as a surprise, since backtracking can be expected
to reveal additional solutions.
NOTE: some tests broken, to be investigated
est-event-log-test.cpp:228: thread_1: verify_callLogging: (log.ensureNot("fun").after("fun").after("fun2"))
...which can be achieved by checking the backtracking loop
always right after the non-backtracking iteration, exploiting
the fact that the guard conditions of both are complimentary.
So the only case when we'd actually enter the backtracking
loop after regular iteration would precisely be when
we drop down due to exahausting an upper layer.
The result now reads
"sausage-bacon-tomato-and-spam-spam-bacon-spam-tomato-and-spam-bacon-tomato-and-bacon-tomato-and-tomato-and"
...which sounds correct, yay!
...since usually such evaluation layers are finally wrapped into
an IterableDecorator and then presented as TreeEplorer -- an exercise
we do not want to perform here, since it is pointless in the typicall
use case. The IterChainSearch is already meant to be ready-for-use.
Thus, instead of wrapping again, the pragmatic solution is simply
to overload the missing operator++ and make it call the augmented
iterNext() function. Related to this, we also need to ensure
proper operation in case no further expansion is mandated
...seems basically sane now.
Just we still need to wrap it one more time into IterableDecorator;
which means the overall scheme how to build and package the whole pipeline
is not correct yet.
Maybe it is not possible to get it packaged all into one single class?
on closer investigation it turned out that the logic of the
first design attempt was broken altogether. It did not properly
support backtracking (which was the reason to start this whole
exercise) and it caused dangling references within the lambda
closure once the produced iterator pipeline was moved out
into the target location.
Reasoning from first principles then indicated that the only sane
way to build such a search evaluation component is to use *two*
closely collaborating layers. The actual filter configuration
and evaluation logic can not reside and work from within the
expander. Rather, it must sit in a layer on top and work in
a conventional, imperative way (with a while loop).
Sometimes, functional programming is *not* the natural way
of doing things, and we should then stop attempting to force
matters against their nature.
this is an rather obvious extension to the TreeExplorer framework.
In some cases, client code wants to define its own very specific
processing layers, beyond of what can be done with filters and
transformers. Obviously, writing such a custom layer requires
intimate knowledge about the internals of TreeExplorer
the actual use case prompting this extension is IterChainSearch;
it turned out that the original design can not be implemented,
unless the resulting object is non-copyable (which violates
the basic traits of a TreeExplorer based pipeline).
Up to now, we had a very simplistic configuration option just
to search for a match, and we had the complete full-blown reconfiguration
builder option, which accepts a functor to work on and reconfigure the
embedded Filter chain.
It occurred to me that in many cases you'd rather want some intermediary
level of flexibility: you want to replace the filter predicate entirely
by some explicitly given functor, yet you don't need the full ability
to re-shape the Filter chain as a whole. In fact the intended use case
for IterChainSearch (which is the EventLog I am about to augment with
backtracking capabilities) will only ever need that intermediate level.
Thus wer're adding this intermediary level of configurability now.
The only twist is that doing so requires us to pass an "arbitrary function like thing"
(captured by universal reference) through a "layer of lambdas". Which means,
we have to capture an "arbitrary thingie" by value.
Fortunately, as I just found out today, C++14 allows something which comes
close to that requirement: the value capture of a lambda is allowe to have
an intialiser. Which means, we can std::forward into the value captured
by the intermediary lambda. I just hope I never need to know or understand
the actual type this captured "value" takes on.... :-)
with the augmented TreeExplorer, we're now able to get rid of the
spurious base layer, and we're able to discard the filter and
continue with the unfiltered sequence starting from current position.
build a special feature into the Explorer component of TreeExplorer,
causing it to "lock into" the current child sequence and discard
all previous sequences from the stack of child explorations
There is an asymetry, insofar the base layer configuration is
evaluated immediately, causing the MutableFilter to be reconfigured
and forwarded to the first match.
to the contrary, when configuring an additional layer, we just
add it to the chain, but then need to iterate once to cause
this configuration actually to be unfolded onto the stack
...which just turns the pipeline into exhausted state,
instead of raising an Assertion failure
The point is, expandChildren() does not guard itself,
since it _requires_ an non-empty iterator as precondition.
Thus, any function downstream, which invokes expandChildren(),
has to check and guard this call apropriately.
In the concrete case at hand we just stop adding further constraints
when the pipeline is already in exhausted state
...the solution built thus far was logically broken, since it retained the unfiltered
source sequence within the base layer. Thus it would backtrack into this unfiltered
sequence eventually.
The idea was to build a special treatment for attaching the first filter condition;
in fact the first one does not need to be added to the chain, but rather should be
planted directly into the base layer.
WIP: this is a solution draft, but does not work yet
- when attaching the base layer, the filter is pulled twice
- an overconstrained filter raises an Assertion failure
(instead of just transitioning into exhausted state)
So we have now a reworked version of the internals of TreeExplorer in place.
It should be easier to debug template instantation traces now, since most
of the redundancy on the type parameters could be remove. Moreover, existing
pipelines can now be re-assigned with similarily built pipelines in many cases,
since the concrete type of the functor is now erased.
The price tag for this refactoring is that we have now to perform a call
through a function pointer on each functor invocation (due to the type erasure).
And seemingly the bloat in the debugging information has been increased slightly
(this overhead is removed by stripping the binary)
Here the design trardeoff becomes clearly visiblie
- on the plus side, we removed that spurous redundant info
from the template parameter, and we simplified functor rebinding
- but as a tradeoff, we now always have two std::function objects
nested into each other, which also means that at least the outer
object resides on the heap and /inevitably/ calls through a
function pointer, even in case the target function is a lambda,
simply because some type erasure happened, and the call site
does not know the actual type anymore
...step by step switch over to the new usage pattern.
Transformer should be the blueprint for all other functor usages.
The reworked solutions behaves as expected;
we see two functor invocations; the outer functor, which does
the argument adaptation, is allocated in heap memory
This does not touch the existing code-path,
but the idea is to use the _FunTraits directly from within the
constructor of the respective processing layer, and to confine the
knowledge of the actual FUN functor type to within that limited context.
Only the generic signature of the resulting std::function need to be
encoded into the type of the processing component, which should help
to simplify the type signatures
...which still needs to be the *concrete* signature of the funcition to pass,
but we'll attempt to loosen that requirement in the next refactoring steps
...and in preparation start with some renamings...
The overall goal is to simplify the type signatures and thereby
to make the generates pipelines more assignment compatible.
The debugging experience form the last days indicated that the
current design is not maintainable on the long run. Both the
template instantiation chains and the call stacks are way to
complicated and hard to understand and diagnose
It is essential that we pass the current state of the filter
into the expand functor, where it needs to be copied (once!)
to create a child state, which can then be augmented.
This augmented state is then pushed onto a stack, to enable backtracking.
Due to the flexible adapters and the wrapping into the TreeExplorer builder,
we ended up performing several spurious copies on the current state
...based on a monadic tree expansion: we define a single step,
which takes the current filter configuration and builds the next
filter configuration, based on a stored chain of configuration functions
The actual exhausting depth-first results just by the greedy application pattern,
and uses the stack embedded in the "Explorer" layer of TreeExplorer
..this resolves the most challenging part of the construction work;
we use the static helper functions to infer a type and construct a suitable
processing pipeline and we invoke the same helper to initialise the base class
in the ctor.
Incidentally... we can now drop all the placeholder stubs,
since we now inherit the full iterator and child explorer API.
The test now starts actually to work... we get spam and sausage!
TODO: now actually fill in the expand functor such as to pick the
concrete filter step in the chain from a sequence of preconfigured
filter bindings
...now matters start to get really nasty,
since we have to pick up an infered type from a partially built pipeline
and use it to construct the signature for a functor to bind into the more elaborate complete pipeline
this is a tricky undertaking, since our treeExplore() helper constructs
a complex wrapped type, depending on the actual builder expressions used.
Solution is to use decltype on the result of a helper function,
and let the _DecoratorTraits from TreeExplorer do the necessary type adaptations
when adapting a functor, the wrapper automatically decides
if this functor was meant to be used in "monadic style" (value -> new monad)
or in "manipulate state-core style" (iter& -> iter)
Unfortunately, in some cases functions accepting a partially built pipeline
will be classified in the wrong way, due to the fact that IterableDecorator
has a templated wildcard constructor and thus seems to accept the conversion
value_type -> IterableDecorator. This causes the "monadic style" to be chosen
erroneously, and leads to a template instantiation failure just at the point when
the generated functor will be used.
The solution is to explicitly *rule out* the monadic usage style
in all those cases, where the *argument* of the functor to bind can
in fact be directly constructed / converted from the source iterator
or state core. Because, if that is the case
- it is superfluous explicitly to dereference the source iterator;
the functor can do the same
- chances are that the "manipulation style" was intended
(as was the case in the concrete example at hand here)
...it should have been this way all the time.
Generic code might otherwise be ill guided to assume a conversion
from the Iterator to its value type, while in fact an explicit dereferentiation is necessary
Need to be more careful when eating trailing spaces after a std::string.
Because the full-blown type is a template, sometimes the compiler adds a spurious
additional space behind the closing angle bracket, due to the now obsolete
"maximum munch rule" of C++98 -- to prevent closing angle brackets to become '>>'.
But in other cases, some type adornments or other language identifiers follow the
string type, like e.g. 'const'. In those cases, the trailing space must be retained.
We solve this by a look-ahead assertion in the regular expression:
consume the trailing space _only_ if a non-word character follows (like '>').
The intention is to augment the iterator based (linear) search
used in EventLog to allow for real backtracking, based on a evaluation tree.
This should be rather staight forward to implement, relying on the
exploreChildren() functionality of TreeExplorer. The trick is to package
the chained search step as a monadic flatMap operation
while this is basically a drop-in replacement,
it marks the switch to the monadic evaluation technology,
which is prerequisite for building real backtracking into the search.
we did an unnecessary copy of the argument, which was uncovered
by the test case manipulating the state core.
Whew.
Now we have a beautiful new overengineered solution
outift the Filter base class with the most generic form of the Functor
wrapper, and rather wrap each functor argument individually. This allows
then to combine various kinds of functors
...this solution works, but has a shortcoming:
the type of the passed lambdas is effectively pinned to conform
with the signature of the first lambda used initially when building the filter.
Well, this is the standard use case, but it kind of turns all the
tricky warpping and re-binding into a nonsense excercise; in this form
the filter can only be used in the monadic case (value -> bool).
Especially this rules out all the advanced usages, where the filter
collaborates with the internals of the source.
while this is basically just code code cosmetics,
at least it marks this as a very distinct special case,
and keeps the API for the standard Filter layer clean.
a quite convoluted construct built from several nested generic lambdas.
When investigated in the debugger, the observed addresses and the
invoked code looks sane and as expected.
The intention is to switch from the itertools-based filter
to the filter available in the TreeExplorer framework.
Thus "basically" we just need to copy the solution over,
since both are conceptually equivalent.
However...... :-(
The TreeExplorer framework is designed to be way more generic
and accepts basically everything as argument and tries to adapt apropriately.
This means we have to use a lot of intricate boilerplate code,
just to get the same effect that was possible in Itertools with
a simple and elegant in-place lambda assignment
Fillter needs to be re-evaluated, when an downstream entity requests
expandChildren() onto an upstream source. And obviously the ordering
of the chained calls was wrong here.
As it turns out, I had discovered that necessity to re-evaluate with
the Transformer layer. There is a dedicated test case for that, but
I cut short on verifying the filter in that situation as well, so
that piece of broken copy-n-paste code went through undetected.
This is in fact a rather esoteric corner case, because it is only
triggered when the expandChildren() call is passed through the filter.
When otoh the filter sits /after/ the entity generating the expandChildren()
calls, everything works as intended. And the latter is the typical standard
usage situation of an recursive evalutation algorithm: the filter is here
used as final part to drive the evaluation ahead and pick the solutions.
There is a bug or shortcoming in the existing ErrorLog matcher implementation.
It is not really difficult to fix, however doing so would require us to intersperse
yet another helper facility into the log matcher. And it occurred to me, that
this helper would effectively re-implement the stack based backtracking ability,
which is already present in TreeExplorer (and was created precisely to support
this kind of recursive evaluation strategies).
Thus I intend to switch the implementation of the EventLog matcher from the
old IterTool framework to the newer TreeExplorer framework. And this intention
made me re-read the code, fixing several comments and re-thinking the design
seemingly my quick-n-dirty implementation was to naiive.
We need real backtracking, if we want to support switches
in the search direction (match("y").after("x").before("z")
Up to now, I have cheated myself around this obvious problem :-/
this is a (hopefully just temporary) workaround to deal with static initialisation
ordering problems. The original solution was cleaner from a code readability viewpoint,
however, when lib::Depend was used from static initialisation code, it could
be observed that the factory constructor was invoked after first use.
And while this did not interfer with the instance lifecycle management itself,
because the zero-initialisation of the instance (atomic) pointer did happen
beforehand, it would discard any special factory functions installed from such
a context (and this counts as bug for my taste).
seemingly this code was brittle: GCC-7 treats int64_t as long,
which leads to preferring the template specialisation over the
explicit version of the operator* -- which means the template
instantiation invokes itself.
usually the ID is hard coded, but when re-throwing errors, it might be
from "somewhere else", which means it is possibly a NULL ptr.
In those cases we fall back to the cannonical ID of the error class.
Basically the mocking mechanism just switches the configuration
and then waits for the service to be accessed in order to cause acutual
instantiation of the mock service implementation. But sometimes we want
to prepare and rig the mock instance prior to the first invocation;
in such cases it can be handy just to trigger the lazy creating process
while switching various services to the new framework,
I noticed the requirement to create a service handle in not-yet-started mode
and then start it explicitly, maybe even from another thread. Thus I introduced
a no-arg default ctor for that purpose, but overlooked that the forwarding ctor
might also need zero arguments for default constructible service implementation
classes. Thus I've now introduced a marker ENUM for disambiguation
SingletonRef was only invented because lib::Depend (or lib::Singleton at that time)
offered only on-demand initialisation, but could not attach to an external service.
But this is required for calling out at the implementation side of a
Lumiera Interface into the actual service implementation.
The recently created DependInject::ServiceInstance now fulfils this task way better
and is seamlessly integrated into the lib::Depend front-end
Actually this is on the implementation side only.
Since Layer-Separation-Interfaces route each call through a binding layer,
we get two Service-"Instances" to manage
- on the client side we have to route into the Lumiera Interface system
- on the implementation side the C-Language calls from the Interface system
need to get to the actual service implementation. The latter is now
managed and exposed via DependInject::ServiceInstance
...still using the FAKE implementation, not a real rules engine.
However, with the new Dependency-Injection framework we need to define
the actual class from the service-provider, not from some service-client.
This is more orthogonal, but we're forced to install a Lifecycle-Hook now,
in order to get this configuration into the system prior to any use
This is borderline yet acceptable;
A service might indeed depend on itself circularly
The concrete example is the Advice-System, which needs to push
the clean-up of AdviceProvicions into a static context. From there
the deleters need to call back into the AdviceSystem, since they have
no wey to find out, if this is an individual Advice being retracted,
or a mass-cleanup due to system shutdown.
Thus the DependencyFactory now invokes the actual deleter
prior to setting the instance-Ptr to NULL.
This sidesteps the whole issue with the ClassLock, which actually
must be already destroyed at that point, according to the C++ standard.
(since it was created on-demand, on first actual usage, *after* the
DependencyFactory was statically initialised). A workaround would be
to have the ctor of DependencyFactory actively pull and allocate the
Monitor for the ClassLock; however this seems a bit overingeneered
to deal with such a borderline issue
...and package the ZombieCheck as helper object.
Also rewrite the SyncClassLock_test to perform an
multithreaded contended test to prove the lock is shared and effective
Static initialisation and shutdown can be intricate; but in fact they
work quite precise and deterministic, once you understand the rules
of the game.
In the actual case at hand the ClassLock was already destroyed, and
it must be destroyed at that point, according to the standard. Simply
because it is created on-demand, *after* the initialisation of the
static DependencyFactory, which uses this lock, and so its destructor
must be called befor the dtor of DependencyFactory -- which is precisely
what happens.
So there is no need to establish a special secure "base runtime system",
and this whole idea is ill-guided. I'll thus close ticket #1133 as wontfix
Conflicts:
src/lib/dependable-base.hpp
When some dependency or singleton violates Lumiera's policy regarding destructors and shutdown,
we are unable to detect this violation reliably and produce a Fatal Error message.
This is due to lib::Depend's de-initialisating being itself tied to template generated
static variables, which unfortunately have a visibility scope beyond the translation unit
responsible for construction and clean-up.
- state-of-the-art implementation of access with Double Checked Locking + Atomics
- improved design for configuration of dependencies. Now at the provider, not the consumer
- support for exposing services with a lifecycle through the lib::Depend<SRV> front-end
...which declare DependencyFactory as friend.
Yes, we want to encourrage that usage pattern.
Problem is, std::is_constructible<X> gives a misleading result in that case.
We need to do the instantiation check within the scope of DependencyFactory
ideally we want
- just a plain unique_ptr
- but with custom deleter delegating to lib::Depend
- Depend can be made fried to support private ctor/dtor
- reset the instance-ptr on deletion
- always kill any instance
all these tests are ported by drop-in replacement
and should work afterwards exactly as before (and they do indeed)
A minor twist was spotted though (nice to have more unit tests indeed!):
Sometimes we want to pass a custom constructor *not* as modern-style lambda,
but rather as direct function reference, function pointer or even member
function pointer. However, we can not store those types into the closure
for later lazy invocation. This is basically the same twist I run into
yesterday, when modernising the thread-wrapper. And the solution is
similar. Our traits class _Fun<FUN> has a new typedef Functor
with a suitable functor type to be instantiated and copied. In case of
the Lambda this is the (anonymous) lamda class itself, but in case of
a function reference or pointer it is a std::function.
- polish the text in the TiddlyWiki
- integrate some new pages in the published documentation
Still mostly placeholder text with some indications
- fill in the relevant sections in the overview document
- adjust, expand and update the Doxygen comments
TODO: could convert the TiddlyWiki page to Asciidoc and
publish it mostly as-is. Especially the nice benchmarks
from yesterday :-D
This solution is considered correct by the experts.
Regarding the dependency-configuration part, we do not care too much about performance
and use the somewhat slower default memory ordering constraint
This is essentially the solution we used since start of the Lumiera project.
This solution is not entirely correct in theory, because the assignment to the
instance pointer can be visible prior to releasing the Mutex -- so another thread
might see a partially initialised object
_not_ using the dependency factory, rather direct access
- to a shared object in the enclosing stack frame
- to a heap allocated existing object accessed through uniqe_ptr
The Lumiera thread-wrapper accepts the operation to be performed
within the new thread as a function object, function reference or lambda.
Some of these types can be directly instantiated in the threadMain
function, and thus possibly inlined altogether. This is especially
relevant for Lambdas. OTOH, we can not instantiate function references
or bound member functions; in those cases we fall back to using a
std::function object, possibly incurring heap allocations.
...written as byproduct from the reimplementation draft.
NOTE there is a quite similar test from 2013, DependencyFactory_test
For now I prefer to retain both, since the old one should just continue
to work with minor API adjustments (and thus prove this rewrite is a
drop-in replacement).
On the long run those two tests could be merged eventually...
This is a complete makeover of our lib::Depend and lib::DependencyFactory templates.
While retaining the basic idea, the configuration has been completely rewritten
to favour configuration at the point where a service is provided rather,
than at the point where a dependency is used.
Note: we use differently named headers, so the entire Lumiera
code base still uses the old implementation. Next step will be
to switch the tests (which should be drop-in)
Most dependencies within Lumiera are singletons and this approach remains adequate.
Singletons are not "EVIL" per se. But in some cases, there is an explicit
lifecycle, managed by some subsystem. E.g. some GUI services are only available
while the GTK event loop is running.
This special case can be integrated transparently into our lib::Depend<TY> front-end,
which defaults to creating a singleton otherwise.
especially std::find is relevant here.
I consider this only a preliminary solution and need to think it over
more in detail. But, judging from the description given in
http://en.cppreference.com/w/cpp/iterator
and
http://en.cppreference.com/w/cpp/concept/InputIterator
...the standard concept of "input iterator" seems to be closest to our
own concept, albeit it requires things like post increment, and a
reference_type convertible to value_type -- requirements we do not
necessarily support with our more limited "Lumiera Forward Iterator"
seems to be the most orthogonal way to strip adornments from the SIG type
Moreover, we want to move the functor into the closure, where it will be stored anyay.
From there on, we can pass as const& into the binder (for creating the partially closed functor)
...but not yet able to get it to compile.
Problem seems to be the generic lambda, which is itself a template.
Thus we need a way to instantiate that template with the correct arguments
prior to binding it into a std::function
been there, seen that recently (-> TreeExplorer, the Expander had a similar problem)
yet some more trickery to get around this design problem.
I just do not want to rework IterSource right now, since this will be
a major change and require more careful consideration.
Thus introduce a workaround and mark it as future work
Using this implementation, "child expansion" should now be possible.
But we do not cover this directly in Unit test yet
...but not yet switched into the main LocationQuery interface,
because that would also break the existing implementation;
recasting this implementation is the next step to do....
...which basically allows us to return any suitable implementation
for the child iterator, even to switch the concrete iteration on each level.
We need this flexibility when implementing navigation through a concrete UI
...at least when using a wrapped Lumiera Iterator as source.
Generally speaking, this is a tricky problem, since real mix-in interfaces
would require the base interface (IterSource) to be declared virtual.
Which incurres a performance penalty on each and every user of IterSource,
even without any mix-in additions. The tricky part with this is to quantify
the relevance of such a performance penalty, since IterSource is meant
to be a generic library facility and is a fundamental building block
on several component interfaces within the architecture.
...yet I do not want to move all of the traits over into the
publicly visible lib::iter_explorer namespace -- I'm quite happy
with these traits being clearly marked as local internal details
surprise: the standard for-Loop causes a copy of the iterator.
From a logical POV this is correct, since the iterator is named,
it can not just be moved into the loop construct and be consumed.
Thus: write a plain old-fashioned for loop and consume the damn thing.
So the top-level call into util::join(&&) decides, if we copy or consume
several extensions and convenience features are conceivable,
but I'll postpone all of them for later, when actual need arises
Note especially there is one recurring design challenge, when creating
such a demand-driven tree evaluation: more often than not it turns out
that "downstream" will need some information about the nested tree structure,
even while, on the surfice, it looks as if the evaluation could be working
completely "linearised". Often, such a need arises from diagnostic features,
and sometimes we want to invoke another API, which in turn could benefit
from knowing something about the original tree structure, even if just
abstracted.
I have no real solution for this problem, but implementing this pipeline builder
leads to a pragmatic workaround: since the iterator already exposes a expandChildren(),
it may as well expose a depth() call, even while keeping anything beyond that
opaque. This is not the clean solution you'd like, but it comes without any
overhead and does not really break the abstraction.
...so sad.
The existing implementation was way more elegant,
just it discarded an exahusted parent element right while in expansion,
so effectively the child sequence took its place. Resolved that by
decomposing the iterNext() operation. And to keep it still readable,
I make the invariant of this class explicit and check it (which
caught yet another undsicovered bug. Yay!)
instead of building a very specific collaboration,
rather just pass the tree depth information over the extended iterator API.
This way, "downstream" clients *can* possibly react on nested scope exploration
...and there is a point where to stop with the mere technicalities,
and return to a design in accordance with the inner nature of things.
Monads are a mere technology, without explicatory power as a concept or pattern
For that reason
- discard the second expansion pattern implemented yesterday,
since it just raises the complexity level for no given reason
- write a summary of my findings while investigating the abilities
of Monads during this design excercise.
- the goal remains to abandon IterExplorer and use the now complete
IterTreeEplorer in its place. Which also defines roughly the extent
to wich monadic techniques can be useful for real world applications
...it can sensibly only be done within the Expander itself.
Question: is this nice-to-have-feature worth the additional complexity
of essentially loading two quite distinct code paths into a single
implementation object?
As it stands, this looks totally confusing to me...
At that time, our home-made Tuple type was replaced by std::tuple,
and then the command framework was extended to also allow command invocation
with arguments packaged as lib::diff::Record<GenNode>
With changeset 0e10ef09ec
A rebinding from std::tuple<ARGS...> to Types<ARGS> was introduced,
but unfortunately this was patched-in on top of the existing Types<ARGS...>
just as a partial specialisation.
Doing it this way is especially silly, since now this rebinding also kicks
in when std::tuple appears as regular payload type within Types<....>
This is what happened here: We have a Lambda taking a std::tuple<int, int>
as argument, yet when extracting the argument type, this rebinding kicks in
and transforms this argument into Types<int, int>
Oh well.
this leads to either unfolding the full tree depth-first,
or, when expanding eagerly, to delve into each sub-branch down to the leaf nodes
Both patterns should be simple to implement on top of what we've built already...
IterSource should be refactored to have an iteration control API similar to IterStateWrapper.
This would resolve the need to pass that pos-pointer over the abstraction barrier,
which is the root cause for all the problems and complexities incurred here
...but for now the price is that we need to punch a hole into IterAdapter.
And obviously, this is all way to tangled and complex on implementation level.
this was a design decision, but now I myself run into that obvious mistake;
thus not sure if this is a good design, or if we need a dedicated operation
to finish the builder and retrieve the iterable result.
as it turned out, when "inheriting" ctors, C++14 removes the base classes' copy ctors.
C++17 will rectify that. Thus for now we need to define explicitly that
we'll accept the base for initialising the derived. But we need do so
only on one location, namely the most down in the chain.
Since this now requires to import iter-adapter-stl.hpp and iter-source.hpp
at the same time, I decided to drop the convenience imports of the STL adapters
into namespace lib. There is no reason to prefer the IterSource-based adapters
over the iter-adapter-stl.hpp variants of the same functionality.
Thus better always import them explicitly at usage site.
...actual implementation of the planned IterSource packaging is only stubbed.
But I needed to redeclare a lot of ctors, which doesn't seem logical
And I get a bad function invocation from another test case which worked correct beforehand.
We need a way for higher layers to discard their caching and re-evaluate,
once some expansion layer was invoked to replace the current element with
its (functionally defined) "children" -- otherwise the first child will
remain obscured by what was there beforehand.
Solution is to pass such manipulation calls through the full chain of
decorators, allowing them to refresh themselves when necessary. To achieve
that technially, we add a base layer to absorb any such call passed down
through the whole decorator chain -- since we can not assume that the
parent, the original source core implements those manipualation calls
like expandChildren()
due to switching from ADL extension points to member functions,
we now need to detect a "state core" type in a different fashion.
The specific twist is that we can not spell out the full signature
in all cases, since the result type will be formed as a consequence
of this type detection. Thus there are now additional detectors to
probe for the presence of a specific function name only, and the
distinction between members and member functions has been sharpened.
Considering the fact that we are bound to introduce yet another iteration control function,
because there is literally no other way to cause a refresh within the IterTreeExplorer-Layers,
it is indicated to reconsider the way how IterStateWrapper attaches to the
iteration control API.
As it turns out, we'll never need an ADL-free function here;
and it seems fully adequate to require all "state core" objects to expose
the API as argument less member function. Because these reflect precisely
the contract of a "state core", so why not have them as member functions.
And as a nice extra, the implementation becomes way more concise in
all the cases refactored with this changeset!
Yet still, we stick to the basic design, *not* relying on virtual functions.
So this is a typical example of a Type Class (or "Concept" in C++ terminology)
- always layer the TreeExplorer (builder) on top of the stack
- always intersperse an IterableDecorator in between adjacent layers
- consequently...
* each layer implementation is now a "state core"
* and the source is now always a Lumiera Iterator
This greatly simplifies all the type rebindings and avoids the
ambiguities in argument converison. Basically now we can always convert
down, and we just need to pick the result type of the bound functor.
Downside is we have now always an adaptation wrapper in between,
but we can assume the compiler is able to optimise such inline
accessors away without overhead.
...yet this seems like a rather bad idea,
it breeds various problems and requires arcane trickery to make it fly
==> abandon this design
==> always intersperse an IterableDecorator between each pair of Layers
...especially relevant in the context of TreeExplorer,
where the general understanding is that the "Data Source" (whatever it is)
will be piggy-backed into the pipeline builder, and this wrapping is
conceived as being essentially a no-op.
It is quite possible we'll even start using such pipeline builders
in concert with move-only types. Just consider a UI-navigator state
hooked up with a massive implementation internal pointer tree attached
to all of the major widgets in the UI. Nothing you want to copy in passing by.
As it turned out, we had two bugs luring in the code base,
with the happy result of one cancelling out the adverse effects of the other
:-D
- a mistake in the invocation of the Itertools (transform, filter,...)
caused them to move and consume any input passed by forwarding, instead
of consuming only the RValue references.
- but util::join did an extraneous copy on its data source, meaning that
in all relevant cases where a *copy* got passed into the Itertools,
only that spurious temporary was consumed by Bug #1.
(Note that most usages of Itertools rely on RValues anyway, since the whole
point of Itertools is to write concise in-line transformation pipelines...)
*** Added additional testcode to prove util::stringify() behaves correct
now in all cases.
Obsoletes and replaces the ad-hoc written type rebindings from
iter-adapter and friends. The new scheme is more consistent and does
less magic, which necessitates an additional remove_pointer<IT> within
the iterator adaptors. Rationale is, "pointer" is treated now just as
a primitive type without additional magic or unwrapping, since it is
impossible to tell generically if the pointer or the pointee was
meant to be the "value"
Oh well.
This kept me busy a whole day long -- and someone less stubborn like myself
would probably supect a "compiler bug" or put the blame on the language C++
So to stress this point: the compiler behaved CORRECT
Just SFINAE is dangerous stuff: the metafunction I concieved yesterday requires
a complete type, yet, under rather specific circumstances, when instantiating
mutually dependent templates (in our case lib::diff::Record<GenNode> is a
recursive type), the distinction between "complete" and "incomplete"
becomes blurry, and depends on the processing order. Which gave the
misleading impression as if there was a side-effect where the presence
of one definition changes the meaning of another one used in the same
program. What happened in fact was just that the evaluation order was
changed, causing the metafunction to fail silently, thus picking
another specialisation.
- we do strip references
- we delegate to nested typedefs
Hoever, we do *not* treat const or pointers in any way special --
if the user want to strip or level these, he has to do so explicitly.
Initially it seemed like a good idea to do something clever here, but
on the long run, such "special treatment" is just good for surprises
...automatically whenever those are present.
Up to now, we hat that as base case, which limited usage to those cases
where we already know such nested definitions are actually present
attempt to re-use the same traits as much as possible
NOTE: new code not passing compiler yet, but refactored old code
does, and still passes unit test
this is a subtle change which, given all interfaces were used in a logically
consistent way, should not cause any observable change to the yielded elements.
But it changes runtime behaviour, insofar now the evalutaion is initiated
lazily, when first requesting a result type. Prior to this change, the
constructor immediately issued a call to the yield() extension point,
which presumably has the side-effect of preparing the core and initiating
any embedded evaluation, in order to get at the first result; it might
even detect an empty state.
Given the fact that all access operations on the iterator front-end perform
an empty check (and possibly throw at that point), this call is redundant.
surprising behaviour encountered while covering more cases
...obviously the return type of ExpandFunctor::operator()
was inferred as value, even while the invoked functor, from which
this type was deduced, clearly returns a reference.
Solution is simple not to rely on inference, moreover since we know
the exact type in the enclosing scope, thanks to the refactoring which
made this ExpandFunctor a nested class
NOTE:
as it turned out, this is not a compiler bug,
but works as defined by the language:
on return type inference, the detected type is decayed,
which usually helps to prevent returning a reference to a temporary
...while this implementation works now, it is still very complex and intricate.
I am still doubtful this is a good approach, but well, we need to try that route....
but possible only for the iterator -> iterator case
Since we can not "probe" a generic lambda, we get only one shot:
we can try to bind it into a std::function with the assumed signature
This is a consequence of the experiments with generic lambdas.
Up to now, lib::meta::_Fun<F> failed with a compilation error
when passing the decltype of such a generic lambda.
The new behaviour is to pick the empty specialisation (std::false_type) in such cases,
allowing to guard explicit specialisations when no suitable functor type
is passed
Basically we want to support two distinct cases, just by slightly adapting
the invocation of the expansion functor:
Case-1: classical monadic flatMap:
the Functor accepts a value yielded by the source iterator
and builds a new "expaneded" iterator
Case-2: manipulation of opaque implementation state
the Functor knows internal details of the source iterator
and thus takes the source iterator as such as argument,
performs some manipulation and then builds a new sub-iterator
A soulution to reconcile those two distinct cases can be built
with the help of a generic lambda
this solution makes me feel somewhat queasy..
stacking several adaptors and wrappers and traits on top of each other.
Well, it type checks and passes the test, so let's trust functional programming
The plan is to use a monad-like scheme, but allow for a lot of leeway
with respect to the src and value types of the expand functor.
A key idea is to allow for a *different* state core than used in the source
...but does not work as intended:
* just forming an IterStateWrapper does not trigger SFINAE cleanly in all cases
* IterStateWrapper can be formed, even when some of the extension points are missing;
this will be uncovered only later, when actually using one of the operations
but beyond that, the basic type selection logic can work this way
Here, the tricky question remains, how to relate this evalutaion scheme
to the well known monadic handling of collections and iterators.
It seems, we can not yet decide upon that question, rather we should
first try to build a concrete implementation of the envisioned algorithm
and then reconsider the question later, to what extent this is "monadic"
This can be seen as a side track, but the hope is
by relying on some kind of monadic evaluation pattern, we'll be
able to to reconcile the IterExplorer draft from 2012 with the requirement
to keep the implementation of "tree position" entirely opaque.
The latter is mandatory in the use case here, since we must not intermingle
the algorithm to resolve UI-coordinates in any way with the code actually
navigating and accessing GTK widgets. Thus, we're forced to build some kind
of abstraction barrier, and this turns out to be surprisingly difficult.
...which can be helpful when a function usually returns a somewhat dressed-up iterator,
but needs to return a specific fixed value under some circumstances
- fix some warnings due to uninitialised members
(no real problem, since these members get assigned anyway)
- use a lambda as example function right in the test
- use move initialisation and the new util::join
up to now, we allowed only initialisation with a precisely matching type.
But this special case seems worth supporting, since it typically occurs
within the "object builder" syntax based on Rec::Mutator
Explicitly assuming that those functions are called solely from IterAdapter
and that they are implemented in a typical standard style, we're able to elide
two redundant calls to the checkPoint() function. Since checkPoint typically performs
some non-trivial checks, this has the potential of a significant performance improvement
- we check (and throw ITER_EXHAUST) anyway from operator++, so we know that pos is valid
- the iterate() function ensures checkPoint is invoked right after iterNext,
and thus the typical standard implementation of iterNext need not do the same
...since that is what it meant to be.
To allow this chance, I've now added a default ctor to lib::Literal,
defaulting to the Symbol::EMPTY (the interned empty string)
The class Literal is used as a thin wrapper to mark the fact that
some string parameter or value is assumed to be given *literally*
For the contract this indicates
- that storage is somewhere
- storage is not owned and managed by Literal
- yet storage guaranteed to exist during the whole lifetime of the program
- Literal can not be altered
- Literal is transparently convertible to const char *
Currently I am in the course of building some path abstraction, and for that
task it makes sense to hold an array of Literals (instead of pointers), just
because it expresses the intent way more clear. I do not see anything in the
above mentioned contract to prohibit a default constructed Literal, with the
empty string being the most obvious choice.
Note: there is the class Symbol, which derives from Literal. Symbol takes
arbitrary strings, but *interns* them into a static symbol table.
...under the assumption that the content is normalised,
which means
- leading NULL is changed to Symbol::EMPTY
- missing elements in the middle are marked as "*"
- trailing NULL in extension storage is handled by adjusting nominal extension size
after various fruitless attempts to rely somehow on the array variant of unique_ptr,
I ended up with a hand coded version of an heap allocated array, managed automatically
as it turned out, the solution from yesterday works only with uniform argument lists,
but not with arbitrarily mixed types. Moreover the whole trickery with the
indices was shitty -- better use a predicate decision on template argument level.
This simple solution somehow just didn't occur to me...
...still somewhat unsatisfactory, because
- no clear compile error message when invoking pickArg with insufficient arguments
- the default initialisation case in SelectVararg is duplicated and messy
some time ago we abandoned our own tuple type in favour of std::tuple
Since then, the helpers and ported utilities provide some generic helpers
to deal with variadic argument sequences, especially to build index sequences,
which in turn can be used to "pick" individual arguments from a variadic parameter pack.
The expectation is for this part of the support library gradually to grow and
in parts to replace the existing type sequence processing helpers. The expectation
is that we'll retain the basic type sequence, lib::meta::Types, but retrofit it
to rely on variadic arguments
since the adoption of C++11, we gradually transition our metaprogramming helpers
to support and rely on variadic template parameters. For the time being,
we just augment existing facilities when it comes in handy, yet some more
heavyweight lifting and overall clean-up remains to be done eventually.
we allow assignment to the element embedded within the wrapper.
Yet obviously we need specific implementations for assignment
to the container itself. Thus we define the templated
assignment operator such as to render the explicit specialisation
a better match than anything generated from the templated
operator
