This is borderline yet acceptable;
A service might indeed depend on itself circularly
The concrete example is the Advice-System, which needs to push
the clean-up of AdviceProvicions into a static context. From there
the deleters need to call back into the AdviceSystem, since they have
no wey to find out, if this is an individual Advice being retracted,
or a mass-cleanup due to system shutdown.
Thus the DependencyFactory now invokes the actual deleter
prior to setting the instance-Ptr to NULL.
This sidesteps the whole issue with the ClassLock, which actually
must be already destroyed at that point, according to the C++ standard.
(since it was created on-demand, on first actual usage, *after* the
DependencyFactory was statically initialised). A workaround would be
to have the ctor of DependencyFactory actively pull and allocate the
Monitor for the ClassLock; however this seems a bit overingeneered
to deal with such a borderline issue
...and package the ZombieCheck as helper object.
Also rewrite the SyncClassLock_test to perform an
multithreaded contended test to prove the lock is shared and effective
Static initialisation and shutdown can be intricate; but in fact they
work quite precise and deterministic, once you understand the rules
of the game.
In the actual case at hand the ClassLock was already destroyed, and
it must be destroyed at that point, according to the standard. Simply
because it is created on-demand, *after* the initialisation of the
static DependencyFactory, which uses this lock, and so its destructor
must be called befor the dtor of DependencyFactory -- which is precisely
what happens.
So there is no need to establish a special secure "base runtime system",
and this whole idea is ill-guided. I'll thus close ticket #1133 as wontfix
Conflicts:
src/lib/dependable-base.hpp
When some dependency or singleton violates Lumiera's policy regarding destructors and shutdown,
we are unable to detect this violation reliably and produce a Fatal Error message.
This is due to lib::Depend's de-initialisating being itself tied to template generated
static variables, which unfortunately have a visibility scope beyond the translation unit
responsible for construction and clean-up.
- state-of-the-art implementation of access with Double Checked Locking + Atomics
- improved design for configuration of dependencies. Now at the provider, not the consumer
- support for exposing services with a lifecycle through the lib::Depend<SRV> front-end
...which declare DependencyFactory as friend.
Yes, we want to encourrage that usage pattern.
Problem is, std::is_constructible<X> gives a misleading result in that case.
We need to do the instantiation check within the scope of DependencyFactory
ideally we want
- just a plain unique_ptr
- but with custom deleter delegating to lib::Depend
- Depend can be made fried to support private ctor/dtor
- reset the instance-ptr on deletion
- always kill any instance
all these tests are ported by drop-in replacement
and should work afterwards exactly as before (and they do indeed)
A minor twist was spotted though (nice to have more unit tests indeed!):
Sometimes we want to pass a custom constructor *not* as modern-style lambda,
but rather as direct function reference, function pointer or even member
function pointer. However, we can not store those types into the closure
for later lazy invocation. This is basically the same twist I run into
yesterday, when modernising the thread-wrapper. And the solution is
similar. Our traits class _Fun<FUN> has a new typedef Functor
with a suitable functor type to be instantiated and copied. In case of
the Lambda this is the (anonymous) lamda class itself, but in case of
a function reference or pointer it is a std::function.
...which showed up under high system load.
The initialisation of the member variables for the check sum
could be delayed while the corresponding thread was already running
- polish the text in the TiddlyWiki
- integrate some new pages in the published documentation
Still mostly placeholder text with some indications
- fill in the relevant sections in the overview document
- adjust, expand and update the Doxygen comments
TODO: could convert the TiddlyWiki page to Asciidoc and
publish it mostly as-is. Especially the nice benchmarks
from yesterday :-D
This solution is considered correct by the experts.
Regarding the dependency-configuration part, we do not care too much about performance
and use the somewhat slower default memory ordering constraint
This is essentially the solution we used since start of the Lumiera project.
This solution is not entirely correct in theory, because the assignment to the
instance pointer can be visible prior to releasing the Mutex -- so another thread
might see a partially initialised object
_not_ using the dependency factory, rather direct access
- to a shared object in the enclosing stack frame
- to a heap allocated existing object accessed through uniqe_ptr
The Lumiera thread-wrapper accepts the operation to be performed
within the new thread as a function object, function reference or lambda.
Some of these types can be directly instantiated in the threadMain
function, and thus possibly inlined altogether. This is especially
relevant for Lambdas. OTOH, we can not instantiate function references
or bound member functions; in those cases we fall back to using a
std::function object, possibly incurring heap allocations.
the operation to run within the thread can be passed through as lambda;
and with the help of C++11 perfect forwarding we can get rid of the
temporary ThreadStartContext entirely, which removes a lot of
pass-through argument copying
...written as byproduct from the reimplementation draft.
NOTE there is a quite similar test from 2013, DependencyFactory_test
For now I prefer to retain both, since the old one should just continue
to work with minor API adjustments (and thus prove this rewrite is a
drop-in replacement).
On the long run those two tests could be merged eventually...
This is a complete makeover of our lib::Depend and lib::DependencyFactory templates.
While retaining the basic idea, the configuration has been completely rewritten
to favour configuration at the point where a service is provided rather,
than at the point where a dependency is used.
Note: we use differently named headers, so the entire Lumiera
code base still uses the old implementation. Next step will be
to switch the tests (which should be drop-in)
Most dependencies within Lumiera are singletons and this approach remains adequate.
Singletons are not "EVIL" per se. But in some cases, there is an explicit
lifecycle, managed by some subsystem. E.g. some GUI services are only available
while the GTK event loop is running.
This special case can be integrated transparently into our lib::Depend<TY> front-end,
which defaults to creating a singleton otherwise.
we'll use a typedef to represent the default case
and provide the level within the UI-Tree as template parameter for the generic case
This avoids wrapping each definition into a builder function, which will be
the same function for 99% of the cases, and it looks rather compact and natural
for the default case, while still retaining genericity.
Another alternative would have been to inject the Tree-level at the invocation;
but doing so feels more like magic for me.
decided to add a very specific preprocessing here, to make the DSL notation more natural.
My guess is that most people won't spot the presence of this tiny bit of magic,
and it would be way more surprising to have rules like
UICoord::currentWindow().panel("viewer").create()
fail in most cases, simply because there is a wildcard on the perspective
and the panel viewer does not (yet) exist. In such a case, we now turn the
perspective into a "existential quantified" wildcard, which is treated as if
the actually existing element was written explicitly into the pattern.
especially std::find is relevant here.
I consider this only a preliminary solution and need to think it over
more in detail. But, judging from the description given in
http://en.cppreference.com/w/cpp/iterator
and
http://en.cppreference.com/w/cpp/concept/InputIterator
...the standard concept of "input iterator" seems to be closest to our
own concept, albeit it requires things like post increment, and a
reference_type convertible to value_type -- requirements we do not
necessarily support with our more limited "Lumiera Forward Iterator"
util::contains used to pick the overload for strings,
i.e. it first converted the UI-Coordinates to diagnostic output format
and then searched that string for '*' to determine if the pattern is explicit
works as expected, but not what you'd intend....
...and breaks spectacularly once you search for something as innocuous as '.'
when used in a pattern for matching against the UI tree,
an element marked as UIC_ELIDED = "." is treated as existentially quantified.
This means, we assume / verify there *is* an element at that level,
but we do not care about what this element actually is. Within the
implementation, the handling is similar to a wildcard, yet such a
spec is not classified as a wildcard (it *is* an explicit element,
just not explicitly named).
The relevant consequence is that such an element matches at a leaf
position, while match on wildcards on leaf positions is prohibited,
to prevent arbitrary and nonsensical wildcard matches against
open ended patterns. Especially we need such an existential pattern
to express a rule to create elements from scratch, but within a
specific window with arbitrary (but existing) perspective.
turns out to be somewhat tricky.
The easy shot would be to use the comma operator,
but I don't like that idea, since in logic programming, comma means "and then".
So I prefer an || operator, similar to short-circuit evaluation of boolean OR
Unfortunately, OR binds stronger than assignment, so we need to trick our way
into a smooth DSL syntax by wrapping into intermediary marker types, and accept
rvalue references only, as additional safeguard to enforce the intended inline
definition syntax typical for DSL usage.
seems to be the most orthogonal way to strip adornments from the SIG type
Moreover, we want to move the functor into the closure, where it will be stored anyay.
From there on, we can pass as const& into the binder (for creating the partially closed functor)
...as it turned out, the result type was the problem: the lambda we provide
typically does not yield an Allocator, but only its baseclass function<UICoord(UICoord)>
solution: make Allocator a typedef, we don't expect any further functionality
...but not yet able to get it to compile.
Problem seems to be the generic lambda, which is itself a template.
Thus we need a way to instantiate that template with the correct arguments
prior to binding it into a std::function
been there, seen that recently (-> TreeExplorer, the Expander had a similar problem)
...this was quite an extensive digression, which basically gave us
a solid foundation for topological addressing and pattern matching
within the "interface space"
rationale: sometimes (likely this is even the standard case) we do not just
want to "extend", rather we want to extent at very specific levels.
This is easy to implement, based on the existing building blocks for path manipulation
the original construction works only as long as we stick to the "classical" Builder syntax,
i.e. use chained calls of the builder functions. But as soon as we just invoke
some builder function for sake of the side-effect on the data within the builder,
this data is destroyed and moved out into the value return type, which unfortunately
is being thrown away right afterwards.
Thus: either make a builder really sideeffect-free, i.e. do each mutation
on a new copy (which is kind of inefficient and counterfeits the whole idea)
or just accept the side-effect and return only a reference.
In this case, we can still return a rvalue-Reference, since at the end
we want to move the product of the build process out into the destination.
This works only due to the C++ concept of sequence points, which ensures
the original object stays alive during the whole evaluation of such a chained
builder expression.
NOTE: the TreeMutator (in namespace lib::diff) also uses a similar Builder construction,
but in *that* case we really build a new product in each step and thus *must*
return a value object, otherwise the reference would already be dangling the
moment we leave the builder function.
- the default should be to look for total coverage
- the predicates should reflect the actual state of the path only
- the 'canXXX' predicates test for possible covering mutation
I set out to "discover" what operations we actually need on the LocationQuery
interface, in order to build a "coordinate resolver" on top. It seems like
this set of operations is clear by now.
It comes somewhat as a surprise that this API is so small. This became possible
through the idea of a ''child iterator'' with the additional ability to delve down and
expand one level of children of the current element. Such can be ''implemented''
by relying on techniques similar to the "Monads" from functional programming.
Let's see if this was a good choice. The price to pay is a high level of ''formal precision''
when dealing with the abstraction barrier. We need to stick strictly to the notion of a
''logical path'' into a tree-like topology, and we need to be strong enough never to
give in and indulge with "the concrete, tangible". The concrete reality of a tree
processing algorithm with memory management plus backtracking is just to complex
to be handled mentally. So either stick to the rules or get lost.
yet some more trickery to get around this design problem.
I just do not want to rework IterSource right now, since this will be
a major change and require more careful consideration.
Thus introduce a workaround and mark it as future work
Using this implementation, "child expansion" should now be possible.
But we do not cover this directly in Unit test yet
we need to layer our Navigator implementation on top,
since this object needs to capture a reference to the "current position".
This is necessary to be able to derive the child position by extending
and then to form a child navigator -- which is the essence of
implementing expandChildren()
...but not yet switched into the main LocationQuery interface,
because that would also break the existing implementation;
recasting this implementation is the next step to do....
...which basically allows us to return any suitable implementation
for the child iterator, even to switch the concrete iteration on each level.
We need this flexibility when implementing navigation through a concrete UI
...at least when using a wrapped Lumiera Iterator as source.
Generally speaking, this is a tricky problem, since real mix-in interfaces
would require the base interface (IterSource) to be declared virtual.
Which incurres a performance penalty on each and every user of IterSource,
even without any mix-in additions. The tricky part with this is to quantify
the relevance of such a performance penalty, since IterSource is meant
to be a generic library facility and is a fundamental building block
on several component interfaces within the architecture.
...yet I do not want to move all of the traits over into the
publicly visible lib::iter_explorer namespace -- I'm quite happy
with these traits being clearly marked as local internal details
NOTE it just type checks right now,
but since meta programming is functional programming, this means
with >90% probability that it might actually work this way....
...which also happens to include sibling and child iteration;
this is an attempt to reconcile the inner contradictions of the design
(we need both absolute flexibility for the type of each child level iterator
yet we want just a single, generic iterator front-end)
...this was a difficult piece of consideration and analysis.
In the end I've settled down on a compromise solution,
with the potential to be extended into the right direction eventually...
surprise: the standard for-Loop causes a copy of the iterator.
From a logical POV this is correct, since the iterator is named,
it can not just be moved into the loop construct and be consumed.
Thus: write a plain old-fashioned for loop and consume the damn thing.
So the top-level call into util::join(&&) decides, if we copy or consume
several extensions and convenience features are conceivable,
but I'll postpone all of them for later, when actual need arises
Note especially there is one recurring design challenge, when creating
such a demand-driven tree evaluation: more often than not it turns out
that "downstream" will need some information about the nested tree structure,
even while, on the surfice, it looks as if the evaluation could be working
completely "linearised". Often, such a need arises from diagnostic features,
and sometimes we want to invoke another API, which in turn could benefit
from knowing something about the original tree structure, even if just
abstracted.
I have no real solution for this problem, but implementing this pipeline builder
leads to a pragmatic workaround: since the iterator already exposes a expandChildren(),
it may as well expose a depth() call, even while keeping anything beyond that
opaque. This is not the clean solution you'd like, but it comes without any
overhead and does not really break the abstraction.
...so sad.
The existing implementation was way more elegant,
just it discarded an exahusted parent element right while in expansion,
so effectively the child sequence took its place. Resolved that by
decomposing the iterNext() operation. And to keep it still readable,
I make the invariant of this class explicit and check it (which
caught yet another undsicovered bug. Yay!)
instead of building a very specific collaboration,
rather just pass the tree depth information over the extended iterator API.
This way, "downstream" clients *can* possibly react on nested scope exploration
...and there is a point where to stop with the mere technicalities,
and return to a design in accordance with the inner nature of things.
Monads are a mere technology, without explicatory power as a concept or pattern
For that reason
- discard the second expansion pattern implemented yesterday,
since it just raises the complexity level for no given reason
- write a summary of my findings while investigating the abilities
of Monads during this design excercise.
- the goal remains to abandon IterExplorer and use the now complete
IterTreeEplorer in its place. Which also defines roughly the extent
to wich monadic techniques can be useful for real world applications
...it can sensibly only be done within the Expander itself.
Question: is this nice-to-have-feature worth the additional complexity
of essentially loading two quite distinct code paths into a single
implementation object?
As it stands, this looks totally confusing to me...
At that time, our home-made Tuple type was replaced by std::tuple,
and then the command framework was extended to also allow command invocation
with arguments packaged as lib::diff::Record<GenNode>
With changeset 0e10ef09ec
A rebinding from std::tuple<ARGS...> to Types<ARGS> was introduced,
but unfortunately this was patched-in on top of the existing Types<ARGS...>
just as a partial specialisation.
Doing it this way is especially silly, since now this rebinding also kicks
in when std::tuple appears as regular payload type within Types<....>
This is what happened here: We have a Lambda taking a std::tuple<int, int>
as argument, yet when extracting the argument type, this rebinding kicks in
and transforms this argument into Types<int, int>
Oh well.
this leads to either unfolding the full tree depth-first,
or, when expanding eagerly, to delve into each sub-branch down to the leaf nodes
Both patterns should be simple to implement on top of what we've built already...
IterSource should be refactored to have an iteration control API similar to IterStateWrapper.
This would resolve the need to pass that pos-pointer over the abstraction barrier,
which is the root cause for all the problems and complexities incurred here
...but for now the price is that we need to punch a hole into IterAdapter.
And obviously, this is all way to tangled and complex on implementation level.
this was a design decision, but now I myself run into that obvious mistake;
thus not sure if this is a good design, or if we need a dedicated operation
to finish the builder and retrieve the iterable result.
as it turned out, when "inheriting" ctors, C++14 removes the base classes' copy ctors.
C++17 will rectify that. Thus for now we need to define explicitly that
we'll accept the base for initialising the derived. But we need do so
only on one location, namely the most down in the chain.
Since this now requires to import iter-adapter-stl.hpp and iter-source.hpp
at the same time, I decided to drop the convenience imports of the STL adapters
into namespace lib. There is no reason to prefer the IterSource-based adapters
over the iter-adapter-stl.hpp variants of the same functionality.
Thus better always import them explicitly at usage site.
...actual implementation of the planned IterSource packaging is only stubbed.
But I needed to redeclare a lot of ctors, which doesn't seem logical
And I get a bad function invocation from another test case which worked correct beforehand.
We need a way for higher layers to discard their caching and re-evaluate,
once some expansion layer was invoked to replace the current element with
its (functionally defined) "children" -- otherwise the first child will
remain obscured by what was there beforehand.
Solution is to pass such manipulation calls through the full chain of
decorators, allowing them to refresh themselves when necessary. To achieve
that technially, we add a base layer to absorb any such call passed down
through the whole decorator chain -- since we can not assume that the
parent, the original source core implements those manipualation calls
like expandChildren()
due to switching from ADL extension points to member functions,
we now need to detect a "state core" type in a different fashion.
The specific twist is that we can not spell out the full signature
in all cases, since the result type will be formed as a consequence
of this type detection. Thus there are now additional detectors to
probe for the presence of a specific function name only, and the
distinction between members and member functions has been sharpened.
Considering the fact that we are bound to introduce yet another iteration control function,
because there is literally no other way to cause a refresh within the IterTreeExplorer-Layers,
it is indicated to reconsider the way how IterStateWrapper attaches to the
iteration control API.
As it turns out, we'll never need an ADL-free function here;
and it seems fully adequate to require all "state core" objects to expose
the API as argument less member function. Because these reflect precisely
the contract of a "state core", so why not have them as member functions.
And as a nice extra, the implementation becomes way more concise in
all the cases refactored with this changeset!
Yet still, we stick to the basic design, *not* relying on virtual functions.
So this is a typical example of a Type Class (or "Concept" in C++ terminology)
- always layer the TreeExplorer (builder) on top of the stack
- always intersperse an IterableDecorator in between adjacent layers
- consequently...
* each layer implementation is now a "state core"
* and the source is now always a Lumiera Iterator
This greatly simplifies all the type rebindings and avoids the
ambiguities in argument converison. Basically now we can always convert
down, and we just need to pick the result type of the bound functor.
Downside is we have now always an adaptation wrapper in between,
but we can assume the compiler is able to optimise such inline
accessors away without overhead.
...yet this seems like a rather bad idea,
it breeds various problems and requires arcane trickery to make it fly
==> abandon this design
==> always intersperse an IterableDecorator between each pair of Layers
...especially relevant in the context of TreeExplorer,
where the general understanding is that the "Data Source" (whatever it is)
will be piggy-backed into the pipeline builder, and this wrapping is
conceived as being essentially a no-op.
It is quite possible we'll even start using such pipeline builders
in concert with move-only types. Just consider a UI-navigator state
hooked up with a massive implementation internal pointer tree attached
to all of the major widgets in the UI. Nothing you want to copy in passing by.
As it turned out, we had two bugs luring in the code base,
with the happy result of one cancelling out the adverse effects of the other
:-D
- a mistake in the invocation of the Itertools (transform, filter,...)
caused them to move and consume any input passed by forwarding, instead
of consuming only the RValue references.
- but util::join did an extraneous copy on its data source, meaning that
in all relevant cases where a *copy* got passed into the Itertools,
only that spurious temporary was consumed by Bug #1.
(Note that most usages of Itertools rely on RValues anyway, since the whole
point of Itertools is to write concise in-line transformation pipelines...)
*** Added additional testcode to prove util::stringify() behaves correct
now in all cases.
Obsoletes and replaces the ad-hoc written type rebindings from
iter-adapter and friends. The new scheme is more consistent and does
less magic, which necessitates an additional remove_pointer<IT> within
the iterator adaptors. Rationale is, "pointer" is treated now just as
a primitive type without additional magic or unwrapping, since it is
impossible to tell generically if the pointer or the pointee was
meant to be the "value"
Oh well.
This kept me busy a whole day long -- and someone less stubborn like myself
would probably supect a "compiler bug" or put the blame on the language C++
So to stress this point: the compiler behaved CORRECT
Just SFINAE is dangerous stuff: the metafunction I concieved yesterday requires
a complete type, yet, under rather specific circumstances, when instantiating
mutually dependent templates (in our case lib::diff::Record<GenNode> is a
recursive type), the distinction between "complete" and "incomplete"
becomes blurry, and depends on the processing order. Which gave the
misleading impression as if there was a side-effect where the presence
of one definition changes the meaning of another one used in the same
program. What happened in fact was just that the evaluation order was
changed, causing the metafunction to fail silently, thus picking
another specialisation.
- we do strip references
- we delegate to nested typedefs
Hoever, we do *not* treat const or pointers in any way special --
if the user want to strip or level these, he has to do so explicitly.
Initially it seemed like a good idea to do something clever here, but
on the long run, such "special treatment" is just good for surprises
...automatically whenever those are present.
Up to now, we hat that as base case, which limited usage to those cases
where we already know such nested definitions are actually present
attempt to re-use the same traits as much as possible
NOTE: new code not passing compiler yet, but refactored old code
does, and still passes unit test
this is a subtle change which, given all interfaces were used in a logically
consistent way, should not cause any observable change to the yielded elements.
But it changes runtime behaviour, insofar now the evalutaion is initiated
lazily, when first requesting a result type. Prior to this change, the
constructor immediately issued a call to the yield() extension point,
which presumably has the side-effect of preparing the core and initiating
any embedded evaluation, in order to get at the first result; it might
even detect an empty state.
Given the fact that all access operations on the iterator front-end perform
an empty check (and possibly throw at that point), this call is redundant.
surprising behaviour encountered while covering more cases
...obviously the return type of ExpandFunctor::operator()
was inferred as value, even while the invoked functor, from which
this type was deduced, clearly returns a reference.
Solution is simple not to rely on inference, moreover since we know
the exact type in the enclosing scope, thanks to the refactoring which
made this ExpandFunctor a nested class
NOTE:
as it turned out, this is not a compiler bug,
but works as defined by the language:
on return type inference, the detected type is decayed,
which usually helps to prevent returning a reference to a temporary
