...which can be achieved by checking the backtracking loop
always right after the non-backtracking iteration, exploiting
the fact that the guard conditions of both are complimentary.
So the only case when we'd actually enter the backtracking
loop after regular iteration would precisely be when
we drop down due to exahausting an upper layer.
The result now reads
"sausage-bacon-tomato-and-spam-spam-bacon-spam-tomato-and-spam-bacon-tomato-and-bacon-tomato-and-tomato-and"
...which sounds correct, yay!
...since usually such evaluation layers are finally wrapped into
an IterableDecorator and then presented as TreeEplorer -- an exercise
we do not want to perform here, since it is pointless in the typicall
use case. The IterChainSearch is already meant to be ready-for-use.
Thus, instead of wrapping again, the pragmatic solution is simply
to overload the missing operator++ and make it call the augmented
iterNext() function. Related to this, we also need to ensure
proper operation in case no further expansion is mandated
...seems basically sane now.
Just we still need to wrap it one more time into IterableDecorator;
which means the overall scheme how to build and package the whole pipeline
is not correct yet.
Maybe it is not possible to get it packaged all into one single class?
on closer investigation it turned out that the logic of the
first design attempt was broken altogether. It did not properly
support backtracking (which was the reason to start this whole
exercise) and it caused dangling references within the lambda
closure once the produced iterator pipeline was moved out
into the target location.
Reasoning from first principles then indicated that the only sane
way to build such a search evaluation component is to use *two*
closely collaborating layers. The actual filter configuration
and evaluation logic can not reside and work from within the
expander. Rather, it must sit in a layer on top and work in
a conventional, imperative way (with a while loop).
Sometimes, functional programming is *not* the natural way
of doing things, and we should then stop attempting to force
matters against their nature.
this is an rather obvious extension to the TreeExplorer framework.
In some cases, client code wants to define its own very specific
processing layers, beyond of what can be done with filters and
transformers. Obviously, writing such a custom layer requires
intimate knowledge about the internals of TreeExplorer
the actual use case prompting this extension is IterChainSearch;
it turned out that the original design can not be implemented,
unless the resulting object is non-copyable (which violates
the basic traits of a TreeExplorer based pipeline).
...and TADAA ... there we get an insidious bug:
we capture *this by reference into the expansion functor,
and then we move *this away, out from the builder into the target....
Up to now, we had a very simplistic configuration option just
to search for a match, and we had the complete full-blown reconfiguration
builder option, which accepts a functor to work on and reconfigure the
embedded Filter chain.
It occurred to me that in many cases you'd rather want some intermediary
level of flexibility: you want to replace the filter predicate entirely
by some explicitly given functor, yet you don't need the full ability
to re-shape the Filter chain as a whole. In fact the intended use case
for IterChainSearch (which is the EventLog I am about to augment with
backtracking capabilities) will only ever need that intermediate level.
Thus wer're adding this intermediary level of configurability now.
The only twist is that doing so requires us to pass an "arbitrary function like thing"
(captured by universal reference) through a "layer of lambdas". Which means,
we have to capture an "arbitrary thingie" by value.
Fortunately, as I just found out today, C++14 allows something which comes
close to that requirement: the value capture of a lambda is allowe to have
an intialiser. Which means, we can std::forward into the value captured
by the intermediary lambda. I just hope I never need to know or understand
the actual type this captured "value" takes on.... :-)
with the augmented TreeExplorer, we're now able to get rid of the
spurious base layer, and we're able to discard the filter and
continue with the unfiltered sequence starting from current position.
build a special feature into the Explorer component of TreeExplorer,
causing it to "lock into" the current child sequence and discard
all previous sequences from the stack of child explorations
There is an asymetry, insofar the base layer configuration is
evaluated immediately, causing the MutableFilter to be reconfigured
and forwarded to the first match.
to the contrary, when configuring an additional layer, we just
add it to the chain, but then need to iterate once to cause
this configuration actually to be unfolded onto the stack
...which just turns the pipeline into exhausted state,
instead of raising an Assertion failure
The point is, expandChildren() does not guard itself,
since it _requires_ an non-empty iterator as precondition.
Thus, any function downstream, which invokes expandChildren(),
has to check and guard this call apropriately.
In the concrete case at hand we just stop adding further constraints
when the pipeline is already in exhausted state
...the solution built thus far was logically broken, since it retained the unfiltered
source sequence within the base layer. Thus it would backtrack into this unfiltered
sequence eventually.
The idea was to build a special treatment for attaching the first filter condition;
in fact the first one does not need to be added to the chain, but rather should be
planted directly into the base layer.
WIP: this is a solution draft, but does not work yet
- when attaching the base layer, the filter is pulled twice
- an overconstrained filter raises an Assertion failure
(instead of just transitioning into exhausted state)
So we have now a reworked version of the internals of TreeExplorer in place.
It should be easier to debug template instantation traces now, since most
of the redundancy on the type parameters could be remove. Moreover, existing
pipelines can now be re-assigned with similarily built pipelines in many cases,
since the concrete type of the functor is now erased.
The price tag for this refactoring is that we have now to perform a call
through a function pointer on each functor invocation (due to the type erasure).
And seemingly the bloat in the debugging information has been increased slightly
(this overhead is removed by stripping the binary)
Here the design trardeoff becomes clearly visiblie
- on the plus side, we removed that spurous redundant info
from the template parameter, and we simplified functor rebinding
- but as a tradeoff, we now always have two std::function objects
nested into each other, which also means that at least the outer
object resides on the heap and /inevitably/ calls through a
function pointer, even in case the target function is a lambda,
simply because some type erasure happened, and the call site
does not know the actual type anymore
...step by step switch over to the new usage pattern.
Transformer should be the blueprint for all other functor usages.
The reworked solutions behaves as expected;
we see two functor invocations; the outer functor, which does
the argument adaptation, is allocated in heap memory
This does not touch the existing code-path,
but the idea is to use the _FunTraits directly from within the
constructor of the respective processing layer, and to confine the
knowledge of the actual FUN functor type to within that limited context.
Only the generic signature of the resulting std::function need to be
encoded into the type of the processing component, which should help
to simplify the type signatures
...and in preparation start with some renamings...
The overall goal is to simplify the type signatures and thereby
to make the generates pipelines more assignment compatible.
The debugging experience form the last days indicated that the
current design is not maintainable on the long run. Both the
template instantiation chains and the call stacks are way to
complicated and hard to understand and diagnose
It is essential that we pass the current state of the filter
into the expand functor, where it needs to be copied (once!)
to create a child state, which can then be augmented.
This augmented state is then pushed onto a stack, to enable backtracking.
Due to the flexible adapters and the wrapping into the TreeExplorer builder,
we ended up performing several spurious copies on the current state
...based on a monadic tree expansion: we define a single step,
which takes the current filter configuration and builds the next
filter configuration, based on a stored chain of configuration functions
The actual exhausting depth-first results just by the greedy application pattern,
and uses the stack embedded in the "Explorer" layer of TreeExplorer
..this resolves the most challenging part of the construction work;
we use the static helper functions to infer a type and construct a suitable
processing pipeline and we invoke the same helper to initialise the base class
in the ctor.
Incidentally... we can now drop all the placeholder stubs,
since we now inherit the full iterator and child explorer API.
The test now starts actually to work... we get spam and sausage!
TODO: now actually fill in the expand functor such as to pick the
concrete filter step in the chain from a sequence of preconfigured
filter bindings
...now matters start to get really nasty,
since we have to pick up an infered type from a partially built pipeline
and use it to construct the signature for a functor to bind into the more elaborate complete pipeline
this is a tricky undertaking, since our treeExplore() helper constructs
a complex wrapped type, depending on the actual builder expressions used.
Solution is to use decltype on the result of a helper function,
and let the _DecoratorTraits from TreeExplorer do the necessary type adaptations
The intention is to augment the iterator based (linear) search
used in EventLog to allow for real backtracking, based on a evaluation tree.
This should be rather staight forward to implement, relying on the
exploreChildren() functionality of TreeExplorer. The trick is to package
the chained search step as a monadic flatMap operation
while this is basically a drop-in replacement,
it marks the switch to the monadic evaluation technology,
which is prerequisite for building real backtracking into the search.
we did an unnecessary copy of the argument, which was uncovered
by the test case manipulating the state core.
Whew.
Now we have a beautiful new overengineered solution
outift the Filter base class with the most generic form of the Functor
wrapper, and rather wrap each functor argument individually. This allows
then to combine various kinds of functors
...this solution works, but has a shortcoming:
the type of the passed lambdas is effectively pinned to conform
with the signature of the first lambda used initially when building the filter.
Well, this is the standard use case, but it kind of turns all the
tricky warpping and re-binding into a nonsense excercise; in this form
the filter can only be used in the monadic case (value -> bool).
Especially this rules out all the advanced usages, where the filter
collaborates with the internals of the source.
while this is basically just code code cosmetics,
at least it marks this as a very distinct special case,
and keeps the API for the standard Filter layer clean.
a quite convoluted construct built from several nested generic lambdas.
When investigated in the debugger, the observed addresses and the
invoked code looks sane and as expected.
The intention is to switch from the itertools-based filter
to the filter available in the TreeExplorer framework.
Thus "basically" we just need to copy the solution over,
since both are conceptually equivalent.
However...... :-(
The TreeExplorer framework is designed to be way more generic
and accepts basically everything as argument and tries to adapt apropriately.
This means we have to use a lot of intricate boilerplate code,
just to get the same effect that was possible in Itertools with
a simple and elegant in-place lambda assignment
Fillter needs to be re-evaluated, when an downstream entity requests
expandChildren() onto an upstream source. And obviously the ordering
of the chained calls was wrong here.
As it turns out, I had discovered that necessity to re-evaluate with
the Transformer layer. There is a dedicated test case for that, but
I cut short on verifying the filter in that situation as well, so
that piece of broken copy-n-paste code went through undetected.
This is in fact a rather esoteric corner case, because it is only
triggered when the expandChildren() call is passed through the filter.
When otoh the filter sits /after/ the entity generating the expandChildren()
calls, everything works as intended. And the latter is the typical standard
usage situation of an recursive evalutation algorithm: the filter is here
used as final part to drive the evaluation ahead and pick the solutions.
There is a bug or shortcoming in the existing ErrorLog matcher implementation.
It is not really difficult to fix, however doing so would require us to intersperse
yet another helper facility into the log matcher. And it occurred to me, that
this helper would effectively re-implement the stack based backtracking ability,
which is already present in TreeExplorer (and was created precisely to support
this kind of recursive evaluation strategies).
Thus I intend to switch the implementation of the EventLog matcher from the
old IterTool framework to the newer TreeExplorer framework. And this intention
made me re-read the code, fixing several comments and re-thinking the design
seemingly my quick-n-dirty implementation was to naiive.
We need real backtracking, if we want to support switches
in the search direction (match("y").after("x").before("z")
Up to now, I have cheated myself around this obvious problem :-/
as it turns out, we need to set the property_expand() on the child widget
within Gtk::Expander explicitly, to cause the child to grab and additional
available screen space (which obviously is what we want in case of a
log display with scrollbars)
basically Gtk::Expander will do the trick.
However, resizing of the enclosing panel is not handled properly,
the log does not expand to take up the available space, as it did
automaticall when just added directly into the frame
no need to define a private function on Wizard anymore, it just forwards the call
to the service actually implementing the view allocation. For now this is the
PanelLocator (and eventually this will be the ViewLocator / ViewSpecDSL)
PanelLocator is a sub component of the WindowLocator (top-level GUI service).
Eventually this shall become a mere widget/component access service, with the
actual lookup and allocation logic layered on top through ViewLocator, configurable
via ViewSpec-DSL.
We can not implement the full scheme right now, since we're lacking knowledge
about internals of a typical Lumiera UI widget
This is only a premature hack, since the whole structure of PanelManager is somewhat broken.
Moreover, the ViewLocator is not really ready for use yet, so this hack at least
allows us to "reach into" a top-level window and "grab" the pannel we need.
* have a dedicated "information hub" controller, which acts a receiver of "error log messages" on the UI-Bus
* let that controller in turn allocate an apropriate view on demand
The goal is to build a (in itself completely meaningless) ping-pong interaction
between the UI and Proc-Layer, for the purpose of driving the integration ahead.
The immediate challenge is how to create and place an apropriate "GuiComponentView",
i.e. a Tangible, which is connected to the UI-Bus with an predictable EntryID.
And the problem is to get that settled right now, without building the envisioned
generic framework for View allocation in the UI. When this is achieved,
it should be a rather small step to actually send those notifications over
the UI-Bus, which is basically implemented and ready by now.
right now this will just end up in the log, since not even the
notification display is implemented beyond the GuiNotification-facade.
Anyway, we get some kind of communication now for real, in the actual application
...because due of #211, we usually don't execute commands yet.
For now there is only the backdoor to prefix the command-ID with "test"
With this change, the TODO message appears now immediately after GUI start!
In the end, I decided against building a generic service here,
since it pretty much looks like a one-time problem.
Preferrably UI content will be pushed or pulled on demand,
rather than actively coding content from within the UI-Layer
- activation signal is a facility offered and used solely by Gtk::Application
- we do not need nor want an Gtk::Application, we deal with our own application
concerns as we see fit.
Gio::Application holds a signal_activation(), which seems to be used for
precisely that task we need here: to do something right after the UI is operative
...and while doing so, also re-check the state of the GTK toolkit initialisation.
Looks like we're still future-proof, while cunningly avoiding all this
Gnome-style "Application" blurb
I will abandon work on the ViewSpec DSL in current shape (everything fine with that)
and instead work on a general UI start-up and content population sequence.
From there, my intention is to return to the docks, the placement of views
and then finally to the TimelineView
This finishes the first round of design drafts in this area.
Right now it seems difficult to get any further, since most of
the actual view creation and management in the UI is not yet coded.
looks like I'm trapped with the choice between a convoluted API design
and an braindead and inefficient implementation. I am leaning towards the latter
looks like we're hitting a design mismatch here....
...and unfortunately I have to abandon this task now and concentrate
on preparation of my talk at LAC.2018 in June
this is a (hopefully just temporary) workaround to deal with static initialisation
ordering problems. The original solution was cleaner from a code readability viewpoint,
however, when lib::Depend was used from static initialisation code, it could
be observed that the factory constructor was invoked after first use.
And while this did not interfer with the instance lifecycle management itself,
because the zero-initialisation of the instance (atomic) pointer did happen
beforehand, it would discard any special factory functions installed from such
a context (and this counts as bug for my taste).
The original goal for #1129 (ViewSpecDSL_test) is impossible to accomplish,
at least within our existing test framework. Thus I'll limit myself to coding
a clean-room integration test with purely synthetic DSL definitions and mock widgets
Problem is, we can not even compile the conversion in the "other branch".
Thus we need to find some way to pick the suitable branch at compile time.
Quite similar to the solution found for binding Rec<GenNode> onto a typed Tuple
Attempt to find my way back to the point
where the digression regarding dependency-injection started.
As it turns out, this was a valuable digression, since we can rid ourselves
from lots of ad-hoc functionality, which basically does in a shitty way
what DependencyFactory now provides as standard solution
FIRST STEP is to expose the Navigator as generic "LocationQuery" service
through lib::Depend<LocationQuery>
more of a layout improvement, to avoid any code duplication.
The mechanics remain the same
- write an explicit specialisation
- trigger template intantiation within a dedicated translation unit
from now on, we'll have dedicated individual translation units (*cpp)
for each distinct interface proxy. All of these will include the
interfaceproxy.hpp, which now holds the boilerplate part of the code
and *must not be included* in anything else than interfac proxy
translation units. The reason is, we now *definie* (with external linkage)
implementations of the facade::Link ctor and dtor for each distinct
type of interface proxy. This allows to decouple the proxy definition code
from the service implementation code (which is crucial for plug-ins
like the GUI)
The recently rewritten lib::Depend front-end for service dependencies,
together with the configuration as lib::DependInject::ServiceInstance
provides all the necessary features and is even threadsafe.
Beyond that, the expectation is that also the instantiation of the
interface proxies can be simplified. The proxies themselves however
need to be hand-written as before
I am fully aware this change has some far reaching ramifications.
Effectively I am hereby abandoning the goal of a highly modularised Lumiera,
where every major component is mapped over the Interface-System. This was
always a goal I accepted only reluctantly, and my now years of experience
confirm my reservation: it will cost us lots of efforts just for the
sake of being "sexy".
Actually this is on the implementation side only.
Since Layer-Separation-Interfaces route each call through a binding layer,
we get two Service-"Instances" to manage
- on the client side we have to route into the Lumiera Interface system
- on the implementation side the C-Language calls from the Interface system
need to get to the actual service implementation. The latter is now
managed and exposed via DependInject::ServiceInstance
...still using the FAKE implementation, not a real rules engine.
However, with the new Dependency-Injection framework we need to define
the actual class from the service-provider, not from some service-client.
This is more orthogonal, but we're forced to install a Lifecycle-Hook now,
in order to get this configuration into the system prior to any use
This is borderline yet acceptable;
A service might indeed depend on itself circularly
The concrete example is the Advice-System, which needs to push
the clean-up of AdviceProvicions into a static context. From there
the deleters need to call back into the AdviceSystem, since they have
no wey to find out, if this is an individual Advice being retracted,
or a mass-cleanup due to system shutdown.
Thus the DependencyFactory now invokes the actual deleter
prior to setting the instance-Ptr to NULL.
This sidesteps the whole issue with the ClassLock, which actually
must be already destroyed at that point, according to the C++ standard.
(since it was created on-demand, on first actual usage, *after* the
DependencyFactory was statically initialised). A workaround would be
to have the ctor of DependencyFactory actively pull and allocate the
Monitor for the ClassLock; however this seems a bit overingeneered
to deal with such a borderline issue
Static initialisation and shutdown can be intricate; but in fact they
work quite precise and deterministic, once you understand the rules
of the game.
In the actual case at hand the ClassLock was already destroyed, and
it must be destroyed at that point, according to the standard. Simply
because it is created on-demand, *after* the initialisation of the
static DependencyFactory, which uses this lock, and so its destructor
must be called befor the dtor of DependencyFactory -- which is precisely
what happens.
So there is no need to establish a special secure "base runtime system",
and this whole idea is ill-guided. I'll thus close ticket #1133 as wontfix
Conflicts:
src/lib/dependable-base.hpp
When some dependency or singleton violates Lumiera's policy regarding destructors and shutdown,
we are unable to detect this violation reliably and produce a Fatal Error message.
This is due to lib::Depend's de-initialisating being itself tied to template generated
static variables, which unfortunately have a visibility scope beyond the translation unit
responsible for construction and clean-up.
- state-of-the-art implementation of access with Double Checked Locking + Atomics
- improved design for configuration of dependencies. Now at the provider, not the consumer
- support for exposing services with a lifecycle through the lib::Depend<SRV> front-end
...which declare DependencyFactory as friend.
Yes, we want to encourrage that usage pattern.
Problem is, std::is_constructible<X> gives a misleading result in that case.
We need to do the instantiation check within the scope of DependencyFactory
ideally we want
- just a plain unique_ptr
- but with custom deleter delegating to lib::Depend
- Depend can be made fried to support private ctor/dtor
- reset the instance-ptr on deletion
- always kill any instance
all these tests are ported by drop-in replacement
and should work afterwards exactly as before (and they do indeed)
A minor twist was spotted though (nice to have more unit tests indeed!):
Sometimes we want to pass a custom constructor *not* as modern-style lambda,
but rather as direct function reference, function pointer or even member
function pointer. However, we can not store those types into the closure
for later lazy invocation. This is basically the same twist I run into
yesterday, when modernising the thread-wrapper. And the solution is
similar. Our traits class _Fun<FUN> has a new typedef Functor
with a suitable functor type to be instantiated and copied. In case of
the Lambda this is the (anonymous) lamda class itself, but in case of
a function reference or pointer it is a std::function.
...which showed up under high system load.
The initialisation of the member variables for the check sum
could be delayed while the corresponding thread was already running
- polish the text in the TiddlyWiki
- integrate some new pages in the published documentation
Still mostly placeholder text with some indications
- fill in the relevant sections in the overview document
- adjust, expand and update the Doxygen comments
TODO: could convert the TiddlyWiki page to Asciidoc and
publish it mostly as-is. Especially the nice benchmarks
from yesterday :-D
This solution is considered correct by the experts.
Regarding the dependency-configuration part, we do not care too much about performance
and use the somewhat slower default memory ordering constraint
This is essentially the solution we used since start of the Lumiera project.
This solution is not entirely correct in theory, because the assignment to the
instance pointer can be visible prior to releasing the Mutex -- so another thread
might see a partially initialised object
_not_ using the dependency factory, rather direct access
- to a shared object in the enclosing stack frame
- to a heap allocated existing object accessed through uniqe_ptr
...written as byproduct from the reimplementation draft.
NOTE there is a quite similar test from 2013, DependencyFactory_test
For now I prefer to retain both, since the old one should just continue
to work with minor API adjustments (and thus prove this rewrite is a
drop-in replacement).
On the long run those two tests could be merged eventually...
This is a complete makeover of our lib::Depend and lib::DependencyFactory templates.
While retaining the basic idea, the configuration has been completely rewritten
to favour configuration at the point where a service is provided rather,
than at the point where a dependency is used.
Note: we use differently named headers, so the entire Lumiera
code base still uses the old implementation. Next step will be
to switch the tests (which should be drop-in)
explicit friendship seems adequate here
DependInject<SRV> becomes more or less a hidden part of Depend<SRV>,
but I prefer to bundle all those quite technical details in a separate
header, and close to the usage
This is a tricky problem an an immediate consequence of the dynamic configuration
favoured by this design. We avoid a centralised configuration and thus there
are no automatic rules to enforce consistency. It would thus be possible
to start using a dependency in singleton style, but to switch to service
style later, after the fact.
An attempt was made to prevent such a mismatch by static initialisiation;
basically the presence of any Depend<SRV>::ServiceInstance<X> would disable
any usage of Depend<SRV> in singleton style. However, such a mechanism
was found to be fragile at best. It seems more apropriate just to fail
when establishing a ServiceInstance on a dependency already actively in
use (and to lock usage after destroying the ServiceInstance).
This issue is considered rather an architectural one, which can not be
solved by any mechanism at implementation level ever
up to now we used placement into a static buffer.
While this approach is somewhat cool, I can't see much practical benefit anymore,
given that we use an elaborate framework which rules out the use of Meyers Singleton.
And given that with C++11 we're able just to use std::unique_ptr to do all work.
Moreover, the intended configurability will become much simpler by relying
on a _closure_ to produce a heap-allocated instance for all cases likewise.
The only possible problem I can see is that critical infrastructure might
rely on failsafe creation of some singleton. Up to now this scenario
remains theoretical however
Meyers Singleton is elegant and fast and considered the default solution
However...
- we want an "instance" pointer that can be rebound and reset,
and thus we are forced to use an explicit Mutex and an atomic variable.
And the situation is such that the optimiser can not detect/verify this usage
and thus generates a spurious additional lock for Meyers Singleton
- we want the option to destroy our singletons explicitly
- we need to create an abstracted closure for the ctor invocation
- we need a compiletime-branch to exclude code generation for invoking
the ctor of an abstract baseclass or interface
All those points would be somehow manageable, but would counterfeit the
simplicity of Meyers Singleton
Problems:
- using Meyers Singleton plus a ClassLock;
This is wasteful, since the compiler will emit additional synchronisation
and will likely not be able to detect the presence of our explicit locking guard
- what happens if the Meyers Singleton can not even be instantiated, e.g. for
an abstract baseclass? We are required to install an explicit subclass configuration
in that case, but the compiler is not able to see this will happen, when just
compiling the lib::Depend
Most dependencies within Lumiera are singletons and this approach remains adequate.
Singletons are not "EVIL" per se. But in some cases, there is an explicit
lifecycle, managed by some subsystem. E.g. some GUI services are only available
while the GTK event loop is running.
This special case can be integrated transparently into our lib::Depend<TY> front-end,
which defaults to creating a singleton otherwise.
we'll use a typedef to represent the default case
and provide the level within the UI-Tree as template parameter for the generic case
This avoids wrapping each definition into a builder function, which will be
the same function for 99% of the cases, and it looks rather compact and natural
for the default case, while still retaining genericity.
Another alternative would have been to inject the Tree-level at the invocation;
but doing so feels more like magic for me.
decided to add a very specific preprocessing here, to make the DSL notation more natural.
My guess is that most people won't spot the presence of this tiny bit of magic,
and it would be way more surprising to have rules like
UICoord::currentWindow().panel("viewer").create()
fail in most cases, simply because there is a wildcard on the perspective
and the panel viewer does not (yet) exist. In such a case, we now turn the
perspective into a "existential quantified" wildcard, which is treated as if
the actually existing element was written explicitly into the pattern.
...actually just more test coverage,
the feature is already implemented.
What *could* be done though is to inject that UIC_ELIDED marker
on missing perspective specs in create clauses automatically...
This looks like YAGNI, and it would be non trivial to implement.
But since the feature looks important for slick UI behaviour,
I've made a new ticket and leave it for now
with the exception of some special situations,
which require additional features from the engine,
especially binding-on-context
Not sure though if I'll implement these or say YAGNI
turns out to be somewhat tricky.
The easy shot would be to use the comma operator,
but I don't like that idea, since in logic programming, comma means "and then".
So I prefer an || operator, similar to short-circuit evaluation of boolean OR
Unfortunately, OR binds stronger than assignment, so we need to trick our way
into a smooth DSL syntax by wrapping into intermediary marker types, and accept
rvalue references only, as additional safeguard to enforce the intended inline
definition syntax typical for DSL usage.
seems to be the most orthogonal way to strip adornments from the SIG type
Moreover, we want to move the functor into the closure, where it will be stored anyay.
From there on, we can pass as const& into the binder (for creating the partially closed functor)
...as it turned out, the result type was the problem: the lambda we provide
typically does not yield an Allocator, but only its baseclass function<UICoord(UICoord)>
solution: make Allocator a typedef, we don't expect any further functionality
...but not yet able to get it to compile.
Problem seems to be the generic lambda, which is itself a template.
Thus we need a way to instantiate that template with the correct arguments
prior to binding it into a std::function
been there, seen that recently (-> TreeExplorer, the Expander had a similar problem)
...this was quite an extensive digression, which basically gave us
a solid foundation for topological addressing and pattern matching
within the "interface space"
rationale: sometimes (likely this is even the standard case) we do not just
want to "extend", rather we want to extent at very specific levels.
This is easy to implement, based on the existing building blocks for path manipulation
the original construction works only as long as we stick to the "classical" Builder syntax,
i.e. use chained calls of the builder functions. But as soon as we just invoke
some builder function for sake of the side-effect on the data within the builder,
this data is destroyed and moved out into the value return type, which unfortunately
is being thrown away right afterwards.
Thus: either make a builder really sideeffect-free, i.e. do each mutation
on a new copy (which is kind of inefficient and counterfeits the whole idea)
or just accept the side-effect and return only a reference.
In this case, we can still return a rvalue-Reference, since at the end
we want to move the product of the build process out into the destination.
This works only due to the C++ concept of sequence points, which ensures
the original object stays alive during the whole evaluation of such a chained
builder expression.
NOTE: the TreeMutator (in namespace lib::diff) also uses a similar Builder construction,
but in *that* case we really build a new product in each step and thus *must*
return a value object, otherwise the reference would already be dangling the
moment we leave the builder function.
- the default should be to look for total coverage
- the predicates should reflect the actual state of the path only
- the 'canXXX' predicates test for possible covering mutation
I set out to "discover" what operations we actually need on the LocationQuery
interface, in order to build a "coordinate resolver" on top. It seems like
this set of operations is clear by now.
It comes somewhat as a surprise that this API is so small. This became possible
through the idea of a ''child iterator'' with the additional ability to delve down and
expand one level of children of the current element. Such can be ''implemented''
by relying on techniques similar to the "Monads" from functional programming.
Let's see if this was a good choice. The price to pay is a high level of ''formal precision''
when dealing with the abstraction barrier. We need to stick strictly to the notion of a
''logical path'' into a tree-like topology, and we need to be strong enough never to
give in and indulge with "the concrete, tangible". The concrete reality of a tree
processing algorithm with memory management plus backtracking is just to complex
to be handled mentally. So either stick to the rules or get lost.
yet some more trickery to get around this design problem.
I just do not want to rework IterSource right now, since this will be
a major change and require more careful consideration.
Thus introduce a workaround and mark it as future work
Using this implementation, "child expansion" should now be possible.
But we do not cover this directly in Unit test yet
...but not yet switched into the main LocationQuery interface,
because that would also break the existing implementation;
recasting this implementation is the next step to do....
...which basically allows us to return any suitable implementation
for the child iterator, even to switch the concrete iteration on each level.
We need this flexibility when implementing navigation through a concrete UI
This is just a temporary solution, until IterSource is properly refactored (#1125)
After that, IterSource is /basically a state core/ and the adaptor will be more or less trivial
- as it stands currently, IterSource has a design problem, (see #1125)
- and due to common problems in C++ with mix-ins and extended super interfaces,
it is surprisingly tricky to build on an extension of IterSource
- thus the idea is to draft a new solution "in green field"
by allowing TreeExplorer to adapt IterSource automatically
- the new sholution should be templated on the concrete sub interface
and ideally even resolve the mix-in-problem by re-linearising the
inheritance line, i.e. replace WrappedLumieraIter by something
able to wrap its source, in a similar vein as TreeExplorer does
...yet I do not want to move all of the traits over into the
publicly visible lib::iter_explorer namespace -- I'm quite happy
with these traits being clearly marked as local internal details
NOTE it just type checks right now,
but since meta programming is functional programming, this means
with >90% probability that it might actually work this way....
...which also happens to include sibling and child iteration;
this is an attempt to reconcile the inner contradictions of the design
(we need both absolute flexibility for the type of each child level iterator
yet we want just a single, generic iterator front-end)
...this was a difficult piece of consideration and analysis.
In the end I've settled down on a compromise solution,
with the potential to be extended into the right direction eventually...
surprise: the standard for-Loop causes a copy of the iterator.
From a logical POV this is correct, since the iterator is named,
it can not just be moved into the loop construct and be consumed.
Thus: write a plain old-fashioned for loop and consume the damn thing.
So the top-level call into util::join(&&) decides, if we copy or consume
several extensions and convenience features are conceivable,
but I'll postpone all of them for later, when actual need arises
Note especially there is one recurring design challenge, when creating
such a demand-driven tree evaluation: more often than not it turns out
that "downstream" will need some information about the nested tree structure,
even while, on the surfice, it looks as if the evaluation could be working
completely "linearised". Often, such a need arises from diagnostic features,
and sometimes we want to invoke another API, which in turn could benefit
from knowing something about the original tree structure, even if just
abstracted.
I have no real solution for this problem, but implementing this pipeline builder
leads to a pragmatic workaround: since the iterator already exposes a expandChildren(),
it may as well expose a depth() call, even while keeping anything beyond that
opaque. This is not the clean solution you'd like, but it comes without any
overhead and does not really break the abstraction.
...so sad.
The existing implementation was way more elegant,
just it discarded an exahusted parent element right while in expansion,
so effectively the child sequence took its place. Resolved that by
decomposing the iterNext() operation. And to keep it still readable,
I make the invariant of this class explicit and check it (which
caught yet another undsicovered bug. Yay!)
instead of building a very specific collaboration,
rather just pass the tree depth information over the extended iterator API.
This way, "downstream" clients *can* possibly react on nested scope exploration
We get conflicting goals here:
- either the child expansion happens within the opaque source data
and is thus abstracted away
- or the actual algorithm evaluation becomes aware of the tree structure
and is thus able to work with nested evaluation contexts and a local stack
...build on top of the core features of TreeExplorer
- completely encapsulate and abstract the source data structure
- build an backtracking evaluation based on layered evaluation
of this abstracted expandable data source
NOTE: test passes compilation, but doesn't work yet
...and there is a point where to stop with the mere technicalities,
and return to a design in accordance with the inner nature of things.
Monads are a mere technology, without explicatory power as a concept or pattern
For that reason
- discard the second expansion pattern implemented yesterday,
since it just raises the complexity level for no given reason
- write a summary of my findings while investigating the abilities
of Monads during this design excercise.
- the goal remains to abandon IterExplorer and use the now complete
IterTreeEplorer in its place. Which also defines roughly the extent
to wich monadic techniques can be useful for real world applications
...it can sensibly only be done within the Expander itself.
Question: is this nice-to-have-feature worth the additional complexity
of essentially loading two quite distinct code paths into a single
implementation object?
As it stands, this looks totally confusing to me...
At that time, our home-made Tuple type was replaced by std::tuple,
and then the command framework was extended to also allow command invocation
with arguments packaged as lib::diff::Record<GenNode>
With changeset 0e10ef09ec
A rebinding from std::tuple<ARGS...> to Types<ARGS> was introduced,
but unfortunately this was patched-in on top of the existing Types<ARGS...>
just as a partial specialisation.
Doing it this way is especially silly, since now this rebinding also kicks
in when std::tuple appears as regular payload type within Types<....>
This is what happened here: We have a Lambda taking a std::tuple<int, int>
as argument, yet when extracting the argument type, this rebinding kicks in
and transforms this argument into Types<int, int>
Oh well.
this leads to either unfolding the full tree depth-first,
or, when expanding eagerly, to delve into each sub-branch down to the leaf nodes
Both patterns should be simple to implement on top of what we've built already...
IterSource should be refactored to have an iteration control API similar to IterStateWrapper.
This would resolve the need to pass that pos-pointer over the abstraction barrier,
which is the root cause for all the problems and complexities incurred here
...but for now the price is that we need to punch a hole into IterAdapter.
And obviously, this is all way to tangled and complex on implementation level.
turns out that -- again -- we miss some kind of refresh after expanding children.
But this case is more tricky; it indicates a design mismatch in IterSource:
we (ab)use the pos-pointer to communicate iteration state. While this might be
a clever trick for iterating a real container, it is more than dangerous when
applied to an opaque source state as in this case. After expanding children,
the pos-pointer still points into the cache buffer of the last transformer.
In fact, we miss an actualisation call, but the IterSource interface does not
support such a call (since it tries to get away with state hidden in the pos pointer)
as it turned out, when "inheriting" ctors, C++14 removes the base classes' copy ctors.
C++17 will rectify that. Thus for now we need to define explicitly that
we'll accept the base for initialising the derived. But we need do so
only on one location, namely the most down in the chain.
Since this now requires to import iter-adapter-stl.hpp and iter-source.hpp
at the same time, I decided to drop the convenience imports of the STL adapters
into namespace lib. There is no reason to prefer the IterSource-based adapters
over the iter-adapter-stl.hpp variants of the same functionality.
Thus better always import them explicitly at usage site.
...actual implementation of the planned IterSource packaging is only stubbed.
But I needed to redeclare a lot of ctors, which doesn't seem logical
And I get a bad function invocation from another test case which worked correct beforehand.
We need a way for higher layers to discard their caching and re-evaluate,
once some expansion layer was invoked to replace the current element with
its (functionally defined) "children" -- otherwise the first child will
remain obscured by what was there beforehand.
Solution is to pass such manipulation calls through the full chain of
decorators, allowing them to refresh themselves when necessary. To achieve
that technially, we add a base layer to absorb any such call passed down
through the whole decorator chain -- since we can not assume that the
parent, the original source core implements those manipualation calls
like expandChildren()
Considering the fact that we are bound to introduce yet another iteration control function,
because there is literally no other way to cause a refresh within the IterTreeExplorer-Layers,
it is indicated to reconsider the way how IterStateWrapper attaches to the
iteration control API.
As it turns out, we'll never need an ADL-free function here;
and it seems fully adequate to require all "state core" objects to expose
the API as argument less member function. Because these reflect precisely
the contract of a "state core", so why not have them as member functions.
And as a nice extra, the implementation becomes way more concise in
all the cases refactored with this changeset!
Yet still, we stick to the basic design, *not* relying on virtual functions.
So this is a typical example of a Type Class (or "Concept" in C++ terminology)
good news: it (almost) works out-of-the-box as expected.
There is only one problem: expandChildren() alters the content of the
data source, yet downstream decorators aren't aware of that fact and
continue to present cached evaluations, until the next iterate() call
is issued. Yet unfortunately this iterate already consumes the first
of the expanded children, which thus gets shadowed by the cached
outcome of parent node already consumed and expanded at that point
See the first example:
"10-8-expand-8-4-2-6-4-2"
should be 6 ^^^
...which happens to be supported out of the box,
due to the generic adaptor magic shared with the explore-operation
Exploiting this feature, some functor could even subvert the layering order
- always layer the TreeExplorer (builder) on top of the stack
- always intersperse an IterableDecorator in between adjacent layers
- consequently...
* each layer implementation is now a "state core"
* and the source is now always a Lumiera Iterator
This greatly simplifies all the type rebindings and avoids the
ambiguities in argument converison. Basically now we can always convert
down, and we just need to pick the result type of the bound functor.
Downside is we have now always an adaptation wrapper in between,
but we can assume the compiler is able to optimise such inline
accessors away without overhead.
...yet this seems like a rather bad idea,
it breeds various problems and requires arcane trickery to make it fly
==> abandon this design
==> always intersperse an IterableDecorator between each pair of Layers
...especially relevant in the context of TreeExplorer,
where the general understanding is that the "Data Source" (whatever it is)
will be piggy-backed into the pipeline builder, and this wrapping is
conceived as being essentially a no-op.
It is quite possible we'll even start using such pipeline builders
in concert with move-only types. Just consider a UI-navigator state
hooked up with a massive implementation internal pointer tree attached
to all of the major widgets in the UI. Nothing you want to copy in passing by.
As it turned out, we had two bugs luring in the code base,
with the happy result of one cancelling out the adverse effects of the other
:-D
- a mistake in the invocation of the Itertools (transform, filter,...)
caused them to move and consume any input passed by forwarding, instead
of consuming only the RValue references.
- but util::join did an extraneous copy on its data source, meaning that
in all relevant cases where a *copy* got passed into the Itertools,
only that spurious temporary was consumed by Bug #1.
(Note that most usages of Itertools rely on RValues anyway, since the whole
point of Itertools is to write concise in-line transformation pipelines...)
*** Added additional testcode to prove util::stringify() behaves correct
now in all cases.
Obsoletes and replaces the ad-hoc written type rebindings from
iter-adapter and friends. The new scheme is more consistent and does
less magic, which necessitates an additional remove_pointer<IT> within
the iterator adaptors. Rationale is, "pointer" is treated now just as
a primitive type without additional magic or unwrapping, since it is
impossible to tell generically if the pointer or the pointee was
meant to be the "value"
Oh well.
This kept me busy a whole day long -- and someone less stubborn like myself
would probably supect a "compiler bug" or put the blame on the language C++
So to stress this point: the compiler behaved CORRECT
Just SFINAE is dangerous stuff: the metafunction I concieved yesterday requires
a complete type, yet, under rather specific circumstances, when instantiating
mutually dependent templates (in our case lib::diff::Record<GenNode> is a
recursive type), the distinction between "complete" and "incomplete"
becomes blurry, and depends on the processing order. Which gave the
misleading impression as if there was a side-effect where the presence
of one definition changes the meaning of another one used in the same
program. What happened in fact was just that the evaluation order was
changed, causing the metafunction to fail silently, thus picking
another specialisation.
attempt to re-use the same traits as much as possible
NOTE: new code not passing compiler yet, but refactored old code
does, and still passes unit test
...which uncovered an error in the test fixture
plus helped to spot the spurious copy when passing the argument to the expand functor
And my GDB crashed when loading the executable, YAY!
so we'll need to coment out some code from now on,
until we're able to switch to a more recent toolchain (#1118)
...while this implementation works now, it is still very complex and intricate.
I am still doubtful this is a good approach, but well, we need to try that route....
but possible only for the iterator -> iterator case
Since we can not "probe" a generic lambda, we get only one shot:
we can try to bind it into a std::function with the assumed signature
...since all those metaprogramming techniques rely on SFINAE,
but *instantiating* a template means to compile it, which is more
than just substituate a type into the signature
If forming the signature fails -> SFINAE, try next one
If instantiating a template fails -> compile error, abort
Basically we want to support two distinct cases, just by slightly adapting
the invocation of the expansion functor:
Case-1: classical monadic flatMap:
the Functor accepts a value yielded by the source iterator
and builds a new "expaneded" iterator
Case-2: manipulation of opaque implementation state
the Functor knows internal details of the source iterator
and thus takes the source iterator as such as argument,
performs some manipulation and then builds a new sub-iterator
A soulution to reconcile those two distinct cases can be built
with the help of a generic lambda
Here, the tricky question remains, how to relate this evalutaion scheme
to the well known monadic handling of collections and iterators.
It seems, we can not yet decide upon that question, rather we should
first try to build a concrete implementation of the envisioned algorithm
and then reconsider the question later, to what extent this is "monadic"
This can be seen as a side track, but the hope is
by relying on some kind of monadic evaluation pattern, we'll be
able to to reconcile the IterExplorer draft from 2012 with the requirement
to keep the implementation of "tree position" entirely opaque.
The latter is mandatory in the use case here, since we must not intermingle
the algorithm to resolve UI-coordinates in any way with the code actually
navigating and accessing GTK widgets. Thus, we're forced to build some kind
of abstraction barrier, and this turns out to be surprisingly difficult.
...which was deliberately represented in an asymmetric way, to verify the
design's ability to cope with such implementation intricacies. So basically
we have to kick in at LEVEL == 1 and access the implementation differently.
This exercise just shows again, that treating tree structures recursively
is the way to go, and we should do similar when coding up the query-API
for the real GTK toolkit based window elements...
...which can be helpful when a function usually returns a somewhat dressed-up iterator,
but needs to return a specific fixed value under some circumstances
this fixes a silly mistake:
obviously we want named sub-nodes, aka. "Attributes",
but we used the anonymous sub-nodes instead, aka. "Children"
Incidentally, this renders the definitions also way more readable;
in fact the strange post-fix naming notation of the original version
was a clear indication of using the system backwards....
obviously, we get a trivial case, when the path is explicit,
and we need a tricky full blown resolution with backtracking
when forced to interpolate wildcards to cover a given UICoord
spec against the actual UI topology.
Do we need it?
* actually not right now
* but already a complete implementation of the ViewSpec concept
requires such a resolution
...to limit them to the UI-Coordinates themselves,
while declining the possibility to mutate the target environment
through the PathResolver. Better handle changes within the
target environment by dedicated API calls on the target elements,
instead of creating some kind of "universal structure"
..this collection of ideas, terms and conclusions has been shaped
since some time within the TiddlyWiki. Since I've now started even
some supporting implementation regarding these concepts, its time
to publish them in the design documentation section of the Website
After completing the self-contained UICoord data elements,
the next thing to consider might be how to resolve UI coordinates
against an actual window topology. We need to define a suitable
command-and-query interface in order to build and verify this
intricate resolution process separated from the actual UI code.
exploring the idea of a configuration DSL.
As a first step, this could be a simple internal DSL,
implemented as a bunch of static functor objects, which are internally bound
and thus implemented by the ViewLocator within InteractionDirector
...we have to face the problem that we need some generic strategy
for access to component views, which possibly will become customisable.
And the allowed patterns of access are quite different for the various
kind of view we know....
responsible for access and allocation of component views.
Internally wired to the PanelLocator within the global WindowLocator
This setup settles those nasty qeustions of crosswise top-level access
this starts work on a new UI global topic (#1004)
- coin a new term: "view component"
- distinction between veiw component and Panel
- consider how to locate view components
- WindowList becomes WindowLocator
actually I do not know much regarding the actual situation when,
within the Builder run, we're able to detect a change and generate
a diff description. However, as a first step, I'll pick IterSrouce
as a base interface and use a "generation context", which is to be
passed by shared-ptr
the (trivial) implementation turned out to be correct as written,
but it was (again) damn challenging to get the mulithreaded chaotic
test fixture and especially the lambda captures to work correct.
- concept for a first preliminary implementation of dispatch into the UI thread
- define an integration effort to build a complete working communication chain
This change was caused by investigation of UI event loop dispatch;
since the GTK UI is designed to run single threaded, any invocation
from other threads need to be diepatched explicitly.
A possible way to achieve this is to use Glib::Dispatcher, which
in turn requires that the current thread (which is in this case the UI thread)
already holds a Glib::MainContext
This prompted me to create a tight link between the external facade interfaces
of the UI and the event loop itself. What remains to be settled is how
to hand over arguments to the action in the main loop
After investigation of current GTK and GIO code, I came to the conclusion
that we do *not* want to rely on the shiny new Gtk::Application, which
provides a lot of additional "convenience" functionality we do neither
need nor want. Most notably, we do not want extended desktop integration
like automatically connecting to D-Bus or exposing application actions
as desktop events.
After stripping away all those optional functions and extensions, it turns
out the basic code to operate the GTK main event loop is quite simple.
This changeset extracts this code from the (deprecated) Gtk::Main and
integrates it directly in Lumiera's UI framework object (UiManager).
this is just a tiny change to make things more othogonal.
Now the unwinding and calls to any GTK / Widget dtors happen *after*
emitting the term signal from UI shutdown. Which means, the other subsystems
are shutting down (in their dedicated threads) as well, thus lowering
the probability of some action still using the UI and triggering an exception
as it turned out, the former functionality was deactivated in 2009
with changeset 6151415
The whole concept seems to be unfinished, and needs to be reworked
and integrated with "Views and Perspectives" (whatever that is...)
See also #1097
Gtk::Main is deprecated, but the new solution, instantiating a
Gtk::Application object does not match our use case, since we handle
all application concerns already and just need a Gtk main loop to run.
Anyway, it became clear that the "main object" will be the new UiManager.
As a first step, I've now moved the (deprecated) Gtk::Main object
down there. Next step (planned) will be to inherit from Gio::Application
and clone some functionality from Gtk::Application
...which opens more questions than it solves at the moment.
Especially note #1096, the question how to refer to object-IDs
Maybe we need to enable sending EntryIDs via GenNode?
Anyway, the magic spell is broken now: we have a way how to
establish commands and how to issue them from the UI, with full integration
of UI-Bus, layer separation facade, instance management and ProcDispatcher
Looks like a stepping stone
after extended analysis, it turned out to be a "placeholder concept"
and introduces an indirection, which can be removed altogether
- simple command invocation happens at gui::model::Tangible
- it is based on the command (definition) ID
- instance management happens automatically and transparently
- the extended case of context-bound commands will be treated later,
and is entirely self-contained
while the initial design treated the commands in a strictly top-down manner,
where the ID is known solely to the CommandRegistry, this change and information
duplication became necessary now, since by default we now always enqueue and
dispatch anonymous clone copies from the original command definition (prototype).
This implementation uses the trick to tag this command-ID when a command-hanlde
is activated, which is also the moment when it is tracked in the registry.
in accordance to the design changes concluded yesterday.
- in the standard cases we now check the global registry first
- automatically create anonymous clone copy from global commands
- reorganise code internally to use common tail implementation
as it turns out, we can always trigger commands right away,
the moment all arguments are known. Thus it is sufficient to
send a single argument binding message, which allows us to
get rid of a lot or ugly complexities (payload visitor).
It seems more adequate to push the somewhat intricate mechanics
for the "fall back" onto generic commands down into the implementation
level of CommandInstanceManager. The point is, we know the standard
usage situation is to rely on the instance manager, and thus we want
to avoid redundant table lookups, only to support the rare case of
fallback to global commands. The latter is currently used only from
unit-tests, but might in future also be used by scripts.
Due to thread safety considerations, I have refrained from handing
out a direct reference to the command token sitting in the registry,
even while not doing so incurs a small runtime penalty (accessing
the shared ref-count for creating a copy of the smart-handle).
This is the typical situation where you'd be tempted to sacrifice
sanity for the sake of an imaginary performance benefit, which
in fact is dwarfed by all the machinery of UI-Bus and argument
passing via GenNode.
but I am not happy with the implementation yet: the maybeGet just
doesn't feel right. Likely it will be a better idea to push that
fallback mechanism generally down into the CommandInstanceManager?
just by reasoning from the concept, an instance should always correspond
to a single invocation trail. Having several sets of invocation state
compete with each other, means to keep them distinct, otherwise the
implicit state is going to be corrupted
This changeset fixes a huge pile of problems, as indicated in the
error log of the Doxygen run after merging all the recent Doxygen improvements
unfortunately, auto-linking does still not work at various places.
There is no clear indication what might be the problem.
Possibly the rather unstable Sqlite support in this Doxygen version
is the cause. Anyway, needs to be investigated further.
this is indeed a change of concept.
A 'command instance' can not be found through the official
Command front-end anymore, since we do not create a registration.
This allows us to avoid decorating command IDs with running counters
interesting new twist: we do not even need to decorate with a running number,
since we'll get away with an anonymous command instance, thanks to Command
being a smart-handle
this is a prerequisite for command instance management:
We have now an (almost) complete framework for writing actual
command definitions in practice, which will be registered automatically.
This could be complemented (future work) by a script in the build process
to regenerate proc/cmd.hpp based on the IDs of those automatic definitions.
The point in question is how to manage these definitions in practice,
since we're about to create a huge lot of them eventually. The solution
attempted here is heavily inspired by the boost-test framework
...because this topic serves as a vehicle to elaborate various core concepts
of the UI backbone, especially how to access, bind and invoke Proc-Layer commands
...turns out to be a nasty subject, now we're able to see
in more concrete detail how this interaction needs to be carried out.
Basically this is a blocker for the top-level, since it is obviously
some service in top-level, which ultimately becomes responsible for
orchestrating this activity
this pretty much resolves most of the uncertainities:
we now get a set of mutually dependent services, each of which
is aware of each other member's capabilities, but accesses those
only through this partner's API
After quite some pondering, it occured to me that we both
- need some top-level model::Tangible to correspond to the RootMO in the session
- need some Controller to handle globally relevant actions
- need a way to link action invocation to transient interaction state (like focus)
This leads to the introduction of a new top-level controller, which is better
suited to fill that role than the depreacted model-controller or the demoted window-manager
looks like we're in management business here ;-)
we chop off heads, slaughter the holy cows and then install -- a new manager
...allows us to get rid of a lot of sigc boilerplate syntax.
The downside is that the resulting functors are not sigc::trackable.
This seems adequate here, since the whole top-level UI backbone is
maintained by GtkLumiera, and thus ensured to exist as long as the
main GTK event loop is running.
WARNING: beware of creating "wild" background thrads in the UI, without
proper scheduling of any communication via the event loop!
This is a very pervasive change and basically turns the whole top-level
of the GTK-UI bottom-up. If this change turns out right, it would likely
solve #1048
WARNING: in parts not implemented, breaks UI
...which itself is obsolete and needs to be redesigned from scratch.
For now we create a local instance of this obsolete PlaybackController
in each viewer panel and we use a static accessor function to just some
instance. Which would break if we start playback with multiple viewer
panels. But we can't anyway, since the Player itself is also a broken
leftover from an obsoleted design study from the early days.
so why care...
- WindowList (ex WindowManager)
- Project & Controller
the latter ones are defunct and can be replicated down into each
of the old timeline pannel instances. They just serve the purpose
to keep this old code barely functional, so it can be used as reference
for building the new timeline
There seems to be a mismatch in the arrangement of the top-level entities
* we support multiple windows, yet from reading the code, you'd ge the impression we aren't really aware we have multiple top-level windows
* the `WindowManager` is the core UI manager, which feels like a mix-up in concerns
* the `WorkspaceWindow::createUI()` does the global UI initialisation. Again, we have multiple workspace windows.
* `GtkLumiera::main()` creates a `Model` and a `Controller` in local function scope, but stores the `WindowManager` in an object field.
* it seems, for that very reason, `GtlLumiera` needed to be a singleton, to allow by-name access to "the" `WindowManager`
* needless to say, this causes a host of problems when shutting down the UI.
The idea is to introduce a dedicated UiManager, to deal with the central
framework induced concerns solely, and to demote the WindowManager and the
WorkspaceWindows to care only for their local concerns
in fact it just does not fulfil any of the behavioural properties
of a full-fledged UI-Element. All it needs is an uplink bus connection,
so let's just keep it as that
Sidenote: I've realised today that such a "free standing" BusTerm
without registration in Nexus is a good idea and acceptable solution.
yes, it's a cycle and indeed quite tricky.
Just verified it (again) with the debugger and saw all
dtor calls happening in the expected order. Also the number
of Nexus registration is sane
Now I've realised that there are two degrees of connectedness.
It is very much possible to have a "free standing" BusTerm, which
only allows to send uplink messages. In fact, this is how CoreService
is implemented, and probably it should also the way how to connect
the GuiNotification service...
Reason was some insideous detail regarding Lambdas:
When a Lambda captures context, a *closure* is created.
And while the Lambda itself is generated code, pretty much
like an anonymous function, the closure depends on the context
that was captured. In our case here, the Lambda used to start
the thread was the problem: it captured the termCallback functor
from the argument of the enclosing function. In fact it did not
help or change anything if we successively package that lambda
into a function objet and store this by value, because the
lambda still refers to the transient function context present
on stack at the moment it was captured.
The solution is to revert back to a bind expression, since this
creates a dedicated storage for the bound function arguments
managed within the bind-functor. This makes us independent
from the call context
...because some Bus connections stem from elements which are
member of CoreService, thus the'll still be connected when the
sanity check in the dtor runs
But even with this fix, we still get a SEGFAULT
TODO
- is this actually a sensible idea, from a design viewpoint?
- in which way to bind GuiNotification for receiving diff messages?
- Problem with disconnnecting from Nexus on shutdown
Writing and debugging such tests is always an interesting challenge...
Fortunately this exercise didn't unveil any problem in the newly written
code, only some insidious problems in the test fixture itself. Which
again highlights the necessity, that each *command instance* needs
to be an independent clone from the original *command prototype*,
since argument binding messages and trigger messages can appear
in arbitrary order.
not quite sure how to get the design straight.
Also a bit concerned because we'll get this much indirections;
the approach to send invocations via the UI-Bus needs to prove its viability
Did a full review of state and locking logic, seems airtight now.
- command processing itself is unimplemented, we log a TODO message for now
- likewise, builder is not implemented
- need to add the deadlock safeguard #1054
We found out that it's best to run it single threaded
within the session loop thread. This does not mean the Builder
itself is necessarily single threaded, but the Builder's top level
will block any other session operation, and this is a good thing.
For this reason it makes more sense to have the Builder integrated
as a component into the session subsystem.
after reading some related code, I am leaning towards a design
to mirror the way command messages are sent over the UI-Bus.
Unfortunately this pretty much abandons the possibility to
invoke these operations from a client written in C or any
other hand made language binding. Which pretty much confirms
my initial reservation towards such an excessively open
and generic interface system.
...this means to turn Looper into a state machine.
Yet it seems more feasible, since the DispatcherLoop has a nice
checkpoint after each iteration through the while loop, and we'd
keep that whole builder-dirty business completely confined within
the Looper (with a little help of the DispatcherLoop)
Let's see if the state transition logic can actually be implemented
based just on such a checkpoint....?
....if by some weird coincidence, a command dispatched into the session
happens to trigger session shutdown or re-loading, this will cause a deadlock,
since decommissioning of session data structures must wait for the
ProcDispatcher to disable command processing -- and this will obviously
never happen when in a callstack below some command execution!
After some consideration, it became clear that this service implementation
is closely tied to the DispatcherLoop -- which will consequently be
responsible to run and expose this service implementation
need to keep state variables on both levels,
since the session manager (lifecycle) "opens" the session
for external access by starting the dispatcher; it may well happen
thus that the session starts up, while the *session subsystem*
is not(yet) started
mark TODOs in code to make that happen.
Actually, it is not hard to do so, it just requires to combine
all the existing building blocks. When this is done, we can define
the "Session" subsystem as prerequisite for "GUI" in main.cpp
Unless I've made some (copy-n-paste) mistake with defining the facades,
this should be sufficient to pull up "the Session" and automatically
let the Gui-Plugin connect against the SessionCommandService
up to now this happened from the GuiRunner, which was a rather bad idea
- it can throw and thus interfer with the startup process
- the GuiNotification can not sensibly be *implemented* just backed
by the GuiRunner. While CoreService offers access to the necessary
implementation facilities to do so
so the true reason is an inner contradiction in the design
- I want it to be completely self similar
- but the connection to CoreService does not conform
- and I do not want to hard code CoreService into the Nexus classdefinition
So we treat CoreService as uplink für Nexus and Nexus as uplink for CoreService,
with the obvious consequences that we're f**ed at init and shutdown.
And since I want to retain the overall design, I resort to implement
a short circuit detector, which suppresses circular deregistration calls
Decision was made to use the CoreService as PImpl to organise
all those technical aspects of running the backbone. Thus,
the Nexus (UI-Bus hub) becomes part of CoreService
...problem is, I actually don't know much about what kinds of markers
we'll get, and how we handle them. Thus introducing a marker kind
is just a wild guess, in order to get *any* tangible attribute
this is a tricky problem and a tough decision.
After quite some pondering I choose to enforce mandatory fields
through the ctor, and not to allow myself cheating my way around it
it occurred to me that effectively we abandoned the use of
a business facade and proxy model in the UI. The connection
becomes entirely message based now.
To put that into context, the originally intended architecture
never came to life. The UI development stalled before this could
happen; possibly it was also hampered by the "impedance mismatch"
between our intentions in the core and such a classical, model centric
architecture. Joel several times complained that he felt blocked; but
I did not really understand this issue. Only recently, when I came to
adapting the timeline display to GTK-3, I realised the model centric
approach can not possibly work with such an open model as intended
in our case. It would lead to endless cascades of introspection.
these are just empty class files, but writing a basic description
for each made me flesh out a lot of organisational aspects of what
I am about to build now
...it seemed first that we'd might run into a very fundamental problem;
but after some consideration it turns out the interspersed display manager
and the decoupling between model/presenter and widget happens to mitigate
this problem as well.
the content of the "GuiTimelineWidgetStructure" tiddler is
actually related to architecture questions related to custom widgets
in general, plus working notes regarding an investigation of the
Gtk::Layout widget.
bottom line
- seems we need to do that manually
- must wait until in the on_draw() callback
- use Container::foreach() to visit all child widgets
- Layout::set_size()
- define tasks to be addressed during investigation
- read documentation, identify problematic aspects
- prepare a child widget class to be placed on the canvas
actually this is a pragmatic extension for some special use cases,
and in general rather discurraged, since it contradicts the
established diff semantics. Yet with some precaution, it should
be possible to transport information via an intermediary ETD
Map -> ETD -> Map
...this is the first attempt to integrate the Diff-Framework into (mock) UI code.
Right now there is a conceptual problem with the representation of attributes;
I tend to reject idea of binding to an "attribute map"
the generic typing to DiffMutatble does not make much sense,
since the desired implementation within gui::ctrl::Nexus
is bound to work on Tangibles only, since that is what
the UI-Bus stores in the routing table
at first, this seemed like a good idea, but it caused already
numerous quirks and headache all over the place. And now, with
the intent to switch to the TreeMutator based implementation,
it would be damn hard to retain these features, if at all
possible.
Thus let's ditch those in time and forget about it!
this is a subtle change in the semantics of the diff language,
actually IMHO a change towards the better. It was prompted by the
desire to integrate diff application onto GenNode-trees into the
implementation framework based on TreeMutator, and do away with
the dedicated implementation.
Now it is a matter of the *selector* to decide if a given layer
is responsible for "attributes". If so, then *all* elements within
this layer count as "attribute" and an after(Ref::ATTRIBS) verb
will fast forward behind *the end of this layer*
Note that the meta token Ref::ATTRIBS is a named GenNode,
and thus trivially responds to isNamed() == true
...instead of using a hand written implementation,
the idea is to rely on the now implemented building blocks,
with just some custom closures to make it work.
- esp. verify the proper inclusion of the Selector closure in all Operations
- straighten the implementation of Attribute binding
- clean-up the error checking helpers
In Theory, acceptSrc and skipSrc are to operate symmetrically,
with the sole difference that skipSrc does not move anything
into the new content.
BUT, since skipSrc is also used to implement the `skip` verb,
which serves to discard garbage left back by a preceeding `find`,
we cannot touch the data found in the src position without risk
of SEGFAULT. For this reason, there is a dedicated matchSrc operation,
which shall be used to generate the verification step to properly
implement the `del` verb.
I've spent quite some time to verify the logic of predicate evaluation.
It seems to be OK: whenever the SELECTOR applies, then we'll perform
the local match, and then also we'll perform the skipSrc. Otherwise,
we'll delegate both operations likewise to the next lower layer,
without touching anything here.
This is the first skeleton to combine all the building blocks,
and it passes compilation, while of course most of the binding
implementation still needs to be filled in...
- default recommendation is to implement DiffMutable interface
- ability to pick up similar non-virtual method on target
- for anything else client shall provide free function mutatorBinding(subject)
PERSONAL NOTE: this is the first commit after an extended leave,
where I was in hospital to get an abdominal cancer removed.
Right now it looks like surgery was successful.
this is at the core of the integration problem: how do we expose
the ability of some opaque data structure to create a TreeMutator?
The idea is
- to use a marker/capability interface
- to use template specialisation to fabricate an instance of that interface
based on the given access point to the opaque data structure
but unfortunately this runs straight into a tough problem,
which I tried to avoid and circumvent all the time:
At some point, we're bound to reveal the concrete type
of the Mutator -- at least to such an extent that we're
able to determine the size of an allocator buffer.
Moreover, by the design chosen thus far, the active
TreeMutator instance (subclass) is assumed to live within
the top-level of a Stack, which means that we need to
place-construct it into that location. Thus, either
we know the type, or we need to move it into place.
the plan is to put together an integration test
of diff application to opaque data through the TreeMutator,
using the now roughly finished binding primitives.
moreover, the idea is to apply precisely the same diff sequence,
as was used in the detail test (TreeMutatorBinding_test).
NOTE: right now, the existing placehoder code applies this sequence
onto a Rec<GenNode>. This should work already -- and it does,
BUT the result of the third step is wrong. Really have to
investigate this accidental finding, because this highlights
a conceptual mismatch in the handling of mixed scopes.
...which mostly just is either ignoring the
operations or indicating failure on attempt to
'reorder' attributes (which don't have any notion of 'ordering')
overall, the structure of this implementation is still rather confusing,
yet any alternatives seem even less convincing
- if we want to avoid the delegation to base-class, we'd have
to duplicate several functions and the combined class would
handle two distinct concerns.
- any attempt to handle the IDs more "symmetrically" seems to
create additional problems on one side or the other
this also supersedes and removes the initial implementation
draft for attribute binding with the 'setAttribute' API
The elementary part of diff application incl. setting
new attribute values works by now.
The way we build this attribute binding, there is no single
entity to handle all attribute bindings. Thus the only way
to detect a missing binding is when none of the binding layers
was able to handle a given INS verb
the idea is again to perform the same sequence of primitives,
this time with a binding to some local variables within the test function
here to enact the role of "object fields"
together with drafting the first segment of the test code,
I've settled down onto an implementation approach
the plan is to use this specific diff sequence
both in the individual binding tests, and in a
more high level integration test. Hopefully this
helps to make these quite technical tests more readable
..as concluded from the preceding analysis.
NOTE this entails a semantical change, since this
predicate is now only meant to be indicative, not conclusive
remarks: the actual implementation of the diff application process
as bound via the TreeMutator remains yet to be written...
how can ordinary object fields be treated as "Attributes"
and thus tied into the Diff framework defined thus far.
This turns out to be really tricky, even questionable
while simple to add into the implementation, this whole feature
seems rather qestionable to me now, thus I've added a Ticket
to be revisited later.
In a nutshell, right here, when implementing the binding layer
for STL collections, it is easy to enable the framework to treat
Ref::THIS properly, but the *actual implementation* will necessarily
be offloaded onto each and every concrete binding implementation.
Thus client code would have to add support for an rather obscure
shortcut within the Diff language. The only way to avoid this
would be to change the semantics of the "match"-lambda: if this
binding would rather be a back-translation of implementation data
into GenNode::ID values, then we'd be able to implement Ref::THIS
natively. But such an approach looks like a way inferiour deisgn
to me; having delegated the meaning of a "match" to the client
seems like an asset, since it is both natural and opens a lot
of flexibility, without adding complexity.
For that reason I tend to avoid that shortcut now, in the hope
to be able to drop it entirely from the language
write down a first draft for a definiton section,
to describe the fundamental parts involved, when
applying a diff message onto implementation defined
data structures
After a break of tree weeks, I found it difficult to find may way
amidst all those various levels of abstraction. In addition to this
definition, we'll probably also need a high level overview of the
whole diff system operation.
...all of this implementation boils down to slightly adjusting
the code written for the test-mutation-target. Insofar it pays off now
having implemented this diagnostic and demonstration first.
Moreover I'm implementing this basic scheme of "diff application"
roughly the fourth time, thus things kindof fall into place now.
What's really hard is all those layers of abstraction in between.
Lesson learned (after being off for three weeks, due to LAC and
other obligations): I really need to document the meaning of the
closures, and I need to document the "abstract operational semantics"
of diff application, otherwise no one will be able to provide
the correct closures.
while I still keep my stance not to allow reflection and
switch-on-type, access to the internal / semantic type of
an embedded record seems a valid compromise to allow
to deal with collections of object-like children
of mixed kind.
Indirectly (and quite intentional) this also opens a loophole
to detect if a given GenNode might constitute a nested scope,
but with the for the actual nested element indeed to cary
a type symbol. Effectively this limits the use of this shortcut
to situations where the handling context does have some pre-established
knowledge about what types *might* be expected. This is precisely
the kind of constraint I intend to uphold: I do not want the
false notion of "total flexibility", as is conveyed by introspection.
the whole implementation will very much be based on
my experiences with the TestMutationTarget and TestWireTap.
Insofar it was a good idea to implement this test dummy first,
as a prototype. Basically what emerges here is a standard pattern
how to implement a tree mutator:
- the TreeMutator will be a one-way-off "throwaway" object.
- its lifecylce starts with sucking away the previous contents
- consuming the diff moves contents back in place
- thus the mutator always attaches onto a target by reference
and needs the ability to manipulate the target
the collection binding can be configured with various
lambdas to supply the basic building blocks of the generated binding.
Since we allow picking up basically anything (functors,
function pointers, function objects, lamdas), and since
we speculate on inlining optimisation of lambdas, we can not
enforce a specific signature in the builder functions.
But at least we can static_assert on the effective signature
at the point where we're generating the actual binding configuration
re-evaluated the decision to build on lambdas, not virtual functions:
- it leads for sure to clearer code at the usag site
- it /might/ offer better, but certainly not worse potential for compiler optimisation
...and write down some insights about the architecure
and design of tree binding and tree description related
to the TreeMutator.
When reading my notes from last year, it became clear
to me that the design of the TreeMutator has evolved
significantly, and became quite something different
than I'd imagined at start
now the full API for the "mutation primitives" is shaped.
Of course the actual implementation is missing, but that
should be low hanging fuit by now.
What still requires some thinking though is how to implement
the selector, so we'll actually get a onion shaped decorator
basically we'll establish a collaboration where both sides
know only the interface (contract) of the partner; a safe margin
for allocation size has to be established through metaprogramming (TODO)
...basically we've now the list mutation primitives working,
albeit in a test/dummy implementation only. Next steps will
be to integrate the assignment and sub scope primitives,
and then to re-do the same implementation respectively
for the case of mutating a standard collection of arbitrary type
now this feels like making progress again,
even when just writing stubs ;-)
Moreover, it became clear that the "typing" of typed child collections
will always be ad hoc, and thus needs to be ensured on a case by case
base. As a consequence, all mutation primitives must carry the
necessary information for the internal selector to decide if this
primitive is applicable to a given decorator layer. Because
otherwise it is not possible to uphold the concept of a single,
abstracted "source position", where in fact each typed sub-collection
of children (and thus each "onion layer" in the decorator chain)
maintains its own private position
after sleeping one night over the problem, this seems to be
the most natural solution, since the possibility of assignment
naturally arises from the fact that, for tree diff, we have
to distinguish between the *identity* of an element node and
its payload (which could be recursive). Thus, IFF the payoad
is an assignable value, why not allow to assign it. Doing so
elegnatly solves the problem with assignment of attributes
Signed-off-by: Ichthyostega <prg@ichthyostega.de>
This basically finishes definition of the fundamental
UI-Element and Bus protocol -- with one notable exception:
how to mutate elements by diff.
This will be the next topic to address
...and I made the decision *not* to consider any kind of
generic properties for now. YAGNI.
UI coding is notorious spaghetti code.
No point in fighting that, it is just the way it is,
because somewhere you're bound to get concrete, hands-on.
still TODO: the ability to use immutable types
within the command framework. In theory, this
shouldn't be had to implement, since we're creating
a new opaque value holder within the command registry
anyway, so it should be sufficient to refrain from
re-assigning a new value tuple. This is relevant,
since e.g. our time framework is built on immutable
value types.
as it turns out, this is a Bug in GCC 4.9 (resolved in 5.x)
See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63723
Problem is, GCC emits a warning on narrowing conversions,
while the standard actually disallows them when building
objects from brace-enclosed initialisers.
Unfortunately GCC also emits such a warning from within
a SFINAE context, instead of letting those spurious dangerous
cases fail. So we end up with additional visitor double dispatch
paths, and a lot of additional warnings.
Temporary solution is to hack a custom trait, which
explicitly declares some conversions paths as "narrowing".
Probably this can be implemented in a way more intelligent
way (using std::numeric_limits), but this doesn't seem
worth the effort, since the problem will go away through
compiler evolution eventually.
now we're able to construct suitable parameter values from the
arguments passed embedded in the GenNodes, as is demonstrated with the
EntryID<long> constructed from an ID-string. We really need a full-blown
double-dispatch, since the content type of the concrete GenNode is only
known at runtime (encoded in the RTTI)
There is still the problem with generating some spurions additional
conversion pathes, some of which are narrowing (and thus dangerous).
The copiler emits several warnings here, and all of them are justified.
E.g. it would be possible to pass an int64_t in the GenNode and initialise
a short from it. This might be convenient at times, but I tend rather to
be prohibitive here and thus consider to built in distinct limitations
on the allowed conversions.
not sure yet if any of this works, because the
technicalities of dealing with variadic types are
quite different to our LISP-style typelist processing.
The good news is that with variadic templates it is
indeed possible, to supply dynamically picked arguments
to another function taking arbitrary arguments.
This all relies on the feature to unpack argument packs,
and, more specifically, about the possiblity to "wrap"
this unpacking around interspersed function call syntax
template<size_t...i>
Xyz
do_something(MyTuple myTuple)
{
return Xyz (std::get<i> (myTuple) ... );
}
Here the '...' will be applied to the i... and then
the whole std::get-construct will be wrapped around
each element. Mind bogging, but very powerful
First part is to define the steps (the protocol) at the
model element level, which gets a command prepared and invoked.
Test fails still, because there is no actual argument binding
invoked in the TestNexus
the initial draft of this concept is in place now, and
the first round of unit tests pass. I've got some understanding
of the purpose of the interactions and involved elements
and I'm confident this design is evolving in a sane way.
Note: extensive documentation is in the TiddlyWiki,
here I've just pasted and reworded some paragraphs from there
and integrated them into the Doxygen docs
next step will be to rig the mock element and set up
and cover the basic / generic element behaviour
This changeset
- adapts the (planned) unit test to the semantic of
the EventLog, which is now fully implemented
- adjusts the function names on the public Tangible interface,
to be better in line with the naming convention of the
corrsponding operations on the UI-Bus:
* "mark" operations are towards the UI element
* "note" messages are from the UI element towards some
state manager, which can be reached via the bus
what you see here now is just the tip of the icebearg...
If we follow this route, the Lumiera UI will become way more
elaborate and responsive than average desktop applications
looked at
- boost property_tree
- boost spirit
- JSON-C
- JsonCpp
- rapidjson
- vjson / gason
the results where quite obvious: --> rapidjson
Rationale: we do not want yet another object system;
rapidjson has an SAX-style API (and additionally an DOM API
if needed). And it is fast, supports in-situ parsing,
extended Unicode support, full JSON support and
an exchangable allocator.
License is MIT
NOTE: we have the policy to always support current Debian/stable
amd at least one Ubuntu LTS release, unless hard dependency problems prevent that.
Currently, Ubuntu/Trusty is already a bit dated, but the only problematic dependency
could be libboost (1.54 in Trusty, 1.55 in Jessie).
GCC-4.8 can be replaced by GCC-4.9 in Trusty without problems
It is always a bit tricky to find out the precise lower boundary,
so we try to upgrade these requirements as our platform progresses.
For now we have used the level available on Ubuntu/Trusty to set
the lower constraints for most libraries
This is a development snaphot pre release of Lumiera.
It features codebase maintenance, upgrade to C++14 and GTK-3
and some work towards a Proc-GUI connection (unfinished)
Update README, AUTHORS, LICENSE and similar release docs.
because otherwise we'd need to send a whole subtree
over the wire and then descend into it just to find an element.
This too is a ripple effect of making '==' deep
well... this was quite a piece of work
Added some documentation, but a complete documentation,
preferably to the website, would be desirable, as would
be a more complete test covering the negative corner cases
It is difficult to reconcile our general architecture for the
linearised diff representation with the processing of recursive,
tree-like data structures. The natural and most clean way to
deal with trees is to use recursion, i.e. the processor stack.
But in our case, this means we'd have to peek into the next
token of the language and then forward the diff iterator
into a recursive call on the nested scope. Essentially, this
breaks the separation between receiving a token sequence and
interpretation for a concrete target data structure.
For this reason, it is preferrable to make the stack an
internal state of the concrete interpreter. The downside of
this approach is the quite confusing data storage management;
we try to make the role of the storage elements a bit more
clear through descriptive accessor functions.
each language token of our "linearised diff representation"
carries a payload data element, which typically is the piece
of data to be altered (added, mutated, etc).
Basically, these elements have value semantics and are
"sent over wire", and thus it seems natural when the
language interpreter functions accept that piece of payload
by-value. But since we're now sending GenNode elements as
parameter data in our diff, which typically are of the
size of 10 data elements (640 bit on a 64bit machine),
it seems more resonable to pass these argument elements
by const& through the interpreter function. This still
means we can (and will indeed) copy the mutated data
values when applying the diff, but we're able to
relay the data more efficiently to the point where
it's consumed.
this boils down to the two alternatives
- manipulate the target data structure
- build an altered copy
since our goal is to handle large tree structures efficiently,
the decision was cast in favour of data manipulation
so basically it's time to explicate the way
our diff language will actually be written.
Similar to the list diff case, it's a linear sequence
of verb tokens, but in this case, the payload value
in each token is a GenNode. This is the very reason
why GenNode was conceived as value object with an
opaque DataCap payload
Initially I intended just to supply an addapter to use
the monadic IterExplorer for this recursive expansion
of GenNode contents. Investigating this approach was
relevant to highlight the minimum requirements for
such an evaluation mechanics: since our GenNode
is an hierarchical structure without back-links,
we are bound to use a stack at some point. And
since an Iterator is a materialised continuation,
we can not use the processor stack and are forced
to represent this stack in memory.
Yet, on second thought, we do not need the full power
of the IterExplorer monad; especially we do not need
to bind arbitrary functions into the monad, just one
single scope exploring function, implemented as
Variant visitor. Based on these observations, we can
"inline" the monad structure into a double nested
iterator, where the outer capsule carries a stack
of scopes to be explored.
remembered that some years ago I had to deal with a very similar problem
for planning the frame rendering jobs. It turned out, that the
iterator monad developed for this looks promising for our task at hand
initially my intention was to use the ID for equality test.
But on a second thought, this seemed like a bad idea, since
it confuses the concepts of equality and identity.
Note: at the moment, I do not know if we even need an equality test,
so it is provided here rather for sake of completeness. And this
means even more that we want an 'equality' implementation that
does what one would naively expect: compare the object identity
*and* compare the contents.
especially setting (changing) attributes turned out to be tricky,
since in case of a GenNode this would mean to re-bind the hash ID;
we can not possibly do that properly without knowing the type of the payload,
and by design this payload type is opaque (erased).
As resort, I changed the semantics of the assign operation:
now it rather builds a new payload element, with a given initialiser.
In case of the strings, this ends up being the same operation,
while in case of GenNode, this is now something entirely different:
we can now build a new GenNode "in place" of the old one, and both
will have the same symbolic ID (attribute key). Incidentally,
our Variant implementation will reject such a re-building operatinon
when this means to change the (opaque) payload type.
in addition, I created a new API function on the Mutator,
allowing to move-in a complete attribute object. Actually this
new function became the working implementation. This way, it is
still possible to emplace a new attribute efficiently (consider
this to be a whole object graph!). But only, if the key (ID)
embedded in the attribute object is already what is the intended
key for this attribute. This way, we elegantly circumvent the
problem of having to re-bind a hash ID without knowing the type seed
OMG, what was all this about?
OK... this cant possibly work this way.
At least we need to trim after splitting the attributes.
But this is not enough, we want the value, which implies
to make the type flexible (since we cant return a const& to
a substring extracted on-the-fly)
especially I've now decided how to handle const-ness:
We're open to all forms of const-ness, the actual usage decides.
const GenNode will only expose a const& to the data values
still TODO is the object builder notation for diff::Record
on a second thought, this "workaround" does not look so bad,
due to the C++11 feature to request the default implementation explicitly.
Maybe we'll never need a generic solution for these cases
This is just a draft for now -- kindof a by-catch, since it is
chep to build that DSL on top of the Rec::Mutator.
This DSL could be of value later, when it comes to define
some configuration data inline, in a copact and clear fashion,
without the need to use a bridge to/from JSON
for the purpose of working out the inner logic, I frequently use the
help of a mindmap -- so why not commiting this alongside? For sure,
it is preliminary and the worked out concepts will be cast in code
and documented on the website. Yet the thought-process leading to
these decisions might be of some interest, at least for myself.
before engaging into the implementation of lib::Record,
I prefer to conduct a round of planning, to get a clearer
view about the requirements we'll meet when extending
our existing list diff to tree structures
Introduce the new term "Fork" at various relevant places
within the documentation. We do not entirely purge the
term "track" though; rather we
- make clear that "Fork" is the entity to build tracks
- use "fork" also synonymous to the "tree of tracks"
This switches the Lumiera UI from GTK-2 to GTK-3
Unfortunately, this move breaks two crucial features, which have been
disabled for now: the display of video and our custom timeline widget.
Since both of these require some reworking, which in fact has already
started, we prefer to do the library and framework switch right away.
to carry out that rather obvious step, I was bound to consider
all the implications of choosing a given layout and handling pattern
for our external structure representation.
Finally, I settled upon the following decisions
- the value space represented within the DataCap is flat, not further structured
- the distinction between "attribute" and "nested object" is merely conceptual
and will be enforced solely by the diff detection / representation protocol
- basically, a nested subtree may appear as an attribute; the difference
between attributes and children lies solely in the way of access and referral:
by-name vs. positional
- it is pointless to save space for the representation of the discriminator ID
- but we can omit any further explicit type tag, because
- we do *not* support programming by switch-on-type, and thus
- we do *not* support full introspection, only a passive type-safety check
- this is *not* a limitation, since we acknowledge that GenNode is a *Monad*
- and the partial function needed within any flatMap implementation
maps naturally onto our Variant-Visitor; thus
- the DataCap can basically just *be* a Variant
- and GenNode has just to supply the neccessary shaffolding
to turn that into a full fledged Monad implementation, including
direct construction by wrapping a value and flatMap with tree walk
This mindmap explains some of the ideas and concepts in more detail.
It can be viewed with the "freeplane" mind mapping software.
("freemind" works as well, but will mess up some of the formatting)
note by the committer: this mindmap was really work in progress.
Christoph shared it with me while in discussion. I'll place it
here in the git history, since it might be interesting to se
how the thoughts evolved. Isn't that what mindmaps are all about?
so yes, it is complicated, and inevitably involves three layers
of indirection. The alternative seems to bind the GUI direcly to
the Session interface -- is there a middle gound?
For the messages from GUI to Proc, we have our commands, based
on PlacementRef entities. But for feeding model updates to the
GUI, whatever I consider, I end up either with diff messages or
an synchronised access to Session attributes, which ties the
responsiveness of the GUI to the Builder operation.
- we use a GenNode element
- this holds a polymorphic value known as DataCap
- besides simple attribute values, this may hold collections of GenNode sub elements
- a special kind of GenNode collection, the Record, is used to represent objects
The purpose of this setup is to enable an external model representation
which is only loosely coupled to the interndal data representation
through the exchange of (tree)diff messages
previously this operation was named 'attach', which an be confused
with attching an object to this location. Indeed, the session interface
even offers such an attach function. By renaming the focus moving
operation into QueryFocus::shift(Scope), this ambiguity is resolved
This is the first step towards a generic backbone to connect
any GUI elements to the session within Proc-Layer.
It is based on a spefic understanding of Model-View-Controller,
which turns the Model-Controller interactions into messages.
Remove some orphaned diagrams and PNG images not actually used
in the TiddlyWiki. Add a page with some hints regarding Bouml
See also #960 -- Bouml has been discontinued and is closed source now
not sure how to proceed with this
This means to discontinue any research into emitting an optimal
diff verb sequence for now and just to document the possible path
I see to reach at such a optimal solution later, when it turns out
to be really necessary performance wise.
Personal note: I arrived at this conclusion while whatching the
New Year fireworks 2014/2015 at the banks of the Isar river in
the centre of the city.
Too sad that 2014 didn't bring us World War III
we want a simple and straight forward way of defining tokens
of the "diff language". Each token is bound to a specific
handler function in the language interpreter interface.
Problem is that likely we'll get a ListDiffLanguage and a TreeDiffLanguage;
after all, I really don't know yet how far to take this whole
diff representation endeavour...
...first step is to design a generic linearised list diff representation.
Basically just need to pull together the theoretical work of the last weeks.
Next steps will be to extend to typed ordered trees.
Actually I arried at the conclusion, that the *receiving* of
a diff representation is actually a typical double-dispatch situation.
This leads to the attempt to come up with a specialised visitor
as standard pattern to handle and apply a diff. Obviously,
we do not want the classical GoF-Visitor, but (yes, we had
that discussion allready) -- well in terms of runtime cost,
we have to deal with at least two indirections anyway;
so now I'm exploring the idea to implement one of these
indirections through a functor object, which at the same time
acts as "Tag" in the diff representation language (instead
of using an enum as tag)
This DSL is usable as wire format for sending
structural change data to another, loosely coupled entity.
A similar format could be used for model serialisation later on.
I know that "reactive" is some kind of a fad currently.
But the term captures the intent very well, and I for sure
wanted such a GUI 5 years ago. Waiting after each stroke or
trim for 5 seconds or even 30 seconds for the UI to update
just totally sucks and kills any creative flow.
We all know that an application needs to be built for reactivity
and exactly that is what we do.
initial considerations; there is a concurrency problem, since
all of session handling within Proc is deliberately not threadsafe.
Thus the decision is to make this the gui::model::SessionFacade's responsibility
Actually this is nothing new, just making explicit what is evident
from the definition of the Proc-Layer model entities. By following
these conventions, it should be possible to come up with a
clearer structure for the custom timeline widget(s).
- upgrade the configuration to a current version
- provide a frontpage with cross-links to other documentation
- define a set of modules; relevant classes and files can be
added to these, to create a exploration path for new readers
- fix a lot of errors in documentation comments
- use a custom configuration for the documentation pages
- tweak the navigation, the sections and further arrangements
Notably TiddlyWiki provides now a fallback mechanism
in case the saving to a local file fails due to security
restrictions. When this happens, TiddlyWiki generates a
download link pointing to the current content; this way
one is at least able to "save as" through the browser
context menu.
Due to some controversial policy changes in recent Firefox versions
the support for saving to local files was removed. The rationale
given by the Firefox developers was that this is a rarely used
and generally outdated concept; preferrably people shall use
extensions and save to cloud services (!)
Anyway, Jeremy Ruston, the original author of TiddlyWiki, wrote
a Firefox plugin called "TiddlyFox" to work around these
arcane limitations.
this draft fills in the structure how to get from an invocation
of the engine service to the starting of actual CalcStream instances.
Basically the EngineService implementation is repsonsile to
instruct the Segmentation to provide a suitable Dispatcher.
we were using an very ancient TiddlyWiki version, basically
unaltered since the start of the Lumiera project.
Recent Firefox versions (starting with FF-17) effectively
removed the ability for JavaScript code to get additional
privileges for writing local files; this change seriuosly
undermines the usability of TiddlyWiki and a lot of similar
portable local JavaScript applications.
The original author of TiddlyWiki, Jeremy Ruston, has written
a Firefox plug-in to work around this restriction; this prompts
us to upgrade to a recent TiddlyWiki version as well.
The good news is: recent TiddlyWiki versions are able to import
Tiddlers from older Wikis without much hassle.
brainstorming how to implement the job planning stage
the idea is to built on top of the IterExplorer,
but have the "stack" of re-evaluation integrated
into a custom type, which exploits the static
node network structure to avoid heap allocations
solution idea: again use a builder function?
this draft is based on
- Cehteh's draft for the scheduler
- my plannings about segmentation and JobTicket
it defines "Job" as a closure which can be invoked
from plain-C, using the information in the
job descriptor datastructure
also touches the question how to represent the job
descriptor datastructure. @Cehteh: I've just pasted
in your preliminary data struct definitinons
from the relevant mailing list discussions.
These pages from the TiddlyWiky feature a complete glossary
of terms relevant for time and timecode handling, plus the
architecural decisions related to this topic
