Damn sideeffect of the suppport for move-only types: since we're
moving our binding now into place /after/ construction, in some cases
the end() iterator (embedded in RangeIter) becomes invalid. Indeed this
was always broken, but didn't hurt, as long as we only used vectors.
Solution: use a dedicated init() hook, which needs to be invoked
*after* the TreeMutator has been constructed and moved into the final
location in the stack buffer.
unintentionally we used copy construction in the builder expression,
wenn passing in the CollectionBinding to the ChildCollectionMutator.
The problem is that CollectionBinding owns a shaddow buffer, where
the contents of the target collection are moved temporarily while
applying the diff. The standard implementation of copy construction
would cause a copy of that shaddow buffer, which boils down to
a copy of the storage of the target collection.
If we want to support move-only types in the collection, most notably
std::unique_ptr, we can thus only use the move constructor. Beyond that
there is no problem, since we're only ever moving elements, and new
elements will be move constructed via emplace() or emplace_back()
actually this is a pragmatic extension for some special use cases,
and in general rather discurraged, since it contradicts the
established diff semantics. Yet with some precaution, it should
be possible to transport information via an intermediary ETD
Map -> ETD -> Map
for the record: while it is indeed sweet-and-simple to support Ref::THIS
here, it is near impossible to represent it in general, in a setup with
multiple "onion-layers". The reason is, we'd have to incorporate such
special treatment into the /selector predicate/, which in turn undermines
the ability to pick the right onion layer to handle a given diff verb,
since "Ref::THIS" is a generic marker and we have no other data to base
the decision in the selector on.
...this is the first attempt to integrate the Diff-Framework into (mock) UI code.
Right now there is a conceptual problem with the representation of attributes;
I tend to reject idea of binding to an "attribute map"
the generic typing to DiffMutatble does not make much sense,
since the desired implementation within gui::ctrl::Nexus
is bound to work on Tangibles only, since that is what
the UI-Bus stores in the routing table
Up to now, InPlaceBuffer used to default construct an instance of the
Interface class, and then you'd need to invoke the `create()` function
to actually create the desired subclass. This is not only inefficient,
but rules out the use of abstract interfaces / base classes.
Unfortunately, there is no way in C++ to specify an explicit template argument list
on ctor calls, so we resort to the trick of passing an additional dummy marker argument
yay! this piece of code has served its purpose:
it was the blueprint to build a way better design and implementation,
which can now cover this "generic tree" case as a special case as well
this adds kind of an extension point to diff::Record<GenNode>::Mutator,
which is then actually defined (implemented) within the diff framework.
This allows the TreeDiffTraits automatically to use this function
to get a TreeMutator for a given Rec::Mutator. Which in turn allows
the generic version of DiffApplicator automatically to attach and
bind to a Record<GenNode>
together this allows us to ditch the explicit specialisation
and dedicated, hand-written implementation of DiffApplication
to GenNode in favour of using the TreeMutator and friends.
this is a subtle change in the semantics of the diff language,
actually IMHO a change towards the better. It was prompted by the
desire to integrate diff application onto GenNode-trees into the
implementation framework based on TreeMutator, and do away with
the dedicated implementation.
Now it is a matter of the *selector* to decide if a given layer
is responsible for "attributes". If so, then *all* elements within
this layer count as "attribute" and an after(Ref::ATTRIBS) verb
will fast forward behind *the end of this layer*
Note that the meta token Ref::ATTRIBS is a named GenNode,
and thus trivially responds to isNamed() == true
needed to use a forward function declaration within the
lambda for recursive scope mutator building, since otherwise
everything is inline and thus the compilation fails when it
comes to deducing the auto return type of the builder.
Other than that, the whole mechanics seem to work out of the box!
previously they where included in the middle of tree-mutator.hpp
This was straight forward, since the builder relies on the classes
defined in the detail headers.
However, the GenNode-binding needs to use a specifically configured
collection binding, and this in turn requires writing a recursive
lambda to deal with nested scopes. This gets us into trouble with
circular definition dependencies.
As a workaround we now only *declare* the DSL builder functions
in the tree-mutator-builder object, and additionally use auto on
all return types. This allows us to spell out the complete builder
definition, without mentioning any of the implementation classes.
Obviously, the detail headers have then to be included *after*
the builder definition, at bottom of tree-mutator.hpp
This also allows us to turn these implementation headers into
completely normal headers, with namespaces and transitive #includes
In the end, the whole setup looks much more "innocent" now.
But beware: the #include of the implementation headers at bottom
of tree-mutator.hpp needs to be given in reverse dependency order,
due to the circular inclusion (back to tree-mutator.hpp) in
conjunction with the inclusion guards!
...instead of using a hand written implementation,
the idea is to rely on the now implemented building blocks,
with just some custom closures to make it work.
- esp. verify the proper inclusion of the Selector closure in all Operations
- straighten the implementation of Attribute binding
- clean-up the error checking helpers
similar reordering for the third part.
This time most operations are either passed down anyway,
or are NOP, since attribute binding has no notion of 'order'
yay! unit testing rocks.
Actually I changed the test definition for another reason, just to discover
that I've missed to implement that operation in this onion layer
now failing due to a contradiction in test fixture:
it is nonsensical to re-order attributes; rather, we should
cover re-ordering of children, to verify that the mutator binding
properly surpasses the attribute layers and forwards operations
to the lower layers responsible for handling child scopes...
In Theory, acceptSrc and skipSrc are to operate symmetrically,
with the sole difference that skipSrc does not move anything
into the new content.
BUT, since skipSrc is also used to implement the `skip` verb,
which serves to discard garbage left back by a preceeding `find`,
we cannot touch the data found in the src position without risk
of SEGFAULT. For this reason, there is a dedicated matchSrc operation,
which shall be used to generate the verification step to properly
implement the `del` verb.
I've spent quite some time to verify the logic of predicate evaluation.
It seems to be OK: whenever the SELECTOR applies, then we'll perform
the local match, and then also we'll perform the skipSrc. Otherwise,
we'll delegate both operations likewise to the next lower layer,
without touching anything here.
--> now it becomes obvious that we've mostly
missed to integrate the Selector predicate properly
in most bindings defined thus far. Which now causes
the sub-object binding to kick in, while actually
the sub-value collection should have handled
the nested values CHILD_B and CHILD_T
OMG, this is intricate stuff....
Questionable if anyone (other than myself) will be able
to get those bindings right???
Probably we'll need yet another abstraction layer to handle
the most common binding situations automatically, so that people
can use the diff framework without intricate knowledge of
TreeMutator construction.
- an extension to our custom toString and typeString helpers.
- currently just for shared_ptr and unique_ptr
- might add further overloads for other smart-ptr types
integrated into the generic DiffApplicationStrategy.
The dedicated, explicit specialisation for DiffMutable is
no longer needed, since the generic template will degrade or
fall back to precisely this functionality, when the target
implements the DiffMutable interface
This is the first skeleton to combine all the building blocks,
and it passes compilation, while of course most of the binding
implementation still needs to be filled in...
It occurred to me, that 90% of this template specialisation
are entirely generic and not dependant on the actual target type.
While the compiler/linker is able to sort such a situation out,
this might lead to template bloat and possibly subtle errors.
So it seems more adequate to emit the generic part of the code
right away from within a dedicated translation unit within the
library module; so the vtable is already in place and only
the flexible part of the code needs to be re-emitted on
each usage site.
- default recommendation is to implement DiffMutable interface
- ability to pick up similar non-virtual method on target
- for anything else client shall provide free function mutatorBinding(subject)
PERSONAL NOTE: this is the first commit after an extended leave,
where I was in hospital to get an abdominal cancer removed.
Right now it looks like surgery was successful.
this is at the core of the integration problem: how do we expose
the ability of some opaque data structure to create a TreeMutator?
The idea is
- to use a marker/capability interface
- to use template specialisation to fabricate an instance of that interface
based on the given access point to the opaque data structure
but unfortunately this runs straight into a tough problem,
which I tried to avoid and circumvent all the time:
At some point, we're bound to reveal the concrete type
of the Mutator -- at least to such an extent that we're
able to determine the size of an allocator buffer.
Moreover, by the design chosen thus far, the active
TreeMutator instance (subclass) is assumed to live within
the top-level of a Stack, which means that we need to
place-construct it into that location. Thus, either
we know the type, or we need to move it into place.
initially, even the diff applicator was meant to be a
"throwaway" object. But then, on writing some tests,
it seemed natural to allow re-using a single applicator,
after having attached it to some target.
With that change, I failed to care for the garbage
left back in the "old" sequence after applying one diff;
since in the typical usage sequence, the first use builds
content from scratch, this problem starts to show up only
with the third usage, where the garbage left from the input
of the second usage appears at the begin of the "new sequence"
Solution is to throw away that garbage explicitly on re-entrance
..because actually we don't know if the intention is
to drop those waste elements -- and for sure this
discarding of waste does not happen through the
invocation logged here; rather it happens by
abandoning the scope
...which mostly just is either ignoring the
operations or indicating failure on attempt to
'reorder' attributes (which don't have any notion of 'ordering')
overall, the structure of this implementation is still rather confusing,
yet any alternatives seem even less convincing
- if we want to avoid the delegation to base-class, we'd have
to duplicate several functions and the combined class would
handle two distinct concerns.
- any attempt to handle the IDs more "symmetrically" seems to
create additional problems on one side or the other
this also supersedes and removes the initial implementation
draft for attribute binding with the 'setAttribute' API
The elementary part of diff application incl. setting
new attribute values works by now.
While in general it is fine to clean-up any entity IDs
to be US-ASCII alphanumerics (plus some allowed interpunction),
the GenNodes and also keys in object-bindings for diff are
considerd internal interfaces, assuming that any passed
ID symbol is already sanitised and checked. So the
sanitise operation can be skipped. This changeset
adds the same option directly to lib::EntryID,
allowing to create an EntryID that matches
a similar GenNode's (hash) ID.
The way we build this attribute binding, there is no single
entity to handle all attribute bindings. Thus the only way
to detect a missing binding is when none of the binding layers
was able to handle a given INS verb
to summarise, it turned out that it is impossible to
provide an airtight 'emptySrc' implementation when binding
to object fields -- so we distinguish into positive and
negative tests, allowing to loosen the sanity check
only for the latter ones when binding to object fields.
..as concluded from the preceding analysis.
NOTE this entails a semantical change, since this
predicate is now only meant to be indicative, not conclusive
remarks: the actual implementation of the diff application process
as bound via the TreeMutator remains yet to be written...
how can ordinary object fields be treated as "Attributes"
and thus tied into the Diff framework defined thus far.
This turns out to be really tricky, even questionable
while simple to add into the implementation, this whole feature
seems rather qestionable to me now, thus I've added a Ticket
to be revisited later.
In a nutshell, right here, when implementing the binding layer
for STL collections, it is easy to enable the framework to treat
Ref::THIS properly, but the *actual implementation* will necessarily
be offloaded onto each and every concrete binding implementation.
Thus client code would have to add support for an rather obscure
shortcut within the Diff language. The only way to avoid this
would be to change the semantics of the "match"-lambda: if this
binding would rather be a back-translation of implementation data
into GenNode::ID values, then we'd be able to implement Ref::THIS
natively. But such an approach looks like a way inferiour deisgn
to me; having delegated the meaning of a "match" to the client
seems like an asset, since it is both natural and opens a lot
of flexibility, without adding complexity.
For that reason I tend to avoid that shortcut now, in the hope
to be able to drop it entirely from the language
...basically this worked right away and was easy to put together.
However, when considering how many components, indirections and
nested lambdas are working together here, I feel a bit dizzy...
:-/
write down a first draft for a definiton section,
to describe the fundamental parts involved, when
applying a diff message onto implementation defined
data structures
After a break of tree weeks, I found it difficult to find may way
amidst all those various levels of abstraction. In addition to this
definition, we'll probably also need a high level overview of the
whole diff system operation.
...all of this implementation boils down to slightly adjusting
the code written for the test-mutation-target. Insofar it pays off now
having implemented this diagnostic and demonstration first.
Moreover I'm implementing this basic scheme of "diff application"
roughly the fourth time, thus things kindof fall into place now.
What's really hard is all those layers of abstraction in between.
Lesson learned (after being off for three weeks, due to LAC and
other obligations): I really need to document the meaning of the
closures, and I need to document the "abstract operational semantics"
of diff application, otherwise no one will be able to provide
the correct closures.
while I still keep my stance not to allow reflection and
switch-on-type, access to the internal / semantic type of
an embedded record seems a valid compromise to allow
to deal with collections of object-like children
of mixed kind.
Indirectly (and quite intentional) this also opens a loophole
to detect if a given GenNode might constitute a nested scope,
but with the for the actual nested element indeed to cary
a type symbol. Effectively this limits the use of this shortcut
to situations where the handling context does have some pre-established
knowledge about what types *might* be expected. This is precisely
the kind of constraint I intend to uphold: I do not want the
false notion of "total flexibility", as is conveyed by introspection.
since we're moving elements around to apply the diff,
dangerous situation might arise in case anyone takes a copy
of the mutator. Thus we effectively limit the possible
usage pattern and only allow to build an anonymous
TreeMutator subclass through the Builder-DSL.
The concrete "onion layers" of the TreeMutator are now limited
- to be created by the chaining operations of the Builder DSl
- to be moved into target location, retaining ownership.
I still feel somewhat queasy with this whole situation!
We need to return the product of the DSL/Builder by value,
but we also want to swap away the current contents before
starting the mutation, and we do not want a stateful lifecycle
for the mutator implementation. Which means, we need to swap
right at construction, and then we copy -- TADAAA!
Thus I'm going for the solution to disallow copying of the
mutator, yet to allow moving, and to change the builder
to move its product into place. Probably should even push
this policy up into the base class (TreeMutator) to set
everyone straight.
Looks like this didn't show up with the test dummy implementation
just because in this case the src buffer also lived within th
TestMutationTarget, which is assumed to sit where it is, so
effectively we moved around only pointers.
the whole implementation will very much be based on
my experiences with the TestMutationTarget and TestWireTap.
Insofar it was a good idea to implement this test dummy first,
as a prototype. Basically what emerges here is a standard pattern
how to implement a tree mutator:
- the TreeMutator will be a one-way-off "throwaway" object.
- its lifecylce starts with sucking away the previous contents
- consuming the diff moves contents back in place
- thus the mutator always attaches onto a target by reference
and needs the ability to manipulate the target
the collection binding can be configured with various
lambdas to supply the basic building blocks of the generated binding.
Since we allow picking up basically anything (functors,
function pointers, function objects, lamdas), and since
we speculate on inlining optimisation of lambdas, we can not
enforce a specific signature in the builder functions.
But at least we can static_assert on the effective signature
at the point where we're generating the actual binding configuration
we can't generate a static assertion so easily here.
Problem is, when forming this type, we don't know if
the user will override and provide a custom binding
in some chained call within the nested DSL.
Might still be able to come up with some clever trick,
like e.g. returing an unsuitable marker type from these
dummy default implementations and then, later on, when
actually building the collection binding, to detect
those marker types and rise a static assert at that point.
This would at least give us a better error message,
and in theory, it should always be possible to
detect this kind of misuse at compile time
...through the use of partial specialisation and SFINAE.
There are some rather specific (yet expectedly not uncommon) cases,
where we'd be able to provide a sensible default for the
- match predicate
- new element constructor
of the binding. While in all other cases, the user
has to provide an explicit implementation for these
crucial building blocks anyway.
the reason is also to enable usage as metafunction,
to disable specialisations for some type which could
never live within a variant record in question
...but does not compile, since all of the fallback functions
will be instantiated, even while in fact we're overriding them
right away with something that *can* be compiled.
this prompts me to reconsider and question the basic approach
with closures for binding, while in fact what I am doing here
is to implement an ABC.
- the test will use some really private data types,
valid only within the scope of the test function.
- invoking the builder for real got me into problems
with the aggregate initialisation I'd used.
Maybe it's the function pointers? Anyway, working
around that by definint a telescope ctor
when setting up a binding to child elements within a STL collection,
all the variable elements are preconfigured to a more or less
disabled and inactive state.
the concern is for the structure of the builder to be
incomprehensible and completely buried within the
implementation details of the various binding layers
most of the mutation primitives return bool(true)
when /any/ layer or part of the TreeMuator was able
to cope with the diff verb.
This is based on the assumption to configure the
TreeMutator in such a way that at most one facility
will actually handle and apply a given verb. That is,
we'll assume that the TreeMutator acutally wraps and
adapts *one* custom data structure, to which the
diff has to be applied.
The TestWireTap is special, insofar it indeed targets
a *second* data structure, albeit not a "real" one,
just a dest and diagnostics dummy.
the first part of the unit test (now passing)
is able to demonstrate the full set of diff operations
just by binding to a TestMutationTarget.
Now, after verifying the design of those primmitive operations,
we can now proceed with bindings to "real" data structures
when implementing the assignment and mutation primitives
it became clear that the original approach of just storing
a log or string rendered elements does not work: for
assignment, we need to locate an element by ID
this one went through unnoticed, because the situation
is not covered in unit-test. The tests written thus fare
are more like a proof-of-concept. I didn't want to spend
weeks on writing extensive coverage of all corner cases,
at least not before all aspects of the tree diff protocol
are settled. Seemingly this backfires already
now the full API for the "mutation primitives" is shaped.
Of course the actual implementation is missing, but that
should be low hanging fuit by now.
What still requires some thinking though is how to implement
the selector, so we'll actually get a onion shaped decorator
basically we'll establish a collaboration where both sides
know only the interface (contract) of the partner; a safe margin
for allocation size has to be established through metaprogramming (TODO)
what's problematic is that we leave back waste in the
internal buffer holding the source. Thus it doesn't make
sense to check if this buffer is empty. Rather the
Mutator must offer an predicate emptySrc().
This will be relevant for other implementations as well
while the original name, 'replace', conveys the intention,
this more standard name 'swap' reveals what is done
and thus opens a wider array of possible usage
now this feels like making progress again,
even when just writing stubs ;-)
Moreover, it became clear that the "typing" of typed child collections
will always be ad hoc, and thus needs to be ensured on a case by case
base. As a consequence, all mutation primitives must carry the
necessary information for the internal selector to decide if this
primitive is applicable to a given decorator layer. Because
otherwise it is not possible to uphold the concept of a single,
abstracted "source position", where in fact each typed sub-collection
of children (and thus each "onion layer" in the decorator chain)
maintains its own private position
after sleeping one night over the problem, this seems to be
the most natural solution, since the possibility of assignment
naturally arises from the fact that, for tree diff, we have
to distinguish between the *identity* of an element node and
its payload (which could be recursive). Thus, IFF the payoad
is an assignable value, why not allow to assign it. Doing so
elegnatly solves the problem with assignment of attributes
Signed-off-by: Ichthyostega <prg@ichthyostega.de>
This basically finishes definition of the fundamental
UI-Element and Bus protocol -- with one notable exception:
how to mutate elements by diff.
This will be the next topic to address
- suppres sending redundant stat mark messages from MockElm
- emit a "reset" state mark when an actual reset happens
- let the PresentationStateManager discard recorded special state
when receiving a "reset" mark for a given element
I assumed that, since GenNode is composed of copyable and
assignable types, the standard implementation will do.
But I overlooked the run time type check on the opaque
payload type within lib::Variant. When a type mismatch
is detected, the default implementation has already
assigned and thus altered the IDs.
So we need to roll our own implementation, and to add
insult to injury, we can't use the copy-and-swap idiom either.
This is actually a STL library feature, and was added precisely
for the reason encountered here: if we want logarithmic search,
we'll have to construct a new GenNode object, just to have something
for the set to invoke the comparison operator.
C++14 introduced the convention that the Comparator of the set
may define a marker type `is_transparent` alongside with a generic
comparison operator. But, as is obvious from the source code of
our GNU Standard library implementation, our std::set has no such
overload to make use of that feature
http://en.cppreference.com/w/cpp/container/set/findhttp://stackoverflow.com/questions/20317413/what-are-transparent-comparators
The only good thing is that, just 10 minutes ago, I felt like
a complete moron because I'm writing a unit test for such a simple
storage class. ;-)
...and I made the decision *not* to consider any kind of
generic properties for now. YAGNI.
UI coding is notorious spaghetti code.
No point in fighting that, it is just the way it is,
because somewhere you're bound to get concrete, hands-on.
right now, what we actually need here is just some integer,
so the GenNode payload is typed to int (or just to anything
different than a Record, because the Record signals that
we intend to bind, not to invoke the command)
the values.child() call would also do a bounds check,
but only to rise a error::Invalid "index out of bounds".
So now we generate a clear message to indicate that
actually a runtime-checked type mismatch caused this problem
the functionality as such is already covered,
but it seems important enough to warrant a dedicated test.
incidentally, Duration still lacked a default ctor.
Time values are default constructible, yet immutable.
incidentally, this uncovered yet another unwanted narrowing conversion,
namely from double via gavl_time_t to TimeValue or alternatively
from double via FSecs (= rational<long>) to Duration.
As in all the previos cases, actually the compiler is to blame,
and GCC-5 is known to get that one right, i.e. let the SFINAE fail
instead of passing it with a "narrowing conversion" warning.
Note: the real test for command binding with immutable types
can be found in BusTerm_test
Completely removed the nested hierarchy, where
the top-level implementation forwarded to yet another
sub-implementation of the same interface. Rather, this
sub-implementation (OpClosure) is now a mere implementation
detail class without VTable, and without half-baked
re-implementation of the CmdClosure interface. And the
state-switch from unbound to bound arguments is now
implemented as a plain-flat boolean flag, instead of
hiding it in the VTable.
To make this possible, without having to rewrite lots of
tests, I've created a clone of StorageHolder as a
"proof-of-concept" dummy implementation, for the sole
purpose of writing test fixtures. This one behaves
similar to the real-world thing, but cares only
for closing the command operation and omits all
the gory details of memento capturing and undo.
...probably just an omission. TimeValue and Time are
also default constructible, and this makes sense, semantically.
Please note that Time values are *immutable* though.
Only TimeVar can be reassigned. This is so by design
Seems this was part of the confusion when looking at
the inheritance graph: Names where almost reversed
to the meaning. the ArgumentHolder was *not* the
argument holder, but the top level closure. And
the class "Closure" was not "the" Closure, but
just the argument holder. ;-)
recently, I've introduced this ability in our toString template.
as it turned out, the bool type was not selected by our
boost::format frontend for special treatment, thus showing
just the fallback «bool»
...when the Test-Nexus processes a command binding message.
In the real system of course we do not want to log every bind message.
The challenge here is the fact that command binding as such
is opaque, and the types of the data within the bind message
are opaque as well. Finally I settled on the compromise
to log them as strings, but only the DataCap part;
most value types applicable within GenNode
have a string representation to match.
the rationale is that I deliberately do not want to provide
a mechanism to iterate "over all contents in stringified form".
Because this could be seen as an invitation to process GenNode-
datastructures in an imperative way. Please recall we do not
want that. Users shall either *match* contents (using a visitor),
or they are required to know the type of the contents beforehand.
Both cases favour structural and type based programming over
dynamic run-time based inspection of contents
The actual task prompting me to add this iteration mechanism
is that I want to build a diagnostic, which allows to verify
that a binding message was sent over the bus with some
specific parameter values.
...also for the existing variant, which packages an
arbitrary number of arguments in stringified form
into a given container type. Moreover, the new
form of stringify allows to write util::join
in a clearer way, eliminating the lambda.
...since, semantically, the template param INT is expected to be
"number like", which implies to base the "in range" notion
on a comparison concept (e.g. we might use floating point numbers)
...this was clearly wrong; it went unnoticed just
because the linker cleans up duplicates of
template instantiations. (I'd expect GCC-5
to spot such errors)
very similar to boost::irange, but without heavyweight boost
includes, and moreover based on our Lumiera Forward Iterator concept
Such a inline-range construct makes writing simple tests easy
based on the new generic tuple builder, we're now able to
add a new binding function into the command implementation
machinery, alongside the existing one. As it stands, the
latter will be used rather by unit tests, while the new
access path is what will be actually taken within
the application, when receiving argument binding
messages dispatched via the UI-Bus.
since this is a quick-n-dirty workariound, until we're using GCC-5,
I'll err for the simple and safe side and disallow any conversion
from LuidH do some algebraic data type. The problem arises,
sincd LuidH defines a conversion to size_t, which depends
on the platform. So, without checking the actual NumericLimits,
there is no way we can allow a conversion to size_t in a
hard wired way, while disallowing a narrowing conversion
to 32bit unsigned int on 64bit platforms.
And in the end, we don't want conversions from LUID to
numeric values to happen automatically anyway. But of
course we *do* want automatic promotion from a LuidH
to a PlacementRef...
...to avoid warnings when deriving a publicly visible type
from that interface. Newer GCC and CLang versions emit
warnings when details from an anonymous implementation
namespace will leak into type signatures visible outside
the translation unit. In this case here, it's the VTable.
because this element picking mechanism for tuples
looks like an instance of something generic.
At least I've written almost the same just some days ago
for the revised version of function-closure, where the
task was to replace a stretch of type arguments in
a given tuple type with a stretch of placeholder types
and then to build a modified ctor, which just fills
in the remaining arguments, while default constructing
the placeholder types. And if we look into the GNU
implementation of std::bind, they're using a similar
concept (with the difference that they're building
a functor object, where we use a type converter)
This refactoring also integrates some generally useful
bits into our standard metaprogramming helper collection
this was a classical example of a muddled and messed-up design,
driven just by the fact that I wanted to "spare" some functions,
with the net effect of writing more functions, plus a proxy class
plus create a lot of confusion for the reader.
This was easy to resolve though, once I resorted to the
general adivice to make public interface methods final,
make the extension ponts protected and never
to chain two extension points
based on the previous experiments, this adds a fake operation
and a definition frame to hook this operation as pseudo Proc-Layer command
WIP: the invocation itself is not yet implemented.
We need to build a custom invocation pattern for that,
in order to be able to capture the instance-ID of the command
on invocation
NOTE: also, because of #989, we can not bind a time value for this test
not sure yet if any of this works, because the
technicalities of dealing with variadic types are
quite different to our LISP-style typelist processing.
The good news is that with variadic templates it is
indeed possible, to supply dynamically picked arguments
to another function taking arbitrary arguments.
This all relies on the feature to unpack argument packs,
and, more specifically, about the possiblity to "wrap"
this unpacking around interspersed function call syntax
template<size_t...i>
Xyz
do_something(MyTuple myTuple)
{
return Xyz (std::get<i> (myTuple) ... );
}
Here the '...' will be applied to the i... and then
the whole std::get-construct will be wrapped around
each element. Mind bogging, but very powerful
we made double use of our Tuple type, not only as a
generic record, but also as a metaprogramming helper.
This changeset replaces these helpers with other
metafunctions available for our typelists or type sequences
(with the exception of code directly related to Tuple itself,
since the intention is to delete this code alltogether shortly)
there was a muddeled mix of type lists and type sequences,
and both where used for processing. Probably the origin
of that confusion was the design of our own Tuple class,
which is implemented based on typelists but accepts a
type sequence at the front-end. From there, a confusing
pattern of equivalence between lists and sequences emerged,
leading to several functions accepting "anything".
This misdesign is not eradicated yet, but in this specific
instance here, has cost me several hours to pinpoint a bug
introduced while refactoring.
See also #967 and #301
This definition -- together with the already existing specialisation
in typeseq-util, allows always to rebind from a given type-list back
to the corresponding type-sequence, by accessing the type member `Seq`
...which causes problems when a preceding include
has already dragged in <functional>
the actual problem is the std::hash hack, which probably
is even no longer possible and could be removed (but
I don't have the time to investigate this somewhat
tricky topic right now)
To prevent this confusing situation, I'm adding the
include of "lib/symbol.hpp", to ensure we do have
the actual definitions of string and Literal,
which trait.hpp just declares forward.
An note, lib/symbol.hpp also includes hash-standard.hpp
first, so we avoid triggering problematic situation
from a header (format-cout.hpp), which is pervasively used
all over the place....
basically this comes down to provide some convenience fixture
within the test::Nexus, which automatically generates and wires
mock commands.
Not sure if this is even possible to the extent envisioned here
since our test.sh runner can be used to verify the
expected output printed by tests, working with these
output transcripts of larger tests can be hard at times.
These separators help to find who produced which output
and they prevent a regexp match to grep beyond the feed
of a single function (which can be a common problem
when using the self-diagnostic output of the facility
currently in test, which obviously will be similar
on any data printed.
- replace remaining usages of typeid(T).name()
- add another type simplification to handle the STL map allocator
- clean-up usage in lib/format-string
- complete the unit tests
- fix some more bugs
quite sure I never really meant to do that, just, at that time,
it seemed logical to treat Placement as yet another smart-ptr.
But in the light of what crucial entity Placement became meanwhile,
I can't imagine a single case where anyone wants to wrap away a
placement as if it was some shrink-wrap
turns out this is a tricky situation.
We want to accept pretty mutch everything, yet we want to get a grip
on anything object-like, so to reveal available RTTI information.
Now, given the way C++ template substitution works, the 'TY const&' overload
wins with only a few exceptions. The reason is, C++ invokes most functions
passing the concrete argument as reference, unless this is not possible,
because the concrete artument is a rvalue. The automatic reduction of
reference expressions does the rest. Consequently the overload with 'const&'
turns out to be the best match even when we invoke the function with a
pointer expression, which would then be made into a pointer-to-a pointer
by our forward call.
There are two remedies for this dilemma:
- make the second overload just typeStr (TY&)
- explicitly remove the second overload for pointers
The first solution unfortunately would rule out passing of anonymous
objects like concatenated strings; in fact it would rule out passing
rvalues as such. While the second solution, chosen here, works really
for everything, and also has the nice side effect of stripping away
any const, pointer and reference adornements elegantly before we
even start to analyse the type.
The only downside of this solution is that it looks intimidating
to the casual reader. Well, I'd say, get used to it.
over time, we got quite a jungle with all those
shome-me-the-type-of helper functions.
Reduced and unified all those into
- typeString : a human readable, slightly simplified full type
- typeSymbol : a single word identifier, extracted lexically from the type
note: this changeset causes a lot of tests to break,
since we're using unmangeled type-IDs pretty much everywhere now.
Beore fixing those, I'll have to implement a better simplification
scheme for the "human readable" type names....
...based on all the clean-up and reorganisation done thus far,
we're now able to rebuild the util::str in a more direct and
sane way, and thus to disentangle the header inclusion problem.
due to the new automatic string conversion in operator<<
the representation of objects has changed occasionally.
I've investigated and verified all those incidents.
...other than intended, the bomb did explode on random occasions,
with an probability of about 4% (when rr >= 96).
Btw, there was also the mistake to throw an heap allocated
object by pointer. Damn Java habits.
- remove unnecessary includes
- expunge all remaining usages of boost::format
- able to leave out the expliti string(elm) in output
- drop various operator<<, since we're now picking up
custom string conversions automatically
- delete diagnostics headers, which are now largely superfluous
- use newer helper functions occasionally
I didn't blindly change any usage of <iostream> though;
sometimes, just using the output streams right away
seems adequate.
the usual drill...
only when wrapped into a factory function, RAII is really
airtight, even when used from within expression evaluation.
Thanks C++11 we're now able to provide such en passant
our lib::P smart-pointer is built on top of std::shared_ptr,
while additionally delegating comparisons to the pointee.
In a similar vein, I've now added a custom string conversion,
delegating to the pointee, with a type-string as fallback.
Together with the built-in string conversion for output streams,
we should now be able to remove most of the explicit string
conversions and calls to util::str in all of our test code.
This removes the last roadblock towards disentangling the
pretty-printing header includes, which in turn should allow
us to remove any conditional code in the built-in string
conversion of GenNode, Variant and the like. Which basically
was the objective for ticket #985
use a shortened display, showing only the last 4 bytes for diagnostics
since we're typically only interested in spotting "same" and "different",
while the full memory address is irrelevant
provide a generic overload for the stream inserter operator<<
to use custom string conversions when applicable.
The overload will be disabled when a direct lexical conversion
is possible (which means, we can expect the output stream to
know allready how to print those values, like e.g. all kinds
of numbers).
Additionally, we provide a pretty-printing mechansim for pointers,
to show the address and possibly invoke a custom string conversion
on the pointee
No more fiddling with printf just to show a number reliably!
simple functions to pretty-print addresses,
doubles and floats (fixed-point, with rounding).
Also make all these basic formatting helpers noexcept
- simple function to pick up the mangled type
- pretty-printing is implemented in format-obj.cpp
- also move the demangleCxx()-Function to that location,
it starts to be used for real, outside the test framework
our minimal compiler requirement is gcc-4.9 since the
transition to Debian/Jessie as reference system.
gcc-4.9 is known to treat SFINAE on private fields properly
this is a stripped-down and very leightweight variant
of the well-known enable_if metaprogramming trick.
Providing this standard variant in a header with minimal
dependencies will allow us to phase out boost inclusions
from many further headers. As a plus, our own variant
is written such as to be more conciese in usage
(no "typename" and no acces of an embedded "::type" menber)
this includes a reorganisation concept for the header includes,
a minimal version (with minimal include dependencies), and
a generic ostream inserter operator<<
...and learned a lot about the new type_traits on the way.
As it seems, it is not possible to get a clean error message
when passing an "object" with no custom string conversion;
instead, some overload for an rvalue-ostream kicks in.
probably I'll go for shoing a type string in these cases
now we use boost::format through our own front-end util::_Fmt
solely, which both helps to reduce compilation time and code size,
and gives us a direct string conversion, which automatically
uses any custom operator string() available on arguments.
While desirable as such, I did this conversion now, since
it allows us to get rid of boost::str, which in turn helps
to drill down any remaning uses of our own util::str
the fixed version is actually more permissive,
insofar it matches any type of event, when ID = classifier
(or alternatively it matches events with type = classifier)
our formatting helper for diagnostics output,
which is primarily used in the unit-tests,
first tries to invoke a custom string conversion.
If that is not possible, it falls back to printing
the demangled type name of the object in question.
With just a minor change we're able to evaluate RTTI here
and print the actual type name, instead of the static
supertype the compiler sees on invocation. We just rely
on the typeid(obj) built-in function.
The only catch is we have to strip the " const*" suffix
(and no, it is not possible to do that on metaprogramming
level, due to the special situation where we have a void*)
This also prompted me to write some util functions for
this often encountered task to check / remove a prefix or suffix
Hopefully I've got those functions correct and safe....
...this is necessary whenever the mocked facility covered
by log matching is managed automatically as singleton,
because then other test cases will leave garbage
in the log
I worked under the erroneous assumption, that Doxygen
will use its internal entity-IDs as the link-IDs when
generating mardown-links. Yes, this seemed logical and
this would be the way I'd implement it....
But seemingly, Doxygen is not so consistent when it
comes to questions of syntax. The same holds true for
markdown, which lacking a coherent definition anyway.
Another problem is that Doxygen's auto-link generation
frequently fails, for reasons not yet clear to me.
Sometimes it seems to be necessary to give it a nudge
by including the \ref command. While I'm not willing
to go into focussed invstigation of Doxygen syntax
right now, at least I've done a search-and-replace
to remove the malformed links I've written the
last days
the initial draft of this concept is in place now, and
the first round of unit tests pass. I've got some understanding
of the purpose of the interactions and involved elements
and I'm confident this design is evolving in a sane way.
Note: extensive documentation is in the TiddlyWiki,
here I've just pasted and reworded some paragraphs from there
and integrated them into the Doxygen docs
Explanation: sigC++ was already linked as transitive dependency
from gtkmm, since it is used for the "signal-slot" system wihin GTK.
But now we want to use sigC++ itself from our generic UI-Backbone,
so we need to pick up the additional compiler and linker flags
and use them when building the relevant parts of both the application
and the test suite
at the point when we're connecting a new node to the UI-Bus,
the new node's BusTerm is not yet initialised, since we need
the uplink connection we're about to create for setting up
this BusTerm.
Consequently, the new nodes's ID is not yet initialised,
so we need to pass this endpoint-ID explicitly to the
registration function.
since, by definition, the Nexus is "the" up-link,
all we need is clever overriding of the relevant
handling functions, so the nexus will care for the routing,
while the CoreService cares for command and presentation
state handling
next step will be to rig the mock element and set up
and cover the basic / generic element behaviour
This changeset
- adapts the (planned) unit test to the semantic of
the EventLog, which is now fully implemented
- adjusts the function names on the public Tangible interface,
to be better in line with the naming convention of the
corrsponding operations on the UI-Bus:
* "mark" operations are towards the UI element
* "note" messages are from the UI element towards some
state manager, which can be reached via the bus
so this turned out to be rather expensive,
while actually not difficult to implement.
On the way, I've learned
- how to build a backtracking matcher, based on
a filtering (monadic) structure and chained lambdas
- learned the hard way how (not) to return a container
by move-reference
- made first contact with the regular expressions
now available from the standard library
this function is of use also for creating a vector of strings
from a bunch of C-Strings, but it could also be used to
construct other stuff initialised by strings (e.g. RegExps)
this deals with a recurring problem in test code:
very common "simple" fractional values can not be represented
precisely as binary floating point. The classical example is 0.1
Since this is a diagnostics facility, we can cheat around this
insidious problem by just setting a limited rendering precision.
Floating point numbers behave deterministic; you just need
to know how to deal with limited precision.
abandon the use of an assertion exception to signal match failure,
rather use a final bool conversion to retrieve the results.
Error messages are now delivered by side effect into STDERR
The reason is we're unable to deliver the desisred behaviour
with the chosen DSL syntax in C++ ; on a second thought the
new approach is even better aligned with the overall way
we're writing tests in Lumiera. And we produce match-trace
messages to indicate the complete matching path now
implemented a solution to determine negative matches.
But because this solution relies on throwing from a destructor,
it is not possible to catch the resulting assertion failure.
Not sure why (AFAIK there is no second exception thrown
while unwinding the stack), but throwing from dtors is
considered "undefined behaviour" anyway.
So this solution is of limited use
beyond that solution, I'm not sure if the desired syntax
can be implemented at all in C++. Seems that we need to build
a bracketing construct, first to initiate a negated match
and finally, after all queries, to detect if there happened
any failure or not
...no need to enclose empty sections when there are no
attributes or no children. Makes test code way more readable.
TestEventLog_test PASS as far as implemented
...and fix an error (header include order of diagnostics facility)
which prevented the first matcher implementation to work
the after()-match now works as expected
this is the tiny bit of operational functionality needed on top:
whenever we're reconfiguring the predicate, we need to re-trigger
the evaluation (and clear the cached value)
n.b.: I've verified in debugger that the closure is
allocated on the heap and the functors are passed by value
after looking into our various iterator tools,
it seems obvious that our filtering iterator implementation
has almost all of the required behaviour; we only need to
add a hook to rewrite and extend the filtering functor,
which can now nicely done with a lambda closure.
This means all memory management, if necessary, is
pushed into std::function and the automated memory
management for closures provided by the runtime.
...providing the standard implementation of UI-Bus connectivity.
It seems reasonable to place all of the UI-Bus implementation into
a single translation unit
what you see here now is just the tip of the icebearg...
If we follow this route, the Lumiera UI will become way more
elaborate and responsive than average desktop applications
..while we should note at this point that the whole techique
of hijacking std::hash is superfluous now, since the standard libray
does no longer define a static assertion which defeats SFINAE
some tests rely on additional diagnostics code being linked in,
which happens, when lib/format-util.hpp is included prior to
the instantiation of lib::diff::Record rsp. lib::Variant.
The reason why i opended this can of worms was to avoid includion
of this formatting and diagnostics code into such basic headers
as lib/variant.hpp or lib/diff/gen-node.hpp
Now it turns out, that on some platforms the linker will use
a later instantiation of lib::Variant::Buff<GenNode>::operator string
in spite of a complete instantiation of this virtual function
being available already in liblumierasupport.so
But the real reason is that -- with this trickery -- we're violating
the single definition rule, so we get what we deserved.
TODO (Ticket #973): at a later point in development we have to re-assess,
the precise impact of including lib/format-util.hpp into
lib/diff/gen-node.hpp
Right now I expect GenNode to be used pervasively, so I am
reluctant to make that header too heavyweight.
preliminary workaround for Ticket #972
On Debian/Jessie, we observed the following error
"gtk-lumiera.css:38:19Theming engine 'adwaita' not found"
even though the package gnome-themes-standard *is* installed
This allows at least to bring the UI up, even if loading
our custom theme and stylesheet fails.
This is a development snaphot pre release of Lumiera.
It features codebase maintenance, upgrade to C++14 and GTK-3
and some work towards a Proc-GUI connection (unfinished)
Update README, AUTHORS, LICENSE and similar release docs.
because otherwise we'd need to send a whole subtree
over the wire and then descend into it just to find an element.
This too is a ripple effect of making '==' deep
well... this was quite a piece of work
Added some documentation, but a complete documentation,
preferably to the website, would be desirable, as would
be a more complete test covering the negative corner cases
yeah, working with open fire is dangerous...
For performace reasons I've undercut the premise
to make GenNode / Record immutable. Now I'm dealing with
raw storage layout together with this quite hairy distinction
between "attribute scope" and "child scope"
In hindsight, it might have been better to implement Record
as a single list, and to maintain a shortcut pointer to jump
to the start of the attributes.
while implementing this, I've discovered a conceptual error:
we allow to accept attributes, even when we've already entered
the child scope. This means that we can not predictable get back
at the "last" (i.e. the currently touched) element, because this
might be such an attribute. So a really correct implementation
would have to memorise the "current" element, which is really
tricky, given the various ways of touching elements in our
diff language.
In the end I've decided to ignore this problem (maybe a better
solution would have been to disallow those "late" attributes?)
My reasoning is that attributes are unlikely to be full records,
rather just values, and values are never mutated. (but note
that it is definitively possible to have an record as attribute!)
...while I must admit that I'm a bit doubtful about that
language feature, but it does come in handy when manually
writing diff messages. The reason is the automatic naming
of child objects, which makes it often hard to refer to
a child after the fact, since the name can not be
reconstructed systematically.
Obviously the downside of this "anonymous pick / delete"
is that we allow to pick (accept) or even delete just
any child, which happens to sit there, without being
able to detect a synchronisation mismatch between
sender and receiver.
i.e. flat match, not deep equality.
This allows to send just an Ref (with the ID) over the
wire to refer to an complete object to be picked, moved
or deleted on the receiver side.
in the first version, I defined equality to just compare the IDs
But that didn't seem right, or what one would expect by the concept
of equality (this is a long standing discussion with persistent
object-relationally mapped data).
So I changed the semantics of equaility to be "deep".
As this means possiblty to visit a whole tree depth-first,
it seems reasonable to provide the shallow "identity-comparison" likewise.
And the most reaonable choice is to use the "matches(object)" API
for that, since, in case of objects, the matches was defined
as full equality, which now seems redundant.
Thus: from now on: obj.matches(otherObj)
means they share the same IDs
The Ref-GenNode is just a specifically constructed GenNode,
and intended to be sliced down to an ordinary GenNode
immediately after construction. It seems, GCC didn't "get that"
and instead emitted an recursive invocation of the same ctor,
which obviously leads to stack overflow.
Problem solved by explicitly coding the copy initialisation,
after the full definition of Ref is available.
the type is the only meta attribute supported by now,
thus the decision was to handle this manually, instead of
introducing a full scope for meta attributes. Unfortunately
this leads to an assymetry: while it is possible to send an
attribute named "type", which will be intercepted and used
as a new type ID, the type will not show up when iterating
or searching through attributes.
When applying a diff, the only possibility is to *insert*
a new type attribute, and we need to check and handle this
likewise manually.
It is difficult to reconcile our general architecture for the
linearised diff representation with the processing of recursive,
tree-like data structures. The natural and most clean way to
deal with trees is to use recursion, i.e. the processor stack.
But in our case, this means we'd have to peek into the next
token of the language and then forward the diff iterator
into a recursive call on the nested scope. Essentially, this
breaks the separation between receiving a token sequence and
interpretation for a concrete target data structure.
For this reason, it is preferrable to make the stack an
internal state of the concrete interpreter. The downside of
this approach is the quite confusing data storage management;
we try to make the role of the storage elements a bit more
clear through descriptive accessor functions.
implement the list handling primitives analogous to the
implementation of list-diff-applicator -- just again with
the additional twist to keep the attribute and child scopes
separated.
...so now the stage is set. We can reimplement
the handling of the list diff cases here in the context
of tree diff application. The additional twist of course
being the distinction between attribute and child scope
each language token of our "linearised diff representation"
carries a payload data element, which typically is the piece
of data to be altered (added, mutated, etc).
Basically, these elements have value semantics and are
"sent over wire", and thus it seems natural when the
language interpreter functions accept that piece of payload
by-value. But since we're now sending GenNode elements as
parameter data in our diff, which typically are of the
size of 10 data elements (640 bit on a 64bit machine),
it seems more resonable to pass these argument elements
by const& through the interpreter function. This still
means we can (and will indeed) copy the mutated data
values when applying the diff, but we're able to
relay the data more efficiently to the point where
it's consumed.
this boils down to the two alternatives
- manipulate the target data structure
- build an altered copy
since our goal is to handle large tree structures efficiently,
the decision was cast in favour of data manipulation
so basically it's time to explicate the way
our diff language will actually be written.
Similar to the list diff case, it's a linear sequence
of verb tokens, but in this case, the payload value
in each token is a GenNode. This is the very reason
why GenNode was conceived as value object with an
opaque DataCap payload
while it's still not really clear how we'll use this helper
and if we need it at all -- some weeks ago I changed its
semantics to be strictly based on the delta to a reference level.
Now this means, we could go below level zero, but this doesn't
make any sense in the context of navigating a tree. Actually,
our test case triggered this situation, which caused the
reference level to wrap around, since it is stored in an
unsigned variable.
Thus I'll add a precondition to keep the level positive,
and I'll change the test to comply.
Initially I've deliberately omitted those, to nudge towards
using time quantisation and TCode formatting for any external
representation of time values.
While this recommendation is still valid, the overloaded
string conversion turns out to be helpful for unit testing
and diagnostics in compound data structures.
See Record<GenNode>
initially the intention was to include a "bracketing construct"
into the values returned by the iterator. After considering
the various implementation and representation approaches,
it seems more appropriate just to expose a measure for the
depth-in-tree through the iterator itself, leaving any concerns
about navigation and structure reconstruction to the usage site.
As rationale we consider the full tree reconstruction as a very
specialised use case, and as such the normal "just iteration" usage
should not pay for this in terms of iterator size and implementation
complexity. Once a "level" measure is exposed, the usage site
can do precisely the same, with the help of the
HierarchyOrientationIndicator.
Whooa!
Templates are powerful.
programming this way is really fun.
under the assumption that the parts are logical,
all conceivable combinations of theses parts are bound to be correct
it passes compilation, but the test still fails, since
I've changed the expected semantics of the iteration,
in the light of the insights I've gained during
re-investigation of the IterExplorer.
What I now actually intend is rather to embed a
HierarchyOrientationIndicator into the iterator,
instead of returning a special "bracket" marker
reference to indicate return from a nested scope.
Only a Record payload constitutes a nested scope.
For all other (primitive) values, we return an empty iterator.
When used within ScopeExplorer, this implementation will just
lead to exposing any simple value once, while delving into
iteration of nested scopes
The only substantial change (besides compilation fixes) is
to confine the iteration to *const access*
This is a good thing; the whole Record/GenNode structure
was designed to represent immutable data, necessitating
a dedicated *Mutator* for any reshaping.
seemingly the operator-> was not yet used in any real scenario.
The whole point with IterAdapter is that it uses an opaque "location type",
which is owned by the controlling container. In many cases this will
actually be just a pointer into the container storage, but we
must not assume it is this way. Thus the only way to obtain a
(language) pointer is to dereference the "position type" and
take the address of the result
Initially I intended just to supply an addapter to use
the monadic IterExplorer for this recursive expansion
of GenNode contents. Investigating this approach was
relevant to highlight the minimum requirements for
such an evaluation mechanics: since our GenNode
is an hierarchical structure without back-links,
we are bound to use a stack at some point. And
since an Iterator is a materialised continuation,
we can not use the processor stack and are forced
to represent this stack in memory.
Yet, on second thought, we do not need the full power
of the IterExplorer monad; especially we do not need
to bind arbitrary functions into the monad, just one
single scope exploring function, implemented as
Variant visitor. Based on these observations, we can
"inline" the monad structure into a double nested
iterator, where the outer capsule carries a stack
of scopes to be explored.
This helper was drafted for the Job / JobPlanning and Scheduler
interface in 2013, but seemingly not yet put into action. While
in the original use case, we have a genuine measuerment for the
tree depth (given by the depth of the processing stack), in other
use cases we want to use to offset embedded within the indicator
itself for keeping track of the depth. Thus I add a second
mark operation, which usess the current offset to set a new
reference level. This has the consequence that the offset
has now to reflect the new reference point immediately
Since C++ is not a real functional programming language and
has unsafe unmanaged pointers, it is not difficult to produce
dangling references within an extended evaluation pipeline
involving transient objects and pass-by-reference.
In the initial implementation, I built in a safeguard copy
into the signature of the Explorer function, to make sure even
a transiently dressed-up input value gets materialised before
proceeding with the source sequence. Unfortunately this safeguard
turns out as a roadblock now; we might as well take the input
by reference and return an "expanded" state by value. We might
even want to do the full "expansion" on referred state, when
we're able to ensure the source values remain in memory
until consumption.
Thus now the full power of decision is placed on the signature
of the explorer function. The expansion strategies of IterExplorer
will no longer attempt to "sanitise" the signature of the passed-in
function to prevent desaster; I've added some warnings into the
documentation to highlight that danger. Basically, if you want
to be clever, then you're bound to read and understand inticacies
of the implementation.
If in doubt, use values and copying. C++ is optimised for that.
allow to pick the bare function signature from any "function like" entity
Note: we're still unsing our own Typelist construct to represent
the function argument types. Since we're now using C++11, this
could be replaced by varargs, and we could get rid of those
various overloads for different numbers of arguments.
Unfortunately this transition is linked to the usage of
argument tuples in our command framework (which could then be
replaced by std::tuple), and this is still a bit of rather
technical work, which I do not want to spend much time on
right now
remembered that some years ago I had to deal with a very similar problem
for planning the frame rendering jobs. It turned out, that the
iterator monad developed for this looks promising for our task at hand
horay!
seems like madness?
well -- found and squashed a bug: equality on RecordRef
implicitly converted to GenNode(RecordRef), which always
generates new (distinct) IDs and so never succeeds. What
we really want is equality test on the references
contrary to the Visitor, accepting a Predicate is const,
and -- of course -- the Predicate invocation returns bool.
This can be used to implement comparison operators or
search functions on Variant based data structures.
There is no generic implementation for these functions, since
they are highly dependent on the payload used within Record<TY>
Here we use Record<GenNode>, which turns the whole setup into an
recursive data type; we especially rely on the fact that each
GenNode has an embedded symbolic ID, and we use this ID to encode
the 'key' for named attributes
while in debugging, it turned out that the short type-prefix
was implemented in a too simplistic way; it fails on stuff
like 'lib::diff::Record<lib::diff::GenNode>'
while I must add, that the whole purpose of these ID functions
is somewhat unclear and needs to reveal itself as we move forward
initially my intention was to use the ID for equality test.
But on a second thought, this seemed like a bad idea, since
it confuses the concepts of equality and identity.
Note: at the moment, I do not know if we even need an equality test,
so it is provided here rather for sake of completeness. And this
means even more that we want an 'equality' implementation that
does what one would naively expect: compare the object identity
*and* compare the contents.
...while on the train back from FrOSCon.
still the same old problem: we need a better hash function
for generating our Entry-IDs. The default hash function from Boost performs
poor on strings with common prefix and trailing number.
We use a hackish workaround, which is sufficient to avoid collisions
among the first 10000 numbers.
not entirely sure about the design, but lets try this approach:
they can be "cloned" and likewise move-assigned, but we do not
allow the regular assignment, because this would enable to use
references like pointers (what we deliberately do not want)
especially setting (changing) attributes turned out to be tricky,
since in case of a GenNode this would mean to re-bind the hash ID;
we can not possibly do that properly without knowing the type of the payload,
and by design this payload type is opaque (erased).
As resort, I changed the semantics of the assign operation:
now it rather builds a new payload element, with a given initialiser.
In case of the strings, this ends up being the same operation,
while in case of GenNode, this is now something entirely different:
we can now build a new GenNode "in place" of the old one, and both
will have the same symbolic ID (attribute key). Incidentally,
our Variant implementation will reject such a re-building operatinon
when this means to change the (opaque) payload type.
in addition, I created a new API function on the Mutator,
allowing to move-in a complete attribute object. Actually this
new function became the working implementation. This way, it is
still possible to emplace a new attribute efficiently (consider
this to be a whole object graph!). But only, if the key (ID)
embedded in the attribute object is already what is the intended
key for this attribute. This way, we elegantly circumvent the
problem of having to re-bind a hash ID without knowing the type seed
initially, the intention was to inject the type as a magic attribute.
But this turned out to make the implementation brittle, asymmetric
and either quite demanding, or inefficient.
The only sane approach would be to introduce a third collection,
the metadata attributes. Then it would be possible to handle these
automatically, but expose them through the iterator.
In the end I decided against it, just the type attribute
allone does not justify that effort. So now the type is an
special magic field and kept apart from any object data.
this solves the problem how to deal with value access
- for the simple default (string) implementation,
we use a 'key = val' syntax and thus have to split strings,
which means we need to return contents by value
- for the actual relevant use case we have GenNode entries,
which may recursively hold further Records. For dealing
with diff messages over this data struture, its a good
idea to allow for const& value access (otherwise we'd
end up copying large subtrees for trivial operaions)
OMG, what was all this about?
OK... this cant possibly work this way.
At least we need to trim after splitting the attributes.
But this is not enough, we want the value, which implies
to make the type flexible (since we cant return a const& to
a substring extracted on-the-fly)
this was an half hearted attempt to satisfy CLang,
but GCC as keen as a razor insists on these inherited
functions not being accessible --
seems like the time is over, when GCC used to be forgiving
and CLang briliantly precise...
So the conclusion of this "round trip" is: whenever GCC
also starts whining about shadowed overloaded virtual functions,
we'll just switch to "-Wno-overloaded-virtual" and be done with
that pointless discussion.
Since C++11, we have the Java style override specifier,
which does a way better job at spotting signature mismatches
Note: not fixing all relevant warnings.
Especially, the "-Woverloaded-virtual" of Clang defeats the whole purpose
of generated generic interfaces. For example, our Variant type is instantiated
with a list of types the variant can hold. Through metaprogramming, this
instantiation generates also an embedded Visitor interface, which has
virtual 'handle(TY)' functions for all the types in question
The client now may implement, or even partially implement this Visitor,
to retrieve specific data out of given Variant instance with unknown conent.
To complain that some other virtual overload is now shaddowed is besides the point,
so we might consider to disable this warning altogether
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=56402
The lambda definition captures the this pointer,
but the ctor of the lamda does not initialise this capture.
In our case, we're lucky, as we don't use the "this" pointer;
otherwise, we'd get a crash a runtime.
Fixed since GCC-4.7.3 --> it's *really* time to upgrade to Debian/Jessie
the object VTable is typically emitted when the compiler
encounters the first non-static non-inline function of
the class or a derived class.
Sometimes this happens within the wrong library and so
the compiler needs a nudge to emit those infrastructure functions.
But in most cases this works out of the box and need no further
magic incanctations, which might have a downside.
Especially because also a non-inline dtor does incur a call overhead,
whereas an inline dtor can be trivially elided.
after sleeping a night over this, it seems obvios
that we do not want to start the build proces "implicitly",
starting from a Record<GenNode>. Rather, we always want
the user to plant a dedicated Mutator object, which then
can remain noncopyable and is passed by reference through
the whole builder chain. Movin innards of *this object*
are moved away a the end of the chain does not pose much risk.
especially I've now decided how to handle const-ness:
We're open to all forms of const-ness, the actual usage decides.
const GenNode will only expose a const& to the data values
still TODO is the object builder notation for diff::Record
forwarding equality to the embedded EntryID
Basically, two GenNodes are equal when they have the same "identity"
Ironically, this is the usual twist with database entities
on a second thought, this "workaround" does not look so bad,
due to the C++11 feature to request the default implementation explicitly.
Maybe we'll never need a generic solution for these cases
I decided to allow for an 'unbound' reference to allow
default construction of elements involving record references.
I am aware of the implications, but I place the focus
on the value nature of GenNode elements; the RecordRef
was introduced only as a means to cary out diff comparisons
and similar computations.
basically this is the well known problem #587
Just it became more pressing with the Upgrade to Jessie and Boost 1.55
So I've pulled off the well known "Knuth trick" to spread the
input data more evenly within the hash domain.
And voilà: now we're able to use 100000 number suffixes without collision
- move the santitise operation up into EntryID's ctor
- turn the recast() operation into a real in-place cast
these changes should be transparent to the existing usages
of EntryID (within the asset framework), but allow for use
as attribute name holder in GenNode, since we're now able
to feed existing name/ID values directly into the ctor
of BareEntryID, without any spurious santitise operation.
this was introduced into namespace mobject and spread from there.
Since the habit is to use more specific typedefs like PClip,
it is preferrable to spell out the full namespace
using the struct-scheme.hpp and the requirements for
EntryID as a guideline. The goal is to move EntryID
over into the support lib, which means we need to get rid
of all direct proc::asset dependencies. Thus, these generic
ID functions shall form a baseline implementation, while
asset::Struct may provide the previously used implementation
through specialisation -- so the behaviour of EntryID will
not change for the structural assets, but we'll get a more
sane and readable default implementation for all other types.
before engaging into the implementation of lib::Record,
I prefer to conduct a round of planning, to get a clearer
view about the requirements we'll meet when extending
our existing list diff to tree structures
Initially, I considered to build an index table like
collection of ordered attributes. But since our actual
use case is Record<GenNode>, this was ruled out in favour
of just a vector<GenNode>, where the keys are embedded
right within the nameID-Field of GenNode.
A decisive factor was the observation, that this design
is basically forced to encode the attribute keys somehow
into the attribute values, because otherwise the whole
collection like initialisation and iteration would break
down. Thus, a fully generic implementation is not possible,
and a pseudo generic implementation just for the purpose of
writing unit tests would be overkill.
Basically this decision means that Record requires an
explicit specialisation to implement the attribute-key
binding for each value type to use.
Ouch!
Why does C++ lack the most basic everyday stuff?
It needn't be performant. It needn't support some fancy
higher order container. Just join the f***ing strings.
use Bosst?? -- OMG!! pulls in half the metra programming library
and tries to work on any concievable range like object. Just
somehow our Lumiera Forward Iterators aren't "range-like" enough
for boost's taste.
Thus let's code up that fucking for-loop ourselves, once and forever.
This is kind of the logic consequence, since we consider our
functional iterator concept still superior and will continue
to rely on it.
For some time now, I've considered to build a generic bridge
function, to use enable_if and metaprogramming to figure out
if some type is a "Lumiera Forward Iterator" automatically.
But since our concept is to some degree a contract regarding
semantics, which never can be captured by any kind of introspection,
such a bridge implementation would be rather heuristic and
bears the danger to trigger on types actually not intended
as iterator at all. So I consider such a solution as dangerous
and we'll settle with just supplying the necessary bridge
functions as free functions injected for ADL on a case by case base
this was spotted by a new GCC warning -Wunused-function
and I must admit, GCC is right here: an externally not visible
function in an anonymous namespace is not what I'd expect to be
picked up by ADL. It is rather weird that the metaprogramming
trait worked at all.
Note that the function is intentionally declared only, never defined.
We want a linker error in case boost::hash ever attempts to
use this 'deliberately ill-definded' catch-all.
I'd never imagine that this superficial draft will sit there
for 5+ years without me getting any chance to continue with that topic.
this is so saddening, so I turned off the warning :-/
In Lumiera, "Tracks" are not what you'd expect from
conventional video editing software. They are a mere
grouping devide, and are also used to implement the
"media bins" and tool palettes.
But having "folders" on the timeline would be likewise
confusing, as would be to have a "branch" or "tree".
To get out of that dilemma, we chose an understandable
but deliberately somewhat strange name: "Fork"
It was common understanding on the Mailinglist that we
should handle this renaming in a tuned-down and discrete
way: The UI will continue to show "Tracks" for a familiar
sight and "Bins" in the Asset section. But Lumiera developers
will be nudged to accomodate by renaming the entity in
source code accordingly
Cockoo hashing is a thrilling algorithm.
We investigated it during the time or our first draft
towards a confirugation system in 2008. This usage turned
up some problems -- not sure if based on the implementation
or the algorithm itself; at that time, we just switched
to the probabilistic splay tree. The whole configuration
system effort stalled afterwards; so the cuckoo implementation
remained in tree as a zombie.
This switches the Lumiera UI from GTK-2 to GTK-3
Unfortunately, this move breaks two crucial features, which have been
disabled for now: the display of video and our custom timeline widget.
Since both of these require some reworking, which in fact has already
started, we prefer to do the library and framework switch right away.
over time, a specific Lumiera code writing style has emerged.
The GUI, as it stood, used somewhat different conventions,
which now have been aligned to the common standard.
Basically we use GNU style, with some adjustments for OO-programming,
we prefer CamelCase, and write TypeNames uppercase, variableNames lowercase
it is a widely accepted rule to shape names with the usage site in mind.
Especially this means, that we use the singular form for all kinds
of collections and assortments.
Thus, the namespace should be called "widget" not "widgets",
because at usage site this becomes gui::widget::TimelineWidget
Likewise for "dialogs" and "pannels"
a long standing TODO to document the actual start-up sequence, which
is implemented this way since a long time now. There was an unwritten
section in the "Linking and Application Structure", which seems the
apropriate place for this kind of intricate techincal details.
Last week, Benny Lyons was here on visit in munich and he was pondering
the idea of an experimental secondary build system, as a way to learn
more about the source structure of Lumiera. This reminded me to fill
some missing parts of the documentation. Possibly this is also the
right moment to land the GTK-3 transition?
The actual trick to make it work is to use decltype on the function operator
http://stackoverflow.com/questions/7943525/is-it-possible-to-figure-out-the-parameter-type-and-return-type-of-a-lambda/7943765#7943765
In addition, we now pick up the functor by template type and
store it under that very type. For one, this cuts the size
of the generated class by a factor of two. And it gives the
compiler the ability to inline a closure as much as is possible,
especially when the created Binder / Mutator lives in the same
reference frame the closure taps into.
to carry out that rather obvious step, I was bound to consider
all the implications of choosing a given layout and handling pattern
for our external structure representation.
Finally, I settled upon the following decisions
- the value space represented within the DataCap is flat, not further structured
- the distinction between "attribute" and "nested object" is merely conceptual
and will be enforced solely by the diff detection / representation protocol
- basically, a nested subtree may appear as an attribute; the difference
between attributes and children lies solely in the way of access and referral:
by-name vs. positional
- it is pointless to save space for the representation of the discriminator ID
- but we can omit any further explicit type tag, because
- we do *not* support programming by switch-on-type, and thus
- we do *not* support full introspection, only a passive type-safety check
- this is *not* a limitation, since we acknowledge that GenNode is a *Monad*
- and the partial function needed within any flatMap implementation
maps naturally onto our Variant-Visitor; thus
- the DataCap can basically just *be* a Variant
- and GenNode has just to supply the neccessary shaffolding
to turn that into a full fledged Monad implementation, including
direct construction by wrapping a value and flatMap with tree walk
All relevant uses will rely on the more strict access policy
implemented with the new util::AccessCasted. Along the same line
of thinking, I've removed the "second try" convenience conversion
from the typed get-Function of OpaqueHolder. Such an unbounded
"convert it somehow" approach is almost never a good idea. Either,
one knows by design the precise type to expect, or alternatively
should rely on the base interface solely.
...with the sole exception of the usage in WrapperPointer,
which in itself looks obsolete to me; we should better re-think
the way we handle "wrapped" objects for the BuilderTools, once
we actually start implementing the Builder
Ticket #450
Note: the new Variant implementation is a re-write from scratch
and does not rely on util::AccessCasted any more. Anyway, both
are now thoroughly covered by unit test
NOTE: this was a one-time verification. Unfortunately there is no way
to verify a failing compilation automatically from a unit-test.
Thus we need to comment out these invalid cases, leaving them
here just for later referral. Need to check those manually
for new compilers to be sure!
this overload will be picked only if none of the more specific
overloads is applicable. Instantiating this overload will then
trigger a static assertion failure. This way we sort out
impossible or dangerous combinations at compile time already.
I found no simple way to include the actual type parameters in
the generated error message (string concatenation at compiletime)
The throw-statement is only there to prevent a warning due
to missing return statement.
...since I consider that a comparatively safe convenience feature.
Of course we *do perform* a NULL check and throw an exception.
So now the actual casting or conversion functions are designed
to work always on the same level of references or pointers,
which means we can just use the standard conversions of the
language. This has the nice effect of ruling out dangerous
combinations (like taking a L-ref from a R-ref) automatically
TODO: might break some unit-tests...
Explanation: our wrapper around boost::format has special
built-in support for custom operator string(). Any type,
which is neiter standard, or printable through such a
custom string conversion, is represented as a type-string.
For this fallback case, we now use our recently added
demangling call (which actually relies on a rather obscure
but standard compiler API)
still passes compilation, but not actually tested.
The visitor-style accees needs to be implemented, and the
whole virtual copy support mechanism extracted into a separate
header and covered by unit test
now the solution with the copy policy class is in place,
I prefer to return to the more verbose yet clearer notion
of distinct constructors for each case on the outer and
the inner capsule likewise.
The idea with the separate builder class would be significant
only if this class would also provide the copy support. This
turns out to be difficult, due to the access restrictions
and the necessary passing of type parameters.
turns out to be quite a tough challenge....
since obviously we want to support usage of types with
partially disabled copy/assignment operations within Variant.
As long as the corresponding operations on the container aren't
invoked, we expect those types to be usable just fine.
The problem arises at the interaction with type erasure;
to support corret copy / assignement in such a situation, we need
virtual copy / assignment operators. And, since these are to be installed
into a VTable, the templated functions will be instantiated allways,
which might cause invocation of inhibited copy / assignement functions
and thus compilation failure, in spite of never actually invoking such
an illegal operation.
The drafted solution is to mix in a specifically configured copy support policy,
which at least raises a runtime error, instead of invoking the incriminating operation(s)
finally got all those copy / assgnment flavours straight.
Still unsolved: unable to instantiate the Variant template
for a type with private assignment operator (like e.g. Time )
The problem is our virtual assignement operator, which forces
instantiation of the implementation (for the VTable), even if
the actual assignment is never invoked.
this was an immature first desgin attempt; we need a lightweight
Variant (typesafe union) implementation, so now is the time for
a second attempt. The existing Variant is used only once, and this
usage as such is in a questionable context, likely to be reworked
when we actually start coding up the builder. So I'll just move
it away and mark it @deprecated for the time being.
After some reconsideration, I decide to stick to the approach with the closures,
but to use a metaprotramming technique to build an inheritance chain.
While I can not decide on the real world impact of storing all those closures,
in theory this approach should enable the compiler to remove all of the
storage overhead. Since, when storing the result into an auto variable
right within scope (as demonstrated in the test), the compiler
sees the concrete type and might be able to boil down the actual
generated virtual function implementations, thereby inlining the
given closures.
Whereas, on the other hand, if we'd go the obvious conventional route
and place the closures into a Map allocated on the stack, I wouldn't
expect the compiler to do data flow analysis to prove this allocation
is not necessary and inline it away.
NOTE: there is now guarantee this inlining trick will ever work.
And, moreover, we don't know anything regarding the runtime effect.
The whole picture is way more involved as it might seem at first sight.
Even if we go the completely conventional route and require every
participating object to supply an implementation of some kind of
"Serializable" interface, we'll end up with a (hand written!)
implementation class for each participating setup, which takes
up space in the code segment of the executable. While the closure
based approach chosen here, consumes data segment (or heap) space
per instance for the functors (or function pointers) representing
the closures, plus code segment space for the closures, but the
latter with a way higher potential for inlining, since the closure
code and the generated virtual functions are necessarily emitted
within the same compilation unit and within a local (inline, not
publickly exposed) scope.
so yes, it is complicated, and inevitably involves three layers
of indirection. The alternative seems to bind the GUI direcly to
the Session interface -- is there a middle gound?
For the messages from GUI to Proc, we have our commands, based
on PlacementRef entities. But for feeding model updates to the
GUI, whatever I consider, I end up either with diff messages or
an synchronised access to Session attributes, which ties the
responsiveness of the GUI to the Builder operation.
- we use a GenNode element
- this holds a polymorphic value known as DataCap
- besides simple attribute values, this may hold collections of GenNode sub elements
- a special kind of GenNode collection, the Record, is used to represent objects
The purpose of this setup is to enable an external model representation
which is only loosely coupled to the interndal data representation
through the exchange of (tree)diff messages
previously this operation was named 'attach', which an be confused
with attching an object to this location. Indeed, the session interface
even offers such an attach function. By renaming the focus moving
operation into QueryFocus::shift(Scope), this ambiguity is resolved
This is the first step towards a generic backbone to connect
any GUI elements to the session within Proc-Layer.
It is based on a spefic understanding of Model-View-Controller,
which turns the Model-Controller interactions into messages.
sans the implementation of the index lookup table(s)
The algorithm is KISS, a variant of insertion sort, i.e.
worst time quadratic, but known to perform well on small data sets.
The mere generation of the diff description is O(n log n), since
we do not verify that we can "find" out of order elements. We leave
this to the consumer of the diff, which at this point has to scan
into the rest of the data sequence (leading to quadratic complexity)
finally....
The problem is that the C++ "dependent types" defeat the typical
DSL usage, where you define some helper function in a generic
language setup class and mix this language in as superclass.
This is, C++ requires us to refer explicitly to any dependent type,
since, due to possible template specialisations, the parser
can't know if a given symbol is a inherited type or a field.
As a solution, we place the token constructor functors into a
static struct "token", which allows to write e.g. token.insert(xyz)
As decided in beb57cde
this changeset switches our basic list diff language to work
in the style of an insertion sort. Rather than 'pushing back'
out-of-order elements, we scan and bring forward missing elements.
Later, when passing the original location of the elements
fetched this way, a 'skip' verb will help to clean up
possible leftowers, so implementation is possible
(and indeed acomplished) without shifting any other elements.
and this adds a twist: conceptually, we identify the token
with the abstract handler function it represents. But C++
does not allow us to compare member pointers to virtual functions,
for good reason: even two pointers with the "same offset" into
the VTable might end up referring to different implementations,
when bound to instances of different subclasses. This is what
polymorphism is all about.
At this point it seems reasonably, albeit a bit uggly, to use the
diagnostic ID as placeholder instead, and just compare these IDs
instead. We assume that in practice tokens will be defined through
the provided helper macro, which ensures unique identifiers.
basically just a function to pick up the container and element type automatically.
The actual implementation is delegated to the exisiting lib::iter_stl::IterSnapshot
Heureka! found out that the C++ standard library exposes a
cross vendor C++ ABI, which amongst others allows to show
object code names and type-IDs in the language-level, human
readable unmangeld form.
Of course, actual application code should not rely on such a
internal representation, yet it is of tremendous help when
writing and debugging unit tests.
Signed-off-by: Ichthyostega <prg@ichthyostega.de>
Actually I arried at the conclusion, that the *receiving* of
a diff representation is actually a typical double-dispatch situation.
This leads to the attempt to come up with a specialised visitor
as standard pattern to handle and apply a diff. Obviously,
we do not want the classical GoF-Visitor, but (yes, we had
that discussion allready) -- well in terms of runtime cost,
we have to deal with at least two indirections anyway;
so now I'm exploring the idea to implement one of these
indirections through a functor object, which at the same time
acts as "Tag" in the diff representation language (instead
of using an enum as tag)
Uniform sequence at start of source files
- copyright claim
- license
- file comment
- header guard
- lumiera includes
- library / system includes
Lumiera uses Brittish spelling. Add an according note to the styleguide.
initial considerations; there is a concurrency problem, since
all of session handling within Proc is deliberately not threadsafe.
Thus the decision is to make this the gui::model::SessionFacade's responsibility
The actual problem is not resolved; the pluginloader
should detect the duplicate and not add the handle
to the database initially. Or it should add it
as "duplicate" or "alternate implementation"
Which probably means we need to coder some additional
corner cases. But certainly not now, we have other
more important stuff to do first... we've already
lost the battle against Duke Nukem Forever :-P
Mark parts of the timeline state handling which will certainly
not be retained: any part where the GUI widgets "hold" some kind
of model. GUI widgets shall be *mapped upon* a model representation
and *wired* with callbacks.
Especially I am suspicious when GUI presentation code "reaches into"
any kind of model data structure to find out something. It should
be the other way round (dont call us, we call you)
actually we should make our timeline a real custom widget,
and do it according to the letter. I.e. really implement
all those callbacks which are recommended, but no other
callbacks.
This has the additional benefit of being able to retrieve
the drawing style in the official way, and define our own
CSS classes, which can be styled by the user in a systematic way.
This is not really a solution, but kind of narrowes down the problem.
Our GUI uses an obsolete C-ish approach at releasing resources at
several points. This is probably a left-over from earlier days.
Especially since we started out with libGDL without C++ wrappers.
And at that time, we didn't use smart-pointers, as we should do,
but we tried to do things manually, which is an approach which never
works in an event driven and condition based environment. Goto fail.
Here I just commented out the manual clean-up code from several dtors.
The real solution would be not to allocate these resources through
the raw C calls at first place, but rather use the mm-wrappers
and leave it to them to unwind at the right moment.
TODO:
- scan the GUI code for *every* instance where we still muck around with gobjects
and either replace that by a mm-wrapper, or wrap it in a smart handle.
- make sure that *all* dtors are either empty, or really airtight and EX_FREE
doh...
this happens when you draft some quite intricate logic and then
get sidelined with other tasks for several years. Mind me, I didn't
even recall that I had treated this whole issue and created
a clean-up thread.
A full fledged implementation would have a real lifecycle and
thus detect the re-entrance; but since none of the components
to be managed by the OutputDirector is even remotely planned
or even coded, the functions were just drafted as stubs.
Which caused us happily to create yet another clean-up thread
whenever the subsystem-runner signalled "please shut down".
Our GUI shutdown logic looks rather confused. Why the hell do
some widgets "unregister" themselves in a dtor. This should never
be necessary. Maybe it's a leftover from C-style programming
and obsolete now, after the switch to GDLmm
the problem seems to be the interplay with the installed user
theme; what looks well with one theme is messed up with another
one, quite insidious.
Attempt to settle down on a set of default definitions
for borders and element background colors, which make the
Lumiera Gui "dark" but still respect the user's theme
for geometry and widget style
verified: basically works
todo: better handling of parse errors.
Currently this is treated as an unexpected exception and just
terminates the whole application, without any suitable diagnostics.
This makes working on the stylesheet somewhat brittle. GTK-3 actually
offers a signal to be invoked in case of CSS parsing errors
(see #953)
Comment out the active part of the GdkDisplayer implementation,
but retain the class, to make compilation pass.
With the Switch to GTK-3, only Cairo drawing is supported.
We need a new solution for video display...
Lots off commented out blocks of code
but most issues are related to simple function name changes,
set/get_flags calls, anything that has to do with a Gtk::Style...
Plan of attack from here is to go one-by one of each commented-out or code and update to gtk3 specs.
trying to track down where these messages
GLib-GObject-CRITICAL **: g_object_unref: assertion G_IS_OBJECT (object) failed
are coming from. These appear when iconifying panels.
as always, it turned out that the alledged "compiler bug"
rather was my own sloppyness: I forgot properly to undo a change
I made while fighting with compilation problems: the wrapper in
the factory didn't use std::forward, resulting in a plain flat
slicing copy. This, rightfully, triggered the assertion in the
session query resolver (since a sliced Goal can not be dynamic cast
to a specific Query subclass).
TODO: the toolfactory needs a redesign anyway,
this was just placeholder code added in a very early
state of the Lumiera project. We have way better memory
managing facilities at hand now
the use of a custom finisihing functor, which is applied
to any generated product. This can be used for registration,
memory management or similar framework aspects
Implement the first simple usage scenario for the
unified MultiFact template, using variadic templates.
NOTE:
- the obvious solution based on std::forward
triggers strange behaviour in GCC-4.7
- the inline lambda in the test case traps the
CLang-3.0 parster with a segfault. Horay!
...but the whole design looks still overengineered. See #388
- should get rid of the explicit specialisation
- always use a function signature and thus have arguments?
- why inheriting from the wrapper?
still puzzled why this instantiation of MultiFact fails to compile with GCC 4.8
so I'm bound to understand why the types involved
need indeed to be are structured the way they are right now.
previous versions used to resolve this ambiguity in favour of a ctor call,
but now the compiler treats such constructs as function definition;
this is reasonable, since C++11 introduced the notion of a "generalised
initialisation", which is always written as a (possibly empty) list
in braces.
In these specific cases here, we just omit the empty parens
Since we have now a generic bridge to supply std::hash functions based on
an exisiting boost::hash function, we removed the explicit std::hash
specialisation for "Placement-ID".
This caused the PlacementIndex implementation to fail at compilation
with a quite obscure error, which in fact seems to be caused by the
absence of any specific specialisation. The symptom is that a iterator
range could not be assigned to the predefined iterator type of
std::unordered_multimap, due to a mismatch in the embedded traits type
__umap_traits<__cache_default<_Key, _Hash>::value>>
While I didn't track down that problem entirely, to verify my hypothesis,
the problem can be avoided by using the default -- which is now to
pick up an existing boost::hash function for this type and use this
to generate the std::hash function.
right now we have to defeat an unfortunate static assertion in
the standard library, which is expected to go away in the future.
We use a hack to hijack the problematic definition with the preprocessor,
which requires our header to be first.
NOTE: this header contains a potentially dangerous, temporary workaround
to defeat the static assertion in the default implementation of std::hash,
as shipped with GCC 4.7.x
This assertion turns out to be detrimental all kinds of metaprogramming
based solutions, since it defeats SFINAE. It is expected to be removed
in GCC 4.8
the rules are:
- our own headers go before any library headers
- all headers need to be spelled relative to include root
- ensure that gtk is always included via gui/gtk-base.hpp
c++11 uses another hashtable implementation.
This uncovered some poorly written tests, which relied on
objects being returned in a specific order. As far as poissible,
we're using generic query functions now to get our test objects.
But these tests still rely on a specifically crafted test index content,
which as such is acceptable IMHO. The only remaining problem is
that we check the order of generated output in some tests, and this
order is still implementation dependent.
a real fix would be to rewrite the test to collect the retrieved
values and do a structural verification of the results. This
would mean to write a lot of code for such a marginal topic,
which was implemented just for sake of completenes anyway.
Hopefully my lack of "motivation" doesn't backfire eventually ;-)
Conversion means automatic conversion. In our case,
what we need ist the ability to *construct* a bool from
our (function) object -- while functors aren't automatically
convertible to bool. Thus we use one of the new predicates
from <type_traits>
...uncovered by switching to c++11
When invoking an individual test, we used to erase
the 0-th cmdline argument, which happens to be allways
the name of the test being invoked. Yet none of our
tests actually complied to that contract. Rather,
all tests taking arguments access them by 1-based
argument index. Previously, the argument values just
happened to be still in memory at the original location
after erasing the 0st element.
"Fixed" that by changing the contract. Now, the 0th argument
remains in place, but when there are no additional arguments,
the whole cmdline is cleared.
This is messy, but the test runer needs to be rewritten
entirely, the whole API is clumsy and dangerous. Ticket #289
* use a development snapshot of lib SigC including the recent C++11 adaptations
* never include whole namespaces. Here we got a clash between std::bind and sigc::bind
* use lambdas
* to make the binding code more readable
* to take the nested invocations apart, which resolves the return type ambiguity
In the November developer meeting, Christian and I agreed that
it's best to remove that offending LUID specifications altogether.
Those embedded LUIDs where one of the issues blocking the transition to C++11
This is a partial and preliminary fix; we had an occasional
numeric overflow on 32bit platforms in some tests.
The complete fix will be to introduce a typedef and then
rework the relevant APIs (which are preliminary anyway,
thus no urge right now)
our front-end for boost::format, the class lib::_Fmt
was lacking an reliable specialisation for long and ulong.
This is due to the notorious problem of these types being
of platform dependant size. As a fix, we're speclialising
explicitly for int16_t, int32_t and int64_t and avoid the
common names 'short', 'int' and 'long' alltogether.
And especially for non-64bit-platform (NONPORTABLE)
we add an explicit specialisation for long
The recommendation is to use the link flag --no-undefined
and to fed *all* dependencies to the respective link step.
This changeset enables this strict linking of dependencies.
It turned out that our dependencies were already sane
(with the sole exception of a direct dependency to X-Lib
in the XV viewer widget)
- upgrade the configuration to a current version
- provide a frontpage with cross-links to other documentation
- define a set of modules; relevant classes and files can be
added to these, to create a exploration path for new readers
- fix a lot of errors in documentation comments
- use a custom configuration for the documentation pages
- tweak the navigation, the sections and further arrangements
to make them stand out more prominently, some entity comments
where started with a line of starts. Unfortunately, doxygen
(and javadoc) only recogise comments which are started exactly
with /**
This caused quite some comments to be ignored by doxygen.
Credits to Hendrik Boom for spotting this problem!
A workaround is to end the line of stars with *//**
When a ctor throws, the dtors of sub-objects have already been
invoked. The object itself never existed, strictly speaking,
and thus the dtor must not be invoked. Usually the runtime system
handles matters automatically this way, but since we're doing
here placement new into an array, we're responsible ourselves
This error was uncovered by compiling with Clang.
GCC automatically neutralised this erroneous dtor invocation.
This removes the central clean-up registry;
Instead, now the InstanceHolder manages the lifecycle of
the service instances placed into static memory; the net effect
is that DependencyFactory and instances are created and destroyed
together, locally for each usage scope
We don't need this ability and it pushes us into using a
central registry. This solution turned out to be problematic
when loading dynamic libraries (plug-ins).
this check may look weird, but in fact a similar check in the
old version of the singleton factory helped us spot a problem
with Clang, most likely but of the compiler or runtime system
Clang doesn't allow to declare a private nested class as friend.
This is unfortunate, but likely correct to the letter of the standard.
As a workaround, now we're creating the instances within a static
function of DependencyFactory -- in the end this improves readability
A second issue fixed with this changeset is the scope of the
marker function. Clang is right, this isn't ADL, thus an inline
friend definition is simply not visible outside the class.
lib::Depend<TY> works as drop-in replacement for lib::Singleton<TY>
This changeset removes the convoluted special cases like
SingletonSub and MockInjector.
explanation: we use pthread_once to define a mutex type descriptor,
used to define some of our mutexes as recursive mutexes. Now,
pthread_once relies on a counter stored in a given location;
we used a non-exported global var for this counter.
Unfortunately this ties the mutex initialisation to the static
initialisation of the compilation unit holding this counter variable.
Theoretically it would be possible (we never observed such an incident)
that, during static initialisation, a singleton was brought up,
which requires a class-scoped lock, implemented as recursive mutex.
And it would be possible for this singleton locking to happen prior
to initialisation of the mentioned counter variable.
As a fix, I've moved the counter varialbe into a function scoped
static variable, since that is guaranteed by the C++ runtime system
to be initialised at first usage of the function, irrespective of the
initialisation order of the enclosing compilation units
clang-3.2 requires a clarification here (while previous versions
of clang and GCC automatically resolved the ambiguity by assuming
use of a nested, dependent template).
Clang is more insistent when it comes to enforcing 'protected' visibility.
Since in this case the basic design can be considered sane and optimal, the
only (and obvious) solution is to nest the PIMPL into a default base class
for implementation; this mirrors the structure of the interface.
Compilation with Clang 3.0 (which is available in Debian/stable) fails,
mostly due to some scoping and naming inconsistencies which weren't detected
by GCC. At some instances, Clang seems to have problems to figure out a
perfectly valid type definition; these can be resolved by more explicit
typing (which is preferrable anyway)
by providing a custom copy function one can adjust otherwise non-copyable
elements. This should be used cautionary because dereferencing elements may
poison the cache and thus have some considerable performance impact
(profile this)
using our util::_Fmt front-end helps to reduce the code size,
since all usages rely on a single inclusion of boost::format
including boost::format via header can cause quite some code bloat
NOTE: partial solution, still some further includes to reorganise
this draft fills in the structure how to get from an invocation
of the engine service to the starting of actual CalcStream instances.
Basically the EngineService implementation is repsonsile to
instruct the Segmentation to provide a suitable Dispatcher.
This is necessary since the implementation of the job functions
calls through the VTable of the interface JobClosure. Thus this
interface (and the VTable definition) needs to reside within
some compilation unit linked together with the basic job class.
TODO: move class Job entirely into the Backend
now builds for me on Debian-7 Wheezy 64bit
unqualified member functions in dependent base classes not found anymore.
Need to qualify either the class or the instance.
...for the very specific situation when we want
to explore an existing data structure, and the
exploration assumes value semantics.
The workaround then is to use pointers as values.
...attempt to build it based on the monadic iterator primitives.
Only problem is: need to find out relation between nodes
after the fact. In the real usage situation, this
is not a problem, since we have a state object
there, which can track the relation as it is established
basically I've changed my mind to prefer an
infinite JobPlanningSequence, which is just
evaluated partially. This removes the need to
embody the logic of planning chunk generation,
which really is a different concern.
decision: the base for any deadline calculations
is the expected real time corresponding to the grid origin.
This value is contained in the Timings record.
this clarifies the relation of TimeAnchor and Timings,
the latter act as a general spec and abstracted grid,
while the latter actually performs the conversion and
deadline checking
Rationale: this is the *support* Library.
The real "Lumiera-Library" does not exist yet.
liblumiera.so will be the *interface* every external
module / plug-in uses to get Lumiera functionality.
Especially the work on Library dependency clean-up
made outright clear, that this interface library
needs to be a separate piece of software, which is
carefully crafted, and more-or-less depends on the
whole application.
the buildsystem will now pick up and link
all test cases according to the layer, e.g.
backend tests will automatically be linked
against the backend + library solely.
tests used to be defined ad hoc and test definitions
are scattered confusingly over various directories.
Now built some simple rules into the buildsystem
to allow organising the tests into layers and
linking them accordingly.
Note: this switches to building shared objects
for the test classes too, which effectively speeds up
both re-building and re-running of test cases
Our libraries constitute a clear dependency hierarchy,
we do not want circular dependencies. Declaring these
dependencies while creating the shared libraries would
allow strict checking by the linker; but unfortunately
this also creates transitive depdendencies stored as
DT_NEEDED tags.
While basically this would be just fine, the resolution of $ORIGIN
on gets confused in case of transitively defined library dependencies
over multiple hops, especially in case when actually no symbol of this
transitive dependency is used. Since these newer systems set the
--as-needed switch for linking by default, these unnecessary
DT_NEEDED entries will be purged from the executable, but of course
not from the shared library causing the transitive dependencies.
As a consequence, when loading the executable, the $ORIGIN resolution
mechanism doesn't act on the dependencies recorded in the library,
causing the shared loader to abort with an "unresolved dependency"
So the resolution for these problems is not to use transitive
dependencies on libraries intended to be found via $ORIGIN
This is kind of a workaround to avoid having to maintain two variants.
Explanation: between Boost 1.42 and 1.52 there was the transition to a
reworked version of the filesystem library, itroducing some breaking changes
The new version distinguishes much clearer between the native and the
generic representation of paths -- which becomes relevant when porting
to non-POXIX operating systems.
Actually the intention was to use the generic path representation in all
configuration; currently this distinction is moot, since we're caring
only for POSIX systems.
So the workaround is to use the fsys::path::string() function, which
is available in both versions, but changed meaning to yield the native
string. Later, when able to deprecate older Boost versions, we should
switch to generic_string()
Note: an alternative solution was found by Mike Fisher in 3b39f35
using the compiletime define BOOST_FILESYSTEM_VERSION=2
See also ticket #896
now this library doesn't refer to any symbols from
Proc-Layer anymore. Resolving these problems
highlighted IMHO a serious shortcomming of our
interface system, which hinders the building
of abstractions at interface level
there is now a mechanism to allow sprcialised queries
to generate this syntactic representation only on demand
The actual concrete representation e.g. for scope queries
still remains TODO, but this won't really change
until we target the integration of a real resoloution engine
while refactoring, I thought it might be a good idea
only to use Query objects. But in this special case,
most often you'd just want to pass in a simple query
with a literal query string. So this convenience shortcut
indeed makes sense.
...to extract the syntetic ordering from
DefsRegistry and make that a responsibility
of the (internal) syntactic representation
of the query.
doesn't pass the compiler yet
effectively this joins the two existing lines
of "Query" classes into one systematic representation
Next step would be to move all mutation operations
over to the Query::Builder
time handling is part of the library, while this
convenience shortcut relies on the Advice system,
which resides in the application lib.
To allow this kind of symbolic acces to a grid
entity defined "elesewhere", client code needs
to be linked against liblumieracore.so
especially this allows to use the Advice system
or the query resolvers from within library facilities
to refer to other implementation level services by name
the rules-based configuration and query system
will be located within the core application,
while the concrete implementation facilities
are expected to reside within the session or
maybe also the GUI.
This is kind of a 'rochade' refactoring to resolve
circular library dependencies and confine the parts
dependant on the session and MObjects to the Proc-Layer
And while we're in the middle of chainsaw surgery,
we'll concentrate further query-based facilities
alongside the config-rules within the App core.
This template was a leftover from the early days
of Lumiera development and doesn't provide any
substantial value as an abstraction.
For the more intricate cases, we're using the
lib::MultiFact template, which allows to install
several "fabrication" functions at runtime
the solution is to introduce a superinterface
and let Dispatcher augment that with the specific parts.
This way, the Job planning only has to rely on the
rather generic stuff (TimeAnchor, FrameCoord)
NOTE: this commit makes the whole JobPlanning machinery
compilable for the first time!
..the Idea is to rely on some kind of service,
to break the cyclic dependency with the Dispatcher.
But I seem unable to find a natural location or
concept to house that service.
basically we had two lines of doubly nested capsules, due to
using the IterAdaptor template. Actually, the evaluation stack
within JobTicket can be considered an implementation detail and
thus doesn't require an iterator interface; the intention is to
use this through JobPlanning solely.
Thus this reworking removes the special iterator within JobTicket,
but retains the idea of exposing the "current" JobTicket through
a smart pointer or operator->()
work done during the FrOSCon travel
especially the exploration stack is pushed down
first successful definition of all the JobPlanning classes
just the framework of classes necessary to pass the compiler;
all implementation is still stubbed
brainstorming how to implement the job planning stage
the idea is to built on top of the IterExplorer,
but have the "stack" of re-evaluation integrated
into a custom type, which exploits the static
node network structure to avoid heap allocations
solution idea: again use a builder function?
the template _Fun started as an internal helper
for function-closure, but seems to be of
general use. Thus move it into meta/function.hpp
(function-closure.hpp is heavyweight)
this enables expansion of a (functional) data structure
until exhaustion -- which is what we need to
build job functors by traversing and expanding
an arbitrarily nested job definition structure
the intention is to use this to simplify
generating render jobs based on the elaborated
dependency network of the render nodes. The key
challenge is to overcome the necessity to
store partially done evaluations as
continuation
the tricky part seems to be how to combine the
source iterators into a new monad instance, while
keeping this "Combinator" Strategy configurable
...just passes the compiler, while still lacking
even the generic implementation of joining
together the source iterators
The idea is to avoid building a data structure
for intermediary results, while still being able
to process a variably sized and arbitrary shaped
set of source data
without any trickery, we'll always get two indirections.
Thus, the decision is to turn the 2nd indirection
into a VTable call; this way, basically the JobClosure
also acts as job functor itself.
implemented as extension to the linear combinations.
I decided to use the same "always floor" rule
as employed for time quantisation. Moreover,
we don't support floating point, only rationals
funny enough this possible memory corruption
didn't happen in the unit test, because my
compiler optimised the additional int field
of class SubDummy, making it the same size
of the baseclass. Now matters should be safe.
this draft is based on
- Cehteh's draft for the scheduler
- my plannings about segmentation and JobTicket
it defines "Job" as a closure which can be invoked
from plain-C, using the information in the
job descriptor datastructure
also touches the question how to represent the job
descriptor datastructure. @Cehteh: I've just pasted
in your preliminary data struct definitinons
from the relevant mailing list discussions.
instead of (ab)using the Timings spect for a
runtime switch, better use the existing
MockInjector facility and thus turn the
mock engine mode into a global switch
I think it's smart to rather use ALSA directly instead of PortAudio.
ALSA is push AFAIK, and talking about it here at the hackspace, seems
like the better choice. It's a bit lower level, but anyway everything
speaks ALSA anyway. It's not like there's any reason to use PortAudio
at all. It's just an extra abstraction.
Coding for ALSA it'll also work with Pulseaudio and esd. Do people
really use other sound systems than Pulseaudio, esd or plain ALSA?
I can't think of it.
I really the idea about building a small tool first. I'll do that.
Also thought about making a small blikning cursor/text output, and
syncing a BEEP-sound to that, so that I can test around with throwing
in lots and lots of latency between "me" and the video, and try to
sync it anyway.
I should be able to read back from the sound card (or pulse audio
underneath, it will just work with alsa as the abstraction) how long
it takes for the bytes I'm pushing to reach the speakers, and do some
buffer tuning on that.
The goal is to make the build scrips more clear
at first sight. So move the main build targets
into a separate SConscript, to make them similar
to the tests, research and tools.
The final goal of this makeover is to reduce
the main SConstruct as much as possible
this turns out to be the typical usage scenario
for ScopedCollection: a manager object owning
the collection will populate it with specially
crafted components and invoke a member function
for creating the individual components.
This shortcut avoids using tr1::bind alltogether
some basic types are passed on directly;
for those, we use explicit specialisations
in the implementation file, and a traits template
to mark those cases in the header.
custom types with custom string conversion will
be converted to string; everything else
just becomes a type id
Meanwhile, BuffHanle became way more concrete,
making the separation of any kind of buffer or channel
type management concievable.
Thus: extract the obsolete ChannelDescriptor
and use switch at any location where the old
(superseeded) buffer handle is still referred
instead of using a vector directy, move the
additionally required functinoality in the
utility class. This forced me to adjust
the IterAdapter, to allow working with
STL algorithms
