the plan is to use this specific diff sequence
both in the individual binding tests, and in a
more high level integration test. Hopefully this
helps to make these quite technical tests more readable
to summarise, it turned out that it is impossible to
provide an airtight 'emptySrc' implementation when binding
to object fields -- so we distinguish into positive and
negative tests, allowing to loosen the sanity check
only for the latter ones when binding to object fields.
..as concluded from the preceding analysis.
NOTE this entails a semantical change, since this
predicate is now only meant to be indicative, not conclusive
remarks: the actual implementation of the diff application process
as bound via the TreeMutator remains yet to be written...
how can ordinary object fields be treated as "Attributes"
and thus tied into the Diff framework defined thus far.
This turns out to be really tricky, even questionable
...basically this worked right away and was easy to put together.
However, when considering how many components, indirections and
nested lambdas are working together here, I feel a bit dizzy...
:-/
...all of this implementation boils down to slightly adjusting
the code written for the test-mutation-target. Insofar it pays off now
having implemented this diagnostic and demonstration first.
Moreover I'm implementing this basic scheme of "diff application"
roughly the fourth time, thus things kindof fall into place now.
What's really hard is all those layers of abstraction in between.
Lesson learned (after being off for three weeks, due to LAC and
other obligations): I really need to document the meaning of the
closures, and I need to document the "abstract operational semantics"
of diff application, otherwise no one will be able to provide
the correct closures.
while I still keep my stance not to allow reflection and
switch-on-type, access to the internal / semantic type of
an embedded record seems a valid compromise to allow
to deal with collections of object-like children
of mixed kind.
Indirectly (and quite intentional) this also opens a loophole
to detect if a given GenNode might constitute a nested scope,
but with the for the actual nested element indeed to cary
a type symbol. Effectively this limits the use of this shortcut
to situations where the handling context does have some pre-established
knowledge about what types *might* be expected. This is precisely
the kind of constraint I intend to uphold: I do not want the
false notion of "total flexibility", as is conveyed by introspection.
I still feel somewhat queasy with this whole situation!
We need to return the product of the DSL/Builder by value,
but we also want to swap away the current contents before
starting the mutation, and we do not want a stateful lifecycle
for the mutator implementation. Which means, we need to swap
right at construction, and then we copy -- TADAAA!
Thus I'm going for the solution to disallow copying of the
mutator, yet to allow moving, and to change the builder
to move its product into place. Probably should even push
this policy up into the base class (TreeMutator) to set
everyone straight.
Looks like this didn't show up with the test dummy implementation
just because in this case the src buffer also lived within th
TestMutationTarget, which is assumed to sit where it is, so
effectively we moved around only pointers.
the collection binding can be configured with various
lambdas to supply the basic building blocks of the generated binding.
Since we allow picking up basically anything (functors,
function pointers, function objects, lamdas), and since
we speculate on inlining optimisation of lambdas, we can not
enforce a specific signature in the builder functions.
But at least we can static_assert on the effective signature
at the point where we're generating the actual binding configuration
...but does not compile, since all of the fallback functions
will be instantiated, even while in fact we're overriding them
right away with something that *can* be compiled.
this prompts me to reconsider and question the basic approach
with closures for binding, while in fact what I am doing here
is to implement an ABC.
- the test will use some really private data types,
valid only within the scope of the test function.
- invoking the builder for real got me into problems
with the aggregate initialisation I'd used.
Maybe it's the function pointers? Anyway, working
around that by definint a telescope ctor
the first part of the unit test (now passing)
is able to demonstrate the full set of diff operations
just by binding to a TestMutationTarget.
Now, after verifying the design of those primmitive operations,
we can now proceed with bindings to "real" data structures
when implementing the assignment and mutation primitives
it became clear that the original approach of just storing
a log or string rendered elements does not work: for
assignment, we need to locate an element by ID
now the full API for the "mutation primitives" is shaped.
Of course the actual implementation is missing, but that
should be low hanging fuit by now.
What still requires some thinking though is how to implement
the selector, so we'll actually get a onion shaped decorator
...basically we've now the list mutation primitives working,
albeit in a test/dummy implementation only. Next steps will
be to integrate the assignment and sub scope primitives,
and then to re-do the same implementation respectively
for the case of mutating a standard collection of arbitrary type
what's problematic is that we leave back waste in the
internal buffer holding the source. Thus it doesn't make
sense to check if this buffer is empty. Rather the
Mutator must offer an predicate emptySrc().
This will be relevant for other implementations as well
while the original name, 'replace', conveys the intention,
this more standard name 'swap' reveals what is done
and thus opens a wider array of possible usage
now this feels like making progress again,
even when just writing stubs ;-)
Moreover, it became clear that the "typing" of typed child collections
will always be ad hoc, and thus needs to be ensured on a case by case
base. As a consequence, all mutation primitives must carry the
necessary information for the internal selector to decide if this
primitive is applicable to a given decorator layer. Because
otherwise it is not possible to uphold the concept of a single,
abstracted "source position", where in fact each typed sub-collection
of children (and thus each "onion layer" in the decorator chain)
maintains its own private position
after sleeping one night over the problem, this seems to be
the most natural solution, since the possibility of assignment
naturally arises from the fact that, for tree diff, we have
to distinguish between the *identity* of an element node and
its payload (which could be recursive). Thus, IFF the payoad
is an assignable value, why not allow to assign it. Doing so
elegnatly solves the problem with assignment of attributes
Signed-off-by: Ichthyostega <prg@ichthyostega.de>
the values.child() call would also do a bounds check,
but only to rise a error::Invalid "index out of bounds".
So now we generate a clear message to indicate that
actually a runtime-checked type mismatch caused this problem
incidentally, this uncovered yet another unwanted narrowing conversion,
namely from double via gavl_time_t to TimeValue or alternatively
from double via FSecs (= rational<long>) to Duration.
As in all the previos cases, actually the compiler is to blame,
and GCC-5 is known to get that one right, i.e. let the SFINAE fail
instead of passing it with a "narrowing conversion" warning.
Note: the real test for command binding with immutable types
can be found in BusTerm_test
the rationale is that I deliberately do not want to provide
a mechanism to iterate "over all contents in stringified form".
Because this could be seen as an invitation to process GenNode-
datastructures in an imperative way. Please recall we do not
want that. Users shall either *match* contents (using a visitor),
or they are required to know the type of the contents beforehand.
Both cases favour structural and type based programming over
dynamic run-time based inspection of contents
The actual task prompting me to add this iteration mechanism
is that I want to build a diagnostic, which allows to verify
that a binding message was sent over the bus with some
specific parameter values.
...also for the existing variant, which packages an
arbitrary number of arguments in stringified form
into a given container type. Moreover, the new
form of stringify allows to write util::join
in a clearer way, eliminating the lambda.
very similar to boost::irange, but without heavyweight boost
includes, and moreover based on our Lumiera Forward Iterator concept
Such a inline-range construct makes writing simple tests easy
we made double use of our Tuple type, not only as a
generic record, but also as a metaprogramming helper.
This changeset replaces these helpers with other
metafunctions available for our typelists or type sequences
(with the exception of code directly related to Tuple itself,
since the intention is to delete this code alltogether shortly)
- replace remaining usages of typeid(T).name()
- add another type simplification to handle the STL map allocator
- clean-up usage in lib/format-string
- complete the unit tests
- fix some more bugs
This clean-up action for Ticket #985 started out as search
for a lightweight generic solution. What is left from this
search now, after including the actual utility code into
our support library, might serve to document this new
feature for later referral
over time, we got quite a jungle with all those
shome-me-the-type-of helper functions.
Reduced and unified all those into
- typeString : a human readable, slightly simplified full type
- typeSymbol : a single word identifier, extracted lexically from the type
note: this changeset causes a lot of tests to break,
since we're using unmangeled type-IDs pretty much everywhere now.
Beore fixing those, I'll have to implement a better simplification
scheme for the "human readable" type names....
due to the new automatic string conversion in operator<<
the representation of objects has changed occasionally.
I've investigated and verified all those incidents.
...other than intended, the bomb did explode on random occasions,
with an probability of about 4% (when rr >= 96).
Btw, there was also the mistake to throw an heap allocated
object by pointer. Damn Java habits.
- remove unnecessary includes
- expunge all remaining usages of boost::format
- able to leave out the expliti string(elm) in output
- drop various operator<<, since we're now picking up
custom string conversions automatically
- delete diagnostics headers, which are now largely superfluous
- use newer helper functions occasionally
I didn't blindly change any usage of <iostream> though;
sometimes, just using the output streams right away
seems adequate.
- simple function to pick up the mangled type
- pretty-printing is implemented in format-obj.cpp
- also move the demangleCxx()-Function to that location,
it starts to be used for real, outside the test framework
our minimal compiler requirement is gcc-4.9 since the
transition to Debian/Jessie as reference system.
gcc-4.9 is known to treat SFINAE on private fields properly
this is a stripped-down and very leightweight variant
of the well-known enable_if metaprogramming trick.
Providing this standard variant in a header with minimal
dependencies will allow us to phase out boost inclusions
from many further headers. As a plus, our own variant
is written such as to be more conciese in usage
(no "typename" and no acces of an embedded "::type" menber)
...this is necessary whenever the mocked facility covered
by log matching is managed automatically as singleton,
because then other test cases will leave garbage
in the log
I worked under the erroneous assumption, that Doxygen
will use its internal entity-IDs as the link-IDs when
generating mardown-links. Yes, this seemed logical and
this would be the way I'd implement it....
But seemingly, Doxygen is not so consistent when it
comes to questions of syntax. The same holds true for
markdown, which lacking a coherent definition anyway.
Another problem is that Doxygen's auto-link generation
frequently fails, for reasons not yet clear to me.
Sometimes it seems to be necessary to give it a nudge
by including the \ref command. While I'm not willing
to go into focussed invstigation of Doxygen syntax
right now, at least I've done a search-and-replace
to remove the malformed links I've written the
last days
so this turned out to be rather expensive,
while actually not difficult to implement.
On the way, I've learned
- how to build a backtracking matcher, based on
a filtering (monadic) structure and chained lambdas
- learned the hard way how (not) to return a container
by move-reference
- made first contact with the regular expressions
now available from the standard library
abandon the use of an assertion exception to signal match failure,
rather use a final bool conversion to retrieve the results.
Error messages are now delivered by side effect into STDERR
The reason is we're unable to deliver the desisred behaviour
with the chosen DSL syntax in C++ ; on a second thought the
new approach is even better aligned with the overall way
we're writing tests in Lumiera. And we produce match-trace
messages to indicate the complete matching path now
...no need to enclose empty sections when there are no
attributes or no children. Makes test code way more readable.
TestEventLog_test PASS as far as implemented
after looking into our various iterator tools,
it seems obvious that our filtering iterator implementation
has almost all of the required behaviour; we only need to
add a hook to rewrite and extend the filtering functor,
which can now nicely done with a lambda closure.
This means all memory management, if necessary, is
pushed into std::function and the automated memory
management for closures provided by the runtime.
some tests rely on additional diagnostics code being linked in,
which happens, when lib/format-util.hpp is included prior to
the instantiation of lib::diff::Record rsp. lib::Variant.
The reason why i opended this can of worms was to avoid includion
of this formatting and diagnostics code into such basic headers
as lib/variant.hpp or lib/diff/gen-node.hpp
Now it turns out, that on some platforms the linker will use
a later instantiation of lib::Variant::Buff<GenNode>::operator string
in spite of a complete instantiation of this virtual function
being available already in liblumierasupport.so
But the real reason is that -- with this trickery -- we're violating
the single definition rule, so we get what we deserved.
TODO (Ticket #973): at a later point in development we have to re-assess,
the precise impact of including lib/format-util.hpp into
lib/diff/gen-node.hpp
Right now I expect GenNode to be used pervasively, so I am
reluctant to make that header too heavyweight.
yet another instance of that obnoxious problem that "long"
is just 32bit on i386 platforms. Why the hell does such
a broken type get the preference of convenient notation??
well... this was quite a piece of work
Added some documentation, but a complete documentation,
preferably to the website, would be desirable, as would
be a more complete test covering the negative corner cases
while implementing this, I've discovered a conceptual error:
we allow to accept attributes, even when we've already entered
the child scope. This means that we can not predictable get back
at the "last" (i.e. the currently touched) element, because this
might be such an attribute. So a really correct implementation
would have to memorise the "current" element, which is really
tricky, given the various ways of touching elements in our
diff language.
In the end I've decided to ignore this problem (maybe a better
solution would have been to disallow those "late" attributes?)
My reasoning is that attributes are unlikely to be full records,
rather just values, and values are never mutated. (but note
that it is definitively possible to have an record as attribute!)
...while I must admit that I'm a bit doubtful about that
language feature, but it does come in handy when manually
writing diff messages. The reason is the automatic naming
of child objects, which makes it often hard to refer to
a child after the fact, since the name can not be
reconstructed systematically.
Obviously the downside of this "anonymous pick / delete"
is that we allow to pick (accept) or even delete just
any child, which happens to sit there, without being
able to detect a synchronisation mismatch between
sender and receiver.
...so now the stage is set. We can reimplement
the handling of the list diff cases here in the context
of tree diff application. The additional twist of course
being the distinction between attribute and child scope
so basically it's time to explicate the way
our diff language will actually be written.
Similar to the list diff case, it's a linear sequence
of verb tokens, but in this case, the payload value
in each token is a GenNode. This is the very reason
why GenNode was conceived as value object with an
opaque DataCap payload
while it's still not really clear how we'll use this helper
and if we need it at all -- some weeks ago I changed its
semantics to be strictly based on the delta to a reference level.
Now this means, we could go below level zero, but this doesn't
make any sense in the context of navigating a tree. Actually,
our test case triggered this situation, which caused the
reference level to wrap around, since it is stored in an
unsigned variable.
Thus I'll add a precondition to keep the level positive,
and I'll change the test to comply.
initially the intention was to include a "bracketing construct"
into the values returned by the iterator. After considering
the various implementation and representation approaches,
it seems more appropriate just to expose a measure for the
depth-in-tree through the iterator itself, leaving any concerns
about navigation and structure reconstruction to the usage site.
As rationale we consider the full tree reconstruction as a very
specialised use case, and as such the normal "just iteration" usage
should not pay for this in terms of iterator size and implementation
complexity. Once a "level" measure is exposed, the usage site
can do precisely the same, with the help of the
HierarchyOrientationIndicator.
Whooa!
Templates are powerful.
programming this way is really fun.
under the assumption that the parts are logical,
all conceivable combinations of theses parts are bound to be correct
it passes compilation, but the test still fails, since
I've changed the expected semantics of the iteration,
in the light of the insights I've gained during
re-investigation of the IterExplorer.
What I now actually intend is rather to embed a
HierarchyOrientationIndicator into the iterator,
instead of returning a special "bracket" marker
reference to indicate return from a nested scope.
This helper was drafted for the Job / JobPlanning and Scheduler
interface in 2013, but seemingly not yet put into action. While
in the original use case, we have a genuine measuerment for the
tree depth (given by the depth of the processing stack), in other
use cases we want to use to offset embedded within the indicator
itself for keeping track of the depth. Thus I add a second
mark operation, which usess the current offset to set a new
reference level. This has the consequence that the offset
has now to reflect the new reference point immediately
remembered that some years ago I had to deal with a very similar problem
for planning the frame rendering jobs. It turned out, that the
iterator monad developed for this looks promising for our task at hand
this design is rather into the blue,
not sure what we actually need for diff generation
and object serialisation. Anyhow, I considered including
a bracketing construct a good idea, and I considered it
sensible to expose inner nodes, not only the leaf nodes.
Obviously, this is not a real monad iteration then.
horay!
seems like madness?
well -- found and squashed a bug: equality on RecordRef
implicitly converted to GenNode(RecordRef), which always
generates new (distinct) IDs and so never succeeds. What
we really want is equality test on the references
while in debugging, it turned out that the short type-prefix
was implemented in a too simplistic way; it fails on stuff
like 'lib::diff::Record<lib::diff::GenNode>'
while I must add, that the whole purpose of these ID functions
is somewhat unclear and needs to reveal itself as we move forward
...while on the train back from FrOSCon.
still the same old problem: we need a better hash function
for generating our Entry-IDs. The default hash function from Boost performs
poor on strings with common prefix and trailing number.
We use a hackish workaround, which is sufficient to avoid collisions
among the first 10000 numbers.
basically the 32/64bit problem was caused by things like 23L, which creates a long.
Unfortunately on 64bit platforms, this is aliased to int64_t,
while on 32bit i386, it is a distinct data type, but just 32bit,
like int.
The code in question here is just test / demonstration code
and actually just needs "some integer number". So let's stick
to good old boring int then.
not entirely sure about the design, but lets try this approach:
they can be "cloned" and likewise move-assigned, but we do not
allow the regular assignment, because this would enable to use
references like pointers (what we deliberately do not want)
especially setting (changing) attributes turned out to be tricky,
since in case of a GenNode this would mean to re-bind the hash ID;
we can not possibly do that properly without knowing the type of the payload,
and by design this payload type is opaque (erased).
As resort, I changed the semantics of the assign operation:
now it rather builds a new payload element, with a given initialiser.
In case of the strings, this ends up being the same operation,
while in case of GenNode, this is now something entirely different:
we can now build a new GenNode "in place" of the old one, and both
will have the same symbolic ID (attribute key). Incidentally,
our Variant implementation will reject such a re-building operatinon
when this means to change the (opaque) payload type.
in addition, I created a new API function on the Mutator,
allowing to move-in a complete attribute object. Actually this
new function became the working implementation. This way, it is
still possible to emplace a new attribute efficiently (consider
this to be a whole object graph!). But only, if the key (ID)
embedded in the attribute object is already what is the intended
key for this attribute. This way, we elegantly circumvent the
problem of having to re-bind a hash ID without knowing the type seed
initially, the intention was to inject the type as a magic attribute.
But this turned out to make the implementation brittle, asymmetric
and either quite demanding, or inefficient.
The only sane approach would be to introduce a third collection,
the metadata attributes. Then it would be possible to handle these
automatically, but expose them through the iterator.
In the end I decided against it, just the type attribute
allone does not justify that effort. So now the type is an
special magic field and kept apart from any object data.
Note: not fixing all relevant warnings.
Especially, the "-Woverloaded-virtual" of Clang defeats the whole purpose
of generated generic interfaces. For example, our Variant type is instantiated
with a list of types the variant can hold. Through metaprogramming, this
instantiation generates also an embedded Visitor interface, which has
virtual 'handle(TY)' functions for all the types in question
The client now may implement, or even partially implement this Visitor,
to retrieve specific data out of given Variant instance with unknown conent.
To complain that some other virtual overload is now shaddowed is besides the point,
so we might consider to disable this warning altogether
the object VTable is typically emitted when the compiler
encounters the first non-static non-inline function of
the class or a derived class.
Sometimes this happens within the wrong library and so
the compiler needs a nudge to emit those infrastructure functions.
But in most cases this works out of the box and need no further
magic incanctations, which might have a downside.
Especially because also a non-inline dtor does incur a call overhead,
whereas an inline dtor can be trivially elided.
