Note: not fixing all relevant warnings.
Especially, the "-Woverloaded-virtual" of Clang defeats the whole purpose
of generated generic interfaces. For example, our Variant type is instantiated
with a list of types the variant can hold. Through metaprogramming, this
instantiation generates also an embedded Visitor interface, which has
virtual 'handle(TY)' functions for all the types in question
The client now may implement, or even partially implement this Visitor,
to retrieve specific data out of given Variant instance with unknown conent.
To complain that some other virtual overload is now shaddowed is besides the point,
so we might consider to disable this warning altogether
the object VTable is typically emitted when the compiler
encounters the first non-static non-inline function of
the class or a derived class.
Sometimes this happens within the wrong library and so
the compiler needs a nudge to emit those infrastructure functions.
But in most cases this works out of the box and need no further
magic incanctations, which might have a downside.
Especially because also a non-inline dtor does incur a call overhead,
whereas an inline dtor can be trivially elided.
after sleeping a night over this, it seems obvios
that we do not want to start the build proces "implicitly",
starting from a Record<GenNode>. Rather, we always want
the user to plant a dedicated Mutator object, which then
can remain noncopyable and is passed by reference through
the whole builder chain. Movin innards of *this object*
are moved away a the end of the chain does not pose much risk.
especially I've now decided how to handle const-ness:
We're open to all forms of const-ness, the actual usage decides.
const GenNode will only expose a const& to the data values
still TODO is the object builder notation for diff::Record
I decided to allow for an 'unbound' reference to allow
default construction of elements involving record references.
I am aware of the implications, but I place the focus
on the value nature of GenNode elements; the RecordRef
was introduced only as a means to cary out diff comparisons
and similar computations.
basically this is the well known problem #587
Just it became more pressing with the Upgrade to Jessie and Boost 1.55
So I've pulled off the well known "Knuth trick" to spread the
input data more evenly within the hash domain.
And voilà: now we're able to use 100000 number suffixes without collision
this was introduced into namespace mobject and spread from there.
Since the habit is to use more specific typedefs like PClip,
it is preferrable to spell out the full namespace
these speical reference-flavours of a GenNode are built
to stand-in for a full fledged "object" GenNode.
The purpose is to be able to handle sub-trees of objects
efficiently in comparisions and processing.
This is just a draft for now -- kindof a by-catch, since it is
chep to build that DSL on top of the Rec::Mutator.
This DSL could be of value later, when it comes to define
some configuration data inline, in a copact and clear fashion,
without the need to use a bridge to/from JSON
I had added this variation just to check compilation and
forgot to revert ist. Of course, we do *not* want to move
the inwards of our Mutator in the test. Rather, we want
to draw a copy from the mutated state
- can build from the supported value types
- is optionally named
- is copyable value, but only assignable within one payload type
- is recursive, for object / tree representation
Ouch!
Why does C++ lack the most basic everyday stuff?
It needn't be performant. It needn't support some fancy
higher order container. Just join the f***ing strings.
use Bosst?? -- OMG!! pulls in half the metra programming library
and tries to work on any concievable range like object. Just
somehow our Lumiera Forward Iterators aren't "range-like" enough
for boost's taste.
Thus let's code up that fucking for-loop ourselves, once and forever.
the valgrind memcheck got more and more flooded with
"possibly lost" memory blocks allocated by GLib and friends.
Linking the vgsuppression binary in the same way than the
testsuite helps us getting ahead of all that noise
Incidentally, we need to rearrange the build dependency tree
here; previously we made the testsuite depend on vgsuppression,
to ensure the latter gets recompiled prior to running tests;
now vgsuppression itself depends on all the test libraries,
so we rather need to make it direcly a prerequisite target
of running the testsuite (this approach is not precisely
correct in a logical sense, yet helps us to get it
recompiled when needed)
this is strange; we need to crank up the VSize to 6 GiB
to get the 5 massively multithreaded tests to run.
I checked those tests with valgrind and can not see
any such huge allocation
In Lumiera, "Tracks" are not what you'd expect from
conventional video editing software. They are a mere
grouping devide, and are also used to implement the
"media bins" and tool palettes.
But having "folders" on the timeline would be likewise
confusing, as would be to have a "branch" or "tree".
To get out of that dilemma, we chose an understandable
but deliberately somewhat strange name: "Fork"
It was common understanding on the Mailinglist that we
should handle this renaming in a tuned-down and discrete
way: The UI will continue to show "Tracks" for a familiar
sight and "Bins" in the Asset section. But Lumiera developers
will be nudged to accomodate by renaming the entity in
source code accordingly
Cockoo hashing is a thrilling algorithm.
We investigated it during the time or our first draft
towards a confirugation system in 2008. This usage turned
up some problems -- not sure if based on the implementation
or the algorithm itself; at that time, we just switched
to the probabilistic splay tree. The whole configuration
system effort stalled afterwards; so the cuckoo implementation
remained in tree as a zombie.
This switches the Lumiera UI from GTK-2 to GTK-3
Unfortunately, this move breaks two crucial features, which have been
disabled for now: the display of video and our custom timeline widget.
Since both of these require some reworking, which in fact has already
started, we prefer to do the library and framework switch right away.
The actual trick to make it work is to use decltype on the function operator
http://stackoverflow.com/questions/7943525/is-it-possible-to-figure-out-the-parameter-type-and-return-type-of-a-lambda/7943765#7943765
In addition, we now pick up the functor by template type and
store it under that very type. For one, this cuts the size
of the generated class by a factor of two. And it gives the
compiler the ability to inline a closure as much as is possible,
especially when the created Binder / Mutator lives in the same
reference frame the closure taps into.
to carry out that rather obvious step, I was bound to consider
all the implications of choosing a given layout and handling pattern
for our external structure representation.
Finally, I settled upon the following decisions
- the value space represented within the DataCap is flat, not further structured
- the distinction between "attribute" and "nested object" is merely conceptual
and will be enforced solely by the diff detection / representation protocol
- basically, a nested subtree may appear as an attribute; the difference
between attributes and children lies solely in the way of access and referral:
by-name vs. positional
- it is pointless to save space for the representation of the discriminator ID
- but we can omit any further explicit type tag, because
- we do *not* support programming by switch-on-type, and thus
- we do *not* support full introspection, only a passive type-safety check
- this is *not* a limitation, since we acknowledge that GenNode is a *Monad*
- and the partial function needed within any flatMap implementation
maps naturally onto our Variant-Visitor; thus
- the DataCap can basically just *be* a Variant
- and GenNode has just to supply the neccessary shaffolding
to turn that into a full fledged Monad implementation, including
direct construction by wrapping a value and flatMap with tree walk
All relevant uses will rely on the more strict access policy
implemented with the new util::AccessCasted. Along the same line
of thinking, I've removed the "second try" convenience conversion
from the typed get-Function of OpaqueHolder. Such an unbounded
"convert it somehow" approach is almost never a good idea. Either,
one knows by design the precise type to expect, or alternatively
should rely on the base interface solely.
...with the sole exception of the usage in WrapperPointer,
which in itself looks obsolete to me; we should better re-think
the way we handle "wrapped" objects for the BuilderTools, once
we actually start implementing the Builder
Ticket #450
Note: the new Variant implementation is a re-write from scratch
and does not rely on util::AccessCasted any more. Anyway, both
are now thoroughly covered by unit test
NOTE: this was a one-time verification. Unfortunately there is no way
to verify a failing compilation automatically from a unit-test.
Thus we need to comment out these invalid cases, leaving them
here just for later referral. Need to check those manually
for new compilers to be sure!
...since I consider that a comparatively safe convenience feature.
Of course we *do perform* a NULL check and throw an exception.
So now the actual casting or conversion functions are designed
to work always on the same level of references or pointers,
which means we can just use the standard conversions of the
language. This has the nice effect of ruling out dangerous
combinations (like taking a L-ref from a R-ref) automatically
(extracted from the git history of file try.cpp, May 2008)
basically this is the draft implementation from which
AccessCasted was extracted. I see two problems
- this version prints from within the access functions
- we do not want the automatic static downcast anymore.
meanwhile, I consider this kind of "do everything for me"
programming style as dangerous. If unchecked donwcasts
are desired, then code them up explicitly
finally got all those copy / assgnment flavours straight.
Still unsolved: unable to instantiate the Variant template
for a type with private assignment operator (like e.g. Time )
The problem is our virtual assignement operator, which forces
instantiation of the implementation (for the VTable), even if
the actual assignment is never invoked.
- we do not want type mutations
- we do not want "empty" records ever
- we do not support "probing" for contents
- visitor style access for generic value handling
After some reconsideration, I decide to stick to the approach with the closures,
but to use a metaprotramming technique to build an inheritance chain.
While I can not decide on the real world impact of storing all those closures,
in theory this approach should enable the compiler to remove all of the
storage overhead. Since, when storing the result into an auto variable
right within scope (as demonstrated in the test), the compiler
sees the concrete type and might be able to boil down the actual
generated virtual function implementations, thereby inlining the
given closures.
Whereas, on the other hand, if we'd go the obvious conventional route
and place the closures into a Map allocated on the stack, I wouldn't
expect the compiler to do data flow analysis to prove this allocation
is not necessary and inline it away.
NOTE: there is now guarantee this inlining trick will ever work.
And, moreover, we don't know anything regarding the runtime effect.
The whole picture is way more involved as it might seem at first sight.
Even if we go the completely conventional route and require every
participating object to supply an implementation of some kind of
"Serializable" interface, we'll end up with a (hand written!)
implementation class for each participating setup, which takes
up space in the code segment of the executable. While the closure
based approach chosen here, consumes data segment (or heap) space
per instance for the functors (or function pointers) representing
the closures, plus code segment space for the closures, but the
latter with a way higher potential for inlining, since the closure
code and the generated virtual functions are necessarily emitted
within the same compilation unit and within a local (inline, not
publickly exposed) scope.
so yes, it is complicated, and inevitably involves three layers
of indirection. The alternative seems to bind the GUI direcly to
the Session interface -- is there a middle gound?
For the messages from GUI to Proc, we have our commands, based
on PlacementRef entities. But for feeding model updates to the
GUI, whatever I consider, I end up either with diff messages or
an synchronised access to Session attributes, which ties the
responsiveness of the GUI to the Builder operation.
- we use a GenNode element
- this holds a polymorphic value known as DataCap
- besides simple attribute values, this may hold collections of GenNode sub elements
- a special kind of GenNode collection, the Record, is used to represent objects
The purpose of this setup is to enable an external model representation
which is only loosely coupled to the interndal data representation
through the exchange of (tree)diff messages
previously this operation was named 'attach', which an be confused
with attching an object to this location. Indeed, the session interface
even offers such an attach function. By renaming the focus moving
operation into QueryFocus::shift(Scope), this ambiguity is resolved
This is the first step towards a generic backbone to connect
any GUI elements to the session within Proc-Layer.
It is based on a spefic understanding of Model-View-Controller,
which turns the Model-Controller interactions into messages.
sans the implementation of the index lookup table(s)
The algorithm is KISS, a variant of insertion sort, i.e.
worst time quadratic, but known to perform well on small data sets.
The mere generation of the diff description is O(n log n), since
we do not verify that we can "find" out of order elements. We leave
this to the consumer of the diff, which at this point has to scan
into the rest of the data sequence (leading to quadratic complexity)
finally....
The problem is that the C++ "dependent types" defeat the typical
DSL usage, where you define some helper function in a generic
language setup class and mix this language in as superclass.
This is, C++ requires us to refer explicitly to any dependent type,
since, due to possible template specialisations, the parser
can't know if a given symbol is a inherited type or a field.
As a solution, we place the token constructor functors into a
static struct "token", which allows to write e.g. token.insert(xyz)
As decided in beb57cde
this changeset switches our basic list diff language to work
in the style of an insertion sort. Rather than 'pushing back'
out-of-order elements, we scan and bring forward missing elements.
Later, when passing the original location of the elements
fetched this way, a 'skip' verb will help to clean up
possible leftowers, so implementation is possible
(and indeed acomplished) without shifting any other elements.
we want a simple and straight forward way of defining tokens
of the "diff language". Each token is bound to a specific
handler function in the language interpreter interface.
Problem is that likely we'll get a ListDiffLanguage and a TreeDiffLanguage;
after all, I really don't know yet how far to take this whole
diff representation endeavour...
Basically attempt to represent the individual diff step
as a tuple of "DiffVerb" and reference element.
The meaning of the reference element depends on the actual verb
...first step is to design a generic linearised list diff representation.
Basically just need to pull together the theoretical work of the last weeks.
Next steps will be to extend to typed ordered trees.
Heureka! found out that the C++ standard library exposes a
cross vendor C++ ABI, which amongst others allows to show
object code names and type-IDs in the language-level, human
readable unmangeld form.
Of course, actual application code should not rely on such a
internal representation, yet it is of tremendous help when
writing and debugging unit tests.
Signed-off-by: Ichthyostega <prg@ichthyostega.de>
the idea ist to build some kind of "smart" enum constants,
which allow for double dispatch through a member function pointer,
invoking a virtual function on a common handler interface
Actually I arried at the conclusion, that the *receiving* of
a diff representation is actually a typical double-dispatch situation.
This leads to the attempt to come up with a specialised visitor
as standard pattern to handle and apply a diff. Obviously,
we do not want the classical GoF-Visitor, but (yes, we had
that discussion allready) -- well in terms of runtime cost,
we have to deal with at least two indirections anyway;
so now I'm exploring the idea to implement one of these
indirections through a functor object, which at the same time
acts as "Tag" in the diff representation language (instead
of using an enum as tag)
Uniform sequence at start of source files
- copyright claim
- license
- file comment
- header guard
- lumiera includes
- library / system includes
Lumiera uses Brittish spelling. Add an according note to the styleguide.
initial considerations; there is a concurrency problem, since
all of session handling within Proc is deliberately not threadsafe.
Thus the decision is to make this the gui::model::SessionFacade's responsibility
- the tests covering threadind support and object monitors
are located in the backend test-library and linked against liblumierabackend.so
- some fundamental facilities of proc-layer moved from the library tree
into the basic components tree, since *testing* them requires at least
to link against liblumieracommon.so
here we're iterating hash table based collections, consequently
the order of items retrieved *is* implementation dependent and indeed
differs on different platforms and compilers.
In Clang, static object fields are initialised from top to bottom,
but before any other variables in anoymous namespaces. To the contrary,
GCC evaluates *any* initialisation expression in the translation
unit together from top to bottom. Thus, in the clang generated
code, in two cases the static initialisation could use a not yet
constructed local lib::_Fmt formatter object.
the use of a custom finisihing functor, which is applied
to any generated product. This can be used for registration,
memory management or similar framework aspects
Implement the first simple usage scenario for the
unified MultiFact template, using variadic templates.
NOTE:
- the obvious solution based on std::forward
triggers strange behaviour in GCC-4.7
- the inline lambda in the test case traps the
CLang-3.0 parster with a segfault. Horay!
right now we have to defeat an unfortunate static assertion in
the standard library, which is expected to go away in the future.
We use a hack to hijack the problematic definition with the preprocessor,
which requires our header to be first.
c++11 uses another hashtable implementation.
This uncovered some poorly written tests, which relied on
objects being returned in a specific order. As far as poissible,
we're using generic query functions now to get our test objects.
But these tests still rely on a specifically crafted test index content,
which as such is acceptable IMHO. The only remaining problem is
that we check the order of generated output in some tests, and this
order is still implementation dependent.
This is a notable difference to the boost or tr1-function objects
we used up to now. Thus the behavour is now straight forward without
any exception. If the function takes an argument by reference,
this is replicated through bind and function expressions
a real fix would be to rewrite the test to collect the retrieved
values and do a structural verification of the results. This
would mean to write a lot of code for such a marginal topic,
which was implemented just for sake of completenes anyway.
Hopefully my lack of "motivation" doesn't backfire eventually ;-)
Conversion means automatic conversion. In our case,
what we need ist the ability to *construct* a bool from
our (function) object -- while functors aren't automatically
convertible to bool. Thus we use one of the new predicates
from <type_traits>
...uncovered by switching to c++11
When invoking an individual test, we used to erase
the 0-th cmdline argument, which happens to be allways
the name of the test being invoked. Yet none of our
tests actually complied to that contract. Rather,
all tests taking arguments access them by 1-based
argument index. Previously, the argument values just
happened to be still in memory at the original location
after erasing the 0st element.
"Fixed" that by changing the contract. Now, the 0th argument
remains in place, but when there are no additional arguments,
the whole cmdline is cleared.
This is messy, but the test runer needs to be rewritten
entirely, the whole API is clumsy and dangerous. Ticket #289
In the November developer meeting, Christian and I agreed that
it's best to remove that offending LUID specifications altogether.
Those embedded LUIDs where one of the issues blocking the transition to C++11
this is rather a workaround.
The problem is a wraparound while calculating the common denominator in
Time rawTime (dirt + frames*F25);
Currently we're using boost_rational<long>, and long is only 32bit
on 32bit platforms. The workaround commited here just avoids
the calculation of the fractional value, and adds 64bit time values
instead. But the real solution would be to use a consistent
approach for dealing with frame counts and frame rates, all
based on 64bit values. See Ticket #939
This is a partial and preliminary fix; we had an occasional
numeric overflow on 32bit platforms in some tests.
The complete fix will be to introduce a typedef and then
rework the relevant APIs (which are preliminary anyway,
thus no urge right now)
our front-end for boost::format, the class lib::_Fmt
was lacking an reliable specialisation for long and ulong.
This is due to the notorious problem of these types being
of platform dependant size. As a fix, we're speclialising
explicitly for int16_t, int32_t and int64_t and avoid the
common names 'short', 'int' and 'long' alltogether.
And especially for non-64bit-platform (NONPORTABLE)
we add an explicit specialisation for long
The recommendation is to use the link flag --no-undefined
and to fed *all* dependencies to the respective link step.
This changeset enables this strict linking of dependencies.
It turned out that our dependencies were already sane
(with the sole exception of a direct dependency to X-Lib
in the XV viewer widget)
- upgrade the configuration to a current version
- provide a frontpage with cross-links to other documentation
- define a set of modules; relevant classes and files can be
added to these, to create a exploration path for new readers
- fix a lot of errors in documentation comments
- use a custom configuration for the documentation pages
- tweak the navigation, the sections and further arrangements
to make them stand out more prominently, some entity comments
where started with a line of starts. Unfortunately, doxygen
(and javadoc) only recogise comments which are started exactly
with /**
This caused quite some comments to be ignored by doxygen.
Credits to Hendrik Boom for spotting this problem!
A workaround is to end the line of stars with *//**
We don't need this ability and it pushes us into using a
central registry. This solution turned out to be problematic
when loading dynamic libraries (plug-ins).
Clang doesn't allow to declare a private nested class as friend.
This is unfortunate, but likely correct to the letter of the standard.
As a workaround, now we're creating the instances within a static
function of DependencyFactory -- in the end this improves readability
A second issue fixed with this changeset is the scope of the
marker function. Clang is right, this isn't ADL, thus an inline
friend definition is simply not visible outside the class.
lib::Depend<TY> works as drop-in replacement for lib::Singleton<TY>
This changeset removes the convoluted special cases like
SingletonSub and MockInjector.
Clang seems to evaluate the terms of a function call in another order
than GCC -- this uncovered re-entrance errors in some metaprogramming tests,
where we re-used a global formatter object in recursive instantiations.
Clang is more insistent when it comes to enforcing 'protected' visibility.
Since in this case the basic design can be considered sane and optimal, the
only (and obvious) solution is to nest the PIMPL into a default base class
for implementation; this mirrors the structure of the interface.
Compilation with Clang 3.0 (which is available in Debian/stable) fails,
mostly due to some scoping and naming inconsistencies which weren't detected
by GCC. At some instances, Clang seems to have problems to figure out a
perfectly valid type definition; these can be resolved by more explicit
typing (which is preferrable anyway)
using our util::_Fmt front-end helps to reduce the code size,
since all usages rely on a single inclusion of boost::format
including boost::format via header can cause quite some code bloat
NOTE: partial solution, still some further includes to reorganise
this draft fills in the structure how to get from an invocation
of the engine service to the starting of actual CalcStream instances.
Basically the EngineService implementation is repsonsile to
instruct the Segmentation to provide a suitable Dispatcher.
...this was quite insidious, but most of the problems
were in the test fixture. Treating the root context
on re-creation is something to be carefull though
While this isn't immediately relevant to the problem at hand,
it looks like a sensible idea to be able to explore
an existing data structure by iterators exposing pointers
(instead of reference wrappers).
Generally speaking, reference wrappers would be preferrable,
but, especially when the data structure relies on STL containers,
the default constructed values for resizing rule out
the standard reference wrapper, which can't be default
constructed. Using a custom variant would be equivalent
to using just a plain pointer (since both can be NULL and can be rebound)
...for the very specific situation when we want
to explore an existing data structure, and the
exploration assumes value semantics.
The workaround then is to use pointers as values.
This test setup is intended to emulate the situation
when adding jobs to the scheduler; thus we should use
an implicit sequence as root element.
I.e. we have to treat a wood, not a single tree
Note: test still fails, since we take a copy
of a Node object somewhere inadvertently
...attempt to build it based on the monadic iterator primitives.
Only problem is: need to find out relation between nodes
after the fact. In the real usage situation, this
is not a problem, since we have a state object
there, which can track the relation as it is established
there was the possibility for the random offset added in this test
to add up to a whole frame, which would cause the
re-quantisation to wrap to the next fame (and thus the
CHECK in line 110 to fail.
DispatcherInterface_test now passes the compiler,
meaning that the interfaces are completely defined,
all the generated types are OK and all operations are
at least stubbed.
Replacing all those stubs will be the next step
decision: the base for any deadline calculations
is the expected real time corresponding to the grid origin.
This value is contained in the Timings record.
this clarifies the relation of TimeAnchor and Timings,
the latter act as a general spec and abstracted grid,
while the latter actually performs the conversion and
deadline checking
the buildsystem will now pick up and link
all test cases according to the layer, e.g.
backend tests will automatically be linked
against the backend + library solely.
tests used to be defined ad hoc and test definitions
are scattered confusingly over various directories.
Now built some simple rules into the buildsystem
to allow organising the tests into layers and
linking them accordingly.
Note: this switches to building shared objects
for the test classes too, which effectively speeds up
both re-building and re-running of test cases
while refactoring, I thought it might be a good idea
only to use Query objects. But in this special case,
most often you'd just want to pass in a simple query
with a literal query string. So this convenience shortcut
indeed makes sense.
This template was a leftover from the early days
of Lumiera development and doesn't provide any
substantial value as an abstraction.
For the more intricate cases, we're using the
lib::MultiFact template, which allows to install
several "fabrication" functions at runtime
the solution is to introduce a superinterface
and let Dispatcher augment that with the specific parts.
This way, the Job planning only has to rely on the
rather generic stuff (TimeAnchor, FrameCoord)
NOTE: this commit makes the whole JobPlanning machinery
compilable for the first time!
brainstorming how to implement the job planning stage
the idea is to built on top of the IterExplorer,
but have the "stack" of re-evaluation integrated
into a custom type, which exploits the static
node network structure to avoid heap allocations
solution idea: again use a builder function?
this enables expansion of a (functional) data structure
until exhaustion -- which is what we need to
build job functors by traversing and expanding
an arbitrarily nested job definition structure
the intention is to use this to simplify
generating render jobs based on the elaborated
dependency network of the render nodes. The key
challenge is to overcome the necessity to
store partially done evaluations as
continuation
the tricky part seems to be how to combine the
source iterators into a new monad instance, while
keeping this "Combinator" Strategy configurable
...just passes the compiler, while still lacking
even the generic implementation of joining
together the source iterators
The idea is to avoid building a data structure
for intermediary results, while still being able
to process a variably sized and arbitrary shaped
set of source data
implemented as extension to the linear combinations.
I decided to use the same "always floor" rule
as employed for time quantisation. Moreover,
we don't support floating point, only rationals
funny enough this possible memory corruption
didn't happen in the unit test, because my
compiler optimised the additional int field
of class SubDummy, making it the same size
of the baseclass. Now matters should be safe.
also touches the question how to represent the job
descriptor datastructure. @Cehteh: I've just pasted
in your preliminary data struct definitinons
from the relevant mailing list discussions.
this turns out to be the typical usage scenario
for ScopedCollection: a manager object owning
the collection will populate it with specially
crafted components and invoke a member function
for creating the individual components.
This shortcut avoids using tr1::bind alltogether
Meanwhile, BuffHanle became way more concrete,
making the separation of any kind of buffer or channel
type management concievable.
Thus: extract the obsolete ChannelDescriptor
and use switch at any location where the old
(superseeded) buffer handle is still referred
- using std::tr1::shared_ptr in namespace gui
- thus removing a lot of std::tr1 qualifications
- fix some includes. Should be relative to 'src'
- interface classes should declare a virtual dtor!
