Commit graph

790 commits

Author SHA1 Message Date
98078b9bb6 Chain-Load: investigate std::function inline-storage
...which is crucial for the solution pursued at the moment;
std::function is known to apply a small-object optimisation,
yet unfortunately there are no guarantees by the C++ standard
(it is only mandated that std::function handles a bare function
 pointer without overhead)

Other people have investigated that behaviour already,
indicating that at least one additional »slot« of data
can be handled with embedded storage in all known implementations
(while libstdc++ seemingly imposes the strongest limitations)
https://stackoverflow.com/a/77202545/444796

This experiment in the unit-test shows that for my setup
(libstdc++ and GCC-8) only a lambda capturing a single pointer
is handled entirely embedded into the std::function; already
a lambda capturing a shared-ptr leads to overflow into heap
2023-12-03 04:59:18 +01:00
3c713a4739 Chain-Load: invent the heart of the trap-mechanism
...the intention is to plant a »trojan lambda« into the target functor,
to set off initialisation (and possibly relocation) on demand.
2023-12-03 04:59:18 +01:00
1892d1beb5 Chain-Load: safety problems with rule initialisation
the RandomDraw rules developed last days are meant to be used
with user-provided λ-adapters; employing these in a context
of a DSL runs danger of producing dangling references.

Attempting to resolve this fundamental problem through
late-initialisation, and then locking the component into
a fixed memory location prior to actual usage. Driven by
the goal of a self-contained component, some advanced
trickery is required -- which again indicates better
to write a library component with adequate test coverage.
2023-12-03 04:59:18 +01:00
8b1326129a Library: RandomDraw - implementation complete and tested. 2023-12-03 04:59:17 +01:00
3808166494 Library: RandomDraw - invent new scheme for dynamic configuration
...now using the reworked partial-application helper...
...bind to *this and then recursively re-invoke the adaptation process
...need also to copy-capture the previously existing mapping-function

first test seems to work now
2023-12-03 04:59:17 +01:00
32b740cd40 Library: RandomDraw - dynamic configuration requires partial application
Investigation in test setup reveals that the intended solution
for dynamic configuration of the RandomDraw can not possibly work.
The reason is: the processing function binds back into the object instance.
This implies that RandomDraw must be *non-copyable*.

So we have to go full circle.
We need a way to pass the current instance to the configuration function.
And the most obvious and clear way would be to pass it as function argument.
Which however requires to *partially apply* this function.

So -- again -- we have to resort to one of the functor utilities
written several years ago; and while doing so, we must modernise
these tools further, to support perfect forwarding and binding
of reference arguments.
2023-12-03 04:59:17 +01:00
75cbfa8991 Library: RandomDraw - adaptor and mapping functions
...the beautiful thing with functions and Metaprogramming is:
it mostly works as designed out of the box, once you make it
past the Compiler.
2023-11-22 04:26:22 +01:00
2578df7c1d Library: RandomDraw - verify numerics (II)
- strive at complete branch coverage for the mapping function
- decide that the neutral value can deliberately lie outside
  the value range, in which case the probability setting
  controls the number of _value_ result incidents vs
  neutral value result incidents.
- introduce a third path to define this case clearly
- implement the range setting Builder-API functions
- absorb boundrary and illegal cases
2023-11-22 02:36:34 +01:00
4f28e8ad6c Library: RandomDraw - verify numerics (I)
- use a Draw with only a few values
- but with an origin within the value range
- verify stepping and distributions for various probabilities
2023-11-21 22:07:51 +01:00
bdb2f12b80 Library: RandomDraw - use dynamic quantiser
For sake of simplicity, since this whole exercise is a byproduct,
the mapping calculations are done in doubles. To get even distribution
of values and a good randomisation, it is thus necessary to break
down the size_t hash value in a first step (size_t can be 64bit
and random numbers would be subject to rounding errors otherwise)

The choice of this quantiser is tricky; it must be a power of two
to guarantee even distribution, and if chosen to close to the grid
of the result values, with lower probabilities we'd fail to cover
some of the possible result values.  If chosen to large, then
of course we'd run danger of producing correlated numbers on
consecutive picks.

Attempting to use 4 bits of headroom above the log-2 of the
required value range. For example, 10-step values would use
a quantiser of 128, which looks like a good compromise.
The following tests will show how good this choice holds up.
2023-11-21 19:50:22 +01:00
5b9a463b38 Library: RandomDraw - rework mapping rule to support origin
The first step was to allow setting a minimum value,
which in theory could also be negative (at no point is the
code actually limited to unsigned values; this is rather
the default in practice).

But reconsidering this extensions, then you'd also want
the "neutral value" to be handled properly. Within context,
this means that the *probability* controls when values other
than the neutral value are produced; especially with p = 1.0
the neutral value shall not be produced at all
2023-11-21 17:49:50 +01:00
75dd4210f2 Library: RandomDraw - must accept generic arguments
...since the Policy class now defines the function signature,
we can no longer assume that "input" is size_t. Rather, all
invocations must rely on the generic adaptaion scheme.

Getting this correct turns out rather tricky again;
best to rely on a generic function-composition.

Indeed I programmed such a helper several years ago,
with the caveat that at that time we used C++03 and
could not perfect-forward arguments. Today this problem
can be solved much more succinct using generic Lambdas.
2023-11-21 04:07:30 +01:00
651e28bac9 Library: RandomDraw - introduce policy template
to define this as a generic library component,
any reference to the actual data source moust be extracted
from the body of the implementation and supplied later
at usage site. In the actual case at hand the source
for randomness would be the node hash, and that is
absolutely an internal implementation detail.
2023-11-20 21:05:18 +01:00
605c1b4a17 Library: RandomDraw - consolidate prototype
...still same functionality as established yesterday in experimentation (try.cpp)
2023-11-20 18:49:00 +01:00
e5f5953b15 Library: RandomDraw - extract as generic component
The idea is to use some source of randomness to pick a
limited parameter value with controllable probability.
While the core of the implementation is nothing more
than some simple numeric adjustments, these turn out
to be rather intricate and obscure; the desire to
package these technicalities into a component
however necessitates to make invocations
at usage site self explanatory.
2023-11-20 16:38:55 +01:00
65fa16b626 Chain-Load: work out DSL for generating DOT scripts
...using a pre-established example as starting point

It seems that building up this kind of generator code
from a set of free functions in a secluded namespace
is the way most suitable to the nature of the C++ language
2023-11-16 03:19:19 +01:00
7a22e7f987 Test: helper for transitory manipulations
Use a simple destructor-trick to set up a concise notation
for temporarily manipulating a value for testing.
The manipulation will automatically be undone
when leaving scope
2023-11-08 19:27:08 +01:00
b61ca94ee5 Scheduler: rectify λ-post API
...to bring it more in line with all the other calls dealing with Activity*
...allows also to harmonise the ActivityLang::dispatchChain()
...and to compose the calls in Scheduler directly

NOTE: there is a twist: our string-formatting helper did not render
custom string conversions for objects passed as pointer. This was a
long standing problem, caused by ambiguous templates overloads;
now I've attempted to solve it one level more down, in util::StringConv.
This solution may turn out brittle, since we need to exclude any direct
string conversion, most notably the ones for C-Strings (const char*)

In case this solution turns out unsustainable, please feel free
to revert this API change, and return to passing Activity& in λ-post,
because in the end this is cosmetics.
2023-10-23 01:48:46 +02:00
3af6a54219 Library/Application: complete technology switch (closes #1279)
As follow-up to the rework of thread-handling, likewise also
the implementation base for locking was switched over from direct
usage of POSIX primitives to the portable wrappers available in
the C++ standard library. All usages have been reviewed and
modernised to prefer λ-functions where possible.

With this series of changes, the old threadpool implementation
and a lot of further low-level support facilities are not used
any more and can be dismantled. Due to the integration efforts
spurred by the »Playback Vertical Slice«, several questions of
architecture could be decided over the last months. The design
of the Scheduler and Engine turned out different than previously
anticipated; notably the Scheduler now covers a wider array of
functionality, including some asynchronous messaging. This has
ramifications for the organisation of work tasks and threads,
and leads to a more deterministic memory management. Resource
management will be done on a higher level, partially superseding
some of the concepts from the early phase of the Lumiera project.
2023-10-16 01:44:04 +02:00
685be1b039 Library/Application: consolidate Monitor API and usage
This is Step-2 : change the API towards application

Notably all invocation variants to support member functions
or a reference to bool flags are retracted, since today a
λ-binding directly at usage site tends to be more readable.

The function names are harmonised with the C++ standard and
emergency shutdown in the Subsystem-Runner is rationalised.

The old thread-wrapper test is repurposed to demonstrate
the effectiveness of monitor based locking.
2023-10-15 20:42:55 +02:00
73737f2aee Library/Application: consolidate Monitor implementation
After the fundamental switch from POSIX to the C++14 wrappers
the existing implementation of the Monitor can now be drastically condensed,
removing several layers of indirection. Moreover, all signatures
shall be changed to blend in with the names and patterns established
by the C++ standard.

This is Step-1 : consolidate the Implementation.

(to ensure correctness, the existing API towards application code was retained)
2023-10-15 02:41:41 +02:00
c37871ca78 Library/Application: switch Locking from POSIX to C++14
While not directly related to the thread handling framework,
it seems indicated to clean-up this part of the application alongside.

For »everyday« locking concerns, an Object Monitor abstraction was built
several years ago and together with the thread-wrapper, both at that time
based on direct usage of POSIX. This changeset does a mere literal
replacement of the POSIX calls with the corresponding C++ wrappers
on the lowest level. The resulting code is needlessly indirect, yet
at API-level this change is totally a drop-in replacment.
2023-10-13 23:46:38 +02:00
5db49afafd Library/Application: now able to switch supervisor-thread in OutputDirector
This, and the GUI thread prompted an further round of
design extensions and rework of the thread-wrapper.

Especially there is now support for self-managed threads,
which can be launched and operate completely detached from the
context used to start them. This resolves an occasional SEGFAULT
at shutdown. An alternative (admittedly much simpler) solution
would have been to create a fixed context in a static global
variable and to attach a regular thread wrapper from there,
managed through unique_ptr.

It seems obvious that the new solution is preferable,
since all the tricky technicalities are encapsulated now.
2023-10-12 02:10:50 +02:00
29b9126c26 Library: test coverage for lifecycle management
Add a complete demonstration for a setup akin to what we use
for the Session thread: a threaded component which manages itself
but also exposes an external interface, which is opened/closed alongside
2023-10-11 22:02:52 +02:00
7b25609896 Library: test coverage for self-managed thread
...extract and improve the tuple-rewriting function
...improve instance tracking test dummy objects
...complete test coverage and verify proper memory handling
2023-10-11 21:06:56 +02:00
f6a6b0b68f Library: allow to bind a member function into self-managed thread
Oh my.
Yet another hideously complex problem and workaround...

Since a week I am like "almost done"
2023-10-11 13:21:08 +02:00
42eba8425a Library: now able to provide a self-managed thread
After quite some detours, with this take I'm finally able to
provide a stringent design to embody all the variants of thread start
encountered in practice in the Lumiera code base.

Especially the *self-managed* thread is now represented as a special-case
of a lifecycle-hook, and can be embodied into a builder front-end,
able to work with any client-provided thread-wrapper subclass.
2023-10-10 21:45:41 +02:00
8b3f9e17cd Library: scaffolding to install thread lifecycle hooks
to cover the identified use-cases a wide variety of functors
must be accepted and adapted appropriately. A special twist arises
from the fact that the complete thread-wrapper component stack works
without RTTI; a derived class can not access the thread-wrapper internals
while the policy component to handle those hooks can not directly downcast
to some derived user provided class. But obviously at usage site it
can be expected to access both realms from such a callback.

The solution is to detect the argument type of the given functor
and to build a two step path for a safe static cast.
2023-10-10 19:47:39 +02:00
5f9683ef10 Library: policy for self-managed thread
...after resolving the fundamental design problems,
a policy mix-in can be defined now for a thread that deletes
its own wrapper at the end of the thread-function.

Such a setup would allow for »fire-and-forget« threads, but with
wrapper and ensuring safe allocations. The prominent use case
for such a setup would be the GUI-Thread.
2023-10-10 02:55:23 +02:00
faa0d3e211 Library: solved embedding arbitrary argument sequences
Concept study of the intended solution successful.

Can now transparently embed any conceivable functor
and an arbitrary argument sequence into a launcher-λ
Materialising into a std::tuple<decay_t<TYPES...>> did the trick.
2023-10-09 02:57:03 +02:00
fd0370bd11 Library: still fighting to get the design straight
Considering a solution to shift the actual launch of the new thread
from the initialiser list into the ctor body, to circumvent the possible
"undefined behaviour". This would also be prerequisite for defining
a self-managed variant of the thread-wrapper.

Alternative / Plan.B would be to abandon the idea of a self-contained
"thread" building block, instead relying on precise setup in the usage
context -- however, not willing to yield yet, since that would be exactly
what I wanted to avoid: having technicalities of thread start, argument
handover and failure detection intermingled with the business code.
2023-10-08 17:26:36 +02:00
08c3e76f14 Library: identified design challenges
On a close look, the wrapper design as pursued here
turns out to be prone to insidious data race problems.
This was true also for the existing solution, but becomes
more clear due to the precise definitions from the C++ standard.

This is a confusing situation, because these races typically do not
materialise in practice; due to the latency of the OS scheduler the
new thread starts invoking user code at least 100µs after the Wrapper
object is fully constructed (typically more like 500µs, which is a lot)

The standard case (lib::Thread) in its current form is correct, but borderline
to undefined behaviour, and any initialisation of members in a derived class
would be off limits (the thread-wrapper should not be used as baseclass,
rather as member)
2023-10-07 03:25:39 +02:00
88b91d204c Library: identified further use-case variants to cover
...while reworking the application code, it became clear that
actually there are two further quite distinct variants of usage.
And while these could be implemented with some trickery based on
the Thread-wrapper defined thus far, it seems prudent better to
establish a safely confined explicit setup for these cases:

- a fire-and-forget-thread, which manages its own memory autonomously
- a thread with explicit lifecycle, with detectable not-running state
2023-10-05 23:35:52 +02:00
ff052ec5a2 Library/Application: switch Microbenchmark + SyncBarrier tests
...these were already written envisionaging he new API,
so it's more or less a drop-in replacement.

- cant use vector anymore, since thread objects are move-only
- use ScopedCollection instead, which also has the benefit of
  allocating the requires space up-front. Allow to deduce the
  type parameter of the placed elements
2023-10-03 22:56:09 +02:00
d879ae7fbd Library: fix cause of the deadlock in Session-Thread
... which became apparent after switching to the new Thread-wrapper implementation
... the reason is a bug in the Thread-Monitor (which will also be reworked soon)
2023-10-01 20:29:11 +02:00
9cb0a9b680 Library: discontinue setting error flag from Exceptions (see #1341)
While seemingly subtle, this is a ''deep change.''
Up to now, the project attempted to maintain two mutually disjoint
systems of error reporting: C-style error flags and C++ exceptions.
Most notably, an attempt was made to keep both error states synced.

During the recent integration efforts, this increasingly turned out
as an obstacle and source for insidious problems (like deadlocks).


As a resolve, hereby the relation of both systems is **clarified**:
 * C-style error flags shall only be set and used by C code henceforth
 * C++ exceptions can (optionally) be thrown by retrieving the C-style error code
 * but the opposite is now ''discontinued'' : Exceptions ''do not set'' the error flag anymore
2023-10-01 20:11:45 +02:00
d79e33f797 Library: verify thread self-recognition
...this function was also ported to the new wrapper,
and can be verified now in a much more succinct way.

''This completes porting of the thread-wrapper''
2023-09-30 00:10:09 +02:00
1d625a01e0 Library: complete and modernise ThreadWrapperJoin_test
Since the decision was taken to retain support for this special feature,
and even extend it to allow passing values, the additional functionality
should be documented in the test. Doing so also highlighted subtle problems
with argument binding.
2023-09-29 23:42:22 +02:00
d37a3abd6c Library: actually verify parallelism
Now the ThreadWrapper_test offers both

- a really simple usage example

- a comprehensive test to verify that actually the
  thread-function is invoked the expected number of times
  and that this invocations must have been parallelised
2023-09-29 19:21:28 +02:00
1d30d47b9a Library: add a simple usage for clarity 2023-09-29 18:45:47 +02:00
bfc4a60a09 Library: rework ThreadWrapper_test
...while the change in the thread-wrapper implementation was drop-in,
this test in the existing from is questionable: it actually tests locking
2023-09-29 17:46:24 +02:00
201672a0ad Library: reconsider join / stringify API
- it is not directly possible to provide a variadic join(args...),
  due to overload resolution ambiguities

- as a remedy, simplify the invocation of stringify() for the typical cases,
  and provide some frequently used shortcuts
2023-09-29 17:00:13 +02:00
691d2b43fa Library: add shortcut-ctor for own-member function
A common usage pattern is to derive from lib::Thread
and then implement the actual thread function as a member function
of this special-Thread-object (possibly also involving other data members)

Provide a simplified invocation for this special case,
also generating the thread-id automatically from the arguments
2023-09-28 17:45:32 +02:00
2c18c39c18 Library: complete the Thread-joining policy
after all this groundwork, implementing the invocation,
capturing and hand-over of results is simple, and the
thread-wrapper classes became fairly understandable.
2023-09-28 02:09:36 +02:00
620639b7ce Library: augment the »Either« wrapper to funciton invocation
This relieves the Thread policy from a lot of technicalities,
while also creating a generally useful tool: the ability to invoke
/anything callable/ (thanks to std::invoke) in a fail-safe way and
transform the exception into an Either type
2023-09-27 23:17:56 +02:00
9c0fa7139d Library: capture and transport the exception itself
...using the std::exception_ptr and helpers, we can now reliably
transport any exception object as the »right« value of the »Either«
2023-09-27 02:51:00 +02:00
4348e110cb Library: change of plan - retain the »Either« wrapper
on second thought, the ability to transport an exception still seems
worthwhile, and can be achieved by some rearrangements in the design.

As preparation, reorganise the design of the Either-wrapper (lib::Result)
2023-09-27 01:27:53 +02:00
3fa4f02737 Library: new thread-wrapper implementation complete
- relocate some code into a dedicated translation unit to reduce #includes
- actually set the thread-ID (the old implementation had only a TODO at that point)
2023-09-26 02:32:48 +02:00
67b010ba7e Library: (re)introduce the distinction join / detach
While it would be straight forward from an implementation POV
to just expose both variants on the API (as the C++ standard does),
it seems prudent to enforce the distinction, and to highlight the
auto-detaching behaviour as the preferred standard case.

Creating worker threads just for one computation and joining the results
seemed like a good idea 30 years ago; today we prefer Futures or asynchronous
messaging to achieve similar results in a robust and performant way.

ThreadJoinable can come in handy however for writing unit tests, were
the controlling master thread has to wait prior to perform verification.

So the old design seems well advised in this respect and will be retained
2023-09-26 01:00:00 +02:00
c9a0203492 Library: gut and remould the existing thread-wrapper
- cut the ties to the old POSIX-based custom threadpool framework
- remove operations deemed no longer necessary
- sync() obsoleted by the new SyncBarrier
- support anything std::invoke supports
2023-09-25 16:27:38 +02:00
11cb53a406 Library: investigate Mutex+Condition-Var for comparison
...which is the technique used in the existing Threadpool framwork.
As expected, such a solution is significantly slower than the new
atomics-based implementation. Yet how much slower is still striking.
2023-09-24 21:52:38 +02:00
7474f56e89 Library: investigate performance of SyncBarrier
Timing measurements in concurrent usage situation.
Observed delay is in the order of magnitude of known scheduling leeway;
assuming thus no relevant overhead related to implementation technique
2023-09-24 20:38:27 +02:00
c183045dfa Library: switch Microbenchmark setup to C++17 threads
Over time, a collection of microbenchmark helper functions was
extracted from occasional use -- including a variant to perform
parallelised microbenchmarks. While not used beyond sporadic experiments yet,
this framework seems a perfect fit for measuring the SyncBarrier performance.

There is only one catch:
 - it uses the old Threadpool + POSIX thread support
 - these require the Threadpool service to be started...
 - which in turn prohibits using them for libary tests

And last but not least: this setup already requires a barrier.

==> switch the existing microbenchmark setup to c++17 threads preliminarily
    (until the thread-wrapper has been reworked).
==> also introduce the new SyncBarrier here immediately
==> use this as a validation test of the setup + SyncBarrier
2023-09-24 18:07:28 +02:00
35ff53a716 Library: generalise pipeline summation into fold-left
Using the same building blocks, this operation can be generalised even more,
leading to a much cleaner implementation (also with better type deduction).

The feature actually used here, namely summing up all values,
can then be provided as a convenience shortcut, filling in std::plus
as a default reduction operator.
2023-09-24 02:45:43 +02:00
b416a67bb9 Library: extract summation of pipeline results
...first used as part of the test harness;
seemingly this is a generic and generally useful shortcut,
similar to algorithm::reduce (or some kind of fold-left operation)
2023-09-23 19:39:08 +02:00
b15281d44b Library: implement and verify SyncBarrier 2023-09-23 18:05:17 +02:00
6735857f3b Library: draft a SyncBarrier latch
Intended as replacement for the Mutex/ConditionVar based barrier
built into the exiting Lumiera thread handling framework and used
to ensure safe hand-over of a bound functor into the starting new
thread. The standard requires a comparable guarantee for the C++17
concurrency framework, expressed as a "synchronizes_with" assertion
along the lines of the Atomics framework.

While in most cases dedicated synchronisation is thus not required
anymore when swtiching to C++17, some special extended use cases
remain to be addressed, where the complete initialisation of
further support framework must be ensured.

With C++20 this would be easy to achieve with a std::latch, so we
need a simple workaround for the time being. After consideration of
the typical use case, I am aiming at a middle ground in terms of
performance, by using a yield-wait until satisfying the latch condition.
2023-09-22 21:55:53 +02:00
416895b5b2 Library: prepare switch of Thread-wrapper to C++17
The investigation for #1279 leads to the following conclusions

- the features and the design of our custom thread-wrapper
  almost entirely matches the design chosen meanwhile by the C++ committee

- the implementation provided by the standard library however uses
  modern techniques (especially Atomics) and is more precisely worked out
  than our custom implementation was.

- we do not need an *active* threadpool with work-assignment,
  rather we'll use *active* workers and a *passive* pool,
  which was easy to implement based on C++17 features

==> decision to drop our POSIX based custom implementation
    and to retrofit the Thread-wrapper as a drop-in replacement

+++ start this refactoring by moving code into the Library
+++ create a copy of the Threadwrapper-code to build and test
    the refactorings while the application itself still uses
    existing code, until the transition is complete
2023-09-21 23:23:55 +02:00
1c6ee62c1a Activity-Lang: allow to verify invocation param in test
requires to supplement EventLog matching primitives
to pick and verify a specific positional argument.

Moreover, it is more or less arbitrary which job invocation parameters
are unpacked and exposed for verification; we'll have to see what is
actually required for writing tests...
2023-08-15 20:03:01 +02:00
e3f1aa4f7c Activity-Lang: support negative assertions for tests
Testcase (detect function invocation) passes now as expected


Some Library / Framework changes

- rename event-log-test.cpp
- allow the ExpectString also to work with concatenated expectation strings


Remark: there was a warning in the comment in event-log.hpp,
pointing out that negative assertions are shallow.

However, after the rework in 9/2018 (commit: d923138d1)
...this should no longer be true, since we perform proper backtracking,
leading to an exhaustive search.
2023-08-14 19:25:56 +02:00
49f2e34e4c Library: extract type rebinding helper
...turns out this is entirely generic and not tied to the context
within ActivityDetector, where it was first introduced to build a
mock functor to log all invocations.

Basically this meta-function generates a new instantiation of the
template X, using the variadic argument pack from template U<ARGS...>
2023-08-01 14:52:20 +02:00
824a626c2e Block-Flow: investigate proper working of on-demand allocation
Library: add "obvious" utility to the IterExplorer, allowing to
         materialise all contents of the Pipeline into a container

...use this to take a snapshot of all currently active Extent addresses
2023-07-12 19:19:41 +02:00
ccf0710903 Block-Flow: maintain an »Epoch« within the raw allocation Extent
- the idea is to use slot-0 in each extent for administrative metadata
- to that end, a specialised GATE-Activity is placed into slot-0
- decision to use the next-pointer for managing the next free slot
- thus we need the help of the underlying ExtentFamily for navigating Extents

Decision to refrain from any attempt to "fix" excessive memory usage,
caused by Epochs still blocked by pending IO operations. Rather, we
assume the engine uses sane parametrisation (possibly with dynamic adjustment)
Yet still there will be some safety limit, but when exceeding this limit,
the allocator will just throw, thereby killing the playback/render process
2023-07-09 01:32:27 +02:00
130bc095d9 the new design takes the old name
The second design from 2017, based on a pipeline builder,
is now renamed `TreeExplorer` ⟼ `IterExplorer` and uses
the memorable entrance point `lib::explore(<seq>)`

✔
2023-06-22 20:23:55 +02:00
d109f5e1fb bye bye Monad (closes #1276)
after completing the recent clean-up and refactoring work,
the monad based framework for recursive tree expansion
can be abandoned and retracted.

This approach from functional programming leads to code,
which is ''cool to write'' yet ''hard to understand.''

A second design attempt was based on the pipeline and decorator pattern
and integrates the monadic expansion as a special case, used here to
discover the prerequisites for a render job. This turned out to be
more effective and prolific and became standard for several exploring
and backtracking algorithms in Lumiera.
2023-06-22 20:23:55 +02:00
a1c1456849 Job-Planning: dispose of FrameCoord in pipeline and Dispatcher interface
...as a preparation for solving a logical problem with the Planning-Pipeline;
it can not quite work as intended just by passing down the pair of
current ticket and dependent ticket, since we have to calculate a chained
calculation of job deadlines, leading up to the root ticket for a frame.

My solution idea is to create the JobPlanning earlier in the pipeline,
already *before* the expansion of prerequisites, and rather to integrate
the representation of the dependency relation direcly into JobPlanning
2023-06-18 03:50:48 +02:00
f84517547b Dispatcher-Pipeline: coordination of base tick and prerequisite expansion
- had to fix a logical inconsistency in the underlying Expander implementation
  in TreeExplorer: the source-pipeline was pulled in advance on expansion,
  in order to "consume" the expanded element immediately; now we retain
  this element (actually inaccessible) until all of the immediate
  children are consumed; thus the (visible) state of the PipeFrameTick
  stays at the frame number corresponding to the top-level frame Job,
  while possibly expanding a complete tree of flexible prerequisites

This test now gives a nice visualisation of the interconnected states
in the Job-Planning pipeline. This can be quite complex, yet I still think
that this semi-functional approach with a stateful pipeline and expand functors
is the cleanest way to handle this while encapsulating all details
2023-06-14 18:12:41 +02:00
ad173540d9 Library: allow for a stop condition in iterator pipeline
...introduced in preparation for building the Dispatcher pipeline,
which at its core means to iterate over a sequence of frame positions;
thus we need a way to stop rendering at a predetermined point...
2023-06-01 16:48:27 +02:00
94cec423d0 Job-Planning: switch to processing references
...which uncovers further deeply nested problems,
especially when referring to non-copyable types.

Thus need to construct a common type that can be used
both to refer to the source elements and the expanded elements,
and use this common type as result type and also attempt to
produce better diagnostic messages on type mismatch....
2023-05-23 01:08:05 +02:00
0df0fd001e Library: fix follow-up problems with const correctness
...the improved const correctness on STL iterators uncovered another
latent problem with out diagnositc format helper, which provide
consistently rounded float and double output, but failed to take
CV-qualifiaction into account
2023-05-23 01:07:53 +02:00
e176e54004 Library: adjust and fix semantics of nested 'value_type' binding
This is a subtle and far reaching fix, which hopefully removes
a roadblock regarding a Dispatcher pipeline: Our type rebinding
template used to pick up nested type definitions, especially
'value_type' and 'reference' from iterators and containers,
took an overly simplistic approach, which was then fixed
at various places driven by individual problems.

Now:
 - value_type is conceptually the "thing" exposed by the iterator
 - and pointers are treated as simple values, and no longer linked
   to their pointee type; rather we handle the twist regarding
   STL const_iterator direcly (it defines a non const value_type,
   which is sensible from the STL point of view, but breaks our
   generic iterator wrapping mechanism)
2023-05-23 01:07:53 +02:00
a940cd25bc Library: extract helper for unloading a sequence into a tuple 2023-05-10 14:49:51 +02:00
256045f91d Segmentation: add test case to cover instance management
- only the parts actually touched by the algo will be re-allocated
- when a segment is split, the clone copies carry on all data


Library: add function to check for a bare address (without type info)
2023-05-05 01:34:41 +02:00
52dad70753 Segmentation: Split-Splice algorithm implementation complete
...and verified including corner cases...
2023-05-04 15:59:17 +02:00
3f2f3db568 Segmentation: verify Split-Splice standard cases
Algorithm seems to work basically...
There was a copy-n-paste error in the treatment of the Successor,
leading to spurious duplication in some cases
2023-05-04 14:43:40 +02:00
c8f879ff3f Segmentation: extract invocation for test
typically you'd write an custom adaptor function
to define all the λ-bindings for the Algorithm
2023-05-04 12:01:25 +02:00
f9a4d6134c Segmentation: setup for simple demo test case works
due to having extracted the Algo implementation as template,
we can now instantiate it for isolated tests with simple integer intervals
2023-05-04 02:07:38 +02:00
00ca84a2aa test-helper for comparison with expected (string) result
...this is something I should have done since YEARS, really...

Whenever working with symbolically represented data, tests
typically involve checking *hundreds* of expected results,
and thus it can be really hard to find out where the
failure actually happens; it is better for readability
to have the expected result string immediately in the
test code; now this expected result can be marked
with a user-defined literal, and then on mismatch
the expected and the real value will be printed.
2023-05-04 00:48:29 +02:00
476c0f6493 Segmentation: build test-setup for Split-Splice-Algo
use simple intervals over integer numbers as test setup
- detect possible memory leaks
- detect corrupted Segmentation sequences
2023-05-03 15:27:46 +02:00
b582c35c9f Segmentation: structure analysis for splitSplice operation
There are 12 distinct cases regarding the orientation of two intervals;
The Segmentation::splitSplice() operation shall insert a new Segment
and adjust / truncate / expand / split / delete existing segments
such as to retain the *Invariant* (seamless segmentation covering
the complete time axis)
2023-05-02 04:29:34 +02:00
a807705185 Segmentation: draft simple mock-based setup for tests
- how to pass-in a specification given as GenNode
- now this might be translated into a MockJobTicket allocated in the MockSegmentation

Unimplemented: actually build the Segment with suitable start/end time
2023-05-01 17:02:11 +02:00
856d8a3b51 Library: allow to reverse intrusive single linked list
Looks like we'll actually retain and use this low-level solution
in cases where we just can not afford heap allocations but need
to keep polymorphic objects close to one another in memory.

Since single linked lists are filled by prepending, it is rather
common to need the reversed order of elements for traversal,
which can be achieved in linear time.

And while we're here, we can modernise the templated emplacement functions
2023-04-20 18:53:17 +02:00
bcd2b3d632 PlaybackVerticalSlice: design analysis for Frame Dispatcher and Scheduler
- decision: the Monad-style iteration framework will be abandoned
- the job-planning will be recast in terms of the iter-tree-explorer
- job-planning and frame dispatch will be disentangled
- the Scheduler will deliberately offer a high-level interface
- on this high-level, Scheduler will support dependency management
- the low-level implementation of the Scheduler will be based on Activity verbs
2023-04-14 04:43:39 +02:00
289f92da7e Timeline: safely calculate sum/difference of large fractional times
...in a similar vein as done for the product calculation.
In this case, we need to check the dimensions carefully and pick
the best calculation path, but as long as the overall result can
be represented, it should be possible to carry out the calculation
with fractional values, albeit introducing a small error.

As a follow-up, I have now also refactored the re-quantisation
functions, to be usable for general requantisation to another grid,
and I used these to replace the *naive* implementation of the
conversion FSecs -> µ-Grid, which caused a lot of integer-wrap-around

However, while the test now works basically without glitch or wrap,
the window position is still numerically of by 1e-6, which becomes
quite noticeably here due to the large overall span used for the test.
2022-12-01 23:23:50 +01:00
cfe3a6618f Lib: cover re-quantisation helper
...which I intend to use for sanitising poisonous rational numbers,
as prerequisite for handling divisor based time scales in the ZoomWindow
2022-11-15 02:13:57 +01:00
ce1220ee72 Lib: test coverage for rational-int corner cases and integer-log
- detailed documentation of known problematic behaviour
  when working with rational fractions
- demonstrate the heuristic predicate to detect dangerous numbers

- add extensive coverage and microbenchmarks for the integer-logarithm
  implementation, based on an example on Stackoverflow. Surprising result:
  The std::ilog(double) function is of comparable speed, at least for
  GCC-8 on Debian-Buster.
2022-11-14 05:20:37 +01:00
f393780845 Lib: fix a bug with diagnostic output
The header "format-cout.hpp" offers a convenience function
to print pretty much any object or data in human readable form.
However, the formatter for pointers used within this framework
switched std::cout into hexadecimal display of numbers and failed
to clean-up this state.

Since the "stickyness" of IOS stream manipulators is generally a problem,
we now provide a RAII helper to capture the previous stream state and
automatically restore it when leaving the scope.
2022-09-27 01:51:21 +02:00
ed7e3b4b32 ElementBox: extract builder qualifier support as library implementation
Complete the investigation and turn the solution into a generic
mix-in-template, which can be used in flexible ways to support
this qualifier notation.

Moreover, recapitulate requirements for the ElementBoxWidget
2022-08-28 23:36:27 +02:00
e15e893a01 Util: use case-insensitive matching for parsing bool values 2021-08-20 14:33:59 +02:00
4caf790339 Library: verify PlantingHandle's extended capabilities
- move construct into the buffer
- directly invoke the payload constructor through PlantingHandle
- reconsider type signature and size constraint
- extend the unit test
- document a corner case of c++ "perfect forwarding",
  which caused me some grief here
2021-05-07 22:50:13 +02:00
5a37bce855 Lib/Diff: extend PlantingHandle to allow for placment-new
...this extension was spurred by the previeous refactoring.
Since 'emplace' now clearly denotes an operation to move-embed an existing object,
we could as well offer a separate 'create' API, which would take forwarding
arguments as usual and just delegates to the placement-new operation 'create'
already available in the InPlaceBuffer class.

Such would be a convenience shortcut and is not strictly necessary,
since move-construction is typically optimised away; yet it would also
allow to support strictly non-copyable payload types.

This refactoring also highlights a fuzziness in the existing design,
where we just passed the interface type, while being sloppy about the
DEFAULT type. In fact this *is* relevant, since any kind of construction
might fail, necessitating to default-construct a placeholder, since
InPlaceBuffer was intended for zero-overhead usage and thus has in itself
no means to know about the state of its buffer's contents. Thus the
only sane contract is that there is always a valid object emplaced
into the buffer, which in turn forces us to provide a loophole for
class hierarchies with an abstract base class -- in such a case the
user has to provide a fallback type explicitly.
2021-05-02 19:40:11 +02:00
5aa41accfc Lib/Diff: prefer the name "emplace" over "build"
...for the operation on a PlantingHandle, which allows
to implant a sub type instance into the opaque buffer.

 * "create" should be used for a constructor invocation
 * "emplace" takes an existing object and move-constructs
2021-05-02 18:31:47 +02:00
acb674a9d2 Project: update and clean-up Doxygen configuration
...in an attempt to clarify why numerous cross links are not generated.
In the end, this attempt was not very successful, yet I could find some breadcrumbs...

- file comments generally seem to have a problem with auto link generation;
  only fully qualified names seem to work reliably

- cross links to entities within a namespace do not work,
  if the corresponding namespace is not documented in Doxygen

- documentation for entities within anonymous namespaces
  must be explicitly enabled. Of course this makes only sense
  for detailed documentation (but we do generate detailed
  documentation here, including implementation notes)

- and the notorious problem: each file needs a valid @file comment

- the hierarchy of Markdown headings must be consistent within each
  documentation section. This entails also to individual documented
  entities. Basically, there must be a level-one heading (prefix "#"),
  otherwise all headings will just disappear...

- sometimes the doc/devel/doxygen-warnings.txt gives further clues
2021-01-24 19:35:45 +01:00
06dbb9fad5 DiffFramework: simplify existing bindings
...by relying on the newly implemented automatic standard binding
Looks like a significant improvement for me, now the actual bindings
only details aspects, which are related to the target, and no longer
such technicalitis like how to place a Child-Mutator into a buffer handle
2021-01-23 12:55:10 +01:00
05b5ee9a7e Diff-Framework: investigate simplification for the most common case
After this long break during the "Covid Year 2020",
I pick this clean-up task as a means to fresh up my knowledge about the code base

The point to note is, when looking at all the existing diff bindings,
seemingly there is a lot of redundancy on some technical details,
which do not cary much meaining or relevance at the usage site:

- the most prominent case is binding to a collection of DiffMutables hold by smart-ptr
- all these objects expose an object identity (getID() function), which can be used as »Matcher«
- and all these objects can just delegate to the child's buildMutator() function
  for entering a recursive mutation.
2021-01-22 12:38:45 +01:00
657b94a4e3 ++ a strange year passed by ++
read the code, documentation and mindmap to find out
at what point I was when this story unfolded
2021-01-20 08:05:30 +01:00
dd016667ad Diff-Listener: add a variant to trigger also on value assignment (see #1206)
As it turned out, it is rather easy to extend the existing listener
for structural changes to detect also value assignments. Actually
it seems we'd need both flavours, so be it.
2020-03-15 23:11:14 +01:00
0f6f09180e Lib: simplified optional access to nested record attributes
Yeah, C++17, finally!

...not totally sure if we want to go that route.
However, the noise reduction in terms of code size at call site looks compelling
2020-03-14 23:52:04 +01:00
c7d157e295 Library: integrate generic min/max function
...built while investigating type deduction problems on PtrDerefIter
...also allow PtrDerefIter to work with std::unique_ptr
2020-03-08 02:05:39 +01:00
b2b5cf0f6d MERGE: upgrade to Debian/Buster and to C++17 2020-02-22 02:16:25 +01:00