Commit graph

2476 commits

Author SHA1 Message Date
73a9e4495a Job-Planning: code up simplest use case 2023-06-16 01:50:11 +02:00
a551314e80 Job-Planning: start rework of the planning data aggregation
The initial implementation effort for Player and Job-Planning
has been reviewed and largely reworked, and some parts are now
obsoleted by the reworked alternative and can be disabled.

The basic idea will be retained though: JobPlanning is a
data aggregator and performs the final step of creating a Job
2023-06-15 03:51:07 +02:00
f84517547b Dispatcher-Pipeline: coordination of base tick and prerequisite expansion
- had to fix a logical inconsistency in the underlying Expander implementation
  in TreeExplorer: the source-pipeline was pulled in advance on expansion,
  in order to "consume" the expanded element immediately; now we retain
  this element (actually inaccessible) until all of the immediate
  children are consumed; thus the (visible) state of the PipeFrameTick
  stays at the frame number corresponding to the top-level frame Job,
  while possibly expanding a complete tree of flexible prerequisites

This test now gives a nice visualisation of the interconnected states
in the Job-Planning pipeline. This can be quite complex, yet I still think
that this semi-functional approach with a stateful pipeline and expand functors
is the cleanest way to handle this while encapsulating all details
2023-06-14 18:12:41 +02:00
08dfe1007c Dispatcher-Pipeline: verify the expansion of prerequisites
- fix a bug in the MockDispatcher, when duplicating the ExitNodes.
  A vector-ctor with curly braces will be interpreted as std::initializer_list

- add visualisation of the contents appearing at the end of the pipeline

*** something still broken here, increments don't happen as expected
2023-06-14 04:20:50 +02:00
542017aa65 Dispatcher-Pipeline: mocked Dispatcher implementation complete (closes: #1294)
`steam/engine/mock-dispatcher.hpp |cpp` now integrates this
''complete mock setup for render jobs and frame dispatching.''
The exising `DummyJob` has been slightly adapted and renamed
to `MockJob` and is tightly integrated with the other mocks.

The implementation of a `MockDispatcher` necessitated to change
the use of `MockJobTicket`. The initial attempts used a complete
mock implementation, but this approach turned out not to be viable.
Instead — based on the ideas developed for the mock setup —
now the prospective real implementation of `JobTicket` is available
and will be used by the mock setup too. Instead of a synthetic spec,
now a setup of recursively connected `ExitNode`(s) is used; the latter
seems to develop into some kind of Facade for the render node network.

Based on this mock setup, we can now demonstrate the (mostly) complete
Job-Planning pipeline, starting from a segmentation up to render jobs,
and verify proper connectivity and job invocation.
✔
2023-06-13 20:23:33 +02:00
0b9705692b Dispatcher-Pipeline: now (finally) able to implement MockDispatcher
MockSupport_test      : PASS
JobPlanningSetup_test : PASS(as far as defined)
2023-06-13 03:47:42 +02:00
122addbff5 Dispatcher-Pipeline: expected behaviour of (mock)Dispatcher 2023-06-13 00:15:16 +02:00
2031a58775 Dispatcher-Pipeline: decide upon the translation into portIDX
- has to be prepared / supported by the RenderEnvironmentClosure
- actual translation happens when building the Dispatcher-Pipeline
- implementation delegate through
    virtual size_t Dispatcher::resolveModelPort (ModelPort)
2023-06-12 19:21:14 +02:00
e6dcb6253c Dispatcher-Pipeline: resolve further problems with re-entrant allocation
...ouch this was insidious: the STL implementation for list does not
return a pointer to the element just allocated, but rather retrieves
and dereferences the back() / front() iterator after returning from emplace_back|front()

...which in case of re-entrant allocations is something wildly different
than the initial allocation. Thus a *cheap* and dirty placeholder implementation
just using a STL container is not possible, and we need at least
to code up likewise cheesy placeholder implementation by hand.
- separate allocation and ctor all
- use an inline buffer in the STL container
- explicitly handle ctor failures to discard allocation
- NOT THREADSAFE and likely WASTFUL in terms of performance


==> MockSupport_test now back to GREEN after complete refactoring
2023-06-12 17:21:41 +02:00
bf3e612c55 Dispatcher-Pipeline: create hook for self-validation
...later to be extended into the render nodes network
2023-06-12 01:18:59 +02:00
0933d2bba8 Dispatcher-Pipeline: simplify JobTicket and remove channel differentiation
The existing implementation of the Player from 2012~2015 inclduded
an additional differentiation by media channel (for multichannel media)
and would build a separate CalcStream for each channel.

The in-depth analysis conducted for the ongoing »Vertical Slice« effort
revealed that this differentiation is besides the point and would never
be materialised: Since -- by definition -- all media processing has
to be done by the engine, also the generation of the final output format
including any channel multiplexing will happen in render nodes.
The only exception would be when only a single channel of multichannel
media is extracted -- yet this case would then translate into a
dedicated ModelPort.

Based on this reasoning, a lot of complexity (and some contradictions)
within the JobTicket implementation can be removed -- together with
some further leftovers of the fist attempt to build JobTickets always
from a Mock specification (we now use construction by the Segment,
based on an ExitNode, which is the expected actual implementation
for production setup)
2023-06-12 00:04:45 +02:00
b18e79d077 Dispatcher-Pipeline: solve allocation of JobTicket instances
...by defining a new scheme for access to custom allocators
...and then passing a reference to such an accessor into the
   JobTicket ctor, thereby allowing the ticket istelf recursively
   to place further JobTicket instances into the allocation space

--> success, test passes (finally)
2023-06-11 04:37:38 +02:00
f25ec2f5ef Dispatcher-Pipeline: switch JobTicket creation to use ExitNode directly
Up to now, a draft/mock implementation was used, relying on a »spec tuple«,
which was fabricated by MockJobTicket. But with the introduction of
NodeGraphAttachment, the MockSequence now generates a nested ExitNode structure,
and thus the JobTicket will be created through the "real" ctor, and
no longer via MockJobTicket.

Thus it is possible to skip this whole interspersed »spec tuple«,
since ExitNode *is* already this aggregated / abstracted Spec
2023-06-10 04:52:40 +02:00
2c3b85a122 Dispatcher-Pipeline: allocate JobTicket in Segment
PROBLEM: can not implement Spec-generation, since
 - we must use a λ for internal allocation of JobTickets
 - but recursive type inference is not possible

Will thus need to abandon the Spec-Tuple and relocate this
traversal-and-generation code into JobTicket itself
2023-06-09 02:48:38 +02:00
c246c21e41 Dispatcher-Pipeline: remould Segment for on-demand JobTicket generation
Use another unit-test (FixtureSegment_test) to guide and cover
the transition from the existing fake-implementation to the
actual implementation, where the JobTicket will be generated
on-demand, from a NodeGraphAttachment
2023-06-08 03:21:43 +02:00
0c57a61545 Dispatcher-Pipeline: implement Generation of fake-ExitNodes
...similar to the Generation of complete JobTickets in the initial implementation
2023-06-08 00:28:44 +02:00
f6af4c6a16 Dispatcher-Pipeline: prepare test for the new NodeGraphAttachment
It turns out that the real (not mocked) implementation of JobTicket creation
is already required now for this planned (mock)Dispatcher setup;
moreover, this real implementation turns out to be almost identical
to the mock implementation written recently -- just nested structure
of prerequiste JobTickets need to be changed into a similar structur
of ExitNodes

-- as an aside: rearrange various tests to be more in-line
   with the envisioned architecture of playback and engine
2023-06-07 04:03:00 +02:00
3b2e5db7b4 Dispatcher-Pipeline: consider how to access render nodes from job
...this opens up yet another difficult question and a host of new problems
- how are prerequisites detected or arranged by the Builder
- how are prerequisites represented?
- what is an ExitNode in terms of implementation? A subclass of ProcNode?
- how will the actual implementation of JobTicket creation (on-demand) work?
- how to adapt the Mock implementation, while retaining the Specification
  for Segments and prerequisites?
2023-06-06 04:25:12 +02:00
7d5c32e6b6 Dispatcher-Pipeline: draft test for JobTicket access 2023-06-05 18:09:42 +02:00
4601c6350e Dispatcher-Pipeline: arrangement of builder types
...it turns out that we actually do not need to wrap TreeExplorer
on the builder types, because basically there is only a single active
builder type, and the complete processing pipeline can be assembled
in a single terminal function.

The type rebinding problem can thus be solved just by a simple
marker struct, which inherits from a template parameter
2023-06-04 02:01:07 +02:00
71ea10bf21 Dispatcher-Pipeline: implement the frame-tick core
splitting into a sequence of builder types seems to have done the trick
2023-06-03 18:38:37 +02:00
81ee9a2e67 Dispatcher-Pipeline: builder type rebinding problems
...hard to tackle...
The idea is to wrap the TreeExplorer builder, so that our specific
builder functions can delegated to the (inherited) generic builder functions
and would just need to supply some cleverly bound lambdas. However,
resulting types are recursive, which does not play nice with type inference,
and working around that problem leads to capturing a self reference,
which at time of invocation is already invalidated (due to moving the
whole pipeline into the final storage)
2023-06-03 03:44:22 +02:00
94fe4a4bec Dispatcher-Pipeline: draft builder-API
...which leads to the next daunting problems:
- we need some mocked ModelPort and DataSink placeholders
- we need a way how to inherit from a partial TreeExplorer pipeline
2023-06-02 05:32:15 +02:00
dda37365cc Dispatcher-Pipeline: direct implementation of frame-tick
start with demonstration of base technique in test setup
2023-06-01 18:09:26 +02:00
ad173540d9 Library: allow for a stop condition in iterator pipeline
...introduced in preparation for building the Dispatcher pipeline,
which at its core means to iterate over a sequence of frame positions;
thus we need a way to stop rendering at a predetermined point...
2023-06-01 16:48:27 +02:00
fbfbd2a078 Dispatcher+Scheduler: decision to dispose of the TimeAnchor
several years ago, it seemed like a good idea to incorporate
the link between nominal time and wall-clock time into a dedicated
anchor point, which also regulates the continued frame planning.

But it turned out that such a design mixes up several concepts
and introduces confusion regarding the meaning of "real time"
- latency can not be reasonably defined for a whole planning chunk
- skipping or sliding due to missed deadlines can not reasonably handled
  within such an abstract entity; it must be handled rather at the
  level of a playback process
- linking the frame grid generation directly to a planning chunk
  undercuts the possible abstraction of a planning pipeline
2023-05-31 03:27:13 +02:00
87f40c8169 Dispatcher+Scheduler: Requirement analysis and planning work 2023-05-29 04:43:10 +02:00
b4c0ffab25 Job-Planning: Analysis for the next step
...which is build a »Job planning pipeline« step by step
in a test setup, and then factor that out as RenderDrive,
to supersede the existing CalcPlanContinuation and get
rid of the Monads this way...

Challenges
- there is a inconsistency with channel usage
- need to establish a way how to transport the output-Sink into the JobFunctor
- need a way to propagate the current frame number to the next planning chunk
2023-05-26 04:20:12 +02:00
e33689e5d6 Job-Planning: verify and complete the build-up of mock structures (see #1294)
The prototypical setup of data structures and test support components
is largely complete by now — with the exception of the `MockDispatcher`,
which will be completed while moving to the next steps pertaining the
setup of a frame dispatch pipeline.

 * the existing `DummyJob` was augmented to allow verification of
   association between Job and `JobTicket`
 * the existing implementation of `JobTicket` was verified and augmented
   to allow coverage of the whole usage cycle
 * a `MockJobTicket` was implemented on top, which can be generated
   from a symbolical test specification (rather than from the real
   Fixture data structure)
 * a complete `MockSegmentation` was developed, allowing to establish
   all the aforementioned data structures without an actual backing
   Render Engine. Moreover, `MockSegmentation` can be generated
   from the aforementioned symbolic test specification.
 * as part of this work, an algorithm to split an existing Segmentation
   and to splice in new segments was developed and verified
2023-05-24 03:38:12 +02:00
4f37b0412c Job-Planning: finally complete the MockSegmentation tests
Last testcase: add deeply nested Prerequisites.
Turns out that the allocator must be able to handle
re-entrant allocations, which std::deque can not fulfil.
Thus using std::list here for the Mock implementation.

In the end, the real allocations will be done by our custom
allocator (AllocationCluster), which can be arranged easily
to support re-entrant allocation calls (since the whole point
is to just place those objects into a pre-allocated large block
and only de-allocate them later in one sway. Thus the allocator
does not need to wait for the object constructor to finish, which
trivially allows for re-entrant calls)
2023-05-23 06:40:18 +02:00
94cec423d0 Job-Planning: switch to processing references
...which uncovers further deeply nested problems,
especially when referring to non-copyable types.

Thus need to construct a common type that can be used
both to refer to the source elements and the expanded elements,
and use this common type as result type and also attempt to
produce better diagnostic messages on type mismatch....
2023-05-23 01:08:05 +02:00
bf6951afcf Job-Planning: verify pipeline can now be constructed (after bugfix)
verify by in-depth investigation that all nested typedefs are now properly constructed
2023-05-23 01:07:53 +02:00
0df0fd001e Library: fix follow-up problems with const correctness
...the improved const correctness on STL iterators uncovered another
latent problem with out diagnositc format helper, which provide
consistently rounded float and double output, but failed to take
CV-qualifiaction into account
2023-05-23 01:07:53 +02:00
e176e54004 Library: adjust and fix semantics of nested 'value_type' binding
This is a subtle and far reaching fix, which hopefully removes
a roadblock regarding a Dispatcher pipeline: Our type rebinding
template used to pick up nested type definitions, especially
'value_type' and 'reference' from iterators and containers,
took an overly simplistic approach, which was then fixed
at various places driven by individual problems.

Now:
 - value_type is conceptually the "thing" exposed by the iterator
 - and pointers are treated as simple values, and no longer linked
   to their pointee type; rather we handle the twist regarding
   STL const_iterator direcly (it defines a non const value_type,
   which is sensible from the STL point of view, but breaks our
   generic iterator wrapping mechanism)
2023-05-23 01:07:53 +02:00
67468f15d5 Job-Planning: Attempt to build a prerequisite-Pipeline failed -- investigate why
To complete the mock setup, the next step would be to extend the GenNode-based spec langage
to allow defining prerequisite Mock-JobTickets. Setting this up seems rather straight forward --

however, defining a simple testcase to cover this extension runs into surprisingly tricky problems..
- for one, the singleValIterator from Itertools has serious difficulties handling references
- but even more surprising, it seems impossible to make the "prerequisites iterator"
  fit into the Tree-Explorer framework (which I intend to use as replacement
  for the monadic approach)

after some extended analysis of generic types and template instances,
it seems that not TreeExplorer as such is the primary problem, but rather
there is a conceptual mismatch somewhere deep down in Itertools or Iter-Adapter
2023-05-23 01:07:07 +02:00
27a8e91fa2 Job-Ticket: consider how to deal with channels and prerequisites
By reasoning and analysis I conclude that the differentiation into
multiple channels is likely misplaced in JobTicket; it belongs ratther
into the Segment and should provide a suitable JobTicket for each ModelPort

Handling of prerequisites also needs to be reshaped entirely after
switching to a pipeline builder for the Job-planning pipeline; as
preliminary access point, just add an iterator over the immediate
prerequisites, thereby shifting the exploration mechanism entirely
out of the JobTicket implementation
2023-05-11 22:47:56 +02:00
db87de3c92 Segmentation: complete implementation of mock / test stub setup
We are now able to build and refine a segmentation
and to get distinct MockJobs from these segments
and to verify invocation of a specific job
2023-05-10 15:00:10 +02:00
a940cd25bc Library: extract helper for unloading a sequence into a tuple 2023-05-10 14:49:51 +02:00
566f73de2a Segmentation: verify standard cases..
Testcase: A simple Sementation with a single and bounded Segment


As aside, figured out how to unpack an iterator such as to
tie a fixed number of references through a structural binding:

auto const& [s1,s2,s3] = seqTuple<3> (mockSegs.eachSeg());
2023-05-10 03:59:46 +02:00
e5cdb86ac3 Segmentation: integrate SplitSplice and build mock-segmentation
...now able to build a mock segmentation which issues dummy jobs,
and is wired such as to verify the right job is invoked for each segment.

And this allows to build and verify the Dispatcher,
without being able to invoke actual render jobs yet.
2023-05-05 03:46:42 +02:00
256045f91d Segmentation: add test case to cover instance management
- only the parts actually touched by the algo will be re-allocated
- when a segment is split, the clone copies carry on all data


Library: add function to check for a bare address (without type info)
2023-05-05 01:34:41 +02:00
52dad70753 Segmentation: Split-Splice algorithm implementation complete
...and verified including corner cases...
2023-05-04 15:59:17 +02:00
3f2f3db568 Segmentation: verify Split-Splice standard cases
Algorithm seems to work basically...
There was a copy-n-paste error in the treatment of the Successor,
leading to spurious duplication in some cases
2023-05-04 14:43:40 +02:00
c8f879ff3f Segmentation: extract invocation for test
typically you'd write an custom adaptor function
to define all the λ-bindings for the Algorithm
2023-05-04 12:01:25 +02:00
f9a4d6134c Segmentation: setup for simple demo test case works
due to having extracted the Algo implementation as template,
we can now instantiate it for isolated tests with simple integer intervals
2023-05-04 02:07:38 +02:00
00ca84a2aa test-helper for comparison with expected (string) result
...this is something I should have done since YEARS, really...

Whenever working with symbolically represented data, tests
typically involve checking *hundreds* of expected results,
and thus it can be really hard to find out where the
failure actually happens; it is better for readability
to have the expected result string immediately in the
test code; now this expected result can be marked
with a user-defined literal, and then on mismatch
the expected and the real value will be printed.
2023-05-04 00:48:29 +02:00
476c0f6493 Segmentation: build test-setup for Split-Splice-Algo
use simple intervals over integer numbers as test setup
- detect possible memory leaks
- detect corrupted Segmentation sequences
2023-05-03 15:27:46 +02:00
b582c35c9f Segmentation: structure analysis for splitSplice operation
There are 12 distinct cases regarding the orientation of two intervals;
The Segmentation::splitSplice() operation shall insert a new Segment
and adjust / truncate / expand / split / delete existing segments
such as to retain the *Invariant* (seamless segmentation covering
the complete time axis)
2023-05-02 04:29:34 +02:00
a807705185 Segmentation: draft simple mock-based setup for tests
- how to pass-in a specification given as GenNode
- now this might be translated into a MockJobTicket allocated in the MockSegmentation

Unimplemented: actually build the Segment with suitable start/end time
2023-05-01 17:02:11 +02:00
56405b2e2d Job-Planning: simulate backing by specific JobTicket
right now we're lacking a complete working implementation of render node invocation,
and thus the Dispatcher implementation can only be verified with the help
of mocked jobs. However, at least a preliminary implementation of tagging the
invocation instance is available, and thus we're able to verify that
a given job instance indeed belongs to and is "backed" by a specific JobTicket.

This is prerequisite for building up a (likewise mocked) Fixture datastructure,
and this in turn was meant to form the basis for attacking an actual Scheduler
implementation, followed by a real render node invocation.
2023-05-01 14:07:21 +02:00
f6fbc15e5f Job-Planning: provide stub implementation for NOP job (see #1296)
- can now create a Job from JobTicket::NIL
- on invocation this Job will to nothing

Only when the first real output backend is implemented,
we can decide if this simplistic implementation is enough,
or if an empty output must be explicitly generated...
2023-05-01 01:48:36 +02:00
fef0c05b64 Job-Planning: base implementation of job instance creation
* using a simplified preliminary implementation of hash chaining (see #1293)
 * simplistic implementation of hashing for time values (half-rotation)
 * for now just hashing the time into the upper part of the LUID

Maybe we can even live with that implementation for some time,
depending on how important uniform distribution of hash values is
for proper usage of the frame cache.

Needless to say, various further fine points need more consideration,
especially questions of portability (32bit anyone?). Moreover, since
frame times are typically quantised, the search space for the hashed
time values is drastically reduced; conceivably we should rather
research and implement a good hash function for 128bit and then combine
all information into a single hash key....
2023-04-30 22:33:42 +02:00
8aa0c258ba Job-Planning: investigate invocation of jobs
...using the MockJobTicket setup as point of reference,
since the actual invocation of render nodes will only be drafted
later in this "Vertical Slice" integration effort...
2023-04-30 02:18:56 +02:00
685b5beba6 Segmentation: simple implementation of time-based access
- introduce a JobTicket::NOP (null-object pattern)
- assuming that the function splitSplice() will retain complete coverage allways

Remark:
`Fixture::getPlaylistForRender()` is a leftover from the very early implementation drafts.
This function was more or less based on the way Cinelerra works; it is clear by now
that Lumiera can not possibly work this way, given that we'll build a low-level model
and dispatch precompiled render jobs....
2023-04-27 22:30:49 +02:00
d73b316ead Segmentation: consider preliminary data structure
...and consider how that can be extended later into the full
structure, which has to support a transactional switch
2023-04-27 19:38:37 +02:00
d58174db4d Segmentation: reorganise namespaces
The Fixture and the low-level model backbone deserve a distinct namespace on their own.
Since it's built by the Builder from the Session contents, and also used by the frame dispatch,
we can expect dependence on some types from Steam-Layer, and thus this namespace
needs to reside in Steam-Layer rather, while the actual low-level Model
might become part of Vault-Layer, creating a hierarchy of data structures.

(Remark: likely also the session related namespaces will need a reorganisation)
2023-04-25 18:27:16 +02:00
90593776f6 Segmentation: a plan to bootstrap into the required structure
The idea is to escape a "design deadlock" by using a test-driven prototype
implementation of the data structure to back a further development
of the Dispatcher and Scheduler implementation, which then can be used
to gradually elaborate and switch over to an actual implementation
data structure
2023-04-25 13:40:20 +02:00
b93a9a7985 Job-Planning: elaborate mock setup for render job 2023-04-21 05:29:10 +02:00
305eb825af Job-Planning: first testcase - empty JobTicket
...requires a first attempt towards defining a `JobTiket`.
This turns out quite tricky, due to using those `LinkedElements`
(intrusive single linked list), which requires all added records
actually to live elsewhere. Since we want to use a custom allocator
later (the `AllocationCluster`), this boils down to allocating those
records only when about to construct the `JobTicket` itself.

What makes matters even worse: at the moment we use a separate spec
per Media channel (maybe these specs can be collapsed later non).
And thus we need to pass a collection -- or better an iterator
with raw specs, which in turn must reveal yet another nested
sequence for the prerequisite `JobTickets`.

Anyhow, now we're able at least to create an empty `JobTicket`,
backed by a dummy `JobFunctor`....
2023-04-20 23:55:02 +02:00
856d8a3b51 Library: allow to reverse intrusive single linked list
Looks like we'll actually retain and use this low-level solution
in cases where we just can not afford heap allocations but need
to keep polymorphic objects close to one another in memory.

Since single linked lists are filled by prepending, it is rather
common to need the reversed order of elements for traversal,
which can be achieved in linear time.

And while we're here, we can modernise the templated emplacement functions
2023-04-20 18:53:17 +02:00
d341f003ca Job-Planning: attempt to stake claims
desperately trying to move forward and define a minimal first test case...
2023-04-18 20:02:36 +02:00
1dd1ec0e79 Job-Planning: decision how to rework bottom-up and test driven
- build the reworked Job-planning pipeline more or less from scratch
- back that with mocked `Dispatcher` and `JobTicket`
- then transfer this into a `RenderDrive`, which can be tested as well
- could continue then to a `CalcStream` integration test....
2023-04-17 17:10:53 +02:00
bcd2b3d632 PlaybackVerticalSlice: design analysis for Frame Dispatcher and Scheduler
- decision: the Monad-style iteration framework will be abandoned
- the job-planning will be recast in terms of the iter-tree-explorer
- job-planning and frame dispatch will be disentangled
- the Scheduler will deliberately offer a high-level interface
- on this high-level, Scheduler will support dependency management
- the low-level implementation of the Scheduler will be based on Activity verbs
2023-04-14 04:43:39 +02:00
dfcb17b890 GUI: close out rework of top-level and timeline
This finishes a long lasting effort to rework the top-level of the Lumiera GTK UI,
to adapt to GTK-3 and the new asynchronous message based architecture.

Special credits and thanks to
 * Joel Holdsworth
 * Stefan Kangas

Without their relentless foundational work, the Lumiera UI could
never be where it is now. Even if some code was rewritten and several
parts of the old GTK-2 implementation are now obsolete, numerous ideas
solutions and inspirations were drawn from those early contributions
and live on as part of the reworked GUI.
2023-03-22 02:58:04 +01:00
52d3231226 Timeline: finish ZoomWindow implementation and boundrary tests 2022-12-18 03:47:40 +01:00
e436023ef9 Timeline: properly handling maximal zoom-in and alignment to µ-ticks 2022-12-17 17:47:10 +01:00
b1514f6632 Timeline: properly handling extreme scroll-steps 2022-12-17 01:15:34 +01:00
3893968502 Timeline: improve handling of window size changes at extreme positions
...again and again surprising how much inconsitencies can hide in just some lines of code...
2022-12-16 19:59:47 +01:00
77bb156615 Timeline: verify handling of extreme time offsets 2022-12-16 02:23:20 +01:00
5e595c57ca Timeline: automatically orient and shift into allowed time domain
Note: changing behaviour of TimeSpan to possibly flip start and end,
and also to use Offset as Offset and then re-orient,
since this seems the least surprising behaviour.

These changes carry over into changed default and limiting
on ZoomWindow constructor and various mutators, and most
notably shifting the time span always into allowed domain.
2022-12-14 03:00:07 +01:00
777024ee40 Timeline: resolve yet another insidious corner case at maximum zoom
...the implementation was way too naive; in some cases we could go
into an infinite loop. In the end, using Newton approximation was not
necessary (and thus there is no loop anymore), but it helped me get
at a much better solution with very small error margin on average case.

All these corner cases are obviously "academic" to some degree,
but it turns out there is no clear-cut point where you'd be able
just so set a limit and be sure that fractional integer arithmetic
works flawless in all cases.

Thus the choice is
 - give up (fractional) integers and work with floats and have to
   deal with error accumulation
 - or do something as chosen here, namely add a boundary zone, where
   fractional integer arithmetic can be kept under control, while admitting
   small errors, and in turn get the absolutely precise integers in all
   everyday standard cases
2022-12-13 01:21:18 +01:00
c31522c236 Timeline: define better internal zoom-out limit
The value used previously was too conservative, and prevented ZommWindow
from zooming out to the complete Time domain. This was due to missing the
Time::SCALE denominator, which increaded the limit by factor 1e6

In fact the code is able to handle even this extremely reduced limit,
but doing so seems over the top, since now detox() kicks in on several
calculations, leading to rather coarse grained errors.

Thus I decided to use a compromise: lower the limit only by factor 1000;
with typical screen pixel widths, we can reach the full time domain,
while most scaling and zoom calculations can be performed precisely,
without detox() kicking in. Obviously this change requires adjusting
a lot of the test case expectations, since we can now zoom out maximally.
2022-12-10 04:26:22 +01:00
40f003a962 Timeline: stress-test with excessive zoom-out reveals further weakness
As it turns out, the calculation path initially choosen for the mutateScale(Rat)
was needlessly indirect, and also duplicated several of the safeguards,
meanwhile implemented way better in conformWindowToMetric(Rat)

Thus, instead of relatively re-scaling the window, now we just
limit the given zoomFactor and pass it to conformWindowToMetric()
2022-12-09 23:13:27 +01:00
ce3713d872 Timeline: now able to increase to maximum pixel count
There is a built-in limitation, which now is even
lowered to 100000 pixels horizontally.

With the techniques introduced in this changeset, it seems possible
to support more -- yet this would be a case of unnecessary genricity;
handling such large numbers will drive more computations into the
danger zone, and doing so incurs cost in terms of testing and debugging.

Placing that into context, contemporary displays are not even 4K on
average, and it does not look likely even for cinema display to go
way beyond 8k -- so yes, I want display hardware with 100000 pixels!!


The key takeaway of this changeset:
 - can calculate px = trunc(zoomFactor * duration) step wise,
   even when the direct calculation would lead to wrap-around
 - can safely adjust and fix the zoomFactor using Newton approximation
2022-12-09 19:37:35 +01:00
068549dace Timeline: now able to handle maximal zoom-out
...even zooming out to span the complete time domain (~19000 years).
But only under the condition that the display window is sufficiently
large in terms of pixels, so we can handle the computation without
glitches.

This should not be a relevant limitation in practice, since a window
size of some 100 pixels is enough to handle Duration::MAX. Needless to add
that it's hard to imagine a media timeline of such tremendous size...
building on these Library changes, plus the safe-add function
developed some days ago, it is now possible to mark a large displacement
as `time::Offset`, and apply this to yield any valid time position,
even extreme negative values
2022-12-08 18:06:37 +01:00
006758f349 Library/Timeline: now able to scroll to extreme positions (closes #1263)
...building on these Library changes, plus the safe-add function
developed some days ago, it is now possible to mark a large displacement
as `time::Offset`, and apply this to yield any valid time position,
even extreme negative values
2022-12-05 03:34:04 +01:00
13adc56f34 Library: rectify confusingly named function on the Grid API
The APIs for time quantisation were drafted in an early stage of the project
and then never followed-up. Especially Grid::gridAlign has no
real-world usage yet, and is only massaged in some tests.

When looking at QuantiserBasics_test, I was puzzled and led astray,
since this function suggests to materialise a continuous time into
a quantised time -- which it doesn't (there is another dedicated
function Quantiser::materialise() to that end); so, without engaging
into the discussion if this function is of any use, I'll hereby
choose a name better reflecting what it does.
2022-12-05 01:05:23 +01:00
50c602ec3f Library: rectify clipping of time::Duration (see #1263)
This is a deep refactoring to allow to represent the distance
between all valid time points as a time::Offset or time::Duration.

By design this is possible, since Time::MAX was defined as 1/30 of
the maximum value technically representable as int64_t. However,
introducing a different limiter for offsets and durations turns
out difficult, due to the inconsistencies in the exiting hierarchy
of temporal entities. Which in turn seems to stem from the unfortunate
decision to make time entities immutable, see #1261

Since the limiter is hard wired into the `time::TimeValue` constructor,
we are forced to create a "backdoor" of sorts, to pass up values
with different limiting from child classes. This would not be so
much of a problem if calculations weren't forced to go through `TimeVar`,
which does not distinguish between time points and time durations.

This solution rearranges all checks to be performed now by time::Offset,
while time::Duration will only take the absolute value at construction,
based on the fact that there is no valid construction path to yield
a duration which does not go through an offset first.

Later, when we're ready to sort out the implementation base of time values
(see #1258), this design issue should be revisited
- either we'll allow derived classes explicitly to invoke the limiter functions
- or we may be able to have an automatic conversion path from clearly
  marked base implementation types, in which case we wouldn't use the
  buildRaw_() and _raw() "backdoor" functions any more...
2022-12-05 00:58:32 +01:00
4d79bdce5f Timeline: sophisticated helper function to resolve problems with numeric precision
While the calculation was already basically under control, I just was not content
with the achieved numeric precision -- and the fact that the test case in fact
misses the bar, making it difficult do demonstrate that the calculation
is not derailed. I just had the gut feeling that it must be somehow possible
to achieve an absolute error level, not just a relative error level of 1e-6

Thus I reworked this part into a generic helper function, see #1262

The end result is:
 * partial failure. we can only ''guarantee'' the relative error margin of 1e-6
 * but in most cases not out to the most extreme numbers, the sophisticated
   solution achieves much better results way below the absolute error level of 1µ-Tick

Thus with using rational numbers, we have now a solution that is absolutely precise
in the regular case, and gradually introduces errors at the domain bound
but with an guaranteed relative error margin of 1e-6 (== Time::SCALE)
2022-12-02 22:23:14 +01:00
289f92da7e Timeline: safely calculate sum/difference of large fractional times
...in a similar vein as done for the product calculation.
In this case, we need to check the dimensions carefully and pick
the best calculation path, but as long as the overall result can
be represented, it should be possible to carry out the calculation
with fractional values, albeit introducing a small error.

As a follow-up, I have now also refactored the re-quantisation
functions, to be usable for general requantisation to another grid,
and I used these to replace the *naive* implementation of the
conversion FSecs -> µ-Grid, which caused a lot of integer-wrap-around

However, while the test now works basically without glitch or wrap,
the window position is still numerically of by 1e-6, which becomes
quite noticeably here due to the large overall span used for the test.
2022-12-01 23:23:50 +01:00
7007101357 Timeline: safely calculate the fraction of a very large timespan
...using a requantisation trick to cancel out some factors in the
product of two rational numbers, allowing to calculate the product
without actual multiplication of (dangerously large) numbers.

with these additional safeguards, the anchorWindowAtPosition()
succeeds without Integer-wrap, but the result is not fully correct
(some further calculation error hidden somewhere??)
2022-11-29 02:00:41 +01:00
b898f1514b Timeline: safeguard agains injecting a poisonous metric factor 2022-11-26 19:45:10 +01:00
90aba4df09 Timeline: demonstrate safeguards against reversed and toxic input 2022-11-18 02:55:28 +01:00
cfe3a6618f Lib: cover re-quantisation helper
...which I intend to use for sanitising poisonous rational numbers,
as prerequisite for handling divisor based time scales in the ZoomWindow
2022-11-15 02:13:57 +01:00
ce1220ee72 Lib: test coverage for rational-int corner cases and integer-log
- detailed documentation of known problematic behaviour
  when working with rational fractions
- demonstrate the heuristic predicate to detect dangerous numbers

- add extensive coverage and microbenchmarks for the integer-logarithm
  implementation, based on an example on Stackoverflow. Surprising result:
  The std::ilog(double) function is of comparable speed, at least for
  GCC-8 on Debian-Buster.
2022-11-14 05:20:37 +01:00
8ab0e1acb5 Lib: consider method to sanitise a poisonous rational
Especially rational numbers with large denominator can be insidious,
since they might cause numeric overflow on seemingly harmless operations,
like adding a small number.

A solution might be to *requantise* the number into a different,
way smaller denominator. Obviously this is a lossy operation;
yet a small and controlled numeric error is always better than
an uncontrolled numeric wrap-around.
2022-11-13 16:52:12 +01:00
d3fc6fd4de Timeline: Analysis of possibly toxic input values
- protection against negative numbers seems adequate
- a possible concern are handling of very large time spans
- definitively have to guard against "poisonous" fractions
  (e.g. n / INT_MAX)
2022-11-12 17:35:47 +01:00
cc16953fd8 Timeline: implement and verify ZoomWindow change notification 2022-11-11 16:30:27 +01:00
3f396ef3b2 Timeline: ZoomWindow now passes all tests according to spec
- some test definitions were simply numerically wrong
- changed some aspects of the specified behaviour, to be more consistent
  + scrolling is more liberal and always allows to extend canvas
  + setting window to a given duration expands around anchor point
2022-11-11 03:06:33 +01:00
93de063ae5 Timeline: fix next ZoomWindow_test failure 2022-11-10 21:00:00 +01:00
fa4fbe5225 Timeline: clarify or fix further ZoomWindow_test failures
- forgot the bias towards the next larger grid aligned size
- implement safeguard against empty window
2022-11-09 03:25:35 +01:00
a9df13f078 Timeline: define basic ZoomWindow setters on top of the normalisation
Rearrange the internal mutator functions to follow a common scheme,
so that most of the setters can be implemented by simple forwarding.
Move the change-listener triggering up into the actual setters.

This makes further test cases pass
 - verify_setup
 - verify_calibration
...implying that the pixel width is now retained
and basic behaviour matches expectations
2022-11-07 03:24:04 +01:00
292be817b7 Timeline: investigate problem with numeric overflow in fractional arithmetic
Extensive tests with corner cases soon highlighted this problem
inherent to integer calculations with fractional numbers: it is
possible to derail the calculation by numeric overflow with values
not excessively large, but using large numbers as denominator.
This problem is typically triggered by addition and subtraction,
where you'd naively not expect any problems.

Thus changed the approach in the normalisation function, relying
on an explicitly coded test rather, and performing the adjustment
only after conversion back to simple integral micro-tick scale.
2022-11-07 00:19:28 +01:00
5ed5905d7f Timeline: work out a scheme of Invariants and Normalisation for the ZoomWindow
Getting all those requirements translated into code turns out to be a challenging task;
and the usual ascent to handle such a situation is to define **Invariants**
in conjunction with a normalisation scheme; each manipulation will then be
translated into invocation of one of the three fundamental mutators,
and these in turn always lead into the common normalisation sequence.

__Invariants__
- oriented and non-empty windows
- never alter given pxWidth
- zoom metric factor < max zoom
- visibleWindow ⊂ Canvas
2022-11-06 02:46:11 +01:00
f2ef893adb Timeline: complete specification of ZoomWindow expected behaviour
Writing this specification unveiled a limitation of our internal
time base implementation, which is a 64bit microsecond grid.
As it turns out, any grid based time representation will always
be not precise enough to handle some relevant time specifications,
which are defined by a divisor. Most notably this affects the precise
display of frame duration in the GUI, and even more relevant,
the sample accurate editing of sound in the timeline.

Thus I decided to perform the internal computation in ZoomWindow
as rational numbers, based on boost::rational

Note: implementation stubbed only, test fails
2022-11-04 03:40:36 +01:00
f1b3f4e666 Timeline: reconsider time handling and Stage/Steam integration
This ZoomWindow_test highlights again the question about the intended usage
of the Lumiera time entities. In which way do we want to perform time calculations,
and under which circumstances is it adequate to perform arithmetic on
raw time values?

These questions made me think about rather far reaching concerns regarding
subsidiarity and implicit or explicit usage context. Basically I could
reconfirm the design choices taken some years ago -- while I must admit
that the project is headed towards a way larger scale and more loose
coupling of the parts, than I could imagine several years ago, at the
time when the design started...

As a side note: we can not avoid that some knowledge about the time implementation
leaks out from the support lib; time codes themselves are tightly coupled
to the usage scenario within the session and can not be used as means
for implementing UI concerns. And the more generic time frameworks,
like std::chrono (as much as it is desirable to have some integration here)
will not be of any help for most of our specific usage patterns.
The reason is, for film editing we do not have a global time scale,
rather the truth is when the film starts....
2022-10-30 23:12:34 +01:00
7145d0d9ce Timeline: ZoomWindow implementation draft
implement the first test case: nudge the zoom factor
⟹ scale factor doubled
⟹ visible window reduced to half size
⟹ visible window placed in the middle of the overall range
2022-10-30 01:31:25 +02:00
b3fe6e16c6 Timeline: requirement analysis to define the ZoomWindow (see #1196) 2022-10-29 01:59:42 +02:00
f393780845 Lib: fix a bug with diagnostic output
The header "format-cout.hpp" offers a convenience function
to print pretty much any object or data in human readable form.
However, the formatter for pointers used within this framework
switched std::cout into hexadecimal display of numbers and failed
to clean-up this state.

Since the "stickyness" of IOS stream manipulators is generally a problem,
we now provide a RAII helper to capture the previous stream state and
automatically restore it when leaving the scope.
2022-09-27 01:51:21 +02:00
ed7e3b4b32 ElementBox: extract builder qualifier support as library implementation
Complete the investigation and turn the solution into a generic
mix-in-template, which can be used in flexible ways to support
this qualifier notation.

Moreover, recapitulate requirements for the ElementBoxWidget
2022-08-28 23:36:27 +02:00