Within the reworked design of the `OutputSlot` a proxy implementation
of the `BufferProvider` API is used (class `OutputBufferProxy`).
After analysis of the invocation patterns, I concluded that it seems adequate
to attach the special-purpose metadata generated by this integration
to the generic system of thread-local replicated `BufferMetadata` storage.
However, since thread-local instances are effectively static(global),
I was concerned that unit-testing might be impeded by accumulation of
stale metadata in the store linked to the main thread. Thus I created
a special configuration variant to retain the previously used setup
for the sake of unit testing (which applies especially to use of the
`DiagnosticOutputSlot`). Using a `SimpleBufferStateRegistry` for in
that case has the benefit that simply a local instance can be used
in the test and all metadata will be discarded afterwards.
Furthermore the factory syntax for obtaining an `OutputSlot` was improved,
drawing on the recent changes to `lib::ScopedCollection`.
Also I looked through some of the documentation and updated
the UML sequence diagram to reflect `OutputSlot`'s improved internal structure
With this third test case, a fairly continuous processing
can be demonstrated, while also coordinating allocations
in a decentralised way.
The implementation behaves correct, and seemingly with
satisfactory performance -- while there is certainly
some room for improvement: with the processing pattern
in this test, a nominal computation load of 4 seconds
can be processed with 4 cores in ~ 1.3 seconds
Yet only in a simple way, by pressing with 100 threads....
This test runs for about 10 seconds, shows a massive allocation
of more than 1 GiB, but comes down clean, without deadlocks,
using all cores and returning all memory blocks at the end.
This test uncovered some missing safety checks,
and it highlighted (again) that this memory management scheme is
elaborate and potentially fragile: I had to deal with carried-on
state in the local metadata registry associated with the main thread
(in fact I did not ever want that registry to be used, but turns out
that the initialisation check, that runs in the constructor, already
pulls up an instance for the main thread, which carries stale state
after the test case is complete)
This is the meat of this test case...
- generate a ''random walk'' with buffer allocation requests
- attempt to make the load pattern nasty, since the goal of this test
is to verify correctness of the complex interplay between local pools
and central allocator.
''correct pattern was verified single-threaded, with print statements''
The intention is to utilise this helper class, that was developed for the
scheduler stress testing; but this time, it should access the actual allocated
render buffers to generate the load.
This implies to use buffers of a significant size; it turned out that
the existing implementation was imprecise, insofar it always used full multiples
of the configured buffer size, which defeats the purpose of performing
a precisely calibrated number of calculation steps. As long as the buffer size
would remain comparatively small, this drift does not matter much.
Thus I changed the implementation now to ensure that exactly the calibrated
number of steps is performed, using a modulus to map it to the buffer memory.
Furthermore, I plan to use a random-walk pattern in the worker thread
to select from a collection of pre-established »buffer types«, similar
to what happens in the actual engine
The mechanism is tuned now to engage rather quickly
whenever excess capacity builds up in a local allocator pool,
but when all buffers are used most of the time, they will be retained.
Can now demonstrate by integration test that capacity is sent back.
The goal is to develop a heuristic when to trigger clean-up of the local pool.
For developing this heuristic, I'll use a unit-test
- represent "allocations" by dummy buffer pointers
- use a queue and a stack to push those "allocations" around
- the "global pool" is just a queue with such dummy pointers
- next step is to define some λ-primitives
- finally, a λ-DSL to allow to perform "N operations of kind X"
This is a fall-back situation,
in case the allocation ''was not announced beforehand''
A direct locked global access is used then, which is verified by this test
...need an elaborate test setup, with the test worker thread
and the main thread proceeding in lockstep, otherwise the
effect would not be visible. Furthermore, the diagnostic
access for LocalMemPool can be re-used here, thereby
mapping requests onto the pool for the current thread.
This makes the test case pass, validating the theory.
Each `AllocReceiver` now integrates a `lib::LiveMark` instance,
and the callbacks are dispatched using the corresponding
`LiveMark::SafeHandle`, so that it is no longer possible
to dispatch allocation messages to a defunct receiver.
Furthermore, the `LiveMark` is explicitly disengaged prior
to unwinding a local pool, so that no further pending requests
can be handled ans dispatched concurrently. The local pool
sends back all the remaining allocations via async queue
to the `EngineBufferManager`
Not that such a shutdown can be triggered any time,
since an exception or failure might cause a worker thread
to unwind, thereby destroying the thread-local `LocalMemPool`
without further notice.
...even while this situation is unlikely, it needs to be considered:
A worker thread might terminate unexpectedly, due to an error;
all thread-local objects will be destroyed automatically — while the
`EngineBufferManager` might be processing further allocations requested
by that very thread, and dispatch them towards an already destroyed `LocalMemPool`
This turns out to be a nasty problem, since the whole setup was created
to allow the workers to operate independently from each other; any handshake
or registration/deregistration mechanism would require a lock of sorts,
thereby defeating the whole purpose of the setup.
The best solution I could find is to rely on a instance marker hash,
that can be checked with an atomic read prior to each dispatch of a
memory allocation. The destructor of the local pool would then be able
to ''disengage'', by invalidating that hash.
To evaluate this solution, I developed this mechanism
as a free-standing library component, called `LiveMark`
Basically the setup should be symmetrical to the `BufferStage` setup,
which implies to install the `LocalMemPool` as a thread-local »slice«.
However, in addition we need a mechanism to trigger an automatic clean-up
of the local pool whenever a worker-thread terminates, with the additional
complication that thread-local variables are effectively static.
Since `LocalMemPool` is written such as to abstract from the setup environment,
we need to inject a shutdown function from the setup within `LocalSlice`,
thereby directly accessing the `EngineBufferManager` via dependency-injection.
Most relevant parts of the `LocalBufferStore` are already available
and just need to be connected. One point poses a challenge however:
when and how to trigger the heuristic clean-up of the local pool,
so that ''on average neither too much memory is wasted, nor
to much unnecessary allocations'' are performed...
Signed-off-by: Ichthyostega <prg@ichthyostega.de>
The test performs a massively concurrent scenario
that presses with 100 threads and 1 MiB allocation for each.
This highlighted my still limited understanding of the situation
and leads to a much better solution, where, on-demand, some worker
performs all the accrued allocation and management work.
So effectively we do not even need to use the synchronous API
Implementing the `EngineBufferManager` the desired way,
with the ability to post allocation requests asynchronously,
raises the difficult question of who will ultimately perform
the allocation / deallocation and further management work.
The point is that this management effort takes up capacity
originally assigned for the render workers themselves, either
by implicitly pushing aside those threads, or as a dedicated
processing assigned to some worker.
In the current stage of development it seems impossible to decide
upon the proper mechanism to schedule this excess work explicitly.
The solution proposed hereby however circumvents this dilemma,
insofar it allows to implement the synchronous, fallback processing
in terms of the asynchronous processing, just performed exclusively
within the current thread. Unless some processing will be scheduled
explicitly (which can be added as performance optimisation later on),
the handling of a direct allocation request will ''fold down'' into
the current worker eventually, thereby using it's processing capability
without the need of any explicit scheduling.
- use a base class with the lock-free queue as building block
- both `LocalMemPool` and `EngineBufferManager` are built on top,
allowing them to exchange `Alloc` entries asynchronously
- ''Allocation requests'' include a receiver callback,
so that a newly allocated memory block can be dispatched
While all of this aggregate can be considered ''deep implementation'',
the structure of dependencies bears danger to get confusing. Even more so,
since an elaborate pooling memory allocator will be added to and interconnected
with this structure.
Now considering the following route:
- `engine-buffer-allocator.hpp` has elements of a common service / interface
- but `engine-buffer-allocator.cpp `is also the most deeply nested implementation backend
- `local-buffer-store.hpp` is a junction point
- `local-mem-pool.hpp` is considered an insolated implementation detail
- ultimately, this will be included and wired in `render-enviroment.cpp`
This implementation and the accompanying test might seem pointless,
yet the point is that the interface is aligned to the emerging
concepts for memory allocators in Lumiera, while also directly
delegating to the std::allocator for the time being.
Before we're able to observe the actual allocator trend and behaviour,
it is pretty much moot to engage into any kind of optimisation;
the generic heap allocator performs quite well on recent C++ runtimes,
and, furthermore, the most important performance tweak is to use
a thread-local pool of recent allocations, which should allow
our jobs to run without impeding each other.
Seems to work now roughly as expected, so that the next layers
of functionality can be built on top. It seems impossible
to determine the viability of this whole approach without
taking the full buffer memory management into use with
a real-world load scenario.
At the begin of each Render Node `pull()`, the render job is expected
to ''announce'' a maximum expected allocation requirement to the `BufferProvider`,
which information is passed-on to the `LocalMemPool` to determine
to which degree it can be satisfied with allocations from that local pool.
Without actual empiric observation, it is impossible to determine
the relevance of this step, how it should be resolved heuristically
against the known contents of the pool, and how much effort to spend
on that step.
Thus I'll settle for the ''simplest thing that can possibly work'' —
which is to check only for close matches and rather rely on additional
allocations, forwarded asynchronously, assuming that the clean-up
heuristics will deal with excess allocations quickly.
- take into account how much memory will be wasted when
using some allocation to satisfy a (smaller) allocation request
- furthermore prefer allocations used frequently in the past
- when an allocation is used, tag it with a score that also
reflects this degree of waste
- select buffers with low score first for clean-up
- use boost::lockfree to implement the in-queue with new allocations
- use a std::list with an internal administrative record
- automatically ingest the in-queue prior to accessing the pool
After several fruitless attempts, I have decided now
to settle for self-contained ''memory pool'' component,
that can be operated **thread-locally** and employs a
**lock-free** message queue for the allotment of memory.
Seemingly the question of how this relates to `BufferMetadata`
must remain unresolved for the time being, until the entire
buffer memory subsystem is available for empirical investigation.
As staring point, it is clear that the `BufferStore` side of the
implementation will rely on a similar structure with a thread-local
base service — which in this case will have to maintain a small pool
of recently used buffer allocations...
...which happens when a buffer type is referred for the first time
from within a new worker thread, while it is already stored in the
global registry; the full key-chain is down-synced then.
...using the first usage situation as guideline,
and verify proper operation through unit-test
- the LocalBufferStage gets a set of thread-local BufferMetadata
- furthermore, it accesses the central hub via dependency-injection
- after each type registration ''from a worker thread'' data is published
As an asside: ensure most metadata registration can ''move'' the TypeHandler
This test setup defines one of the problematic situations
where a synchronisation is necessary.
__TODO__: actually add locking and that synchronisation to LocalBufferStage
What I am building step by step right now is the first draft
of a production-grade render buffer management scheme.
For this I need suitable implementations of `BufferStag` (and `BufferStore`)
These will include a `LocalSlice` to hold thread-local »satelitte services«,
which in turn need a connection to some central coordination hub.
The thread-local instances can access the hub via our `lib::Depend` (I have just
verified that this works as intended, with on-demand creation of the
central instance on first access from some thread. This is important,
since very likely we need to install a dedicated and speficically wired
service instance, which must happen prior to first dependency-request
via `lib::Depend`. And since the worker-threads are created by the Engine,
we get ample opportunity to do so before starting those threads.
Included in this changeset is a first rough draft of the central registry
`EngineBufferMetadata` — I have modelled it analogous to the `SimpleBufferStateRegistry`,
which is the test-level implementation of the `BufferStage` API.
However, we do not want to expose direct buffer allocation operations
on the central hub, and thus I have left out the respecive parts of the API.
Furthermore, I have added an `abandon()` operation, without there beeing
a need for such an operation right now. But since this operation is tricky
and potentially dangerous, I prefer implementing it the correct way,
since I have a clear understanding what this entails at the moment.
The recursive synchronisation up the parent-chain can in most cases be skipped,
by relying on the fact that inconsistent data is never stored in the BufferMetadata
table. If thus an entry is locally present and matches the entry from the
other side, then it can be concluded that its parent must also be already in sync.
Since the intention is to run several thread-local instances of `BufferMetadata`
in a concurrent setup, we need a way to publish and exchange chains of
registered type-keys and metadata entries.
The first step is to develop a mechanic for data exchange,
and to define the rules when such synchronisation is (not) allowed.
During the last days, I started with analysis to find a way
to operate the `BufferProvider`, as implemented thus far,
within a massively concurrent environment.
Since I intend to create thread-local caches,
the initialisation of the `BufferMetadata` store must be static.
It turns out that the root-ID, which acts like a mandator ID,
is only used at a single code path, when creating or accessing
an abstract type-Key. It is thus easy to extract and relocate
this distinguishing ID into the actual `BufferStage` implementation.
Looking through all my work notes, it seems that all the
problematic aspects are now solved in a satisfactory way.
* it is clear how to dependency-inject a `BufferProvider`
* the internals of the `BufferProvider` front-End have been restructured
* there are now two ''pillars'' of the implementation: `BufferStage` and `BufferStore`
* both can be subclassed independently
* `BufferProviderSetup` exposes a proxying-mechanism (used for `DiagnosticBufferProvider`)
* the `BufferMetadata` have been reviewed and some minor details were rectified
* after a re-Design of `OutputSlot`, `BuffHandle` can now be used as unified front-End
* both `DiagnosticBufferProvider` and `DiagnosticOutputSlot` have been rewritten
* they are now independent, self-contained specialisations, layered on top
Overall, what might seem like a detour, was an important step towards
well-balanced structures and a better understanding of the Render Engine.
After an extended round of analysis and cascading reworks,
the long-standing discrepancy between `OutputSlot` and the
''Buffer Provider Protocol'' (#1395) has finally been resolved.
The foundation for this success was laid by means of a restructured
implementation within `BufferProvider` – which has now been
successfully completed and validated.
__note__: regarding hash-standard.hpp see Ticket #1422
Mostly this hack worked -- but it is very likely not needed any more.
Today I had to re-wire a partial specialisation for hashing pointers,
that was added somewhere in the past into the STL.
Now able to reap the rewards: since all that convoluted and tangled
old ''state tracking'' has been resolved and replaced by dedicated
and self-contained diagnostic subclasses, the old dummy BufferProvider
implementation can be detached, using a similar yet very simple
fake implementation.
For context: the `HeapMemBufferStore` is part of the default configuration
and thus automatically used whenever a unit test needs a `BufferProvider`.
This default implementation is not optimised for speed, and actually
never releases any memory, which allows to verify proper handling
of buffer memory allocation.
''This is a breakthrough...''
Using the realigned implementation structure in `BufferProvider`,
and especially the ability to establish an overarching context by
passing an connection-ID through the `LocalTag` in the `BufferMetadata`,
it was possible to resolve the longstanding mismatch buried deep within
the `OutputSlot` implementation.
Based on the new internal APIs in `BufferProvider`, it was thus possible
to setup an `OututBufferProxy` that acts as a hub to connect the
`DataSink` handles with the actual `OutputSlot::Connection` implementation.
This change allowed to rectify the convoluted control structure in
`OutputSlot` and finally to manage the Lifecycle properly. Furthermore,
one level of unnecessary implementation-inheritance could be removed,
so that it is no longer necessary to subclass `OutputSlot` in order to
configure the link to a specific output backend.
Using this redesigned internal structure, it was now fairly simple to
re-implement a tracking `DiagnosticOutputSlot` and then to verify proper
integration by passing the `OutputSlotProtocol_test` again.
...to emplace some object into the buffer space.
I wouldn't expect that to be necessary for the real usage,
but it comes in handy for tests, where we often want to place
some sentinel value into a buffer.
Straight-forward:
- add a map keyed by observed frame numbers
- individual connection needs to know its own seqNr :-(
- also need to store and recall the frameNr in the buffer-Index
Fix bug with DataSink constructor
(value constructor shadows copy constructor => Stackoverflow)
''Complete construction and wiring sequence works as intended''
- can now throw away the old convoluted `TrackingInMemoryBlockSequence`
- instead use a simplistic implementation of the Connection interface
- do not perform any kind of validation, allocate new buffer for each frame
- use the `NaiveBufferSetup` (`BufferProvider`) as implementation backend.
This will be the first ''reality test'' for the new `OutputSlot` internal design:
Is it possible to implement a tracking test implementation in a clean
and straight-forward way, avoiding all the tangling and indirection
that plagued the old prototype implementation from 2012?
After resolving the inner tensions in the old prototype from 2012,
it was now possible to resolve all issues in a single strike
* `DataSink` becomes a functor, yet embeds a shared ref-count
* in the λ-closure a ''specific Buffer Descriptor'' is prepared for each feed
* building upon the prototyping work, a dedicated `OutputBufferProxy`
exposes a `BuffHandle` to the client, with `emit()` and `release()` calls
* the implementation ''delegates directly'' to the `OutputSlot::Connection`
* an this is where all actual output technology is encapsulated
...this is the gordian λ
Note
- embedding a copy of the shared-ptr in the closure
- pre-configure a buffer descriptor (buffer type)
* with the buffer size as retrieved from the connection
* and pass the connection itself marked as LocalTag
Thus, on invocation of the DataSink (functor), it suffices
to invoke the lock() function on the BufferDescriptor,
using the new additional argument introduced with the
preceding commit
Reaping the benefits of the preceding refactorings, most of the complex
wiring within `OutputBufferProxy` can be omitted, and the proxy calls
can be delegated directly to the actual `Connection`
There is a catch however: we need to create an additional pathway
to feed the frame-number information into the `BufferStore` backend.
The solution adopted here is the result of extensive reflection.
Taken literally, or naively, the reworked architecture would imply
to route every call through a delegation-chain that is dispatched
in the state-managing "PImpl" (`AllocState<CON>`).
However, in keeping with the idea of an opaque functor embedded into
`DataSink`, direct and explicitly typed calls can be bound in the
associated closure. Doing so will strengthen the role of the
interface `Connection` — which defines the boundary to the
actual domain of output technology.
This changeset attempts to breach into the implementation design of OutputSlot
(while largely retaining the API and the »Output Slot Protocol«).
While the actual problem that brought me to this point can be traced back to some
rather obvious mistake, namely to conflate the `BufferProvider` used as
backing implementation with a `BufferProvider` used as front-end API,
creating the new connections to use an `OutputBufferProxy` unveils some
further long-standing structural mismatch; notably this leads to the
difficulties to get the lifecylce management of `OutputSlot` and `DataSink`
under control.
So I remove one entire level from the implementation,
in a bold move to realign the design with the prospective usage pattern.
Obviously, this breaks a lot of further code, and I need to rewire
the Allocation implementation altogether, and provide a new functor based
implementation for `DataSink`
