The goal is to develop a heuristic when to trigger clean-up of the local pool.
For developing this heuristic, I'll use a unit-test
- represent "allocations" by dummy buffer pointers
- use a queue and a stack to push those "allocations" around
- the "global pool" is just a queue with such dummy pointers
- next step is to define some λ-primitives
- finally, a λ-DSL to allow to perform "N operations of kind X"
This is a fall-back situation,
in case the allocation ''was not announced beforehand''
A direct locked global access is used then, which is verified by this test
...need an elaborate test setup, with the test worker thread
and the main thread proceeding in lockstep, otherwise the
effect would not be visible. Furthermore, the diagnostic
access for LocalMemPool can be re-used here, thereby
mapping requests onto the pool for the current thread.
This makes the test case pass, validating the theory.
Each `AllocReceiver` now integrates a `lib::LiveMark` instance,
and the callbacks are dispatched using the corresponding
`LiveMark::SafeHandle`, so that it is no longer possible
to dispatch allocation messages to a defunct receiver.
Furthermore, the `LiveMark` is explicitly disengaged prior
to unwinding a local pool, so that no further pending requests
can be handled ans dispatched concurrently. The local pool
sends back all the remaining allocations via async queue
to the `EngineBufferManager`
Not that such a shutdown can be triggered any time,
since an exception or failure might cause a worker thread
to unwind, thereby destroying the thread-local `LocalMemPool`
without further notice.
...even while this situation is unlikely, it needs to be considered:
A worker thread might terminate unexpectedly, due to an error;
all thread-local objects will be destroyed automatically — while the
`EngineBufferManager` might be processing further allocations requested
by that very thread, and dispatch them towards an already destroyed `LocalMemPool`
This turns out to be a nasty problem, since the whole setup was created
to allow the workers to operate independently from each other; any handshake
or registration/deregistration mechanism would require a lock of sorts,
thereby defeating the whole purpose of the setup.
The best solution I could find is to rely on a instance marker hash,
that can be checked with an atomic read prior to each dispatch of a
memory allocation. The destructor of the local pool would then be able
to ''disengage'', by invalidating that hash.
To evaluate this solution, I developed this mechanism
as a free-standing library component, called `LiveMark`
Most relevant parts of the `LocalBufferStore` are already available
and just need to be connected. One point poses a challenge however:
when and how to trigger the heuristic clean-up of the local pool,
so that ''on average neither too much memory is wasted, nor
to much unnecessary allocations'' are performed...
Signed-off-by: Ichthyostega <prg@ichthyostega.de>
This is a collection with unique-ownership,
and the ability to create all content objects in ''RAII style''
from the constructor call, using a ''population functor''.
It is often used to maintain a collection of resource handles,
and especially to start a series of threads (for testing).
With the variadic λ-captures from C++20, it is possible to rewrite
the most frequently used population functors to be easier to use.
Notably we can now bind constructor arguments, that will be passed
to each of the created elements — a prominent example being the
thread function when launching several threads. Furthermore,
I added a constructor variant that consumes a Lumiera Iterator.
Adding this kind of generic variants is possible now with C++20,
since overload resolution takes concepts into account.
The test performs a massively concurrent scenario
that presses with 100 threads and 1 MiB allocation for each.
This highlighted my still limited understanding of the situation
and leads to a much better solution, where, on-demand, some worker
performs all the accrued allocation and management work.
So effectively we do not even need to use the synchronous API
Implementing the `EngineBufferManager` the desired way,
with the ability to post allocation requests asynchronously,
raises the difficult question of who will ultimately perform
the allocation / deallocation and further management work.
The point is that this management effort takes up capacity
originally assigned for the render workers themselves, either
by implicitly pushing aside those threads, or as a dedicated
processing assigned to some worker.
In the current stage of development it seems impossible to decide
upon the proper mechanism to schedule this excess work explicitly.
The solution proposed hereby however circumvents this dilemma,
insofar it allows to implement the synchronous, fallback processing
in terms of the asynchronous processing, just performed exclusively
within the current thread. Unless some processing will be scheduled
explicitly (which can be added as performance optimisation later on),
the handling of a direct allocation request will ''fold down'' into
the current worker eventually, thereby using it's processing capability
without the need of any explicit scheduling.
While all of this aggregate can be considered ''deep implementation'',
the structure of dependencies bears danger to get confusing. Even more so,
since an elaborate pooling memory allocator will be added to and interconnected
with this structure.
Now considering the following route:
- `engine-buffer-allocator.hpp` has elements of a common service / interface
- but `engine-buffer-allocator.cpp `is also the most deeply nested implementation backend
- `local-buffer-store.hpp` is a junction point
- `local-mem-pool.hpp` is considered an insolated implementation detail
- ultimately, this will be included and wired in `render-enviroment.cpp`
This implementation and the accompanying test might seem pointless,
yet the point is that the interface is aligned to the emerging
concepts for memory allocators in Lumiera, while also directly
delegating to the std::allocator for the time being.
Before we're able to observe the actual allocator trend and behaviour,
it is pretty much moot to engage into any kind of optimisation;
the generic heap allocator performs quite well on recent C++ runtimes,
and, furthermore, the most important performance tweak is to use
a thread-local pool of recent allocations, which should allow
our jobs to run without impeding each other.
Seems to work now roughly as expected, so that the next layers
of functionality can be built on top. It seems impossible
to determine the viability of this whole approach without
taking the full buffer memory management into use with
a real-world load scenario.
At the begin of each Render Node `pull()`, the render job is expected
to ''announce'' a maximum expected allocation requirement to the `BufferProvider`,
which information is passed-on to the `LocalMemPool` to determine
to which degree it can be satisfied with allocations from that local pool.
Without actual empiric observation, it is impossible to determine
the relevance of this step, how it should be resolved heuristically
against the known contents of the pool, and how much effort to spend
on that step.
Thus I'll settle for the ''simplest thing that can possibly work'' —
which is to check only for close matches and rather rely on additional
allocations, forwarded asynchronously, assuming that the clean-up
heuristics will deal with excess allocations quickly.
- take into account how much memory will be wasted when
using some allocation to satisfy a (smaller) allocation request
- furthermore prefer allocations used frequently in the past
- when an allocation is used, tag it with a score that also
reflects this degree of waste
- select buffers with low score first for clean-up
- use boost::lockfree to implement the in-queue with new allocations
- use a std::list with an internal administrative record
- automatically ingest the in-queue prior to accessing the pool
After several fruitless attempts, I have decided now
to settle for self-contained ''memory pool'' component,
that can be operated **thread-locally** and employs a
**lock-free** message queue for the allotment of memory.
Seemingly the question of how this relates to `BufferMetadata`
must remain unresolved for the time being, until the entire
buffer memory subsystem is available for empirical investigation.
As staring point, it is clear that the `BufferStore` side of the
implementation will rely on a similar structure with a thread-local
base service — which in this case will have to maintain a small pool
of recently used buffer allocations...
...which happens when a buffer type is referred for the first time
from within a new worker thread, while it is already stored in the
global registry; the full key-chain is down-synced then.
...using the first usage situation as guideline,
and verify proper operation through unit-test
- the LocalBufferStage gets a set of thread-local BufferMetadata
- furthermore, it accesses the central hub via dependency-injection
- after each type registration ''from a worker thread'' data is published
As an asside: ensure most metadata registration can ''move'' the TypeHandler
This test setup defines one of the problematic situations
where a synchronisation is necessary.
__TODO__: actually add locking and that synchronisation to LocalBufferStage
What I am building step by step right now is the first draft
of a production-grade render buffer management scheme.
For this I need suitable implementations of `BufferStag` (and `BufferStore`)
These will include a `LocalSlice` to hold thread-local »satelitte services«,
which in turn need a connection to some central coordination hub.
The thread-local instances can access the hub via our `lib::Depend` (I have just
verified that this works as intended, with on-demand creation of the
central instance on first access from some thread. This is important,
since very likely we need to install a dedicated and speficically wired
service instance, which must happen prior to first dependency-request
via `lib::Depend`. And since the worker-threads are created by the Engine,
we get ample opportunity to do so before starting those threads.
Included in this changeset is a first rough draft of the central registry
`EngineBufferMetadata` — I have modelled it analogous to the `SimpleBufferStateRegistry`,
which is the test-level implementation of the `BufferStage` API.
However, we do not want to expose direct buffer allocation operations
on the central hub, and thus I have left out the respecive parts of the API.
Furthermore, I have added an `abandon()` operation, without there beeing
a need for such an operation right now. But since this operation is tricky
and potentially dangerous, I prefer implementing it the correct way,
since I have a clear understanding what this entails at the moment.
The recursive synchronisation up the parent-chain can in most cases be skipped,
by relying on the fact that inconsistent data is never stored in the BufferMetadata
table. If thus an entry is locally present and matches the entry from the
other side, then it can be concluded that its parent must also be already in sync.
Since the intention is to run several thread-local instances of `BufferMetadata`
in a concurrent setup, we need a way to publish and exchange chains of
registered type-keys and metadata entries.
The first step is to develop a mechanic for data exchange,
and to define the rules when such synchronisation is (not) allowed.
During the last days, I started with analysis to find a way
to operate the `BufferProvider`, as implemented thus far,
within a massively concurrent environment.
Since I intend to create thread-local caches,
the initialisation of the `BufferMetadata` store must be static.
It turns out that the root-ID, which acts like a mandator ID,
is only used at a single code path, when creating or accessing
an abstract type-Key. It is thus easy to extract and relocate
this distinguishing ID into the actual `BufferStage` implementation.
Looking through all my work notes, it seems that all the
problematic aspects are now solved in a satisfactory way.
* it is clear how to dependency-inject a `BufferProvider`
* the internals of the `BufferProvider` front-End have been restructured
* there are now two ''pillars'' of the implementation: `BufferStage` and `BufferStore`
* both can be subclassed independently
* `BufferProviderSetup` exposes a proxying-mechanism (used for `DiagnosticBufferProvider`)
* the `BufferMetadata` have been reviewed and some minor details were rectified
* after a re-Design of `OutputSlot`, `BuffHandle` can now be used as unified front-End
* both `DiagnosticBufferProvider` and `DiagnosticOutputSlot` have been rewritten
* they are now independent, self-contained specialisations, layered on top
Overall, what might seem like a detour, was an important step towards
well-balanced structures and a better understanding of the Render Engine.
''This is a breakthrough...''
Using the realigned implementation structure in `BufferProvider`,
and especially the ability to establish an overarching context by
passing an connection-ID through the `LocalTag` in the `BufferMetadata`,
it was possible to resolve the longstanding mismatch buried deep within
the `OutputSlot` implementation.
Based on the new internal APIs in `BufferProvider`, it was thus possible
to setup an `OututBufferProxy` that acts as a hub to connect the
`DataSink` handles with the actual `OutputSlot::Connection` implementation.
This change allowed to rectify the convoluted control structure in
`OutputSlot` and finally to manage the Lifecycle properly. Furthermore,
one level of unnecessary implementation-inheritance could be removed,
so that it is no longer necessary to subclass `OutputSlot` in order to
configure the link to a specific output backend.
Using this redesigned internal structure, it was now fairly simple to
re-implement a tracking `DiagnosticOutputSlot` and then to verify proper
integration by passing the `OutputSlotProtocol_test` again.
This is a design study to find out what would be required
to build an opaque function wrapper, similar to `std::function`,
yet without using any heap memory ever.
Such a building block would allow for some interesting options
for a well structured design of elaborate data structures.
This changeset provides an implementation draft.
__Two important findings to note__:
* it is impossible to get away with just a single function pointer
* because at the very least we need to invoke the destructor of the
embedded delegate. Thus something akin to a "manager" is required.
* this implies that the component would not be so efficient as I hoped,
since it requires two pointers and requires to emit code for each
type of delegate.
* by making the component **NonCopyable**, I avoided the complexities
of virtual copy support. In case the latter is desired, it would be
much simpler to switch to a classical OO approach with a VTable
and use our `PolymorphicValue` wrapper to handle copy and cloning.
...to emplace some object into the buffer space.
I wouldn't expect that to be necessary for the real usage,
but it comes in handy for tests, where we often want to place
some sentinel value into a buffer.
- visible Signature of TestFrame changed due to being relocated into the Vault
- attempt to make the WorkForce_test more robust;
frequently fails on testsuite run after full build;
so better ''wait'' for the workers to have started working,
instead of using a fixed delay (that must be extremely short
and is hard to predict)
This changeset attempts to breach into the implementation design of OutputSlot
(while largely retaining the API and the »Output Slot Protocol«).
While the actual problem that brought me to this point can be traced back to some
rather obvious mistake, namely to conflate the `BufferProvider` used as
backing implementation with a `BufferProvider` used as front-end API,
creating the new connections to use an `OutputBufferProxy` unveils some
further long-standing structural mismatch; notably this leads to the
difficulties to get the lifecylce management of `OutputSlot` and `DataSink`
under control.
So I remove one entire level from the implementation,
in a bold move to realign the design with the prospective usage pattern.
Obviously, this breaks a lot of further code, and I need to rewire
the Allocation implementation altogether, and provide a new functor based
implementation for `DataSink`
...which still looks like a nasty task
* the good part is, we have a working `BufferProxyAdaptor` now
* yet this setup does not seem to fit into the `OutputSlot::AllocState`
As an aside, several test definitions need to be relocated to the Vault,
since the whole compound around `BufferProvider` migrated down
This is a nasty new requirement, discovered as an aside while looking
into ways to resolve the ''crucial mismatch'' between the `OutputSlot`
and `BufferProvider`, using the proxy-implementation of `BufferProvider`
(thereby expanding on the recent draft with the `BufferProxyAdaptor`)
The point is, if the BufferMetadata will be used beyond the internal coordination
within BufferProvider, then it will be used as part of actual buffer storage.
And this implies that the actual size of the allocation may differ (be greater)
than the nominally required size (since we'll very likely employ some kind
of memory tiling, for reasons of performance).
This leads again to an analysis of `BufferMetadata` usage,
with the conclusion to add the actual storage size directly into the API,
but treat it as an ''extra'' and not integrated into the ''buffer type''
Phew ... quite complex alltogether
This might have seemed like a good idea, adding an additional ''consistency check''...
However — we ''do have'' a generic check, insofar `BufferMetadata` is a **state engine**.
As long as we're able to prevent, reliably, that the same buffer address
ends up being classified as a different buffer type, this should be enough
to prevent double-use of any buffer (letting aside that `BufferProvider` is
backed by an implementing memory allocator, which in itself should prevent
any double-use.
Removing this feature resolves some ambiguity in the overall design,
and allows to turn `BufferMetadata::lock(type, address)` into a private
implementation function.
In the old code, there were some external access paths directly entering
into that implementation function, which caused a lot of confusion (at least for me...)
This refacotring prepares for the planned changes in `OutputSlot`
* introduce a new compound parameter (tuple) BufferAlloc
* wire this through all calls involved with storage handling
__Background__:
* `BufferMetadata` deemed important for the implementation of buffer storage
* ''new requirement detected'': must be able to handle an actual storage block
that is ''larger'' than the requested buffer size (⟼ tiling)
* need compact and uniform info, also for the `BufferProxyAdaptor` callbacks
The old implementation of `DiagnosticOutputSlot` is now disabled
and will be re-written from scratch, and no longer tangled to `BufferProvider`
- `OutputManager` remains in Steam-Layer for now
- the `Timings` and `OutputSlot` are now a self-contained
building upon all the preparations done during the last days,
the compound of code connected to the `BufferProvide` can now
be migrated down into the vault:
- `BuffHandle`
- `BufferProvider`
- `BufferMetadata`
- `BufferProviderSetup`
- the simple standard implementations
- `DiagnosticBufferProvider`
- all directly related unit tests
These are generic vehicles to verify computation structures.
Since they are in no way related to our models, they belong into »Vault« layer
Furthermore this reorganisation of test support components is prerequisite
to also move further generic facilities down into the Vault
This is something I wanted to clean-up since a very long time indeed,
but always postponed it, since it requires a lot of nasty clean-up and disentangling....
Several parts had been stashed away into `src/lib/test`, which are not really related:
* the testrunner and testsuite builder
* generic helper utilities
* specialised test-support facilities, e.g. the EventLog
This changeset introduces a new top-level test folder and a namespace `::test`
Code from this folder is linked into a shared library `libtestsupport.so`,
which resides in the linkage hierarchy between »Vessel« and »Vault«,
yet is no longer part of the Lumiera application and only linked into the test-suite.
Yet another chainsaw massacre.
One of the most obnoxious annoyances with C++ metaprogramming
is the need to insert `typename` and `template` qualifiers into
most definitions, to help the compiler to cope with the syntax,
which is not context-free.
The recent standards adds several clarifications, so that most
of these qualifiers are redundant now, at least at places where
it is unambiguously clear that only a type can be given.
GCC already supports most of these relaxing rules
(Clang unfortunately lags way behind with support of newer language features...)
The Lumiera »Reference Platform« is now upgraded to Debian/Buster, which provides GCC-14 and Clang-20.
Thus the compiler support for C++20 language features seems solid enough, and C++23,
while still in ''experimental stage'' can be seen as a complement and addendum.
This changeset
* upgrades the compile switches for the build system
* provides all the necessary adjustments to keep the code base compilable
Notable changes:
* λ-capture by value now requires explicit qualification how to handle `this`
* comparison operators are now handled transparently by the core language,
largely obsoleting boost::operators. This change incurs several changes
to implicit handling rules and causes lots of ambiguities — which typically
pinpoint some long standing design issues, especially related to MObjects
and the ''time entities''. Most tweaks done here can be ''considered preliminary''
* unfortunately the upgraded standard ''fails'' to handle **tuple-like** entities
in a satisfactory way — rather an ''exposition-only'' concept is introduced,
which applies solely to some containers from the STL, thereby breaking some
very crucial code in the render entities, which was built upon the notion of
''tuple-like'' entities and the ''tuple protocol''. The solution is to
abandon the STL in this respect and **provide an alternative implementation**
of the `apply` function and related elements.
This resolves an intricate problem related to metaprogramming with
variadic templates and function signatures. Due to exceptional complexity,
a direct solution was blocked for several years, and required a better
organisation of the support code involved; several workarounds were
developed, gradually leading to a transition path, which could now
be completed in an focused clean-up effort over the last week.
Metaprogramming with sequences of types is organised into three layers:
- simple tasks can be solved with the standard facilities of the language,
using pattern match with variadic template specialisations
- the ''type-sequence'' construct `Types<T...>` takes the centre stage
for the explicit definition of collections of types; it can be re-bound
to other variadic templates and supports simple direct manipulation
- for more elaborate and advanced processing tasks, a ''Loki-style type list''
can be obtained from a type-sequence, allowing to perform recursive
list processing task with a technique similar to LISP.
Since I've convinced myself during the last years that this kind
of typelist programming is ''not a workaround'' — it is even
superior to pattern matching on variadics for certain kinds
of tasks — the empty struct defined as `NullType` got into
more widespread use as a marker type in the Lumiera code base.
It seems adequate though to give it a much more evocative name
...initially, this header was a collection of small helpers made on occasion;
one of them, the `ItemWrapper` used in transforming pipelines came
into widespread use and was much augmented and improved over the years.
many other tiny helpers could be replaced by standard library facilities...
