benn/lumiera_documentation
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
lumiera_documentation/tests/vault/mem/local-mem-pool-test.cpp

182 lines
4.7 KiB
C++
Raw Normal View History

BufferProvider: start planning for the memory handling part As staring point, it is clear that the `BufferStore` side of the implementation will rely on a similar structure with a thread-local base service — which in this case will have to maintain a small pool of recently used buffer allocations...
2026-04-21 23:30:35 +02:00
/*
LocalMemPool(Test) - verify low-level handling of buffer blocks in a local pool
Copyright (C)
2026, Hermann Vosseler <Ichthyostega@web.de>
  **Lumiera** is free software; you can redistribute it and/or modify it
  under the terms of the GNU General Public License as published by the
  Free Software Foundation; either version 2 of the License, or (at your
  option) any later version. See the file COPYING for further details.
* *****************************************************************/
/** @file local-mem-pool-test.cpp
** unit test \ref LocalMemPool_test
*/
#include "test/run.hpp"
#include "test/test-helper.hpp"
//#include "vault/mem/engine-buffer-metadata.hpp"
#include "vault/mem/local-mem-pool.hpp"
BufferProvider: decide to use only a simple heuristics for reservation At the begin of each Render Node `pull()`, the render job is expected to ''announce'' a maximum expected allocation requirement to the `BufferProvider`, which information is passed-on to the `LocalMemPool` to determine to which degree it can be satisfied with allocations from that local pool. Without actual empiric observation, it is impossible to determine the relevance of this step, how it should be resolved heuristically against the known contents of the pool, and how much effort to spend on that step. Thus I'll settle for the ''simplest thing that can possibly work'' — which is to check only for close matches and rather rely on additional allocations, forwarded asynchronously, assuming that the clean-up heuristics will deal with excess allocations quickly.
2026-04-26 20:24:50 +02:00
#include "lib/iter-stack.hpp"
BufferProvider: start planning for the memory handling part As staring point, it is clear that the `BufferStore` side of the implementation will rely on a similar structure with a thread-local base service — which in this case will have to maintain a small pool of recently used buffer allocations...
2026-04-21 23:30:35 +02:00
//#include "lib/depend-inject.hpp"
//#include "lib/thread.hpp"
//#include "lib/error.hpp"
BufferProvider: decide to use only a simple heuristics for reservation At the begin of each Render Node `pull()`, the render job is expected to ''announce'' a maximum expected allocation requirement to the `BufferProvider`, which information is passed-on to the `LocalMemPool` to determine to which degree it can be satisfied with allocations from that local pool. Without actual empiric observation, it is impossible to determine the relevance of this step, how it should be resolved heuristically against the known contents of the pool, and how much effort to spend on that step. Thus I'll settle for the ''simplest thing that can possibly work'' — which is to check only for close matches and rather rely on additional allocations, forwarded asynchronously, assuming that the clean-up heuristics will deal with excess allocations quickly.
2026-04-26 20:24:50 +02:00
#include "lib/format-util.hpp"
#include "test/diagnostic-output.hpp"
BufferProvider: start planning for the memory handling part As staring point, it is clear that the `BufferStore` side of the implementation will rely on a similar structure with a thread-local base service — which in this case will have to maintain a small pool of recently used buffer allocations...
2026-04-21 23:30:35 +02:00
//using std::this_thread::yield;
//using lib::Thread;
BufferProvider: decide to use only a simple heuristics for reservation At the begin of each Render Node `pull()`, the render job is expected to ''announce'' a maximum expected allocation requirement to the `BufferProvider`, which information is passed-on to the `LocalMemPool` to determine to which degree it can be satisfied with allocations from that local pool. Without actual empiric observation, it is impossible to determine the relevance of this step, how it should be resolved heuristically against the known contents of the pool, and how much effort to spend on that step. Thus I'll settle for the ''simplest thing that can possibly work'' — which is to check only for close matches and rather rely on additional allocations, forwarded asynchronously, assuming that the clean-up heuristics will deal with excess allocations quickly.
2026-04-26 20:24:50 +02:00
using util::join;
BufferProvider: start planning for the memory handling part As staring point, it is clear that the `BufferStore` side of the implementation will rely on a similar structure with a thread-local base service — which in this case will have to maintain a small pool of recently used buffer allocations...
2026-04-21 23:30:35 +02:00
namespace vault {
namespace mem {
namespace test {
// using LERR_(LOGIC);
// using LERR_(LIFECYCLE);
namespace { // Test helper
BufferProvider: decision to use a free-standing pool with lock-free queue After several fruitless attempts, I have decided now to settle for self-contained ''memory pool'' component, that can be operated **thread-locally** and employs a **lock-free** message queue for the allotment of memory. Seemingly the question of how this relates to `BufferMetadata` must remain unresolved for the time being, until the entire buffer memory subsystem is available for empirical investigation.
2026-04-24 17:36:44 +02:00
inline constexpr Buff*
fake_Buffer(size_t id)
{
return reinterpret_cast<Buff*> (id);
}
Buff* const MEM1 = fake_Buffer(1);
Buff* const MEM2 = fake_Buffer(2);
Buff* const MEM3 = fake_Buffer(3);
const size_t SIZ10{10};
const size_t SIZ20{20};
const size_t SIZ30{30};
BufferProvider: start planning for the memory handling part As staring point, it is clear that the `BufferStore` side of the implementation will rely on a similar structure with a thread-local base service — which in this case will have to maintain a small pool of recently used buffer allocations...
2026-04-21 23:30:35 +02:00
}
/**********************************************************************//**
* @test verify handling of prospective buffer allocations in a local pool.
*/
class LocalMemPool_test : public Test
{
virtual void
run (Arg)
{
seedRand();
BufferProvider: decision to use a free-standing pool with lock-free queue After several fruitless attempts, I have decided now to settle for self-contained ''memory pool'' component, that can be operated **thread-locally** and employs a **lock-free** message queue for the allotment of memory. Seemingly the question of how this relates to `BufferMetadata` must remain unresolved for the time being, until the entire buffer memory subsystem is available for empirical investigation.
2026-04-24 17:36:44 +02:00
simpleUse();
verify_matchAlloc();
verify_selectAlloc();
verify_pruneAlloc();
BufferProvider: start planning for the memory handling part As staring point, it is clear that the `BufferStore` side of the implementation will rely on a similar structure with a thread-local base service — which in this case will have to maintain a small pool of recently used buffer allocations...
2026-04-21 23:30:35 +02:00
}
BufferProvider: decision to use a free-standing pool with lock-free queue After several fruitless attempts, I have decided now to settle for self-contained ''memory pool'' component, that can be operated **thread-locally** and employs a **lock-free** message queue for the allotment of memory. Seemingly the question of how this relates to `BufferMetadata` must remain unresolved for the time being, until the entire buffer memory subsystem is available for empirical investigation.
2026-04-24 17:36:44 +02:00
/** @test show how to place an allocation into the pool */
void
simpleUse()
{
LocalMemPool pool;
pool.add (MEM1, SIZ20);
CHECK (pool.canServe(SIZ20));
CHECK (pool.canServe(SIZ10));
BufferProvider: setup implementation structure for the pool - use boost::lockfree to implement the in-queue with new allocations - use a std::list with an internal administrative record - automatically ingest the in-queue prior to accessing the pool
2026-04-24 23:36:07 +02:00
auto [mem,siz] = pool.retrieve (SIZ10);
BufferProvider: decision to use a free-standing pool with lock-free queue After several fruitless attempts, I have decided now to settle for self-contained ''memory pool'' component, that can be operated **thread-locally** and employs a **lock-free** message queue for the allotment of memory. Seemingly the question of how this relates to `BufferMetadata` must remain unresolved for the time being, until the entire buffer memory subsystem is available for empirical investigation.
2026-04-24 17:36:44 +02:00
CHECK (MEM1 == mem);
CHECK (SIZ20 == siz);
CHECK (not pool.canServe(SIZ10));
pool.reAdd (MEM1);
CHECK (pool.canServe(SIZ20));
}
BufferProvider: setup implementation structure for the pool - use boost::lockfree to implement the in-queue with new allocations - use a std::list with an internal administrative record - automatically ingest the in-queue prior to accessing the pool
2026-04-24 23:36:07 +02:00
/** @test allocations are accounted by size
BufferProvider: decision to use a free-standing pool with lock-free queue After several fruitless attempts, I have decided now to settle for self-contained ''memory pool'' component, that can be operated **thread-locally** and employs a **lock-free** message queue for the allotment of memory. Seemingly the question of how this relates to `BufferMetadata` must remain unresolved for the time being, until the entire buffer memory subsystem is available for empirical investigation.
2026-04-24 17:36:44 +02:00
*/
void
verify_matchAlloc()
{
LocalMemPool pool;
CHECK (watch(pool).isEmpty());
pool.add (MEM1, SIZ10);
BufferProvider: setup implementation structure for the pool - use boost::lockfree to implement the in-queue with new allocations - use a std::list with an internal administrative record - automatically ingest the in-queue prior to accessing the pool
2026-04-24 23:36:07 +02:00
CHECK (watch(pool).cnt(SIZ10) == 0); // new allotment not ingested yet,....
CHECK (not watch(pool).isEmpty()); // however, there is an entry in the in-queue
CHECK (pool.canServe(SIZ10)); // query causes a scan and retrieval from the in-queu
BufferProvider: decision to use a free-standing pool with lock-free queue After several fruitless attempts, I have decided now to settle for self-contained ''memory pool'' component, that can be operated **thread-locally** and employs a **lock-free** message queue for the allotment of memory. Seemingly the question of how this relates to `BufferMetadata` must remain unresolved for the time being, until the entire buffer memory subsystem is available for empirical investigation.
2026-04-24 17:36:44 +02:00
CHECK (watch(pool).cnt(SIZ10) == 1);
BufferProvider: setup implementation structure for the pool - use boost::lockfree to implement the in-queue with new allocations - use a std::list with an internal administrative record - automatically ingest the in-queue prior to accessing the pool
2026-04-24 23:36:07 +02:00
CHECK (not watch(pool).isEmpty());
BufferProvider: devise a heuristic for buffer selection - take into account how much memory will be wasted when using some allocation to satisfy a (smaller) allocation request - furthermore prefer allocations used frequently in the past - when an allocation is used, tag it with a score that also reflects this degree of waste - select buffers with low score first for clean-up
2026-04-25 23:56:13 +02:00
pool.add (MEM2, SIZ20);
pool.add (MEM3, SIZ30);
CHECK (pool.canServe(SIZ20));
CHECK (pool.canServe(SIZ10));
BufferProvider: decide to use only a simple heuristics for reservation At the begin of each Render Node `pull()`, the render job is expected to ''announce'' a maximum expected allocation requirement to the `BufferProvider`, which information is passed-on to the `LocalMemPool` to determine to which degree it can be satisfied with allocations from that local pool. Without actual empiric observation, it is impossible to determine the relevance of this step, how it should be resolved heuristically against the known contents of the pool, and how much effort to spend on that step. Thus I'll settle for the ''simplest thing that can possibly work'' — which is to check only for close matches and rather rely on additional allocations, forwarded asynchronously, assuming that the clean-up heuristics will deal with excess allocations quickly.
2026-04-26 20:24:50 +02:00
CHECK (watch(pool).size() == 3);
BufferProvider: devise a heuristic for buffer selection - take into account how much memory will be wasted when using some allocation to satisfy a (smaller) allocation request - furthermore prefer allocations used frequently in the past - when an allocation is used, tag it with a score that also reflects this degree of waste - select buffers with low score first for clean-up
2026-04-25 23:56:13 +02:00
CHECK (watch(pool).cnt(SIZ10) == 1);
CHECK (watch(pool).cnt(SIZ20) == 1);
CHECK (watch(pool).cnt(SIZ30) == 1);
CHECK (watch(pool).cntFree() == 3);
auto [m1,s1] = pool.retrieve (SIZ20 -2);
CHECK (SIZ20 == s1);
CHECK (MEM2 == m1);
CHECK (watch(pool).cntFree() == 2);
auto [m2,s2] = pool.retrieve (SIZ10);
CHECK (SIZ10 == s2);
CHECK (MEM1 == m2);
BufferProvider: decide to use only a simple heuristics for reservation At the begin of each Render Node `pull()`, the render job is expected to ''announce'' a maximum expected allocation requirement to the `BufferProvider`, which information is passed-on to the `LocalMemPool` to determine to which degree it can be satisfied with allocations from that local pool. Without actual empiric observation, it is impossible to determine the relevance of this step, how it should be resolved heuristically against the known contents of the pool, and how much effort to spend on that step. Thus I'll settle for the ''simplest thing that can possibly work'' — which is to check only for close matches and rather rely on additional allocations, forwarded asynchronously, assuming that the clean-up heuristics will deal with excess allocations quickly.
2026-04-26 20:24:50 +02:00
CHECK (watch(pool).size() == 3);
BufferProvider: devise a heuristic for buffer selection - take into account how much memory will be wasted when using some allocation to satisfy a (smaller) allocation request - furthermore prefer allocations used frequently in the past - when an allocation is used, tag it with a score that also reflects this degree of waste - select buffers with low score first for clean-up
2026-04-25 23:56:13 +02:00
CHECK (watch(pool).cntFree() == 1);
CHECK (pool.canServe(SIZ10));
CHECK (pool.canServe(SIZ20));
CHECK (pool.canServe(SIZ30));
BufferProvider: decide to use only a simple heuristics for reservation At the begin of each Render Node `pull()`, the render job is expected to ''announce'' a maximum expected allocation requirement to the `BufferProvider`, which information is passed-on to the `LocalMemPool` to determine to which degree it can be satisfied with allocations from that local pool. Without actual empiric observation, it is impossible to determine the relevance of this step, how it should be resolved heuristically against the known contents of the pool, and how much effort to spend on that step. Thus I'll settle for the ''simplest thing that can possibly work'' — which is to check only for close matches and rather rely on additional allocations, forwarded asynchronously, assuming that the clean-up heuristics will deal with excess allocations quickly.
2026-04-26 20:24:50 +02:00
pool.reAdd (m2);
pool.reAdd (m1);
CHECK (watch(pool).cntFree() == 3);
lib::IterQueue<size_t> returned;
uint cnt = pool.cleanup(0.5
,[&](auto, size_t siz){ returned.feed (siz);});
CHECK (cnt == 1);
CHECK (join(returned) == "30"_expect);
CHECK (watch(pool).cntFree() == 2);
CHECK (watch(pool).size() == 2);
BufferProvider: decision to use a free-standing pool with lock-free queue After several fruitless attempts, I have decided now to settle for self-contained ''memory pool'' component, that can be operated **thread-locally** and employs a **lock-free** message queue for the allotment of memory. Seemingly the question of how this relates to `BufferMetadata` must remain unresolved for the time being, until the entire buffer memory subsystem is available for empirical investigation.
2026-04-24 17:36:44 +02:00
}
/** @test
*/
BufferProvider: start planning for the memory handling part As staring point, it is clear that the `BufferStore` side of the implementation will rely on a similar structure with a thread-local base service — which in this case will have to maintain a small pool of recently used buffer allocations...
2026-04-21 23:30:35 +02:00
void
BufferProvider: decision to use a free-standing pool with lock-free queue After several fruitless attempts, I have decided now to settle for self-contained ''memory pool'' component, that can be operated **thread-locally** and employs a **lock-free** message queue for the allotment of memory. Seemingly the question of how this relates to `BufferMetadata` must remain unresolved for the time being, until the entire buffer memory subsystem is available for empirical investigation.
2026-04-24 17:36:44 +02:00
verify_selectAlloc()
BufferProvider: start planning for the memory handling part As staring point, it is clear that the `BufferStore` side of the implementation will rely on a similar structure with a thread-local base service — which in this case will have to maintain a small pool of recently used buffer allocations...
2026-04-21 23:30:35 +02:00
{
}
/** @test
*/
void
BufferProvider: decision to use a free-standing pool with lock-free queue After several fruitless attempts, I have decided now to settle for self-contained ''memory pool'' component, that can be operated **thread-locally** and employs a **lock-free** message queue for the allotment of memory. Seemingly the question of how this relates to `BufferMetadata` must remain unresolved for the time being, until the entire buffer memory subsystem is available for empirical investigation.
2026-04-24 17:36:44 +02:00
verify_pruneAlloc()
BufferProvider: start planning for the memory handling part As staring point, it is clear that the `BufferStore` side of the implementation will rely on a similar structure with a thread-local base service — which in this case will have to maintain a small pool of recently used buffer allocations...
2026-04-21 23:30:35 +02:00
{
}
};
/** Register this test class... */
LAUNCHER (LocalMemPool_test, "unit engine");
}}} // namespace vault::mem::test
Reference in a new issue Copy permalink