...oh my.
This is getting messy. I am way into danger territory now....
I've made a nifty cool design with automatically adapted functors;
yet at the end of the day, this does not bode well with a DSL usage,
where objects appear to be simple values from a users point of view.
- Helper function to find out of two objects are located
"close to each other" -- which can be used as heuristics
to distinguish heap vs. stack storage
- further investigation shows that libstdc++ applies the
small-object optimisation for functor up to »two slots«
in size -- but only if the copy-ctor is trivial. Thus
a lambda capturing a shared_ptr by value will *always*
be maintained in heap storage (and LazyInit must be
redesigned accordingly)...
- the verify_inlineStorage() unit test will now trigger
if some implementation does not apply small-object optimisation
under these minimal assumptions
the RandomDraw rules developed last days are meant to be used
with user-provided λ-adapters; employing these in a context
of a DSL runs danger of producing dangling references.
Attempting to resolve this fundamental problem through
late-initialisation, and then locking the component into
a fixed memory location prior to actual usage. Driven by
the goal of a self-contained component, some advanced
trickery is required -- which again indicates better
to write a library component with adequate test coverage.
...now using the reworked partial-application helper...
...bind to *this and then recursively re-invoke the adaptation process
...need also to copy-capture the previously existing mapping-function
first test seems to work now
Investigation in test setup reveals that the intended solution
for dynamic configuration of the RandomDraw can not possibly work.
The reason is: the processing function binds back into the object instance.
This implies that RandomDraw must be *non-copyable*.
So we have to go full circle.
We need a way to pass the current instance to the configuration function.
And the most obvious and clear way would be to pass it as function argument.
Which however requires to *partially apply* this function.
So -- again -- we have to resort to one of the functor utilities
written several years ago; and while doing so, we must modernise
these tools further, to support perfect forwarding and binding
of reference arguments.
- strive at complete branch coverage for the mapping function
- decide that the neutral value can deliberately lie outside
the value range, in which case the probability setting
controls the number of _value_ result incidents vs
neutral value result incidents.
- introduce a third path to define this case clearly
- implement the range setting Builder-API functions
- absorb boundrary and illegal cases
For sake of simplicity, since this whole exercise is a byproduct,
the mapping calculations are done in doubles. To get even distribution
of values and a good randomisation, it is thus necessary to break
down the size_t hash value in a first step (size_t can be 64bit
and random numbers would be subject to rounding errors otherwise)
The choice of this quantiser is tricky; it must be a power of two
to guarantee even distribution, and if chosen to close to the grid
of the result values, with lower probabilities we'd fail to cover
some of the possible result values. If chosen to large, then
of course we'd run danger of producing correlated numbers on
consecutive picks.
Attempting to use 4 bits of headroom above the log-2 of the
required value range. For example, 10-step values would use
a quantiser of 128, which looks like a good compromise.
The following tests will show how good this choice holds up.
This highly optimised function was introduced about one year ago
for handling of denomals with rational values (fractions), as
an interim solution until we'll switch to C++20.
Since this function uses an unrolled loop and basically
just does a logarithmic search for the highest set bit,
it can just be declared constexpr. Moreover, it is now
relocated into one of the basic utility headers
Remark: the primary "competitor" is the ilogb(double),
which can exploit hardware acceleration. For 64bit integers,
the ilog2() is only marginally faster according to my own
repeated invocation benchmarks.
The first step was to allow setting a minimum value,
which in theory could also be negative (at no point is the
code actually limited to unsigned values; this is rather
the default in practice).
But reconsidering this extensions, then you'd also want
the "neutral value" to be handled properly. Within context,
this means that the *probability* controls when values other
than the neutral value are produced; especially with p = 1.0
the neutral value shall not be produced at all
...since the Policy class now defines the function signature,
we can no longer assume that "input" is size_t. Rather, all
invocations must rely on the generic adaptaion scheme.
Getting this correct turns out rather tricky again;
best to rely on a generic function-composition.
Indeed I programmed such a helper several years ago,
with the caveat that at that time we used C++03 and
could not perfect-forward arguments. Today this problem
can be solved much more succinct using generic Lambdas.
to define this as a generic library component,
any reference to the actual data source moust be extracted
from the body of the implementation and supplied later
at usage site. In the actual case at hand the source
for randomness would be the node hash, and that is
absolutely an internal implementation detail.
The idea is to use some source of randomness to pick a
limited parameter value with controllable probability.
While the core of the implementation is nothing more
than some simple numeric adjustments, these turn out
to be rather intricate and obscure; the desire to
package these technicalities into a component
however necessitates to make invocations
at usage site self explanatory.
...these were developed driven by the immediate need
to visualise ''random generated computation patterns''
for ''Scheduler load testing.''
The abstraction level of this DSL is low
and structures closely match some clauses of the DOT language;
this approach may not yet be adequate to generate more complex
graph structures and was extracted as a starting point
for further refinements....
With all the preceding DSL work, this turns out to be surprisingly easy;
the only minor twist is the grouping of nodes into (time)levels,
which can be achieved with a "lagging" update from the loop body
Note: next step will be to extract the DSL helpers into a Library header
...refine the handling of FrameRates close to the definition bounds
...implement the actual rule to scale allocator capacity on announcement
...hook up into the seedCalcStream() with a default of +25FPS
+ test coverage
...whenever a new CalcStream is seeded, it would be prudent
not only to step up the WorkForce (which is already implemented),
but also to provide a hint to the BlockFlow allocator regarding
the expected calculation density.
Such a hint would allow to set a more ample »epoch« spacing,
thereby avoiding to drive the allocator into overload first.
The allocator will cope anyway and re-balance in a matter of
about 2 seconds, but avoiding this kind of control oscillations
altogether will lead to better performance at calculation start.
The test case "scheduleRenderJob()" -- while deliberately operated
quite artificially with a disabled WorkForce (so the test can check
the contents in the queue and then progress manually -- led to discovery
of an open gap in the logic: in the (rare) case that a new task is
added ''from the outside'' without acquiring the Grooming-Token, then
the new task could sit in the entrace queue, in worst case for 50ms,
until the next Scheduler-»Tick« routinely sweeps this queue. Under
normal conditions however, each dispatch of another activity will
also sweep the entrance queue, yet if there happens to be no other
task right now, a new task could be stuck.
Thinking through this problem also helped to amend some aspects
of Grooming-Token handling and clarified the role of the API-functions.
Use a simple destructor-trick to set up a concise notation
for temporarily manipulating a value for testing.
The manipulation will automatically be undone
when leaving scope
For now, the `EngineObserver` is defined as an empty shell,
outfitted with a low-level binary message dispatch API.
Messages are keyed by a Symbol, which allows evolution of private message types.
Routing and Addressing is governed by an opaque size_t hash.
The `EngineEvent` data base class provides »4 Slots« of inline binary storage;
concrete subclasses shall define the mapping of actual data into this space
and provide a convenience constructor for events.
For use by the Scheduler, a `WorkTiming`-Event is defined based on this scheme;
this allows to implement the λ-work and λ-done of the Scheduler-`ExecutionCtx`.
These hooks will be invoked at begin and end of any render calculations.
...especially to prevent a deadline way too far into the future,
since this would provoke the BlockFlow (epoch based) memory manager
to run out of space.
Just based on gut feeling, I am now imposing a limit of 20seconds,
which, given current parametrisation, with a minimum spacing of 6.6ms
and 500 Activities per Block would at maximum require 360 MiB for
the Activities, or 3000 Blocks. With *that much* blocks, the
linear search would degrade horribly anyway...
WorkForce scales down automatically after 2 seconds when
workers fall idle; thus we need to step up automatically
with each new task.
Later we'll also add some capacity management to both the
LoadController and the Job-Planning, but for now this rather
crude approach should suffice.
NOTE: most of the cases in SchedulerService_test verify parts
of the component integration and thus need to bypass this
automatism, because the test code wants to invoke the
work-Function directly (without any interference
from running workers)
While testing, I repeatedly had SEGFAULT in the new thread-wrapper,
but only when running under debugger. While the language spec guarantees
that exit from the thread handle initialisation synchronizes-with
the start of the new thread, there is no guarantee in the reverse
direction. Here this means that the new thread may not see the
newly initialised thread handle ID at start. Thus I've added
a yield-wait at the very beginning of the new thread function.
Under normal conditions, the startup of a thread takes at least
100 - 500µs and thus I've never seen the problematic behaviour
without debugger. However, adding a yield-wait loop at that point
seems harmless (it typically checks back every 400ns or so).
All real usages of the thread wrapper in the application use
some kind of additional coordination or even a sync barrier
to ensure the thread can pick up all further data before
going into active work.
WARNING: if someone would detach() the thread immediately after
creating it, then this added condition would cause the starting
thread function to hang forever. In our current setup for the
thread wrapper, this is not possible, since the thread handle
is embedded into protected code. The earliest point you could
do that would be in the handle_begin_thread(), which is called
from the thread itself *after* the new check. And moreover,
this would require to write a new variation of the Policy.
While building increasingly complex integration tests for the Scheduler,
it turns out helpful to be able to manipulate the "full concurreency"
as used by Scheduler, WorkForce and LoadController.
In the current test, I am facing a problem that new entries from the
threadsafe entrance queue are not propagated to the priority queue
soon enough; partly this is due to functionality still to be added
(scaling up when new tasks are passed in) -- but this will further
complicate the test setup.
The invocation structure is effectively determined by the
Activity-chain builder from the Activity-Language; but, taking
into account the complexity of the Scheduler code developed thus far,
it seems prudent to encapsulate the topic of "Activities" altogether
and expose only a convenience builder-API towards the Job-Planning
With the previous change, we allways have an execution scope now,
which (among other things) defines a time-window (start,deadline).
However, the entrance point to an Activity-chain, the POST-Activity
also defines a time window, which is now combined with this scope
by maximum / minimum constraining.
The problem with passing the deadline was just a blatant symptom
that something with the overall design was not quite right, leading
to mix-up of interfaces and implementation functions, and more and more
detail parameters spreading throughout the call chains.
The turning point was to realise the two conceptual levels
crossing and interconnected within the »Scheduler-Service«
- the Activity-Language describes the patterns of processing
- the Scheduler components handle time-bound events
So by turning the (previously private) queue entry into an
ActivationEvent, the design could be balanced.
This record becomes the common agens within the Scheduler,
and builds upon / layers on top of the common agens of the
Language, which is the Activity record.
This is the first step to address the conceptual problems identified yesterday,
and works largely as a drop-in replacement. Instead of just retrieving
the Activity*, now the Queue entry itself is exposed to the rest of the
scheduler implementation, augmented with implicit conversion, allowing
all of the tests to remain unaltered (and legible, without boilerplate)
the attempt to integrate additional deadline and significance parameters
unveils a design problem due to the layering of contexts
- the Activity-Language attempts to abstract away the ''Scheduler mechanics''
- but this implementation logic now needs to pass additional parameters
- and notably there is the possibility of direct re-scheduling from within
the Activity-Dispatch
The symptom of this problem is that it's no longer possible
to implement the ExecutionCtx.post() function in the real Scheduler-context
...it is clear that there must be a way to flush the scheduler queues
an thereby silently drop any obsoleted or irrelevant entries. This topic
turns out to be somewhat involved, as it requires to consider the
deadline (due to the memory management, which is based on deadlines).
Furthermore there is a relation to yet another challenging conceptual
requirement, which is the support for other operation modes beyond
just time-bound rendering; these concerns make it desirable to
expand the internal representation of entries in the queue.
Concerns regarding performance are postponed deliberately,
until we can demonstrate the Scheduler-Service running under
regular operational conditions.
This is the first kind of integration,
albeit still with a synthetic load.
- placed two excessive load peaks in the scheduling timeline
- verified load behaviour
- verified timings
- verified that the scheduler shuts down automatically when done
- sample distance to scheduler head whenever a worker asks for work
- moving average with N = worker-pool size and damp-factor 2
- multiply with the current concurrency fraction
as an aside, the header lib/test/microbenchmark.hpp
turns out to be prolific for this kind of investigation.
However, it is somewhat obnoxious that the »test subject«
must expose the signature <size_t(size_t)>.
Thus, with some metaprogramming magic, an generic adaptor
can be built to accept a range of typical alternatives,
and even the quite obvious signature void(void).
Since all these will be wrapped directly into a lambda,
the optimiser will remove these adaptations altogether.
- An important step towards a complete »Scheduler Service«
- Correct timing pattern could be verified in detail by tracing
- Spurred some further concept and design work regarding Load-control
- draft the duty cycle »tick«
- investigate corner cases of state updates and allocation managment
- implement start and forcible stop of the scheduler service
Obviously the better choice and a perfect fit for our requirements;
while the system-clock may jump and even move backwards on time service
adjustments, the steady clock just counts the ticks since last boot.
In libStdC++ both are implemented as int64_t and use nanoseconds resolution
- Ensure the grooming-token (lock) is reliably dropped
- also explicitly drop it prior to trageted sleeps
- properly signal when not able to acquire the token before dispatch
- amend tests broken by changes since yesterday
Notably the work-function is now completely covered, by adding
this last test, and the detailed investigations yesterday
ultimately unveiled nothing of concern; the times sum up.
Further reflection regarding the overall concept led me
to a surprising solution for the problem with priority classes.
