Largely this test covers already established functionality.
It is ''somewhat difficult'' to prove that an opaque type is present,
beyond showing that values are truncated or accommodated.
At the beginning of the implementation-draft, I considered a double-dispatch,
but concluded that this would be too much overhead for the standard case;
the drafted visitor-style dispatch was "parked" as `Domain::transferTo`.
It turned out that this implementation is not directly usable, since it
assumes that »the value buffer« is accessible, albeit as opaque ValBuff&.
As the design matured, I decided that the presence of a »Buffer« should better
not be assumed, and thus flipped the high-level-APIs around. But it is
easily possible to implement this double-dispatch scheme anew on top of
the now settled implementation scheme:
- `Disposition::assignValueTo (Disposition&)` invoked on the source side
- implemented in `ParamData<VAL, IFA>::assignValueTo (Disposition&)`,
and there delegating to the templated front-end setter
`Disposition::setVal(X)` (which performs the second dispatch step)
It remains to be seen if such an opaque value assignement is required in practice
We are still using the first draft implementation of that concept.
At that time, I was not able to implement ''virtual copy support''
in a fully generic fashion -- rather I put up the requirement that
the payload type must expose a `CopySupportAPI`
Typically this does not fit well into a given domain-specific
interface- and implementation hierarchy; here I used a band-aid,
by injecting it via an additional template parameter -- and promptly
forgot using the same secondary parameter in some type aliases.
The result was that we got two disjoint instances of the ScaleRegistry,
leading to quite puzzling "ID not yet registered" errors.
- define lower bound, upper bound or both for a limited scale
- setup a cyclic wrapping scale
- mark the scale as logarithmic, binary log or decibels
- mark the parameter as a factor value (important in interplay with log scales)
BUT there is a problem: when the original parameter (used as prototype)
''does already define a scale'', then we need a way to get a clone
of this scale definition -- and this involves type conversion
In-depth analysis indicated that an ''opaque solution'' would be possible,
yet complex to implement in detail
- a Builder interface would be required
- this interface needs to package all given param values as ValBuff
- furthermore, each such value must be accompanied with a BaseTypeID
- the actual builder would then become ''a PImpl''
- it could be either implemented as heap-allocated with unique_ptr
- or be a `PolymorphicValue` itself (in which case the builder front-end
should be non-copyable and used by reference, inline in a builder expression
- in any case, the front-End builder would proxy all calls to the actual builder
- and the latter would return an opaque parameter at the end
This setup would even allow to use ''full type erasure'' on the Parameter front-end,
and might reduce the ''header inclusion tax''
BUT -- that being said, if we just require the user to specify the desired type explicitly,
not only when building a new parameter, but also when cloning a parameter from a prototype,
then the actual implementation could use one simple builder object and all the complexities
outlined above could be skipped.
So the latter is what we do for now -- since this whole effort is rather a
design draft and should enable to explore actual usages of the parameter framework.
At a later point, the elaborate version could still be implemented
...since we do not check beforehand what value was specified;
it seems preferable to do this ''in the parameter implementation'',
as the builder can be expected to get a broad API, while the ParamData
is the natural place to enforce an invariant
This demonstrates how a Scale (descriptor) can be handled
in the simple case where the value type is known.
There is ''no deduplication'' -- every Scale definition
gets a new Slot.
The scale is then retrieved later and used to conform values,
as demonstrated with a simple `maxValue()`
Use the simplest possible example, a parameter with limited `maxVal()`.
Then find out where in the data patch the Scale-conforming operation
must be hooked in.
The decision was made to integrate **everything inline** for maximum performance.
An increased ''include tax'' is incurred, and thus the Parameter-Builder
should be split off into a separate header.
The static Scale-Registry is now part of `TypedEntity` (type-handler.hpp)
and will thus instantiated once per actual parameter value type.
⟹ the Scale-ID is a **runtime value** and **depends** on the BaseType-ID
- Fix several subtle (and not so subtle) bugs.
- Verify correct behaviour under concurrent pressure
- leaving out the Mutex-lock leads to corrupted internal state
- effect of the Atomics can not be demonstrated
Note: occasional failures because the threads sleep to allow
the other threads to complete their work. If a thread wakes up
too early, it might see stale state and access index-slots that
are not yet published. This confirms: you **must not** access
registration IDs unless you got them from somewhere, which
ensures they were registered and the change is visible.
This turns out to be tricky, since all registry operations are written
to be fast and should operate in the nanoseconds range; any scheduling
or thread sleeping/awaking activity would thus be slower by factor 1000.
It seems impossible to ''really press'' the visibility aspect,
but we can ensure the internal structures are built and extended
without corruption under concurrent pressure:
- place numbers 1...N into the registry
- and alternating read the numbers added by the other threads
At the end, each thread must see the same (predictable) sum, even while
the order or the numbers in the registry is slightly random
- as it turns out: we need `UninitialisedStorage` to avoid spurious ctor calls
- and the extent-index is computed by div, not mod !!
With these fixes, I can now demonstrate that the allocation procees
exactly as expected, the registry can be filled completely, and no
memory is leaked or corrupted (the latter only verified with some probes)
- extract an accessor for the Extent
- increment the `size_` field only after successfully completing slot allocation
- properly manage allocated memory without leak; we have a policy in Lumiera
that prohibits to leak memory ''just because we know that it will not be de-allocated''
__NOTE__: implementation passes first unit test now
Since the design established thus far calls for a fixed size container
and a size-constrained stack of virtual interfaces and implementation within,
it is clear that more diverse parts of the type and scale handling must be
forwarded into a second layer, maintained within a registry and keyed by ID.
Taking the prospective use into account, a two-level structure with a
fixed-size top-level index table and dynamically added extents seems to
cover all requirements, and allows for a lightweight, atomic-protected
read-access path.
The selection of the intermediary working type, used for exponential
and logarithmic functions when combining values from different scales,
was implemented too simplistic.
Following the example of the `NumberJoinType`, it seems preferrable
to spell out the decision logic explicitly. This allows to pick `long doubl`
as working type especially in cases where 64bit integral parameters are
involved, either as source or destination.
With these final tweaks, the `verify_valueJoining()` test is not complete,
presumably with path coverage of the conversion functions.
The Scale descriptor and operational logic is now defined to the point
that it can be integrated into the general Parameter implementation machinery.
Scrutinising the bounds of numeric precision.
As it turns out, most cases work as expected,
including the massive errors incurred by input-values given as float.
However: a logarithmic combination integral -> integral runs in loss of precision
So the selection logic for the intermediary working type is still overly simplistic!
The control logic for the various combinations of linear and logarithmic scales is nasty,
and the added path coverage indeed unveiled several bugs.
One of these issues, an unjustified limitation on the input value type of `asLogarithm`,
convinced me to refactor the conversion functions into free functions,
since they effectively ''always span the relation of two scales'' --
writing them as member function introduces an unnecessary asymetry.
There are some corner cases where the C++ number promotion rules can hurt us badly.
Devised a type selection logic to pick a suitable common numeric type explicitly,
using `std::common_type<T1,T2>` in most cases, yet handling explicitly the
case when the computation would wrap into unsigned; in such a mixed sign integral case,
we either use int64_t (when its precision suffices), or long double,
which can hold 64bit of mantissa without precision loss.
LIN -> LIN yet either additive or multiplicative
LOG -> LIN
LIN -> LOG
and different kinds of logarithms
all computations using a common intermediary numeric type,
preferably a suitably sized floating-point
In the general case, a base value and a feed can also be combined
when the metric of both arguments differs; notably a linear multiplier
can be applied to an logarithmic base value by adding its logarithm,
and a logarithmic feed can be multiplied to a linear base value by
taking the exponential function.
Furthermore there are some cases involving nominal scales, which we can mark only, but leave unimplemented
The `join()`-operation allows to combine a value pulled from automation
to be combined with a local ''base value'' stored in the parameter.
When both the source and the target parameter use the same Scale,
this operation is simple; it becomes more tricky once they don't --
because then linear / logarithmic and type conversions must be handled.
I now decided to provide a generic implementation in the Scale (descriptor),
while the enclosing parameter shall handle the dispatch when connecting
to a opaque source with another Scale. And this generic implementation
relies on the same technique as was used for the parameter type conversions,
thus it should be factored out into a library header.
As with all those C++ base types, unfortunately they are giving you
only rough size guarantees. For Lumiera, I defined short hand names
some time ago, notably because the explicit long X and short X qualifiers
do not work with ''constructor syntax'' for direct conversions.
But I choose the name `f128` at that time -- which might raise ill-guided
expectations, since the `long double` is not that long on relevant linux 64bit
platforms; it can just hold 64bit of mantissa at least however.
As replacement, I'll use `lflp` now, because it has also 4 letters,
looks somewhat weird (and thus catches attention) and points into the
right direction. I considered `lfloat` as alternative, but that looks
too much like `float`, visually...
finally I was able to come up with a rather concise test
that works with float, double and long double; especially
the aliasing due to rounding could be demonstrated, and
the precision limit is made plausible.
Interestingly, `std::fabs()` really works precise enough
so that a very large positive domain can be implemented
and tested with exact match
The floating-point variant turns out to be difficult to test,
since it is never ''actually exact'' -- due to the nature of floating-points.
We can not expect reasonable behaviour at the domain boundaries,
even while in some cases the computation works (and in others it
goes infinite and is sanitised to `minVal`).
The limit for rejection is itself rather a heuristic;
thus it is difficult to show "wrong" behaviour beyond the limit,
while the limit itself is typically already so large that it is
hard to handle in test code -- at least when expecting visible results.
The implementation should cover all integral data types,
both signed and signed, over the entire value domain.
Verify that by testing the bounds and the complex computation path
These two helpers ''just happen to work'' in the full domain, and also for unsigned types.
I can not recall that I actually was aware of that, but it's nice.
Thus add some test cases to cover the bounds of the domain
Turns out that implementing a cyclically wrapping domain properly is much more tricky than initially anticipated.
After initial analysis I decided where to draw the line and what ''esoteric features'' aspects to leave out.
Then I used Knowledge-Technology (Claude) to scrutinise and improve my implementation,
and to learn a lot more about integral arithmetics in general.
- build a default value
- handle cyclic scales
- implement clamping behaviour
- identify a muliplicative factor
- join a feed either multiplicative or additive
Scales are defined as descriptor records and will be used as const.
Besides some information functions, a Scale
* allows to conform a value to be compliant with the Scale's constraints
* can ''join'' and thus apply a ''feed'' on top of a ''base value'';
this is the foundation for applying an automation function, while
handling multiplicative and logarithmic cases properly
Notably I considered how and to what extent we will need to represent ''nominal'' or ''ordinal scales''...
In fact we should represent them, yet as cascaded delegate, since that is a relevant topic,
yet not the primary use case -- and a topic with very deep complexity in itself.
It is clear by now that a `Scale` will act as a refinement of a full-domain value type.
However, in order to develop the definition and builder notation for parameters,
a much more detailed understanding of scales is necessary -- including the question
where to draw a line between the complex structured `ParamSet` and a simple
`Parameter` that is conceived to be number-value-like.
...yet the actual problem remains that the value type of the
prototype remains a runtime-dynamic hidden implementation trait.
How can we remould the scale hidden within a parameter, while
in the regular setup, the scale is implemented as an immutable
entry placed into a registry?
Overall, this is a questionable endeavour...
- I want a framework that can be extended flexibly
- I am concerned about the performance penalty of that flexibility
What makes this situation problematic is the fact that I am deep in
prototyping work here, and have not the slightest idea as to what would be
the actual usage context. So I am lacking a point of reference, yet still
I am engaged into a very tricky (premature) optimisation...
Anyway. I have now found a way to integrate an optimised access code-path
in a safe way, while retaining the general case with flexible type conversion:
- whenever a BaseDomain<X> is used, a corresponding type-ID is marked into the ID-Record
- this allows when it is possible to side-step any type conversion,
- in which case we can also forcibly side-step the virtual dispatch,
and invoke directly the value-accessors of the default implementation,
reaching into the embedded value buffer.
Can now express the automatic conversion between all allowed `BaseTypes`
by invoking the `TypeHandler` interface directly. The interfaces were streamlined
and storage size is managed automatically at compile time.
`PolymorphicValue` incurs some quite confusing requirements due to
the clumsy way how the instance handling and copy support must be injected.
As pointed out in #1197, with modern C++, this could be much simplified.
An overhaul of `PolymorphicValue` seems urgent thus -- in the meanwhile
we need to introduce more flexibiltiy with base-class template parameters,
so that we can inject the CopySupport interface at usage site of the
complete implementation class chain. Furthermore, the size of the VTable
is added automatically to the storage requirements in `PolymorphicValue`,
so we must subtract it when specifying the payload size by a compile-time
expression based on `sizeof(ParamData<double>)`
With these tweaks, we can now keep the limit of 4 «slots» (32 bytes)
and still reserve 64bit for ID and type flags.
The solution from yesterday works, but is needlessly indirect.
However, if we want to circumvent ''any indirection'' in the
simple standard case, then it turns out to be necessary to
have some kind of ''flag store'' already in the `Disposition`
(the implementation service interface). Because otherwise
it would be impossible to trigger the short-circuit access
without going through some kind of indirection.
Need to define a generic ID-Record with some flexible extension space
that can be used by subclasses; and notably we need to generate some
base-type-IDs on a static level -- which should be possible, since all
allowed BaseTypes are defined as Type-Sequence.
Add value access implementation of the Provision interface
by direct cast and access to the embedded value in `ParamData<VAL>`;
obviously this is an indirect dispatch, based on the actual type `VAL`
The test demonstrates the basic machinery built thus far
- have an embedded container with inline storage
- dispatch using type information encoded into the VTable
...and in addition check that the defined value type
is consistent with the list of pre-configured `BaseTypes`
Signed-off-by: Ichthyostega <prg@ichthyostega.de>
turns out to be quite tricky, due to the overly convoluted implementation
of PolymorphicValue (which looks quite dated, generally speaking,
and could use a complete overhaul).
- pass the basic value type as template parameter through the Builder
- allow for the ability to define an initial value for the parameter
- invoke the protected constructor of PolymorphicValue, directly passing
a ''type tag'' to specify the implementation data package to be placed
into the embedded data buffer
- `Disposition` is the service-API used by the front-End container
- `ParamData<TY>` is a complete implementation with a inline-value buffer
- The front-End is based on the `PolymorphicValue` container from our support-Lib
Finally able to settle upon a layering for the implementation facilities
Provision:: represents the core idea of a parameter
Domain:: assumes the ability to handle typed values
Disposition:: complete implementation capabilities
BaseDomain:: establish type conversion functionality
TypedEntity:: provide a type and ID descriptor and scale configuration
ParamData:: full implementation based on an embedded value buffer
