diff --git a/README b/README index 2ebbe23df..f8f4236ce 100644 --- a/README +++ b/README @@ -61,8 +61,8 @@ For building Lumiera, you'll need: * https://gtkmm.gnome.org/en/[gtkmm] * https://cgit.freedesktop.org/xorg/lib/libXv[libXv] * https://wiki.gnome.org/Projects/LibRsvg[lib rSVG] - * https://git.lumiera.org/?p=debian/gdl;a=summary[lib GDL] - * https://git.lumiera.org/?p=debian/gdlmm;a=summary[GDL-mm bindings] + * https://git.lumiera.org/gitweb?p=debian/gdl;a=summary[lib GDL] + * https://git.lumiera.org/gitweb?p=debian/gdlmm;a=summary[GDL-mm bindings] Further information: diff --git a/admin/scons/Platform.py b/admin/scons/Platform.py index ea11fd9c3..fc308a45e 100644 --- a/admin/scons/Platform.py +++ b/admin/scons/Platform.py @@ -97,6 +97,8 @@ def configure(env): problems.append('We need boost::lexical_cast') if not conf.CheckCXXHeader('boost/format.hpp'): problems.append('We need boost::format (header).') + if not conf.CheckCXXHeader('boost/lockfree/queue.hpp'): + problems.append('We need boost::lockfree (header).') if not conf.CheckLibWithHeader('boost_program_options','boost/program_options.hpp','C++'): problems.append('We need boost::program_options (including binary lib for linking).') diff --git a/doc/design/application/ApplicationStart.txt b/doc/design/application/ApplicationStart.txt index a460039e4..56d625ec4 100644 --- a/doc/design/application/ApplicationStart.txt +++ b/doc/design/application/ApplicationStart.txt @@ -1,42 +1,47 @@ -The Application : Start-up and Subsystems -========================================= +Application : Start-up and Subsystems +===================================== :Date: 2009 :Author: Ichthyo //MENU: label Start-up .the purpose of »Application-main« -Lumiera is envisioned as a heavyweight yet coherent »**Application**« -- not so much -as platform, framework or operating system. And, in accordance with this focus, we -place no emphasis on possible configuration changes and state transitions within -the running application _as a whole._ And, while in fact it is possible to close -and load a new Session, most of the time we assume the application just to be -``up and running'' -- all required services to be available and interfaces to -be accessible. Without doubt, these are simplifications, but serve us well -to cut down complexity -- yet still there needs to be one dedicated realm -to deal with these specific concerns of configuration, dependency and lifecyle. +Lumiera is envisaged as a heavyweight yet coherent »**Application**« -- which implies +it is not so much seen as ``a platform'', framework or operating system. And, in line +with this understanding, we place no emphasis on possible configuration changes and state +transitions within the running application _as a whole._ The ability to react ``life'' to +some global setting changes can be limited to relevant special cases. And, while it is +indeed possible to close and load a new Session from a running application instance, in +most parts of the code base we assume the application just to be ``up and running'' -- all +required services are assumed to be available and interfaces to be accessible. If some of +these assumptions break, an exception is raised and the application unwinds. +Without doubt, these decisions represent a forcible simplification, and might be limiting +at times, yet serve us well to cut down complexity in large parts of the code base -- +however, there needs to be _one dedicated realm_ to deal with all these specific concerns +related to configuration, interdependency and lifecyle. -The Application Realm ---------------------- -So we treat all these concerns within a small and self contained structure, clearly -set apart from all the other layers, services and subsystems. This dedicated -_Application Realm_ is organised around the ``Application main object''.footnote:[ -This is the singleton `lumiera::AppState`, which is triggered by the `main` function -of the Lumiera Application. The sourcecode is kept in a separate folder 'src/common' -and linked into the shared library 'liblumieracommon.so'] -It serves the purpose of pulling up and tearing down the application in a controlled -fashion. And additionally, it provides the Interface and Config core services. -The act of building or tearing down this core realm and main object is what creates -the *Lifecycle* of the application. This is a succession of ``lifecycle phases'' -- -and almost all activities happen within the _operational phase,_ when everything -is ``up and running'' or ``just available''. +The Vessel +---------- +So we treat all these concerns within a focused and self contained structure, called +the »**Vessel**«, and clearly set apart from all the other layers, services and subsystems. +This dedicated _Application Realm_ is organised around the ``Application main object''.footnote:[ + This is the singleton `vessel::Voyage`, which is triggered by the `main` function + of the Lumiera Application. The sourcecode is kept in a separate folder 'src/vessel' + and linked into the shared library 'liblumieravessel.so'] +Its purpose is to bootstrap and tear-down the application in a controlled way, +which includes to provide the backbone services, evaluate the start-up parameters +and to connect and start the necessary core components. The beginning and end of +this vessel's voyage therefore marks the *Lifecycle* of the application, as a +succession of ``lifecycle phases'' -- where almost any activity happens within +the _operational phase,_ when everything is considered ``up and running''. Subsystems ~~~~~~~~~~ -However, initially the application needs to be brought up, and at the end, all -parts need to be unwound cleanly. To organise this process, we identify a limited -number of *Subsystems* within the Application, which are more or less independent. +However, before becoming operational, the application must be bootstrapped, and at the end, +all parts need to be unwound cleanly. To organise this process, we identify a limited number +of *Subsystems* within the Application, which are self-sufficient and operate independently, +at least to some degree. Each link:{ldoc}/design/architecture/Subsystems.html[Subsystem] is self contained and groups some other parts and services, which typically work together and may be mutually dependent. These subsystems represent a grouping, applied for the purpose @@ -61,7 +66,7 @@ the startup phase. It is the responsibility of the _Application main object_ to pull up required functionality, which in turn might result in pulling up further subsystems as dependencies. -Subsystems are defined by implementing the interface `lumiera::Subsys`, which acts +Subsystems are defined by implementing the interface `vessel::Subsys`, which acts as façade to conduct the lifecycle, find out about dependencies and shut down the subsystem in the end. So this interface, together with the _Subsystem Runner,_ define a lifecycle protocol; each subsystem is free to implement this as it @@ -97,20 +102,21 @@ within the engine independently, relying on worker jobs and a thread pool. Initialisation and Lifecycle ---------------------------- -After some discussion,footnote:[See the -link:{rfc}/GlobalInitialization.html[GlobalInitialisation] RfC -from spring 2008. In the beginning, we all agreed to ``keep matters simple'' -and build an `init()` function within one central piece of code everyone knows -and hooks into. However, while the outline of the application emerged, there -was a tension between the concern about _over-engineering_ versus the concern -about _tangled and unmanageable complexity._ At some point, an alternative -implementation based on lifecycle callbacks was elaborated, which then turned -into the solution described here. Lumiera then ceased to be the typical UI -application started by GTK, and the existing GTK code was retrofitted to -launch from within a plug-in.] +After some discussion,footnote:[ + See the link:{rfc}/GlobalInitialization.html[GlobalInitialisation] RfC + from spring 2008. In the beginning, we all agreed to ``keep matters simple'' + and build an `init()` function within one central piece of code everyone knows + and hooks into. However, while the outline of the application emerged, there + was a tension between the concern about _over-engineering_ versus the concern + about _tangled and unmanageable complexity._ At some point, an alternative + implementation based on lifecycle callbacks was elaborated, which then turned + into the solution described here. At that point, Lumiera ceased to be the + typical UI based application started-up by GTK-main, and the existing GTK + code was retrofitted to be launched from within a plug-in. +] the design leaned toward loosely coupled parts and a formal lifecycle; which saves us from investigating and coding up the various interdependencies -explicitly. Rather, the parts of the application have to comply to +explicitly. Rather, all parts of the application have to comply to link:{ldoc}/design/architecture/Subsystems.html#lifecycle[Lifecycle Phases], and each part has to care for its own state transitions, invoked through _lifecycle callbacks._ We can distinguish two distinct models how to deal @@ -128,9 +134,9 @@ Application Start ~~~~~~~~~~~~~~~~~ * some fundamental language-level facilities will be prepared during _static initialisation._ At some point, execution enters `main(argc,arvv)`. - * `AppState::init()` brings up the plugin loader and opens the config-interface. + * `Voyage::init()` brings up the plugin loader and opens the config-interface. * ...followed by triggering *ON_GLOBAL_INIT* - * the main thread then pulls up the subsystems (`AppState::maybeStart(subsystem)`), + * the main thread then pulls up the subsystems (`Voyage::maybeStart(subsystem)`), according to the command line options. * within each subsystem, façade interfaces will be opened through the interface/plug-in system. @@ -142,7 +148,7 @@ Application Start the main window. * there is an *ON_GLOBAL_SHUTDOWN* event, which can be used for normal cleanup; In case of an emergency exit, the *ON_EMERGENCY_EXIT* event is triggered alternatively. - * the AppState destructor tears down the core systems (config-interface and pluginloader). + * the Voyage destructor tears down the core systems (config-interface and pluginloader). * we establish a policy to _prohibit any non-local and non-trivial activities_ during the tear-down phase after leaving the `main()` function. diff --git a/doc/design/application/Config.txt b/doc/design/application/Config.txt index dec73272b..631c2362c 100644 --- a/doc/design/application/Config.txt +++ b/doc/design/application/Config.txt @@ -1,15 +1,71 @@ -Lumiera Design: Application and Session Configuration -===================================================== +Application and Session Configuration +===================================== -*TODO* : 'write design document' +.After initial discussion, this topic was postponed. +[CAUTION] +In its current form, as of 2026, the Lumiera application is headed towards a first +integration of the parts developed thus far. A simple configuration file, 'setup.ini' +is used to supply some minimal parameters for bootstrap of the application; all further +settings are defined as local constants in the source code. -The Lumiera application uses two quite different sources for configuration +**Configuration and Parametrisation** of software describe a state of setting, that lies +between the fixed structure embodied into the code and the transient work state of the +application. It encompasses all settings and options that determine how a software system +behaves in a specific environment, without changing its underlying source code. -- individual __settings values__ can be loaded from an application- and user configuration in the conventional way -- various facilities, especially in the Steam-Layer, base the actual behavour on __queries__, which are - to be resolve employing a rules based system '(planned)'. Configuration rules will be provided by the - application (defaults), a session template and rules stored in the actual session. +The purpose of configuration +---------------------------- +For a video editing application, the scope of ``configuration'' can be far reaching. +Not only are there numerous parameters in the engine, that must be fine-tuned for optimal +performance, but also several patterns of behaviour are available that can not be decided by +first principle alone and must thus be left open for the user to configure to their preference. +Some of these are linked to the actual user, some to the setup in the system, and some to the +current project. +Furthermore, many of these ``settings'' turn out, upon closer inspection, to be a form of +rule-based knowledge. In a project with the scale of a movie, it is essential for a fluid +and intuitive working style to handle some default setup and configuration of connections +and conversions automatically, while the user can have very valid reasons to prohibit +some other specific class of automatic conversions to be applied without explicit confirmation. --> see also the link:rfc/ConfigLoaderDraft2008.html[Config Loader brainstorming from 2008] (implementation details) +After some initial, prototyping for a ``config loader'', it was deemed too risky to engage +into any kind of preliminary design without knowing the possible ramifications. + +-> link:/x/rfc/ConfigLoaderDraft2008.html[Config Loader brainstorming from 2008] +The following preliminary measures have now been established for the ongoing development: +- whenever a _tweaking parameter_ is discovered, it will be extracted and marked + with a **language constant** in the setup or integration header of the software component. +- whenever a _decision point_ is encountered, that can not be related to technical dependencies, + it is marked as a **query call** -- with a corresponding, hard-wired _faked configuration rule_. + +The integration of some kind of configuration rule language is considered part of the primary agenda. + +Requirements +------------ +To track any insights gained regarding the requirements for configuration, +https://issues.lumiera.org/ticket/1224[Ticket #1224 (»Layered Configuration«)] +was added to the list of medium-term +https://issues.lumiera.org/report/17[»Focus Topics«] + +The following requirements have been identified to date (2026) + +- It is pointless to build one huge flat global "configuration file"; + rather, groups of configuration settings are scoped and closely related to a _local context_ +- the degree of _consistency checking_ must be tied to such a local context, where the + situation is reasonably clear: the consumer of specific configuration usually has + clear-cut requirements to demand some configuration to be mandatory, while other + configuration is optional. Misspelled or misplaced configuration that does not + fit in such a category must be flagged as error. +- such a locally defined set of expected and optional configuration might become + subject to versioning (forward- / backward compatibility) +- Configuration must be assembled from several configuration sources, + with clearly defined precedence and shadowing. Notably we have default configuration, + system-wide configuration, user specific configuration and project related configuration. +- a _preferences UI_ must be able to alter configuration settings, thereby intelligently + routing the changes into the appropriate configuration source or create a suitable overlay +- when configuration causes a decision from a rule to flip, this fact must be recorded + permanently in the session; the reason for this mechanism is that configuration might + change later, and the application must be able then to point out all places + affected by the change. +- Any detected configuration change must be captured and logged as Event and thus + be un-doable cleanly, with all possible ripple effects. diff --git a/doc/design/application/index.txt b/doc/design/application/index.txt index 11b8b02d5..1bfe09a51 100644 --- a/doc/design/application/index.txt +++ b/doc/design/application/index.txt @@ -1,10 +1,48 @@ -Design Documents: Application Framework -======================================= +The Application Realm +===================== // Menu : prepend child ApplicationStart - * link:Config.html[Configuration] - * link:plugin_loader.html[Plugin Loader] - * link:ApplicationStart.html[Application Subsystems and Lifecycle] +Shaping the outside world through software requires a variety of effective mechanisms +that capture, represent, and transform elements of reality. A well-designed part is +focused on a single aspect, is coherent and cohesive in itself, with links to other +parts based on concepts and abstraction rather then on shared technology. Yet this +structural perspective must be complemented by a meaningful overall arrangement that +supports the desired workflow and can be adapted to circumstances. +In software systems, such coherency does not emerge spontaneously. It must be supported +by definition work to establish suitable terms and interfaces, components must be +interconnected and configured, and the overall state needs coordination. +An application is therefore more than just a collection of software components. +In the Lumiera Architecture, this application realm is embodied into a structure +called the »**Vessel**«: + +- it is built as a library with only basic linkage to the support library +- it can be invoked from a `main()` function, with an argument parser +- at source code level, it provides all the types and definitions + required to express the _binding_ between components +- _conceptually,_ all layers are encased in the _application vessel_ +- and while the Vessel can not just ``control'' the configuration of parts + on upper layers, it has to bootstrap their ability to adapt to circumstances. + +-- +  +-- + +Dependencies:: + While the code base is organised into _three layers,_ the actual implementation + structures are arranged into **Subsystems** that _depend_ on each other and + can be _started and shut down_ ++ +-> link:ApplicationStart.html[Subsystems and Lifecycle] + +Configuration:: + As a means to adapt to the environment and enable flexibility, + this topic is surprisingly intricate and development has barely scratched + the surface: while the effective configuration is assembled from several + layered sources, the meaning (and documentation) of individual settings + is scoped, and users can expect the ability to override some settings + from within the application. ++ +-> link:Config.html[Configuration] diff --git a/doc/design/application/plugin_loader.txt b/doc/design/application/plugin_loader.txt deleted file mode 100644 index a0e2d9a1a..000000000 --- a/doc/design/application/plugin_loader.txt +++ /dev/null @@ -1,5 +0,0 @@ -Lumiera Design: Plugin system -============================= - -Eventually, this will have design documentation for the Plugin system. - diff --git a/doc/design/architecture/Subsystems.txt b/doc/design/architecture/Subsystems.txt index 67406f860..74e276d25 100644 --- a/doc/design/architecture/Subsystems.txt +++ b/doc/design/architecture/Subsystems.txt @@ -237,7 +237,7 @@ Lifecycle Events The Application as a whole conducts a well defined lifecycle; whenever transitioning to the next phase, a _Lifecycle Event_ is issued. Components may register a notification hook with the central _Lifecycle Manager_ (see 'include/lifecycle.h') to be invoked whenever a specific event is emitted. The process of registration -can be simplified by planting a static variable of type `lumiera::LifecycleHook`. +can be simplified by planting a static variable of type `vessel::LifecycleHook`. WARNING: A callback enrolled this way needs to be callable at the respective point in the lifecycle, otherwise the application will crash. @@ -246,7 +246,7 @@ WARNING: A callback enrolled this way needs to be callable at the respective poi Invoked as early as possible, somewhere in the static initialisation phase prior to entering `main()`. In order to install a callback hook for this event, the client must plant a static variable somewhere. -`AppState`:: +`Voyage`:: This is the Lumiera »Application Object«. It is a singleton, and should be used by `main()` solely. While not a lifecycle event in itself, it serves to bring up some very fundamental application services: + @@ -255,7 +255,7 @@ WARNING: A callback enrolled this way needs to be callable at the respective poi - the application configuration -- + -After starting those services within the `AppState::init()` function, +After starting those services within the `Voyage::init()` function, the event `ON_GLOBAL_INIT` is emitted. `ON_GLOBAL_INIT`:: diff --git a/doc/design/plugins/PluginBrainstorm.txt b/doc/design/extension/PluginBrainstorm.txt similarity index 100% rename from doc/design/plugins/PluginBrainstorm.txt rename to doc/design/extension/PluginBrainstorm.txt diff --git a/doc/design/plugins/PluginVersioning.txt b/doc/design/extension/PluginVersioning.txt similarity index 100% rename from doc/design/plugins/PluginVersioning.txt rename to doc/design/extension/PluginVersioning.txt diff --git a/doc/design/extension/index.txt b/doc/design/extension/index.txt new file mode 100644 index 000000000..4291329a8 --- /dev/null +++ b/doc/design/extension/index.txt @@ -0,0 +1,19 @@ +Extending the Application +========================= + +During the inception phase, the link:/x/rfc/AllPluginInterfacesAreC.html[idea was commended] +to bootstrap the application as a multi-lingual compound of plug-ins, for maximum flexibility +and ease of contribution. Although this approach garnered much support initially, the ensuing +discussions questioned its viability, with well-known plugin-based systems and frameworks +cited as cautionary example. + +A prototype implementation +(-> link:PluginBrainstorm.html[draft for the plugin interface]) +was nevertheless developed, but it proved problematic in practical use. +The goal of a flexibly expandable application is still maintained, +but the path to achieving it should be based on actual requirements. + +Development of an https://issues.lumiera.org/ticket/1212[Extension Concept #1212] +was thus placed on the list of medium-term +https://issues.lumiera.org/report/17[»Focus Topics«] + diff --git a/doc/design/index.txt b/doc/design/index.txt index 9e3d27811..80e513f4a 100644 --- a/doc/design/index.txt +++ b/doc/design/index.txt @@ -7,22 +7,23 @@ Lumiera Design Documents // Menu : prepend child design/gui // Menu : prepend child architecture -// Menu : append child plugins +// Menu : append child extension // Menu : append child workflow // Menu : append child governance -Lumiera is to be a professional tool for video editing on +Lumiera aspires to be a professional tool for video editing on GNU/Linux systems. The vision of the development team defines a modern design for the core of a Non-Linear Editing software. A key aspect of Lumiera is the strong separation between the user interface and the processing core. -We are developing Lumiera with a GTK GUI but that will in no way be exclusive; -other GUIs can be written, as well as scripts to drive the core. +We are developing Lumiera with a general purpose GTK UI -- which is loaded as +an optional component, so that the core can also be invoked from scripts to +use the editing and processing capabilities of the engine. This becomes possible by an ongoing effort to decrease coupling. Each major component in the application strives to be open to extensions, but closed against external modification. The presentation, the model and the ``performing'' -of the model are separate and self-contained parts of the application. Lumiera -as a processing core will be able to perform any conceivable -task on video and audio, maybe even tasks entirely unrelated to video editing. +of the model are separate and self-contained parts of the application. +Lumiera as a processing core will be able to perform any conceivable task +on video and audio, maybe even tasks entirely unrelated to video editing. .Workflow ***************** @@ -52,7 +53,8 @@ specialised GUIs. The interface is the component that is closest to the user. I provides purely visual interaction with the user. Within this work environment, the user manipulates, organizes, loads, configures all sorts of data, especially MObjects (media objects) and Assets. These elements are contained within a structure called the Session. + --> link:gui/index.html[GUI/Stage-Layer design documents] +-> link:gui/index.html[GUI/Stage-Layer design documents] + +-> link:{ldoc}/technical/stage/index.html[GUI technical documentation] Steam: the Transformation Layer @@ -87,24 +89,23 @@ the screen or to external output. + Extra Components ---------------- -The Application -~~~~~~~~~~~~~~~ - -The application controls the initialization and shutdown procedures as well as -loading external elements such as plug-ins, which are widely used throughout -Lumiera. It acts as the framework which supports core component operations. This -framework is complemented by a library of commonly used components, algorithms and data structures. + --> link:application/index.html[Application framework] +The Vessel +~~~~~~~~~~ +A structure that is anchored below all layers, yet contains everything, conceptually. +Definitions for building common interfaces are located here, as is the mechanism for initialisation +and shutdown of the application, the loading of bootstrap configuration and optional components. +It encloses and coordinates all parts to work together as a coherent whole. +This -> link:application/index.html[Application framework] is complemented by +a link:{ldoc}/technical/library/index.html[Support-Lbrary] with commonly used components, +algorithms and data structures. -link:plugins/index.html[Plugins] -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Plug-ins play an important role within Lumiera since the application relies -heavily on current FLOSS implementations and external libraries. Moreover, the -application will be configurable and can be extended in various -ways; whenever some extension isn't permanent or will be used only on demand, it -is packaged as a separate module into a plug-in. For example, the GUI of Lumiera -is a plugin. + --> design documents regarding the link:plugins/index.html[Plugins] +Extensions +~~~~~~~~~~ +Lumiera uses proven and field-tested implementations for the actual media processing, +which are available as FLOSS or open-source libraries. To avoid extensive external dependencies +on the main application however, such libraries need to be loaded through _adaptor plug-ins._ +Furthermore, the application will be configurable and it is conceivable that external +extensions will be created. Designing and building a future-proof framework for configuration +and extension is an important medium-term goal for further development. diff --git a/doc/design/lowlevel/index.txt b/doc/design/lowlevel/index.txt index e4bf2de20..431e0f3ab 100644 --- a/doc/design/lowlevel/index.txt +++ b/doc/design/lowlevel/index.txt @@ -1,14 +1,30 @@ -Design Documents: Vault -======================= +Low-level Services +================== -What follows is a summary of Lumiera's *Data Handling Backend* +Towards the _upper side_ of the Lumiera core, in »Steam Layer«, +several components and subsystems work together to provide functionality +that is close to the requirements of the _problem domain_ -- editing and +rendering of media. The implementation of these features however requires +a foundation of generic, technical services: Memory needs to be allocated, +working buffers must be provided and managed, work tasks coordinated. +It is hard to associate these responsibilities directly to +some specific domain, other than just _data processing._ -This is the foundation layer responsible for any high performance or high volume -data access. Within Lumiera, there are two main kinds of data handling: +A common trait of all those special aspects of processing is that they +require special care and attention to achieve the necessary level of +performance, while they are _technical_ in nature and do not directly +reflect and model structures of the real world. + +Data Handling +------------- +The task of editing video and processing media requires to read and write +a significant amount of data, while using the capabilities of the hardware +to the best degree possible. Lumiera thus needs a foundation layer for +high volume data access; two kinds of data handling can be distinguished: * The Session and the object models manipulated through the GUI are kept in memory. - They are backed by a _storage backend,_ which provides database-like storage and - especially logging, replaying and ``Undo'' of all ongoing modifications.. + They will be backed by a _storage backend,_ which provides database-like storage + and especially logging, and controlled execution of _mutating actions._ * Media data is handled _frame wise_ -- prefetching data asynchronously. The goal is to optimize CPU utilization; large scale data IO will be performed diff --git a/doc/design/plugins/index.txt b/doc/design/plugins/index.txt deleted file mode 100644 index 6ad7e6138..000000000 --- a/doc/design/plugins/index.txt +++ /dev/null @@ -1,4 +0,0 @@ -Design Documents: Plug-In -========================= - - * link:PluginBrainstorm.html[early draft for the plugin interface] diff --git a/doc/design/workflow/Verwijlen/WorkflowProposals.txt b/doc/design/workflow/Verwijlen/WorkflowProposals.txt index cfe579ef4..cd778d74d 100644 --- a/doc/design/workflow/Verwijlen/WorkflowProposals.txt +++ b/doc/design/workflow/Verwijlen/WorkflowProposals.txt @@ -1,16 +1,15 @@ Lumiera Workflow Proposals ========================== :Author: Wouter Verwijlen -:Date: 19 August 2025 +:Date: 16 March 2026 :TOC: -I would like to share a first version of a collection of workflow ideas for Lumiera. These ideas come -from having cut hundreds (or possibly thousands) of videos over the past 20 years as a professional -editor, working in Avid Media Composer, Adobe Premiere Pro (from before it was called "Pro"), -Lightworks and DaVinci Resolve. +I would like to share a collection of workflow ideas for Lumiera. These ideas come from +having cut hundreds (or possibly thousands) of videos over the past 20 years as a professional +editor, working in Avid Media Composer, Adobe Premiere, Lightworks and DaVinci Resolve. -This document focuses on analysing the behaviour of Non-linear Editors (NLE's), in other words -software in which a user can edit films and videos, in order to discuss possible solutions for the +This document focuses on analysing the behaviour of Non-linear Editors (NLE's): software +in which a user can edit films and videos, in order to discuss possible solutions for the design of Lumiera. The main ideas behind these proposals are: @@ -178,14 +177,354 @@ specific project in order to select the parts needed for the final edit. I will which different NLE's handle this and will propose possible directions for Lumiera to support certain workflows. -[red]#To be written:# +Project creation +~~~~~~~~~~~~~~~~ +When starting a new project, a user often needs to set the project resolution and framerate. Adobe +Premiere (optionally) shows an import panel immediately after this. The first time you drop a clip +on a timeline, Resolve and Premiere will ask if you want to change your project settings to match +the clip settings, when these are different. + +.Premiere's import panel +image::{imgg}/wouter/v4-01-importPanel.jpg[width="100%", alt="Premiere's import panel"] + +Projects themselves exist within an application's database (Resolve), or as a file/folder on local or +shared storage (Avid, Premiere, Lightworks, FCP). + +One of the first things that needs to be setup is the organisational structure of the project, in other +words: the structure that's going to house all of the project's contents. Most applications use bins for +this, a bin being simply a container for items like video clips. In a sense they are similar to folders +on a computer's storage, except bins usually only exist within the project in your NLE. Avid's bins +also exist as folders on local or shared storage, but this is an exception. + +For top level organisation, bins can either be placed in other bins (Premiere, Resolve) or in +groups/folders (Avid, Lightworks). + +As we'll see throughout this document, Final Cut Pro is usually the only application that takes a +different approach. Projects are called ``libraries'' and FCP sorts footage based on events (top level +containers of timelines, footage and keyword collections), keywords (tags) and collections, instead +of bins (timelines are confusingly called ``projects''). Footage, when imported, is placed in an event +and from there all emphasis is on the metadata, in how you rate and tag (parts of) the footage. This +also serves as a replacement of the subclips feature found in other NLE's. + +.Final Cut Pro's Keywords dialog. + +image::{imgg}/wouter/v4-02-keywords.jpg[width="100%", alt="Final Cut Pro's Keywords dialog"] +^↑``Travel part 1'' and ``Travel part 2'' on the left are events.^ + + +In any case, when working in a professional environment, you will probably want to start with a +template that defines your top level structure, instead of having to recreate your entire bin setup +for every project. Surprisingly, at the time of writing only Adobe Premiere has introduced a project +template feature (in version 2024). In other apps you will need to manually create your own template project +and duplicate that, or even literally copy and paste bins from a template project in a file browser (Avid). + +Importing footage +~~~~~~~~~~~~~~~~~ +Every application works with clips: virtual representations of (parts of) the video files that exist on +your preferred type of storage. Getting footage into your project (``ingest'') will create these clips in +your project. Most apps will offer several options to do this: + +. Creating links to the original video files, wherever these may reside (on your computer, +external storage, network storage or in the cloud). +. Copying your files to a dedicated folder managed by the application (this could again be on +local or network storage). Importing to a dedicated application folder is very useful for +feature film editing and TV production, but is less common elsewhere in the media industry. +Its strong point is that when your media is managed by the application, you won't have to +re-link files when moving to a different editing station. +. Optionally, files can be transcoded during import. New media files will be created in either +the app's dedicated folder or to a user specified location. The transcoded files will be much +more optimised for editing, resulting in better playback performance. + +The latter is part of what is known as the *offline-online* workflow, in which you work with +optimised files, and once an edit is finished (``picture lock''), the clips on the timeline +will be exchanged with clips that link to the original camera files. Often, after colour grading +in an external application, all colour graded shots are rendered with their colour corrections +and placed back onto the timeline in the NLE. + +A specific type of offline-online workflow is the *proxy workflow*, that allows switching +on-the-fly between the proxy files and the original camera files and will ensure that exports +of your video will use these original camera files when rendering. When using a proxy workflow, +you can either create optimised files inside the application itself (or in Premiere's case, via +Adobe Media Encoder) or attach existing proxy files that were created by a third party application +or by the camera during recording. + +After importing media and possibly creating/attaching proxies, clip attributes might need altering. +You might need to change incorrectly determined frame rates or, more commonly, change the audio +channel mapping of your clips (e.g. setup clips with two audio channels as dual mono instead of +stereo). + +Clip types +^^^^^^^^^^ +We usually find the following types of clips in NLE's: + +Source clips:: they represent media files (containing video and/or audio streams) to its full +extent, and for other types of media (most notably still images) -- these usually get a default +duration upon import. +Subclips:: they represent a portion of a video or audio file. +Timeline clips:: duplicates of source clips that only exist within a timeline. Properties of +these clips can often be altered without altering the same properties of the source clip. +Compound clips/nested sequences:: these are in fact (sub)timelines that can contain any +number of tracks and clips. +Multicam clips:: (group clips in Avid), these are compound clips, in which multiple clips are +synchronised and combined into one. This compound contains clips with different camera angles +(for video) or audio recordings for the same event. +Filler (Avid), FX clips (Lightworks) or Adjustment Clips (Premiere, Resolve):: these special clips +can hold effects that are applied to all clips on lower tracks of a timeline, directly underneath. + +Bin types +^^^^^^^^^ +Often there are two types of bins in which these clips sit + +. ``Normal'' bins. +. Search bins (Premiere) / filters (Lightworks) / smart bins (Resolve) / smart collections +(FCP): for these you setup certain rules based on clip names or metadata. All clips to which +these rules apply are automatically added to these bins. This means that the same clip can +simultaneously exist in a regular bin and a search bin, in contrast to normal bins, where each +clip will only exist in a single bin and you'd have to duplicate this clip for it to be also +placed in another bin (Lightworks being the exception here). + +.Filter options in Lightworks +image::{imgg}/wouter/v4-03-filterOptions.jpg[alt="Filter options in Lightworks"] + + +Resolve adds a third kind of bins, known as ``Power bins'': these are bins that are accessible by all projects. + +Bin views +^^^^^^^^^ +Bins offer several views to see its contents. These include: + +* List View / Text View: all items in the bin are listed vertically. Multiple columns will show +different kinds of metadata -- the user can choose which fields are displayed and can +rearrange their order. + +* Icon View (Premiere) / Frame View (Avid) / Thumbnail View (Resolve) / Tile View +(Lightworks): this will show a small frame from each clip in a raster. In general you're able +to change which frame is displayed if you wish to see another. + +* Freeform View (Premiere): same as Icon View, but you can freely position the clips +wherever you want. Note that Avid and Lightworks allow you to do this within Icon/Frame +View itself. + +* Script View (Avid): similar to Text View, but with a frame being displayed for each clip and +a large field for entering comments. In Script View, the vertical order of clips can be +rearranged, in contrast to Text View, which only supports automatic sorting. + +* Metadata View (Resolve): similar to Avid's Script View, except you can't reorder clips. + +* Filmstrip View (Final Cut Pro): similar to Icon View, but [red]#TODO: missing text??# + +Avid is famous for allowing you to save your Text or Script View configuration as a preset that can +be quickly recalled. Premiere allows this through its Metadata Display window, Final Cut Pro by +Control+left clicking any column header and Lightworks via the Columns > Presets menu, but these +methods are less straightforward than Avid's. Another neat feature that Avid has is a bin map: an +overview of bin contents that can be toggled when in Frame View. + +.Creating a custom view preset in Avid Media Composer, which can be selected from the drop-down menu in the top bar. +image::{imgg}/wouter/v4-04-customView.jpg[width="120%", alt="Creating a custom view preset"] + + +Note that all apps offer a search/filter box (or button) on top of each bin view. Final Cut Pro and +Resolve allow skimming clips from the bin view, simply by hovering the mouse on top of the +thumbnail and moving it from left to right. + + +Organising footage +~~~~~~~~~~~~~~~~~~ +Once your footage is represented by clips in bins, organisation can begin. Wildly different methods +exist for this. In the simplest version you will just drop your footage in one or several bins and be +done with it. In professional environments there are often strict bin structures and naming +conventions, and additional methods for organising are being used: + +* First and foremost: with multi-camera shoots and/or when audio has been recorded +separately, now is the time to synchronise these. Create multicam clips and/or synchronise +sound, either in the NLE itself, or in an external application. I personally create source reels +first (see further down this list), export these to another application for syncing, import the +synced reel back, and in case of a multicam shoot, split this reel into multiple reels and +convert each to a multicam clip (Premiere and Resolve both support this). + +* Metadata. This is a very powerful way to search for or filter specific types of content and it +could be a chapter on its own. Metadata is often created by the camera during recording, +added during offloading by a DIT (digital imaging technician) or applied after ingest by +assistant editors. Nowadays metadata is also added by AI/machine learning algorithms to +help identify the contents of a clip, create transcriptions from audio, etc. Avid provides the +option to (automatically or manually) connect clips to lines in a script (ScriptSync). + +* The aforementioned subclips are often used to extract sections out of a longer clip. + +* Sequence/timeline markers to mark points in time, or sections (so-called ranged or spanned +markers; terminology is different per app). ++ +.Plenty of marker options in Adobe Premiere +image::{imgg}/wouter/v4-05-markerOptions.png[width="70%", alt="Marker options in Premiere"] ++ + +* Timeline clip notes (Avid): these can be added to a timeline clip, and will stay connected to +this clip. You can't trim them off and they will automatically move whenever a clip moves. +They can be burned into a render upon export. Premiere and Resolve offer clip markers +instead -- these are similar to sequence markers, but are attached to clips. + +* Tagging or rating (parts of) clips (FCP and to a lesser extent Resolve). FCP's rating only +goes as far as letting you mark clips or sections as ``favourite'' or reject clips entirely, +but its keyword system (as described later in this chapter) is much more elaborate and useful. +Resolve has an option to add keywords, but its workflow is quite clunky when trying to +apply this to a section within a clip. + +* Clip colours. Often there exist multiple levels at which a clip can get a colour. Colours can be +applied based on source clip (usually different colours for different media types), timeline +clip, timeline track, offline footage, different framerates, and more. Avid has the most +options in this field and lets you choose exactly which types of colours to display on the +timeline. Manually set clip colours are a very useful organisational tool, e.g. to differentiate +between several cameras, interviewed persons in a documentary or different locations. ++ +.Avid has many options for choosing how clips are coloured in the timeline +image::{imgg}/wouter/v4-06-clipColor.png[width="50%", alt="Coosing clip colours in Avid"] ++ + +* Flags (Resolve). Similar to markers, but these apply to entire (source) clips, while markers +apply to a single point in time or a time range within a clip. Multiple flags can be added to +each source clip. + +* Reels: source footage is divided over one or multiple timelines to be able to quickly navigate +many clips without having to load individual clips in the source viewer. These are called +``source reels''. A special kind of source reel is a ``sync map'', which contains all footage +of a day of shooting from multiple cameras synced and laid out chronologically. +FCP doesn't quite support source reels and instead offers continuous playback from the +browser window. Resolve has a ``source tape'' option that works in a similar way. + + +.Simple example of a source reel (top), converted to a sync map (bottom) +image::{imgg}/wouter/v4-07-syncMap.png[width="100%", alt="Example of a sync map", link="{imgg}/wouter/v4-07-syncMap.png"] + +Viewing footage +^^^^^^^^^^^^^^^ +Once basic organisation is in place, the editor can start watching and adding further notes/comments +to narrow down large amounts of footage and seek out options of useful content for the edit. Most +apps offer two types of monitors for video playback: source and record/program monitors. FCP has +only a single monitor that switches between these two functions. Some others (Avid, Resolve) give +users the choice between single or multi viewer modes. Lightworks and Avid support an unlimited +amount of floating viewers. + +.You can open (and position) as many viewers as you like in Lightworks +image::{imgg}/wouter/v4-08-multiViewer.jpg[width="120%", alt="Multiple viewers in Lightworks", link="{imgg}/wouter/v4-08-multiViewer.jpg"] + +Viewers are usually coupled with timecode display and have a ton of settings. +I won't go into details about these. + +In most applications you can also watch your source footage from the bin when in Frame/Icon +View. You'll be able to use JKL shortcuts for playback, mark in and out points and add it to the +timeline, similar as to when the clip is loaded in the source viewer. Final Cut Pro lets you click and +drag in-out regions in the thumbnail. + +Selects +^^^^^^^ +Time to create a selection of material to consider for the rough cut. One way to do this, as +previously described, is by creating subclips or by tagging. Drop the subclips in a bin in +Icon / Frame View and you have a good visual overview of selected shots. This is great for selecting +B-roll material, or for projects that require a very visual approach, f.e. trailers, music videos +or aftermovies (not hoping to ever cut one of those again though, but that's a different story 😉). + +Another way to select footage is to cut selections from source clips or source reels to one or more +selects reels: these are timelines that hold your selections and are only used for that purpose. Selects +reels can then be loaded again as if they were a source clip, in order for an another selection to be +made. This is called *reel-to-reel editing*. Many apps allow seeing the source timeline, either in their +own panel (Premiere) or window (Lightworks), or by switching the timeline panel between a source +and record mode (Avid, Resolve). A method known as _pancake timeline editing_ works when an +application supports viewing multiple timelines simultaneously. By stacking timelines above each +other you can easily drag selections from one timeline to another. This is possible in Premiere, +Lightworks and Resolve. + +.Pancake timeline editing in Lightworks +image::{imgg}/wouter/v4-09-pancake.png[width="110%", alt="Example of a sync map", link="{imgg}/wouter/v4-09-pancake.png"] + +Reels can be further managed by using markers to describe certain points in time or sections, and +these markers can be viewed as a list in their own panel. When selecting a marker in this panel, the +playhead of the corresponding monitor will jump to the (start) frame of this marker. Markers can +have multiple input fields and a colour. They are one of the most important tools to be able to make +notes and navigate large amounts of footage over extended periods of time. As an aside: Lightworks +has an option to turn ranged markers into subclips, quite useful, as it allows you to use markers +to make your selection. + +Some NLE's have a ``ripple markers'' option for timeline markers. This means markers will move +when clips in the timeline are shortened or extended. + +FCP has ``todo markers'' that can be checked. And when tagging clips with keywords in FCP, +keyword collections are automatically created in which all sections of clips with the same keyword +can be found. It's also possible to manually create keyword collections and drag and drop clips or +tagged sections (``content ranges'') there. + +Back to the reels: one advantage of reel-to-reel editing is that it's very useful to see the audio +waveforms of your source clips on an actual timeline: this way you can freely zoom and scroll. +Final Cut Pro will only display a clip's waveform at the bottom of the clip preview in the browser, +but when an interview is an hour or longer, this waveform will be squeezed in a way that's of little +use. + +Another advantage of reel-to-reel editing is a sense of space: you get an idea of where certain +footage can be found (more to the left, in the middle, or at the end of a reel). Combined with +markers, clip colours and gaps between sections, this makes it easier to find things back later. A third +advantage is that it's easy to filter in rounds: you can narrow down your footage in multiple reel-to-reel +selection passes. + +I personally often use a variation of reel-to-reel editing, in which I lift sections in my source reel. +I then duplicate this reel, remove the first video and audio track(s) and keep the lifted parts to create +my selects reel. The advantage here is that I can rate sections as _not useful_ (V1), _possibly useful_ +(V2), _quite useful_ (V3) or _highly useful_ (V4) by lifting the video clip one, two or three tracks up -- +see it as a 4-star rating system. I have seen other editors use this method for selecting footage, +although in all fairness I'm not sure how widespread this technique is. + +.Lifting selections; also note the use of clip colours, gaps and markers for organising the selects reel. +image::{imgg}/wouter/v4-10-selectsReel.png[width="100%", alt="Example of a sync map", link="{imgg}/wouter/v4-10-selectsReel.png"] + + +In FCP you can create your own rating system by using keywords (as shown in the screenshot at the +start of this chapter) and then having everything filtered through keyword collections, as described +in this https://blog.frame.io/2016/10/18/make-fcp-x-log-better-1/[blog post at Frame.io]. +Quite handy as well! + +Proposals +~~~~~~~~~ +* Which system(s) of organising a project and selecting footage should Lumiera support? I +would say: most. They don't interfere with each other, so let the user choose. Definitely +support source timelines and let people rate, tag and mark all they want. Subclips might not +be necessary when a tagging/keyword system is present, as Final Cut has proven. + +* Metadata-based organising, searching and filtering of content are _must-haves_. + +* I would include a native rating system (1 to 5 stars should suffice) that works both on clip +sections and entire clips, rather than FCP's favourite/reject system, as it's more granular. + +* As for bins-in-bins vs bins-in-folders: the first one is more flexible (bins can contain both +media files and other bins, while folders can only contain bins) and more closely resembles +how folders work on a PC, so my preference would be to go that route, although there's also +something to be said for the strict division of function that folders+bins have. + +* I would like to propose an option to automatically create source reels based on rules you can +set, for example: one source reel for every day of shooting. These automatic source reels +could have a filter function to select which clips are being displayed on the timeline. This +way it becomes easy to say something like ``show me all clips from the second day of the +shoot from camera A''. + +* A versioning system for timelines is something that only Resolve has recently picked up +with its Create (and Restore) Timeline Backup feature, and honestly I've been amazed by +how this hasn't been a core feature in NLE's for decades now, as it makes so much sense to +have. Let users make snapshots and perhaps make one automatically every day, without the +need to manually duplicate and rename timelines and putting them in some sort of archive +bin. + +* Importing: auto-import based on watch folders could be very useful during certain stages of +an edit. Setup which folders to watch and whenever a new file is dropped in that folder it +will be imported into a bin of choice. Especially handy when working with sound effects. +However, we might not want to be scanning folders all the time. + +* About sound effects: there's a free application called Metadigger that let's you quickly filter +your sound collection by metadata and then copy sound files to your project folder with a +keyboard shortcut. Something like this, built-in, would be quite handy for those of us with +local sound effect libraries. + +* A small one: I'd like each project to remember the last location you imported from. I often +switch between projects and each time I want to import something in current NLE's, the file +dialog points to the last used directory, which is often the wrong one. +Not a big deal, but not ideal either. + -- Asset organisation: bins vs metadata-based structures (+ search bins/filters) -- Viewing footage: single clips vs source reels; continuous playback from browser -- Source monitor + program monitor vs a single monitor -- Creating selects: selects reels, subclips and keyword collections -- On markers and notes -- From selects to editing in the timeline: Timelines in the source viewer: reel to reel editing Chapter 2: the timeline ----------------------- @@ -283,7 +622,7 @@ Editors often proudly share screenshots of their timelines on social media, and impressive, but these are in fact pretty fragile structures. .One of my own timelines, which is relatively simple by comparison to those of many Hollywood movies -image::{imgg}/wouter/01-timeline.jpg[width="110%", alt="Screenshot of a timeline", link="{imgg}/wouter/01-timeline.jpg"] +image::{imgg}/wouter/v3-01-timeline.jpg[width="110%", alt="Screenshot of a timeline", link="{imgg}/wouter/v3-01-timeline.jpg"] So naturally, a question would be: what will happen if we would let go of the track paradigm? This is what Final Cut Pro has done, starting from the rewrite of Final Cut Pro X. At the time, a @@ -300,10 +639,10 @@ primary storyline, and all connected clips automatically move with it. With a mo ignore clip connections, so that you can also easily move a primary clip elsewhere without its connected siblings coming along. -image::{imgg}/wouter/02-trim-magnetic-1.jpg[width="100%", alt="(before trim)"] +image::{imgg}/wouter/v3-02-trim-magnetic-1.jpg[width="100%", alt="(before trim)"] .Extending one clip will automatically move all connected clips in order to keep sync -image::{imgg}/wouter/02-trim-magnetic-2.jpg[width="100%", alt="(after trim)"] +image::{imgg}/wouter/v3-02-trim-magnetic-2.jpg[width="100%", alt="(after trim)"] _Sounds good._ Why not just copy this? One reason is that FCP assumes that all clips that are not on the primary storyline should be connected to this primary storyline. This might work well for fiction @@ -318,7 +657,7 @@ How else could we group clips together that ought to be connected? + We could create groups that are directly editable, like so: .Compound clips -image::{imgg}/wouter/03-grouped-1.png[width="100%", alt="Timeline with compound clips"] +image::{imgg}/wouter/v3-03-grouped-1.png[width="100%", alt="Timeline with compound clips"] However, there are a few problems with this: we expect rendering to happen from top to bottom, in which case the b-roll would cover the subtitles. On top of that, it's hard to see at which points video @@ -327,7 +666,7 @@ clips overlap. Last but not least: it looks rather unorganised. Can we restructure this? Perhaps like this: .Overlapping compound clips -image::{imgg}/wouter/03-grouped-2.png[width="100%", alt="Timeline with stacked overlapping compound clips"] +image::{imgg}/wouter/v3-03-grouped-2.png[width="100%", alt="Timeline with stacked overlapping compound clips"] But how exactly would a user interact with a timeline like this? I will leave the answer to that question for an actual design document, as for now I just wanted to show the concept of grouping so @@ -354,7 +693,7 @@ FCP's precision editor. It doesn't change the working of keyboard shortcuts or t alters the timeline via animations in a very fluid way. .Final Cut Pro's precision editor -image::{imgg}/wouter/04-fcp-precision-editor.jpg[precision editor] +image::{imgg}/wouter/v3-04-fcp-precision-editor.jpg[precision editor] Another example is the Trim View in DaVinci Resolve: whenever you perform a trimming operation, the program monitor displays a trim view (showing the frames left and right of the cut @@ -386,7 +725,7 @@ the use of a _contextual bar_ that appears over the bottom part of the timeline mode is active. .Contextual bar with options for the mode/tool -image::{imgg}/wouter/05-contextual-bar.png[width="100%", alt="contextual bar"] +image::{imgg}/wouter/v3-05-contextual-bar.png[width="100%", alt="contextual bar"] Currently I'm thinking of using this bar on two or three occasions: @@ -467,10 +806,10 @@ One last thing I would like to address is the destruction that an insert operati this can be prevented by proper track management, but you can wonder why you would ever want an insert action to cause the following: -image::{imgg}/wouter/06-sync-insert-1.png[height=110, alt="sync insert"] +image::{imgg}/wouter/v3-06-sync-insert-1.png[height=110, alt="sync insert"] .Inserting a clip with sync locks enabled for V2 might split a clip in Avid Media Composer. -image::{imgg}/wouter/06-sync-insert-2.png[height=110, alt="sync insert"] +image::{imgg}/wouter/v3-06-sync-insert-2.png[height=110, alt="sync insert"] In this case, if you'd like the clips on V2 to stay in sync, the overlapping clip will be split in two. If you don't want this to happen you would have to move the overlapping clip to another track @@ -622,7 +961,7 @@ will move both the viewport and playhead. Clicking the left mouse button will ac position and close the widget. + .Skim widget -image::{imgg}/wouter/07-1-skim.png[width="100%", alt="A skim widget rendered as overlay"] +image::{imgg}/wouter/v3-07-1-skim.png[width="100%", alt="A skim widget rendered as overlay"] Autoscroll widget:: when the user moves the mouse cursor slightly left or right, it will enter zones @@ -631,14 +970,14 @@ When the user moves the mouse farther away from the initial starting point, the increase. This will also work vertically. The playhead will move along as well. + .Autoscroll widget -image::{imgg}/wouter/07-2-autoscroll.png[width="100%", alt="A widget for auto-scrolling"] +image::{imgg}/wouter/v3-07-2-autoscroll.png[width="100%", alt="A widget for auto-scrolling"] Zoom widget:: moving the mouse left or right from the centre will zoom horizontally, up and down will zoom vertically. + .Zoom widget -image::{imgg}/wouter/07-3-zoom.png[width="100%", alt="An overlay widget to control zooming"] +image::{imgg}/wouter/v3-07-3-zoom.png[width="100%", alt="An overlay widget to control zooming"] Mouse-only navigation: @@ -648,7 +987,7 @@ mouse button. When released, it will commit to the new location/view. If a user immediately releases the button, a regular context menu will appear. + .Combined skim, zoom and scroll widget -image::{imgg}/wouter/07-4-combined.png[width="100%", alt="Overlay widget to combin skim, zoom and autoscrol function"] +image::{imgg}/wouter/v3-07-4-combined.png[width="100%", alt="Overlay widget to combin skim, zoom and autoscrol function"] Why popup widgets? @@ -708,7 +1047,7 @@ what any other NLE does at the moment, but enabling multi-clip selection by keyb good reason to go in this direction. .Selection mockup: selected clips are highlighted in yellow, the cursor is a white dot. -image::{imgg}/wouter/08-selection.png[width="100%", alt="selecting clips"] +image::{imgg}/wouter/v3-08-selection.png[width="100%", alt="selecting clips"] For selecting, we can introduce an ``add to selection'' key which adds the highlighted clip to the current selection (apart from the convention of using Ctrl+mouse click). We could also think of @@ -723,10 +1062,10 @@ Most NLE's have functions for selecting many clips at once, called ``select forw backward''. In Adobe Premiere these are tools: click and you select all clips to either the left or right of the mouse cursor, and using Shift will toggle between selecting all clips on a single track or on all tracks (it doesn't respect track activation status though, which it should). Other NLE's (Resolve, -Avid, Lightworks) only have keyboard shortcuts for these functions, which do only affect activated -tracks. This is a bit faster: in Premiere you need to switch to the Select Forward/Backward Tool, -make your selection, then switch back to your previous tool. With a keyboard shortcut you only -have to press a single key (if your track selection is correct). +Avid, Lightworks) have keyboard shortcuts for these functions (in Avid you can place a button in the +GUI as well), which do affect activated tracks only. This is a bit faster: in Premiere you need to +switch to the Select Forward/Backward Tool, make your selection, then switch back to your previous +tool. With a keyboard shortcut you only have to press a single key (if your track selection is correct). Arranging clips ~~~~~~~~~~~~~~~ @@ -789,7 +1128,7 @@ entering trim mode. * Lightworks has a trim mode that defaults to ripple trimming, unless you specifically use the “drag to black” feature by Control-clicking trim edges when entering trim mode. When using the keyboard, one needs to mark a section and use the Remove command (similar to -Avid''s ``lift'') to create a gap. +Avid's ``lift'') to create a gap. Next to ripple/non-ripple trimming there are three other edit operations closely related to trimming that need to be mentioned: @@ -816,10 +1155,10 @@ tracks automatically stay in sync. This means (manually) enabling or disabling t track. Once set, trimming a single clip can be enough to have all tracks move along, although you might still need to manually select trim sides for overlapping clips to get the desired result. -image::{imgg}/wouter/09-sync-trim-1.png[height=140, alt="trimming with sync-lock(before)"] +image::{imgg}/wouter/v3-09-sync-trim-1.png[height=140, alt="trimming with sync-lock(before)"] .Avid's sync locks will split longer clips by default, in order to keep sync. Such destructive behaviour should be avoided in Lumiera. -image::{imgg}/wouter/09-sync-trim-2.png[height=140, alt="trimming with sync-lock(after)"] +image::{imgg}/wouter/v3-09-sync-trim-2.png[height=140, alt="trimming with sync-lock(after)"] Without sync locks you will have to select trim sides on every track that needs to stay in sync, although, as we'll see, even that can be accomplished by a single keystroke. FCP is the exception @@ -850,7 +1189,7 @@ These commands can be combined: when pressing multiple keys, new trim sides are selection. This allows for quickly selecting many trim sides on many tracks. .One keystroke led to the selection of these four trim sides in Lightworks -image::{imgg}/wouter/10-trim-side.png["select trim side"] +image::{imgg}/wouter/v3-10-trim-side.png["select trim side"] Another feature that was already mentioned in a previous chapter, is FCP's precision editor, which can be triggered by double clicking a cut or by keyboard shortcut. This shows how much footage @@ -937,7 +1276,7 @@ exited automatically whenever the user clicks in an empty spot of the timeline o mode key (T). This key can also be used to return to the last-used trim configuration. .Mockup of the trim mode. The striped area marks the pre- and post-roll that will be previewed. -image::{imgg}/wouter/11-trim-mode.png[width="100%", alt="using trim mode"] +image::{imgg}/wouter/v3-11-trim-mode.png[width="100%", alt="using trim mode"] Splitting and merging clips @@ -973,7 +1312,7 @@ I would like to propose the ability to divide a timeline into multiple sections. will have a header in the ruler that can be edited, to give each section a name (similar to how duration markers in Premiere are displayed). -image::{imgg}/wouter/12-sections.png[width="100%", alt="Timeline with sections"] +image::{imgg}/wouter/v3-12-sections.png[width="100%", alt="Timeline with sections"] The benefits of sections: @@ -1030,7 +1369,7 @@ transition from a clip's start or end to the position of the playhead. * Avid has a configuration panel for adding transitions: -image::{imgg}/wouter/13-transition.png[align="center", alt="transition properties"] +image::{imgg}/wouter/v3-13-transition.png[align="center", alt="transition properties"] * Some apps allow adding transitions to every cut in the in-out range (Avid, Lightworks). Avid gives you the option to skip cuts that already have a transition. Very useful for adding short @@ -1100,6 +1439,12 @@ Chapter 3: finishing -------------------- [red]#To be written.# +- Nodes vs Layers +- Audio mixing +- Color grading +- Visual effects compositing +- Titling + Chapter 4: a broader GUI concept -------------------------------- [red]#To be written.# diff --git a/doc/devel/Status.txt b/doc/devel/Status.txt index ea1389ec1..d35493187 100644 --- a/doc/devel/Status.txt +++ b/doc/devel/Status.txt @@ -51,7 +51,7 @@ Current Release [red]#Lumiera is not usable yet#. - A »Preview Release« was - https://git.lumiera.org/?p=LUMIERA;a=tag;h=refs/tags/v0.pre.04[published at Dec 2, 2025] + https://git.lumiera.org/gitweb?p=LUMIERA;a=tag;h=refs/tags/v0.pre.04[published at Dec 2, 2025] - link:/debian[DEB packages] were built for »Trixie« and »Noble« diff --git a/doc/devel/meeting_summary/2008-10-10.txt b/doc/devel/meeting_summary/2008-10-10.txt index 0f33162ed..f5ea10590 100644 --- a/doc/devel/meeting_summary/2008-10-10.txt +++ b/doc/devel/meeting_summary/2008-10-10.txt @@ -214,7 +214,7 @@ Interface Definition Language ----------------------------- In his work on the plugin loader, _Cehteh_ created a first draft how to export an interface, and calls for review. An example can be found in -https://git.lumiera.org/?p=LUMIERA;a=blob;f=tests/backend/test-interfaces.c;h=09fa25afa3cb57b79a2717d0bf3a8253afeba000;hb=ebcc5c7fd2d8700f3375e3309b724ac2517a0f40[tests/backend/test-interfaces.c] +https://git.lumiera.org/gitweb?p=LUMIERA;a=blob;f=tests/backend/test-interfaces.c;h=09fa25afa3cb57b79a2717d0bf3a8253afeba000;hb=ebcc5c7fd2d8700f3375e3309b724ac2517a0f40[tests/backend/test-interfaces.c] An interface is a list of »slots« mapping functions. The interface descriptor is itself realised as interface, and thus can be versioned, extended and specialised. diff --git a/doc/devel/meeting_summary/2009-01-14.txt b/doc/devel/meeting_summary/2009-01-14.txt index 4531e2702..ab0b52f45 100644 --- a/doc/devel/meeting_summary/2009-01-14.txt +++ b/doc/devel/meeting_summary/2009-01-14.txt @@ -95,7 +95,7 @@ For now the focus is on getting a single frame into the gui and there will be ti The devs agreed to meet up on Saturday to get the latest version up and running. -NOTE: Demonstration in https://git.lumiera.org/?p=LUMIERA;a=blob;f=src/common/guifacade.cpp;h=60a16d0f67bd95b43e44662daa9c0525235e14f5;hb=5f10e65852de2c51364c2e2a5a1cf29cae81c6b2#l64[guifacade.cpp] +NOTE: Demonstration in https://git.lumiera.org/gitweb?p=LUMIERA;a=blob;f=src/common/guifacade.cpp;h=60a16d0f67bd95b43e44662daa9c0525235e14f5;hb=5f10e65852de2c51364c2e2a5a1cf29cae81c6b2#l64[guifacade.cpp] (The thread shall be forked from within the GUI code not from the facade) This was discussed and ichthyo has already pushed a new cleaned up state and resolved most of the dependency problems with the liblumiera. It raised the question: do we prefer a UI facade to a GUI facade or do we want an extra script facade someday? diff --git a/doc/devel/rfc/ApplicationStructure.txt b/doc/devel/rfc/ApplicationStructure.txt index f08c94c3e..1732b466e 100644 --- a/doc/devel/rfc/ApplicationStructure.txt +++ b/doc/devel/rfc/ApplicationStructure.txt @@ -191,7 +191,7 @@ don't think we get much program options, maybe something to set a GUI skin. Moreover, I've written last year a thin wrapper around the commandline and integrated it with the boost options parser such that user code can receive the remaining options as a vector of `std::string`s. Please have a look at -https://git.lumiera.org/?p=LUMIERA;a=blob;f=tests/common/mainsuite.cpp;h=455bfd98effd0b7dbe6597f712a1bdfa35232308;hb=80e1e382f42512ebf2e10a802f77e50327b8fb73[the test class runner main] +https://git.lumiera.org/gitweb?p=LUMIERA;a=blob;f=tests/common/mainsuite.cpp;h=455bfd98effd0b7dbe6597f712a1bdfa35232308;hb=80e1e382f42512ebf2e10a802f77e50327b8fb73[the test class runner main] for an usage example. I really want our Lumiera main to be clean and expressive in the way showed there. Probably the most important part of the startup is pulling up the session core; because of that I think most of the startup diff --git a/doc/devel/rfc/ArchitectureOverview.txt b/doc/devel/rfc/ArchitectureOverview.txt index 625060e25..ac4390c45 100644 --- a/doc/devel/rfc/ArchitectureOverview.txt +++ b/doc/devel/rfc/ArchitectureOverview.txt @@ -15,8 +15,8 @@ This proposal intends to capture envisioned Architecture of the Application. image:{ldoc}/devel/draw/rendered/Lumi.Architecture-1.png[Overview of Lumiera Architecture] ^The SVG source of this drawing is -https://git.lumiera.org/?p=LUMIERA;a=blob_plain;f=doc/devel/draw/Lumi.Architecture-1.svg;hb=HEAD[maintained in GIT(2008)]· -https://git.lumiera.org/?p=LUMIERA;a=blob_plain;f=doc/devel/draw/Lumi.Architecture-2.svg;hb=HEAD[(current: 2022)]^ +https://git.lumiera.org/gitweb?p=LUMIERA;a=blob_plain;f=doc/devel/draw/Lumi.Architecture-1.svg;hb=HEAD[maintained in GIT(2008)]· +https://git.lumiera.org/gitweb?p=LUMIERA;a=blob_plain;f=doc/devel/draw/Lumi.Architecture-2.svg;hb=HEAD[(current: 2022)]^ Description @@ -75,7 +75,7 @@ Conclusion ---------- Accepted. The drawing was moved into the GIT tree. + It was upgraded and reworked -https://git.lumiera.org/?p=LUMIERA;a=blob_plain;f=doc/devel/draw/Lumi.Architecture-2.svg;hb=0b9f2e2c3139cad542013caac593cb5972a2e3f6[in 2022] +https://git.lumiera.org/gitweb?p=LUMIERA;a=blob_plain;f=doc/devel/draw/Lumi.Architecture-2.svg;hb=0b9f2e2c3139cad542013caac593cb5972a2e3f6[in 2022] Back to link:/x/DesignProcess.html[Lumiera Design Process overview] diff --git a/doc/devel/rfc/DesignRenderNodesInterface.txt b/doc/devel/rfc/DesignRenderNodesInterface.txt index 0dc00a399..7c716f071 100644 --- a/doc/devel/rfc/DesignRenderNodesInterface.txt +++ b/doc/devel/rfc/DesignRenderNodesInterface.txt @@ -106,7 +106,7 @@ details of carrying out the operations are much more involved. Please see the link/wiki/renderengine.html#Rendering[Development internal doc (TiddlyWiki)] and the -https://git.lumiera.org/?p=LUMIERA;a=blob;f=src/proc/engine/procnode.hpp;h=9cf3a2ea8c33091d0ee992ec0fc8f37bb5874d34;hb=f9452f654cf39a4abbbeecfd81bef51aa347b321[Draft: ProcNode (src)] +https://git.lumiera.org/gitweb?p=LUMIERA;a=blob;f=src/proc/engine/procnode.hpp;h=9cf3a2ea8c33091d0ee992ec0fc8f37bb5874d34;hb=f9452f654cf39a4abbbeecfd81bef51aa347b321[Draft: ProcNode (src)] for details (and/or contact Ichthyo for in-depth discussion of those technical details) diff --git a/doc/devel/rfc/DevelopmentFramework.txt b/doc/devel/rfc/DevelopmentFramework.txt index 0a430d65e..8a4000f53 100644 --- a/doc/devel/rfc/DevelopmentFramework.txt +++ b/doc/devel/rfc/DevelopmentFramework.txt @@ -149,7 +149,7 @@ We soon settled upon the following toolchain, which is used up to today: history is consolidated eventually into the link:{rfc}/MasterRepositorySetup.html#historical_remark[»official Lumiera repository«], which is made accessible at - https://git.lumiera.org/?p=LUMIERA;a=summary[`git://git.lumiera.org/LUMIERA`] + https://git.lumiera.org/gitweb?p=LUMIERA;a=summary[`git://git.lumiera.org/LUMIERA`] - We never used link:{rfc}/GitSubmoduleTransistion.html#_comments[Git submodules]... - https://www.scons.org/[SCons] became the primary buildsystem soon, with Autotools discontinued in 2011 @@ -158,14 +158,14 @@ We soon settled upon the following toolchain, which is used up to today: - Website and documentation pages are generated by https://asciidoc-py.github.io/[Asciidoc], with source files for documentation maintained - https://git.lumiera.org/?p=LUMIERA;a=blob;f=doc/index.txt;h=9bc0f11e6fb67588388e7616320b695841571278;hb=20f3252892d5b53a05ecd8aeb571122806baba17[in-tree], + https://git.lumiera.org/gitweb?p=LUMIERA;a=blob;f=doc/index.txt;h=9bc0f11e6fb67588388e7616320b695841571278;hb=20f3252892d5b53a05ecd8aeb571122806baba17[in-tree], published - https://git.lumiera.org/?p=website;a=blob;f=build_website.sh;h=60b90e18d5732d66566baf2c20a217212a5dcece;hb=aefc91e13a95a3c742dc8143c374ec982e0aa56b[automatically on push]. + https://git.lumiera.org/gitweb?p=website;a=blob;f=build_website.sh;h=60b90e18d5732d66566baf2c20a217212a5dcece;hb=aefc91e13a95a3c742dc8143c374ec982e0aa56b[automatically on push]. - A https://classic.tiddlywiki.com/[TiddlyWiki] is used for development notes - for Unit-Tests, we use our own minimalist - https://git.lumiera.org/?p=LUMIERA;a=blob;f=src/lib/test/run.hpp;h=035633af9bde7a8053bcd304baffc21693dcf031;hb=20f3252892d5b53a05ecd8aeb571122806baba17[Test-runner]. + https://git.lumiera.org/gitweb?p=LUMIERA;a=blob;f=src/lib/test/run.hpp;h=035633af9bde7a8053bcd304baffc21693dcf031;hb=20f3252892d5b53a05ecd8aeb571122806baba17[Test-runner]. and a - https://git.lumiera.org/?p=LUMIERA;a=blob;f=tests/test.sh;h=8c13810eb475bbacd97c528607e6750f47bb145e;hb=20f3252892d5b53a05ecd8aeb571122806baba17[shell script] + https://git.lumiera.org/gitweb?p=LUMIERA;a=blob;f=tests/test.sh;h=8c13810eb475bbacd97c528607e6750f47bb145e;hb=20f3252892d5b53a05ecd8aeb571122806baba17[shell script] to support definition of test collections. - A Debian package is maintained with https://wiki.debian.org/PackagingWithGit[git-buildpackage] and published diff --git a/doc/devel/rfc/DistributedDevelopmentFramework.txt b/doc/devel/rfc/DistributedDevelopmentFramework.txt index 9f8e50592..e2d43d7cc 100644 --- a/doc/devel/rfc/DistributedDevelopmentFramework.txt +++ b/doc/devel/rfc/DistributedDevelopmentFramework.txt @@ -68,9 +68,9 @@ rather -- a task no one was willing to take on. Yet the Lumiera project succeeded in building a web presence based on https://asciidoc-py.github.io/[Asciidoc] sources -https://git.lumiera.org/?p=website;a=summary[checked into Git], +https://git.lumiera.org/gitweb?p=website;a=summary[checked into Git], with a lightweight -https://git.lumiera.org/?p=website;a=blob;f=build_website.sh;h=60b90e18d5732d66566baf2c20a217212a5dcece;hb=aefc91e13a95a3c742dc8143c374ec982e0aa56b[commit-hook script] +https://git.lumiera.org/gitweb?p=website;a=blob;f=build_website.sh;h=60b90e18d5732d66566baf2c20a217212a5dcece;hb=aefc91e13a95a3c742dc8143c374ec982e0aa56b[commit-hook script] to automatically publish any changes on git-push -- a scheme which comes close to the gist of this RfC. diff --git a/doc/devel/rfc/MasterRepositorySetup.txt b/doc/devel/rfc/MasterRepositorySetup.txt index 81d092acc..86f499f76 100644 --- a/doc/devel/rfc/MasterRepositorySetup.txt +++ b/doc/devel/rfc/MasterRepositorySetup.txt @@ -16,7 +16,7 @@ This describes how the shared MASTER is set up and syncronized. Description ~~~~~~~~~~~ We have now a shared master repository in -https://git.lumiera.org/?p=LUMIERA;a=summary[/git/LUMIERA]. +https://git.lumiera.org/gitweb?p=LUMIERA;a=summary[/git/LUMIERA]. - The public/anonymous git url is [yellow-background]#`git://git.lumiera.org/LUMIERA`# - for people with ssh access it is pushable at [purple]#`UID@git.lumiera.org:/git/LUMIERA`#. @@ -71,10 +71,10 @@ That being said -- the shared master at [yellow-background]#`git://git.lumiera.o still contains the _official_ state of the Lumiera codebase, while actual development is published first in a repository per developer (so effectively in -https://git.lumiera.org/?p=lumiera/ichthyo;a=summary[the Ichthyo repository] +https://git.lumiera.org/gitweb?p=lumiera/ichthyo;a=summary[the Ichthyo repository] for the time being); development is consolidated intermittently into the -https://git.lumiera.org/?p=LUMIERA;a=summary[LUMIERA Git]. +https://git.lumiera.org/gitweb?p=LUMIERA;a=summary[LUMIERA Git]. '''' diff --git a/doc/devel/rfc/NoBugFlags.txt b/doc/devel/rfc/NoBugFlags.txt index e6b4ad19d..0d2afdbfc 100644 --- a/doc/devel/rfc/NoBugFlags.txt +++ b/doc/devel/rfc/NoBugFlags.txt @@ -18,7 +18,7 @@ Propose a documentation/planning about the setup. Description ~~~~~~~~~~~ Take a look at my draft at: -https://git.lumiera.org/?p=LUMIERA;a=blob;f=doc/devel/nobug_flags.txt;h=325141acb52a7be7f75425359b6057c74ed30253;hb=a03e3c5e73a4c0be2264ac4e4e0f02eb8749837b[nobug_flags.txt] +https://git.lumiera.org/gitweb?p=LUMIERA;a=blob;f=doc/devel/nobug_flags.txt;h=325141acb52a7be7f75425359b6057c74ed30253;hb=a03e3c5e73a4c0be2264ac4e4e0f02eb8749837b[nobug_flags.txt] I've added the things I planning for the backend, others might add their own plans there too. So far this is an early draft, comments welcome. diff --git a/doc/devel/rfc/SchedulerRequirements.txt b/doc/devel/rfc/SchedulerRequirements.txt index e3a90503d..bfc3c1cc0 100644 --- a/doc/devel/rfc/SchedulerRequirements.txt +++ b/doc/devel/rfc/SchedulerRequirements.txt @@ -210,7 +210,7 @@ during this effort leads to the following refinements: - Workers manage their capacity through a statistical distribution of _sleep times._ In this form, the proposal can be considered as implemented. + -^(see -> https://git.lumiera.org/?p=LUMIERA;a=commit;h=d71eb37[d71eb37] +^(see -> https://git.lumiera.org/gitweb?p=LUMIERA;a=commit;h=d71eb37[d71eb37] and https://issues.lumiera.org/ticket/1344[#1344])^ Ichthyostega:: '2024-04-20 01:55 CEST' diff --git a/doc/devel/uml/BoUML-diagrams.txt b/doc/devel/uml/BoUML-diagrams.txt index 72ea556da..869c4195b 100644 --- a/doc/devel/uml/BoUML-diagrams.txt +++ b/doc/devel/uml/BoUML-diagrams.txt @@ -49,6 +49,4 @@ On occasion, we'll replace them by drawings from our current UML model. 128517: classdiagram 142725 "Time flavours" 133637: classdiagram 143877 "Player Entities" 133637: objectdiagram 144005 "Play Process Structure" -133637: sequencediagram 145157 "output data exchange" -133637: classdiagram 151685 "Player Output" 128773: classdiagram 158213 "UIBus" diff --git a/doc/devel/uml/OutputSlot_components.svg b/doc/devel/uml/OutputSlot_components.svg new file mode 100644 index 000000000..98dca4bee --- /dev/null +++ b/doc/devel/uml/OutputSlot_components.svg @@ -0,0 +1,8639 @@ + + + OutputSlot components + Generated with Qt + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + create + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + «for each channel» + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 1 + + + + + + «active» + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + «smart_handle» + + + + + + + + + + BufferProxyAdaptor + + + + + + + + + + + + + + + getDescriptor() + + + + + + + + + + + + + + + + BufferProvider + + + + + + + + + + + + + + + getDescriptor() + + + + + + + + lockBuffer() + + + + + + + + + + + + + + + + OutputManager + + + + + + + + + + + + + + + openOutput() + + + + + + + + + + + + + + + + SpecificConnection + + + + + + + + + + + + + + + claimBufferFor() + + + + + + + + publish() + + + + + + + + release() + + + + + + + + + + + + + + + + BuffHandle + + + + + + + + + + + + + + + emit() + + + + + + + + release() + + + + + + + + + + + + + + + + Connection + + + + + + + + + + + + + + + getBufferSize() + + + + + + + + claimBufferFor() + + + + + + + + publish() + + + + + + + + release() + + + + + + + + shutDown() + + + + + + + + + + + + + + + + OutputSlot + + + + + + + + + + + + alloc_ + + + + + + + + + + + + + + isActive() + + + + + + + + getTimings() + + + + + + + + getOpenedSinks() + + + + + + + + + + + + + + + + DataSink + + + + + + + + + + + + descriptor + + + + + + + + + + + + + + invoke() + + + + + + + + + + + + + + + + Allocation + + + + + + + + + + + + + + + connect() + + + + + + + + release() + + + + + + + + getTimings() + + + + + + + + + + + + + + + + BuffDescr + + + + + + + + + + + + + + + lock() + + + + + + + + + + + + + + + + + + + CON + + + + + + + + AllocState + + + + + + + + + + + + connections_ + + + + + + + + + + + + + + AllocState() «constructor» + + + + + + + + connect() + + + + + + + + release() + + + + + + + + getTimings() + + + + + + + + + + + + Output Management + + + + + + Actual Output Driver + + + + + + + + + + + + Knowledge specific + + + + + + + for this kind of + + + + + + + output driver + + + + + + + + + + + + + + + Client + + + + + + + front-end + + + + + + + + + + + OutputSlot components + + + + Ichthyostega + + + Lumiera documentation + + + GPL v2+ or CC-By-SA 4 + + + + + + + + + + + + + + diff --git a/doc/devel/uml/OutputSlot_sequence.svg b/doc/devel/uml/OutputSlot_sequence.svg new file mode 100644 index 000000000..5a70461b5 --- /dev/null +++ b/doc/devel/uml/OutputSlot_sequence.svg @@ -0,0 +1,1672 @@ + + + OutputSlot call sequence + Generated with Umbrello + + + + + + + GPL v2+ or CC-By-SA 4 + + + + + Ichthyostega + + + OutputSlot call sequence + + Lumiera documentation +(UML created with umbrello 5.0) + + + + + + + + + + + + + + + loop [N] + + + + + + + + : OutputManager + + + + + : Client + + + + + : BuffHandle + + + + + : DataSink + + + + + : OutputSlot + + + + + : AllocState + + + + + : BufferProxy + + + + + + : Connection + + + + + + + + mark_emitted() + destroy + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + publish() + + + + + + + + + + + lockBuffer + claimBufferFor() + access buffer + frame() + accessAs() + getOpenedSinks() + getOutput + new + allocate() + generate data + emit() + release() + detachBuffer() + destroy + shutDown() + + + + + + + + + + + + + + + + + + + + release() + done + + + + + + + + + Data producer + + + + + + Output Slot + + + + + + Output + implementation + + + diff --git a/doc/devel/uml/fig145157.png b/doc/devel/uml/fig145157.png deleted file mode 100644 index dac8e23cb..000000000 Binary files a/doc/devel/uml/fig145157.png and /dev/null differ diff --git a/doc/devel/uml/fig151685.png b/doc/devel/uml/fig151685.png deleted file mode 100644 index 77b41e446..000000000 Binary files a/doc/devel/uml/fig151685.png and /dev/null differ diff --git a/doc/index.txt b/doc/index.txt index 676804e85..0439abeb2 100644 --- a/doc/index.txt +++ b/doc/index.txt @@ -1,7 +1,7 @@ Documentation ============= -This documentation section contains documentation for both users and developers. +This documentation section contains documentation for users and developers alike. == User == @@ -60,8 +60,8 @@ deeper documentation dungeons. At the second layer, the reader must make the first major decision on which router to pursue: documentation with a heavy code based leaning; or more descriptive -documentation on the design of Lumiera which avoids a code narrative. This -section has two main sections: +documentation on the design of Lumiera, which omits most technical details of the implementation. +Consequently, the documentation is divided into two main parts: * link:design/index.html[Lumiera Developer Design Documents] * link:technical/index.html[Lumiera Developer Technical Documents] @@ -75,14 +75,14 @@ section has two main sections: [width="90%",frame="topbot",options="header",cols="^,2<"] |================================================================================================ -|Sub-System|Remarks +|Section|Remarks |link:design/architecture/[Architecture]|Structure of Lumiera |link:design/gui/[GUI]|The Lumiera ⟷ user interaction layer |link:design/model/[Model]|Low-level and high-level models |link:design/engine/[Engine]|Renderer, output generation, ... |link:design/lowlevel/[Low-level]|Low-level and system services |link:design/application/[Application]|Application framework -|link:design/plugins/[Plugins]|Dynamically loaded Extensions +|link:design/extension/[Extension]|Dynamically loaded Extensions |link:design/workflow/[Workflow]|Workflow and Interaction Design |link:design/governance/[Governance]|Meta project concerns |================================================================================================ @@ -93,12 +93,12 @@ section has two main sections: [width="90%",frame="topbot",options="header",cols="^,2<"] |================================================================================================ -|Sub-System|Remarks +|Section|Remarks |link:technical/build/[Build System]|Build, Packaging and CI |link:technical/stage/[Stage]|GUI (technical aspects) |link:technical/steam/[Steam]|Session and processing coordination |link:technical/vault/[Vault]|Low-level operations -|link:technical/library/[Support lib]|Interface and support libraries +|link:technical/library/[Library]|Utilities and custom frameworks |link:technical/code/[Code base]|Code management, organisation, ... |link:technical/infra/[Infrastructure]|Website and developer tooling ... |link:technical/howto/[Developer HowTos]|Instructions and guides for developers diff --git a/doc/technical/build/BuildDependencies.txt b/doc/technical/build/BuildDependencies.txt index f841c1583..29a1851d6 100644 --- a/doc/technical/build/BuildDependencies.txt +++ b/doc/technical/build/BuildDependencies.txt @@ -40,7 +40,7 @@ Lumiera expects a ``standard'' desktop installation running a XServer.footnote:[ Wayland is ``just around the corner'' since 10 years now. Well, recently it seems it is _really_ just around the corner. We are aware of the necessary changes, which are not very complicated to implement (since we are using - GTK-3). However, as long as Wayland simply crashes or hands in our + GTK-3). However, as long as Wayland simply crashes or hangs in our everyday work setup with XFCE, we will not consider this topic a priority. In the meantime, please report any problems you encounter with Lumiera running on XWayland. @@ -97,7 +97,8 @@ Languages and Tools * BOOST (listed below are the Debian package names) - libboost-dev (at least *1.83*) - - libboost-program-options-dev + - notably including `boost::lockfree` + - libboost-program-options-dev (link dependency) * Script languages - Python (*3.10*) required for building (->SCons) @@ -123,7 +124,7 @@ Build Tools * pkg-config * Doxygen (optional) * Asciidoc (optional) -* Git -> https://git.lumiera.org/?p=LUMIERA;a=summary[git://git.lumiera.org/LUMIERA] +* Git -> https://git.lumiera.org/gitweb?p=LUMIERA;a=summary[git://git.lumiera.org/LUMIERA] footnote:[ _Ichthyo_ maintains a repo at https://github.com/Ichthyostega/Lumiera/[Github ^~Ichthyostega/Lumiera^] ] @@ -160,9 +161,9 @@ Libraries which is possible, but would require some considerable effort. As of 2025, GDL is still usable, but no longer available through the standard distribution channels. + We provide a Git repository at our Website for - https://git.lumiera.org/?p=debian/gdl;a=summary[libGDL (3.40) +git://git.lumiera.org/debian/gdl+] + https://git.lumiera.org/gitweb?p=debian/gdl;a=summary[libGDL (3.40) +git://git.lumiera.org/debian/gdl+] and the - https://git.lumiera.org/?p=debian/gdlmm;a=summary[GDL C++ bindings (3.7.3) +git://git.lumiera.org/debian/gdlmm+] + https://git.lumiera.org/gitweb?p=debian/gdlmm;a=summary[GDL C++ bindings (3.7.3) +git://git.lumiera.org/debian/gdlmm+] and we maintain custom Debian packages for both in our link:/debian/[Debian DEB depot]. ] diff --git a/doc/technical/build/LumieraDebianPackage.txt b/doc/technical/build/LumieraDebianPackage.txt index 9d374613f..f94617aed 100644 --- a/doc/technical/build/LumieraDebianPackage.txt +++ b/doc/technical/build/LumieraDebianPackage.txt @@ -79,8 +79,8 @@ How the Lumiera Debian package works Since we maintain both the Debian package and our build system, we can always try to do _the heavy lifting_ in our link:{ldoc}/technical/build/SCons.html[SCons build system], as this helps to keep the Debian packaging as simple as possible. Please have a look at the --> https://git.lumiera.org/?p=debian/lumiera;a=blob;hb=refs/heads/deb;f=debian/control[control file] -· https://git.lumiera.org/?p=debian/lumiera;a=blob;hb=refs/heads/deb;f=debian/rules[debian/rules] +-> https://git.lumiera.org/gitweb?p=debian/lumiera;a=blob;hb=refs/heads/deb;f=debian/control[control file] +· https://git.lumiera.org/gitweb?p=debian/lumiera;a=blob;hb=refs/heads/deb;f=debian/rules[debian/rules] - we rely on the _Debhelper Sequencer_ "`dh_*`" in our 'debian/rules' + -> https://manpages.debian.org/trixie/debhelper/dh.1.en.html[Manpage] @@ -106,7 +106,7 @@ into `master` and marked with a tag there. For the actual packaging, we use these release tags as _upstream tag_ and perform a merge from that tag into the **debian branch**.footnote:[ This _debian branch_ is called `deb` by convention. As mentioned above, it is published through - a separate https://git.lumiera.org/?p=debian/lumiera;a=summary[Git repository `git://git.lumiera.org/debian/lumiera`], + a separate https://git.lumiera.org/gitweb?p=debian/lumiera;a=summary[Git repository `git://git.lumiera.org/debian/lumiera`], which is kept separate, since packaging is considered a ``downstream process''. ] This branch carries only the additions and tweaks done for debian packaging. Yet still, this `deb` @@ -115,7 +115,7 @@ repository as the `master` branch. When the packaging process is complete, the p DEB version is marked with a tag `debian/` Note however that there is another branch in the aforementioned `debian/lumiera` repository, -which is called https://git.lumiera.org/?p=debian/lumiera;a=shortlog;h=refs/heads/depot[depot] +which is called https://git.lumiera.org/gitweb?p=debian/lumiera;a=shortlog;h=refs/heads/depot[depot] and is related to our link:/debian[DEB package depot] on Lumiera.org. This branch carries a completely different directory structure, since it is used with the https://wiki.debian.org/DebianRepository/SetupWithReprepro[»reprepro« tool from Debian] diff --git a/doc/technical/build/SCons.txt b/doc/technical/build/SCons.txt index cd4acb7e2..b5e24e1b3 100644 --- a/doc/technical/build/SCons.txt +++ b/doc/technical/build/SCons.txt @@ -352,6 +352,6 @@ shared object (dynamic library), which is then again a SCons target and can be passed to other executable builders as library dependency for compilation and linking. Look into 'src/SConscript' or 'test/SConscript' to see examples of that technique -- which we also use to define that -global compound target variables like `core`, `app_lib`, `core_lib`, +global compound target variables like `core`, `vessel_lib`, `core_lib`, `vault_lib`, and `support_lib`. These in turn are essential for building the layered dependency hierarchy in our code. diff --git a/doc/technical/code/LinkingStructure.txt b/doc/technical/code/LinkingStructure.txt index c1ab3700a..4618afe19 100644 --- a/doc/technical/code/LinkingStructure.txt +++ b/doc/technical/code/LinkingStructure.txt @@ -322,8 +322,8 @@ actively step by step modules of the application will be placed by default . after the core application has been loaded and all direct dependencies are resolved, - but still before entering `main()`, the class `lumiera::AppState` will be initialised, - which in turn holds a member of type `lumiera::BasicSetup`. The latter will figure out + but still before entering `main()`, the class `vessel::Voyage` will be initialised, + which in turn holds a member of type `vessel::BasicSetup`. The latter will figure out the location of the executable footnote:[this is a Linux-only trick, using `/proc/self/exe`] and require a 'setup.ini' file in the same directory. This setup file is mandatory. diff --git a/doc/technical/howto/crackNuts.txt b/doc/technical/howto/crackNuts.txt index d5741c03e..d92ac46f2 100644 --- a/doc/technical/howto/crackNuts.txt +++ b/doc/technical/howto/crackNuts.txt @@ -88,6 +88,33 @@ test with random numbers:: several threads; each thread should use its own generator then (yet this can be handled sloppy if the _quality_ of the random number actually does not matter!) +special test setup:: + It can be very helpful to use _policy based design_ for some complex evaluation or algorithm, + even in cases where it is clear that the actual usage will only ever involve one specific data type + and configuration. Not only does such a structured approach help with clarifying the design itself, + since the core of the evaluation logic can be abstracted from data representation, but only the + testing and verification of correctness is simplified, as the test can be written to use simple, + human-readable data (strings, symbols), and, furthermore, many kinds of instrumentation can be + included into the test data to gain detailed insight what is happening in the test subject. + +verify behaviour sequence:: + Use the `lib::EventLog`: ++ +- within the test, notable points are marked into the log, possibly with arguments +- after the test, the Event Log can be verified with a search-sequence +- this way it is possible to ensure that some events did happen, and happened + in a very specific order, or that some event did _not_ happen after another event +- the `test::Tracker` defined in 'src/test/tracking-dummy.hpp' will mark all constructor, + destructor and copy events into an embedded Event Log instance. By inheriting or mix-in + from that type, the allocation and instance behaviour of some service can be scrutinised. + +verify incidences and concurrency:: + Aside from using some generic tracing utility, sometimes it is much easier to setup + a _specifically tailored instrumentation_. This can be achieved with 'incidence-count.hpp' ++ +- for instrumentation, the `markEnter()` and `markLeave()` functions need to be invoked +- after the test run, various statistics can be computed and evaluated, including + cumulated time and concurrency for the "cases" marked with enter / leave. Common Tasks @@ -109,6 +136,12 @@ structured data:: - also useful as output format, both for debugging and because it can be matched against an expected structure, which is generated with the DSL notation. - [yellow-background]#(planned)# will be able to map this from/to JSON easily. +scoped collection:: + unique-ownership collection that holds a sequence of objects in a preallocated fixed + storage block. Notably even non-copyable objects can be managed (which is not possible + with a `std::vector`), and type-erased instances of subclasses can be mixed. Furthermore + there is a constructor variant that allows to populate a `ScopedCollection` in RAII-style. + This can e.g. be used to fire-off a set of Threads or open a set of wrapped connections. Iterating ~~~~~~~~~ @@ -134,6 +167,26 @@ IterSource:: even the production-state can be hooked up into the IterSource instance (with a smart-ptr). This allows e.g. to send a Diff to the UI through the UI-Bus, while the actual generation and application both happen demand-driven or _lazy..._ +State Core:: + When some algorithm involves a statefull evaluation, it might be helpful to package that into + a _State Core_ container. This is a class that is expected to expose three operations, which + allow to build an iteration on top. The header 'iter-adapter.hpp' provides several adapters + and wrappers, notable `lib::IterStateWrapper` wraps any _state core_ and provides a + Lumiera Iterator interface; likewise, `IterExplorer` heavily relies on that concept and + is able to accept and iterate any state core in a pipeline. +special iterators:: + Some additional adaptors are provided as building block so that specific evaluations + can be made to conform with the »Lumiera Iterator« concept ++ +- several given iterators can be combined into a iteration over tuples with 'lib/iter-zip.hpp' +- there is also a variation 'lib::izip' that adds a running index into the _zipping operation_ +- IterExplorer provides a `flatten()` operation, that can be used to join an iterator-of-iterators + into a single seamless iteration lazily. +- when a STL container also supports _backwards iteration_, then a lumiera iterator can be + created that can _switch direction_ dynamically, with the help of 'iter-cursor.hpp' +- a STL »deque« container can be used both as a stack and a queue; the header 'iter-stack.hpp' + provides an iteration adapter for both usage schemes, so that either the stack or the queue + can be _"discharged"_ as a Lumiera Iterator (possibly with an IterExplorer pipeline) Formatting @@ -155,6 +208,25 @@ Formatting because there are way more capable solutions _out there_ ☺) +Setup and wiring +~~~~~~~~~~~~~~~~ +Generally speaking, we prefer _constructor wiring_ whenever possible, to avoid creating +unnecessary statefulness (as would be the case when connecting dependencies through setters). +This design also favours decomposition into self-contained components with a single purpose, +and simplifies error handling with the help of the RAII pattern. + +However, sometimes a piece of code depends on some remote and even global service; in these +cases, the `lib::Depend` framework can be used to access that service via type name. The +actual service can either be created on-demand as a singleton, or it can be installed +explicitly as a service instance. Furthermore, it is possible to "push aside" the +current service instance temporarily and replace it by a _test mock_. + +One point to note is that `lib::Depend` is a _monostate_, and not a _singleton_. +The actual service should be written as a free-standing class, that can be instantiated +directly for the purpose of testing. + + + Language constructs ------------------- Templates @@ -235,4 +307,14 @@ unpack iterator into tuple:: - see tuple-record-init-test.cpp - used in 'command-simple-closure.hpp' to receive parameter values sent via UI-Bus and package them into a tuple for invocation of a Steam-Layer command. ++ +index and iterate over tuple elements:: + This is another example of bridging the compile-time / runtime barrier, using a trampoline. + Obviously such a solution requires that it is somehow possible to define a generic functor / λ that + can accept any of the tuple elements. And, in order to build a runtime index access, we additionally + need to convert each of the tuple elements into a _common reconciled type_. ++ +- If such a type exists, can be figured out with the help of the `meta::CommonReturn` _metafunction_ +- `lib::meta::TupleIdxAdaptor` provides the necessary machinery to define a subscript `operator[]` +- see `TupleIdxAdaptor_test` for some usage examples, including a runtime iteration... diff --git a/doc/technical/index.txt b/doc/technical/index.txt index f3ace95bd..9fcd9007e 100644 --- a/doc/technical/index.txt +++ b/doc/technical/index.txt @@ -9,34 +9,34 @@ Technical Documentation // Menu : prepend child overview -This documentation section contains technical documentation about Lumiera. To -get an *overview* of all the internals and components, you might want to read +This documentation section contains technical documentation about Lumiera. + +To get an *overview* of all the internals and components, you might want to read link:overview.html[Lumiera the Inner Core] == Three Layers -The technical documentation is split in three parts, one for each of the three -main layers of Lumiera. You may want to read the -link:{ldoc}/design/index.html[Design Documents] first to get an overview of all the -components. +The technical documentation is divided into three main parts and some supplements, +corresponding to the code organisation in three layers. The information here can be +very detailed and technical at times, and the topics are organised by individual components. * link:stage/index.html[*Stage Layer*] : Documents related to the GTK based Lumiera GUI - -* link:steam/index.html[*Steam Layer*] : Documents regarding Session, models and the engine structure - +* link:steam/index.html[*Steam Layer*] : Documents regarding Session, the models and the engine * link:vault/index.html[*Vault Layer*] : Documents about the data and processing backend +To better understand the underlying concepts, it might be helpful to also consult the +link:{ldoc}/design/index.html[Design Documents]. + == Tools .Development -* link:/doxy/[*Doxygen* generated documentation] : API documentation of the Lumiera code. +* link:/doxy/[*Doxygen*] : API documentation of the Lumiera code. * organisation of the link:code/index.html[Code Base] in general * link:/x/CodingGuidelines.html[Coding Style] and further guidelines .Building -* link:build/index.html[Buildsystem] : Installation & compilation tools, dependencies and packaging. +* link:build/index.html[Buildsystem] : Installation and compilation tools, dependencies and packaging. .HOWTO -* link:/x/DevHowto.html[Developer *HOWTOs*] : Collection of instructions, recipes, hints and similar - materials intended for developers +* link:/x/DevHowto.html[Developer *HOWTOs*] : Collection of instructions, recipes, hints, misdeeds + and confessions and similar helpful resources for developers diff --git a/doc/technical/infra/TestSupport.txt b/doc/technical/infra/TestSupport.txt index ca5146c76..59d134cbf 100644 --- a/doc/technical/infra/TestSupport.txt +++ b/doc/technical/infra/TestSupport.txt @@ -8,30 +8,32 @@ NOTE: This page holds content copied from the old "main" TiddlyWiki + It needs to be reworked, formatted and generally brought into line ******************************************** -.Lumiera relies on *test-driven development* +.Lumiera employs test-driven development This page discusses the overall organisation of test code, and the tools used for running test cases ******************************************** -Tests are the only form of documentation, known to provides some resilience against -becoming outdated. Tests help to focus on the usage, instead of engaging in spurious +Tests are the only known form of documentation that offers some resilience against +getting outdated. Tests help to focus on the usage, instead of engaging in spurious implementation details. Developers are highly encouraged to write the tests _before_ the actual implementation, or at least alongside and interleaved with expanding the feature set of the actual code. + There may be exceptions to this rule. Not every single bit needs to be covered by tests. Some features are highly cross-cutting and exceptionally difficult to cover with tests. -And sometimes, just an abstract specification is a better choice. +Test code must not duplicate the logic of implementation code, it should represent an +abstracted, outside, logical view of the matter at hand. And sometimes it is the better +choice to simply write an abstract specification in understandable language. As a rule of thumb, consider to write test code which is easy to read and understand, _like a narration_ to show off the relevant properties of the test subject. + Test Structure -------------- -- a *test case* is an executable, expected to run without failure, and optionally producing - some verifiable output -- simple test cases may be written as stand-alone application, while the more tightly - integrated test cases can be written as classes within the Lumiera application framework. +- a *test case* is an executable, expected to run without failure; test code not only invokes + a _subject under test_ -- the important part is to _verify_ its behaviour. - test cases should use the +CHECK+ macro of NoBug to verify test results, since the normal assertions may be de-configured for optimised release builds. - our test runner script 'test.sh' provides mechanisms to check for expected output @@ -41,15 +43,13 @@ several functions within the same test fixture. This allows to create a ``narrat in the code: first do this, than do that, and now that, and now this should happen... Generally speaking, it is up to the individual test to take care or isolate himself from any _dependencies_. Test code and application code uses the same mechanisms -for accessing other components within the application. Up to now (2014), there -was no need for any kind of _dependency injection_, nor did we face any -difficulties with tainted state. +for accessing other components within the application. Test classes are organised into a tree closely mirroring the main application source code tree. Large sections of this test tree are linked together into *test libraries*. Some of these are linked against a specific (sub)scope of the application, like e.g. only against the support library, the application framework or the vault. Since we -use _strict dependencies_, this linking step will spot code not being placed at the +use _strict dependencies_, this linking step will also spot code not placed at the correct scope within the whole system. As a final step, the build system creates a *test runner* application (`target/test-suite`), which links dynamically against _all_ the test libraries and thus against all application dependencies. @@ -81,9 +81,9 @@ to help with automating the build and test execution, test code should adhere to - test classes should be placed into a sub namespace +...::test+ - specific definitions for a single test case may rely on global variables, but these should live within an anonymous namespace -- all test executables should be named according to the pattern +test-*+ +- free standing test executables should be named according to the pattern `test-*` - all test source code is within the 'tests' subtree -- immediately within the 'tests/' directory are the test collection definitions +*.tests+, +- immediately within the 'tests/' directory are the test collection definitions `*.tests`, ordered by number prefixes - below are the directories with test cases, to be grouped into the aforementioned test-executables. - we distinguish two ways to write and link tests: _standalone_ and _suite_ @@ -107,7 +107,7 @@ See the following .hello-world-test example [source,C] ---------------------------------------------------------------- -#include "lib/test/run.hpp" +#include "test/run.hpp" #include using std::cout; diff --git a/doc/technical/infra/debianDepot.txt b/doc/technical/infra/debianDepot.txt index 12f5cb32f..3fad649bc 100644 --- a/doc/technical/infra/debianDepot.txt +++ b/doc/technical/infra/debianDepot.txt @@ -109,7 +109,7 @@ Configuration ~~~~~~~~~~~~~ All the current configuration and the state of the Debian depot index files is tracked in the aforementioned ``debian/lumiera'' Git repository, on a separate branch called -https://git.lumiera.org/?p=debian/lumiera;a=shortlog;h=refs/heads/depot[`depot'] +https://git.lumiera.org/gitweb?p=debian/lumiera;a=shortlog;h=refs/heads/depot[`depot'] TIP: this is a slightly unconventional setup, insofar the `depot' branch carries a directory tree that is completely separate from the main Lumiera source tree, which diff --git a/doc/technical/library/index.txt b/doc/technical/library/index.txt index 67e0a7946..f15c65b9f 100644 --- a/doc/technical/library/index.txt +++ b/doc/technical/library/index.txt @@ -1,41 +1,70 @@ Library documentation ===================== -//Menu: label Support Lib -Alongside with the core Lumiera Application, we're also creating several *Libraries*. + -This section holds pages with in-depth technical documentation regarding those. Please don't -expect a coherent or complete coverage for the time being -- library functionality is typically -``discovered'' while in the middle of the implementation, then factored out, reused and remoulded -over time. +Alongside with the core Lumiera Application, and as part of ongoing development work, +a lot of *Library code* is written. Many of this happens on occasion. Library solutions +are frequently ``discovered'', while in the middle of the implementation, driven by the +desire to re-use some code structure that was built just recently and in a similar way. + + +Support Library +--------------- +Over time, a huge and growing number of generic building blocks has been collected. +Hoever, not all parts of this collection are equally well maintained and presentable. +It happened time and again that some pattern extracted into the library turned out +to be rather some kind or ephemeral habit -- maybe even a bad habit in hindsight -- +and turned into a dead-end eventually. Furthermore, a lot of our library code has +been superseded by very similar features of newer language standards.footnote:[ + It seems like developers working on similar problems tend to have similar ideas + to improve productivity. Unfortunately, the solutions resulting from this convergence + of ideas have a tendency not to ``converge'' so well in practical terms. + For example, Lumiera had a tuple type a long time before a mature tuple implementation + was available in the Boost libraries, yet in the end, we could retrofit most of our + tuple-handling library. However, other cases did not play out so well: we evolved + the idea of ``iterators'' to become a lazy processing pipeline, but unfortunately + our design leans more into the direction of functional programming and is much + more expressive and powerful as abbreviation than the »Range-TS« from C++20. +] +The entrance barrier for something to be added to our _Support Library_ is rather low: + +- extracting it as a library function _must make the client code clearer_ +- it must be possible to describe its purpose and its usage in _generic terms_ -- + without referring to data types and definitions from our domain models. +- it must be _essentially stateless_ and self-contained +- it must be easy to document with an unit-test. + +Over time, parts of the Support Library have evolved into some kind of a framework: +Notably the link:{ldoc}/design/architecture/time/index.html[time and timecode] handling, +our link:Dependencies.html[dependency and service handling], +the link:DiffFramework.html[»Diff framework«], together with the recursive property trees +used for link:{ldoc}/design/architecture/ETD.html[»External Tree Description«], the +link:iterator.html[iterator pipeline builders], the metaprogramming helpers for transforming +function signatures and generic sequences, and our collection of string conversion, parsing +and rendering functions. These all turned into distinguishing traits of the Lumiera code base. + +The link:{ldoc}/technical/overview.html#_library[»Library« section of the »Overview« document] +contains a long list of short descriptions for relevant features of the Support Library. + + +.how to crack nuts... +[NOTE] +In the how-to section, there is a page with a loose collection of +link:{ldoc}/technical/howto/crackNuts.html[successful solution patterns], +many of which rely on unique features of our Support Lib. + Interface Library ----------------- The Lumiera Application is designed deliberately in a way to be controllable by scripts. All of the Session, the models and the vault are fully operational without the GUI. Any significant -functionality can be addressed through the primary Façade Interfaces. +functionality shall be usable through the primary Façade Interfaces alone. Yet in order to create +such an open and extensible application as envisioned, we also need to build up an _interface layer._ +Eventually, this should evolve into a binding library for using Lumiera -- however, none of this +interface building work has been accomplished yet, as of 2026. We do not even have a complete +picture how _event sourcing_ meshes together with our UI commands and the configuration rules. -But, in order to make such an open and extensible application a reality, we need to build up -an 'interface layer' -- provided as library for client code and script adapters to link against. -This library allows to access the primary external interfaces and exposes everything needed to -work with the specifics of the application. - -[,yellow]#TODO as of 1/2012:# _this is just a nebulous plan for now._ Nothing of the work required -really to address facilities from the outside has been done. All we've managed to achieve is actually -to use clean interfaces and decouple the parts of the application appropriately. - -- we still need to identify those core concepts which need to be exposed +- we still need to identify those select core concepts which need to be exposed - we need to refactor the corresponding data types to resolve cross dependencies to the implementation -- then we could think about moving those data types into an _official Lumiera interface_ - -Support Library ---------------- -There is a huge collection of generic building blocks, re-used more or less frequently. -The're written in a generic way, but in accordance to the rest of the source base, relying only -on the general error handling and logging framework used within Lumiera. (Some facilities may depend -on other parts of the Application; this can be qualified as a bug -- please complain when you've found -something of that sort). - - - +- only then we could think about moving those data types into an _official Lumiera interface_ diff --git a/doc/technical/overview.txt b/doc/technical/overview.txt index a37f5d649..9565c6f9e 100644 --- a/doc/technical/overview.txt +++ b/doc/technical/overview.txt @@ -1,7 +1,7 @@ Lumiera: The Inner Core ======================= :Author: Lumiera_Core_Developers -:Date: 2012 / 2025 +:Date: 2012 / 2026 :toc: [abstract] @@ -30,7 +30,7 @@ This three Layers are: The Steam Layer:: Keeps the Session, generates the rendering graphs for sequences, - arranges what to do when and how. + prearranges elements and decides what to do when and how. The Vault Layer:: Manages worker pools, schedules jobs, does the memory management for the @@ -39,13 +39,11 @@ This three Layers are: The extra components are: - Lumiera:: - The main program itself, which starts all Subsystems and controls the - overall lifecycle. - - Common:: - Vital components and common services which must be available for pulling up - any part of the system. + The Vessel:: + This is the »backbone« of the application, it is rooted below the three + layers and encompasses everything. It defines the periphery, starts all + Subsystems, controls the overall lifecycle and provides vital services + which must be available for engaging any part of the system. Library:: A lot of (largely) stateless helper functionality used throughout the code. @@ -55,9 +53,9 @@ Coding Style ~~~~~~~~~~~~ The Lumiera team agreed on using GNU coding style with slight adaptations. -Generally speaking, we strive to keep the code base consistent and stick to -widely accepted guidelines and best practices. See our separate -link:{ldoc}/technical/code/codingGuidelines.html[Coding Guidelines] page. +In general, we strive to maintain source code consistency and adhere to widely +accepted guidelines and practices of good craftsmanship. See our separate +link:{ldoc}/technical/code/CodingGuidelines.html[Coding Guidelines] page. @@ -148,7 +146,7 @@ contained within the Session, which belongs to the _steam layer_ below. User interfaces are implemented as plug-ins and are pulled up on demand, they won't contain any relevant persistent state beyond presentation. -_As of 2023, the one and only interface under active development is +_As of 2026, the one and only interface under active development is the Lumiera GTK GUI,_ based on GTK-3 / gtkmm. The sources are in tree (directory 'src/stage') and it is integrated into the standard build and installation process. By default, running the 'lumiera' executable will @@ -250,7 +248,7 @@ for each output channel is required. The player service ^^^^^^^^^^^^^^^^^^ -Client code accesses the player (subsystem) through the play-facade (`lumiera::Play`). +Client code accesses the player (subsystem) through the play-facade (`Play`). The exposed service allows to _set up an output connection for playback or rendering,_ resulting in a play-controller object. @@ -285,11 +283,6 @@ performance critical. - the OutputSlot exposed by the output manager is responsible for accepting timed frame delivery - - -Vault Layer ------------ - Engine Interface ~~~~~~~~~~~~~~~~ While on itself just a thin interface and adaptation layer forwarding calls to @@ -318,16 +311,21 @@ the job ticket acts as blueprint for the actual jobs to be enqueued with the _Scheduler._ + +Vault Layer +----------- Scheduler ~~~~~~~~~ The Scheduler serves as the central hub in the implementation of the RenderEngine and coordinates the processing resources of the application. Its purpose is to invoke and control the dependency and time based execution of the _Render Jobs._ -Regarding architecture, the Scheduler is located in the Vault-Layer and running the Scheduler -is equivalent to activating the »Vault Subsystem«. An EngineFaçade acts as entrance point, -providing high-level render services to other parts of the application: render jobs can be -activated under various timing and dependency constraints, which can be grouped +As far as architecture is concerned, the Scheduler is located in the Vault-Layer and +is a _passive service_ -- like most components in the vault it needs to be put into +use by some coordinating and integrating facility. An EngineFaçade, itself located +above, in the »Steam« layer, acts as entrance point, to provide the high-level +render services to other parts of the application: render jobs can be activated +under various timing and dependency constraints, which can be grouped into three basic modes of control: Deadline:: @@ -362,6 +360,29 @@ The scheduler is able to observe and maintain dependencies between render jobs, to break down an extended calculation into a chain of small steps. +The Work Force +~~~~~~~~~~~~~~ +Since the typical work job dimensions can be expected to lie in the range of _some milliseconds,_ +the Lumiera engine can adopt a statistical model of capacity distribution. This is notably +different to an _operating system scheduler_ where the response times are measured in +microseconds and the coordination of work approaches hardware limits. These observations +lead to the (maybe slightly surprising approach) to rely on an _actively pulling_ work force +combined with a _passive scheduling service._ + +The _Work Force_ is a **pool of worker threads** with a size typically matching the nominal +number of independent CPU cores in the system. Spinning up the Work Force is equivalent to +activating the »Vault Subsystem«. Deliberately, there is no dedicated ``engine'' or +``manager'' thread. All workers participate in the coordination work on equal footing. +An idle worker enters the (passive) scheduling code, and thereby performs management +work up to the point where the next render job is determined. The worker then leaves +the management realm and continues with performing this job. However, the scheduling +code preloads each worker with some random sleep delay in order to distribute the +``incidence'' of idle workers and to avoid contention. This way, the available +computing capacity is distributed stochastically into that time window where +demand is foreseeable, based on the next job's starting time, the currently +dispatched jobs and the observed load factor. + + Resource Management ~~~~~~~~~~~~~~~~~~~ @@ -378,12 +399,12 @@ the system and current circumstances. -Common Services ---------------- +Vessel: Common Services +----------------------- Subsystem runner ~~~~~~~~~~~~~~~~ -Conceptually, the application is structured into layers. However, as actual runtime +Conceptually, the application is structured into layers. However, as the actual runtime entities we rely on a small number of link:{ldoc}/design/architecture/Subsystems.html[Subsystems], which can be started and stopped and have well defined dependencies: @@ -418,17 +439,14 @@ link:{ldoc}/design/architecture/Subsystems.html#_lifecycle[documentation] [small]#(likewise incomplete, but we're getting there, eventually [yellow-background]##2018##)# -Plugin loader -~~~~~~~~~~~~~ -_tbw_ Advice framework ~~~~~~~~~~~~~~~~ This is a ``whiteboard'' system, allowing implementation-level services to _publish_ some piece of information as _advice_, while other parts of the system may pick up this advice just by a name token, without requiring a direct knowledge of the -original _advisor._ The _Advice System_ is a singleton service maintaining a -lookup and registration data structure. Individual _piece of advice_ elements +original _advisor._ The _Advice System_ is a singleton service that maintains +a lookup and registration data structure. Individual _piece of advice_ elements are stored _as value copy_. Publishing new advice requires locking, but accessing advice is lock-free (actually there needs to be a memory barrier ``somewhere'', otherwise the advice requesting client might not see new advice) @@ -458,13 +476,27 @@ Library ------- The Lumiera support library contains lots of helper functionality -factored out from the internals and re-used. It is extended as we go. +factored out from the internals and re-used. Some parts evolved into +basic frameworks, that are used pervasively throughout the code base. + + - link:{ldoc}/design/architecture/time/index.html[time and timecode] + (-> xref:_time[below]) + - link:Dependencies.html[dependency and service handling] + (-> xref:_singletons_and_dependencies[below]), + - the link:DiffFramework.html[»Diff framework«] + - property trees for link:{ldoc}/design/architecture/ETD.html[»External Tree Description«] + (-> xref:_external_tree_description[below]) + - the link:iterator.html[iterator pipeline builders] + (-> xref:_iterators[below]) + - various metaprogramming helpers (function signatures and generic sequences) + - string conversion, parsing and rendering functions + practical shortcuts ~~~~~~~~~~~~~~~~~~~ -The header 'lib/util.hpp' defines some shortcuts heavily used throughout -the code base. The idea is to highlight a common semantic meaning, while -hide differentiation on the technical details. +The header 'lib/util.hpp' defines some shortcuts that are used almost +everywhere in the code base. The general idea is to highlight a common semantic meaning, +while hide the differentiation related to the technical details. isnil:: indicate a _missing value_, irrespective if this is a NULL pointer, an empty string or an empty container. Several of our custom wrappers also @@ -473,8 +505,29 @@ isnil:: indicate a _missing value_, irrespective if this is a NULL pointer, contains:: indicate that a value is _somehow contained_ within a collection, irrespective if this is a set, a map, a vector or string (substring test) -sanitise:: make any string usable as identifier. +limited:: to bracket some value. +unConst:: a functional notation to cast away `const`, which is used typically + at the barrier between a type safe front-end and a low-level back-end, + and to avoid needless duplication of accessor code implementation + for sake of providing `const` information functions. + +isSameObject:: identity test based on bare memory location, + which is used a lot in assertions and unit tests + +string shorthands:: + trim, startsWith, endsWith, boolVal, sanitise (turn arbitrary string into an identifier) + + +Literals and Symbols +~~~~~~~~~~~~~~~~~~~~ +Besides the well-known C++ strings (and `string_view`), Lumiera introduces +two marker types for string tokens + +Literal:: a C-string literal directly given in the Source code + +Symbol:: an _interned_ token stored in a Symbol table, so that its + address hash can be used as unique identity Locking @@ -892,6 +945,17 @@ and assignment incur the cost of an additional virtual call, assuming the target objects cooperate by mixing in a copy management interface. +.uninitialised storage +A cornerstone of the C++ language is the deterministic and reliable invocation +of constructors and destructors. However, when building low-level storage schemes, +a frequent requirement is to cover the precise storage of an object collection, +_without_ actually invoking any constructor or destructor. The `UninitialisedStorage` +template is a building block to support this task. It is parametrised with a type +and an element count, and provides a force-casted array-like access to the storage cells. +This can be used either to placement-construct some elements on-demand, or to ``project'' +an object-like view over existing data in the storage. + + Unique Identifiers ~~~~~~~~~~~~~~~~~~ diff --git a/doc/technical/stage/index.txt b/doc/technical/stage/index.txt index ad34a0085..a7252f702 100644 --- a/doc/technical/stage/index.txt +++ b/doc/technical/stage/index.txt @@ -1,5 +1,5 @@ Technical Documentation: Stage Layer ==================================== -Eventually, this will have technical documentation for the GUI. +This section offers a loose collection with detailed information regarding the GUI implementation. diff --git a/doc/technical/steam/index.txt b/doc/technical/steam/index.txt index 56c69524b..c48e8741f 100644 --- a/doc/technical/steam/index.txt +++ b/doc/technical/steam/index.txt @@ -1,27 +1,36 @@ Technical Documentation: Steam-Layer ==================================== -Eventually, this section will hold technical documentation for the Steam-Layer. +As implementation progresses, this section will gradually compile +the technical documentation for the Steam layer. Development Scrapbook TiddlyWiki -------------------------------- -Hermann Voßeler, core developer of the Steam-Layer uses an separate embedded wiki -for his day-to-day design sketches, notes and also for the more persistent planning. -This *TiddlyWiki* is a nifty JavaScript wiki embedded in a single HTML page; as such -it lives right within the source tree, where it can be worked on alongside with the -code. Besides the actual planning and descriptions, this wiki also links to the -UML diagrams created with *BOUML* during the ongoing planning and design. +It is sometimes difficult to write complete documentation while a system is still under construction. +For taking work notes and formulating new ideas while working, _Ichthyostega_ frequently uses the +https://classic.tiddlywiki.com/[TiddlyWiki] -- which is a nifty JavaScript wiki embedded within a +single HTML page; as such it lives right within the source tree, where it can be worked on alongside +with the code. Notes and sketches for the planning often evolve into detailed descriptions, complete +with test data and UML diagrams to document the structure. Material drafted in the TiddlyWiki +will be moved gradually into this documentation section, as time permits. -This setup started as a temporary solution -- but as uWiki is not expected to come -to life soon, the current website can't serve as replacement for a real wiki, an -we'll stick to that situation for the time being. As the content in the TiddlyWiki -becomes more mature and stable, it will be moved over into this section here. +This arrangement started as a temporary solution at a time where we did not have a suitable website +for permanent documentation -- and it was kept ever since, as the TiddlyWiki is very convenient +to work with, once you are accustomed to its quirks.footnote:[ + It is quite fluid and inspiring to work with indeed. The content is an unordered collection + of small notepad pages, called ``Tiddlers'', which can be opened and closed conveniently. + Any "CamelCaseWord" becomes a new Tiddler, recent changes appear in a timeline, and Tiddlers + can be organised with tags. The only point to be careful is that you need to _save_ + your work, which can be achieved with the `save' action in the menu to overwrite + the old version in the Git tree. Sometimes this might produce Git conflicts, + which are easy to resolve, since the content is plain text. +] +Additions to the TiddlyWiki are usually made during coding, and are deliberately committed always +together with the corresponding code. As such, these additions will propagate to Lumiera/Master eventually; +yet to make the newest development notes easier to access, we have put up a `rsync`ed version of the +TiddlyWiki online for direct access over the internet (which then is of course read-only). -While additions to the TiddlyWiki generally propagate to Lumiera/Master through -the normal merges, we've put up a rsynced version of the TiddlyWiki online for -direct access (of course read-only). - --> access the Proy-Layer link:/wiki/renderengine.html[TiddlyWiki online here] +-> access the link:/wiki/renderengine.html[TiddlyWiki online here] diff --git a/doc/user/intro/intro.txt b/doc/user/intro/intro.txt index 3f34822d2..2e49a6c27 100755 --- a/doc/user/intro/intro.txt +++ b/doc/user/intro/intro.txt @@ -113,8 +113,11 @@ each have a distinct _type_ which explicitly identifies the possible connections modules. Within these constraints, any combination is possible without hidden restrictions. -Lumiera is not a set of Lego bricks, nor is it a business application -driven by finite usage stories. +However, the application strives for a certain degree of internal coherence. + +Lumiera is not a set of Lego bricks allowing you to build ``just anything'' +(in a clumsy way) -- nor is it the next universal programming platform or +application framework driven by pre-canned success stories, where you +``just push the button'' and magic does the rest. Medium Level Abstraction and Project Specific Conventions @@ -131,9 +134,8 @@ details such as header format conversion or access to individual channels. To complement this approach, Lumiera does _not_ rely on hard-wired, global conventions -- rather we allow the building up of project specific conventions and rules xref:rules[<-] to fit a given requirement and preferred working style. - -Lumiera will be supplied with a conventional template and a default -configuration to ease the learning curve for new users. +However, in its default configuration, Lumiera will be supplied with a conventional +UI template and key bindings to ease the learning curve for new users. [[graphs]] @@ -218,16 +220,15 @@ Lumiera consists of three major parts: In this section, we discuss the user's perspective when working with the GUI. -Although Lumiera will initially ship with a standard, default GUI, we do not presume that this GUI will be suitable for all uses. -We expect there to be multiple, different GUIs, each designed for different -kinds of tasks. We expect that some of these will be written by users, and the -steam-layer is designed to facilitate this easily. +Although Lumiera ships with a conventional desktop UI by default, we do not presume +that this GUI will be suitable for all uses. We expect there to be several specialised +UIs eventually, designed to handle very specific kinds of tasks. Indeed, Lumiera can even work satisfactorily without a GUI, for example, for special purposes (e.g. as headless render-node xref:rendernode[<-] or frameserver -xref:frameserver[<-]). Scripts will eventually be written -to facilitate automated processing. +xref:frameserver[<-]). A script interface will be added at some +point to facilitate automated processing. Now it is time to take a look at the preliminary Lumiera GUI: @@ -505,21 +506,19 @@ A Plug-in is a kind of generalisation of a library. All applications use, to varying degrees of intensity, libraries. A programmer will not reinvent the wheel each time he sits down to programme an application. A programmer will typically borrow and use features and -functionality from other programmers---or even borrow from himself, stuff +functionality from other programmers -- or even borrow from himself, stuff written long ago in the past. Such features are collected together in libraries. -A library is used in an application by _linking_ the library into the -application. (There are other things to be done, but we'll call these 'details', -which wont concern us here.) There are different ways to _link_ a library -into an application: linking statically and linking dynamically. +A library is used in an application by _linking_ the library into the application. +However, there are different ways to _link_ a library: linking statically and +linking dynamically. _Static Linking_ is done while the application is being built, or compiled. It is performed by the linker, which executes as the last step -of the compilation process. The linker can perform some sanity checks -(mostly checks regarding formally correct function invocation) and warn the user -hat some particular feature is being used incorrectly. The user can then correct -the offending code, and recompile. + +of the compilation process. The linker can perform some sanity checks and +some optimisation. In the end, the functionality of the library is ``baked into'' +the resulting executable. + There are a number of disadvantages associated with static linking. Features and libraries are being constantly improved. If the application wants to use new features, it will have to be recompiled with the new library which provides the @@ -532,15 +531,14 @@ features can be used by the application. The features provided by a dynamic library are loaded when the application starts to run. However both methods exhibit a number of shortcomings. Wouldn't it be better if -all features could be loaded only when needed? If features could be loaded only -when needed, then they could also be unloaded when not required, thus saving -memory and possibly increasing performance. This scheme of making features -available to an application is known as run-time linking, aka plug-ins. +all features could be loaded only when needed? This scheme of making features +available to an application on-demand is known as run-time linking. A component +intended to be added this way, at run time, by the user, is known as a Plug-in. Plug-ins offer other benefits: the application can continue to use both the old features and the new features together, side-by-side, by using the version -number associated with the plug-in. This saves the application from the havoc -of library incompatibility that is -associated with other linking methods. +number associated with the plug-in. Furthermore, Plug-ins can load other Plug-ins +and thus extend the application in creative ways, beyond anything the original +developers of the application had planned. Most modern applications use plug-ins, some are heavily dependent on plug-ins and only provide limited functionality without any plug-ins. diff --git a/doc/user/tutorials/building.txt b/doc/user/tutorials/building.txt index 3638e73b1..3ce26088b 100644 --- a/doc/user/tutorials/building.txt +++ b/doc/user/tutorials/building.txt @@ -54,7 +54,7 @@ in addition to the following tools and libraries: * https://www.boost.org/[Boost libraries] * https://github.com/bplaum/gavl[GAVL library] * https://nobug.pipapo.org/[NoBug] ~([purple]#⚠ see below#)~ - * Lumiera https://git.lumiera.org/?p=LUMIERA;a=summary[source code] + * Lumiera https://git.lumiera.org/gitweb?p=LUMIERA;a=summary[source code] The GUI depends on the following: diff --git a/doc/user/tutorials/contributing.txt b/doc/user/tutorials/contributing.txt index aa53825db..6fe451999 100644 --- a/doc/user/tutorials/contributing.txt +++ b/doc/user/tutorials/contributing.txt @@ -115,9 +115,9 @@ sudo make install but was orphaned in 2021. Since it is currently a prerequisite for building Lumiera, we maintain a clone of the original repository and a Debian package on Lumiera.org. This is relevant both for - https://git.lumiera.org/?p=debian/gdl;a=summary[libGDL (3.40) +git://git.lumiera.org/debian/gdl+] + https://git.lumiera.org/gitweb?p=debian/gdl;a=summary[libGDL (3.40) +git://git.lumiera.org/debian/gdl+] and the - https://git.lumiera.org/?p=debian/gdlmm;a=summary[GDL C++ bindings (3.7.3) +git://git.lumiera.org/debian/gdlmm+]. + https://git.lumiera.org/gitweb?p=debian/gdlmm;a=summary[GDL C++ bindings (3.7.3) +git://git.lumiera.org/debian/gdlmm+]. ] * first you have to build libGTL itself... diff --git a/research/try.cpp b/research/try.cpp index b6e9d0a82..972451b90 100644 --- a/research/try.cpp +++ b/research/try.cpp @@ -1,6 +1,7 @@ /* try.cpp - to try out and experiment with new features.... * scons will create the binary bin/try */ +// 03/26 - runtime index access and iteration over tuple-likes // 06/25 - provide a concept to accept _tuple-like_ objects // 06/25 - investigate function type detection of std::bind Binders // 12/24 - investigate problem when perfect-forwarding into a binder @@ -10,22 +11,28 @@ /** @file try.cpp - * Develop a concept to detect _tuple-like_ classes, based on the requirements - * of the »tuple protocol«. Using some ideas from [Stackoverflow] as starting point. - * However, we model both a _friend function_ `get` and a similar member function - * as alternatives for element access, and we break down the checks into sub-concepts. - * [Stackoverflow]: https://stackoverflow.com/q/68443804/444796 + * Work out a construction scheme to build an efficient element-accessor to data + * in a tuple-like with run-time indexing. As an extension, run-time pulled iteration + * can be established, using the IterExplorer framework. + * @see lib::meta::FunTrampoline + * @see lib::meta::TupleIdxAdaptor */ #include "lib/format-cout.hpp" -#include "lib/test/test-helper.hpp" -#include "lib/test/diagnostic-output.hpp" +#include "lib/format-util.hpp" +#include "test/test-helper.hpp" +#include "test/diagnostic-output.hpp" +#include "lib/meta/common-result.hpp" +#include "lib/meta/tuple-concept.hpp" #include "lib/meta/tuple-helper.hpp" +#include "lib/meta/function.hpp" +#include "lib/iter-adapter.hpp" #include "lib/hetero-data.hpp" +#include "lib/meta/trait.hpp" #include "lib/util.hpp" -#include +#include using std::string; @@ -41,14 +48,263 @@ template void show() { - cout << "Tup!! "<< lib::test::showType() <() < ([](auto i) { using Elm = std::tuple_element_t; - cout <<" "<() <() <` + */ + template + concept typeseq = requires (TSEQ seq) + { + // TSEQ can invoke a function with a Types subclass + [](Types&){/*placeholder*/}(seq); + }; +#endif ///////////////////////////////////////////////////////////////////////////////////////////////////(End) : new generic library functions + + + template class BUILD, typeseq TSEQ> + class FunTrampoline + { + using AllElm = ElmTypes; + using Indices = AllElm::Idx; + using Builder = BUILD; + using Functor = Builder::Functor; + using Table = std::array; + + template + static constexpr Table + buildFunctors (std::index_sequence) + { + return {Builder().template build()...}; + } + + static constexpr Table functors = buildFunctors (Indices{}); + + public: + static constexpr size_t size() { return functors.size(); } + static constexpr size_t memSiz() { return sizeof(functors); } + + static Functor const& + dispatch (size_t idx) + { + ENSURE (idx < functors.size()); + return functors[idx]; + } + + using ResType = _Fun::Ret; + }; + + +#if false ///////////////////////////////////////////////////////////////////////////////////////////////////TODO : new generic library functions + template + using isConst = std::is_const>; + template + using isLRef = std::is_lvalue_reference; + + + template + using Any_Const = ElmTypes::template OrAll; + + template + using All_LRef = ElmTypes::template AndAll; + + template + static constexpr bool Any_Const_v = Any_Const::value; + + template + static constexpr bool All_LRef_v = All_LRef::value; + + + + + template + struct ProbeCommonType + { + + template + using Reconciled = decltype(true ? std::declval() : std::declval()); + + + template + struct FoundResult + : std::true_type + { + using Type = RES; + }; + struct NoResultFound + : std::false_type + { }; + + template> + static FoundResult> + check(int); + + template + static NoResultFound + check(...); + + using FirstResult = decltype(check(0)); + + template + struct Join + { + using Result = FirstResult; + }; + + template + struct Join + { + //wenn RES erfolgreich + using CT = FirstResult::Type; + using Result = ProbeCommonType::Result; + }; + + using Result = Join::Result; + }; + + + + template + struct CommonResult + : ProbeCommonType::Result + { }; + + template + struct CommonResult + : CommonResult + { }; + + template<> + struct CommonResult<> + : std::false_type + { }; + + template + struct CommonResult> + : CommonResult + { }; + + template + using CommonResult_t = CommonResult::Type; + + template + static constexpr bool has_CommonResult_v = CommonResult::value; + + + template + concept reconcilable = requires + { + typename CommonResult::Type; + }; + + template + concept reconcilable_ref = reconcilable + and isLRef_v>; + + template + struct CommonResultTraits + { + static_assert (reconcilable, "unable to reconcile these types"); + + using CommonVal = std::common_type_t; + using ResType = CommonResult_t; + //NOTE: std::common_type decays (strips CV and ref) + static constexpr bool makeConst = isConst_v; + static constexpr bool exposeRef = isLRef_v; + }; +#endif ///////////////////////////////////////////////////////////////////////////////////////////////////(End) : new generic library functions + + + + template + class TupleIndexAdaptor + : public TUP + { + template + struct SubscriptBuilder + { + using Tuple = RefTraits::value_type; + using AddRefs = ElmTypes::template Apply; + using ResType = CommonResult_t; + using Functor = ResType(*)(Tuple&); + + template + constexpr auto + build() + { + return [](Tuple& tuple) -> ResType + { + return getElm (tuple); + }; + } + }; + + using TypeSeq = ElmTypes::Seq; + using Accessor = FunTrampoline; + public: + using ResType = Accessor::ResType; + using reference = ResType; + using value_Type = RefTraits::value_type; + + template + TupleIndexAdaptor(T&& dataSrc) + : TUP{forward (dataSrc)} + { } + + using TUP::TUP; + + static constexpr size_t size() { return Accessor::size(); } + + ResType + operator[] (size_t idx) + { + return Accessor::dispatch(idx)(*this); + } + }; + + template + TupleIndexAdaptor(TUP) -> TupleIndexAdaptor>; +}} + +using namespace lib::meta; + +template +struct Stringify + { + using Tuple = ElmTypes::Tup; + using Functor = string(*)(Tuple const&); + + template + constexpr auto + build() + { + using ElmT = std::tuple_element_t; + + return [](Tuple const& tuple) + { + return util::toString (getElm (tuple)); + }; + } + + }; + + +struct Uber { }; +struct Untr : Uber { }; +struct Unta : Uber { }; + int main (int, char**) @@ -57,20 +313,10 @@ main (int, char**) using Arr = std::array; using Hetero = lib::HeteroData::Chain::ChainExtent::ChainType; - SHOW_EXPR((lib::meta::tuple_sized )) - SHOW_EXPR((lib::meta::tuple_sized )) - SHOW_EXPR((lib::meta::tuple_sized )) - SHOW_EXPR((lib::meta::tuple_sized )) - - SHOW_EXPR((lib::meta::tuple_element_accessible)) -// SHOW_EXPR((lib::meta::tuple_element_accessible)) - SHOW_EXPR((lib::meta::tuple_element_accessible)) - SHOW_EXPR((lib::meta::tuple_accessible)) - SHOW_EXPR((lib::meta::tuple_accessible)) - SHOW_EXPR((lib::meta::tuple_like )) SHOW_EXPR((lib::meta::tuple_like )) SHOW_EXPR((lib::meta::tuple_like )) + SHOW_EXPR((lib::meta::tuple_like )) SHOW_EXPR((lib::meta::tuple_like )) show(); @@ -78,25 +324,112 @@ main (int, char**) show(); show(); - SHOW_EXPR((std::tuple_size_v)) - using Elm1 = std::tuple_element_t<1, const Tup>; - SHOW_TYPE(Elm1) - using TupConstSeq = lib::meta::ElmTypes::Seq; - SHOW_TYPE(TupConstSeq) + using TupStringy = FunTrampoline::Seq>; +SHOW_EXPR(TupStringy::size()) - using T1 = decltype(lib::meta::getElm<0> (std::declval())); - SHOW_TYPE(T1) - using T2 = decltype(lib::meta::getElm<0> (std::declval())); - SHOW_TYPE(T2) - using T3 = decltype(lib::meta::getElm<0> (std::declval())); - SHOW_TYPE(T3) - - using H1 = decltype(lib::meta::getElm<4> (std::declval())); - SHOW_TYPE(H1) - using H2 = decltype(lib::meta::getElm<4> (std::declval())); - SHOW_TYPE(H2) - using H3 = decltype(lib::meta::getElm<4> (std::declval())); - SHOW_TYPE(H3) + Tup tup23{2,3}; + Tup tup42{4,2}; +SHOW_TYPE(decltype(std::get<1>(tup42))) +SHOW_EXPR(std::get<1>(tup42)) +SHOW_EXPR(TupStringy::dispatch(0)(tup23)) +SHOW_EXPR(TupStringy::dispatch(0)(tup42)) +SHOW_EXPR(TupStringy::dispatch(1)(tup23)) +SHOW_EXPR(TupStringy::dispatch(1)(tup42)) + + using T1 = Types; + using T2 = Types; + using T3 = Types; +SHOW_EXPR((has_CommonResult_v)) +SHOW_EXPR((has_CommonResult_v)) +SHOW_EXPR((has_CommonResult_v)) +SHOW_EXPR((has_CommonResult_v<>)) +SHOW_EXPR((has_CommonResult_v)) +SHOW_EXPR((has_CommonResult_v)) + +SHOW_TYPE(CommonResult_t) +SHOW_TYPE(CommonResult_t) +SHOW_TYPE(CommonResult_t) +using CoTx = CommonResult_t; +SHOW_TYPE(CoTx) + + + TupleIndexAdaptor idxAdapt{tup42}; +SHOW_EXPR(sizeof(tup42)) +SHOW_EXPR(sizeof(idxAdapt)) +SHOW_TYPE(TupleIndexAdaptor::ResType) +SHOW_EXPR(idxAdapt.size()) +SHOW_EXPR(idxAdapt[0]) +SHOW_EXPR(idxAdapt[1]) +SHOW_EXPR(getElm<0>(static_cast (idxAdapt))) +SHOW_EXPR(getElm<1>(static_cast (idxAdapt))) +SHOW_TYPE(decltype(idxAdapt[1])) + + using Tup2 = std::tuple; + int ii{23}; + Tup2 tup21{-1,ii,55}; + TupleIndexAdaptor idxApt2{tup21}; +SHOW_TYPE(decltype(tup21)) + using IdxApt2 = decltype(idxApt2); +SHOW_TYPE(IdxApt2) +SHOW_TYPE(IdxApt2::ResType) +SHOW_EXPR(idxApt2[0]) +SHOW_EXPR(idxApt2[1]) +SHOW_EXPR(idxApt2[2]) +SHOW_EXPR(++idxApt2[0]) +SHOW_EXPR(idxApt2[1]++) +SHOW_EXPR(++++idxApt2[2]) +SHOW_EXPR(tup21) +SHOW_EXPR(Tup2(idxApt2)) +SHOW_EXPR(ii) + + using TupIt = lib::IterableDecorator>; + TupIt tupIt{move(idxApt2)}; +SHOW_EXPR( bool(tupIt) ) +SHOW_EXPR( *tupIt ) +SHOW_TYPE(decltype(*tupIt)) + ++*tupIt; + --ii; +SHOW_EXPR( *tupIt ) +SHOW_EXPR(util::join(tupIt, "▶▶")) + +SHOW_EXPR(( reconcilable )) +SHOW_EXPR(( reconcilable )) +SHOW_EXPR(( reconcilable )) +SHOW_EXPR(( reconcilable )) +SHOW_EXPR(( reconcilable_ref )) +SHOW_EXPR(( reconcilable_ref )) +SHOW_EXPR(( reconcilable_ref )) +SHOW_EXPR(( reconcilable_ref )) + using TX = CommonResult_t; +SHOW_TYPE(TX) +SHOW_EXPR(( reconcilable )) +SHOW_EXPR(( reconcilable )) +SHOW_EXPR(( reconcilable )) + using TX2 = CommonResult_t; +SHOW_TYPE(TX2) +SHOW_EXPR(( reconcilable )) +SHOW_EXPR(( reconcilable )) +SHOW_EXPR(( reconcilable )) +SHOW_EXPR(( reconcilable )) +SHOW_EXPR(( reconcilable_ref )) +SHOW_EXPR(( reconcilable_ref )) +SHOW_EXPR(( reconcilable )) +SHOW_EXPR(( reconcilable )) + using TX3 = CommonResult_t; +SHOW_TYPE(TX3) + using TX4 = CommonResult_t; +SHOW_TYPE(TX4) +SHOW_EXPR(( reconcilable )) + using TX5 = CommonResult_t; +SHOW_TYPE(TX5) + using TX6 = CommonResult_t; +SHOW_TYPE(TX6) +SHOW_EXPR(( CommonResult::value )) +SHOW_EXPR(( CommonResult::value )) +SHOW_EXPR(( CommonResult::value )) +SHOW_EXPR(( reconcilable )) +SHOW_EXPR(( reconcilable )) +SHOW_EXPR(( reconcilable )) cout << "\n.gulp." < @@ -11,32 +11,30 @@ */ -/** @file lifecycle.h +/** @file lifecycle.hpp ** Installing and invoking of application lifecycle event callbacks. - ** This service is a facade for and implemented by the lumiera::LifecycleRegistry. + ** This service is a facade for and implemented by the vessel::LifecycleRegistry. ** By placing a static LifecycleHook variable or by calling LifecycleHook::add, ** a callback can be registered to be executed on a specific application lifecycle ** event. Examples being #ON_BASIC_INIT, #ON_GLOBAL_INIT. Other subsystems may ** register additional events for more specific purpose. ** ** @note the implementation resides in lib/lifecycle.cpp and gets linked into liblumiera.so - ** @see lumiera::AppState + ** @see vessel::Voyage ** @see main.cpp */ -#ifndef LUMIERA_LIFECYCLE_H -#define LUMIERA_LIFECYCLE_H +#ifndef VESSEL_LIFECYCLE_H +#define VESSEL_LIFECYCLE_H -#ifdef __cplusplus - #include "lib/symbol.hpp" #include "lib/nocopy.hpp" -namespace lumiera { +namespace vessel { using lib::Symbol; @@ -81,21 +79,5 @@ namespace lumiera { -} // namespace lumiera - - - -#else /* =========== C interface ====================== */ - - -//defined in liblumiera.so -extern const char * lumiera_ON_BASIC_INIT; -extern const char * lumiera_ON_GLOBAL_INIT; -extern const char * lumiera_ON_GLOBAL_SHUTDOWN; - - -void lumiera_LifecycleHook_add (const char* eventLabel, void callbackFun(void)); -void lumiera_Lifecycle_trigger (const char* eventLabel); - -#endif -#endif +} // namespace vessel +#endif /*VESSEL_LIFECYCLE_H*/ diff --git a/src/include/session-command-facade.h b/src/include/session-command-facade.h index a23dbf593..6851ca146 100644 --- a/src/include/session-command-facade.h +++ b/src/include/session-command-facade.h @@ -116,7 +116,7 @@ extern "C" { #endif /* =========================== CL Interface ===================== */ -#include "common/interface.h" +#include "vessel/interface.h" LUMIERA_INTERFACE_DECLARE (lumieraorg_SessionCommand, 0, LUMIERA_INTERFACE_SLOT (const char*, cycle, (const char*, const char*)), diff --git a/src/lib/allocator-handle.hpp b/src/lib/allocator-handle.hpp index d7c0477f3..4676bef02 100644 --- a/src/lib/allocator-handle.hpp +++ b/src/lib/allocator-handle.hpp @@ -51,6 +51,7 @@ #include #include +#include #include diff --git a/src/lib/builder-qualifier-support.hpp b/src/lib/builder-qualifier-support.hpp index 5513f7cc8..3f9e32cdc 100644 --- a/src/lib/builder-qualifier-support.hpp +++ b/src/lib/builder-qualifier-support.hpp @@ -31,7 +31,7 @@ ** @warning since these qualifiers manipulate by side-effect, it is up to the user to ensure ** consistent settings and behaviour (typically within a terminal `build()` function). ** @note the `qualify()` function accepts the qualifier functors by-reference; thus it is up - ** to the user to either pass them by-value, by reverence or even by perfect forwarding. + ** to the user to either pass them by-value, by reference or even by perfect forwarding. ** ** @see BuilderQualifierSupport_test ** @see ElementBoxWidget as usage example diff --git a/src/lib/depend-inject.hpp b/src/lib/depend-inject.hpp index 6a3edebd6..c68009d56 100644 --- a/src/lib/depend-inject.hpp +++ b/src/lib/depend-inject.hpp @@ -376,7 +376,7 @@ namespace lib { using ResultVal = meta::_Fun::Ret; using Subclass = meta::Strip::TypePlain; - static_assert (std::is_pointer::value, + static_assert (std::is_pointer_v, "Function must yield a pointer to a heap allocated instance"); }; diff --git a/src/lib/diff/gen-node.cpp b/src/lib/diff/gen-node.cpp index 69cc70ca7..ed0b18c7a 100644 --- a/src/lib/diff/gen-node.cpp +++ b/src/lib/diff/gen-node.cpp @@ -44,7 +44,7 @@ using util::join; using boost::lexical_cast; using lib::time::TimeValue; -using lib::transformIterator; +using lib::transformIter; using util::almostEqual; using lib::hash::LuidH; @@ -297,9 +297,9 @@ namespace diff{ return (Rec::TYPE_NIL==rec.getType()? "" : rec.getType()) + "{" - + join (transformIterator (rec.attribs(), renderAttrib)) + + join (transformIter (rec.attribs(), renderAttrib)) + (isnil(rec.scope())? "" : "|") - + join (transformIterator (rec.scope() , renderChild)) + + join (transformIter (rec.scope() , renderChild)) + "}" ; } diff --git a/src/lib/diff/gen-node.hpp b/src/lib/diff/gen-node.hpp index 5a794743a..9f21f9d15 100644 --- a/src/lib/diff/gen-node.hpp +++ b/src/lib/diff/gen-node.hpp @@ -375,8 +375,6 @@ namespace diff{ iterator end() const; - using ChildDataIter = TransformIter; - /** visit the _data_ of nested child elements * @return an iterator over the DataCap elements of all children, * in case this GenNode actually holds a Record. @@ -384,26 +382,26 @@ namespace diff{ * @note this iterator visits _only_ the children, which are * by definition unnamed. It does _not_ visit attributes. */ - friend ChildDataIter + friend auto childData (GenNode const& n) { - return ChildDataIter{ n.data.childIter() - , [](GenNode const& child) ->DataCap const& + return transformIter(n.data.childIter() + ,[](GenNode const& child) ->DataCap const& { return child.data; } - }; + ); } - friend ChildDataIter + friend auto childData (Rec::scopeIter&& scopeIter) { - return ChildDataIter{ std::forward(scopeIter) - , [](GenNode const& child) ->DataCap const& + return transformIter(std::forward(scopeIter) + ,[](GenNode const& child) ->DataCap const& { return child.data; } - }; + ); } @@ -776,7 +774,7 @@ namespace diff{ inline std::optional DataCap::retrieveAttribute (string key) const { - static_assert (not std::is_reference_v + static_assert (not meta::isRef_v ,"optional access only possible by value"); Rec* nested = unConst(this)->maybeAccessNestedRec(); diff --git a/src/lib/diff/mutation-message.hpp b/src/lib/diff/mutation-message.hpp index 735c6ca07..b4ae1a1bc 100644 --- a/src/lib/diff/mutation-message.hpp +++ b/src/lib/diff/mutation-message.hpp @@ -100,7 +100,7 @@ namespace diff{ using std::__and_; using std::__not_; using lib::meta::enable_if; - using lib::meta::can_IterForEach; + using lib::meta::can_LumieraIter; using lib::meta::can_STL_ForEach; using DiffStep = TreeDiffLanguage::DiffStep; @@ -168,7 +168,7 @@ namespace diff{ * @note source iterator is copied into a heap allocated IterSource */ template - MutationMessage(IT&& ii, enable_if< can_IterForEach, void*> =nullptr) + MutationMessage(IT&& ii, enable_if< can_LumieraIter, void*> =nullptr) : _FrontEnd{iter_source::wrapIter (std::forward(ii))} { } @@ -179,7 +179,7 @@ namespace diff{ */ template MutationMessage(CON& container, enable_if< __and_< can_STL_ForEach - ,__not_>>, void*> =nullptr) + ,__not_>>, void*> =nullptr) : _FrontEnd{iter_source::eachEntry(container)} { } diff --git a/src/lib/diff/record.hpp b/src/lib/diff/record.hpp index 89c9f7637..e5da73aba 100644 --- a/src/lib/diff/record.hpp +++ b/src/lib/diff/record.hpp @@ -308,18 +308,16 @@ namespace diff{ using iterator = IterAdapter; using scopeIter = iter_stl::_SeqT::Range; - using keyIter = TransformIter; - using valIter = TransformIter; /** default iteration exposes all data within this "object", starting with the attributes */ iterator begin () const { return iterator(this, attribs_.empty()? children_.begin() : attribs_.begin()); } iterator end () const { return iterator(); } - scopeIter attribs() const { return iter_stl::eachElm(attribs_); } - scopeIter scope() const { return iter_stl::eachElm(children_); } + scopeIter attribs() const { return iter_stl::eachElm (attribs_); } + scopeIter scope() const { return iter_stl::eachElm (children_); } - keyIter keys() const { return transformIterator(attribs(), extractKey); } - valIter vals() const { return transformIterator(attribs(), extractVal); } + auto keys() const { return transformIter (attribs(), extractKey); } + auto vals() const { return transformIter (attribs(), extractVal); } protected: /* ==== API for the IterAdapter ==== */ @@ -792,11 +790,11 @@ namespace diff{ Record::operator std::string() const { using util::join; - using lib::transformIterator; + using lib::transformIter; return "Rec(" + (TYPE_NIL==type_? "" : type_) - + (isnil(this->attribs())? "" : "| "+join (transformIterator (this->attribs(), renderAttribute))+" ") + + (isnil(this->attribs())? "" : "| "+join (transformIter (this->attribs(), renderAttribute))+" ") + (isnil(this->scope())? "" : "|{"+join (this->scope())+"}") + ")" ; diff --git a/src/lib/diff/test-mutation-target.hpp b/src/lib/diff/test-mutation-target.hpp index 713fe23b6..7a308ed73 100644 --- a/src/lib/diff/test-mutation-target.hpp +++ b/src/lib/diff/test-mutation-target.hpp @@ -43,7 +43,7 @@ #include "lib/idi/genfunc.hpp" #include "lib/format-string.hpp" #include "lib/format-util.hpp" -#include "lib/test/event-log.hpp" +#include "test/event-log.hpp" #include "lib/nocopy.hpp" #include "lib/util.hpp" @@ -58,8 +58,8 @@ namespace diff{ namespace error = lumiera::error; using lib::Literal; - using lib::test::EventLog; - using lib::test::EventMatch; + using test::EventLog; + using test::EventMatch; using iter_stl::eachElm; using util::unConst; using util::isnil; @@ -110,12 +110,12 @@ namespace diff{ renderRecord (Rec const& record) { using util::join; - using lib::transformIterator; + using lib::transformIter; return "Rec(" + (Rec::TYPE_NIL==record.getType()? "" : record.getType()) - + (isnil(record.attribs())? "" : "| "+join (transformIterator (record.attribs(), renderAttribute))+" ") - + (isnil(record.scope())? "" : "|{"+join (transformIterator (record.scope(), renderChild))+"}") + + (isnil(record.attribs())? "" : "| "+join (transformIter (record.attribs(), renderAttribute))+" ") + + (isnil(record.scope())? "" : "|{"+join (transformIter (record.scope(), renderChild))+"}") + ")" ; } @@ -289,14 +289,14 @@ namespace diff{ string showContent () const { - return join (transformIterator (begin(), renderNode)); + return join (transformIter (begin(), renderNode)); } /** render elements waiting in source buffer to be accepted */ string showSrcBuffer () const { - return join (transformIterator (eachElm(prev_content_), renderNode)); + return join (transformIter (eachElm(prev_content_), renderNode)); } EventMatch diff --git a/src/lib/diff/tree-mutator-collection-binding.hpp b/src/lib/diff/tree-mutator-collection-binding.hpp index ef237a583..fc36284d4 100644 --- a/src/lib/diff/tree-mutator-collection-binding.hpp +++ b/src/lib/diff/tree-mutator-collection-binding.hpp @@ -76,6 +76,7 @@ #include "lib/error.hpp" #include "lib/nocopy.hpp" #include "lib/meta/trait.hpp" +#include "lib/meta/function.hpp" #include "lib/diff/gen-node.hpp" #include "lib/diff/tree-mutator.hpp" #include "lib/iter-adapter-stl.hpp" diff --git a/src/lib/error-exception.cpp b/src/lib/error-exception.cpp index 74669ddfe..36191a7e5 100644 --- a/src/lib/error-exception.cpp +++ b/src/lib/error-exception.cpp @@ -24,7 +24,7 @@ #include "lib/error.hpp" -#include "include/lifecycle.h" +#include "include/lifecycle.hpp" #include "lib/meta/util.hpp" #include "lib/util.hpp" @@ -154,6 +154,9 @@ namespace lumiera { namespace error { namespace { + using vessel::LifecycleHook; + using vessel::ON_BASIC_INIT; + void install_unexpectedException_handler () { std::set_terminate (lumiera_unexpectedException); diff --git a/src/lib/format-obj.cpp b/src/lib/format-obj.cpp index 57dc1191c..676a4713f 100644 --- a/src/lib/format-obj.cpp +++ b/src/lib/format-obj.cpp @@ -197,14 +197,15 @@ apologies for that." "|(\\w+::)+\\(anonymous namespace\\)::" "|lib::meta::" "|lib::time::" - "|lib::test::" "|lib::diff::" "|lib::" "|util::" + "|\\btest::" "|steam::(asset::|mobject::(session::)?|play::)?" "|stage::model" "|stage::ctrl" "|lumiera::" + "|vessel::" , regex::ECMAScript | regex::optimize}; static regex lolong {"long long" diff --git a/src/lib/format-obj.hpp b/src/lib/format-obj.hpp index 85f65a2b1..842a26ff6 100644 --- a/src/lib/format-obj.hpp +++ b/src/lib/format-obj.hpp @@ -100,6 +100,119 @@ namespace util { + + namespace { // helper for printing type diagnostics + + template + struct TypeDiagnostics + { + using Type = X; + static constexpr auto prefix = ""; + static constexpr auto postfix = ""; + }; + template + struct TypeDiagnostics + { + using Type = X; + static constexpr auto prefix = "const "; + static constexpr auto postfix = ""; + }; + template + struct TypeDiagnostics + { + using Type = X; + static constexpr auto prefix = ""; + static constexpr auto postfix = "&"; + }; + template + struct TypeDiagnostics + { + using Type = X; + static constexpr auto prefix = ""; + static constexpr auto postfix = " &&"; + }; + template + struct TypeDiagnostics + { + using Type = X; + static constexpr auto prefix = ""; + static constexpr auto postfix = " const&"; + }; + template + struct TypeDiagnostics + { + using Type = X; + static constexpr auto prefix = "const "; + static constexpr auto postfix = " &&"; + }; + template + struct TypeDiagnostics + { + using Type = X; + static constexpr auto prefix = ""; + static constexpr auto postfix = " *"; + }; + template + struct TypeDiagnostics + { + using Type = X; + static constexpr auto prefix = "const "; + static constexpr auto postfix = " *"; + }; + template + struct TypeDiagnostics + { + using Type = X; + static constexpr auto prefix = "const "; + static constexpr auto postfix = " * const"; + }; + template + struct TypeDiagnostics + { + using Type = X; + static constexpr auto prefix = ""; + static constexpr auto postfix = " * const"; + }; + template + struct TypeDiagnostics + { + using Type = X; + static constexpr auto prefix = ""; + static constexpr auto postfix = " * const *"; + }; + }//(End)decoration helper + + /** diagnostic type output, including const and similar adornments + * @warning operates after-the-fact and relies on mangled type names + * plus several heuristics. Output might thus not be entirely correct, + * especially when several levels of const, pointer and references are involved. + * If in doubt, place the test::TypeDebugger to reveal the type as the compiler sees it. + * @remarks the function lib::meta::typeStr was deliberately written such as to remove type adornments + * and does not work on all kinds of reference. This helper complements this and + * attempts to preserve const-ness, pointer and reference decoration. + */ + template + inline std::string + showType() + { + using Case = TypeDiagnostics; + using Type = Case::Type; + + return Case::prefix + + lib::meta::humanReadableTypeID (typeid(Type).name()) + + Case::postfix; + } + + template + inline std::string + showTypes() + { + return "<| " + ((showType()+", ") + ... + "|>"); + } + + + + namespace { /** toggle to prefer specialisation with direct lexical conversion */ template diff --git a/src/lib/format-util.hpp b/src/lib/format-util.hpp index 01e7c8616..80d044666 100644 --- a/src/lib/format-util.hpp +++ b/src/lib/format-util.hpp @@ -46,7 +46,7 @@ namespace util { - using lib::meta::can_IterForEach; + using lib::meta::lum_iter; using std::string; using std::forward; using std::move; @@ -135,13 +135,13 @@ namespace util { { using Val = lib::meta::ValueTypeBinding::value_type; - return lib::transformIterator(forward(src), util::toString); + return lib::transformIter (forward(src), util::toString); } namespace { // helper to build range iterator on demand - template + template struct _RangeIter { using StlIter = CON::const_iterator; @@ -153,8 +153,8 @@ namespace util { { } }; - template - struct _RangeIter> > + template + struct _RangeIter { IT iter; diff --git a/src/lib/hash-standard.hpp b/src/lib/hash-standard.hpp index b603c8e6f..2703a14b8 100644 --- a/src/lib/hash-standard.hpp +++ b/src/lib/hash-standard.hpp @@ -44,9 +44,11 @@ ** Please ensure it is always included _before_ the latter. Failing to do so will result ** in mysterious failures. ** - ** @todo 4/2014 doesn't work as expected. My suspicion is that in the actual use case (PlacementIndex), - ** the type providing the hasher is mixed in through inheritance, and the template specialisation - ** for this base type is not considered on lookup. ///////TICKET #722 + ** @todo 4/2026 this shady trickery works without problems since many years now, + ** while it is not clear if we actually need to do these horrible things any more. + ** Even while we may retain that automatic bridge to boost-functional hash, doing so + ** should be possible with the current STDLIB without having to patch and rewire + ** the library implementation. //////////////////////////////////////////////////////////////////////TICKET #1422 : re-assess the situation ! ** ** @see HashIndexed ** @see LUID @@ -106,10 +108,6 @@ namespace std { { static_assert (sizeof(TY) < 0, "No hash implementation found. " "Either specialise std::hash or provide a boost-style hash_value via ADL."); - - // the default provides *no* hash implementation - // and adds a marker type for metaprogramming - typedef int NotHashable; }; /** @@ -129,8 +127,8 @@ namespace std { /** * Primary class template for std::hash. - * We provide no default implementation, but a marker type - * to allow detection of custom implementation through metaprogramming + * We provide no default implementation, but bridge automatically + * when a boost-style hash function can be picked up via ADL. * * @note this definition has been \em hijacked by Lumiera * to add an automatic bridge to use boost::hash functions @@ -140,12 +138,14 @@ namespace std { : public _HashImplementationSelector { }; -} +}// namespace std -/* ==== Evil Evil ==== */ + + +/* ==== Evil Evil ==== */ ////////////////////////////////////////////////////////////////TICKET #1422 : very likely we do not need this manipulation any more !!! #define hash hash_HIDDEN #define _Hash_impl _Hash_impl_HIDDEN @@ -204,6 +204,12 @@ namespace std { STD_HASH_IMPL (long double) #undef STD_HASH_IMPL -} + + /** rewire the partial specialisation for pointer types */ + template + struct hash : public hash_HIDDEN { }; + + +}// namespace std #endif /*LIB_HASH_STANDARD_H*/ diff --git a/src/lib/index-iter.hpp b/src/lib/index-iter.hpp index b8bb7915f..be709bff2 100644 --- a/src/lib/index-iter.hpp +++ b/src/lib/index-iter.hpp @@ -50,8 +50,8 @@ namespace lib { using ResVal = decltype(data_->operator[](0)); - using value_type = meta::RefTraits::Value; - using reference = meta::RefTraits::Reference; + using value_type = meta::RefTraits::value_type; + using reference = meta::RefTraits::reference; bool checkPoint() const diff --git a/src/lib/item-wrapper.hpp b/src/lib/item-wrapper.hpp index bd45621ed..aca3fbc78 100644 --- a/src/lib/item-wrapper.hpp +++ b/src/lib/item-wrapper.hpp @@ -31,7 +31,7 @@ ** this to the pattern of _type erasure_, where we discard specific ** knowledge after construction; for _such_ cases, opaque-holder.hpp ** might be a better fit. - ** @see lib::TransformIter + ** @see lib::iter::Transform ** @see lib::explore ** */ diff --git a/src/lib/iter-adapter-ptr-deref.hpp b/src/lib/iter-adapter-ptr-deref.hpp index 3458efe80..22d89658f 100644 --- a/src/lib/iter-adapter-ptr-deref.hpp +++ b/src/lib/iter-adapter-ptr-deref.hpp @@ -61,7 +61,7 @@ namespace lib { public: /** this iterator adapter is meant to wrap an iterator yielding pointer values */ using pointer = meta::ValueTypeBinding::value_type; - static_assert(std::is_pointer_v); + static_assert(is_pointer_v); using value_type = std::remove_pointer_t; using reference = value_type&; @@ -74,8 +74,8 @@ namespace lib { using ValueTypeBase = std::remove_const_t; // value_type without const - using WrappedIterType = IterType::template SimilarIter< ValueTypeBase* * >::Type; - using WrappedConstIterType = IterType::template SimilarIter::Type; + using WrappedIterType = IterRebind::template SimilarIter< ValueTypeBase* * >::Type; + using WrappedConstIterType = IterRebind::template SimilarIter::Type; using IterType = PtrDerefIter; using ConstIterType = PtrDerefIter; diff --git a/src/lib/iter-adapter-stl.hpp b/src/lib/iter-adapter-stl.hpp index 96f68cf7c..8fa6617ac 100644 --- a/src/lib/iter-adapter-stl.hpp +++ b/src/lib/iter-adapter-stl.hpp @@ -90,15 +90,18 @@ namespace iter_stl { /** - * helper baseclass to simplify - * defining customised wrapped STL iterators + * @internal helper baseclass to simplify defining customised wrapped STL iterators. + * @remark this is meant to be compliant to a STL input-iterator; + * it's _not_ a »Lumiera Forward Iterator« (lacks the bool check for iteration end) + * @warning minimal definition, since it is only intended to be wrapped into lib::RangeIter */ template struct WrappedStlIter : DEF { - using Iter = DEF::Iter; - using reference = DEF::reference; - using pointer = DEF::pointer; + using Iter = DEF::Iter; + using value_type = DEF::value_type; + using reference = DEF::reference; + using pointer = DEF::pointer; WrappedStlIter() : i_() { } @@ -112,6 +115,10 @@ namespace iter_stl { friend bool operator== (WrappedStlIter const& i1, WrappedStlIter const& i2) { return i1.i_ == i2.i_; } friend bool operator!= (WrappedStlIter const& i1, WrappedStlIter const& i2) { return i1.i_ != i2.i_; } + /** @note the following dummy definitions are required to fulfil the concept std::input_iterator */ + WrappedStlIter& operator++(int) { NOTREACHED("never used as STL iterator"); return *this; } + using difference_type = ptrdiff_t; + private: mutable Iter i_; }; @@ -424,11 +431,14 @@ namespace iter_stl { /** create empty snapshot */ IterSnapshot() { } + // standard copy operations acceptable + + /** take snapshot by discharging the given Lumiera Forward iterator * @warning depending on the implementation backing the source iterator, * this might or might not yield side-effects. */ - template + template> IterSnapshot (IT&& src) { for ( ; src; ++src) @@ -439,7 +449,7 @@ namespace iter_stl { * @warning depending on the implementation backing the source iterator, * this might or might not yield side-effects. */ - template + template> IterSnapshot (IT const& src) { for (IT copy{src}; copy; ++copy) @@ -462,11 +472,6 @@ namespace iter_stl { buffer_.emplace_back (*pos); } - IterSnapshot(IterSnapshot &&) = default; - IterSnapshot(IterSnapshot const&) = default; - IterSnapshot& operator= (IterSnapshot const&) = default; - IterSnapshot& operator= (IterSnapshot &&) = default; - explicit operator bool() const { diff --git a/src/lib/iter-adapter.hpp b/src/lib/iter-adapter.hpp index 453f51f87..7d3d85875 100644 --- a/src/lib/iter-adapter.hpp +++ b/src/lib/iter-adapter.hpp @@ -30,7 +30,8 @@ ** - very similar is IterableDecorator, but this time directly as ** decorator to inherit from the »state core«, and without checks. ** - the RangeIter allows just to expose a range of elements defined - ** by a STL-like pair of "start" and "end" iterators + ** by a STL-like pair of "start" and "end" iterators; thus it + ** serves a bridge between STL iterators and Lumiera iterators. ** ** Some more specific use cases are provided in the extension header ** iter-adapter-ptr-deref.hpp @@ -62,14 +63,14 @@ ** Similar to the STL, instead of using a common "Iterator" base class, ** we rather define a common set of functions and behaviour which can be ** expected from any such iterator. These rules are similar to STL's - ** "forward iterator", with the addition of an bool check to detect - ** iteration end. The latter is inspired by the \c hasNext() function + ** "input iterator", with the addition of an bool check to detect + ** iteration end. The latter is inspired by the `hasNext()` function ** found in many current languages supporting iterators. However, by - ** inspiration from functional programming, we deliberately do not - ** support the various extended iterator concepts from STL and boost + ** drawing from insights promoted in functional programming, deliberately + ** we do not support the various extended iterator concepts from the STL ** (random access iterators, output iterators, arithmetics, difference - ** between iterators and the like). According to this concept, - ** _an iterator is a promise for pulling values once,_ + ** between iterators and the like). According to this concept proposed + ** here, _an iterator is a promise for pulling values once,_ ** and nothing beyond that. ** ** Notably, @@ -87,10 +88,48 @@ ** Conceptually, a Lumiera Iterator represents a lazy stream of calculations ** rather than a target value considered to be »within« a container. And while ** the result is in may cases deliberately _exposed as a reference,_ in order - ** to keep the door open for special-case manipulations, for the typical usage - ** it is _discouraged_ to assume anything about the source, beyond the limited - ** access to some transient state as exposed during active iteration. Together, - ** these rules enable a _loose coupling_ to the source of data. + ** to allow for tightly coupled implementation in special cases -- for the + ** typical usage it is _discouraged_ to assume anything about the source, + ** beyond the limited access to some transient state as exposed during + ** active iteration. Together, these rules enable a _loose coupling_ + ** to the source of data. + ** + ** ## Usage + ** + ** All the building blocks defined her provide the bridge definitions to allow + ** using a »Lumiera Forward Iterator« in common language constructs: It can be + ** used directly in for-each loops (notably this implies that a Lumiera iterator + ** is used in the same position where usually the container would be referred). + ** Yet, beyond that, Lumiera iterators can also be used in normal `for` loops + ** and even in while loops, due to the bool-check to detect iteration end. + ** A default constructed Lumiera iterator is always in _invalid state_ + ** and can be used as _sentinel value_ like the `end()` iterator. + ** Yet, while Lumiera iterators can be copied and assigned, the effect of + ** these operations is implementation defined; in some cases, taking a copy + ** may allow to create a snapshot, while in other cases (especially when the + ** Lumiera iterator uses an reference to tap into some remote service) all + ** copies will refer to the same shared state. The rationale for this design + ** was to simplify both building and using Lumiera Iterators; they are meant + ** as an implementation tool, and not for creating external interfaces. + ** + ** However, there is a special provision for the latter use case, in the form + ** of the lib::IterSource interface and framework; an `IterSource` embeds a + ** classic OO interface with virtual methods, and thus allows to abstract the + ** backing implementation entirely away. + ** + ** ## Concepts and Traits + ** + ** The »Lumiera Forward Iterators« do not map directly to any specification from + ** the C++ standard. The concept std::input_iterator comes close, but a Lumiera + ** iterator can not fulfil this concept, since it lacks the postfix increment. + ** In the opposite direction, there is also no subsumption possible, since a + ** STL iterator does not offer a bool-check to detect iteration end. + ** + ** Notably the header `lib/meta/trait.hpp` defines some detector traits, allowing + ** to recognise if something is an iterable STL container, or a Lumiera iterator. + ** Furthermore, lib::iter::Trait defines a transparent mechanism to extract the + ** basic type definitions (`reference` ≙ what the iterator yields and + ** `value_type` ≙ what to use to store results from an iterator) ** ** @see iter-adapter-test.cpp ** @see itertools.hpp @@ -105,6 +144,7 @@ #include "lib/error.hpp" +#include "lib/nocopy.hpp" #include "lib/meta/value-type-binding.hpp" #include @@ -120,6 +160,9 @@ namespace util { // see lib/util.hpp namespace lib { + using std::is_pointer_v; + using std::remove_pointer_t; + using std::remove_reference_t; namespace { // internal helpers inline void @@ -149,7 +192,14 @@ namespace lib { namespace iter { - /** type binding helper: an iterato's actual result type */ + using meta::is_StateCore; + using meta::can_LumieraIter; + using meta::can_STL_ForEach; + using meta::ValueTypeBinding; + using meta::RefTraits; + + + /** type binding helper: an iterator's actual result type */ template using Yield = decltype(std::declval().operator*()); @@ -157,6 +207,35 @@ namespace lib { template using CoreYield = decltype(std::declval().yield()); + + /** some legacy STL type iterators do not strictly fulfil all requirements, + * yet were marked by a specialisation of `std::iterator_traits` */ + template + concept legacy_traits_marked = requires + { + typename std::iterator_traits::iterator_category; + }; + + + /** Iterator-Traits: definitions to level the difference between + * Lumiera-Iterators and the STL usage of ranges / iterators. + */ + template + struct Trait + { + static constexpr bool is_LumieraIter = meta::is_LumieraIter_v; + static constexpr bool is_STLIter = std::input_iterator; + static constexpr bool is_Legacy = legacy_traits_marked; + + static_assert(is_LumieraIter or is_STLIter or is_Legacy + ,"source type must be an iterator (either Lumiera or STL"); + + using _ValTrait = std::conditional_t + , RefTraits>>; + using value_type = _ValTrait::value_type; + using reference = _ValTrait::reference; + using pointer = _ValTrait::pointer; + }; } @@ -302,7 +381,7 @@ namespace lib { protected: - using ConRef = meta::RefTraits::Reference; + using ConRef = meta::RefTraits::reference; /** allow derived classes to access backing container */ ConRef source() { return source_; } @@ -370,15 +449,15 @@ namespace lib { * @see iter-explorer-test.hpp * @see iter-adaptor-test.cpp */ - template> + template> class IterStateWrapper { ST core_; public: - using value_type = meta::RefTraits::Value; - using reference = meta::RefTraits::Reference; - using pointer = meta::RefTraits::Pointer; + using value_type = meta::RefTraits::value_type; + using reference = meta::RefTraits::reference; + using pointer = meta::RefTraits::pointer; IterStateWrapper (ST&& initialState) : core_(std::forward(initialState)) @@ -399,9 +478,9 @@ namespace lib { } - /* === lumiera forward iterator concept === */ + /* === Lumiera Forward Iterator concept === */ - T + RES operator*() const { __throw_if_empty(); @@ -411,13 +490,13 @@ namespace lib { pointer operator->() const { - if constexpr (meta::isLRef_v) + if constexpr (meta::isLRef_v) { __throw_if_empty(); return & core_.yield(); // core interface: yield } else - static_assert (!sizeof(T), + static_assert (!sizeof(RES), "can not provide operator-> " "since iterator pipeline generates a value"); } @@ -500,7 +579,7 @@ namespace lib { class IterStateCore : public IT { - static_assert (lib::meta::can_IterForEach::value + static_assert (iter::can_LumieraIter::value ,"Lumiera Iterator required as source"); protected: IT& @@ -545,7 +624,7 @@ namespace lib { class CheckedCore : public COR { - static_assert (lib::meta::is_StateCore::value + static_assert (iter::is_StateCore::value ,"Adapted type must expose a »state core« API"); protected: COR& @@ -608,6 +687,7 @@ namespace lib { * @warning be sure to understand the implications of this setup * - when initialised by reference, the container's contents will be copied * - when move-initialised, the container will be destroyed with this iterator + * - when moved / swapped during iteration, the iterator is invalidated (-> data corruption) * - the container API remains visible (baseclass), which could confuse trait detection */ template @@ -649,6 +729,67 @@ namespace lib { + template + concept subscriptable = requires (CON& con, size_t idx) + { + typename CON::reference; + { con[idx] } -> std::convertible_to; + { con.size() } -> std::convertible_to; + }; + + template + using is_subscriptable = std::bool_constant>>; + + /** + * Adapter to package some container store inline and access it bey sequenced subscript. + * This setup works similar to the ContainerCore, yet has the benefit of being movable. + * @warning using any kind of inline storage in the context of IterExplorer is dangerous. + * Anything beyond the actual iteration state and especially an extended data storage + * might lead to surprising and harmful behaviour, due to the contradiction to the + * common mental image associated with an iterator. These dangers are amplified + * when such embedded data is transformed, filtered or modified by reference. + */ + template + class IdxStoreCore + : public CON + { + size_t i_; + + public: + IdxStoreCore (CON&& container) + : CON(std::forward (container)) + , i_{0} + { } + + IdxStoreCore() + : CON{} + , i_{CON::size()} + { } // explicitly ensure that iterator is empty + + // copy and assignment acceptable (warning!) + + + /* === »state core« protocol API === */ + bool + checkPoint() const + { + return i_ < CON::size(); + } + + decltype(auto) + yield() const + { + return util::unConst(*this).operator[] (i_); + } // rationale: const iterator ≠ const_iterator + + void + iterNext() + { + ++i_; + } + }; + + /** * Decorator-Adapter to make a »*State Core*« iterable as Lumiera Forward Iterator. @@ -694,9 +835,9 @@ namespace lib { public: using YieldRes = iter::CoreYield; - using value_type = meta::RefTraits::Value; - using reference = meta::RefTraits::Reference; - using pointer = meta::RefTraits::Pointer; + using value_type = meta::RefTraits::value_type; + using reference = meta::RefTraits::reference; + using pointer = meta::RefTraits::pointer; /** by default, pass anything down for initialisation of the core. @@ -782,17 +923,23 @@ namespace lib { /** - * Accessing a STL element range through a Lumiera forward iterator, + * Access a STL element range through a »Lumiera Forward Iterator«. * An instance of this iterator adapter is completely self-contained - * and allows to iterate once over the range of elements, until + * and allows to iterate _once_ over the range of elements, until * `pos==end`. Thus, a custom container may expose a range of * elements of an embedded STL container, without controlling - * the details of the iteration (as is possible using the + * the details of the iteration (as would be possible using the * more generic IterAdapter). * @note - * - when IT is just a pointer, we use the pointee as value type - * - but when IT is a class, we expect the usual STL style nested typedefs - * `value_type`, `reference` and `pointer` + * - when IT is a STL iterator, we rely on the iterator concepts and + * especially on `std::iter_reference_t` to find the `reference_type` + * - but when IT is a »Lumiera Forward Iterator«, we need to evaluate + * nested typedefs `value_type`, `reference` and `pointer` + * @remark the reason for this tricky distinction is a _clash of concepts:_ + * In Lumiera, »Iterator« has a special meaning and can used on itself, + * while in the STL, an »Iterator« is a pointer in disguise and is + * only meaningful when the backing container is known. This adaptor, + * RangeIter, acts as a bridge and levels those usage styles. */ template class RangeIter @@ -800,14 +947,10 @@ namespace lib { IT p_; IT e_; - using _ValTrait = meta::ValueTypeBinding>; - public: - using pointer = _ValTrait::pointer; - using reference = _ValTrait::reference; - - /// @note special twist, since a STL const_iterator would yield a non-const `value_type` - using value_type = std::remove_reference::type; + using pointer = iter::Trait::pointer; + using reference = iter::Trait::reference; + using value_type = iter::Trait::value_type; RangeIter (IT const& start, IT const& end) @@ -1027,13 +1170,13 @@ namespace lib { /** * Helper for type rewritings: - * get the element type for an iterator like entity + * build a similar iterator with another value type */ template - struct IterType; + struct IterRebind; template class Iter, class TY, class CON> - struct IterType> + struct IterRebind> { using Container = CON; using ElemType = TY ; @@ -1046,13 +1189,13 @@ namespace lib { }; template - struct IterType> - : IterType + struct IterRebind> + : IterRebind { template struct SimilarIter ///< rebind to rewritten Iterator wrapped into RangeIter { - using WrappedIter = IterType::template SimilarIter::Type; + using WrappedIter = IterRebind::template SimilarIter::Type; using Type = RangeIter; }; }; diff --git a/src/lib/iter-cursor.hpp b/src/lib/iter-cursor.hpp index 532ab5279..45b764691 100644 --- a/src/lib/iter-cursor.hpp +++ b/src/lib/iter-cursor.hpp @@ -56,9 +56,9 @@ namespace lib { public: - using pointer = meta::ValueTypeBinding::pointer; - using reference = meta::ValueTypeBinding::reference; - using value_type = std::remove_reference::type; ///< @note will be const for const iterators (while const_iterator::value_type isn't) + using pointer = iter::Trait::pointer; + using reference = iter::Trait::reference; + using value_type = iter::Trait::value_type; CursorGear() diff --git a/src/lib/iter-explorer.hpp b/src/lib/iter-explorer.hpp index ba50bb54c..33a366f34 100644 --- a/src/lib/iter-explorer.hpp +++ b/src/lib/iter-explorer.hpp @@ -13,13 +13,13 @@ */ /** @file iter-explorer.hpp - ** Building tree expanding and backtracking evaluations within hierarchical scopes. + ** Build tree expanding and backtracking evaluations within hierarchical scopes. ** Based on the *Lumiera Forward Iterator* concept and using the basic IterAdapter templates, ** these components allow to implement typical evaluation strategies, like conditional expanding ** or depth-first exploration of a hierarchical structure. Since the access to this structure is ** abstracted through the underlying iterator, what we effectively get is a functional datastructure. - ** The implementation is based on the idea of a "state core", which is wrapped right into the iterator - ** itself (value semantics) -- similar to the IterStateWrapper, which is one of the basic helper templates + ** The implementation is based on the idea of a **State Core**, which is wrapped right into the iterator + ** itself (value semantics) — similar to the IterStateWrapper, which is one of the basic helper templates ** provided by iter-adapter.hpp. ** ** @remark historically, this template, as well as the initial IterExplorer (draft from 2012) can be @@ -36,7 +36,7 @@ ** calculations with exception handling but also simple data structures like lists or trees). The key point with any ** monad is the ability to _bind a function_ into the monad; this function will work on the _contained base values_ ** and produce a modified new monad instance. In the simple case of a list, "binding" a function basically means - ** to _map the function onto_ the elements in the list. (strictly speaking, it means the `flatMap` operation) + ** to _map the function onto_ the elements in the list. (more precisely, this indicates a `flatMap` operation) ** ** # A Pipeline builder ** Based on such concepts, structures and evaluation patterns, the IterExplorer serves the purpose to provide @@ -49,8 +49,9 @@ ** on each element, rather it is triggered by issuing a dedicated `expandChildren()` call on the processing ** pipeline. Thus, binding the expansion functor has augmented the data source with the ability to explore ** some part in more detail _when required_. - ** - the *transform operation* installs a function to be mapped onto each element retrieved from the underlying source - ** - in a similar vein, the *filter operation* binds a predicate to decide about using or discarding data + ** - the **transform** operation installs a function to be mapped onto each element retrieved from the underlying source + ** - in a similar vein, the **filter** operation binds a predicate to decide about using or discarding iterator results + ** - the **flatten** operation allows to combine a sequence of nested _iterables_ into a single seamless iterator ** - in concert, expand- and transform operation allow to build hierarchy evaluation algorithms without exposing ** any knowledge regarding the concrete hierarchy used and explored as data source. ** - further special convenience adaptors and _terminal functions_ are provided. @@ -101,6 +102,7 @@ #include "lib/error.hpp" #include "lib/uninitialised-storage.hpp" #include "lib/meta/duck-detector.hpp" +#include "lib/meta/common-result.hpp" #include "lib/meta/function.hpp" #include "lib/meta/trait.hpp" #include "lib/item-wrapper.hpp" @@ -217,6 +219,14 @@ namespace lib { } }; + + // (forward declarations to allow defining all traits together) + template + class RedressAdapter; + + template + class WrapperAdapter; + }//(End) namespace iter_explorer : basic iterator wrappers @@ -237,9 +247,10 @@ namespace lib { using std::common_type_t; using std::conditional_t; using std::is_convertible; + using std::is_lvalue_reference; using std::remove_reference_t; using meta::is_StateCore; - using meta::can_IterForEach; + using meta::can_LumieraIter; using meta::can_STL_ForEach; using meta::ValueTypeBinding; using meta::has_TypeResult; @@ -248,17 +259,24 @@ namespace lib { template struct shall_wrap_STL_Iter : __and_ - ,__not_> + ,__not_> > { }; template struct shall_use_Lumiera_Iter - : __and_ + : __and_ ,__not_> > { }; + template + struct shall_use_StateCore + : __and_<__not_> + ,is_StateCore + > + { }; + /** @@ -273,8 +291,8 @@ namespace lib { template struct _DecoratorTraits>> { - using SrcRaw = lib::meta::Strip::Type; - using SrcVal = meta::RefTraits>::Value; + using SrcRaw = lib::meta::Strip::TypeReferred; + using SrcVal = meta::RefTraits>::value_type; using SrcIter = lib::IterableDecorator>; }; @@ -288,7 +306,7 @@ namespace lib { template struct _DecoratorTraits>> { - static_assert (not std::is_rvalue_reference::value, + static_assert (not std::is_rvalue_reference_v, "container needs to exist elsewhere during the lifetime of the iteration"); using SrcIter = iter_explorer::StlRange; using SrcVal = SrcIter::value_type; @@ -312,6 +330,78 @@ namespace lib { { }; + /** + * @internal Type-selector to adapt a recursive iterable for flattening... + * @tparam SRC the underlying source iterator (which yields a nested iterable as result) + * @tparam RES result type of the source iterator (this is the nested iterable itself) + * @remark to explain the general scheme employed here... + * - we want to add a processing layer with a iter_explorer::Flattener + * - this requires a source-iterator that yields a nested iterable of sorts + * - the source-iterator as such has already been adapted by the [generic mechanism](\ref _DecoratorTraits) + * - however there remains the twist that the _nested result_ (which is some kind of iterable) + * also needs to be adapted, so that it can be handled as Lumiera Forward Iterator... + * - this can be achieved by interspersing an adapter layer _below_ the Flattener, + * so that the result of the underlying iterator will be adapted / casted / wrapped + * - furthermore, we have to distinguish the cases where the underlying iterator produces a value result + * - overall this trait template is instantiated with the type \a SRC of the underlying iterator + * - with the help of the second parameter's default, the \a RES result type of this iterator is determined + * - the specialisation matcher then classifies this result type and determines an adaptation strategy + * - finally, the Flattener is parametrised with this `SrcAdapter` type, but we pass `move(*this) to + * its constructor; since `*this` inherits from \a SRC, the Adapter will pick up and wrap \a SRC + * There are two basic adaptation strategies. First, if the source iterator yields an iterator by reference, + * and we can adapt this iterator on-the-fly (without requiring additional storage), we can just cast the result. + * In all other cases, a setup similar to a Transformer is required: for each source-result (which is in itself + * iterable), a wrapper is constructed into an _inline storage space_ (implemented as \ref lib::ItemWrapper), + * and this wrapper is exposed by-ref as "result value", so that the Flattener can iterate it until exhausted. + */ + template, typename SEL=void> + struct _FlatteningTraits + { + static_assert (!sizeof(SRC), "Can not build Flattener: Unable to figure out how to iterate the nested iterable."); + }; + + template + struct _FlatteningTraits, can_LumieraIter >>> + { + using SrcAdapter = meta::RefTraits::value_type; + }; + template + struct _FlatteningTraits>, can_LumieraIter >>> + { + using NestedIter = meta::RefTraits::value_type; + using SrcAdapter = iter_explorer::WrapperAdapter; + }; + + template + struct _FlatteningTraits, shall_use_StateCore >>> + { + using CoreRaw = lib::meta::Strip::TypeReferred; + using NestedIter = IterableDecorator; + using SrcAdapter = iter_explorer::RedressAdapter; + }; + template + struct _FlatteningTraits>, shall_use_StateCore >>> + { + using CoreRaw = lib::meta::Strip::TypeReferred; + using NestedIter = IterableDecorator; + using SrcAdapter = iter_explorer::WrapperAdapter; + }; + + template + struct _FlatteningTraits, shall_wrap_STL_Iter >>> + { + using NestedIter = iter_explorer::StlRange; + using SrcAdapter = iter_explorer::WrapperAdapter; + }; + template + struct _FlatteningTraits>, is_subscriptable >>> + { + using StoreRaw = lib::meta::Strip::TypeReferred; + using NestedIter = IterableDecorator>; + using SrcAdapter = iter_explorer::WrapperAdapter; + }; + + /** * helper to derive a suitable common type when expanding children @@ -327,19 +417,15 @@ namespace lib { using ResIter = _DecoratorTraits::SrcIter; using SrcYield = iter::Yield; using ResYield = iter::Yield; - using _CommonT = meta::CommonResultYield; + using _CommonT = meta::CommonResult; static constexpr bool can_reconcile = _CommonT::value; - static constexpr bool isRefResult = _CommonT::isRef; static_assert (can_reconcile, "source iterator and result from the expansion must yield compatible values"); static_assert (is_const_v == is_const_v, "source and expanded types differ in const-ness"); - using YieldRes = _CommonT::ResType; - using value_type = _CommonT::value_type; - using reference = _CommonT::reference; - using pointer = _CommonT::pointer; + using YieldRes = _CommonT::Type; }; }//(End) IterExplorer traits @@ -478,13 +564,15 @@ namespace lib { struct _ReduceTraits { using Result = iter_explorer::_FunTraits::Res; - using ResVal = lib::meta::RefTraits::Value; + using ResVal = lib::meta::RefTraits::value_type; }; + /* ========= IterExplorer pipeline building blocks ========= */ + /** * @internal Base of pipe processing decorator chain. * IterExplorer allows to create a stack out of various decorating processors @@ -516,45 +604,81 @@ namespace lib { }; + /** + * @internal Iterator adapter to _virtually project_ an arbitrary type \a RES + * onto the result of the underlying Lumiera Iterator, by means of a force-cast. + */ + template + class RedressAdapter + : public SRC + { + static_assert(can_LumieraIter::value, "Lumiera Iterator required as source."); + + using SrcRes = iter::Yield; + static_assert(meta::isLRef_v, "Source iterator must yield its result by-ref."); + + public: + using value_type = meta::RefTraits::value_type; + using reference = meta::RefTraits::reference; + using pointer = meta::RefTraits::pointer; + + reference + operator*() const + { + return reinterpret_cast (SRC::operator*()); + } + + pointer + operator->() const + { + return & operator*(); + } + + using SRC::SRC; + }; + + /** * @internal Decorator for IterExplorer adding the ability to "expand children". - * The expandChildren() operation is the key element of a depth-first evaluation: it consumes - * one element and performs a preconfigured _expansion functor_ on that element to yield - * its "children". These are given in the form of another iterator, which needs to be - * compatible to the source iterator ("compatibility" boils down to both iterators - * yielding a compatible value type). Now, this _sequence of children_ effectively - * replaces the expanded source element in the overall resulting sequence; which - * means, the nested sequence was _flattened_ into the results. Since this expand() - * operation can again be invoked on the results, the implementation of such an evaluation - * requires a stack datastructure, so the nested iterator from each expand() invocation - * can be pushed to become the new active source for iteration. Thus the primary purpose - * of this Expander (decorator) is to integrate those "nested child iterators" seamlessly - * into the overall iteration process; once a child iterator is exhausted, it will be - * popped and iteration continues with the previous child iterator or finally with - * the source iterator wrapped by this decorator. The source pipeline is only pulled - * once the expanded children are exhausted. + * The expandChildren() operation is the key element of a _depth-first evaluation:_ + * it consumes one element and performs a preconfigured _expansion functor_ on that element + * to yield its "children". These are given in the form of another iterator, which needs to be + * compatible to the source iterator (within this context, "compatibility" boils down to both + * iterators yielding a compatible value type). Now, this _sequence of children_ effectively + * replaces the expanded source element in the overall resulting sequence; which means that + * the nested sequence was _flattened_ into the results. Since this expand() operation can + * again be invoked on the results, the implementation of such an evaluation requires a + * **stack datastructure**, so the nested iterator from each expand() invocation can be + * _pushed on top_ in order to become the new active source for iteration. Thus the prime + * purpose of this Expander (decorator) is to integrate those "nested child iterators" + * seamlessly into the overall iteration process; once a child iterator is exhausted, + * it will be _popped_ and iteration continues with the previous child iterator, or, + * finally, with the source iterator wrapped by this decorator. Overall, the given + * source pipeline is only pulled, whenever the expanded children are exhausted. * @remark since we allow a lot of leeway regarding the actual form and definition of the - * _expansion functor_, there is a lot of minute technical details, mostly confined - * within the _FunTraits traits. For the same reason, we need to prepare two different - * bindings of the passed raw functor, one to work on the source sequence, and the other - * one to work on the result sequence of a recursive child expansions; these two sequences - * need not be implemented in the same way, which simplifies the definition of algorithms. - * @tparam SRC the wrapped source iterator, typically a IterExplorer or nested decorator. - * @tparam FUN the concrete type of the functor passed. Will be dissected to find the signature + * _expansion functor_, various technical details must be observed, and these minutiae + * are largely maintained within the _FunTraits traits. For the same reason, we need to + * prepare two different bindings of the passed raw functor, where one of them is used + * to work on the source sequence, while the other one handles the result sequence of + * _recursive child expansions;_ these two sequences need not be implemented precisely + * in the same way — what matters here is compatible behaviour — and this flexible + * setup helps to simplify the definition of search- and evaluation algorithms. + * @tparam SRC the wrapped source iterator, typically an iterator adaptor or nested decorator. + * @tparam FUN actual type of the functor passed. Will be type-matched to detect its signature. * @note the _return type_ of #yield depends _both_ on the return type produced from the original * sequence and the return type of the sequence established through the expand functor. * An attempt is made to _reconcile these_ and this attempt may fail (at compile time). - * The reason is, any further processing downstream can not tell if data was produced - * by the original sequence of the expansion sequence. Notably, if one of these two - * delivers results by-value, then the Expander will _always_ deliver all results - * by-value, because it would not be possible to expose a reference to some value + * The rationale is that any further processing downstream can not tell if a data result + * was produced by the original sequence of the expansion sequence. Notably, if one of + * these two delivers results by-value, then the Expander will _always_ choose to deliver + * all its results by-value, since it is impossible to expose a reference to some value * that was just delivered temporarily from a source iterator. */ template class Expander : public SRC { - static_assert(can_IterForEach::value, "Lumiera Iterator required as source"); + static_assert(can_LumieraIter::value, "Lumiera Iterator required as source"); using _Trait = _ExpanderTraits; using ResIter = _Trait::ResIter; @@ -622,9 +746,9 @@ namespace lib { /** @note result type bindings based on a common type of source and expanded result */ using YieldRes = _Trait::YieldRes; - using value_type = _Trait::value_type; - using reference = _Trait::reference; - using pointer = _Trait::pointer; + using value_type = meta::RefTraits::value_type; + using reference = meta::RefTraits::reference; + using pointer = meta::RefTraits::pointer; bool @@ -699,7 +823,7 @@ namespace lib { /** * @internal extension to the Expander decorator to perform expansion automatically on each iteration step. - * @todo as of 12/2017, this is more like a proof-of concept and can be seen as indication, that there might + * @todo as of 12/2017, this is more like a proof-of concept and can be seen as indication that there might * be several flavours of child expansion. Unfortunately, most of these conceivable extensions would * require a flexibilisation of Expander's internals and thus increase the complexity of the code. * Thus, if we ever encounter the need of anything beyond the basic expansion pattern, we should @@ -726,7 +850,7 @@ namespace lib { /** - * @internal extension to the Expander decorator to perform expansion delayed on next iteration. + * @internal extension to the Expander decorator to perform a delayed expansion on next iteration. */ template class ScheduledExpander @@ -761,14 +885,94 @@ namespace lib { }; + /** + * @internal Decorator for IterExplorer to expand one level of nested iterables. + * This mechanism requires that the _result value_ yielded by the underlying iterator + * is itself iterable. Applying this decorator thus has the effect to »flatten« exactly + * one level of nesting. + * @note The result produced by underlying \a SRC _must be a Lumiera iterator_. + * Other kinds of nested iterables can be handled only when adapted accordingly. + */ + template + class Flattener + : public SRC + { + static_assert(can_LumieraIter::value, "Lumiera Iterator required as source"); + + using NestedIT = iter::Yield; + static_assert(can_LumieraIter::value, "flatten() operation expects nested iterator"); + + public: + using value_type = meta::ValueTypeBinding::value_type; + using reference = meta::ValueTypeBinding::reference; + using pointer = meta::ValueTypeBinding::pointer; + + Flattener() =default; + // inherited default copy operations + + Flattener (SRC&& dataSrc) + : SRC{move (dataSrc)} + { + pullNested(); + } + + + /** refresh state when other layers manipulate the source sequence. + * @note possibly pulls pipeline to re-establish the invariant */ + void + expandChildren() + { + SRC::expandChildren(); + pullNested(); + } + + public: /* === Iteration control API for IterableDecorator === */ + + bool + checkPoint() const + { + ENSURE (not srcIter() or * srcIter()); + return bool(srcIter()); + } + + reference + yield() const + { + return **srcIter(); + } + + void + iterNext() + { + ++ *srcIter(); + pullNested(); + } + + protected: + SRC& + srcIter() const + { + return unConst(*this); + } + + /** @note Invariant: positioned at next nested source element, if any */ + void + pullNested() + { + while (srcIter() and not * srcIter()) + ++srcIter(); + } + }; + + /** - * @internal Decorator for IterExplorer to map a transformation function on all results. + * @internal Decorator for IterExplorer to map a transformation function onto all results. * The transformation function is invoked on demand, and only once per item to be treated, * storing the treated result into an universal value holder buffer. The given functor * is adapted in a similar way as the "expand functor", so to detect and convert the * expected input on invocation. - * @note the result-type of the #yield() function _must be_ `reference`, even when + * @note the result-type of the #yield() function _must be_ a `reference`, even when * the TransformFunctor produces a value; otherwise we can not provide a safe * `operator->` on any iterator downstream. This is also the reason why the * ItemWrapper is necessary, precisely _because we want to support_ functions @@ -782,7 +986,7 @@ namespace lib { class Transformer : public SRC { - static_assert(can_IterForEach::value, "Lumiera Iterator required as source"); + static_assert(can_LumieraIter::value, "Lumiera Iterator required as source"); using TransformFunctor = function; using TransformedItem = wrapper::ItemWrapper; @@ -790,10 +994,10 @@ namespace lib { TransformFunctor trafo_; TransformedItem treated_; - public: - using value_type = meta::ValueTypeBinding::value_type; - using reference = meta::ValueTypeBinding::reference; - using pointer = meta::ValueTypeBinding::pointer; + public: // Note: result might be a nested iterator -- not using ValueTypeBinding here! + using value_type = meta::RefTraits::value_type; + using reference = meta::RefTraits::reference; + using pointer = meta::RefTraits::pointer; template @@ -826,6 +1030,7 @@ namespace lib { t1.treated_.reset(); t2.treated_.reset(); swap (t1.trafo_, t2.trafo_); + swap (t1.srcIter(), t2.srcIter()); } @@ -880,6 +1085,125 @@ namespace lib { + /** + * Iterator adapter to package each iterator result into a **wrapper object** of type \a WRA. + * Built as a simplified variant of the Transformer, it can not invoke an arbitrary function, + * yet will destroy the previous wrapper and construct a new wrapper into the embedded storage + * buffer on each iteration. + * @note similar as the Transformer, WrapperAdapter will _re-engage_ after copy, move and swap. + * This implies to discard a wrapped object residing in the ItermWrapper's storage; the next + * result-pull will then cause it to be regenerated from source, since the absolute storage + * locations both of the ItemWrapper storage, and the underlying source iterator has changed. + * Notably this protects against insidious data corruption in cases where some part of the + * implementation (typically indirectly) happens to take a reference into the source. + * @warning this mechanism might have surprising ramifications, especially when the underlying + * iterator has generated a collection by-value; this data will be discarded whenever the + * iterator pipeline is moved, which might happen automatically when adding further layers. + * Notably there is an interference with possible iterator decorators _added downstream_, + * that need to pull ahead for grouping and filtering. + * @remark Users of WrapperAdapter, and of IterExplorer are responsible in general to ensure + * that any embodied structure preferably relies on value copy of iterator results rather, + * and otherwise only employs references into stable memory locations. + */ + template + class WrapperAdapter + : public SRC + { + using WrappedItem = wrapper::ItemWrapper; + + WrappedItem wrapped_{/*created empty*/}; + + public: // Note: result might be a nested iterator -- not using ValueTypeBinding here! + using value_type = meta::RefTraits::value_type; + using reference = meta::RefTraits::reference; + using pointer = meta::RefTraits::pointer; + + WrapperAdapter (SRC&& dataSrc) + : SRC{move (dataSrc)} + { } + + WrapperAdapter() =default; + WrapperAdapter (WrapperAdapter const& o) + : SRC{o} + , wrapped_{/* deliberately empty: force re-engage */} + { } + WrapperAdapter (WrapperAdapter && o) + : SRC{move (o)} + , wrapped_{/* deliberately empty: force re-engage */} + { } + WrapperAdapter& + operator= (WrapperAdapter changed) + { + swap (*this,changed); + return *this; + } + friend void + swap (WrapperAdapter& t1, WrapperAdapter& t2) + { + using std::swap; + t1.wrapped_.reset(); + t2.wrapped_.reset(); + swap (t1.srcIter(), t2.srcIter()); + } + + + /* === decorated Lumiera Forward Iterator === */ + + reference + operator*() const + { + return unConst(this)->reWrapIt(); + } + + pointer + operator->() const + { + return & operator*(); + } + + WrapperAdapter& + operator++() + { + wrapped_.reset(); // discard current wrapper + ++ srcIter(); + return *this; + } + + /** refresh state when other layers manipulate the source sequence */ + void + expandChildren() + { + wrapped_.reset(); + SRC::expandChildren(); + } + + + ENABLE_USE_IN_STD_RANGE_FOR_LOOPS (WrapperAdapter); + + friend bool + operator== (WrapperAdapter const& w1, WrapperAdapter const& w2) + { + return w1.srcIter() == w2.srcIter(); + } + + private: + SRC& + srcIter() const + { + return unConst(*this); + } + + reference + reWrapIt () + { + if (not wrapped_) // build a wrapper instance once per src item + wrapped_ = * srcIter(); + return *wrapped_; + } + }; + + + /** * @internal Decorator for IterExplorer to group consecutive elements into fixed sized chunks. * One group of elements is always prepared eagerly, and then the next one on iteration. @@ -892,7 +1216,7 @@ namespace lib { class Grouping : public SRC { - static_assert(can_IterForEach::value, "Lumiera Iterator required as source"); + static_assert(can_LumieraIter::value, "Lumiera Iterator required as source"); protected: using Group = std::array; @@ -1002,14 +1326,15 @@ namespace lib { /** * @internal Decorator for IterExplorer to group consecutive elements controlled by some - * grouping value \a GRP and compute an aggregate value \a AGG for each such group as + * **grouping value** \a GRP and compute an aggregate value \a AGG for each such group as * iterator yield. The first group is consumed eagerly, each further group on iteration; * thus when the aggregate for the last group appears as result, the source iterator has - * already been exhausted. The aggregate is default-initialised at start of each group - * and then the computation functor \a FAGG is invoked for each consecutive element marked - * with the same _grouping value_ — and this grouping value itself is obtained by invoking - * the functor \a FGRP on each source value. All computation are performed on-the-fly. No - * capturing or reordering of the source elements takes place, rather groups are formed + * already been exhausted. The aggregate is default-initialised at start of each group, + * followed by invoking the [computation functor](\ref #Aggregator) for each consecutive + * element that has been marked with the same _grouping value_ — and this grouping value + * itself is obtained by invoking the [grouping functor](\ref #Grouping) on each value + * produced by the underlying source iterator. All computation is performed on-the-fly. + * No capturing or reordering of the source elements takes place, rather groups are formed * based on the changes of the grouping value over the source iterator's result sequence. * @tparam AGG data type to collect the aggregate; must be default constructible and assignable * @tparam GRP value type to indicate a group @@ -1018,14 +1343,14 @@ namespace lib { * This limitation was deemed acceptable (adapting a function with several arguments would * require quite some nasty technicalities). The first argument of this `aggFun` refers * to the accumulator by value, and thereby also implicitly defines the aggregate result type. - * @warning the Aggregator \a AGG *must not capture references* to upstream internal state, because + * @warning the #Aggregator *must not capture references* to upstream internal state, because * the overall pipeline will be moved into final location _after the initial ctor call._ */ template class GroupAggregator : public SRC { - static_assert(can_IterForEach::value, "Lumiera Iterator required as source"); + static_assert(can_LumieraIter::value, "Lumiera Iterator required as source"); protected: using SrcValue = meta::ValueTypeBinding::value_type; @@ -1038,9 +1363,9 @@ namespace lib { Aggregator aggregate_; public: - using value_type = meta::RefTraits::Value; - using reference = meta::RefTraits::Reference; - using pointer = meta::RefTraits::Pointer; + using value_type = meta::RefTraits::value_type; + using reference = meta::RefTraits::reference; + using pointer = meta::RefTraits::pointer; GroupAggregator() =default; // inherited default copy operations @@ -1108,13 +1433,13 @@ namespace lib { * Similar to the Transformer, the given functor is adapted as appropriate. However, * we require the functor's result type to be convertible to bool, to serve as approval test. * The filter predicate and thus the source iterator is evaluated _eagerly_, to establish the - * *invariant* of this class: _if a "current element" exists, it has already been approved._ + * **invariant** of this class: _if a "current element" exists, it has already been approved._ */ template class Filter : public SRC { - static_assert(can_IterForEach::value, "Lumiera Iterator required as source"); + static_assert(can_LumieraIter::value, "Lumiera Iterator required as source"); protected: using FilterPredicate = function; @@ -1201,7 +1526,7 @@ namespace lib { * capturing it _by value._ After building, this remoulded version can be assigned to the * original filter functor, under the assumption that both are roughly compatible. Moreover, * since we wrap the actual lambda into an adapter, allowing for generic lambdas to be used - * as filter predicates, this setup allows for a lot of leeway regarding the concrete predicates. + * as filter predicates, this setup provides a lot of wiggle room for defining predicates. * @note whenever the filter is remoulded, the invariant is immediately * [re-established](\ref Filter::pullFilter() ), possibly forwarding the sequence * to the next element approved by the new version of the filter. @@ -1215,7 +1540,7 @@ namespace lib { : public Filter { using _Filter = Filter; - static_assert(can_IterForEach::value, "Lumiera Iterator required as source"); + static_assert(can_LumieraIter::value, "Lumiera Iterator required as source"); public: MutableFilter() =default; @@ -1365,6 +1690,7 @@ namespace lib { + /** * @internal Decorator for IterExplorer to cut iteration once a predicate ceases to be true. * Similar to Filter, the given functor is adapted as appropriate, yet is required to yield @@ -1379,7 +1705,7 @@ namespace lib { class StopTrigger : public IterStateCore { - static_assert(can_IterForEach::value, "Lumiera Iterator required as source"); + static_assert(can_LumieraIter::value, "Lumiera Iterator required as source"); using Core = IterStateCore; using Cond = function; @@ -1579,7 +1905,7 @@ namespace lib { class IterExplorer : public SRC { - static_assert(can_IterForEach::value, "Lumiera Iterator required as source"); + static_assert(can_LumieraIter::value, "Lumiera Iterator required as source"); public: @@ -1698,6 +2024,25 @@ namespace lib { } + /** adapt this IterExploer to flatten _one level of nested Lumiera Iterators_. + * Obviously this processing can only be applied when the underlying pipeline yields a result + * that can be handled as being iterable; the resulting pipeline will seamlessly produce all + * results from all non-empty nested iterators, in sequence. + * @remark when chained behind a suitable [transformer(fun)](\ref transform()), the processing + * pattern is similar to the `flatMap(fun)` or `M >= fun` operation known from Monad theory. + * @return processing pipeline with attached [flattening](\ref iter_explorer::Flattener) decorator + */ + auto + flatten() + { + using Adapted = _FlatteningTraits::SrcAdapter; + using ResCore = iter_explorer::Flattener; + using ResIter = _DecoratorTraits::SrcIter; + + return IterExplorer (ResCore{Adapted{move(*this)}}); + } + + /** adapt this IterExplorer to pipe each result value through a transformation function. * Several "layers" of mapping can be piled on top of each other, possibly mixed with the * other types of adaptation, like the child-expanding operation, or a filter. Obviously, @@ -1760,7 +2105,7 @@ namespace lib { static_assert (meta::is_BinaryFun()); using ArgType1 = _Fun::Args::List::Head; - using Aggregate = meta::RefTraits::Value; + using Aggregate = meta::RefTraits::value_type; using ResCore = iter_explorer::GroupAggregator; using ResIter = _DecoratorTraits::SrcIter; @@ -1905,7 +2250,7 @@ namespace lib { asPtr() { using Val = meta::ValueTypeBinding::value_type; - static_assert (not std::is_pointer_v); + static_assert (not is_pointer_v); return IterExplorer::transform ([](Val& ref){ return &ref; }); } diff --git a/src/lib/iter-source.hpp b/src/lib/iter-source.hpp index 8a157a336..352a88f66 100644 --- a/src/lib/iter-source.hpp +++ b/src/lib/iter-source.hpp @@ -44,6 +44,7 @@ #include "lib/meta/util.hpp" +#include "lib/meta/function.hpp" #include "lib/iter-adapter.hpp" #include "lib/itertools.hpp" #include "lib/nocopy.hpp" @@ -310,7 +311,7 @@ namespace lib { { using Src = std::remove_reference::type; using ResVal = lib::meta::_Fun::Ret; - using TransIter = TransformIter; + using TransIter = decltype(lib::transformIter (std::declval(), std::declval())); using Iter = IterSource::iterator; }; @@ -326,26 +327,23 @@ namespace lib { // we can strip the const added by the STL map types using KeyType = std::remove_const::type; - using KeyIter = TransformIter; - using ValIter = TransformIter; - static KeyType takeFirst (PairType const& pair) { return pair.first; } static ValType takeSecond(PairType const& pair) { return pair.second;} }; template - _PairIterT::KeyIter + auto takePairFirst (IT&& source) { - return transformIterator(forward(source), _PairIterT::takeFirst ); + return transformIter (forward(source), _PairIterT::takeFirst ); } template - _PairIterT::ValIter + auto takePairSecond (IT&& source) { - return transformIterator(forward(source), _PairIterT::takeSecond ); + return transformIter (forward(source), _PairIterT::takeSecond ); } } //(END) type helpers... @@ -375,10 +373,10 @@ namespace lib { auto singleVal (VAL&& something) { - using Src = decltype(singleValIterator (forward(something))); + using Src = decltype(singletonIter (forward(something))); using Val = _IterT::Val; - return IterSource::build (new WrappedLumieraIter{singleValIterator (forward(something))}); + return IterSource::build (new WrappedLumieraIter{singletonIter (forward(something))}); } @@ -400,7 +398,7 @@ namespace lib { return IterSource::build ( new WrappedLumieraIter ( - transformIterator (forward(source), processingFunc))); + transformIter (forward(source), processingFunc))); } @@ -408,7 +406,7 @@ namespace lib { * all the keys of the given Map or Hashtable */ template - _MapT::KeyIter + auto eachMapKey (MAP& map) { using Range = RangeIter; @@ -438,13 +436,13 @@ namespace lib { * the distinct keys */ template - _MapT::KeyIter + auto eachDistinctKey (MAP& map) { using Range = RangeIter; Range contents (map.begin(), map.end()); - return wrapIter (filterRepetitions (takePairFirst(contents))); + return wrapIter (filterRepeat (takePairFirst(contents))); } diff --git a/src/lib/iter-zip.hpp b/src/lib/iter-zip.hpp index 1ee811f59..9f34581aa 100644 --- a/src/lib/iter-zip.hpp +++ b/src/lib/iter-zip.hpp @@ -31,7 +31,7 @@ #include "lib/iter-adapter.hpp" #include "lib/iter-explorer.hpp" -#include "lib/meta/tuple-helper.hpp" +#include "lib/meta/tuple-concept.hpp" #include diff --git a/src/lib/itertools.hpp b/src/lib/itertools.hpp index b4eb78320..405fd9ce6 100644 --- a/src/lib/itertools.hpp +++ b/src/lib/itertools.hpp @@ -2,7 +2,7 @@ ITERTOOLS.hpp - collection of tools for building and combining iterators Copyright (C) - 2009, Hermann Vosseler + 2009,2026 Hermann Vosseler   **Lumiera** is free software; you can redistribute it and/or modify it   under the terms of the GNU General Public License as published by the @@ -12,55 +12,50 @@ */ /** @file itertools.hpp - ** Helpers for working with iterators based on the pipeline model. - ** Iterators abstract from the underlying data container and provide - ** the contained data as a source to pull values from. Based on this - ** model, we can build pipelines, with filters, valves, junction points - ** and transforming facilities. The templates in this header enable such, - ** based on the Lumiera Forward Iterator concept. They build on generic - ** programming techniques, thus are intended to be combined at compile time - ** using definitive type information. Contrast this to an iterator model - ** as in Java's Collections, where Iterator is an Interface based on - ** virtual functions. Thus, the basic problem to overcome is the lack - ** of a single common interface, which could serve as foundation for - ** type inference. As a solution, we use a "onion" approach, where a - ** generic base gets configured with an active core, implementing - ** the filtering or processing functionality, while the base class - ** (IterTool) exposes the operations necessary to comply to the - ** Forward Iterator Concept. + ** Collection of tools for adapting and combining Lumiera iterators. + ** These abstract from the underlying data container and provide the + ** exposed data as a source to pull values from. Based on this model + ** of on-demand processing, pipelines can be built with data filtering + ** and transforming defined by functions. The templates and builders in + ** his header support this approach, based on the **Lumiera Forward Iterator** + ** concept. The implementation relies techniques of _generic programming_, + ** so that any kind of _invocable_ can be used as processing function and + ** actual type information is used at compile time to generate an optimised + ** calculation scheme. The tools provided in this header are fairly basic + ** however, and will typically be used to apply simple adaptations to be + ** used inline. + ** @remark A similar technique for building much more elaborate processing, + ** search and evaluation schemes is provided by iter-explorer.hpp ** - ** \par filtering Iterator - ** The FilterIter template can be used to build a filter into a pipeline, + ** # filtering Iterator + ** The iter::Filter template can be used to build a filter into a pipeline, ** as it forwards only those elements from its source iterator, which pass - ** the predicate evaluation. Anything acceptable as ctor parameter for a - ** std::function object can be passed in as predicate, but of course the - ** signature must be sensible. Please note, that -- depending on the - ** predicate -- already the ctor or even a simple `bool` test might - ** pull and exhaust the source iterator completely, in an attempt - ** to find the first element passing the predicate test. + ** the predicate evaluation. Anything invocable that yields a bool convertible + ** result can be passed in as predicate. Please note, that — depending on the + ** predicate — already the ctor or even a simple `bool` test might pull and + ** exhaust the source iterator completely, in an attempt to find the + ** first element to pass the predicate test. ** - ** \par extensible Filter - ** Based on the FilterIter, this facility allows to elaborate the filter - ** function while in the middle of iteration. The new augmented filter - ** will be in effect starting with the current element, which might even - ** be filtered away now due to a more restrictive condition. However, - ** since this is still an iterator, any "past" elements are already - ** extracted and gone and can thus not be subject to changed filtering. - ** The ExtensibleFilterIter template provides several _builder functions_ - ** to elaborate the initial filter condition, like adding conjunctive or - ** disjunctive clauses, flip the filter's meaning or even replace it - ** altogether by a completely different filter function. - ** - ** \par processing Iterator - ** the TransformIter template can be used as processing (or transforming) + ** # processing Iterator + ** The iter::Trans template can be used as processing (or transforming) ** step within the pipeline. It is created with a functor, which, when ** pulling elements, is invoked for each element pulled from the - ** source iterator. The signature of the functor must match the - ** desired value (output) type. + ** source iterator. The functor need to accept the result type from + ** the underlying source iterator and its result value determines + ** the result (output) type of the resulting pipeline. + ** + ** # convenience utilities + ** In addition, this header provides an assortment of convenience helpers + ** and utilities, that were created over time to simplify the formulation + ** of common tasks, while the fundamental building blocks for definition and + ** combination of Lumiera iterators can be found in \ref iter-adapter.hpp. + ** @warning since this header is widely used in the Lumiera codebase, + ** excessive complexity and rarely used includes should be avoided. ** - ** @see iter-adapter.hpp ** @see itertools-test.cpp - ** @see event-log.hpp + ** @see iter-explorer.hpp + ** @see iter-adapter.hpp + ** @see gen-node.hpp */ @@ -70,735 +65,502 @@ #include "lib/iter-adapter.hpp" #include "lib/item-wrapper.hpp" -#include "lib/meta/value-type-binding.hpp" -#include "lib/meta/function.hpp" #include "lib/meta/trait.hpp" #include "lib/util.hpp" #include +#include #include - namespace lib { + using std::move; using std::forward; - using std::function; using util::unConst; - using lib::meta::ValueTypeBinding; - - - - /** - * A neutral \em identity-function core, - * also serving as point-of reference how any - * core is intended to work. Any core is intended - * to serve as inner part of an iterator tool template. - * - it provides the trait typedefs - * - it abstracts the "source" - * - it abstracts the local operation to be performed - * - the ctor of the core sets up the configuration. - * @note cores should be copyable without much overhead - */ - template - struct IdentityCore - { - IT source_; - - IdentityCore (IT&& orig) - : source_{forward(orig)} - { } - IdentityCore (IT const& orig) - : source_{orig} - { } - - IT& - pipe () + namespace iter { // Implementation building blocks + using lib::meta::ValueTypeBinding; + + /** + * Predicate to be applied to an iterator or container element. + * Requires a non-strict function (side-effects acceptable) + * that operates on the value yielded by the iterator. + */ + template::reference> + concept iter_predicate = requires (FUN fun, REF ref) + { + requires std::invocable; + { fun(ref) } -> std::convertible_to; + }; + + template::reference> + concept iter_transformer = std::invocable; + + + + /** + * Pseudo-Iterator to yield just a single value. + * When incremented, the value is destroyed and + * the Iterator transitions to _exhausted state_. + * @remark can be useful to integrate special cases + * into an established interface based on iterators. + * @tparam VAL anything, value or reference to store + * @note relies on ItemWrapper, which includes a flag. + */ + template + class Single + { + using Item = wrapper::ItemWrapper; + + Item theValue_; + + void + __throw_if_empty() const + { + if (not isValid()) + _throwIterExhausted(); + } + + public: + using value_type = meta::RefTraits::value_type; + using reference = meta::RefTraits::reference; + using pointer = meta::RefTraits::pointer; + + + template> + Single (X&& val) + : theValue_{forward(val)} + { } + + Single() =default; + // standard copy operations + + + /* === Lumiera Forward Iterator concept === */ + + operator bool() const + { + return isValid(); + } + + reference + operator*() const + { + __throw_if_empty(); + return *theValue_; + } + + pointer + operator->() const + { + return & operator*(); + } + + Single& + operator++() + { + __throw_if_empty(); + theValue_.reset(); + return *this; + } + + bool + isValid() const + { + return theValue_.isValid(); + } + + bool + empty() const + { + return not isValid(); + } + + ENABLE_USE_IN_STD_RANGE_FOR_LOOPS (Single); + + friend bool + operator== (Single const& s1, Single const& s2) { - return source_; + return s1.theValue_ == s2.theValue_; } - - IT const& - pipe () const + }; + + + + /** + * Iterator adapter that applies filtering to a source iterator. + * Any invocable that can be used on the iterator's result to + * yield something bool-convertable can be employed here, and + * will be embedded as-is (which might be by reference). + * The source iterator will be pulled immediately on construction, + * until the first result is found that satisfies the filter predicate. + * @note the filter is thus invoked _at most once_ per value yielded. + * @warning this eager pull can be insidious when the predicate happens + * to store a reference to the filtered value, and the adapter object + * is then moved to another location after construction. + */ + template + class Filter + { + SRC srcIter_; + PRED predicate_; + + void + __throw_if_empty() const + { + if (not isValid()) + _throwIterExhausted(); + } + + /** @note establishes the **invariant**: + * whatever the source yields as current element, + * has already been approved by our predicate */ + void + pullFilter() + { + while (srcIter_ and not predicate_(*srcIter_)) + ++srcIter_; + } + + public: + using value_type = ValueTypeBinding::value_type; + using reference = ValueTypeBinding::reference; + using pointer = ValueTypeBinding::pointer; + + + template FUN> + Filter (IT&& src, FUN&& filter) + : srcIter_{forward (src)} + , predicate_{forward (filter)} + { + pullFilter(); // initially pull to establish the invariant + } + + Filter() =default; + // standard copy operations + + + /* === Lumiera Forward Iterator concept === */ + + operator bool() const + { + return isValid(); + } + + reference + operator*() const + { + __throw_if_empty(); + return *srcIter_; + } + + pointer + operator->() const + { + return & operator*(); + } + + Filter& + operator++() + { + __throw_if_empty(); + ++srcIter_; + pullFilter(); + return *this; + } + + bool + isValid() const + { + return bool(srcIter_); + } + + bool + empty() const + { + return not isValid(); + } + + ENABLE_USE_IN_STD_RANGE_FOR_LOOPS (Filter); + + friend bool + operator== (Filter const& s1, Filter const& s2) { - return source_; + return s1.srcIter_ == s2.srcIter_; } - - void - advance () + }; + + + /** + * Iterator adapter to map a function onto each result + * of the given source iterator. The result of this function + * invocation is captured into an ItemWrapper storage buffer. + * Consecutive value access will thus use this cached result. + */ + template + class Trans + { + using Res = std::invoke_result_t>; + using Item = wrapper::ItemWrapper; + + SRC srcIter_; + PROC transformer_; + Item transformed_; + + void + __throw_if_empty() const + { + if (not isValid()) + _throwIterExhausted(); + } + + decltype(auto) + process() + { + if (not transformed_) // invoke transform function once per src item + transformed_ = transformer_(*srcIter_); + return *transformed_; + } + + public: + using value_type = ValueTypeBinding::value_type; + using reference = ValueTypeBinding::reference; + using pointer = ValueTypeBinding::pointer; + + + template FUN> + Trans (IT&& src, FUN&& processor) + : srcIter_{forward (src)} + , transformer_{forward (processor)} + , transformed_{/* initially empty */} + { } + + Trans() = default; + // standard copy operations + + + /* === Lumiera Forward Iterator concept === */ + + operator bool() const + { + return isValid(); + } + + reference + operator*() const + { + __throw_if_empty(); + return unConst(this)->process(); + } + + pointer + operator->() const + { + return & operator*(); + } + + Trans& + operator++() + { + __throw_if_empty(); + ++srcIter_; + transformed_.reset(); + return *this; + } + + bool + isValid() const + { + return bool(srcIter_); + } + + bool + empty() const + { + return not isValid(); + } + + ENABLE_USE_IN_STD_RANGE_FOR_LOOPS (Trans); + + friend bool + operator== (Trans const& s1, Trans const& s2) { - ++source_; - } - - bool - evaluate () const - { - return bool(source_); - } - - using pointer = IT::pointer; - using reference = IT::reference; - using value_type = IT::value_type; - }; - - - /** - * Standard functionality to build up any iterator tool. - * IterTool exposes the frontend functions necessary to - * comply to the Lumiera Forward Iterator concept. - * The protected part provides the _iteration control_ - * building blocks to drive the processing/filter logic, - * which is implemented in the specific core for each tool. - */ - template - class IterTool - { - - protected: /* == iteration control == */ - CORE core_; - - bool - hasData() const - { - return core_.evaluate() - or unConst(this)->iterate(); - } // to skip irrelevant results doesn't count as "mutation" - - bool - iterate () - { - if (not core_.pipe()) return false; - - do core_.advance(); - while (core_.pipe() and not core_.evaluate()); - - return bool{core_.pipe()}; - } - - void - _maybe_throw() const - { - if (!isValid()) - _throwIterExhausted(); - } - - - - public: - using pointer = CORE::pointer; - using reference = CORE::reference; - using value_type = CORE::value_type; - - - IterTool (CORE&& setup) - : core_{std::move(setup)} - { - hasData(); - } - - explicit - operator bool() const - { - return isValid(); - } - - - - /* === lumiera forward iterator concept === */ - - reference - operator*() const - { - _maybe_throw(); - return *core_.pipe(); - } - - pointer - operator->() const - { - _maybe_throw(); - return & *core_.pipe(); - } - - IterTool& - operator++() - { - _maybe_throw(); - iterate(); - return *this; - } - - bool - isValid () const - { - return hasData(); - } - - bool - empty () const - { - return not isValid(); - } - - }; - - - template - inline bool - operator== (IterTool const& it1, IterTool const& it2) - { - return (!it1 and !it2 ) - or ( it1 and it2 and (*it1) == (*it2) ) - ; - } - - template - inline bool - operator!= (IterTool const& ito1, IterTool const& ito2) - { - return not (ito1 == ito2); - } - - - + return s1.srcIter_ == s2.srcIter_ + and (not s1 or not s2 + or *s1 == *s2); + } // even same type might hold different function + }; + + + /** + * Iterator adapter to yield a value, + * initialised from the result of an underlying iterator. + * @tparam SRC a Lumiera Forward Iterator to wrap and decorate + * @tparam forceMove if the value shall be initialised by a forced `std::move`; + * otherwise it will be initialised by `std::forward` + * @warning using this adapter may lead to _consuming_ of results; beware! + */ + template + class ValueAdapter + : public SRC + { + static_assert (can_LumieraIter::value, "Lumiera Iterator required as source."); + + public: + SRC::value_type + operator*() const + { + using Val = SRC::value_type; + if constexpr (forceMove) + return {move (SRC::operator*())}; + else + return {forward (SRC::operator*())}; + } + + SRC::pointer operator->() const =delete; + + using SRC::SRC; + }; + + + /** + * Helper: predicate returning `true` + * whenever the argument value changes + * during a sequence of invocations. + */ + template + class ChangeDetector + { + using Item = wrapper::ItemWrapper; + + Item prev_; + + public: + bool + operator() (VAL const& elm) + { + if (prev_ and + (*prev_ == elm)) + return false; + + // element differs from predecessor + prev_ = elm; + return true; + } + }; + + }// namespace iter (building blocks) /** - * Implementation of the filter logic. - * This core stores a function object instance, - * passing each pulled source element to this - * predicate function for evaluation. - * @note predicate is evaluated at most once - * for each value yielded by the source + * Build a filtering iterator that wraps a source iterator. + * Only elements classified as `true` will appear as results. + * @param filterPredicate to be invoked for each source element + * @note Both the source iterator and the predicate are _embodied_ + * into the generated filter-adapter object, yet both can be + * initialised by RValue. Furthermore this implies that both + * types should be default constructible and assignable, to + * support the typical usage pattern of an iterator. */ - template - struct FilterCore - : IdentityCore - { - using Raw = IdentityCore; - using Val = iter::Yield; - - - function predicate_; - - bool - evaluate () const - { - return Raw::pipe() - and currVal_isOK(); - } - - - mutable bool cached_; - mutable bool isOK_; - - bool - currVal_isOK () const ///< @return (maybe cached) result of filter predicate - { - return (cached_ and isOK_) - or (cached_ = true - and (isOK_ = predicate_(*Raw::pipe()))); - } - - void - advance () - { - cached_ = false; - Raw::advance(); - } - - - template - FilterCore (IT&& source, PRED prediDef) - : Raw{forward(source)} - , predicate_(prediDef) // induces a signature check - , cached_(false) // not yet cached - , isOK_(false) // not yet relevant - { } - - template - FilterCore (IT const& source, PRED prediDef) - : Raw{source} - , predicate_(prediDef) - , cached_(false) - , isOK_(false) - { } - }; - - - /** - * Iterator tool filtering pulled data according to a predicate - */ - template - class FilterIter - : public IterTool> - { - typedef FilterCore _Filter; - typedef IterTool<_Filter> _Impl; - - public: - static bool acceptAll(_Filter::Val) { return true; } - - - FilterIter () - : _Impl{FilterCore(IT(), acceptAll)} - { } - - template - FilterIter (IT const& src, PRED filterPredicate) - : _Impl{_Filter(src, filterPredicate)} - { } - - template - FilterIter (IT&& src, PRED filterPredicate) - : _Impl{_Filter(forward(src), filterPredicate)} - { } - - ENABLE_USE_IN_STD_RANGE_FOR_LOOPS (FilterIter) - }; - - - /** Build a FilterIter: convenience free function shortcut, - * picking up the involved types automatically. - * @param filterPredicate to be invoked for each source element - * @return Iterator filtering contents of the source - */ - template + template inline auto - filterIterator (IT const& src, PRED filterPredicate) + filterIter (IT&& src, FUN&& filterPredicate) { - return FilterIter{src, filterPredicate}; - } - - template - inline auto - filterIterator (IT&& src, PRED filterPredicate) - { - using SrcIT = std::remove_reference::type; - return FilterIter{forward(src), filterPredicate}; + using SrcIT = std::decay_t; + using Predi = std::decay_t; + return iter::Filter{forward (src) + ,forward(filterPredicate)}; } /** - * Additional capabilities for FilterIter, - * allowing to extend the filter condition underway. - * This wrapper enables remoulding of the filer functor - * while in the middle of iteration. When the filter is - * modified, current head of iteration gets re-evaluated - * and possible fast-forwarded to the next element - * satisfying the now extended filter condition. - * @note changing the condition modifies a given iterator in place. - * Superficially this might look as if the storage remains - * the same, but in fact we're adding a lambda closure, - * which the runtime usually allocates on the heap, - * holding the previous functor and a second functor - * for the added clause. - * @warning the addition of disjunctive and negated clauses might - * actually weaken the filter condition. Yet still there is - * _no reset of the source iterator,_ i.e. we don't - * re-evaluate from start, but just from current head. - * Which means we might miss elements in the already consumed - * part of the source sequence, which theoretically would - * pass the now altered filter condition. - * @see IterTools_test::verify_filterExtension + * Build filtering adapter that removes + * consecutive identical values emitted by source iterator */ template - class ExtensibleFilterIter - : public FilterIter - { - using _Filter = FilterCore; - using Val = _Filter::Val; - - void - reEvaluate() - { - this->core_.cached_ = false; - this->hasData(); // re-evaluate head element - } - - public: - ExtensibleFilterIter() { } - - template - ExtensibleFilterIter (IT&& src, PRED initialFilterPredicate) - : FilterIter{forward(src), initialFilterPredicate} - { } - template - ExtensibleFilterIter (IT const& src, PRED initialFilterPredicate) - : FilterIter{src, initialFilterPredicate} - { } - - ExtensibleFilterIter (IT&& src) - : ExtensibleFilterIter{forward(src), FilterIter::acceptAll} - { } - - // standard copy operations acceptable - - - /** access the unfiltered source iterator - * in current state */ - IT& - underlying() - { - return this->core_.source_; - } - - - template - ExtensibleFilterIter& - andFilter (COND conjunctiveClause) - { - function& filter = this->core_.predicate_; - - filter = [=](Val val) - { - return filter(val) - and conjunctiveClause(val); - }; - reEvaluate(); - return *this; - } - - template - ExtensibleFilterIter& - andNotFilter (COND conjunctiveClause) - { - function& filter = this->core_.predicate_; - - filter = [=](Val val) - { - return filter(val) - and not conjunctiveClause(val); - }; - reEvaluate(); - return *this; - } - - template - ExtensibleFilterIter& - orFilter (COND disjunctiveClause) - { - function& filter = this->core_.predicate_; - - filter = [=](Val val) - { - return filter(val) - or disjunctiveClause(val); - }; - reEvaluate(); - return *this; - } - - template - ExtensibleFilterIter& - orNotFilter (COND disjunctiveClause) - { - function& filter = this->core_.predicate_; - - filter = [=](Val val) - { - return filter(val) - or not disjunctiveClause(val); - }; - reEvaluate(); - return *this; - } - - - template - ExtensibleFilterIter& - setNewFilter (COND entirelyDifferentPredicate) - { - this->core_.predicate_ = entirelyDifferentPredicate; - reEvaluate(); - return *this; - } - - ExtensibleFilterIter& - flipFilter () - { - function& filter = this->core_.predicate_; - - filter = [=](Val val) - { - return not filter(val); - }; - reEvaluate(); - return *this; - } - }; - - - - /** - * Helper: predicate returning `true` - * whenever the argument value changes - * during a sequence of invocations. - */ - template - class SkipRepetition - { - using Item = wrapper::ItemWrapper; - - Item prev_; - - public: - bool - operator() (VAL const& elm) - { - if (prev_ and - (*prev_ == elm)) - return false; - - // element differs from predecessor - prev_ = elm; - return true; - } - - using result_type = bool; - }; - - - - - - - - + inline auto + filterRepeat (IT&& source) + { + using Val = meta::ValueTypeBinding::value_type; + return filterIter (forward (source) + ,iter::ChangeDetector{} ); + } /** - * Implementation of a _singleton value_ holder, - * which discards the contained value once "iterated" - */ - template - class SingleValCore - { - using Item = wrapper::ItemWrapper; - - Item theValue_; - - public: - SingleValCore() { } ///< passive and empty - - SingleValCore (VAL&& something) - : theValue_{forward (something)} - { } - - Item const& - pipe () const - { - return theValue_; - } - - void - advance () - { - theValue_.reset(); - } - - bool - evaluate () const - { - return theValue_.isValid(); - } - - using pointer = std::remove_reference_t *; - using reference = std::remove_reference_t &; - using value_type = std::remove_reference_t ; - }; - - - /** - * Pseudo-Iterator to yield just a single value. - * When incremented, the value is destroyed and - * the Iterator transitions to _exhausted state_. - * @remark as such might look nonsensical, but proves - * useful when a function yields an iterator, while - * producing an explicit value in some special case. - * @tparam VAL anything, value or reference to store - */ - template - class SingleValIter - : public IterTool> - { - using _ValHolder = SingleValCore; - using _IteratorImpl = IterTool<_ValHolder> ; - - public: - SingleValIter () - : _IteratorImpl{_ValHolder{}} - { } - - SingleValIter (VAL&& something) - : _IteratorImpl{_ValHolder{forward(something)}} - { } - - ENABLE_USE_IN_STD_RANGE_FOR_LOOPS (SingleValIter) - }; - - - - /** Build a SingleValIter: convenience free function shortcut, - * to pick up just any value and wrap it as Lumiera Forward Iterator. - * @return Iterator to yield the value once - * @warning be sure to understand that we literally pick up and wrap anything - * provided as argument. If you pass a reference, we wrap a reference. - * If you want to wrap a copy, you have to do the copy yourself inline + * A »singleton« iterator that yields a single value and is empty after iteration. + * @warning Be sure to understand that we literally pick up and wrap anything + * provided as argument. If you pass a reference, we wrap a reference. + * If you want to wrap a copy, you have to create a copy value argument. */ template inline auto - singleValIterator (VAL&& something) + singletonIter (VAL&& something) { - return SingleValIter{forward(something)}; + return iter::Single{forward (something)}; } - /** not-anything-at-all iterator */ template inline auto - nilIterator() + nilIter() { - return SingleValIter(); + return iter::Single(); } - + + /** + * Build a transforming iterator that maps a function to the source. + * @tparam processingFun to be invoked for each source element + * @note both the source iterator and the transforming functor + * are embodied, together with a result cache buffer. + */ + template + inline auto + transformIter (IT&& src, FUN&& processingFun) + { + using SrcIT = std::decay_t; + using ProcF = std::decay_t; + return iter::Trans{forward (src) + ,forward(processingFun)}; + } /** - * Implementation of custom processing logic. - * This core stores a function object instance - * to treat each source element pulled. + * Build an adapter that wraps the given iterator + * and returns by-value, taking a copy from each result. */ - template - class TransformingCore - { - using InType = iter::Yield; - using Item = wrapper::ItemWrapper; - - function trafo_; - - IT source_; - Item treated_; - - void - processItem () - { - if (source_) - treated_ = trafo_(*source_); - else - treated_.reset(); - } - - - - public: - TransformingCore () ///< deactivated core - : trafo_() - , source_() - , treated_() - { } - - template - TransformingCore (IT&& orig, FUN processor) - : trafo_(processor) // induces a signature check - , source_(forward (orig)) - { - processItem(); - } - - template - TransformingCore (IT const& orig, FUN processor) - : trafo_(processor) // induces a signature check - , source_(orig) - { - processItem(); - } - - Item const& - pipe () const - { - return treated_; - } - - void - advance () - { - ++source_; - processItem(); - } - - bool - evaluate () const - { - return bool(source_); - } - - using pointer = ValueTypeBinding::pointer; - using reference = ValueTypeBinding::reference; - using value_type = ValueTypeBinding::value_type; - }; - + template + auto + copyIt (IT&& iterator) + { + using Iter = meta::RefTraits::value_type; + using CopyAdapter = iter::ValueAdapter; + return CopyAdapter{std::forward (iterator)}; + } /** - * Iterator tool treating pulled data by a custom transformation (function) - * @tparam IT source iterator - * @tparam VAL result (output) type + * Build an adapter that wraps the given iterator + * and returns by-value, thereby **forcibly moving** the result. + * @warning the resulting behaviour breaks the usual assumptions + * regarding iterators, since the first access **consumes** the result. + * Use with extreme care and be sure to understand all ramifications; + * the forced move might even disrupt the underlying source. */ - template - class TransformIter - : public IterTool> - { - using _Trafo = TransformingCore; - using _IteratorImpl = IterTool<_Trafo> ; - - public: - TransformIter () - : _IteratorImpl(_Trafo()) - { } - - template - TransformIter (IT&& src, FUN trafoFunc) - : _IteratorImpl{_Trafo(forward(src), trafoFunc)} - { } - template - TransformIter (IT const& src, FUN trafoFunc) - : _IteratorImpl{_Trafo(src, trafoFunc)} - { } - - ENABLE_USE_IN_STD_RANGE_FOR_LOOPS (TransformIter) - }; - - - - - - /** Build a TransformIter: convenience free function shortcut, - * picking up the involved types automatically. - * @tparam processingFunc to be invoked for each source element - * @return Iterator processing the source feed - */ - template - inline auto - transformIterator (IT const& src, FUN processingFunc) + template + auto + moveIt (IT&& iterator) { - using OutVal = lib::meta::_Fun::Ret; - return TransformIter{src,processingFunc}; + using Iter = meta::RefTraits::value_type; + using ForceMoveAdapter = iter::ValueAdapter; + return ForceMoveAdapter{std::forward (iterator)}; } - template - inline auto - transformIterator (IT&& src, FUN processingFunc) - { - using SrcIT = std::remove_reference::type; - using OutVal = lib::meta::_Fun::Ret; - return TransformIter{forward(src), processingFunc}; - } @@ -813,12 +575,11 @@ namespace lib { container.push_back (*iter); } - template - inline IT::value_type + inline meta::ValueTypeBinding::value_type pull_last (IT iter) { - using Val = IT::value_type; + using Val = meta::ValueTypeBinding::value_type; using Item = wrapper::ItemWrapper; Item lastElm; @@ -838,28 +599,5 @@ namespace lib { } - - /** filters away repeated values - * emitted by source iterator */ - template - inline auto - filterRepetitions (IT const& source) - { - using Val = meta::ValueTypeBinding::value_type; - return filterIterator (source, SkipRepetition()); - } - - template - inline auto - filterRepetitions (IT&& source) - { - using Val = meta::ValueTypeBinding::value_type; - return filterIterator (forward(source), SkipRepetition() ); - } - - - - - } // namespace lib #endif diff --git a/src/lib/lifecycle.cpp b/src/lib/lifecycle.cpp index 5d38e7090..d4de5b8a8 100644 --- a/src/lib/lifecycle.cpp +++ b/src/lib/lifecycle.cpp @@ -18,14 +18,14 @@ #include "lib/error.hpp" -#include "include/lifecycle.h" +#include "include/lifecycle.hpp" #include "lib/lifecycleregistry.hpp" #include "lib/util.hpp" -namespace lumiera { +namespace vessel { // ==== implementation Lifecycle Registry ======= @@ -64,7 +64,6 @@ namespace lumiera { if (isNew and eventLabel == ON_BASIC_INIT) callbackFun(); // when this code executes, // then per definition we are already post "basic init" - // (which happens in the AppState ctor); thus fire it immediately } @@ -83,31 +82,4 @@ namespace lumiera { CStr ON_EMERGENCY ("ON_EMERGENCY"); -} // namespace lumiera - - -extern "C" { /* ==== implementation C interface for lifecycle hooks ======= */ - - - const char * lumiera_ON_BASIC_INIT = lumiera::ON_BASIC_INIT; - const char * lumiera_ON_GLOBAL_INIT = lumiera::ON_GLOBAL_INIT; - const char * lumiera_ON_GLOBAL_SHUTDOWN = lumiera::ON_GLOBAL_SHUTDOWN; - - const char * lumiera_ON_EMERGENCY = lumiera::ON_EMERGENCY; - - - - void - lumiera_LifecycleHook_add (const char* eventLabel, void callbackFun(void)) - { - lumiera::LifecycleHook (eventLabel, callbackFun); - } - - - void - lumiera_Lifecycle_trigger (const char* eventLabel) - { - lumiera::LifecycleRegistry::instance().execute (eventLabel); - } - -} +} // namespace vessel diff --git a/src/lib/lifecycleregistry.hpp b/src/lib/lifecycleregistry.hpp index 912aec541..5e5da74fb 100644 --- a/src/lib/lifecycleregistry.hpp +++ b/src/lib/lifecycleregistry.hpp @@ -14,7 +14,7 @@ /** @file lifecycleregistry.hpp ** Helper for registering lifecycle event callbacks, which are - ** provided as a global service by lumiera::AppState. This service + ** provided as an application service within the »vessel«. This service ** allows to enrol functions under a given label and then to call ** all those registered functions. ** @note this is in fact an event mechanism, and if we start using @@ -22,12 +22,12 @@ ** boost::signals. (which has the downside of being an binary ** dependency). ** - ** @see appstate.hpp + ** @see voyage.hpp */ -#ifndef LUMIERA_LIFECYCLEREGISTRY_H -#define LUMIERA_LIFECYCLEREGISTRY_H +#ifndef VESSEL_LIFECYCLE_REGISTRY_H +#define VESSEL_LIFECYCLE_REGISTRY_H #include "lib/util.hpp" @@ -39,7 +39,7 @@ #include -namespace lumiera { +namespace vessel { using util::contains; using std::function; @@ -50,7 +50,7 @@ namespace lumiera { /** * Registry of callback functions accessible by a label (ID) * provided at registration. Registered functions will be added - * to a list, which can be triggered via label. Used by AppState + * to a list, which can be triggered via label. Used by vessel::Voyage * to implement the lumiera lifecycle (init, shutdown) hooks. */ class LifecycleRegistry @@ -98,5 +98,5 @@ namespace lumiera { -} // namespace lumiera -#endif +} // namespace vessel +#endif /*VESSEL_LIFECYCLE_REGISTRY_H*/ diff --git a/src/lib/linked-elements.hpp b/src/lib/linked-elements.hpp index dbba6f767..9a2419446 100644 --- a/src/lib/linked-elements.hpp +++ b/src/lib/linked-elements.hpp @@ -32,7 +32,7 @@ ** under the assumption that the allocator is a _monostate_, and ** exploiting _empty-base-optimisation,_ footprint is minimal. ** @note this linked list container is _intrusive_ and thus needs the help - ** of the element type, which must *provide a pointer member* `next`. + ** of the element type, which must **provide a pointer member** `next`. ** Consequently, each such node element can not be member in ** several collections at the same time (unless they share all elements) ** @note as usual, any iterator relies on the continued existence and diff --git a/src/lib/live-mark.hpp b/src/lib/live-mark.hpp new file mode 100644 index 000000000..1ef70784f --- /dev/null +++ b/src/lib/live-mark.hpp @@ -0,0 +1,179 @@ +/* + LIVE-MARK.hpp - safety-tag to prevent invoking a deceased partner + + Copyright (C) + 2026, Hermann Vosseler + +  **Lumiera** is free software; you can redistribute it and/or modify it +  under the terms of the GNU General Public License as published by the +  Free Software Foundation; either version 2 of the License, or (at your +  option) any later version. See the file COPYING for further details. + +*/ + +/** @file live-mark.hpp + ** Memory marker tag to prevent segfault in communication with a defunct receiver. + ** The tag comprises an unique hash value, that can be embedded into another object + ** and tied to the lifecycle. This setup allows to build a safe-handle, that offers + ** a liveness check of the communication partner and prevents dereferencing and + ** invoking into the stale memory of an expired receiver. Notably this allows + ** to safeguard asynchronous communication between thread-local entities, + ** even when some of the participating threads terminates unexpectedly. + ** + ** @todo WIP 5/2026 developing a thread-local allocation pool + ** + ** @see engine-buffer-manager.hpp usage example + ** @see LiveMark_test + ** + */ + +#ifndef LIB_LIVE_MARK_H_ +#define LIB_LIVE_MARK_H_ + +#include "lib/error.hpp" +#include "lib/hash-combine.hpp" +#include "lib/util.hpp" + +#include +#include + +namespace lib { + namespace err = lumiera::error; + + using lib::HashVal; + + + class LiveMark; + + template + concept alive_marked = requires (SUBJ const& subject) + { + { subject.getLiveMark() } -> std::convertible_to; + }; + + + /** + * Liveness marker tag to be incorporated into another object, + * so that a communication link can be severed automatically when + * the receiver object goes out of scope. + * @see AllocReceiver + * @see LiveMark_test + */ + class LiveMark + { + std::atomic id_; + + public: + LiveMark() = default; + ~LiveMark() { disengage(); } ///< @note the actual protection happens here + + /** + * Create a LiveMark that is connected + * to the Subject to be protected. + */ + LiveMark (alive_marked auto& subject) + : id_{generateID (subject)} + { } + + // using default copy/assignment + + /** deactivate and mark as dead (irreversible) */ + void disengage() { id_.store (0, std::memory_order_relaxed); } + + + /** + * A smart-handle for safe protected access to some object + * that might go out of scope eventually. This works under the + * assumption that the target object embeds a LiveMark, which + * is expressed through the concept alive_marked. The marker + * HashVal is unique for each instance and distinct memory + * location, and thus even access to a new object instance + * with the same type, placed into the same memory location + * will be rejected, after the original target was destroyed. + */ + template + class SafeHandle + { + SUBJ * target_{nullptr}; + HashVal check_{0}; + + public: + SafeHandle() = default; + // default copy operations + + /** liveness check */ + operator bool() const + { + return bool(target_) + and check_ == retrieveID (*target_); + } + + /* ==== subject access ==== */ + + SUBJ& + operator*() const + { + if (not bool(*this)) + throw err::State{"Accessing defunct target" + , LERR_(BOTTOM_VALUE)}; + return * target_; + } + + SUBJ* + operator->() const + { + return & SafeHandle::operator*(); + } + + private: + SafeHandle (SUBJ& target) + : target_{& target} + , check_{retrieveID(target)} + { } + + friend class LiveMark; + + /** The heart of the implementation: peek into the + * target location to see if a valid LiveMark is embedded. + */ + static HashVal + retrieveID (SUBJ& target) + { + return target.getLiveMark().id_.load (std::memory_order_relaxed); + } + }; + + /** + * Create a safe smart-handle to refer to the given subject, + * similar to a raw pointer. Yet the handle will bool-check + * to `false` when the target object was destroyed + */ + template + SafeHandle + makeHandle (SUBJ& subject) + { + return SafeHandle{subject}; + } + + private: + /** Generate unique HashVal. + * @remark seeded by a nonce (counter) and the memory address, + * which makes it impossible to get the same hash at the same + * target address more than once (unless there is a collision + * in the 64bit hash space, which is extremely rare) + */ + template + static HashVal + generateID (X const& targetAddress) + { + static std::atomic_size_t nonce; + HashVal mark = nonce.fetch_add (1, std::memory_order_relaxed); + + hash::combine (mark, util::addrID (targetAddress)); + return mark; + } + }; + + +} // namespace lib +#endif /*LIB_LIVE_MARK_H_*/ diff --git a/src/lib/local-slice.hpp b/src/lib/local-slice.hpp new file mode 100644 index 000000000..7978a95b8 --- /dev/null +++ b/src/lib/local-slice.hpp @@ -0,0 +1,86 @@ +/* + LOCAL-SLICE.hpp - access front-end to thread-local instances + + Copyright (C) + 2026, Hermann Vosseler + +  **Lumiera** is free software; you can redistribute it and/or modify it +  under the terms of the GNU General Public License as published by the +  Free Software Foundation; either version 2 of the License, or (at your +  option) any later version. See the file COPYING for further details. + +*/ + +/** @file local-slice.hpp + ** Front-end to attach to a set of thread-local instances. + ** This setup helps to adapt a given service implementation to be deployed + ** as per-thread instances, operating in a massively concurrent situation. + ** The services installed this way must be able to work as self-contained + ** instances transparently, possibly cooperating with a central hub behind + ** the scenes, which however must be coordinated by the services themselves. + ** + ** Since thread-local storage is effectively static, the configuration of + ** instances happens at some point _prior_ to the first use from within a + ** thread, and otherwise not further specified. The LocalSlice access front-end + ** is a _monostate_ and all instances thus expose the same set of per-thread + ** service instances, which are created on-demand (by the C++ runtime system). + ** + ** @warning as of 2022 the implementation of `thread_local` in the GNU runtime + ** was observed to incur seemingly unnecessary overhead, which however + ** is relevant only in a high-performance environment. However, some + ** [casual benchmarks](\ref LocalSlice_test::investigatePerformance) + ** indicated that the problem does not seem to be _that serious_... + ** @see LocalSlice_test + ** @see LocalBufferStage (usage example) + ** @see buffer-provider-load-test.cpp + ** + */ + +#ifndef LIB_LOCAL_SLICE_H_ +#define LIB_LOCAL_SLICE_H_ + +#include "lib/integral.hpp" + + +namespace lib { + + /** Extension point for LocalSlice: Service initialisation */ + template + struct LocalSlice_DefaultSetup + { + using Service = SRV; + static Service init() { return Service(); } + }; + + + /** + * Common access point to thread-local service instances. + * @tparam CONF setup to initialise the service. + */ + template> + struct LocalSlice + { + LocalSlice() = default; + // using default copy/assignment + + + using Service = CONF::Service; + + /** service access */ + Service& + operator*() const + { + thread_local Service srvInstance{CONF::init()}; + return srvInstance; + } + + Service* + operator->() const + { + return & LocalSlice::operator*(); + } + }; + + +} // namespace lib +#endif /*LIB_LOCAL_SLICE_H_*/ diff --git a/src/lib/meta/common-result.hpp b/src/lib/meta/common-result.hpp new file mode 100644 index 000000000..318a240a0 --- /dev/null +++ b/src/lib/meta/common-result.hpp @@ -0,0 +1,236 @@ +/* + COMMON-RESULT.hpp - reconciled return types + + Copyright (C) + 2024,2026 Hermann Vosseler + +  **Lumiera** is free software; you can redistribute it and/or modify it +  under the terms of the GNU General Public License as published by the +  Free Software Foundation; either version 2 of the License, or (at your +  option) any later version. See the file COPYING for further details. + +*/ + +/** @file common-result.hpp + ** Determine a common result type for combined functors. + ** This header defines a metafunction to determine if, and how the return types + ** of several functors or accessors can be combined. This task becomes relevant + ** when building exploration strategies defined over a functional datastructure. + ** A notable example happens when performing a _tree expansion_ from within a + ** [iterator pipeline](\ref iter-explorer.hpp) or when mapping an aggregating + ** functor onto a [tuple of values](\ref tuple-concept.hpp). + ** + ** The basic technique to find such a _reconciled type_ is to rely on the + ** _ternary operator_ (`test? yes_expr : no_expr;`), which is specified to + ** great detail in the C++ standard. However, the STDLIB provides only two + ** tools tailored for specific cases: + ** - std::common_type is what you can use to _store_ such a reconciled result, + ** and includes _decaying_ the type. + ** - std::common_reference is oriented towards passing of references, + ** and includes elaborate mapping logic for mixture of RValues and LValues. + ** + ** So there is a need to expose the foundational logic for type reconciliation, + ** without any special adaptation rules. Furthermore, direct support for passing + ** the selection of types in the form of a [Type Sequence](\ref typelist.hpp) + ** is included. + ** + ** @see CommonResult_test + ** @see lib::iter_explorer::Expander + ** @see tuple-idx-adaptor.hpp + */ + + +#ifndef LIB_META_COMMON_RESULT_H +#define LIB_META_COMMON_RESULT_H + + +#include "lib/meta/trait.hpp" +#include "lib/meta/typelist.hpp" + + + +namespace lib { +namespace meta { + + namespace detail {// Implementation: reconcile N return types... + + /** @internal helper to find a reconciled result type. + * This is the technical part, and relies on a SFINAE technique: + * The test succeeds, if a ternary operator with both type results + * can be instantiated in the signature of a `check` template. + * + * Furthermore, the reconciliation for a pair of types can be + * generalised to N types by gradually reconciling the following + * types with the already established result. + * + * @remark 2026 the GNU Libstdc++ relies on precisely the same + * technique, which implies that this approach works reliably. + */ + template + class _ProbeCommonType + { + /******************************************************************************//** + * Core of the type reconciliation logic: determine the _common type_. + * The implementation relies on the **ternary operator** — the C++ language + * specifies a detailed and elaborate algorithm for this operator to reconcile + * the results of the "true" and "false" parts of `test? true_part : false_part`; + * such an expression compiles if and only if these criteria are met and a common + * type can be determined. The results of this procedure are quite solid, and + * satisfactory in most usage situations. A mixture of references, values const + * and volatile is handled properly. Only some rare corner cases are not covered, + * like upcasting two subclasses to a common interface (actually in some cases, + * involving multiple inheritance, a single authoritative solution for this + * special problem does not exist, and thus these cases were left out + * from the language spec). + */ + template + using Reconciled = decltype(true ? std::declval() : std::declval()); + + + /** Marker type: successful test */ + template + struct FoundResult + : std::true_type + { + using Type = RES; + }; + struct NoResultFound + : std::false_type + { }; + + + /** SFINAE setup to probe for a reconciled type */ + template> + static FoundResult> + check(int); + + template + static NoResultFound + check(...); + + /** perform the reconciliation test */ + using FirstResult = decltype(check(42)); + + + /** Helper: Join results of pairwise reconciliation + * to extend the result to a sequence of types */ + template + struct Join + { + using Result = FirstResult; + }; + + template + struct Join ///<@note matches when FirstResult was successful + { + using CT = FirstResult::Type; + using Result = _ProbeCommonType::Result; + }; + + + public: + /** Retrieve the overall result of this metafunction */ + using Result = Join::Result; + }; + + }//(End)implementation: type reconciliation + + + + /** + * Metafunction to find a _common result type_. + * @tparam TYPES a variadic sequence of types to be reconciled. + * Finding such a common result type can be relevant when results + * from several functors shall be combined, e.g. when mapping and + * aggregating results of a function application to a tuple. + * It is important to note that such a function may return arbitrary + * types, either by value or by reference, and possibly marked as const. + * + * \par Interface + * The `struct` CommonResult itself is a std::bool_constant that + * indicates if the given types can be reconciled. When successful, + * a nested typedef `Type` is present and aliased to a type that + * can represent all given result types as a function return. + * + * @remarks + * - the C++ standard offers similar functionality as + * + std::common_type + * + std::common_reference + * - however, the standard solution, while based on the same logic + * as this implementation, adds additional transformations, like + * decaying the type (for std::common_type) or handling various + * corner cases with RValue references. The specification of + * expected behaviour is quite elaborate for those cases + * - the implementation presented here is focused on _returning_ + * results from a function and thus relies _solely_ on the + * logic of the ternary operator (`? : `) + * - when several LValue-reference types are given, the result + * is only a reference when all results can be presented safely + * through such a reference; in most cases the result falls back + * to returning by-value. + * @note Concepts are provided to require that a selection of types + * is reconcilable, or even reconcilable into a reference type. + * @see CommonResult_test + */ + template + struct CommonResult + : detail::_ProbeCommonType::Result + { }; + + template + struct CommonResult + : CommonResult + { }; + + template<> + struct CommonResult<> + : std::false_type + { }; + + /** Specialisation to work transparently also on a type sequence, + * passed-in as a single template parameter. */ + template + struct CommonResult> + : CommonResult + { }; + + + + template + using CommonResult_t = CommonResult::Type; + + template + static constexpr bool has_CommonResult_v = CommonResult::value; + + + + template + concept reconcilable = requires + { + typename CommonResult::Type; + }; + + template + concept reconcilable_ref = reconcilable + and isLRef_v>; + + + + template + struct CommonResultTraits + { + static_assert (reconcilable, "unable to reconcile these types"); + + using CommonVal = std::common_type_t; + using ResType = CommonResult_t; + //NOTE: std::common_type decays (strips CV and ref) + static constexpr bool makeConst = isConst_v; + static constexpr bool exposeRef = isLRef_v; + }; + + + +}} // namespace lib::meta +#endif /*LIB_META_COMMON_RESULT_H*/ diff --git a/src/lib/meta/duck-detector.hpp b/src/lib/meta/duck-detector.hpp index 301279d4c..6f04e0bbe 100644 --- a/src/lib/meta/duck-detector.hpp +++ b/src/lib/meta/duck-detector.hpp @@ -108,13 +108,15 @@ * and check the static bool value field. * @warning none of these checks can detect private members */ -#define META_DETECT_NESTED(_TYPE_) \ - template \ - class HasNested_##_TYPE_ \ - { \ - \ - template \ - static Yes_t check(X::_TYPE_ *); \ +#define META_DETECT_NESTED(_TYPE_) \ + template \ + class HasNested_##_TYPE_ \ + { \ + \ + template \ + > \ + static Yes_t check(int); \ template \ static No_t check(...); \ \ diff --git a/src/lib/meta/fun-inline.hpp b/src/lib/meta/fun-inline.hpp new file mode 100644 index 000000000..bc3338c04 --- /dev/null +++ b/src/lib/meta/fun-inline.hpp @@ -0,0 +1,237 @@ +/* + FUN-INLINE.hpp - abstract functor with local delegate + + Copyright (C) + 2026 Hermann Vosseler + +  **Lumiera** is free software; you can redistribute it and/or modify it +  under the terms of the GNU General Public License as published by the +  Free Software Foundation; either version 2 of the License, or (at your +  option) any later version. See the file COPYING for further details. + +*/ + +/** @file fun-inline.hpp + ** A generic abstracted functor with the implementation stored locally. + ** This template is an implementation building block and allows to create + ** an abstraction barrier, similar to what std::function does. In contrast + ** to the latter however, FunInline places the actual implementation delegate + ** into an embedded opaque buffer, never using any heap memory. + ** + ** @todo WIP-WIP 4/2026 this is more like a conceptual draft, + ** not sure where this design exercise will lead to. + ** @warning after investigating the implementation technique to some degree, + ** it became clear that **copy support** is the greatest challenge. + ** All »the rest« might be surprisingly simple to implement, yet + ** the demand for a movable / copyable value-object effectively + ** translates into something like a VTable — and the result + ** would be similar to lib::PolymorphicValue — just with + ** some special binding to the function operator. + ** For this reason, FunInline is now declared to be util::NonCopyable + ** @todo 4/2026 I've finally got this draft hammered out. + ** It is far from what I wanted to achieve... + ** Notably a second function pointer is used + ** That is ugly, but it does the job. + ** + ** @see FunInline_test + ** + */ + + +#ifndef LIB_META_FUN_INLINE_H_ +#define LIB_META_FUN_INLINE_H_ + + +#include "lib/error.hpp" +#include "lib/nocopy.hpp" +#include "lib/meta/function.hpp" +#include "lib/meta/typeseq-util.hpp" +#include "lib/meta/variadic-helper.hpp" + +#include + + +namespace lib { +namespace meta{ + namespace err = lumiera::error; + + using std::forward; + + namespace test { class FunInline_test; } + + /** + * An opaque front-end for a functor stored inline. + * The type of the implementation delegate is erased + * from the type, retaining only the generic signature. + * @note this wrapper object is defined as NonCopyable, + * because anything else would require a VTable, + * and raise serious concerns regarding identity. + * @warning deliberately **not threadsafe** + * @see FunInline_test + */ + template + class FunInline + { + static_assert (!sizeof(SIG), "function signature required"); + }; + + + template + class FunInline + : util::NonCopyable + { + using _F = _Fun; + static_assert (_F{}, "function signature required"); + + using Sig = _F::Sig; + using Ret = _F::Ret; + using Args = _F::Args; + using RArgs = ElmTypes::template Apply; + using IArgs = Prepend::Seq; + using ISig = BuildFunType::Sig; + using Dtor = void(void*); + + /** virtualised delegation through function pointer(s) */ + ISig* invoker_{nullptr}; + Dtor* destroy_{nullptr}; + + /** opaque storage buffer to emplace the functor delegate */ + alignas(ALI) mutable + std::byte buf_[siz]; + + + /** + * The Invoker template is instantiated with the type of the concrete + * delegate functor (which resides in the opaque storage buffer). + * The purpose of this template is to cause generation of a static + * delegation function that can be invoked through the #invoker_ pointer. + * In the typical usage situation, the delegate type \a FUNC is implemented + * mostly with inline functions, and thus the bulk of the actual implementation + * code will be emitted into this static delegation function. The template + * \ref InvokerBuilder provides the blueprint for this code generation. + */ + template + struct Invoker + { + static Ret + dispatch (void* instance, ARGS&&...args) + { + return functor(instance) (forward (args)...); + } + + static void + destroy (void* instance) + { + if (instance) + functor(instance).~FUN(); + } + + static FUN& + functor (void* instance) + { + REQUIRE (instance); + return * static_cast (instance); + } + + using Sig = _F::Sig; + using ISig = decltype(dispatch); + }; + + /** allow the Unit test to poke into internals */ + friend class test::FunInline_test; + + + + public: + ~FunInline(){ destroy(); } + + FunInline() = default; + + template + FunInline (FUNC&& functor) + { + using Functor = std::decay_t; + emplace (forward (functor)); + activate(); + } + + // NonCopyable + + + operator bool() const + { + return bool(destroy_) and bool(invoker_); + } + + + /* == Functor interface == */ + + Ret + operator() (ARGS&&... args) + { + if (not invoker_) + throw err::State{"Attempt to invoke inactive function."}; + + return (*invoker_) (&buf_, forward (args)...); + } + + + private: /* == Instance management == */ + + void + destroy() + { + if (destroy_) + (*destroy_) (&buf_); + destroy_ = nullptr; + invoker_ = nullptr; + } + + template + void + activate() + { + ASSERT_VALID_SIGNATURE (FUNC, Sig); + REQUIRE (!destroy_ and !invoker_, "Lifecycle logic broken"); + + destroy_ = Invoker::destroy; + invoker_ = Invoker::dispatch; + } + + /** Abbreviation for placement new */ + template + TY& + emplace (INIT&& ...args) + { + static_assert (siz >= sizeof(TY), "Embedded storage buffer too small"); + static_assert (alignof(ALI) >= alignof(TY), "Alignment requirements not fulfilled"); + + destroy(); + try { + return *new(&buf_) TY {std::forward (args)...}; + } + catch (...) + { + ENSURE (not destroy_); + throw; + } + } + + /// @todo unused ... not required? + template + XXX& + accessAs() const + { + return * std::launder (reinterpret_cast (&buf_)); + } + }; + + +}}// namespace lib::meta +#endif /*LIB_META_FUN_INLINE_H_*/ diff --git a/src/lib/meta/fun-trampoline.hpp b/src/lib/meta/fun-trampoline.hpp new file mode 100644 index 000000000..7a7948426 --- /dev/null +++ b/src/lib/meta/fun-trampoline.hpp @@ -0,0 +1,125 @@ +/* + FUN-TRAMPOLINE.hpp - generate static functor table + + Copyright (C) + 2026 Hermann Vosseler + +  **Lumiera** is free software; you can redistribute it and/or modify it +  under the terms of the GNU General Public License as published by the +  Free Software Foundation; either version 2 of the License, or (at your +  option) any later version. See the file COPYING for further details. + +*/ + +/** @file fun-trampoline.hpp + ** Build a trampoline table with functors. + ** This Metaprogramming helper uses a _builder template_ to fill an embedded + ** static dispatcher table with generated functors. These can then be accessed + ** by index, typically to be invoked. + ** + ** The setup works entirely on a static level, so that any mention of some + ** FunTrampoline template instantiation causes the compiler to populate the + ** corresponding functor table. This arrangement can be used, for example, + ** to auto-generate an index accessor for a specific tuple, so that a + ** runtime subscript or iteration can be derived. Obviously doing so + ** requires the types in the tuple to be [reconcilable](\ref common-type.hpp), + ** so that a generic functor can be defined to handle all fields in the tuple + ** based on a common generic scheme. Without that, tuples can in general + ** only be iterated and transformed by compile-time metaprogramming. + ** + ** @see FunTrampoline_test + ** @see CommonResult_test + ** @see tuple-idx-adaptor.hpp + */ + + +#ifndef LIB_META_FUN_TRAMPOLINE_H +#define LIB_META_FUN_TRAMPOLINE_H + + +#include "include/logging.h" +#include "lib/meta/function.hpp" +#include "lib/meta/variadic-helper.hpp" + +#include +#include + + +namespace lib { +namespace meta { + + /** + * FunTrampoline relies on a helper template + * to be used as _trampoline builder_ to generate + * a sequence of functors that can be placed into a + * static dispatcher table. Thus, a trampoline_builder + * - must be default constructible + * - must define a Functor type... + * - that can be used as array element + * - and must provide a templated build() function + */ + template + concept trampoline_builder = requires(B builder) + { + typename B::Functor; + std::array{}; + requires std::default_initializable; + { builder.template build<0>() } -> std::convertible_to; + }; + + + /** + * A static functor table, populated automatically. + * @tparam BUILD helper Template to generate functors + * that are placed into a static table + * @tparam TSEQ sequence of types to instantiate + * the functors with, iteratively. + * @remarks + * - the given type sequence is handled with the + * ElmTypes helper template (see \ref variadic-helper.hpp) + * - the BUILD template is instantiated with that \a TSEQ + * - statically-for-each-index, the build() function + * on the builder is invoked, which is expected to produce + * an instance of BUILD::Functor + * @warning as soon as some instance of this FunTrampoline + * is used / referred, the compiler will emit the trampoline + * table into static storage; the resulting increase of the + * executable size is something to be aware of, when using + * this construct for transient processing, driven by + * actual argument types. + */ + template class BUILD, typeseq TSEQ> + requires trampoline_builder> + class FunTrampoline + { + using AllElm = ElmTypes; + using Indices = AllElm::Idx; + using Builder = BUILD; + using Functor = Builder::Functor; + using Table = std::array; + + template + static constexpr Table + buildFunctors (std::index_sequence) + { + return {Builder().template build()...}; + } + + static constexpr Table functors = buildFunctors (Indices{}); + + public: + static constexpr size_t size() { return functors.size(); } + static constexpr size_t memSiz() { return sizeof(functors);} + + static Functor const& + dispatch (size_t idx) + { + ENSURE (idx < functors.size()); + return functors[idx]; + } + + using ResType = _Fun::Ret; + }; + +}}// namespace lib::meta +#endif /*LIB_META_FUN_TRAMPOLINE_H*/ diff --git a/src/lib/meta/generator.hpp b/src/lib/meta/generator.hpp index 4585f396e..2cf37b381 100644 --- a/src/lib/meta/generator.hpp +++ b/src/lib/meta/generator.hpp @@ -41,7 +41,7 @@ This code is heavily inspired by ** instantiation is mixed or inherited into the resulting product. ** ** @see generator-test.cpp - ** @see lumiera::query::ConfigRules usage example + ** @see vessel::query::ConfigRules usage example ** @see typelist.hpp ** */ diff --git a/src/lib/meta/maybe-compare.hpp b/src/lib/meta/maybe-compare.hpp deleted file mode 100644 index dfe87c20c..000000000 --- a/src/lib/meta/maybe-compare.hpp +++ /dev/null @@ -1,56 +0,0 @@ -/* - MAYBE-COMPARE.hpp - guarded invocation of comparisons - - Copyright (C) - 2009, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -*/ - - -/** @file maybe-compare.hpp - ** helpers for fail-safe invocation of comparison operations from generic code. - */ - - -#ifndef LIB_META_MAYBE_COMPARE_H -#define LIB_META_MAYBE_COMPARE_H - - - - -namespace lib { -namespace meta{ - - - /** - * Trait template for invoking equality comparison. - * This allows to treat some types specifically.... - */ - template - struct Comparator - { - static bool - equals (X const& x1, X const& x2) - { - return x1 == x2; - } - }; - - /* == add specialisations here == */ - - - template - inline bool - equals_safeInvoke (X const& x1, X const& x2) - { - return Comparator::equals(x1,x2); - } - - -}} // namespace lib::meta -#endif diff --git a/src/lib/meta/prop-builder.hpp b/src/lib/meta/prop-builder.hpp new file mode 100644 index 000000000..4482bb33a --- /dev/null +++ b/src/lib/meta/prop-builder.hpp @@ -0,0 +1,147 @@ +/* + PROP-BUILDER.hpp - generate data record with arbitrary named fields + + Copyright (C) + 2026, Hermann Vosseler + +  **Lumiera** is free software; you can redistribute it and/or modify it +  under the terms of the GNU General Public License as published by the +  Free Software Foundation; either version 2 of the License, or (at your +  option) any later version. See the file COPYING for further details. + +*/ + + +/** @file prop-builder.hpp + ** A builder for compile-time generation of data records with arbitrary named fields. + ** Generally speaking, a "record" in this context means just a struct or class with some + ** data fields, and can be easily defined to suit any needs. However, such a class definition + ** always incurs some _syntactical overhead_ — which is fine for spelling out some type in + ** general, but can be problematic when the data record is only used once, locally and + ** is without further significance beyond that single usage. Such a situation often + ** arises in the context of _policy based design_ and flexible configuration of + ** some generic facility, especially when part of this configuration involves + ** the definition of _callback functions._ + ** + ** A common requirement that lends itself well to policy based design arises when such callbacks + ** establish a deeply interwoven link into local implementation details; this kind of interconnected + ** implementation can be expressed succinctly through Lambda functions with contextual capture, and + ** allows to setup a tight interconnection between implementation back-ends, without exposing any + ** of these implementation details at the interface / dependency-injection level. However, the task + ** to package several such specific callbacks into a policy instance and pass this configured policy + ** for creation of service instances, can be surprisingly complex, verbose, confusing and leads + ** to code that is hard to maintain. The reasons for this challenge are manifold: + ** - contextual capture works only well from directly within an implementation function + ** - such a setup can not be expressed well through types and interfaces, since the + ** generated closure is _essentially a value_ and its type can not be spelled out. + ** - a policy with several callbacks thus turns into a template with several type parameters, + ** and must be initialised through a likewise template based constructor; the resulting code + ** tends to become very technical, verbose, surprisingly tricky and hard to decipher for + ** any reader not familiar with all the intricate details of the implementation. + ** + ** The root of these difficulties of proper formulation seems to be related to the nature + ** of cohesive, closely interwoven implementation code. As long as this kind of structure + ** can be kept close together, it works surprisingly well and expresses intent clearly. + ** Yet as soon as it must be taken apart, expressed in names, types and interfaces, + ** a high conceptual burden is added, which leads to the danger of generalisation + ** and re-use beyond the very specific case, to which the original formulation + ** of the problem was well adapted and adequate. + ** + ** The solution promoted here thus aims to remove the need to define any type and interface + ** explicitly; a builder-expression is created rather, which allows to bind all callbacks + ** and further parameters directly inline, within a single constructor invocation. + ** Such a setup becomes possibly through a combination of some C++20 features, linked + ** together by lambda expressions generated from a _macro_. + ** - the PropBuilder acts as base class for the generated records, and is used to + ** initiate the build-up of the definition expression. + ** - several »layers« of derived classes are added step by step, where each such + ** layer adds one further _property_ attribute, that is initialised by a given value. + ** - the _junctor_ to combine those layer is provided by the template function + ** `PropBuilder::define(builder-λ, value)`. It uses the new _explicit object_ + ** syntax introduced with C++20, to pick up the "self type" as template argument. + ** - by invoking this builder function, the current "this"-object is _moved_ into + ** the builder, where it is used to create and initialise a derived class. + ** - for each »layer« a specific lambda is generated by a macro — and it is this + ** technique that allows to accept a lexical token form the program text and + ** use this token as field name in the next derived class to generate. + ** @see PropBuilder_test for detailed explanation + ** @see OutputProxyProvider_test for a usage example + */ + + +#ifndef LIB_META_PROP_BUILDER_H +#define LIB_META_PROP_BUILDER_H + + +#include "lib/meta/trait.hpp" + +#include + + +namespace lib { +namespace meta{ + + using std::move; + using std::forward; + + + /** + * Anchor record and builder mechanics for generating data records. + * - start a builder expression by constructing an anonymous `PropBulder()` + * - invoke PropBuilder::define() for each further property to add + * - pass a builder-λ as first argument, and the data value as second + * - generate a suitable builder-λ with the macro `PROP_FIELD(fieldname)` + * - overall, the builder expression generates a RValue, that will usually + * be passed directly to the constructor of another class. + * @see PropBuilder_test::simpleUsage + */ + struct PropBuilder + { + /** + * Core builder operation: invoke a builder-λ to extend »this« + * @param layerBuilder a specially crafted invokable, that takes _this object_ + * and the given property value and creates an instance of a derived class. + * @note the macro \ref PROP_FIELD below will generate a suitable argument. + * @param val an arbitrary (movable) data element to be stored as property. + * @remark C++20 introduced the syntax for _explicit object member functions_, + * which allow to bind the "self" object instance to an explicitly defined + * first function parameter. Since this builder operation will be invoked + * on the generated compound object, the template parameter \a BAS will + * denote this compound, and the next builder-λ will derive from it. + */ + template + auto + define (this BAS&& chain, LAY const& layerBuilder, VAL&& val) + { + return layerBuilder (move(chain), forward(val)); + } + }; + + +#define PROP_FIELD(FIELD_NAME) \ + [] \ + (BAS&& bas, VAL&& val) \ + { \ + using BaseLayer = std::decay_t;\ + using Property = std::decay_t; \ + \ + struct DataLayer_ ## FIELD_NAME \ + : BaseLayer \ + { \ + Property FIELD_NAME; \ + \ + DataLayer_ ## FIELD_NAME (BaseLayer b \ + ,Property v) \ + : BaseLayer{move(b)} \ + ,FIELD_NAME{move(v)} \ + { } \ + }; \ + return DataLayer_ ## FIELD_NAME {forward(bas)\ + ,forward(val) \ + }; \ + } + + + +}} // namespace lib::meta +#endif /*LIB_META_PROP_BUILDER_H*/ diff --git a/src/lib/meta/trait.hpp b/src/lib/meta/trait.hpp index 7ab1ff8a6..cb58b4a98 100644 --- a/src/lib/meta/trait.hpp +++ b/src/lib/meta/trait.hpp @@ -57,9 +57,6 @@ //Forward declarations for the Unwrap helper.... -namespace boost{ - template class reference_wrapper; -} namespace std { template class reference_wrapper; template class shared_ptr; @@ -90,9 +87,7 @@ namespace meta { using std::remove_pointer_t; using std::remove_reference_t; using std::conditional_t; - using std::is_reference_v; - using std::is_lvalue_reference_v; - using std::is_rvalue_reference_v; + using std::is_pointer_v; using std::is_pointer; using std::is_base_of; using std::is_convertible; @@ -108,13 +103,22 @@ namespace meta { using std::__or_; template - static constexpr bool isConst_v = std::is_const_v>; + using isConst = std::is_const>; template - static constexpr bool isLRef_v = std::is_lvalue_reference_v; + using isLRef = std::is_lvalue_reference; template - static constexpr bool isRRef_v = std::is_rvalue_reference_v; + using isRRef = std::is_rvalue_reference; template - static constexpr bool isRef_v = std::is_reference_v; + using isRef = std::is_reference; + + template + static constexpr bool isConst_v = isConst::value; + template + static constexpr bool isLRef_v = isLRef::value; + template + static constexpr bool isRRef_v = isRRef::value; + template + static constexpr bool isRef_v = isRef::value; /** @@ -159,19 +163,6 @@ namespace meta { } }; - template - struct Unwrap> - : std::true_type - { - using Type = X; - - static X& - extract (boost::reference_wrapper wrapped) - { - return wrapped; - } - }; - template struct Unwrap> : std::true_type @@ -249,8 +240,8 @@ namespace meta { template struct Strip { - using TypeUnconst = conditional_t - , conditional_t + using TypeUnconst = conditional_t + , conditional_t , remove_cv_t> && , remove_cv_t> & > , remove_cv_t>; @@ -274,33 +265,33 @@ namespace meta { template struct RefTraits { - typedef TY Value; - typedef TY* Pointer; - typedef TY& Reference; + typedef TY value_type; + typedef TY& reference; + typedef TY* pointer; }; template struct RefTraits { - typedef TY* Value; - typedef TY** Pointer; - typedef TY*& Reference; + typedef TY* value_type; + typedef TY*& reference; + typedef TY** pointer; }; template struct RefTraits { - typedef TY Value; - typedef TY* Pointer; - typedef TY& Reference; + typedef TY value_type; + typedef TY& reference; + typedef TY* pointer; }; template struct RefTraits { - typedef TY Value; - typedef TY* Pointer; - typedef TY& Reference; + typedef TY value_type; + typedef TY& reference; + typedef TY* pointer; }; @@ -328,7 +319,11 @@ namespace meta { , is_same > { }; - + + template + inline constexpr bool is_Subclass_v = is_Subclass(); + + /** compare for unadorned base type, disregarding const and references */ template struct is_basically @@ -339,8 +334,8 @@ namespace meta { /** verify the first (special) type can stand-in for the second */ template struct can_StandIn - : std::is_convertible::Reference - ,typename RefTraits::Reference + : std::is_convertible::reference + ,typename RefTraits::reference > { }; @@ -505,24 +500,36 @@ namespace meta { * This is just a heuristic, based on some common properties * of such iterators; it is enough to distinguish it from an * STL container, but can certainly be refined. + * @note deliberately a Lumiera iterator does _not fulfil_ + * the concept std::input_iterator. And vice versa: + * a STL iterator lacks the `bool` check for iteration end. + * @see IterableClassification_test */ template - class can_IterForEach + class can_LumieraIter { - using Type = Strip::Type; + using Type = Strip::TypeReferred; META_DETECT_NESTED(value_type); + META_DETECT_NESTED(reference); META_DETECT_OPERATOR_DEREF(); META_DETECT_OPERATOR_INC(); public: enum{ value = std::is_constructible::value and HasNested_value_type::value + and HasNested_reference::value and HasOperator_deref::value and HasOperator_inc::value }; }; + template + constexpr bool is_LumieraIter_v = meta::can_LumieraIter::value; + + template + concept lum_iter = is_LumieraIter_v; + /** Trait template to detect a type exposing a »state core« API. @@ -530,10 +537,10 @@ namespace meta { * with the help of lib::IterStateWrapper or lib::IterableDecorator. * This check is heuristic, based on the presence of function names. */ - template + template class is_StateCore { - using Type = Strip::Type; + using Type = Strip::TypeReferred; META_DETECT_FUNCTION_ARGLESS(checkPoint); META_DETECT_FUNCTION_ARGLESS(iterNext); @@ -546,15 +553,24 @@ namespace meta { }; }; + template + constexpr bool is_StateCore_v = is_StateCore::value; + + template + concept state_core = is_StateCore_v; + /** Trait template to detect a type usable with the STL for-each loop. * Basically we're looking for the functions to get the begin/end iterator + * @remark many Lumiera Iterators define those additional functions any + * can thus be used in a for-each loop, since a default-constructed + * Lumiera Iterator can act as an "end" sentinel. */ template class can_STL_ForEach { - using Type = Strip::Type; + using Type = Strip::TypeReferred; struct is_iterable { @@ -618,7 +634,7 @@ namespace meta { template class can_STL_backIteration { - using Type = Strip::Type; + using Type = Strip::TypeReferred; struct is_backIterable { diff --git a/src/lib/meta/tuple-concept.hpp b/src/lib/meta/tuple-concept.hpp new file mode 100644 index 000000000..207e07b2f --- /dev/null +++ b/src/lib/meta/tuple-concept.hpp @@ -0,0 +1,305 @@ +/* + TUPLE-CONCEPT.hpp - metaprogramming support for tuple-like data types + + Copyright (C) + 2025, Hermann Vosseler + +  **Lumiera** is free software; you can redistribute it and/or modify it +  under the terms of the GNU General Public License as published by the +  Free Software Foundation; either version 2 of the License, or (at your +  option) any later version. See the file COPYING for further details. + +*/ + + +/** @file tuple-concept.hpp + ** Metaprogramming with tuples-of-types and the `std::tuple` record. + ** This header complements typelist.hpp and provides a bridge from type sequences + ** to the tuple type provided by the standard library, including traits and + ** helpers to build tuple types from metaprogramming and to pretty-print tuples. + ** + ** Notably, a `concept tuple_like` is provided here, which is satisfied for any type + ** in compliance with the »[tuple protocol]«. This solution reaches beyond the scope + ** of the C++ standard — and in fact opposes the decision of the C++ Committee + ** _not to provide such a concept_, as this can not be defined in full genericity + ** for any type that can be handled by the _structural bindings of the C++ language. + ** + ** Since the Lumiera support library is part of a specific application framework, and + ** does not intend to be a general purpose library, we prefer a partially complete + ** definition that allows us to subsume and handle custom types under a »Tuple« concept. + ** @note Unfortunately the C++ committee did not only decline to provide a tuple_like concept, + ** rather they choose to lock-down essential parts of the accompanying library to their + ** internal and hard-wired selection of what is considered "tuple-like". This forces us + ** to redefine and replace also the crucial function `std::apply`, so that a generic + ** function can be applied to any _tuple-like_ entity. + ** @see TupleHelper_test + ** @see tuple-helper.hpp extended tuple support functions + ** [tuple protocol]: https://en.cppreference.com/w/cpp/language/structured_binding.html#Case_2:_binding_a_type_implementing_the_tuple_operations + */ + + +#ifndef LIB_META_TUPLE_CONCEPT_H +#define LIB_META_TUPLE_CONCEPT_H + +#include "lib/meta/util.hpp" +#include "lib/meta/variadic-helper.hpp" + +#include +#include +#include + + +namespace lib { +namespace meta { + + using std::remove_cv_t; + using std::is_reference_v; + using std::remove_reference_t; + + + /** @internal building-block: a type supporting the `tuple_size` metafunction */ + template + concept tuple_sized = requires + { + { std::tuple_size::value } -> std::convertible_to; + }; + + + /** @internal building-block: a type where elements can be accessed through a `get` friend function */ + template + concept tuple_adl_accessible = requires(TUP tup) + { + typename std::tuple_element_t; + { get(tup) } -> std::convertible_to&>; + }; + + /** @internal building-block: a type where elements can be accessed through a `get` member function */ + template + concept tuple_mem_accessible = requires(TUP tup) + { + typename std::tuple_element_t; + { tup.template get() } -> std::convertible_to&>; + }; + + template + concept tuple_element_accessible = tuple_mem_accessible or tuple_adl_accessible; + + template + concept tuple_accessible = + tuple_sized and + WithIdxSeq>::andAll([](auto idx) + { + return tuple_element_accessible; + }); + + + /** + * Concept to mark any type compliant to the »tuple protocol« + * @note a type satisfying this concept can participate in _structured bindings_, + * yet not every type that can be used in such bindings will fulfil this concept, + * since structured bindings can also be applied to _some POD like_ data records. + */ + template + concept tuple_like = not is_reference_v + and tuple_sized> + and tuple_accessible>; + + + + /** + * Concept to require an instance of std::tuple` + * @note other than `concept` \ref tuple_like, this aims + * explicitly at the implementation form STDLIB. + */ + template + concept tup = requires (TUP tup) + { + // check that a function can be invoked with a std::tuple + [](std::tuple){/*placeholder*/}(tup); + }; + + template + struct is_Tuple : std::bool_constant> { }; + + template + static constexpr bool is_Tuple_v = is_Tuple::value; + + + + + /** + * Helper for abstracted / unified access to member elements of any _tuple-like_ + * @remark preferably uses a `get` member function, falling back to a + * free function `get`, which is found by ADL. + */ + template + requires tuple_like> + decltype(auto) + getElm (TUP&& tup) + { + using Tup = std::remove_reference_t; + static_assert (0 < std::tuple_size_v); + if constexpr (tuple_mem_accessible) + { + if constexpr (std::is_reference_v) + return tup.template get(); + else + { // return value copy when tuple given as RValue + using Elm = std::tuple_element_t; + Elm elm(tup.template get()); + return elm; + } + } + else + { // ▽▽▽ ADL + using std::get; + return get (std::forward (tup)); + } + } + + + + + /** + * Specialisation of variadic access for any tuple-like + * @see variadic-helper.hpp + */ + template + struct ElmTypes + { + template + struct Extract; + template + struct Extract> + { + using ElmTypes = Types ...>; + }; + + static constexpr size_t SIZ = std::tuple_size_v; + + using Idx = std::make_index_sequence; + using Seq = Extract::ElmTypes; + using Tup = RebindVariadic::Type; + + template class META> + using Apply = ElmTypes::template Apply; + + template class O> + using Rebind = RebindVariadic::Type; + + template class PRED> + using AndAll = ElmTypes>::template Rebind; + + template class PRED> + using OrAny = ElmTypes>::template Rebind; + }; + + + + + + + + namespace { // apply to tuple-like : helpers... + + /** @internal invocation helper similar to C++17 — but preferring a custom `get` impl */ + template + constexpr decltype(auto) + __unpack_and_apply (FUN&& f, TUP&& tup, std::index_sequence) + { + return std::invoke (std::forward (f) + ,getElm (std::forward(tup))... + ); + } + + + /** @internal invoke a metafunction with \a FUN and all element types from the _tuple-like_ \a TUP */ + template class META, class FUN, class TUP> + struct _InvokeMetafunTup + { + using Tupl = std::decay_t; + using Elms = ElmTypes::Seq; + using Args = Prepend::Seq; + using Type = RebindVariadic::Type; + }; + template class META, class FUN, class TUP> + struct _InvokeMetafunTup + { + using Tupl = std::decay_t; + using Elms = typename ElmTypes::template Apply; + using Args = Prepend::Seq; + using Type = RebindVariadic::Type; + }; + + template + inline constexpr bool can_nothrow_invoke_tup = _InvokeMetafunTup::Type::value; + } + + /** + * Replacement for `std::apply` — yet applicable to _tuple-like custom types_. + * For unclear reasons, the standard chooses to reject such custom types, and + * only allows a fixed set of explicitly defined facilities from the Stdlib + * (tuple, pair, array, and some ranges stuff). + * @todo 6/2025 as a first step, this replicates the implementation from C++17; + * the second step would be to constrain this to a concept `tuple_like` + */ + template requires tuple_like> + constexpr decltype(auto) + apply (FUN&& f, TUP&& tup) noexcept (can_nothrow_invoke_tup ) + { + using Indices = std::make_index_sequence>>; + + return __unpack_and_apply (std::forward (f) + ,std::forward (tup) + ,Indices{} + ); + } + + + /** + * Tuple iteration: perform some arbitrary operation on each element of a tuple. + * @note the given functor must be generic, since each position of the tuple + * may hold a data element of different type. + * @remark credits to David Vandevoorde (member of C++ committee) for using + * std::apply to unpack the tuple's contents into an argument pack and + * then employ a fold expression with the comma operator. + */ + template requires tuple_like> + constexpr void + forEach (TUP&& tuple, FUN fun) + { + lib::meta::apply ([&fun](ELMS&&... elms) + { + (fun (std::forward(elms)), ...); + } + ,std::forward (tuple)); + } + + /** + * Apply some arbitrary function onto all elements of a tuple. + * @return a new tuple constructed from the results of this function + * @note the functor must be generic and able to work with all element types + * @warning pay attention to references; the function can take arguments by-ref + * and manipulate them (side-effect), and it may return references, + * which will be placed as such into the result tuple; furthermore, + * the argument tuple can also be taken as reference... + * @remark The tuple constructor invocation is preceded by a regular argument evaluation, + * which has unspecified evaluation order (even in C++17); _no assumptions_ can be + * made regarding the order the functor will see the source tuple elements. + * Notably this differs from #forEach, where a fold-expression with comma-operator + * is used, which is guaranteed to evaluate from left to right. + */ + template requires tuple_like> + constexpr auto + mapEach (TUP&& tuple, FUN fun) + { + return lib::meta::apply ([&fun](ELMS&&... elms) + { //..construct the type explicitly (make_tuple would decay fun result types) + using Tuple = std::tuple(elms))) ...>; + return Tuple (fun (std::forward(elms)) ...); + } + ,std::forward (tuple)); + } + +}} // namespace lib::meta +#endif /*LIB_META_TUPLE_CONCEPT_H*/ diff --git a/src/lib/meta/tuple-helper.hpp b/src/lib/meta/tuple-helper.hpp index 44d89fc5c..01b9f0c67 100644 --- a/src/lib/meta/tuple-helper.hpp +++ b/src/lib/meta/tuple-helper.hpp @@ -2,7 +2,7 @@ TUPLE-HELPER.hpp - metaprogramming utilities for type and data tuples Copyright (C) - 2016, Hermann Vosseler + 2016,2025 Hermann Vosseler   **Lumiera** is free software; you can redistribute it and/or modify it   under the terms of the GNU General Public License as published by the @@ -18,17 +18,38 @@ ** to the tuple type provided by the standard library, including traits and ** helpers to build tuple types from metaprogramming and to pretty-print tuples. ** - ** Notably, a `concept tuple_like` is provided, which is satisfied for any type in compliance - ** with the »tuple protocol«. Together with a [generic accessor][\ref lib::meta::getElm], - ** this allows to handle all _tuple-like_ types uniformly. - ** @note Due to an unfortunate limitation of the standard, we're forced to provide our own alternative - ** implementation to replace `std::apply`, so that a function can be applied to any _tuple-like_ + ** Notably the handling of tuples is based on a **Concept** `tuple_like`, + ** that is satisfied for any type in compliance with the »[tuple protocol]«. + ** Together with a [generic accessor][\ref lib::meta::getElm], this allows + ** to handle all _tuple-like_ types uniformly. + ** This solution reaches beyond the scope of the C++ standard, which declines + ** to define such a concept and rather provides an "exposition-only" concept + ** that is limited to a hard-wired selection of library types, yet does not + ** cover custom user provided types able to participate in _structured bindings_. + ** @note Due to that unfortunate limitation of the standard, we're forced to + ** provide our own alternative implementation of `std::apply`. ** ** Furthermore, a generic iteration construct is provided, to instantiate ** a generic Lambda for each element of a given tuple, which allows to write ** generic code »for each tuple element«. ** - ** @see control::CommandDef usage example + ** @remark 2026-03 this header underwent some significant rework in recent years. + ** The Lumiera project had developed a `Tuple` type way before something similar + ** was considered for inclusion into the C++ language standard; flexible and + ** specifically typed storage records were used, in conjunction with LISP-style + ** metaprogramming (»Loki Typelists«), from the very start of the project. + ** Storing an arbitrary function, combined with a tuple of arguments, was a + ** major undertaking at those days, and required a huge body of support + ** functionality — which needed to be remoulded gradually, over the + ** following years, to become compatible with newer C++ features. + ** - basic handling, like access and compile-time iteration became trivial + ** - however, the project still relies on specific metaprogramming tools, + ** to simplify handling and transforming of variadic metafunctions. + ** + ** [tuple protocol]: https://en.cppreference.com/w/cpp/language/structured_binding.html#Case_2:_binding_a_type_implementing_the_tuple_operations + ** @see variadic-helper.hpp + ** @see tuple-closure.hpp : usage example + ** @see feed-manifold.hpp : use of unified variadics and tuples ** @see TupleHelper_test ** @see typelist.hpp ** @see function.hpp @@ -40,268 +61,31 @@ #ifndef LIB_META_TUPLE_HELPER_H #define LIB_META_TUPLE_HELPER_H -#include "lib/meta/typelist.hpp" +#include "lib/integral.hpp" +#include "lib/meta/tuple-concept.hpp" +#include "lib/meta/variadic-helper.hpp" #include "lib/meta/typelist-util.hpp" #include "lib/meta/typeseq-util.hpp" -#include "lib/meta/variadic-helper.hpp" #include "lib/meta/util.hpp" #include #include +#include #include -namespace util { // forward declaration +namespace util { // forward -> see lib/format-obj.hpp ... template - std::string - toString (TY const& val) noexcept; -} + std::string toString (TY const&) noexcept; + template + struct StringConv; +} namespace lib { namespace meta { - /** trait to detect tuple types */ - template - struct is_Tuple - : std::false_type - { }; - - template - struct is_Tuple> - : std::true_type - { }; - - template - struct is_Tuple> - : std::true_type - { }; - - using std::remove_cv_t; - using std::is_reference_v; - using std::remove_reference_t; - - - /** @internal building-block: a type supporting the `tuple_size` metafunction */ - template - concept tuple_sized = requires - { - { std::tuple_size::value } -> std::convertible_to; - }; - - - /** @internal building-block: a type where elements can be accessed through a `get` friend function */ - template - concept tuple_adl_accessible = requires(TUP tup) - { - typename std::tuple_element_t; - { get(tup) } -> std::convertible_to&>; - }; - - /** @internal building-block: a type where elements can be accessed through a `get` member function */ - template - concept tuple_mem_accessible = requires(TUP tup) - { - typename std::tuple_element_t; - { tup.template get() } -> std::convertible_to&>; - }; - - template - concept tuple_element_accessible = tuple_mem_accessible or tuple_adl_accessible; - - template - concept tuple_accessible = - tuple_sized and - WithIdxSeq>::andAll([](auto idx) - { - return tuple_element_accessible; - }); - - - /** - * Concept to mark any type compliant to the »tuple protocol« - */ - template - concept tuple_like = not is_reference_v - and tuple_sized> - and tuple_accessible>; - - - /** - * Helper for abstracted / unified access to member elements of any _tuple-like_ - * @remark preferably uses a `get` member function, falling back to a - * free function `get`, which is found by ADL. - */ - template - requires(tuple_like>) - decltype(auto) - getElm (TUP&& tup) - { - using Tup = std::remove_reference_t; - static_assert (0 < std::tuple_size_v); - if constexpr (tuple_mem_accessible) - { - if constexpr (std::is_reference_v) - return tup.template get(); - else - { // return value copy when tuple given as RValue - using Elm = std::tuple_element_t; - Elm elm(tup.template get()); - return elm; - } - } - else - { // ▽▽▽ ADL - using std::get; - return get (std::forward (tup)); - } - } - - - - - namespace { // apply to tuple-like : helpers... - - /** @internal invocation helper similar to C++17 — but preferring a custom `get` impl */ - template - constexpr decltype(auto) - __unpack_and_apply (FUN&& f, TUP&& tup, std::index_sequence) - { - return std::invoke (std::forward (f) - ,getElm (std::forward(tup))... - ); - } - - - /** @internal invoke a metafunction with \a FUN and all element types from the _tuple-like_ \a TUP */ - template class META, class FUN, class TUP> - struct _InvokeMetafunTup - { - using Tupl = std::decay_t; - using Elms = ElmTypes::Seq; - using Args = Prepend::Seq; - using Type = RebindVariadic::Type; - }; - template class META, class FUN, class TUP> - struct _InvokeMetafunTup - { - using Tupl = std::decay_t; - using Elms = typename ElmTypes::template Apply; - using Args = Prepend::Seq; - using Type = RebindVariadic::Type; - }; - - template - inline constexpr bool can_nothrow_invoke_tup = _InvokeMetafunTup::Type::value; - } - - /** - * Replacement for `std::apply` — yet applicable to _tuple-like custom types_. - * For unclear reasons, the standard chooses to reject such custom types, and - * only allows a fixed set of explicitly defined facilities from the Stdlib - * (tuple, pair, array, and some ranges stuff). - * @todo 6/2025 as a first step, this replicates the implementation from C++17; - * the second step would be to constrain this to a concept `tuple_like` - */ - template requires(tuple_like>) - constexpr decltype(auto) - apply (FUN&& f, TUP&& tup) noexcept (can_nothrow_invoke_tup ) - { - using Indices = std::make_index_sequence>>; - - return __unpack_and_apply (std::forward (f) - ,std::forward (tup) - ,Indices{} - ); - } - - - /** - * Tuple iteration: perform some arbitrary operation on each element of a tuple. - * @note the given functor must be generic, since each position of the tuple - * may hold a data element of different type. - * @remark credits to David Vandevoorde (member of C++ committee) for using - * std::apply to unpack the tuple's contents into an argument pack and - * then employ a fold expression with the comma operator. - */ - template requires(tuple_like>) - constexpr void - forEach (TUP&& tuple, FUN fun) - { - lib::meta::apply ([&fun](ELMS&&... elms) - { - (fun (std::forward(elms)), ...); - } - ,std::forward (tuple)); - } - - /** - * Apply some arbitrary function onto all elements of a tuple. - * @return a new tuple constructed from the results of this function - * @note the functor must be generic and able to work with all element types - * @warning pay attention to references; the function can take arguments by-ref - * and manipulate them (side-effect), and it may return references, - * which will be placed as such into the result tuple; furthermore, - * the argument tuple can also be taken as reference... - * @remark The tuple constructor invocation is preceded by a regular argument evaluation, - * which has unspecified evaluation order (even in C++17); _no assumptions_ can be - * made regarding the order the functor will see the source tuple elements. - * Notably this differs from #forEach, where a fold-expression with comma-operator - * is used, which is guaranteed to evaluate from left to right. - */ - template requires(tuple_like>) - constexpr auto - mapEach (TUP&& tuple, FUN fun) - { - return lib::meta::apply ([&fun](ELMS&&... elms) - { //..construct the type explicitly (make_tuple would decay fun result types) - using Tuple = std::tuple(elms))) ...>; - return Tuple (fun (std::forward(elms)) ...); - } - ,std::forward (tuple)); - } - - - - /** - * Specialisation of variadic access for any tuple-like - * @see variadic-helper.hpp - */ - template - struct ElmTypes - { - template - struct Extract; - template - struct Extract> - { - using ElmTypes = Types ...>; - }; - - static constexpr size_t SIZ = std::tuple_size_v; - - using Idx = std::make_index_sequence; - using Seq = Extract::ElmTypes; - using Tup = RebindVariadic::Type; - - template class META> - using Apply = ElmTypes::template Apply; - - template class O> - using Rebind = RebindVariadic::Type; - - template class PRED> - using AndAll = ElmTypes>::template Rebind; - - template class PRED> - using OrAll = ElmTypes>::template Rebind; - }; - - - - - namespace { // rebinding helper to create std::tuple from a type sequence template @@ -389,8 +173,8 @@ namespace meta { * selecting a (partial) specialisation based on the source type given. * @see ExtractArg */ - template< typename TYPES - , template class _ElmMapper_ + template class _ElmMapper_ > struct TupleConstructor : Tuple @@ -398,16 +182,18 @@ namespace meta { /** meta-sequence to drive instantiation of the ElmMapper */ using SequenceIterator = BuildIdxIter::Ascending; - template + template + using ElmMapperFor = _ElmMapper_, Tuple, idx>; + + template static auto mapElm (SRC&& init) ///< initialise an instance of the element-mapper { - return _ElmMapper_ - ,Tuple - , idx - >{std::forward (init)}; + return ElmMapperFor{std::forward (init)}; } + + protected: template TupleConstructor (SRC&& initVals, IndexSeq) @@ -454,135 +240,66 @@ namespace meta { { return TupleConstructor{std::forward (values)}; } - - + /* ======= Tuple toString rendering ======= */ /** - * Decorating a tuple type with auxiliary data access operations. - * This helper template builds up a subclass of the given TUP (base) type - * (which is assumed to be a Tuple or at least need to be copy constructible - * from `Tuple` ). The purpose is to use the Tuple as storage record, but - * to add a layer of access functions, which in turn might rely on the exact - * type of the individual elements within the Tuple. To achieve this, for each - * type within the Tuple, the TUP type is decorated with an instance of the - * template passed in as template template parameter _X_. Each of these - * decorating instances is provided with an index number, allowing to - * access "his" specific element within the underlying tuple. - * - * The decorating template _X_ need to take its own base class as template - * parameter. Typically, operations on _X_ will be defined in a recursive fashion, - * calling down into this templated base class. To support this, an instantiation - * of _X_ with the empty type sequence is generated for detecting recursion end - * (built as innermost decorator, i.e. the immediate subclass of TUP) + * convenience function to dump a given tuple's contents, converted to string. + * @note this is the basic functionality to place rendered data into a `ostringstream.` */ - template - < template class _X_ ///< user provided template - , typename TYPES ///< Sequence of types to use within the Accessor - , class TUP =Tuple ///< the tuple type to build on - , uint i = 0 ///< tuple element index counter - > - class BuildTupleAccessor + template + inline void + joinTuple (TUP const& tup, std::string delim, std::ostringstream& buffer) { - // prepare recursion... - using Head = Split::Head; - using Tail = Split::Tail; - using NextBuilder = BuildTupleAccessor<_X_, Tail,TUP, i+1>; - using NextAccessor = NextBuilder::Product; - public: - - /** type of the product created by this template. - * Will be a subclass of TUP */ - using Product = _X_< Head // the type to use for this accessor - , NextAccessor // the base type to inherit from - , TUP // the tuple type we build upon - , i // current element index - >; - }; + forEachIDX ([&](auto i) + { + using Elm = std::tuple_element_t; + if (i) + buffer << delim; + buffer << util::toString (getElm (tup)); + }); + } - - template - < template class _X_ - , class TUP - , uint i - > - class BuildTupleAccessor< _X_, Types<>, TUP, i> + template + inline void + joinTuple (TUP const& tup, std::ostringstream& buffer) { - public: - using Product = _X_; // Note: i == tuple size - }; + joinTuple (tup, ","); + } - - - /** - * Helper to dump tuple contents. - * Defined to act as "Accessor" for BuildTupleAccessor, this helper template - * allows to create a recursive operation to invoke string conversion on - * all elements within any given tuple. + /** convenience function to join the tuple's contents into a single string. + * @param delim optional delimiter to intersperse, defaults to `","` */ - template - < typename TY - , class BASE - , class TUP - , uint idx - > - struct TupleElementDisplayer - : BASE - { - using BASE::BASE; - - std::string - dump (std::string const& prefix ="(") const - { - return BASE::dump (prefix + util::toString(std::get(*this))+","); - } - }; - - template - struct TupleElementDisplayer - : TUP - { - TupleElementDisplayer (TUP const& tup) - : TUP(tup) - { } - - std::string - dump (std::string const& prefix ="(") const - { - if (1 < prefix.length()) - // remove the trailing comma - return prefix.substr (0, prefix.length()-1) +")"; - else - return prefix+")"; - } - }; - - - /** - * convenience function to dump a given tuple's contents. - * Using the BuildTupleAccessor, we layer a stack of Instantiations of - * the TupleElementDisplayer temporarily on top of the given tuple, - * just to invoke a recursive call chain through these layers - * and get a string representation of each element in the - * tuple. - */ - template + template inline std::string - dump (std::tuple const& tuple) - { - using BuildAccessor = BuildTupleAccessor>; - using Displayer = BuildAccessor::Product ; - - return static_cast (tuple) - .dump(); - } - + joinTuple (TUP const& tup, std::string delim =",") + { + std::ostringstream buffer; + joinTuple (tup, delim, buffer); + return buffer.str(); + } + template + inline void + joinTupleParen (TUP const& tup, std::ostringstream& buffer) + { + buffer << '('; + joinTuple (tup, ",", buffer); + buffer << ')'; + } + template + inline std::string + joinTupleParen (TUP const& tup) + { + std::ostringstream buffer; + joinTupleParen (tup, buffer); + return buffer.str(); + } }} // namespace lib::meta @@ -595,12 +312,15 @@ namespace util { static std::string invoke (std::tuple const& tuple) noexcept try { - return "«"+typeStr(tuple) - + "»──" + lib::meta::dump (tuple); + std::ostringstream buffer; + buffer << "«" + << typeStr(tuple) + << "»──"; + lib::meta::joinTupleParen (tuple, buffer); + return buffer.str(); } catch(...) { return FAILURE_INDICATOR; } }; - } // namespace util #endif /*LIB_META_TUPLE_HELPER_H*/ diff --git a/src/lib/meta/tuple-idx-adaptor.hpp b/src/lib/meta/tuple-idx-adaptor.hpp new file mode 100644 index 000000000..4b3b348e8 --- /dev/null +++ b/src/lib/meta/tuple-idx-adaptor.hpp @@ -0,0 +1,131 @@ +/* + TUPLE-IDX-ADAPTOR.hpp - runtime-indexed access to tuples + + Copyright (C) + 2026 Hermann Vosseler + +  **Lumiera** is free software; you can redistribute it and/or modify it +  under the terms of the GNU General Public License as published by the +  Free Software Foundation; either version 2 of the License, or (at your +  option) any later version. See the file COPYING for further details. + +*/ + +/** @file tuple-idx-adaptor.hpp + ** Adaptor to provide access to tuple elements by runtime subscript. + ** Tuples are data records with distinct fields of different type; thus + ** most of indexing, iteration and transformation of tuples need to be + ** performed statically, at compile-time. However, under the proviso that + ** the types in a given tuple are somehow _reconcilable_, a dispatcher + ** table with access functors can be built, so that effectively all fields + ** can be handled by instances of a generic functor. This setup allows to + ** define a subscript `operator[]`, so that the suitable functor is picked + ** with runtime-index and applied to the tuple to yield the indicated field, + ** albeit converted into a _common type_. + ** + ** While this is certainly a very specific setup, and incurs some cost, + ** since the functor dispatcher tables must be prepared statically, such + ** a runtime-controlled index access allows for dynamic and data-driven + ** processing of tuple_like structures, which are fully type-safe, + ** compact in storage and easy to nest. Furthermore, run-time + ** processing of arguments passed to variadic functions + ** is enabled, providing a new kind of flexible API. + ** + ** @see TupleIdxAdaptor_test + ** @see FunTrampoline_test + */ + + +#ifndef LIB_META_TUPLE_IDX_ADAPTOR_H +#define LIB_META_TUPLE_IDX_ADAPTOR_H + + +#include "include/logging.h" +#include "lib/iter-adapter.hpp" +#include "lib/meta/tuple-helper.hpp" +#include "lib/meta/common-result.hpp" +#include "lib/meta/fun-trampoline.hpp" + +#include + + +namespace lib { +namespace meta { + + template + class TupleIndexAdaptor + : public TUP + { + /** Helper to generate a dispatch table for + * this specific instance of tuple element types */ + template + struct SubscriptBuilder + { + static_assert (reconcilable + ,"A common run-time index accessor can only be built " + "if all types in the tuple can be reconciled into a " + "common type used for the access"); + + using Tuple = RefTraits::value_type; + using AddRefs = ElmTypes::template Apply; + using ResType = CommonResult_t; + using Functor = ResType(*)(Tuple&); + + template + constexpr auto + build() + { + return [](Tuple& tuple) -> ResType + { + return getElm (tuple); + }; + } + }; + + using TypeSeq = ElmTypes::Seq; + using Accessor = FunTrampoline; + + public: + using Tuple = TUP; + using ResType = Accessor::ResType; + using reference = ResType; + using value_Type = RefTraits::value_type; + static constexpr bool isRefAccess = isLRef_v; + + + /** construct by passing a source tuple */ + template + TupleIndexAdaptor(T&& dataSrc) + : TUP{forward (dataSrc)} + { } + + /// lift all constructors of the adapted tuple + using TUP::TUP; + + Tuple & tuple() { return *this; } + Tuple const& tuple() const { return *this; } + + operator std::string() const + { + return "iTup" + joinTupleParen (this->tuple()); + } + + + /* === subscriptable === */ + + static constexpr size_t size() { return Accessor::size(); } + + ResType + operator[] (size_t idx) + { + return Accessor::dispatch(idx)(*this); + } + }; + + /// deduction guide + template + TupleIndexAdaptor(TUP) -> TupleIndexAdaptor>; + + +}}// namespace lib::meta +#endif /*LIB_META_TUPLE_IDX_ADAPTOR_H*/ diff --git a/src/lib/meta/typelist.hpp b/src/lib/meta/typelist.hpp index fd80e16d1..5fa47c0db 100644 --- a/src/lib/meta/typelist.hpp +++ b/src/lib/meta/typelist.hpp @@ -115,5 +115,21 @@ namespace meta { using Seq = Types<>; }; + + + + /** + * Concept to require a generic type sequence `Types` + */ + template + concept typeseq = requires (TSEQ seq) + { + // TSEQ can invoke a function with a Types subclass + [](Types&){/*placeholder*/}(seq); + }; + + template + static constexpr bool is_Typeseq_v = typeseq; + }} // namespace lib::meta #endif diff --git a/src/lib/meta/typeseq-util.hpp b/src/lib/meta/typeseq-util.hpp index 33756c0ac..c38021480 100644 --- a/src/lib/meta/typeseq-util.hpp +++ b/src/lib/meta/typeseq-util.hpp @@ -44,6 +44,44 @@ namespace lib { namespace meta { + template + struct Prepend; + + + + /* ========= Specialisations ========= */ + + /** + * Additional specialisation of the basic type sequence type, + * allowing to re-create a (flat) type sequence from a typelist. + * @remark recursively prepend-to-tail of the type sequence. + */ + template + struct Types< Node > + { + using List = Node; + using Seq = Prepend< H + , typename Types::Seq + >::Seq; + }; + template<> + struct Types + { + using List = Nil; + using Seq = Types<>; + }; + template<> + struct Types + : Types + { }; + + + + + + + + /* ========= Information functions ========= */ /** * Find the index of the first incidence of a type in a type-sequence. @@ -82,6 +120,13 @@ namespace meta { { }; + + + + + + /* ========= Type Sequence Manipulation ========= */ + /** * Helper: prepend a type to an existing type sequence, * thus shifting all elements within the sequence @@ -99,33 +144,6 @@ namespace meta { - /** - * Additional specialisation of the basic type sequence type, - * allowing to re-create a (flat) type sequence from a typelist. - * @remark can now be built with the help of \ref Prepend. - */ - template - struct Types< Node > - { - using List = Node; - using Seq = Prepend< H - , typename Types::Seq - >::Seq; - }; - template<> - struct Types - { - using List = Nil; - using Seq = Types<>; - }; - template<> - struct Types - : Types - { }; - - - - /** Helper: separate parts of a type sequence */ diff --git a/src/lib/meta/util.hpp b/src/lib/meta/util.hpp index 12a455e90..7fb4967be 100644 --- a/src/lib/meta/util.hpp +++ b/src/lib/meta/util.hpp @@ -505,19 +505,19 @@ namespace util { /* === Literals for common size designations === */ -inline uint +constexpr inline uint operator""_KiB (ullong const siz) { return uint(siz) * 1024u; } -inline uint +constexpr inline uint operator""_MiB (ullong const siz) { return uint(siz) * 1024u*1024u; } -inline ullong +constexpr inline ullong operator""_GiB (ullong const siz) { return siz * 1024uLL*1024uLL*1024uLL; diff --git a/src/lib/meta/value-type-binding.hpp b/src/lib/meta/value-type-binding.hpp index c816db729..1103124d5 100644 --- a/src/lib/meta/value-type-binding.hpp +++ b/src/lib/meta/value-type-binding.hpp @@ -27,9 +27,9 @@ ** Within the STL, there is a convention to provide nested typedefs to indicate ** type variations in relation to the basic payload type of the container. We ** follow this convention and support especially the - ** - `value_type` (what is conceived to be "in" the container or iterator) - ** - a simple (LValue) reference to the payload - ** - a pointer at the payload. + ** - `value_type` : how to store what is conceived to be "in" the container or iterator) + ** - a simple (LValue) reference to the payload: how you access some content + ** - a to form pointer at the payload, which is relevant for `operator->` ** ** A custom container should likewise provide such type definitions, and the ** type rebinding helper template defined in this header makes it easy to @@ -37,6 +37,12 @@ ** creates an *Extension Point*: when some payload type requires special ** treatment, an explicit specialisation to this rebinding trait may be ** injected alongside with the definition of the payload type. + ** @warning A special twist may happen when processing a function in an iterator (pipeline). + ** In this special case, `ValueTypeBinding` should _not_ be used, since the function + ** might yield again a nested iterator; yet we want to treat that iterator as a value + ** in that special case, and not look into the iterator or retrieve results from it. + ** Notably this situation applies to any kind of transforming iterator. + ** lib::meta::RefTraits should be used in such cases. ** ** The CommonResultYield type rebinding helper allows to reconcile several ** essentially compatible result types; it is used in iterator pipelines, @@ -53,7 +59,6 @@ #define LIB_META_VALUE_TYPE_BINDING_H -#include "lib/error.hpp" #include "lib/meta/trait.hpp" @@ -107,64 +112,22 @@ namespace meta { template struct ValueTypeBinding { - using value_type = RefTraits::Value; - using reference = RefTraits::Reference; - using pointer = RefTraits::Pointer; + using value_type = RefTraits::value_type; + using reference = RefTraits::reference; + using pointer = RefTraits::pointer; }; /** specialisation for classes providing STL style type binding definitions */ template struct ValueTypeBinding> > { - using _SrcType = RefTraits::Value; + using _SrcType = RefTraits::value_type; using value_type = _SrcType::value_type; using reference = _SrcType::reference; using pointer = _SrcType::pointer; }; - - - - /** - * Decision helper to select between returning results by value or reference. - * - when both types can not be reconciled, _no type result_ is provided; - * this case can be detected by a compile-time bool-check - * - when one of both types is `const`, the `ResType` will be const - * - when both types are LValue-references, then the result will be a reference, - * otherwise the result will be a value type - * @see IterExplorer::expand() - */ - template>()> - struct CommonResultYield - : std::false_type - { }; - - template - struct CommonResultYield - : std::true_type - { - using _Common = std::common_type_t; - // NOTE: unfortunately std::common_type decays (strips cv and reference) - static constexpr bool isConst = isConst_v or isConst_v; - static constexpr bool isRef = isLRef_v and isLRef_v; - - using _ConstT = std::conditional_t; - using _ValRef = std::conditional_t - , std::remove_reference_t<_ConstT> - >; - - using ResType = _ValRef; - using value_type = RefTraits::Value; - using reference = RefTraits::Reference; - using pointer = RefTraits::Pointer; - }; - - - + }} // namespace lib::meta #endif /*LIB_META_VALUE_TYPE_BINDING_H*/ diff --git a/src/lib/meta/variadic-helper.hpp b/src/lib/meta/variadic-helper.hpp index dcfcad5cf..f60401a5a 100644 --- a/src/lib/meta/variadic-helper.hpp +++ b/src/lib/meta/variadic-helper.hpp @@ -16,9 +16,14 @@ ** Metaprogramming with type sequences based on variadic template parameters. ** The type rebinding- and helper templates in this header allow to perform ** simple sequence manipulations on sequences of template parameters extracted - ** from variadic parameter packs. The goal is to (pre)process flexible argument - ** lists _at compile time,_ driven by template instantiation, allowing to specialise - ** and react specifically on some concrete pattern of argument types. + ** from variadic parameter packs. Among other things, this allows the sequence + ** of function arguments to be flexibly reshaped and remoulded at compile-time. + ** Driven by template instantiation, this enables adaptation in response to + ** specific argument types for each actual instance. Such metaprogramming + ** capabilities can be amplified significantly through the use of generic-λ. + ** @note with C++23, this framework could be expanded to cover also any + ** »tuple-like« product type, drawing from our non-standard extension + ** with the `concept tuple_like`, as defined in \ref tuple-concept.hpp. ** ** @remark in Lumiera, over time three different approaches were developed for ** handling sequences of types in metaprogramming; some of these techniques @@ -26,8 +31,10 @@ ** - templates with variadic arguments (e.g. std::tuple) can be manipulated directly ** - a type-sequence `Types` can be primed / rebound from other variadic templates ** - Loki-style type-lists are created from type-sequences and enable elaborate manipulations + ** ** @see feed-manifold.hpp advanced usage example in the Render Engine - ** @see TupleHelper_test + ** @see VariadicHelper_test + ** @see tuple-concept.hpp ** @see typelist.hpp ** @see function.hpp ** @see generator.hpp @@ -42,7 +49,7 @@ #include "lib/meta/typelist-util.hpp" #include "lib/meta/typeseq-util.hpp" #include "lib/meta/variadic-rebind.hpp" -#include "lib/meta/util.hpp" +#include "lib/meta/trait.hpp" namespace lib { namespace meta { @@ -160,7 +167,7 @@ namespace meta { * - the nested template #Apply wraps each type into another template * - #Rebind likewise instantiates another template with the element types * - #AndAll applies a predicate and combines the result with _logical and_ - * - #OrAll similarly evaluates _logical or_ on the application results + * - #OrAny similarly evaluates _logical or_ on the application results */ template struct ElmTypes @@ -177,7 +184,7 @@ namespace meta { template class PRED> using AndAll = std::__and_>; template class PRED> - using OrAll = std::__or_>; + using OrAny = std::__or_>; }; /** Partial specialisation to handle type sequences */ @@ -199,10 +206,23 @@ namespace meta { using AndAll = ElmTypes>::template Rebind; template class PRED> - using OrAll = ElmTypes>::template Rebind; + using OrAny = ElmTypes>::template Rebind; }; - // Note: a further specialisation for any »tuple-like« is defined in tuple-helper.hpp + // Note: a further specialisation for any »tuple-like« is defined in tuple-concept.hpp + + + + template + using Any_Const = ElmTypes::template OrAny; + template + using All_LRef = ElmTypes::template AndAll; + + template + static constexpr bool Any_Const_v = Any_Const::value; + template + static constexpr bool All_LRef_v = All_LRef::value; + diff --git a/src/lib/nobug-init.cpp b/src/lib/nobug-init.cpp index b00add93d..adcf83fb6 100644 --- a/src/lib/nobug-init.cpp +++ b/src/lib/nobug-init.cpp @@ -26,7 +26,7 @@ -namespace lumiera { +namespace lib { void initialise_NoBug () diff --git a/src/lib/nobug-init.hpp b/src/lib/nobug-init.hpp index da66b4f23..4f3bec90f 100644 --- a/src/lib/nobug-init.hpp +++ b/src/lib/nobug-init.hpp @@ -27,17 +27,17 @@ #ifndef NOBUG_INIT_H #define NOBUG_INIT_H -#include "include/lifecycle.h" +#include "include/lifecycle.hpp" #include -namespace lumiera { +namespace lib { void initialise_NoBug (); namespace { - LifecycleHook trigger_init_ (ON_BASIC_INIT, &initialise_NoBug); + vessel::LifecycleHook trigger_init_ (vessel::ON_BASIC_INIT, &initialise_NoBug); } } diff --git a/src/lib/random-reseed.cpp b/src/lib/random-reseed.cpp new file mode 100644 index 000000000..5d571ef15 --- /dev/null +++ b/src/lib/random-reseed.cpp @@ -0,0 +1,95 @@ +/* + RandomReseed - establish controlled seed for deterministic randomness + + Copyright (C) + 2026, Hermann Vosseler + +  **Lumiera** is free software; you can redistribute it and/or modify it +  under the terms of the GNU General Public License as published by the +  Free Software Foundation; either version 2 of the License, or (at your +  option) any later version. See the file COPYING for further details. + +* *****************************************************************/ + + +/** @file random-reseed.cpp + ** Implementation of coordinated reseeding of random number generators. + ** @todo 3/2024 work out how some executions or performances are configured reproducibly. + */ + + +#include "lib/error.hpp" +#include "lib/random.hpp" +#include "lib/random-reseed.hpp" +#include "lib/format-string.hpp" +#include "lib/iter-adapter-stl.hpp" +#include "lib/util.hpp" + +#include + +namespace lib { + namespace err = lumiera::error; + + using util::_Fmt; + using util::contains; + using iter_stl::eachVal; + + typedef void (*Hook)(); + + + namespace {// Implementation: a central registry for callbacks... + + class RandomReseed + : util::NonCopyable + { + std::map registry_; + + public: + void + enrol (uint idx, Hook callback) + { + if (contains (registry_, idx)) + throw err::Fatal{_Fmt{"Duplicate registration for random-reseeding-slot %d"} % idx}; + + registry_[idx] = callback; + } + + void + triggerAll() + { + for (Hook hook : eachVal (registry_)) + (*hook)(); + } + }; + + /** + * Static internal access to a global (hidden) registry. + * @note use Meyer's Singleton to allow registration from + * a static-init context, without prerequisites. + */ + inline RandomReseed& + randomReseedService() + { + static RandomReseed registry; + return registry; + } + }//(End)hidden impl. + + + + uint + register_at_seq (uint idx, Hook callback) + { + randomReseedService().enrol (idx, callback); + return idx; + } + + void + reseed_for_test (SeedNucleus& reseedingNucleus) + { + lib::defaultGen.reseed (reseedingNucleus); + randomReseedService().triggerAll(); + } + + +} // namespace lib diff --git a/src/lib/random-reseed.hpp b/src/lib/random-reseed.hpp new file mode 100644 index 000000000..96b886619 --- /dev/null +++ b/src/lib/random-reseed.hpp @@ -0,0 +1,76 @@ +/* + RANDOM-RESEED.hpp - establish controlled seed for deterministic randomness + + Copyright (C) + 2026, Hermann Vosseler + +  **Lumiera** is free software; you can redistribute it and/or modify it +  under the terms of the GNU General Public License as published by the +  Free Software Foundation; either version 2 of the License, or (at your +  option) any later version. See the file COPYING for further details. + +*/ + + +/** @file random-reseed.hpp + ** A setup to allow for coordinated reseeding of otherwise independent components. + ** The primary purpose of this setup is to allow for probabilistic unit-testing, + ** yet including some facilities which are more intricate to setup. These will have + ** to configure themselves to be registered with this random-reseeding service. + ** Each registration-»slot« allocates an unique index number, and for each such + ** index, some random values will be drawn from the global lib::defaultGen. + ** @remark reseeding for »deterministic randomness« is an intricate topic. + ** This setup is intended to be used as "one-stop solution" for all basic stuff, + ** commonly used in unit tests whenever "some random XYZ" seems appropriate. + ** Obviously, such a solution can only be provisional, and must be augmented, + ** once new requirements arise. In 2026, using pseudo-random values and chained + ** hash computations has become quite commonplace in the Lumiera test suite; + ** since 2024, there is a framework in place to allow for controlled re-seeding + ** and deterministic computation. However, in practice, the necessary setup + ** and initialisation was overlooked frequently, and thus we now have to raise + ** to the next level, by providing this one and central implementation function. + ** @warning _deliberately_ there is no coordination in place to ensure consistency + ** of registration or initialisation. This is only a convenient front-end, while + ** the actual registration and setup remains essentially hard-wired. + ** @see Test::seedRand() + ** @see Random_test + */ + + +#ifndef LIB_RANDOM_RESEED_H +#define LIB_RANDOM_RESEED_H + + +#include "lib/integral.hpp" + + +namespace lib { + + class SeedNucleus; // see lib/random.hpp + + + /** + * Install a callback \a Hook to be invoked whenever + * _reseeding_ of random number generators becomes necessary. + * @param idx number of a distinct »slot« to allocate; + * invocation will happen in ascending order + * @remark this function can be used to plant a static variable + * to ensure registration happens early and once within + * some implementation translation unit. The callback + * is called way later, typically when the first test + * case invokes test::Test::seedRand() + */ + uint register_at_seq (uint idx, void(*)(void) ); + + /** + * Reseed generic randomness used for probabilistic unit-testing. + * After calling this function, the default random generator in the Lumiera library + * is reconfigured and reseeded based on the given seeding nucleus; furthermore, a + * small, hard-wired collection of test facilities will be re-set and re-seeded + * from this newly configured random source. + */ + void reseed_for_test (SeedNucleus&); + + +} // namespace lib +#endif /*LIB_RANDOM_RESEED_H*/ diff --git a/src/lib/random.cpp b/src/lib/random.cpp index cebf81a18..bcce7fa0d 100644 --- a/src/lib/random.cpp +++ b/src/lib/random.cpp @@ -85,6 +85,27 @@ namespace lib { entropyGen.reseed(entropyNucleus); defaultGen.reseed(entropyNucleus); } + + + + /** + * @remark this is a general-purpose util mostly used in unit-tests + * to generate a short string item that looks readable; probabilities + * of individual letters were chosen non-uniformely to improve readability. + */ + string + randStr (size_t len) + { + static const string alpha{"aaaabbccdddeeeeeffgghiiiiiijjkkllmmnnnoooopqqrrssttuuuuvwwxxxxxxxxxyyyyzz0123456789"}; + static const size_t MAXCH{alpha.size()}; + + string garbage(len,'\0'); + size_t p = len; + while (p) + garbage[--p] = alpha[rani (MAXCH)]; + return garbage; + } + } // namespace lib diff --git a/src/lib/random.hpp b/src/lib/random.hpp index fb2c800b3..f03744d64 100644 --- a/src/lib/random.hpp +++ b/src/lib/random.hpp @@ -166,6 +166,14 @@ namespace lib { + /* ===== general-purpose utils ===== */ + + /** create garbage string of given length. + * @return string containing arbitrary lower case letters and numbers + */ + std::string randStr (size_t len); + + /* ===== Implementation details ===== */ diff --git a/src/lib/scoped-collection.hpp b/src/lib/scoped-collection.hpp index 8d4d9d82d..a53e18e79 100644 --- a/src/lib/scoped-collection.hpp +++ b/src/lib/scoped-collection.hpp @@ -2,7 +2,7 @@ SCOPED-COLLECTION.hpp - managing a fixed collection of noncopyable polymorphic objects Copyright (C) - 2012, Hermann Vosseler + 2012,2026 Hermann Vosseler   **Lumiera** is free software; you can redistribute it and/or modify it   under the terms of the GNU General Public License as published by the @@ -12,44 +12,46 @@ */ /** @file scoped-collection.hpp - ** Managing a collection of non-copyable polymorphic objects in compact storage. - ** This helper supports the frequently encountered situation where a service - ** implementation internally manages a collection of implementation related - ** sub-components with reference semantics. Typically, those objects are - ** being used polymorphically, and often they are also added step by step. - ** The storage holding all those child objects is allocated in one chunk - ** and never adjusted. + ** Manage a collection of non-copyable polymorphic objects in compact storage. + ** This helper supports the common situation where a service implementation has + ** to manage a fixed collection of implementation related sub-components, treated + ** with reference semantics. Typically, those objects are used polymorphically, + ** and often even with _type erasure_ and added step by step through a builder. + ** The storage to hold all those child objects is allocated in one chunk + ** and never adjusted. Individual children are never re-ordered or removed, + ** only the collection as a whole is discarded eventually. ** ** # usage patterns ** ** The common ground for all usage of this container is to hold some elements ** with exclusive ownership; when the enclosing container goes out of scope, ** all the dtors of the embedded objects will be invoked. Frequently this - ** side effect is the reason for using the container: we want to own some - ** resource handles to be available exactly as long as the managing object - ** needs and accesses them. + ** side effect is one of the reasons for using the container: we want to own + ** some resource handles and be sure they are available exactly as long as + ** the managing scope is alive and able to access them. ** ** There are two different usage patterns for populating a ScopedCollection - ** - the "stack style" usage creates an empty container (using the one arg - ** ctor just to specify the maximum size). The storage to hold up to this - ** number of objects is (heap) allocated right away, but no objects are + ** - the **stack style** usage creates an empty container (using the one arg + ** ctor to specify the maximum size only). The storage capacity to hold + ** a number of objects, up to the predefined maximum, is (heap) allocated + ** immediately at construction of the container, yet no child objects are ** created. Later on, individual objects are "pushed" into the collection ** by invoking #emplaceElement() to create a new element of the default ** type `I`) or #emplace(args) to create some subtype. This way, - ** the container is being filled successively. - ** - the "RAII style" usage strives to create all of the content objects - ** right away, immediately after the memory allocation. This usage pattern - ** avoids any kind of "lifecycle state". Either the container comes up sane - ** and fully populated, or the ctor call fails and any already created - ** objects are discarded. + ** the container is gradually filled up. + ** - the **RAII-style** usage attempts to create all the contained objects + ** right away, immediately after memory allocation. This usage pattern + ** avoids any kind of _lifecycle state_. Either the container is in sane + ** state and fully populated from the outset, or the call to the ctor fails + ** and all objects that have already been created are discarded. ** @note intentionally there is no operation to discard individual objects, - ** all you can do is to #clear() the whole container. + ** the only option is to #clear() the whole container. ** @note the container can hold instances of a subclass of the type defined ** by the template parameter `I`. But you need to ensure in this case ** that the defined buffer size for each element (2nt template parameter) ** is sufficient to hold any of these subclass instances. This condition ** is protected by a static assertion (compilation failure). - ** @warning when using subclasses, a virtual dtor is mandatory + ** @warning when using subclasses, defining a virtual dtor is mandatory! ** @warning deliberately **not threadsafe** ** ** @see ScopedCollection_test @@ -67,7 +69,7 @@ #include "lib/iter-adapter.hpp" #include -#include +#include namespace lib { @@ -76,6 +78,7 @@ namespace lib { using LERR_(INDEX_BOUNDS); using LERR_(CAPACITY); + using meta::lum_iter; /** @@ -98,8 +101,8 @@ namespace lib { public: /** * Storage Frame to hold one Child object. - * The storage will be an heap allocated - * array of such Wrapper objects. + * The storage will be an heap allocated array + * of such Wrapper objects. * @note doesn't manage the Child */ class ElementHolder @@ -124,13 +127,12 @@ namespace lib { - /** place object of type TY, forwarding ctor arguments */ template TY& create (ARGS&& ...args) { - static_assert ( meta::is_Subclass() + static_assert ( meta::is_Subclass_v && sizeof(TY) <= siz, "ElementHolder buffer too small"); @@ -148,6 +150,9 @@ namespace lib { clear(); } + /** create an empty ScopedCollection with given max capacity + * @remark this ctor variant is intended for _stack-like_ usage + */ explicit ScopedCollection (size_t maxElements) : level_(0) @@ -160,9 +165,9 @@ namespace lib { * Ctor fails in case of any error during element creation. * @param builder functor to be invoked for each "slot". * It gets an ElementHolder& as parameter, and should - * use this to create an object of some I-subclass + * use this to create an object of suitable I-subclass */ - template + template CTOR> ScopedCollection (size_t maxElements, CTOR builder) : level_(0) , capacity_(maxElements) @@ -188,22 +193,44 @@ namespace lib { populate_by (builder,instance); } + /** variation of RAII-style: "pull" elements from an iterator. + * Each element will be created directly by invoking either the copy ctor, + * or a suitable 1-argument ctor, depending on the type yielded by the iterator. + * No further elements will be created however, once the iterator is exhausted. + * @note anything in accordance to the Lumiera Forward concept is acceptable. + * This rules out just passing a plain STL iterator (because these can't + * tell for themselves when they're exhausted). Use an suitable iter-adapter + * instead, e.g. by invoking lib::iter_stl::eachElm(stl_container) + */ + template + ScopedCollection (size_t maxElements, IT&& iter) + : level_(0) + , capacity_(maxElements) + , elements_(new ElementHolder[maxElements]) + { + pull_from (std::forward (iter)); + } + /* == some pre-defined Builders == */ - class FillAll; ///< fills the ScopedCollection with default constructed I-instances + /** Builder: fills the given storage slot with a I-instance + * constructed from copies of the given arguments. + * @return builder-λ to pass into ScopeCollection's ctor. */ + template + static auto fill (ARGS&&... args); - template - class FillWith; ///< fills the ScopedCollection with default constructed TY-instances + /** Builder: fills the given storage slot with a TY-instance + * constructed from copies of the given arguments. */ + template + static auto fillWith (ARGS&&... args); - template - class PullFrom; ///< fills by copy-constructing values pulled from the iterator IT + /** Builder: fills the given storage slot from invoking the + * provided factory functor, with either an I-element or a + * I-subtype, depending on the functor's result type. + * @note the functor will be copied in the resulting λ */ + template FUN> + static auto invoke (FUN&& factory); - template - static PullFrom - pull (IT iter) ///< convenience shortcut to pull from any given Lumiera Forward Iterator - { - return PullFrom (iter); - } void clear() @@ -245,7 +272,7 @@ namespace lib { * ElementHolder#create function, which places a new element * into the storage frame passed as parameter. */ - template + template CTOR> void populate_by (CTOR builder) try { @@ -280,11 +307,46 @@ namespace lib { catch(...) { WARN (progress, "Failure while populating ScopedCollection. " - "All elements will be discarded"); + "All elements will be discarded."); clear(); throw; } + template + struct _BuildFromIter ///< @internal helper to determine what to build from an iterator + { + using Yield = iter::Yield; + using Value = meta::ValueTypeBinding::value_type; + using Type = std::conditional_t, Value, I>; + static_assert (std::is_constructible_v + ,"Iterator must produce something " + "that can be used to build contents of this container"); + }; + + /** variation of element initialisation, + * creating further elements from the results of an iterator, + * until either the container is filled up or the iterator exhausted. + */ + template + void + pull_from (IT iter) + try { + while (iter and level_ < capacity_) + { + using Element = _BuildFromIter::Type; + ElementHolder& storageFrame (elements_[level_]); + storageFrame.template create (*iter); + ++level_; + ++iter; + } } + catch(...) + { + WARN (progress, "Failure while pulling Iterator into ScopedCollection. " + "All elements will be discarded."); + clear(); + throw; + } + /** push a new element of default type @@ -338,8 +400,8 @@ namespace lib { size_t size () const { return level_; } - size_t capacity () const { return capacity_; } - bool empty () const { return 0 == level_; } + size_t capacity() const { return capacity_; } + bool empty() const { return 0 == level_; } // use in standard range for loops... @@ -352,7 +414,7 @@ namespace lib { private: /* ==== Storage: heap allocated array of element buffers ==== */ - typedef std::unique_ptr ElementStorage; + using ElementStorage = std::unique_ptr; size_t level_; size_t capacity_; @@ -415,66 +477,66 @@ namespace lib { /* === Supplement: pre-defined element builders === */ /** \par usage - * Pass an instance of this builder functor as 2nd parameter - * to ScopedCollections's ctor. (an anonymous instance is OK). + * On invocation, this function generates a builder-λ to pass as + * the 2nd parameter to ScopedCollections's ctor. When arguments are given, + * these are perfect-forwarded into a copy in the closure, which implies that + * all instances will receive either a copy or const-reference to these values. + * (It is not possible to pass references this way). * Using this variant of the ctor switches the collection to RAII-style: - * It will immediately try to create all the embedded objects, invoking this + * It will immediately try to create all the embedded objects, invoking the * builder functor for each "slot" to hold such an embedded object. Actually, * this "slot" is an ElementHolder instance, which provides functions for * placement-creating objects into this embedded buffer. + * @note allows also to default-create or copy-create */ template - class ScopedCollection::FillAll - { - public: - void - operator() (ScopedCollection::ElementHolder& storage) - { - storage.template create(); - } - }; + template + auto + ScopedCollection::fill (ARGS&&... args) + { + return [... args = std::forward (args)] + (ScopedCollection::ElementHolder& storage) + { + storage.template create (args...); + }; + } template - template - class ScopedCollection::FillWith - { - public: - void - operator() (ScopedCollection::ElementHolder& storage) - { - storage.template create(); - } - }; + template + auto + ScopedCollection::fillWith (ARGS&&... args) + { + return [... args = std::forward (args)] + (ScopedCollection::ElementHolder& storage) + { + storage.template create (args...); + }; + } - /** \par usage - * This variant allows to "pull" elements from an iterator. - * Actually, the collection will try to create each element right away, - * by invoking the copy ctor and passing the value yielded by the iterator. - * @note anything in accordance to the Lumiera Forward Iterator pattern is OK. - * This rules out just passing a plain STL iterator (because these can't - * tell for themselves when they're exhausted). Use an suitable iter-adapter - * instead, e.g. by invoking lib::iter_stl::eachElm(stl_container) + /** + * @remark this builder handles the technical aspect of creating elements + * into the slots of the ElementHolder storage, so that client code + * has only to define the constructor argument, which notably can + * also be a subclass of \a I (assuming the storage size suffices) + * @return builder-λ that can be passed into ScopeCollection's ctor. */ template - template - class ScopedCollection::PullFrom - { - IT iter_; - - using ElementType = meta::ValueTypeBinding::value_type; - - public: - PullFrom (IT source) - : iter_(source) - { } - - void - operator() (ScopedCollection::ElementHolder& storage) - { - storage.template create (*iter_); - ++iter_; - } - }; + template FUN> + auto + ScopedCollection::invoke (FUN&& factory) + { + using Res = std::invoke_result_t; + using Elem = std::conditional_t, Res, I>; + static_assert (std::is_constructible_v + ,"Iterator must produce something " + "that can be used to build contents of this container"); + + return [fac = std::forward (factory)] + (ScopedCollection::ElementHolder& storage) mutable + { + storage.template create (fac()); + }; + } diff --git a/src/lib/several-builder.hpp b/src/lib/several-builder.hpp index 9bc5f6795..ea86d26f8 100644 --- a/src/lib/several-builder.hpp +++ b/src/lib/several-builder.hpp @@ -137,7 +137,7 @@ namespace lib { using std::is_volatile_v; using std::is_const_v; using std::is_same_v; - using lib::meta::is_Subclass; + using lib::meta::is_Subclass_v; using several::ArrayBucket; @@ -743,7 +743,7 @@ namespace lib { typename Policy::Fac& factory(*this); - if (is_Subclass() and has_virtual_destructor_v) + if (is_Subclass_v and has_virtual_destructor_v) { __ensureMark (VIRTUAL); return [factory](ArrayBucket* bucket){ unConst(factory).template destroy (bucket); }; @@ -753,7 +753,7 @@ namespace lib { __ensureMark (TRIVIAL); return [factory](ArrayBucket* bucket){ unConst(factory).template destroy (bucket); }; } - if (is_same_v and is_Subclass()) + if (is_same_v and is_Subclass_v) { __ensureMark (ELEMENT); return [factory](ArrayBucket* bucket){ unConst(factory).template destroy (bucket); }; diff --git a/src/lib/several.hpp b/src/lib/several.hpp index b6767c71c..6ffb285d3 100644 --- a/src/lib/several.hpp +++ b/src/lib/several.hpp @@ -231,8 +231,8 @@ namespace lib { friend auto begin (Several const& svl) { return svl.begin();} friend auto end (Several const& svl) { return svl.end(); } - using value_type = meta::RefTraits::Value; - using reference = meta::RefTraits::Reference; + using value_type = meta::RefTraits::value_type; + using reference = meta::RefTraits::reference; using const_reference = value_type const&; diff --git a/src/lib/stat/csv.hpp b/src/lib/stat/csv.hpp index c55680cd0..21a05bf97 100644 --- a/src/lib/stat/csv.hpp +++ b/src/lib/stat/csv.hpp @@ -40,8 +40,9 @@ #include "lib/error.hpp" #include "lib/null-value.hpp" -#include "lib/meta/tuple-helper.hpp" +#include "lib/meta/tuple-concept.hpp" #include "lib/format-string.hpp" +#include "lib/format-obj.hpp" #include "lib/regex.hpp" #include diff --git a/src/lib/sync.hpp b/src/lib/sync.hpp index 3cd730da8..90ab042c5 100644 --- a/src/lib/sync.hpp +++ b/src/lib/sync.hpp @@ -282,5 +282,5 @@ namespace lib { }; -} // namespace lumiera +} // namespace lib::sync #endif diff --git a/src/lib/thread.cpp b/src/lib/thread.cpp index 0f227f21e..7c14942aa 100644 --- a/src/lib/thread.cpp +++ b/src/lib/thread.cpp @@ -56,7 +56,7 @@ namespace thread{ ThreadWrapper::decorate_with_global_count (string const& rawID) { static atomic_uint globalCnt{1}; - return _Fmt{"%s.%03i"} % rawID % globalCnt.fetch_add (+1, std::memory_order_acq_rel); + return _Fmt{"%s.%03i"} % rawID % globalCnt.fetch_add (+1, std::memory_order_relaxed); } diff --git a/src/lib/thread.hpp b/src/lib/thread.hpp index abfbdfc7f..7f12c954b 100644 --- a/src/lib/thread.hpp +++ b/src/lib/thread.hpp @@ -451,6 +451,8 @@ namespace lib { * and possibly configure further details, depending on the actual Policy used. * @remark the primary ThreadLifecycle-ctor accepts such a Launch-instance * and invokes a chain of λ-functions collected in the member #launch + * @note a Launch instance can be built by direct conversion from an arg-less + * thread function (because the ctor is _not_ explicit) */ struct Launch : util::MoveOnly @@ -571,6 +573,10 @@ namespace lib { * Primary constructor: Launch the new thread with flexible configuration. * @param launcher a #Launch builder with a λ-chain to configure and * finally trigger start of the thread + * @note The Launch instance can be built by implicit conversion from any + * arg-less invocable; this implies that ThreadLifecycle (and its + * relevant subclasses) can be built without much ado directly + * from a Lambda that defines the thread-function. */ ThreadLifecycle (Launch launcher) : Policy{launcher.id} @@ -644,6 +650,12 @@ namespace lib { * thread has already copied the arguments and indeed actively started to run. * @warning The destructor waits for a short grace period of 20ms, but calls * `std::terminate` afterwards, should the thread still be active then. + * @remark creating a Thread instance immediately starts the thread; + * it can be built from + * - any argument-less invocable or lambda (that is invoked in the thread) + * - a string argument for the thread-id, an invocable, further arguments... + * - a member pointer and further arguments; useful for derived classes + * - a ThreadLifecycle::Launch, which allows fine grained configuration */ class Thread : public thread::ThreadLifecycle @@ -741,7 +753,7 @@ namespace lib { inline void launchDetached (ThreadHookable::Launch&& launchBuilder) { - static_assert (lib::meta::is_Subclass()); + static_assert (lib::meta::is_Subclass_v); new TAR{move(launchBuilder) .atExit([](TAR& selfAllocation) diff --git a/src/lib/util-coll.hpp b/src/lib/util-coll.hpp index 06d819994..b06c01323 100644 --- a/src/lib/util-coll.hpp +++ b/src/lib/util-coll.hpp @@ -57,14 +57,14 @@ namespace util { typedef lib::meta::Unwrap::Type TaT; enum{ value = lib::meta::can_STL_ForEach::value - &&!lib::meta::can_IterForEach::value + &&!lib::meta::can_LumieraIter::value }; }; template struct treat_as_LumieraIterator { - enum{ value = lib::meta::can_IterForEach::value + enum{ value = lib::meta::can_LumieraIter::value }; }; diff --git a/src/lib/util-foreach.hpp b/src/lib/util-foreach.hpp index 438936a14..b67b785eb 100644 --- a/src/lib/util-foreach.hpp +++ b/src/lib/util-foreach.hpp @@ -56,7 +56,7 @@ namespace util { using lib::meta::disable_if; using lib::meta::can_STL_ForEach; - using lib::meta::can_IterForEach; + using lib::meta::can_LumieraIter; @@ -108,7 +108,7 @@ namespace util { template - inline disable_if< can_IterForEach, + inline disable_if< can_LumieraIter, FUN > for_each (Container const& coll, FUN doIt) { @@ -124,7 +124,7 @@ namespace util { template - inline enable_if< can_IterForEach, + inline enable_if< can_LumieraIter, FUN > for_each (IT const& ii, FUN doIt) { @@ -152,7 +152,7 @@ namespace util { template - inline enable_if< can_IterForEach, + inline enable_if< can_LumieraIter, bool > and_all (IT const& ii, FUN predicate) { @@ -179,7 +179,7 @@ namespace util { template - inline enable_if< can_IterForEach, + inline enable_if< can_LumieraIter, bool > has_any (IT const& ii, FUN predicate) { diff --git a/src/lib/util-quant.hpp b/src/lib/util-quant.hpp index a9f67f9eb..a171bce1c 100644 --- a/src/lib/util-quant.hpp +++ b/src/lib/util-quant.hpp @@ -20,6 +20,9 @@ #ifndef LIB_UTIL_QUANT_H #define LIB_UTIL_QUANT_H +#include "lib/integral.hpp" + +#include #include #include #include @@ -31,7 +34,7 @@ namespace util { template inline constexpr bool - isPow2 (N n) + isPow2 (N n) noexcept { return n > 0 and !(n & (n-1)); }; // at each power of 2, a new bit is set for the first time @@ -44,7 +47,8 @@ namespace util { I quot; I rem; - IDiv (I num, I den) + constexpr + IDiv (I num, I den) noexcept : quot(num/den) , rem(num - quot*den) { } @@ -54,7 +58,8 @@ namespace util { struct IDiv : div_t { - IDiv (int num, int den) + constexpr + IDiv (int num, int den) noexcept : div_t(div (num,den)) { } }; @@ -63,7 +68,8 @@ namespace util { struct IDiv : ldiv_t { - IDiv (long num, long den) + constexpr + IDiv (long num, long den) noexcept : ldiv_t(ldiv (num,den)) { } }; @@ -72,14 +78,15 @@ namespace util { struct IDiv : lldiv_t { - IDiv (llong num, llong den) + constexpr + IDiv (llong num, llong den) noexcept : lldiv_t(lldiv (num,den)) { } }; template - inline IDiv - iDiv (I num, I den) ///< support type inference and auto typing... + inline constexpr IDiv + iDiv (I num, I den) noexcept ///< support type inference and auto typing... { return IDiv{num,den}; } @@ -93,8 +100,8 @@ namespace util { * @see UtilFloordiv_test */ template - inline I - floordiv (I num, I den) + inline constexpr I + floordiv (I num, I den) noexcept { if (0 < (num^den)) return num/den; @@ -115,8 +122,8 @@ namespace util { * @see UtilFloorwarp_test */ template - inline IDiv - floorwrap (I num, I den) + inline constexpr IDiv + floorwrap (I num, I den) noexcept { IDiv res(num,den); if (0 > (num^den) and res.rem) @@ -129,6 +136,23 @@ namespace util { } + /** quantise towards ceiling. + * This function models to fit something into a tiled structure. + */ + template requires std::unsigned_integral + inline constexpr U + ceilDiv (U num, U den) noexcept + { + return (num + den - 1u) / den; + } + + static_assert (0 == ceilDiv (0u,5u)); + static_assert (1 == ceilDiv (1u,5u)); + static_assert (1 == ceilDiv (4u,5u)); + static_assert (1 == ceilDiv (5u,5u)); + static_assert (2 == ceilDiv (6u,5u)); + + /** * epsilon comparison of doubles. @@ -146,8 +170,8 @@ namespace util { * @see https://en.wikipedia.org/wiki/Unit_in_the_last_place * @todo 3/2024 seems we have solved this problem several times meanwhile /////////////////////////////////TICKET #1360 sort out floating-point rounding and precision */ - inline bool - almostEqual (double d1, double d2, unsigned int ulp =2) + inline constexpr bool + almostEqual (double d1, double d2, unsigned int ulp =2) noexcept { using std::fabs; return fabs (d1-d2) < DBL_EPSILON * fabs (d1+d2) * ulp diff --git a/src/lib/util.hpp b/src/lib/util.hpp index f834299a8..02f22e198 100644 --- a/src/lib/util.hpp +++ b/src/lib/util.hpp @@ -367,6 +367,12 @@ namespace util { return const_cast (ro); } + /** similar shortcut to repurpose a const accessor + * for implementing a similar non-const variant + * @warning beware! you might be subverting the contract + */ + #define UN_CONST(_CALL_) util::unConst (std::as_const(*this). _CALL_ ) + /* ======== address and identity ========= */ diff --git a/src/lib/variant.hpp b/src/lib/variant.hpp index 1561e7e83..30dbb8308 100644 --- a/src/lib/variant.hpp +++ b/src/lib/variant.hpp @@ -71,6 +71,7 @@ #include "lib/error.hpp" +#include "lib/meta/trait.hpp" #include "lib/meta/typelist.hpp" #include "lib/meta/typelist-util.hpp" #include "lib/meta/generator.hpp" diff --git a/src/lumiera/main.cpp b/src/lumiera/main.cpp index bdb698040..f91739cbb 100644 --- a/src/lumiera/main.cpp +++ b/src/lumiera/main.cpp @@ -20,8 +20,8 @@ #include "include/logging.h" #include "lib/error.hpp" -#include "common/appstate.hpp" -#include "common/option.hpp" +#include "vessel/voyage.hpp" +#include "vessel/option.hpp" #include "vault/enginefacade.hpp" #include "vault/netnodefacade.hpp" @@ -30,8 +30,8 @@ #include "stage/guifacade.hpp" using lib::Cmdline; -using lumiera::Subsys; -using lumiera::AppState; +using vessel::Subsys; +using vessel::Voyage; namespace { Subsys& engine = vault::EngineFacade::getDescriptor(); @@ -49,12 +49,12 @@ main (int argc, const char* argv[]) { NOTICE (main, "*** Lumiera NLE for Linux ***"); - AppState& application = AppState::instance(); + Voyage& voyage = Voyage::access(); try { Cmdline args (argc,argv); - lumiera::Option options (args); - application.init (options); + vessel::Option options (args); + voyage.init (options); netNode.depends (session); netNode.depends (engine); @@ -65,21 +65,21 @@ main (int argc, const char* argv[]) script.depends (session); script.depends (engine); - application.maybeStart (playOut); - application.maybeStart (netNode); - application.maybeStart (lumigui); - application.maybeStart (script); + voyage.maybeStart (playOut); + voyage.maybeStart (netNode); + voyage.maybeStart (lumigui); + voyage.maybeStart (script); - return application.maybeWait(); + return voyage.maybeWait(); } catch (lumiera::Error& problem) { - return application.abort (problem); + return voyage.abort (problem); } catch (...) { - return application.abort(); + return voyage.abort(); } } diff --git a/src/stage/config-keys.cpp b/src/stage/config-keys.cpp index b1a370571..1d7a3676b 100644 --- a/src/stage/config-keys.cpp +++ b/src/stage/config-keys.cpp @@ -16,8 +16,8 @@ /** @file config-keys.hpp ** Definition of access keys for global UI configuration. ** - ** @see lumiera::AppState general Lumiera application main - ** @see lumiera::Config + ** @see vessel::Voyage general Lumiera application main + ** @see vessel::spine::Config ** */ diff --git a/src/stage/config-keys.hpp b/src/stage/config-keys.hpp index e4902d34b..28210e671 100644 --- a/src/stage/config-keys.hpp +++ b/src/stage/config-keys.hpp @@ -18,7 +18,7 @@ ** ** # configuration and resource search ** - ** The UI layer retrieves the necessary configuration values from lumiera::Config, + ** The UI layer retrieves the necessary configuration values from vessel::spine::Config, ** the config facade in the application core. Currently as of 2/2011 these values are ** loaded from setup.ini, because the full-blown config system is not yet implemented. ** Amongst others, this configuration defines a _search path_ for icons and a @@ -27,9 +27,9 @@ ** This allows for a relocatable Lumiera installation bundle. ** ** @see gtk-lumiera.cpp the plugin to pull up this GUI - ** @see lumiera::BasicSetup definition of the acceptable configuration values - ** @see lumiera::AppState general Lumiera application main - ** @see lumiera::Config + ** @see vessel::BasicSetup definition of the acceptable configuration values + ** @see vessel::Voyage general Lumiera application main + ** @see vessel::spine::Config ** */ @@ -38,14 +38,14 @@ #include "lib/symbol.hpp" -#include "include/config-facade.h" +#include "vessel/spine/config-facade.h" namespace stage { using lib::Literal; - using lumiera::Config; + using vessel::spine::Config; extern Literal KEY_TITLE; extern Literal KEY_VERSION; diff --git a/src/stage/ctrl/demo-controller.cpp b/src/stage/ctrl/demo-controller.cpp index 4617beb26..7a21047eb 100644 --- a/src/stage/ctrl/demo-controller.cpp +++ b/src/stage/ctrl/demo-controller.cpp @@ -37,7 +37,7 @@ namespace ctrl { using std::make_unique; using steam::node::DummyTickService; using steam::node::DummyImageGenerator; - using lumiera::DisplayerInput; + using vessel::spine::DisplayerInput; DemoController::DemoController() @@ -54,7 +54,7 @@ namespace ctrl { * @param displayFormat format for the frames expected in the passed image buffer. */ void - DemoController::activate (lumiera::DisplayerInput displayFormat) + DemoController::activate (DisplayerInput displayFormat) { REQUIRE (imageGen_); imageGen_->configure (displayFormat); diff --git a/src/stage/ctrl/demo-controller.hpp b/src/stage/ctrl/demo-controller.hpp index 0474f793f..510b47bc4 100644 --- a/src/stage/ctrl/demo-controller.hpp +++ b/src/stage/ctrl/demo-controller.hpp @@ -32,7 +32,7 @@ #define DEMO_CONTROLLER_H #include "stage/gtk-base.hpp" -#include "include/display-handles.hpp" +#include "vessel/spine/display-handles.hpp" #include "lib/nocopy.hpp" #include @@ -68,7 +68,7 @@ namespace ctrl { void pause(); void stop(); - void activate (lumiera::DisplayerInput); + void activate (vessel::spine::DisplayerInput); sigc::signal output_; private: bool playing_; diff --git a/src/stage/dialog/test-control.hpp b/src/stage/dialog/test-control.hpp index 45ccf97a4..628a7db55 100644 --- a/src/stage/dialog/test-control.hpp +++ b/src/stage/dialog/test-control.hpp @@ -40,7 +40,7 @@ #include "include/gui-notification-facade.h" #include "steam/mobject/session/dummy-session-connection.hpp" -#include "lib/test/test-helper.hpp" +#include "test/test-helper.hpp" #include "lib/diff/gen-node.hpp" #include "lib/scoped-ptrvect.hpp" #include "lib/format-string.hpp" @@ -285,7 +285,7 @@ namespace dialog { pickDummyID() { string dummyID = sanitise (dummy_.get_text()); - dummy_.set_text (string{_Fmt{"d%s%02d"} % lib::test::randStr(2) % (1 + lib::rani(99))}); + dummy_.set_text (string{_Fmt{"d%s%02d"} % lib::randStr(2) % (1 + lib::rani(99))}); return dummyID; } diff --git a/src/stage/gtk-base.hpp b/src/stage/gtk-base.hpp index a84a8a631..2e07354d6 100644 --- a/src/stage/gtk-base.hpp +++ b/src/stage/gtk-base.hpp @@ -25,7 +25,7 @@ ** ** # configuration and resource search ** - ** The GUI backbone retrieves the necessary configuration values from lumiera::Config, + ** The GUI backbone retrieves the necessary configuration values from vessel::spine::Config, ** the config facade in the application core. Currently as of 2/2011 these values are ** loaded from setup.ini, because the full-blown config system is not yet implemented. ** Amongst others, this configuration defines a _search path_ for icons and a @@ -39,9 +39,9 @@ ** @see gtk-lumiera.cpp the plugin to pull up this GUI ** @see stage::GuiFacade access point for starting the GUI ** @see stage::GuiNotification interface for communication with the gui from the lower layers - ** @see lumiera::Config - ** @see lumiera::BasicSetup definition of the acceptable configuration values - ** @see lumiera::AppState general Lumiera application main + ** @see vessel::spine::Config + ** @see vessel::BasicSetup definition of the acceptable configuration values + ** @see vessel::Voyage general Lumiera application main */ diff --git a/src/stage/gtk-lumiera.cpp b/src/stage/gtk-lumiera.cpp index fd2d21a09..4f83196a4 100644 --- a/src/stage/gtk-lumiera.cpp +++ b/src/stage/gtk-lumiera.cpp @@ -19,7 +19,7 @@ ** This plugin is linked together with the Lumiera GUI code; when loaded as ** Lumiera plugin, it allows to kick off the GTK main event loop and thus to bring ** up the GUI. The loading and shutdown process is carried out by stage::GuiFacade and - ** controlled by lumiera::AppState, which in turn is activated by Lumiera main(). + ** controlled by the vessel::Voyage, which in turn is activated by Lumiera main(). ** ** After successfully loading this module, a call to GuiFacade::launchUI is expected to ** happen, passing a termination signal (callback) to be executed when the GUI terminates. @@ -31,7 +31,7 @@ ** This entails also to open the primary "business" interface(s) of the GUI ** (currently as of 1/16 this is the interface stage::GuiNotification.) ** - ** @see lumiera::AppState + ** @see vessel::Voyage ** @see stage::GuiFacade ** @see guifacade.cpp ** @see ui-manager.hpp @@ -44,19 +44,19 @@ #include "stage/ui-bus.hpp" #include "stage/guifacade.hpp" #include "stage/ctrl/ui-manager.hpp" -#include "common/subsys.hpp" +#include "vessel/subsys.hpp" #include "lib/nocopy.hpp" extern "C" { -#include "common/interface.h" -#include "common/interface-descriptor.h" +#include "vessel/interface.h" +#include "vessel/interface-descriptor.h" } #include -using lumiera::Subsys; +using vessel::Subsys; using lib::launchDetached; using lumiera::error::LUMIERA_ERROR_STATE; using stage::LUMIERA_INTERFACE_INAME(lumieraorg_Gui, 1); diff --git a/src/stage/guifacade.hpp b/src/stage/guifacade.hpp index c410b8f1b..66f0d3165 100644 --- a/src/stage/guifacade.hpp +++ b/src/stage/guifacade.hpp @@ -18,8 +18,8 @@ ** contained within the GUI dynamic module), because it's job is to load and ** activate this module as plug-in and to startup the GUI. ** - ** @see lumiera::AppState - ** @see lumiera::Option + ** @see vessel::Voyage + ** @see vessel::Option ** @see guifacade.cpp ** @see main.cpp */ @@ -29,11 +29,11 @@ #define STAGE_FACADE_H -#include "common/subsys.hpp" +#include "vessel/subsys.hpp" #include "lib/nocopy.hpp" extern "C" { -#include "common/interface.h" +#include "vessel/interface.h" } @@ -72,10 +72,10 @@ namespace stage { { public: - /** provide a descriptor for lumiera::AppState, + /** provide a descriptor for vessel::Voyage, * wired accordingly to allow main to load, * start and stop the Lumiera GTK GUI. */ - static lumiera::Subsys& getDescriptor(); + static vessel::Subsys& getDescriptor(); /** weather the GUI has been started and all interfaces are opened */ diff --git a/src/stage/interact/ui-location-solver.hpp b/src/stage/interact/ui-location-solver.hpp index 3312ed199..68d0e49af 100644 --- a/src/stage/interact/ui-location-solver.hpp +++ b/src/stage/interact/ui-location-solver.hpp @@ -277,7 +277,7 @@ namespace interact { // append ID of the new element to be created // unless it's already there (and thus exists) resolver.append (elementTypeID); - return move (resolver); //////////////////////////////////////////////////////////////////TICKET #1402 : need a better solution for the builder-terminal-op. (it collides with the templated UICoord ctor) |NOTE: the compiler waring is wrong here, the move() is essential + return move (resolver); //////////////////////////////////////////////////////////////////TICKET #1402 : need a better solution for the builder-terminal-op. (it collides with the templated UICoord ctor) |NOTE: the compiler warning is wrong here, the move() is essential // use the first suitable solution and exit } else diff --git a/src/stage/model/zoom-metric.hpp b/src/stage/model/zoom-metric.hpp index 27b6b8a09..c8d11544a 100644 --- a/src/stage/model/zoom-metric.hpp +++ b/src/stage/model/zoom-metric.hpp @@ -64,7 +64,7 @@ namespace model { class ZoomMetric : public I { - static_assert(lib::meta::is_Subclass()); + static_assert(lib::meta::is_Subclass_v); protected: ZoomWindow zoomWindow_; diff --git a/src/stage/notification-service.cpp b/src/stage/notification-service.cpp index ef6fc2f62..90fbff503 100644 --- a/src/stage/notification-service.cpp +++ b/src/stage/notification-service.cpp @@ -57,7 +57,7 @@ #include "lib/util.hpp" extern "C" { -#include "common/interface-descriptor.h" +#include "vessel/interface-descriptor.h" } #include diff --git a/src/stage/notification-service.hpp b/src/stage/notification-service.hpp index 7d88da63e..7c1596d45 100644 --- a/src/stage/notification-service.hpp +++ b/src/stage/notification-service.hpp @@ -32,7 +32,7 @@ #include "include/gui-notification-facade.h" -#include "common/instancehandle.hpp" +#include "vessel/instancehandle.hpp" #include "stage/ctrl/bus-term.hpp" #include @@ -77,9 +77,9 @@ namespace stage { /* === Interface Lifecycle === */ - using ServiceInstanceHandle = lumiera::InstanceHandle< LUMIERA_INTERFACE_INAME(lumieraorg_GuiNotification, 0) - , GuiNotification - > ; + using ServiceInstanceHandle = vessel::InstanceHandle< LUMIERA_INTERFACE_INAME(lumieraorg_GuiNotification, 0) + , GuiNotification + > ; ServiceInstanceHandle serviceInstance_; diff --git a/src/stage/output/displayer.cpp b/src/stage/output/displayer.cpp index a80fbcc2a..e7f8613b6 100644 --- a/src/stage/output/displayer.cpp +++ b/src/stage/output/displayer.cpp @@ -38,7 +38,7 @@ namespace output { DisplayerInput Displayer::format() { - return lumiera::DISPLAY_NONE; + return vessel::spine::DISPLAY_NONE; } void diff --git a/src/stage/output/displayer.hpp b/src/stage/output/displayer.hpp index b10d4f106..9dece1eaf 100644 --- a/src/stage/output/displayer.hpp +++ b/src/stage/output/displayer.hpp @@ -28,12 +28,12 @@ #include "lib/nocopy.hpp" -#include "include/display-handles.hpp" +#include "vessel/spine/display-handles.hpp" namespace stage { namespace output { - using lumiera::DisplayerInput; + using vessel::spine::DisplayerInput; /** diff --git a/src/stage/output/pixbuf-displayer.hpp b/src/stage/output/pixbuf-displayer.hpp index 469affd3d..e186129f0 100644 --- a/src/stage/output/pixbuf-displayer.hpp +++ b/src/stage/output/pixbuf-displayer.hpp @@ -71,7 +71,7 @@ class PixbufDisplayer DisplayerInput format() override { - return lumiera::DISPLAY_RGB; + return vessel::spine::DISPLAY_RGB; } }; diff --git a/src/stage/output/xv-displayer.hpp b/src/stage/output/xv-displayer.hpp index 9cc1ae048..f69d75147 100644 --- a/src/stage/output/xv-displayer.hpp +++ b/src/stage/output/xv-displayer.hpp @@ -72,7 +72,7 @@ namespace output { DisplayerInput format() override { - return lumiera::DISPLAY_YUV; + return vessel::spine::DISPLAY_YUV; } diff --git a/src/stage/timeline/body-canvas-widget.cpp b/src/stage/timeline/body-canvas-widget.cpp index ad5b79da8..2483ae704 100644 --- a/src/stage/timeline/body-canvas-widget.cpp +++ b/src/stage/timeline/body-canvas-widget.cpp @@ -43,7 +43,7 @@ #include "stage/timeline/track-body.hpp" #include "stage/style-scheme.hpp" -#include "common/advice.hpp" +#include "vessel/advice.hpp" #include "lib/util.hpp" #include @@ -71,8 +71,8 @@ namespace timeline { const int MINIMAL_CONTENT_WIDTH_px = 100; /** request a pre-defined CSS style context for the track body */ - lumiera::advice::Request trackBodyStyle{"style(trackBody)"}; - lumiera::advice::Request trackRulerStyle{"style(trackRuler)"}; + vessel::advice::Request trackBodyStyle{"style(trackBody)"}; + vessel::advice::Request trackRulerStyle{"style(trackRuler)"}; cuString slopeClassName(int depth) diff --git a/src/stage/timeline/timeline-layout.cpp b/src/stage/timeline/timeline-layout.cpp index e997b5685..c2ce4ae4a 100644 --- a/src/stage/timeline/timeline-layout.cpp +++ b/src/stage/timeline/timeline-layout.cpp @@ -25,7 +25,7 @@ #include "stage/timeline/timeline-layout.hpp" #include "stage/timeline/track-body.hpp" -#include "common/advice.hpp" +#include "vessel/advice.hpp" #include "lib/time/timevalue.hpp" using lib::time::Time; diff --git a/src/stage/widget/video-display-widget.hpp b/src/stage/widget/video-display-widget.hpp index c749d3feb..1eb69a2d5 100644 --- a/src/stage/widget/video-display-widget.hpp +++ b/src/stage/widget/video-display-widget.hpp @@ -33,7 +33,7 @@ namespace stage { namespace widget { using stage::output::Displayer; - using lumiera::DisplayerInput; + using vessel::spine::DisplayerInput; /** diff --git a/src/stage/workspace/ui-style.cpp b/src/stage/workspace/ui-style.cpp index 5b7c79660..832d228d2 100644 --- a/src/stage/workspace/ui-style.cpp +++ b/src/stage/workspace/ui-style.cpp @@ -52,7 +52,7 @@ namespace workspace { * either from the default theme of via the given Lumiera icon search paths, * typically from `setup.ini`. * - * @see lumiera::Config + * @see vessel::spine::Config */ UiStyle::UiStyle() : Gtk::UIManager() diff --git a/src/stage/workspace/ui-style.hpp b/src/stage/workspace/ui-style.hpp index 600e1366f..21ee115a5 100644 --- a/src/stage/workspace/ui-style.hpp +++ b/src/stage/workspace/ui-style.hpp @@ -28,7 +28,7 @@ #define STAGE_WORKSPACE_UI_STYLE_H #include "stage/gtk-base.hpp" -#include "common/advice.hpp" +#include "vessel/advice.hpp" #include "lib/symbol.hpp" #include "lib/nocopy.hpp" @@ -49,7 +49,7 @@ namespace workspace { using lib::Literal; using std::string; - using StyleAdvice = lumiera::advice::Provision; + using StyleAdvice = vessel::advice::Provision; @@ -99,7 +99,7 @@ namespace workspace { * Sets the theme to use for the Lumiera GUI. * @param stylesheetName GTK CSS stylesheet to load from the resourceSearchPath_ * @throw error::Config if this stylesheet can't be resolved on the searchpath - * @see lumiera::Config + * @see vessel::spine::Config */ void setTheme (string const& stylesheetName); diff --git a/src/steam/asset/meta/time-grid.cpp b/src/steam/asset/meta/time-grid.cpp index e6d098366..2b4d42576 100644 --- a/src/steam/asset/meta/time-grid.cpp +++ b/src/steam/asset/meta/time-grid.cpp @@ -23,7 +23,7 @@ #include "lib/time/quantiser.hpp" #include "lib/time/timevalue.hpp" #include "lib/format-string.hpp" -#include "common/advice.hpp" +#include "vessel/advice.hpp" #include "lib/util.hpp" #include @@ -62,7 +62,7 @@ namespace meta { using lib::time::FixedFrameQuantiser; using std::dynamic_pointer_cast; - namespace advice = lumiera::advice; + namespace advice = vessel::advice; namespace { diff --git a/src/steam/asset/pipe.hpp b/src/steam/asset/pipe.hpp index d1ad00362..5ca1857f0 100644 --- a/src/steam/asset/pipe.hpp +++ b/src/steam/asset/pipe.hpp @@ -27,7 +27,7 @@ #include "steam/asset/struct.hpp" #include "steam/asset/procpatt.hpp" #include "steam/streamtype.hpp" -#include "common/query.hpp" +#include "vessel/query.hpp" #include @@ -36,7 +36,7 @@ namespace steam { namespace asset { - using lumiera::Query; + using vessel::Query; using std::string; class Pipe; diff --git a/src/steam/asset/struct-factory-impl.hpp b/src/steam/asset/struct-factory-impl.hpp index 01666240a..016e294d2 100644 --- a/src/steam/asset/struct-factory-impl.hpp +++ b/src/steam/asset/struct-factory-impl.hpp @@ -33,8 +33,8 @@ #include "steam/mobject/session/element-query.hpp" #include "steam/mobject/session/session-query.hpp" #include "steam/mobject/session/scope.hpp" -#include "common/config-rules.hpp" -#include "common/query.hpp" +#include "vessel/config-rules.hpp" +#include "vessel/query.hpp" #include "steam/asset/timeline.hpp" #include "steam/asset/sequence.hpp" #include "steam/asset/procpatt.hpp" @@ -60,7 +60,7 @@ namespace asset { using util::uNum; using util::isnil; using util::contains; - using lumiera::Query; + using vessel::Query; using LERR_(CAPABILITY_QUERY); using lib::query::extractID; diff --git a/src/steam/asset/struct.cpp b/src/steam/asset/struct.cpp index 71451aef3..a814d3a78 100644 --- a/src/steam/asset/struct.cpp +++ b/src/steam/asset/struct.cpp @@ -39,12 +39,12 @@ #include "lib/query-util.hpp" #include "lib/format-string.hpp" #include "include/logging.h" -#include "common/query.hpp" +#include "vessel/query.hpp" using util::_Fmt; using lib::Symbol; using lib::query::normaliseID; -using lumiera::query::QueryHandler; +using vessel::query::QueryHandler; using steam::ConfigResolver; using util::contains; diff --git a/src/steam/asset/struct.hpp b/src/steam/asset/struct.hpp index bb9d50007..317338b4d 100644 --- a/src/steam/asset/struct.hpp +++ b/src/steam/asset/struct.hpp @@ -54,7 +54,7 @@ #include "lib/symbol.hpp" #include "steam/asset.hpp" -#include "common/query.hpp" +#include "vessel/query.hpp" #include "lib/nocopy.hpp" #include @@ -67,7 +67,7 @@ namespace asset { using std::string; using std::unique_ptr; - using lumiera::Query; + using vessel::Query; using lib::Symbol; class Struct; diff --git a/src/steam/asset/typed-id.hpp b/src/steam/asset/typed-id.hpp index 4b6a2bcb5..73fc6b8d2 100644 --- a/src/steam/asset/typed-id.hpp +++ b/src/steam/asset/typed-id.hpp @@ -47,7 +47,7 @@ #include -namespace lumiera{ ///////TODO: shouldn't that be namespace lib? or steam? +namespace vessel { ///////TODO: shouldn't that be namespace lib? or steam? namespace query { /** diff --git a/src/steam/config-resolver.cpp b/src/steam/config-resolver.cpp index fc733c367..22fc26b6b 100644 --- a/src/steam/config-resolver.cpp +++ b/src/steam/config-resolver.cpp @@ -21,13 +21,13 @@ #include "lib/error.hpp" -#include "include/lifecycle.h" +#include "include/lifecycle.hpp" #include "steam/config-resolver.hpp" #include "steam/mobject/session/query/fake-configrules.hpp" #include "lib/depend-inject.hpp" -using lumiera::LifecycleHook; -using lumiera::ON_GLOBAL_INIT; +using vessel::LifecycleHook; +using vessel::ON_GLOBAL_INIT; namespace steam { @@ -66,7 +66,7 @@ namespace steam { -#include "common/query/defs-manager-impl.hpp" +#include "vessel/query/defs-manager-impl.hpp" @@ -80,7 +80,7 @@ namespace steam { #include "steam/asset/sequence.hpp" #include "steam/mobject/session/fork.hpp" -namespace lumiera{ +namespace vessel { namespace query { using steam::asset::Pipe; @@ -105,4 +105,4 @@ namespace query { template bool DefsManager::forget (PPipe const&); -}} // namespace lumiera::query +}} // namespace vessel::query diff --git a/src/steam/config-resolver.hpp b/src/steam/config-resolver.hpp index cdecb0de4..9f7800be5 100644 --- a/src/steam/config-resolver.hpp +++ b/src/steam/config-resolver.hpp @@ -25,7 +25,7 @@ ** @note this is placeholder code using a preliminary/mock implementation... don't take this code too literal! ** @todo clarify the relation of config query and query-for-defaults ///////////////TICKET #705 ** - ** @see lumiera::Query + ** @see vessel::Query ** @see mobject::session::DefsManager ** @see asset::StructFactory ** @see fake-configrules.hpp currently used dummy-implementation @@ -37,8 +37,8 @@ #ifndef STEAM_CONFIG_RESOLVER #define STEAM_CONFIG_RESOLVER -#include "common/query.hpp" -#include "common/config-rules.hpp" +#include "vessel/query.hpp" +#include "vessel/config-rules.hpp" #include "lib/depend.hpp" // types for explicit specialisations.... @@ -73,7 +73,7 @@ namespace steam { * Configured as Singleton (with hidden Implementation class) */ class ConfigResolver - : public lumiera::query::ConfigRules + : public vessel::query::ConfigRules { public: diff --git a/src/steam/control/command-op-closure.hpp b/src/steam/control/command-op-closure.hpp index fb8e676ca..0a4dc04bb 100644 --- a/src/steam/control/command-op-closure.hpp +++ b/src/steam/control/command-op-closure.hpp @@ -16,6 +16,16 @@ ** Implementation of the concrete (sub)-closure of a command, responsible for ** invoking the actual command operation with the concrete (binding) arguments. ** + ** @todo 4/2026 this header seems to be more of a concept draft from the early days. + ** At that time, the C++ language did not have much function tools, did not + ** support lambdas and tuples, and thus invoking a functor object and passing + ** an arbitrary sequence of stored arguments was an innovative concept to be + ** implemented from scratch. And during that development process, some ideas + ** regarding the specific handling of relevant argument types was intermingled + ** with the creation of generic general purpose tooling. However, with the + ** switch to C++23, in summer 2025, most of that old implementation was + ** removed and replaced by standard library facilities. The #activated_ + ** flag seems to be the only piece of actual functionality here. ** @see Command ** @see command-closure.hpp ** @see command-storage-holder.hpp @@ -28,110 +38,35 @@ #define CONTROL_COMMAND_OP_CLOSURE_H #include "lib/meta/function.hpp" -#include "lib/meta/function-closure.hpp" -#include "lib/meta/tuple-helper.hpp" -#include "lib/meta/tuple-record-init.hpp" -#include "lib/meta/maybe-compare.hpp" #include "steam/control/command-closure.hpp" #include "steam/control/argument-erasure.hpp" -#include "lib/typed-allocation-manager.hpp" -#include "lib/format-cout.hpp" +#include "lib/meta/tuple-helper.hpp" +#include "lib/nocopy.hpp" -#include -#include #include #include - - namespace steam { namespace control { using lib::meta::_Fun; using lib::meta::Tuple; - using lib::meta::BuildTupleAccessor; - using lib::meta::buildTuple; - using lib::meta::Nil; - using lib::TypedAllocationManager; - using std::function; - using std::ostream; using std::string; - - - - - /** Helper for accessing an individual function parameter */ - template - < typename TY - , class BASE - , class TUP - , uint idx - > - class ParamAccessor - : public BASE - { - TY & element() { return std::get (*this); } - TY const& element() const { return std::get (*this); } - - public: - using BASE::BASE; - - - ////////////////////TODO the real access operations (e.g. for serialising) go here - - /////////////////////////////////////////////////////////////TICKET #798 : we need to pick up arguments from a lib::diff::Record. - - ostream& - dump (ostream& output) const - { - return BASE::dump (output << element() << ','); - } - }; - - template - class ParamAccessor ///< used for recursion end of implementation functions - : public TUP - { - public: - ParamAccessor (TUP const& tup) - : TUP(tup) - { } - - /** we deliberately support immutable types as command arguments */ - ParamAccessor& operator= (ParamAccessor const&) =delete; - ParamAccessor& operator= (ParamAccessor&&) =delete; - ParamAccessor (ParamAccessor const&) =default; - ParamAccessor (ParamAccessor&&) =default; - - - ////////////////////TODO the recursion-end of the access operations goes here - - ostream& - dump (ostream& output) const - { - return output; - } - }; - - - /** * closure to deal with the actual command operation. * This includes holding the invocation parameter tuple */ template class OpClosure + : util::NonAssign { - using Args = _Fun::Args; - using Builder = BuildTupleAccessor; + using Args = _Fun::Args; - using ParamStorageTuple = Builder::Product; - - ParamStorageTuple params_; + Tuple params_; bool activated_; public: @@ -139,8 +74,8 @@ namespace control { OpClosure() - : params_(Tuple()) - , activated_(false) + : params_{} + , activated_{false} { } explicit @@ -149,13 +84,6 @@ namespace control { , activated_(true) { } - /** we deliberately support immutable types as command arguments */ - OpClosure& operator= (OpClosure const&) =delete; - OpClosure& operator= (OpClosure&&) =delete; - OpClosure (OpClosure const&) =default; - OpClosure (OpClosure&&) =default; - - bool isValid () const { @@ -166,36 +94,25 @@ namespace control { /** Core operation: use the embedded argument tuple for invoking a functor * @param unboundFunctor an function object, whose function arguments are * required to match the types of the embedded ParamStorageTuple - * @note ASSERTION failure if the function signature - * doesn't match the argument types tuple. - * @note the functor might actually \em modify the param values. + * @note the functor might actually _modify the param values._ * Thus this function can't be const. */ void invoke (CmdFunctor const& unboundFunctor) { - ArgTuple& paramTuple{params_}; - std::apply (unboundFunctor.getFun(), paramTuple); + std::apply (unboundFunctor.getFun(), params_); } - operator string() const { std::ostringstream buff; - params_.dump (buff << "OpClosure(" ); - - string dumped (buff.str()); - if (10 < dumped.length()) - // remove trailing comma... - return dumped.substr (0, dumped.length()-1) +")"; - else - return dumped+")"; + buff << "OpClosure"; + lib::meta::joinTupleParen (params_, buff); + return buff.str(); } }; - - }} // namespace steam::control #endif /*CONTROL_COMMAND_OP_CLOSURE_H*/ diff --git a/src/steam/control/command-setup.cpp b/src/steam/control/command-setup.cpp index 630029a54..d9aef6b7e 100644 --- a/src/steam/control/command-setup.cpp +++ b/src/steam/control/command-setup.cpp @@ -28,7 +28,7 @@ #include "lib/error.hpp" #include "include/logging.h" -#include "include/lifecycle.h" +#include "include/lifecycle.hpp" #include "steam/control/command.hpp" #include "steam/control/command-setup.hpp" #include "steam/control/command-instance-manager.hpp" @@ -45,8 +45,8 @@ using std::get; using std::function; using std::move; using lib::Symbol; -using lumiera::LifecycleHook; -using lumiera::ON_GLOBAL_INIT; +using vessel::LifecycleHook; +using vessel::ON_GLOBAL_INIT; using std::string; using util::unConst; using util::_Fmt; diff --git a/src/steam/control/command-simple-closure.hpp b/src/steam/control/command-simple-closure.hpp index 5ebd56085..a81dfdbb4 100644 --- a/src/steam/control/command-simple-closure.hpp +++ b/src/steam/control/command-simple-closure.hpp @@ -38,7 +38,10 @@ #define CONTROL_COMMAND_SIMPLE_CLOSURE_H #include "lib/typed-allocation-manager.hpp" +#include "steam/control/command-closure.hpp" +#include "steam/control/argument-erasure.hpp" #include "steam/control/command-op-closure.hpp" +#include "lib/meta/tuple-record-init.hpp" #include "lib/opaque-holder.hpp" #include @@ -50,6 +53,7 @@ namespace control { namespace err = lumiera::error; using lib::InPlaceBuffer; + using lib::meta::buildTuple; using std::string; diff --git a/src/steam/control/command-storage-holder.hpp b/src/steam/control/command-storage-holder.hpp index dbb0b8d0b..81de90f25 100644 --- a/src/steam/control/command-storage-holder.hpp +++ b/src/steam/control/command-storage-holder.hpp @@ -36,9 +36,12 @@ #define CONTROL_COMMAND_STORAGE_HOLDER_H #include "lib/typed-allocation-manager.hpp" +#include "steam/control/command-closure.hpp" +#include "steam/control/argument-erasure.hpp" #include "steam/control/command-op-closure.hpp" #include "steam/control/memento-tie.hpp" #include "steam/control/command-impl-clone-builder.hpp" +#include "lib/meta/tuple-record-init.hpp" #include "lib/opaque-holder.hpp" #include @@ -50,6 +53,7 @@ namespace control { namespace err = lumiera::error; using lib::InPlaceBuffer; + using lib::meta::buildTuple; using std::string; diff --git a/src/steam/control/memento-tie.hpp b/src/steam/control/memento-tie.hpp index 0c5fc7272..f8d4d9d7c 100644 --- a/src/steam/control/memento-tie.hpp +++ b/src/steam/control/memento-tie.hpp @@ -31,7 +31,6 @@ #ifndef CONTROL_MEMENTO_TIE_H #define CONTROL_MEMENTO_TIE_H -#include "lib/meta/maybe-compare.hpp" #include "lib/meta/function-closure.hpp" #include "steam/control/command-signature.hpp" #include "lib/item-wrapper.hpp" @@ -48,7 +47,6 @@ namespace control { using lib::meta::func::bindLast; using lib::meta::func::chained; - using lib::meta::equals_safeInvoke; using lib::wrapper::ItemWrapper; diff --git a/src/steam/control/session-command-interface-proxy.cpp b/src/steam/control/session-command-interface-proxy.cpp index 32595db90..25cfb7dcd 100644 --- a/src/steam/control/session-command-interface-proxy.cpp +++ b/src/steam/control/session-command-interface-proxy.cpp @@ -24,14 +24,14 @@ #include "include/session-command-facade.h" -#include "include/interfaceproxy.hpp" +#include "vessel/spine/interfaceproxy.hpp" namespace steam { namespace control{ /** static storage for the facade access front-end */ lib::Depend SessionCommand::facade; }} -namespace lumiera { +namespace vessel { namespace facade { @@ -65,4 +65,4 @@ namespace facade { template class Link; -}} //namespace facade::lumiera +}} //namespace vessel::facade diff --git a/src/steam/control/session-command-service.cpp b/src/steam/control/session-command-service.cpp index 7ff9c66ec..c0128900a 100644 --- a/src/steam/control/session-command-service.cpp +++ b/src/steam/control/session-command-service.cpp @@ -30,7 +30,7 @@ #include "lib/util.hpp" extern "C" { -#include "common/interface-descriptor.h" +#include "vessel/interface-descriptor.h" } #include diff --git a/src/steam/control/session-command-service.hpp b/src/steam/control/session-command-service.hpp index 8ffe6fd99..1ebd8c4c5 100644 --- a/src/steam/control/session-command-service.hpp +++ b/src/steam/control/session-command-service.hpp @@ -36,7 +36,7 @@ #include "include/session-command-facade.h" #include "steam/control/command-dispatch.hpp" #include "steam/control/command-instance-manager.hpp" -#include "common/instancehandle.hpp" +#include "vessel/instancehandle.hpp" #include "lib/diff/gen-node.hpp" #include "lib/nocopy.hpp" #include "lib/symbol.hpp" @@ -77,9 +77,9 @@ namespace control { /* === Interface Lifecycle === */ - using ServiceInstanceHandle = lumiera::InstanceHandle< LUMIERA_INTERFACE_INAME(lumieraorg_SessionCommand, 0) - , SessionCommand - >; + using ServiceInstanceHandle = vessel::InstanceHandle< LUMIERA_INTERFACE_INAME(lumieraorg_SessionCommand, 0) + , SessionCommand + >; ServiceInstanceHandle serviceInstance_; public: diff --git a/src/steam/control/steam-dispatcher.hpp b/src/steam/control/steam-dispatcher.hpp index 95ef51a9e..48f7448c8 100644 --- a/src/steam/control/steam-dispatcher.hpp +++ b/src/steam/control/steam-dispatcher.hpp @@ -45,7 +45,7 @@ #ifndef STEAM_CONTROL_PROC_DISPATCHER_H #define STEAM_CONTROL_PROC_DISPATCHER_H -#include "common/subsys.hpp" +#include "vessel/subsys.hpp" #include "lib/depend.hpp" #include "lib/sync.hpp" @@ -57,7 +57,7 @@ namespace steam { namespace control { using std::unique_ptr; - using lumiera::Subsys; + using vessel::Subsys; class DispatcherLoop; diff --git a/src/steam/control/stypemanager.cpp b/src/steam/control/stypemanager.cpp index 7f8450268..18763cc59 100644 --- a/src/steam/control/stypemanager.cpp +++ b/src/steam/control/stypemanager.cpp @@ -20,7 +20,7 @@ #include "steam/common.hpp" #include "steam/control/stypemanager.hpp" #include "steam/control/styperegistry.hpp" -#include "include/lifecycle.h" +#include "include/lifecycle.hpp" namespace steam { @@ -49,7 +49,7 @@ namespace control { STypeManager::reset() { reg_.reset(new Registry); - lumiera::LifecycleHook::trigger (ON_STREAMTYPES_RESET); + vessel::LifecycleHook::trigger (ON_STREAMTYPES_RESET); } /** \par @@ -111,12 +111,3 @@ namespace control { } }} // namespace steam::control - - -// ==== C interface for registering setup of basic stream type configuration ======= - -void -lumiera_StreamType_registerInitFunction (void setupFun(void)) -{ - lumiera::LifecycleHook (steam::control::ON_STREAMTYPES_RESET, setupFun); -} diff --git a/src/steam/control/stypemanager.hpp b/src/steam/control/stypemanager.hpp index f25ec7331..3de3d8f46 100644 --- a/src/steam/control/stypemanager.hpp +++ b/src/steam/control/stypemanager.hpp @@ -110,20 +110,4 @@ namespace proc_interface { } // namespace proc_interface - - -extern "C" { //TODO provide a separate header if some C code or plugin happens to need this... - - /** any stream type implementation, which needs to be present on the - * pristine default level (without any session specific configuration), - * should register a setup function, which will be called on each - * STypemanager::reset() - */ - void - lumiera_StreamType_registerInitFunction (void setupFun(void)); - - // TODO provide a C interface usable from such a setupFun to access the STypeManager registration functions. -} - - -#endif +#endif /*STEAM_CONTROL_STYPEMANAGER_H*/ diff --git a/src/steam/control/styperegistry.hpp b/src/steam/control/styperegistry.hpp index 1a02b9d81..2c75dd2ff 100644 --- a/src/steam/control/styperegistry.hpp +++ b/src/steam/control/styperegistry.hpp @@ -17,7 +17,7 @@ ** Only used in stypemanager.cpp and accompanying unit tests. ** ** @see control::STypeManager - ** @see lumiera::StreamType + ** @see steam::StreamType ** */ @@ -30,7 +30,7 @@ #include "steam/control/stypemanager.hpp" #include "lib/nocopy.hpp" -//#include "common/query.hpp" +//#include "vessel/query.hpp" //#include "lib/util.hpp" //#include "lib/p.hpp" diff --git a/src/steam/engine/buffer-provider.cpp b/src/steam/engine/buffer-provider.cpp deleted file mode 100644 index 8f73e3797..000000000 --- a/src/steam/engine/buffer-provider.cpp +++ /dev/null @@ -1,334 +0,0 @@ -/* - BufferProvider - Abstraction for Buffer management during playback/render - - Copyright (C) - 2008, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -* *****************************************************************/ - - -/** @file buffer-provider.cpp - ** Implementation details related to buffer lifecycle management in the render engine. - */ - - -#include "lib/error.hpp" -#include "steam/engine/buffer-provider.hpp" -#include "steam/engine/buffer-metadata.hpp" -#include "lib/util.hpp" - -using util::isSameAdr; - -namespace steam { -namespace engine { - - // storage for the default-marker constants - const TypeHandler TypeHandler::RAW{}; - const LocalTag LocalTag::UNKNOWN{}; - - - namespace { // impl. details and definitions - - const uint DEFAULT_DESCRIPTOR = 0; - - } - - LUMIERA_ERROR_DEFINE (BUFFER_MANAGEMENT, "Problem providing working buffers"); - - - /** build a new provider instance, managing a family of buffers. - * The metadata of these buffers is organised hierarchically based on - * chained hash values, using the #implementationID as a seed. - * @param implementationID symbolic ID setting these family of buffers apart. - */ - BufferProvider::BufferProvider (Literal implementationID) - : meta_(new BufferMetadata (implementationID)) - { } - - BufferProvider::~BufferProvider() { } - - - /** @internal verify the given descriptor. - * @return true if it corresponds to a buffer - * currently locked and usable by client code - */ - bool - BufferProvider::verifyValidity (BuffDescr const& bufferID) const - { - return meta_->isLocked (bufferID); - } - - - BuffDescr - BufferProvider::getDescriptorFor (size_t storageSize) - { - return BuffDescr (*this, meta_->key (storageSize)); - } - - - BuffDescr - BufferProvider::getDescriptorFor(size_t storageSize, TypeHandler specialTreatment) - { - return BuffDescr (*this, meta_->key (storageSize, specialTreatment)); - } - - - size_t - BufferProvider::getBufferSize (HashVal typeID) const - { - metadata::Key& typeKey = meta_->get (typeID); - return typeKey.storageSize(); - } - - - /** callback from implementation to build and enrol a BufferHandle, - * to be returned to the client as result of the #lockBuffer call. - * Performs the necessary metadata state transition leading from an - * abstract buffer type to a metadata::Entry corresponding to an - * actual buffer, which is locked for exclusive use by one client. - */ - BuffHandle - BufferProvider::buildHandle (HashVal typeID, Buff* storage, LocalTag const& localTag) - { - metadata::Key& typeKey = meta_->get (typeID); - metadata::Entry& entry = meta_->markLocked(typeKey, storage, localTag); - - return BuffHandle (BuffDescr(*this, entry), storage); - } - - - /** BufferProvider API: declare in advance the need for working buffers. - * This optional call allows client code to ensure the availability of the - * necessary working space, prior to starting the actual operations. The - * client may reasonably assume to get the actual number of buffers, as - * indicated by the return value. A provider may be able to handle - * various kinds of buffers (e.g. of differing size), which are - * distinguished by _the type embodied into_ the BuffDescr. - * @return maximum number of simultaneously usable buffers of this type, - * to be retrieved later through calls to #lockBuffer. - * @throw error::State when no buffer of this kind can be provided - * @note the returned count may differ from the requested count. - */ - uint - BufferProvider::announce (uint count, BuffDescr const& type) - { - uint actually_possible = prepareBuffers (count, type); - if (!actually_possible) - throw error::State ("unable to fulfil request for buffers" - ,LUMIERA_ERROR_BUFFER_MANAGEMENT); - return actually_possible; - } - - - /** BufferProvider API: retrieve a single buffer for exclusive use. - * This call actually claims a buffer of this type and marks it for - * use by client code. The returned handle allows for convenient access, - * but provides no automatic tracking or memory management. The client is - * explicitly responsible to invoke #releaseBuffer (which can be done directly - * on the BuffHandle). - * @return a copyable handle, representing this buffer and this usage transaction. - * @throw error::State when unable to provide this buffer - * @note this function may be used right away, without prior announcing, but then - * the client should be prepared for exceptions. The #announce operation allows - * to establish a reliably available baseline. - * @todo 2/2025 might need to accept a cache key as additional parameter //////////////////////////////////TICKET #1392 : get cache key from computation to the cache backend - */ - BuffHandle - BufferProvider::lockBuffer (BuffDescr const& type) - { - REQUIRE (was_created_by_this_provider (type)); - - return provideLockedBuffer (type); - } // is expected to call buildHandle() --> state transition - - - /** BufferProvider API: state transition to \em emitted state. - * Client code may signal a state transition through this optional operation. - * The actual meaning of an "emitted" buffer is implementation defined; similarly, - * some back-ends may actually do something when emitting a buffer (e.g. commit data - * to cache), while others just set a flag or do nothing at all. This state transition - * may be invoked at most once per locked buffer. - * @throw error::Fatal in case of invalid state transition sequence. Only a locked buffer - * may be emitted, and at most once. - * @warning by convention, emitting a buffer implies that the contained data is ready and - * might be used by other parts of the application. - * While an emitted buffer should not be modified anymore, it can still be accessed. - */ - void - BufferProvider::emitBuffer (BuffHandle const& handle) - { - metadata::Entry& metaEntry = meta_->get (handle.entryID()); - mark_emitted (metaEntry.parentKey(), metaEntry.localTag()); - metaEntry.mark(EMITTED); - } - - - /** BufferProvider API: declare done and detach. - * Client code is required to release \em each previously locked buffer eventually. - * @warning invalidates the BuffHandle, clients mustn't access the buffer anymore. - * Right after releasing, an access through the handle will throw; - * yet the buffer might be re-used and the handle become valid - * later on accidentally. - * @note EX_FREE - */ - void - BufferProvider::releaseBuffer (BuffHandle const& handle) - try { - metadata::Entry& metaEntry = meta_->get (handle.entryID()); - metaEntry.mark(FREE); // might invoke embedded dtor function - detachBuffer (metaEntry.parentKey(), metaEntry.localTag(), *handle); - meta_->release (metaEntry); - } - ERROR_LOG_AND_IGNORE (engine, "releasing a buffer from BufferProvider") - - - - /** @warning this operation locally modifies the metadata entry of a single buffer - * to attach a TypeHandler taking ownership of an object embedded within the buffer. - * The client is responsible for actually placement-constructing the object; moreover - * the client is responsible for any damage done to already existing buffer content. - * @note the buffer must be in locked state and the underlying buffer type must not define - * an non-trivial TypeDescriptor, because there is no clean way of superseding an - * existing TypeDescriptor, which basically is just a functor and possibly - * could perform any operation on buffer clean-up. - * @note EX_STRONG - */ - void - BufferProvider::attachTypeHandler (BuffHandle const& target, BuffDescr const& reference) - { - metadata::Entry& metaEntry = meta_->get (target.entryID()); - metadata::Entry& refEntry = meta_->get (reference); - REQUIRE (refEntry.isTypeKey()); - REQUIRE (!metaEntry.isTypeKey()); - if (!metaEntry.isLocked()) - throw error::Logic{"unable to attach an object because buffer isn't locked for use" - , LERR_(LIFECYCLE)}; - - metaEntry.useTypeHandlerFrom (refEntry); // EX_STRONG - } - - - /** @internal abort normal lifecycle, reset the underlying buffer and detach from it. - * This allows to break out of normal usage and reset the handle to _invalid state_ - * @param invokeDtor if possibly the clean-up function of an TypeHandler registered with - * the buffer metadata should be invoked prior to resetting the metadata state. - * Default is \em not to invoke anything - * @note EX_FREE - */ - void - BufferProvider::emergencyCleanup (BuffHandle const& target, bool invokeDtor) - try { - metadata::Entry& metaEntry = meta_->get (target.entryID()); - metaEntry.invalidate (invokeDtor); - detachBuffer (metaEntry.parentKey(), metaEntry.localTag(), *target); - meta_->release (metaEntry); - } - ERROR_LOG_AND_IGNORE (engine, "cleanup of buffer metadata while handling an error") - - - - bool - BufferProvider::was_created_by_this_provider (BuffDescr const& descr) const - { - return isSameAdr (this, descr.provider_); - } - - - - - - /* === BuffDescr and BuffHandle === */ - - bool - BuffDescr::verifyValidity() const - { - ENSURE (provider_); - return provider_->verifyValidity(*this); - } - - - size_t - BuffDescr::determineBufferSize() const - { - ENSURE (provider_); - return provider_->getBufferSize (*this); - } - - - uint - BuffDescr::announce (uint count) - { - ENSURE (provider_); - return provider_->announce(count, *this); - } - - - BuffHandle - BuffDescr::lockBuffer() - { - ENSURE (provider_); - return provider_->lockBuffer(*this); - } - - - void - BuffHandle::emit() - { - REQUIRE (isValid()); - descriptor_.provider_->emitBuffer(*this); - } - - - void - BuffHandle::release() - { - if (pBuffer_) - { - REQUIRE (isValid()); - descriptor_.provider_->releaseBuffer(*this); - pBuffer_ = 0; - } - ENSURE (!isValid()); - } - - - void - BuffHandle::emergencyCleanup() - { - descriptor_.provider_->emergencyCleanup(*this); // EX_FREE - pBuffer_ = 0; - } - - - /** Install a standard TypeHandler for an already locked buffer. - * This causes the dtor function to be invoked when releasing this buffer. - * The assumption is that client code will placement-construct an object - * into this buffer right away, and thus we're taking ownership on that object. - * @param type a reference BuffDescr defining an embedded TypeHandler to use - * A copy of this TypeHandler will be stored into the local metadata for - * this buffer only, not altering the basic buffer type in any way - * @throw lifecycle error when attempting to treat an buffer not in locked state - * @throw error::Logic in case of insufficient buffer space to hold the - * intended target object - * @note EX_STRONG - */ - void - BuffHandle::takeOwnershipFor(BuffDescr const& type) - { - if (!this->isValid()) - throw error::Logic ("attaching an object requires an buffer in locked state", LERR_(LIFECYCLE)); - if (this->size() < type.determineBufferSize()) - throw error::Logic ("insufficient buffer size to hold an instance of that type"); - - descriptor_.provider_->attachTypeHandler(*this, type); // EX_STRONG - } - - - - -}} // namespace engine diff --git a/src/steam/engine/buffer-provider.hpp b/src/steam/engine/buffer-provider.hpp deleted file mode 100644 index 6f24bc314..000000000 --- a/src/steam/engine/buffer-provider.hpp +++ /dev/null @@ -1,168 +0,0 @@ -/* - BUFFER-PROVIDER.hpp - Abstraction for Buffer management during playback/render - - Copyright (C) - 2011, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -*/ - -/** @file buffer-provider.hpp - ** Abstraction to represent buffer management and lifecycle within the render engine. - ** It turns out that -- throughout the render engine implementation -- we never need - ** direct access to the buffers holding media data. Buffers are just some entity to be \em managed, - ** i.e. "allocated", "locked" and "released"; the actual meaning of these operations is an implementation detail. - ** The code within the render engine just pushes around BufferHandle objects, which act as a front-end, - ** being created by and linked to a BufferProvider implementation. There is no need to manage the lifecycle - ** of buffers automatically, because the use of buffers is embedded into the render calculation cycle, - ** which follows a rather strict protocol anyway. Relying on the capabilities of the scheduler, - ** the sequence of individual jobs in the engine ensures... - ** - that the availability of a buffer was ensured prior to planning a job ("buffer allocation") - ** - that a buffer handle was obtained ("locked") prior to any operation requiring a buffer - ** - that buffers are marked as free ("released") after doing the actual calculations. - ** - ** @see state.hpp - ** @see output-slot.hpp - */ - -#ifndef STEAM_ENGINE_BUFFR_PROVIDER_H -#define STEAM_ENGINE_BUFFR_PROVIDER_H - - -#include "lib/error.hpp" -#include "lib/symbol.hpp" -#include "lib/hash-value.h" -#include "steam/engine/buffhandle.hpp" -//#include "steam/engine/engine-ctx.hpp" -#include "steam/engine/type-handler.hpp" -#include "steam/engine/buffer-local-tag.hpp" -#include "lib/nocopy.hpp" - -#include -#include - - -namespace steam { -namespace engine { - - using lib::Literal; - using std::unique_ptr; - using std::forward; - - - class BufferMetadata; - - - LUMIERA_ERROR_DECLARE (BUFFER_MANAGEMENT); ///< Problem providing working buffers - - - /** - * Interface: a facility providing and managing working buffers for media calculations. - * The pointer to actual buffer storage can be retrieved by - * - optionally announcing the required buffer(s) beforehand - * - "locking" a buffer to yield a buffer handle - * - then dereferencing the obtained smart-handle - * - * @warning all of BufferProvider is assumed to run within a threadsafe environment. - * - * @todo as of 6/2011 buffer management within the engine is still a bit vague - * @todo as of 11/11 thread safety within the engine remains to be clarified ///////////////////////////TICKET #854 - */ - class BufferProvider - : util::NonCopyable - { - unique_ptr meta_; - - - protected: /* === for Implementation by concrete providers === */ - - /// placeholder marker type for an actual data buffer - using Buff = StreamType::ImplFacade::DataBuffer; - - BufferProvider (Literal implementationID); - - virtual uint prepareBuffers (uint count, HashVal typeID) =0; - - virtual BuffHandle provideLockedBuffer (HashVal typeID) =0; - virtual void mark_emitted (HashVal, LocalTag const&) =0; - virtual void detachBuffer (HashVal, LocalTag const&, Buff&) =0; - - - public: - virtual ~BufferProvider(); ///< this is an ABC - - - uint announce (uint count, BuffDescr const&); - - BuffHandle lockBuffer (BuffDescr const&); - void emitBuffer (BuffHandle const&); - void releaseBuffer (BuffHandle const&); - - template - BuffHandle lockBufferFor (ARGS ...args); - - /** allow for attaching and owing an object within an already created buffer */ - void attachTypeHandler (BuffHandle const& target, BuffDescr const& reference); - - void emergencyCleanup (BuffHandle const& target, bool invokeDtor =false); - - - /** describe the kind of buffer managed by this provider */ - BuffDescr getDescriptorFor(size_t storageSize=0); - BuffDescr getDescriptorFor(size_t storageSize, TypeHandler specialTreatment); - - template - BuffDescr getDescriptor (ARGS ...args); - - - - /* === API for BuffHandle internal access === */ - - bool verifyValidity (BuffDescr const&) const; - size_t getBufferSize (HashVal typeID) const; - - protected: - BuffHandle buildHandle (HashVal typeID, Buff* storage, LocalTag const& =LocalTag::UNKNOWN); - - bool was_created_by_this_provider (BuffDescr const&) const; - }; - - - - - /* === Implementation === */ - - /** convenience shortcut: - * prepare and claim ("lock") a buffer suitable - * to hold an object of the given type. - * @return a handle embedding a suitably configured - * buffer descriptor. The corresponding buffer - * has been allocated and marked for exclusive use - */ - template - BuffHandle - BufferProvider::lockBufferFor (ARGS ...args) - { - BuffDescr attach_object_automatically = getDescriptor (forward (args)...); - return lockBuffer (attach_object_automatically); - } - - - /** define a "buffer type" for automatically creating - * an instance of the template type embedded into the buffer - * and destroying that embedded object when releasing the buffer. - */ - template - BuffDescr - BufferProvider::getDescriptor (ARGS ...args) - { - return getDescriptorFor (sizeof(BU), TypeHandler::create (forward (args)...)); - } - - -}} // namespace steam::engine -#endif /*STEAM_ENGINE_BUFFR_PROVIDER_H*/ diff --git a/src/steam/engine/buffer-proxy-provider.cpp b/src/steam/engine/buffer-proxy-provider.cpp deleted file mode 100644 index f26628109..000000000 --- a/src/steam/engine/buffer-proxy-provider.cpp +++ /dev/null @@ -1,55 +0,0 @@ -/* - BufferProxyProvider - Adapt existing allocation for access through the Buffer protocol - - Copyright (C) - 2024, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -* *****************************************************************/ - - -/** @file buffer-proxy-provider.cpp - ** Implementation details of a forwarding BufferProvider stub implementation. - */ - - -#include "lib/error.hpp" -#include "steam/engine/buffer-proxy-provider.hpp" -//#include "steam/engine/buffer-metadata.hpp" -//#include "lib/util.hpp" - -//using util::isSameAdr; - -namespace steam { -namespace engine { - - // storage for the default-marker constants -// const TypeHandler TypeHandler::RAW{}; - - - namespace { // impl. details and definitions - -// const uint DEFAULT_DESCRIPTOR = 0; - - } - - - /** build a new - */ -// BufferProvider::BufferProvider (Literal implementationID) -// : meta_(new BufferMetadata (implementationID)) -// { } -// -// BufferProvider::~BufferProvider() { } - - - /** @internal verify the given descriptor. - */ - - - -}} // namespace engine diff --git a/src/steam/engine/buffer-proxy-provider.hpp b/src/steam/engine/buffer-proxy-provider.hpp deleted file mode 100644 index 4b724f242..000000000 --- a/src/steam/engine/buffer-proxy-provider.hpp +++ /dev/null @@ -1,129 +0,0 @@ -/* - BUFFER-PROXY-PROVIDER.hpp - Adapter to access existing allocation via buffer handling protocol - - Copyright (C) - 2024, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -*/ - -/** @file buffer-proxy-provider.hpp - ** Adapter to expose a given memory block through a BuffHandle. - ** This allows to integrate a specific data access (e.g. related to input / output) - ** with the buffer lifecycle protocol as defined by BufferProvider. - ** @todo BROKEN as of 12/2024 //////////////////////////////////////////////////////////////////////////////TICKET #1387 : can not properly compose BufferProvider - ** @see output-slot.hpp - */ - -#ifndef STEAM_ENGINE_BUFFER_PROXY_PROVIDER_H -#define STEAM_ENGINE_BUFFER_PROXY_PROVIDER_H - - -#include "lib/error.hpp" -#include "lib/symbol.hpp" -#include "lib/meta/util.hpp" -//#include "lib/hash-value.h" -#include "steam/engine/buffer-provider.hpp" -#include "steam/engine/buffer-metadata.hpp" -//#include "steam/engine/engine-ctx.hpp" -//#include "steam/engine/type-handler.hpp" -//#include "steam/engine/buffer-local-tag.hpp" -#include "lib/nocopy.hpp" - -#include -#include -//#include - - -namespace steam { -namespace engine { - - using lib::Literal; -// using std::unique_ptr; -// using std::forward; - - - - - /** - * Adapter to expose access to data blocks via BuffHandle and the BufferProvider protocol. - * @todo WIP-WIP 12/2024 this is a design sketch to explore extension capabilities of BufferProvider - */ - class BufferProxyProvider - : util::NonCopyable - { - - using Listener = std::function; - - class ForwardingBufferProvider - : public BufferProvider - { - Listener listener_; - - /* === BufferProvider API === */ - - uint - prepareBuffers (uint, HashVal) override - { - NOTREACHED ("this part of the API should not be used"); - return 1; // can not sensibly do anything for "pre-allocation", - } // other than telling the caller that we only "have one buffer to provide" - - BuffHandle - provideLockedBuffer (HashVal typeID) override - { - /////////////////////////////////////////////////////////////////////////////////////TICKET #1387 : BufferProvider default impl. is lacking means to compose and delegate -// return buildHandle (typeID, asBuffer(newBlock.accessMemory()), &newBlock); - } - - void - mark_emitted (HashVal, LocalTag const&) override - { - - } - - void - detachBuffer (HashVal, LocalTag const&, Buff&) override - { - - } - public: - ForwardingBufferProvider (Listener listener) - : BufferProvider{"BufferProxyProvider"} - , listener_{std::move (listener)} - { } - }; - - ForwardingBufferProvider passThroughProvider_; - - - public: - template> - BufferProxyProvider (LIS&& listener) - : passThroughProvider_{std::forward (listener)} - { } - - template - BuffHandle - lockBuffer (TAR& dataBlock) - { - //////////////////////////////////////////////////////////////////////////////////////////////////TICKET #1387 : impossible due to inner contradictions in BufferProvider and OutputSlot - UNIMPLEMENTED ("setup type handler and then create a locked BuffHandle"); - } - - }; - - - - - /* === Implementation === */ - - /** convenience shortcut: */ - - -}} // namespace steam::engine -#endif /*STEAM_ENGINE_BUFFER_PROXY_PROVIDER_H*/ diff --git a/src/steam/engine/buffhandle-attach.hpp b/src/steam/engine/buffhandle-attach.hpp deleted file mode 100644 index b5c5b7c56..000000000 --- a/src/steam/engine/buffhandle-attach.hpp +++ /dev/null @@ -1,121 +0,0 @@ -/* - BUFFHANDLE-ATTACH.hpp - Buffer handle extension to attach objects into the buffer - - Copyright (C) - 2008, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -*/ - -/** @file buffhandle-attach.hpp - ** Extension to allow placing objects right into the buffers, taking ownership. - ** This extension is mostly helpful for writing unit-tests, and beyond that for the - ** rather unusual case where we need to place an full-blown object into the buffer, - ** instead of just plain data. A possible use case for this mechanism is to allow for - ** state per calculation stream, feeding this local state to the individual render node - ** embedded into a "state frame". Some effect processors indeed need to maintain state - ** beyond the single frame (e.g. averaging, integrating, sound compression), which usually - ** is handled by applying an "instance" of that processor to the frames to be calculated - ** in a straight sequence. - ** - ** BuffHandle and the underlying BufferProvider standard implementation support that case - ** by attaching an object managing functor to the metadata. This way, the state can live - ** directly embedded into the frame and still be accessed like an object. To keep the - ** header and compilation footprint low, the implementation of the functions supporting - ** this special case was split out of the basic buffhandle.hpp - ** - ** @see BuffHandle - ** @see BufferProviderProtocol_test usage demonstration - */ - -#ifndef ENGINE_BUFFHANDLE_ATTACH_H -#define ENGINE_BUFFHANDLE_ATTACH_H - - -#include "lib/error.hpp" -#include "steam/engine/buffer-provider.hpp" -#include "steam/engine/buffhandle.hpp" - - -namespace steam { -namespace engine { - - - - - /* === BuffHandle Implementation === */ - - -#define _EXCEPTION_SAFE_INVOKE(_CTOR_) \ - try \ - { \ - return *new(pBuffer_) _CTOR_; \ - } \ - catch(...) \ - { \ - emergencyCleanup(); /* EX_FREE */ \ - pBuffer_ = 0; \ - throw; \ - } - - /** convenience shortcut: place and maintain an object within the buffer. - * This operation performs the necessary steps to attach an object; - * if the buffer isn't locked yet, it will do so. Moreover, the created - * object will be owned by the buffer management facilities, i.e. the - * destructor is registered as cleanup function. - * @throw error::Logic in case there is already another TypeHandler registered - * in charge of managing the buffer contents, or when the object to create - * would not fit into this buffer. - */ - template - inline BU& - BuffHandle::create() - { - takeOwnershipFor(); - _EXCEPTION_SAFE_INVOKE (BU()); - } - -#undef _EXCEPTION_SAFE_INVOKE - - - - /** @internal helper to attach an TypeHandler after-the fact. - * @note this prepares the buffer for placement-creating an embedded object. - * It doesn't actually create an object - * @throw error::Logic in case there is already another TypeHandler registered - * in charge of managing the buffer contents, or when the object to create - * would not fit into this buffer. - */ - template - inline void - BuffHandle::takeOwnershipFor() - { - BuffDescr howto_attach_object_automatically - = descriptor_.provider_->getDescriptor(); - takeOwnershipFor (howto_attach_object_automatically); // EX_STRONG - } - - - /** convenience shortcut: access the buffer contents casted to a specific type. - * @warning this is a \em blind cast, there is no type safety. - * @note clients can utilise the metadata::LocalTag to keep track of some - * specific property of the buffer, like e.g. the type of object. - */ - template - inline BU& - BuffHandle::accessAs() - { - if (!pBuffer_) - throw error::Logic ("buffer not (yet) locked for access by clients" - , LERR_(LIFECYCLE)); - return *reinterpret_cast (pBuffer_); - } - - - -}} // namespace steam::engine -#endif diff --git a/src/steam/engine/buffhandle.hpp b/src/steam/engine/buffhandle.hpp deleted file mode 100644 index 4b62d5ce0..000000000 --- a/src/steam/engine/buffhandle.hpp +++ /dev/null @@ -1,185 +0,0 @@ -/* - BUFFHANDLE.hpp - Buffer handling support for the render engine - - Copyright (C) - 2008, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -*/ - -/** @file buffhandle.hpp - ** A front-end to support the buffer management within the render nodes. - ** When pulling data from predecessor nodes and calculating new data, each render node - ** needs several input and output buffers. These may be allocated and provided by various - ** different "buffer providers" (for example the frame cache). Typically, the real buffers - ** will be passed as parameters to the actual job instance when scheduled, drawing on the - ** results of prerequisite jobs. Yet the actual job implementation remains agnostic with - ** respect to the way actual buffers are provided; the invocation just pushes BuffHandle - ** objects around. The actual render function gets an array of C-pointers to the actual - ** buffers, and for accessing those buffers, the node needs to keep a table of buffer - ** pointers, and for releasing the buffers later on, we utilise the buffer handles. - ** - ** These buffer handles are based on a [Buffer Descriptor record](\ref BuffDescr), - ** which is opaque as far as the client is concerned. BuffDescr acts as a representation - ** of the type or kind of buffer. The only way to obtain such a BuffDescr is from a concrete - ** BufferProvider implementation. A back-link to this owning and managing provider is embedded - ** into the BuffDescr, which thus may be used as a _configuration tag,_ allowing to retrieve a - ** concrete buffer handle when needed, corresponding to an actual buffer provided and managed - ** behind the scenes. There is no automatic resource management; clients are responsible to - ** invoke BuffHandle#release when done. - ** - ** @warning buffer management via BuffHandle and BuffDescr does _not automatically maintain - ** proper alignment._ Rather, it relies on the storage allocator to provide a buffer - ** suitably aligned for the target type to hold. In most cases, this target location - ** will actually be storage maintained on heap through some STL collection; - ** this topic is a possible subtle pitfall non the less. - ** - ** @see BufferProvider - ** @see BufferProviderProtocol_test usage demonstration - ** @see OutputSlot - ** @see bufftable.hpp storage for the buffer table /////////////////////OOO where to point into the new structure for documentation? - ** @see engine::RenderInvocation - */ - -#ifndef ENGINE_BUFFHANDLE_H -#define ENGINE_BUFFHANDLE_H - - -#include "lib/error.hpp" -#include "lib/hash-value.h" -#include "steam/streamtype.hpp" - - -namespace steam { -namespace engine { - namespace error = lumiera::error; - - using lib::HashVal; - - class BuffHandle; - class BufferProvider; - - - - /** - * An opaque descriptor to identify the type and further properties of a data buffer. - * For each kind of buffer, there is somewhere a BufferProvider responsible for the - * actual storage management. This provider may "lock" a buffer for actual use, - * returning a BuffHandle. - * @note this descriptor and especially the #subClassification_ is really owned - * by the BufferProvider, which may use (and even change) the opaque contents - * to organise the internal buffer management. - */ - class BuffDescr - { - protected: - BufferProvider* provider_; - HashVal subClassification_; - - BuffDescr(BufferProvider& manager, HashVal detail) - : provider_(&manager) - , subClassification_(detail) - { } - - friend class BufferProvider; - friend class BuffHandle; - - public: - // using standard copy operations - - bool verifyValidity() const; - size_t determineBufferSize() const; - - operator HashVal() const { return subClassification_; } - - /** convenience shortcut to start a buffer handling cycle */ - uint announce (uint count); - BuffHandle lockBuffer(); - }; - - - - - /** - * Handle for a buffer for processing data, abstracting away the actual implementation. - * The real buffer pointer can be retrieved by dereferencing this smart-handle class. - */ - class BuffHandle - { - using Buff = StreamType::ImplFacade::DataBuffer; ///< marker type for an actual data buffer - - BuffDescr descriptor_; - Buff* pBuffer_; - - - public: - using PBuff = Buff*; - - /** @internal a buffer handle may be obtained by "locking" - * a buffer from the corresponding BufferProvider */ - BuffHandle(BuffDescr const& typeInfo, void* storage = 0) - : descriptor_(typeInfo) - , pBuffer_(static_cast(storage)) - { } - - // using standard copy operations - - explicit operator bool() const { return isValid(); } - - - void emit(); - void release(); - - - template - BU& create(); - - template - BU& accessAs(); - - - //////////////////////////////////////////TICKET #249 this operator looks obsolete. The Buff type is a placeholder type, - //////////////////////////////////////////TODO it should never be accessed directly from within Lumiera engine code - Buff& - operator* () const - { - ENSURE (pBuffer_); - return *pBuffer_; - } - - bool - isValid() const - { - return bool(pBuffer_) - and descriptor_.verifyValidity(); - } - - HashVal - entryID() const - { - return HashVal(descriptor_); - } - - size_t - size() const - { - return descriptor_.determineBufferSize(); - } - - private: - template - void takeOwnershipFor(); - void takeOwnershipFor(BuffDescr const& type); - - void emergencyCleanup(); - }; - - - - -}} // namespace steam::engine -#endif diff --git a/src/steam/engine/bufftable.hpp b/src/steam/engine/bufftable.hpp deleted file mode 100644 index 4b7d16088..000000000 --- a/src/steam/engine/bufftable.hpp +++ /dev/null @@ -1,158 +0,0 @@ -/* - BUFFTABLE.hpp - Table of buffer pointers to be used by the render nodes - - Copyright (C) - 2008, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -*/ - - -/** @file bufftable.hpp - ** helper for organisation of render data buffers - ** Used during the process of _"pulling"_ a render node, which recursively might - ** pull further nodes. Any node has a _render calculation function,_ which in turn - ** needs input and output buffers - ** @see buffer-provider.hpp - */ - - -#ifndef ENGINE_BUFFHTABLE_H -#define ENGINE_BUFFHTABLE_H - - -#include "lib/error.hpp" -#include "steam/engine/buffhandle.hpp" -#include "steam/engine/proc-node.hpp" -#include "lib/iter-adapter.hpp" - -#include -#include - - - -namespace steam { -namespace engine { - - using std::pair; - using std::vector; - - - /** - * Tables of buffer handles and corresponding dereferenced buffer pointers. - * Used within the invocation of a processing node to calculate data. - * The tables are further differentiated into input data buffers and output - * data buffers. The tables are supposed to be implemented as bare "C" arrays, - * thus the array of real buffer pointers can be fed directly to the - * processing function of the respective node. - * - * @todo this whole design is a first attempt and rather clumsy. It should be reworked - * to use a single contiguous memory area and just layer the object structure on top - * (by using placement new). Yet the idea of an stack-like organisation should be retained - */ - struct BuffTable - { - using PHa = BuffHandle *; - using PBu = BuffHandle::PBuff *; - - struct StorageChunk - { }; - - template - struct Storage - { - enum{size = count * sizeof(StorageChunk)}; - }; - - class Builder - { - public: - Builder& announce (uint count, BuffDescr const& type); - BuffTable& build(); - }; - - static Builder& prepare (const size_t STORAGE_SIZE, void* storage); - - void lockBuffers(); - void releaseBuffers(); - - using BuffHandleTable = vector; - using iterator = lib::RangeIter; - - iterator buffers(); - iterator inBuffers(); - iterator outBuffers(); - }; - - - - - /* === Implementation === */ - - inline BuffTable::Builder& - BuffTable::prepare (const size_t STORAGE_SIZE, void* storage) - { - UNIMPLEMENTED ("expose a builder object for outfitting a buffer pointer table"); - } - - - inline BuffTable::Builder& - BuffTable::Builder::announce (uint count, BuffDescr const& type) - { - UNIMPLEMENTED ("accept announcement of additional buffer table entries required"); - } - - - inline BuffTable& - BuffTable::Builder::build() - { - UNIMPLEMENTED ("finally drop off the newly configured buffer pointer table"); - } - - - inline void - BuffTable::lockBuffers() - { - UNIMPLEMENTED ("convenience shortcut: lock all preconfigured buffers within this table through the underlying buffer provider"); - } - - - inline void - BuffTable::releaseBuffers() - { - UNIMPLEMENTED ("convenience shortcut: release all the buffers managed through this buffer table, by forwarding to the underlying buffer provider"); - } - - - - inline BuffTable::iterator - BuffTable::buffers() - { - UNIMPLEMENTED ("expose an iterator to yield all prepared buffers within this buffer table"); - } - - - inline BuffTable::iterator - BuffTable::inBuffers() - { - UNIMPLEMENTED ("expose an iterator to access all the input buffer slots of this buffer table"); - } - - - inline BuffTable::iterator - BuffTable::outBuffers() - { - UNIMPLEMENTED ("expose an iterator to access all the output buffer slots of this buffer table"); - } - - - - - - -}} // namespace steam::engine -#endif diff --git a/src/steam/engine/calc-stream.hpp b/src/steam/engine/calc-stream.hpp index 6eaa711c9..359cd80a8 100644 --- a/src/steam/engine/calc-stream.hpp +++ b/src/steam/engine/calc-stream.hpp @@ -37,12 +37,11 @@ #include "lib/error.hpp" -#include "steam/play/timings.hpp" -#include "steam/play/output-slot.hpp" +#include "vault/out/output-slot.hpp" #include "steam/engine/render-drive.hpp" //#include "include/dummy-player-facade.h" //#include "include/display-facade.h" -//#include "common/instancehandle.hpp" +//#include "vessel/instancehandle.hpp" //#include "lib/singleton-ref.hpp" // //#include @@ -56,9 +55,9 @@ namespace engine{ namespace error = lumiera::error; // using std::string; -// using lumiera::Subsys; -// using lumiera::Display; -// using lumiera::DummyPlayer; +// using vessel::Subsys; +// using vessel::Display; +// using vessel::DummyPlayer; class RenderEnvironment; @@ -98,7 +97,7 @@ namespace engine{ void - sendToOutput (play::DataSink) + sendToOutput (vault::out::DataSink) { UNIMPLEMENTED ("set up dispatcher to start calculating and feeding to the given output sink"); } diff --git a/src/steam/engine/diagnostic-buffer-provider.cpp b/src/steam/engine/diagnostic-buffer-provider.cpp deleted file mode 100644 index 82429073f..000000000 --- a/src/steam/engine/diagnostic-buffer-provider.cpp +++ /dev/null @@ -1,109 +0,0 @@ -/* - DiagnosticBufferProvider - helper for testing against the BufferProvider interface - - Copyright (C) - 2011, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -* *****************************************************************/ - - -/** @file diagnostic-buffer-provider.cpp - ** Implementation details of unit test support regarding internals of the render engine. - ** - ** @todo WIP from 2013, half finished, stalled - ** @see buffer-provider-protocol-test.cpp - */ - - -#include "lib/error.hpp" -#include "lib/meta/function.hpp" - -#include "steam/engine/diagnostic-buffer-provider.hpp" -#include "steam/engine/tracking-heap-block-provider.hpp" - - -namespace steam { -namespace engine { - - - /** Storage for the diagnostics frontend */ - lib::Depend DiagnosticBufferProvider::diagnostics; - - - - - DiagnosticBufferProvider::DiagnosticBufferProvider() - : pImpl_() - { } - - DiagnosticBufferProvider::~DiagnosticBufferProvider() { } - - - BufferProvider& - DiagnosticBufferProvider::build() - { - return diagnostics().reset(); - } - - - DiagnosticBufferProvider& - DiagnosticBufferProvider::access (BufferProvider const& provider) - { - if (!diagnostics().isCurrent (provider)) - throw error::Invalid("given Provider doesn't match (current) diagnostic data record." - "This might be an lifecycle error. Did you build() this instance beforehand?"); - - return diagnostics(); - } - - - - - TrackingHeapBlockProvider& - DiagnosticBufferProvider::reset() - { - if (pImpl_) - pImpl_->markAllEmitted(); - pImpl_.reset(new TrackingHeapBlockProvider()); - return *pImpl_; - } - - bool - DiagnosticBufferProvider::isCurrent (BufferProvider const& implInstance) - { - return &implInstance == pImpl_.get(); - } - - - - - - /* === diagnostic API === */ - - bool - DiagnosticBufferProvider::buffer_was_used (uint bufferID) const - { - return pImpl_->access_emitted(bufferID).was_used(); - } - - - bool - DiagnosticBufferProvider::buffer_was_closed (uint bufferID) const - { - return pImpl_->access_emitted(bufferID).was_closed(); - } - - - void* - DiagnosticBufferProvider::accessMemory (uint bufferID) const - { - return pImpl_->access_emitted(bufferID).accessMemory(); - } - - -}} // namespace engine diff --git a/src/steam/engine/diagnostic-buffer-provider.hpp b/src/steam/engine/diagnostic-buffer-provider.hpp deleted file mode 100644 index dc2178bd2..000000000 --- a/src/steam/engine/diagnostic-buffer-provider.hpp +++ /dev/null @@ -1,102 +0,0 @@ -/* - DIAGNOSTIC-BUFFER-PROVIDER.hpp - helper for testing against the BufferProvider interface - - Copyright (C) - 2011, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -*/ - -/** @file diagnostic-buffer-provider.hpp - ** A facility for writing unit-tests targeting the BufferProvider interface. - ** - ** @see buffer-provider-protocol-test.cpp - */ - -#ifndef STEAM_ENGINE_DIAGNOSTIC_BUFFR_PROVIDER_H -#define STEAM_ENGINE_DIAGNOSTIC_BUFFR_PROVIDER_H - - -#include "lib/error.hpp" -#include "lib/depend.hpp" -#include "lib/util.hpp" -#include "steam/engine/type-handler.hpp" -#include "steam/engine/buffer-provider.hpp" -#include "lib/nocopy.hpp" - -#include - - -namespace steam { -namespace engine { - - namespace error = lumiera::error; - - - /** - * simple BufferProvider implementation - * with additional allocation tracking - */ - class TrackingHeapBlockProvider; - - - /****************************************************************//** - * Helper for unit tests: Buffer provider reference implementation. - * - * @todo write type comment - */ - class DiagnosticBufferProvider - : util::NonCopyable - { - - std::unique_ptr pImpl_; - static lib::Depend diagnostics; - - - TrackingHeapBlockProvider& reset(); - bool isCurrent (BufferProvider const&); - - - DiagnosticBufferProvider(); - ~DiagnosticBufferProvider(); - - friend class lib::DependencyFactory; - - public: - /** build a new Diagnostic Buffer Provider instance, - * discard the existing one. Use the static query API - * for investigating collected data. */ - static BufferProvider& build(); - - - /** access the diagnostic API of the buffer provider - * @throw error::Invalid if the given provider doesn't allow - * for diagnostic access or wasn't registered beforehand. - */ - static DiagnosticBufferProvider& - access (BufferProvider const& ); - - - - - /* === diagnostic API === */ - - bool buffer_was_used (uint bufferID) const; - bool buffer_was_closed (uint bufferID) const; - void* accessMemory (uint bufferID) const; - bool all_buffers_released() const; - - - - private: - - }; - - - -}} // namespace steam::engine -#endif diff --git a/src/steam/engine/dispatcher.hpp b/src/steam/engine/dispatcher.hpp index e8178ccab..82ffd1f63 100644 --- a/src/steam/engine/dispatcher.hpp +++ b/src/steam/engine/dispatcher.hpp @@ -30,8 +30,8 @@ #include "steam/mobject/model-port.hpp" #include "steam/engine/job-ticket.hpp" #include "steam/engine/job-planning.hpp" -#include "steam/play/timings.hpp" -#include "steam/play/output-slot.hpp" +#include "vault/out/output-slot.hpp" +#include "vault/out/timings.hpp" #include "lib/iter-explorer.hpp" #include "lib/time/timevalue.hpp" #include "lib/nocopy.hpp" @@ -47,8 +47,8 @@ namespace engine { using std::move; using std::make_pair; using mobject::ModelPort; - using play::Timings; - using play::DataSink; + using vault::out::Timings; + using vault::out::DataSink; using lib::time::TimeValue; using lib::time::TimeSpan; using lib::time::FrameCnt; @@ -264,7 +264,7 @@ namespace engine { * wire in the DataSink...) */ auto - feedTo (play::DataSink& sink) + feedTo (DataSink& sink) { return terminatePipeline ( SRC::transform( diff --git a/src/steam/engine/engine-ctx-facilities.cpp b/src/steam/engine/engine-ctx-facilities.cpp deleted file mode 100644 index 512570739..000000000 --- a/src/steam/engine/engine-ctx-facilities.cpp +++ /dev/null @@ -1,77 +0,0 @@ -/* - EngineCtxFacilities - Implementation of global render engine operational services - - Copyright (C) - 2024, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -* *****************************************************************/ - - -/** @file engine-ctx-facilities.hpp - ** Implementation parts of render engine operational services. - ** @warning as of 10/2024 a preliminary integration of Render-Engine underway ////////////////////////////////////////TICKET #1367 - */ - - -#include "steam/engine/engine-ctx-facilities.hpp" -#include "steam/engine/tracking-heap-block-provider.hpp" - -//#include -//#include -//#include - - - -namespace steam { -namespace engine{ - -// using std::string; -// using lumiera::Subsys; -// using std::function; -// using std::bind; - using std::make_unique; -// using std::ref; - - - namespace { // hidden local details of the service implementation.... - - } // (End) hidden service impl details - - - - - - /** storage for the EngineService interface object */ - lib::Depend EngineCtx::access; - - - - EngineCtx::~EngineCtx() { } - - /** */ - EngineCtx::EngineCtx() - : services_{make_unique()} - , mem {services_->getMemProvider()} - , cache{services_->getCacheProvider()} - { } - - - /** */ - EngineCtx::Facilities::Facilities() - : memProvider_{make_unique()} ////////////////////////////////////////TICKET #1367 : only suitable for first prototype - , cacheProvider_{} - { } - - - - /** core operation: blah blubb.... - */ - - - -}} // namespace steam::engine diff --git a/src/steam/engine/engine-ctx.cpp b/src/steam/engine/engine-ctx.cpp new file mode 100644 index 000000000..ff5f0a31b --- /dev/null +++ b/src/steam/engine/engine-ctx.cpp @@ -0,0 +1,59 @@ +/* + EngineCtx - Base implementation of global render engine operational services + + Copyright (C) + 2024, Hermann Vosseler + +  **Lumiera** is free software; you can redistribute it and/or modify it +  under the terms of the GNU General Public License as published by the +  Free Software Foundation; either version 2 of the License, or (at your +  option) any later version. See the file COPYING for further details. + +* *****************************************************************/ + + +/** @file engine-ctx.cpp + ** Base implementation of render engine operational service access. + ** This singleton implementation provides and configures the access point, + ** while the actual [»facilities«](engine-facilities.hpp), which are retrieved + ** through interface methods, can be configured with the help of the + ** [Dependency-Injection framework](depend-inject.hpp) + ** @warning as of 1/2026 a preliminary integration of Render-Engine underway ////////////////////////////////////////TICKET #1367 + */ + + +#include "steam/engine/engine-ctx.hpp" +#include "steam/engine/engine-facilities.hpp" + + +namespace steam { +namespace engine{ + + + + /** storage for the EngineService interface object */ + lib::Depend EngineCtx::access; + + /** configurable access point to the »Engine Facilities« */ + lib::Depend accessEngineFacilities; + + + + /** + * Bootstrap the EngineCtx access front-end. + * The EngineCtx object instance is created on-demand through + * lib::Depend; at that point, a service implementation + * of the EngineFacilities is accessed; when used from a test setup, + * this will be a "naive" implementation, based on heap allocations. + * For the actual Render Engine, the EngineEnvironment must been + * bootstrapped prior to this first access, to make the full set + * of Render Engine services available through this interface. + */ + EngineCtx::EngineCtx() + : mem {accessEngineFacilities().setupBufferProvider()} + , cache{accessEngineFacilities().setupCacheProvider()} + { } + + + +}} // namespace steam::engine diff --git a/src/steam/engine/engine-ctx.hpp b/src/steam/engine/engine-ctx.hpp index 863e59a09..35ab90618 100644 --- a/src/steam/engine/engine-ctx.hpp +++ b/src/steam/engine/engine-ctx.hpp @@ -16,15 +16,24 @@ ** Notably the services to provide access to working buffers are linked directly into the ** render node connectivity, where they are used for each invocation of a render job. ** - ** The EngineCtx itself is meant to be [dependency injected](\ref lib::Depend), so that - ** tests can use suitably adapted variants for verifications. The default instantiation - ** provides a naive self-contained implementation suitable for demonstration and test. + ** The EngineCtx itself is meant to be [dependency injected](\ref lib::Depend), and acts + ** as a front-end for the support facilities of the render engine — which are likewise + ** dependency-injected, so that tests can use suitably adapted variants for verification. + ** The default instantiation provides a naive self-contained implementation suitable for + ** demonstration and test. + ** + ** For productive use within the Lumiera Render Engine, a much more elaborate setup + ** is necessary, including setup of a frame cache, and of timing strategies; these + ** are configured as part of starting the steam::engine::RenderEnvironment. ** @todo who is responsible for setup of the services for the actual render engine? ** Might be closely related to brining up façade interfaces. - ** @todo WIP-WIP 2/2025 provide actual service implementation and find a way - ** how to populate the Facilities with these actual services... + ** @todo WIP-WIP 1/2026 + ** - ✔ provide a default implementation for test + ** - 🔁 find a way to populate the Facilities with the + ** production variant of the services... ** - ** @see engine-ctx-facilities.hpp implementation + ** @see engine-ctx-test.cpp + ** @see engine-ctx.cpp base implementation ** @see buffer-provider.hpp ** @see buffhandle.hpp */ @@ -34,41 +43,31 @@ #include "lib/depend.hpp" -#include "steam/engine/buffhandle.hpp" +#include "vault/mem/buffhandle.hpp" #include "lib/nocopy.hpp" -//#include -#include - namespace steam { namespace engine { -// using lib::Literal; -// using std::unique_ptr; -// using std::forward; + using vault::mem::BufferProvider; + class EngineCtx : util::NonCopyable { - class Facilities; - std::unique_ptr services_; public: BufferProvider& mem; - BufferProvider& cache; -// BufferProvider& output; /////////////////////////OOO presumably no longer necessary + BufferProvider& cache; /////////////////////////////////////////////////////////////////////////////TICKET #1223 : very likely a super interface for the CacheService static lib::Depend access; private: - ~EngineCtx(); EngineCtx(); - friend class lib::DependencyFactory; }; - }} // namespace steam::engine #endif /*STEAM_ENGINE_ENGINE_CTX_H*/ diff --git a/src/steam/engine/engine-diagnostics.hpp b/src/steam/engine/engine-diagnostics.hpp index 0395ee3c1..70ba59b3d 100644 --- a/src/steam/engine/engine-diagnostics.hpp +++ b/src/steam/engine/engine-diagnostics.hpp @@ -30,14 +30,15 @@ #include "lib/error.hpp" #include "lib/nocopy.hpp" +#include "vault/out/timings.hpp" #include "steam/engine/engine-service.hpp" //#include "include/dummy-player-facade.h" //#include "include/display-facade.h" //#include "steam/engine/calc-stream.hpp" //#include "steam/mobject/model-port.hpp" -#include "steam/play/timings.hpp" -//#include "steam/play/output-slot.hpp" -//#include "common/instancehandle.hpp" +//#include "vault/out/timings.hpp" +//#include "vault/out/output-slot.hpp" +//#include "vessel/instancehandle.hpp" //#include "lib/singleton-ref.hpp" //#include "lib/polymorphic-value.hpp" //#include "lib/depend.hpp" @@ -49,10 +50,10 @@ namespace steam { namespace engine{ // using std::string; -// using lumiera::Subsys; -// using lumiera::Display; -// using lumiera::DummyPlayer; - using steam::play::Timings; +// using vessel::Subsys; +// using vessel::Display; +// using vessel::DummyPlayer; + using vault::out::Timings; @@ -101,6 +102,5 @@ namespace engine{ -} // namespace engine -} // namespace steam +}}// namespace steam::engine #endif diff --git a/src/steam/engine/engine-ctx-facilities.hpp b/src/steam/engine/engine-facilities.hpp similarity index 50% rename from src/steam/engine/engine-ctx-facilities.hpp rename to src/steam/engine/engine-facilities.hpp index 4774006b0..95255e9af 100644 --- a/src/steam/engine/engine-ctx-facilities.hpp +++ b/src/steam/engine/engine-facilities.hpp @@ -1,5 +1,5 @@ /* - ENGINE-CTX-FACILITIES.hpp - Implementation of global render engine operational services + ENGINE-FACILITIES.hpp - Access and coordination of global render engine operational services Copyright (C) 2024, Hermann Vosseler @@ -11,64 +11,45 @@ */ -/** @file engine-ctx-facilities.hpp - ** Implementation of render engine operational configuration and services. +/** @file engine-facilities.hpp + ** Interface to setup and access the render engine configuration and services. ** Without further setup, [dependency injection](\ref depend.hpp) will establish a ** minimalistic default instantiation of these services, sufficient for demonstration ** and tests. For the real render engine however, dedicated service instances will be - ** created and managed actively. + ** created and managed actively, by the [»Render Environment«](\ref RenderEnvironment). ** @todo and we'll have yet to »figure out« what specifically to do for these ** services and how to start an active render engine for productive use. - ** @todo WIP-WIP 10/2024 strive at first complete prototypical engine usage ////////////////////////////////////////TICKET #1367 + ** @remark this seemingly convoluted setup with several layers of abstraction is + ** necessary to allow for "simple use without much ado" in a test scenario, + ** while having a fully controlled lifecycle for the actual Render Engine. ** ** @see engine-ctx.hpp + ** @see engine-service.hpp ** @see weaving-pattern-builder.hpp */ -#ifndef STEAM_ENGINE_ENGINE_CTX_FACILITIES_H -#define STEAM_ENGINE_ENGINE_CTX_FACILITIES_H +#ifndef STEAM_ENGINE_ENGINE_FACILITIES_H +#define STEAM_ENGINE_ENGINE_FACILITIES_H #include "steam/engine/engine-ctx.hpp" -#include "steam/engine/buffer-provider.hpp" #include "lib/nocopy.hpp" -//#include -#include - namespace steam { -namespace engine { +namespace engine{ -// using lib::Literal; - using std::unique_ptr; -// using std::forward; - - class EngineCtx::Facilities + class EngineFacilities : util::NonCopyable { - unique_ptr memProvider_; - unique_ptr cacheProvider_; public: - Facilities(); + virtual ~EngineFacilities() { } ///< this is an interface - BufferProvider& - getMemProvider() - { - REQUIRE (memProvider_); - return *memProvider_; - } - - BufferProvider& - getCacheProvider() - { - return cacheProvider_? *cacheProvider_ - : *memProvider_; - } + virtual BufferProvider& setupBufferProvider(); + virtual BufferProvider& setupCacheProvider(); ///////////////////////////////////////////////////////TICKET #1223 : very likely super interface required for the CacheService }; - }} // namespace steam::engine #endif /*STEAM_ENGINE_ENGINE_CTX_FACILITIES_H*/ diff --git a/src/steam/engine/engine-service.cpp b/src/steam/engine/engine-service.cpp index a438cf6ea..7a9dee174 100644 --- a/src/steam/engine/engine-service.cpp +++ b/src/steam/engine/engine-service.cpp @@ -19,24 +19,23 @@ #include "steam/engine/engine-service.hpp" +#include "steam/engine/render-environment.hpp" #include "lib/itertools.hpp" -//#include -//#include #include +using std::ref; +using std::bind; +using std::function; +using std::placeholders::_1; +using lib::iter_source::transform; +using lib::append_all; +using vault::out::DataSink; namespace steam { namespace engine{ -// using std::string; -// using lumiera::Subsys; - using std::function; - using std::bind; - using std::ref; - using lib::iter_source::transform; - using lib::append_all; namespace { // hidden local details of the service implementation.... @@ -70,22 +69,22 @@ namespace engine{ CalcStreams EngineService::calculate(ModelPort mPort, Timings nominalTimings, - OutputConnection& output, + OutputSlot& outputSlot, Quality serviceQuality) { //////////////////////////////////////////////////TICKET #1301 : prepare proper Dispatcher here, including translation ModelPort -> portIDX RenderEnvironment& renderConfig = configureCalculation (mPort,nominalTimings,serviceQuality); - function triggerRenderStart = bind (activateCalculation, _1, ref(renderConfig)); + function triggerRenderStart = bind (activateCalculation, _1, ref(renderConfig)); CalcStreams runningCalculations; - append_all (transform (output.getOpenedSinks() + append_all (transform (outputSlot.getOpenedSinks() ,triggerRenderStart ) ,runningCalculations); return runningCalculations; -/////////////////////////////////////////////////////////////////////////////////////////////TICKET #874 : use a pipeline builder to write it as follows: -// treat_all(output.getOpenedSinks()) -// .apply (activateCalculation, _1, renderConfig) -// .buildVector(); +/////////////////////////////////////////////////////////////////////////////////////////////TICKET #874 : use our pipeline builder (lib::IterSource) to write it as follows: +// lib::explore(output.getOpenedSinks()) +// .transform ([this,&renderConfig](DataSink sink){ return activateCalculation(sink, renderConfig); }) +// .effuse(); } @@ -107,7 +106,7 @@ namespace engine{ * to be re-configured and adjusted while running. */ CalcStream - EngineService::activateCalculation (play::DataSink sink, RenderEnvironment& engineCallback) + EngineService::activateCalculation (DataSink sink, RenderEnvironment& engineCallback) { CalcStream calcStream(engineCallback); calcStream.sendToOutput (sink); ////////////////////////////////////////////////////TICKET #1297 : need to re-think the association ModelPort ⟷ output sink diff --git a/src/steam/engine/engine-service.hpp b/src/steam/engine/engine-service.hpp index 786a7378b..dd4c77de3 100644 --- a/src/steam/engine/engine-service.hpp +++ b/src/steam/engine/engine-service.hpp @@ -49,11 +49,11 @@ #include "lib/nocopy.hpp" //#include "include/dummy-player-facade.h" //#include "include/display-facade.h" +#include "vault/out/timings.hpp" #include "steam/engine/calc-stream.hpp" #include "steam/mobject/model-port.hpp" -#include "steam/play/timings.hpp" -#include "steam/play/output-slot.hpp" -//#include "common/instancehandle.hpp" +#include "vault/out/output-slot.hpp" +//#include "vessel/instancehandle.hpp" //#include "lib/singleton-ref.hpp" #include "lib/polymorphic-value.hpp" #include "lib/depend.hpp" @@ -66,14 +66,14 @@ namespace steam { namespace engine{ // using std::string; -// using lumiera::Subsys; -// using lumiera::Display; -// using lumiera::DummyPlayer; +// using vessel::Subsys; +// using vessel::Display; +// using vessel::DummyPlayer; // using std::function; using mobject::ModelPort; - using steam::play::Timings; + using vault::out::Timings; - using OutputConnection = steam::play::OutputSlot::Allocation; + using vault::out::OutputSlot; @@ -141,7 +141,7 @@ namespace engine{ CalcStreams calculate(ModelPort mPort, Timings nominalTimings, - OutputConnection& output, + OutputSlot& outputSlot, Quality serviceQuality =QoS_DEFAULT); CalcStreams @@ -159,7 +159,7 @@ namespace engine{ friend class EngineDiagnostics; private: - static CalcStream activateCalculation (play::DataSink, RenderEnvironment&); + static CalcStream activateCalculation (vault::out::DataSink, RenderEnvironment&); }; diff --git a/src/steam/engine/feed-manifold.hpp b/src/steam/engine/feed-manifold.hpp index f91ba8f44..d38a80181 100644 --- a/src/steam/engine/feed-manifold.hpp +++ b/src/steam/engine/feed-manifold.hpp @@ -75,7 +75,7 @@ #include "lib/error.hpp" #include "lib/nocopy.hpp" -#include "steam/engine/buffhandle.hpp" +#include "vault/mem/buffhandle.hpp" #include "lib/uninitialised-storage.hpp" #include "lib/meta/function.hpp" #include "lib/meta/trait.hpp" @@ -83,7 +83,7 @@ #include "lib/meta/variadic-helper.hpp" #include "lib/meta/tuple-helper.hpp" #include "lib/meta/generator.hpp" -#include "lib/test/test-helper.hpp" +#include "test/test-helper.hpp" namespace steam { @@ -296,12 +296,13 @@ namespace engine { }//(End)Introspection helpers. + using vault::mem::BuffHandle; + template class FeedPrototype; - /** * Configuration context for a FeedManifold. * This type-rebinding helper provides a storage configuration diff --git a/src/steam/engine/job-planning.hpp b/src/steam/engine/job-planning.hpp index 288e888bd..d9e5528fb 100644 --- a/src/steam/engine/job-planning.hpp +++ b/src/steam/engine/job-planning.hpp @@ -61,8 +61,8 @@ #include "steam/common.hpp" #include "vault/gear/job.h" #include "steam/engine/job-ticket.hpp" -#include "steam/play/output-slot.hpp" -#include "steam/play/timings.hpp" +#include "vault/out/output-slot.hpp" +#include "vault/out/timings.hpp" #include "lib/time/timevalue.hpp" #include "lib/itertools.hpp" #include "lib/nocopy.hpp" @@ -74,11 +74,11 @@ namespace engine { namespace error = lumiera::error; - using play::DataSink; - using play::Timings; using lib::time::Time; using lib::time::TimeVar; using lib::time::Duration; + using vault::out::DataSink; + using vault::out::Timings; using vault::gear::Job; @@ -139,11 +139,11 @@ namespace engine { { switch (timings.playbackUrgency) { - case play::ASAP: - case play::NICE: + case vault::out::ASAP: + case vault::out::NICE: return Time::ANYTIME; - case play::TIMEBOUND: + case vault::out::TIMEBOUND: return doCalcDeadline (timings); } NOTREACHED ("unexpected playbackUrgency"); @@ -176,11 +176,11 @@ namespace engine { auto buildDependencyPlanning() { - return lib::transformIterator (jobTicket_.getPrerequisites() - ,[this](JobTicket& prereqTicket) - { - return JobPlanning{*this, prereqTicket}; - }); + return lib::transformIter (jobTicket_.getPrerequisites() + ,[this](JobTicket& prereqTicket) + { + return JobPlanning{*this, prereqTicket}; + }); } private: diff --git a/src/steam/engine/job-ticket.hpp b/src/steam/engine/job-ticket.hpp index 661a6f89a..63ca5c04d 100644 --- a/src/steam/engine/job-ticket.hpp +++ b/src/steam/engine/job-ticket.hpp @@ -154,12 +154,12 @@ using lib::LUID; auto getPrerequisites () { - return lib::transformIterator (this->empty()? Prerequisites::iterator() - : provision_.prerequisites.begin() - ,[](Prerequisite& prq) -> JobTicket& - { - return prq.prereqTicket; - }); + return lib::transformIter (this->empty()? Prerequisites::iterator() + : provision_.prerequisites.begin() + ,[](Prerequisite& prq) -> JobTicket& + { + return prq.prereqTicket; + }); } /** diff --git a/src/steam/engine/media-weaving-pattern.hpp b/src/steam/engine/media-weaving-pattern.hpp index 0e2d6f799..771080622 100644 --- a/src/steam/engine/media-weaving-pattern.hpp +++ b/src/steam/engine/media-weaving-pattern.hpp @@ -138,25 +138,15 @@ #include "steam/engine/turnout.hpp" #include "steam/engine/turnout-system.hpp" #include "steam/engine/feed-manifold.hpp" -/////////////////////////////////////////////////////////////////////////////////////////////////////////////TICKET #1367 : Rebuild the Node Invocation -//#include "vault/gear/job.h" -//#include "steam/engine/exit-node.hpp" -//#include "lib/time/timevalue.hpp" -//#include "lib/linked-elements.hpp" +#include "vault/mem/buffhandle.hpp" #include "lib/several.hpp" -//#include "lib/util-foreach.hpp" -//#include "lib/iter-adapter.hpp" -//#include "lib/meta/function.hpp" -//#include "lib/itertools.hpp" -//#include "lib/util.hpp" ////////OOO wegen manifoldSiz() - -//#include namespace steam { namespace engine { using lib::Several; + using vault::mem::BuffDescr; /** diff --git a/src/steam/engine/proc-node.hpp b/src/steam/engine/proc-node.hpp index 5f09ba990..3b0e68bc1 100644 --- a/src/steam/engine/proc-node.hpp +++ b/src/steam/engine/proc-node.hpp @@ -99,7 +99,8 @@ #include "lib/error.hpp" #include "lib/nocopy.hpp" #include "lib/hash-value.h" -#include "steam/engine/buffhandle.hpp" +#include "include/limits.hpp" +#include "vault/mem/buffhandle.hpp" #include "steam/engine/turnout-system.hpp" #include "lib/format-string.hpp" #include "lib/several.hpp" @@ -124,6 +125,7 @@ namespace engine { class ProcNode; class ProcNodeDiagnostic; + using vault::mem::BuffHandle; using ProcNodeRef = std::reference_wrapper; using OptionalBuff = std::optional; diff --git a/src/steam/engine/render-drive.hpp b/src/steam/engine/render-drive.hpp index 21934a30f..720d2f6b5 100644 --- a/src/steam/engine/render-drive.hpp +++ b/src/steam/engine/render-drive.hpp @@ -31,9 +31,9 @@ #define STEAM_ENGINE_RENDER_DRIVE_H #include "steam/common.hpp" +#include "steam/engine/render-environment.hpp" #include "steam/mobject/model-port.hpp" -#include "steam/engine/dispatcher.hpp" -#include "steam/play/timings.hpp" +#include "vault/out/timings.hpp" #include "vault/gear/job.h" //#include "lib/nocopy.hpp" @@ -42,9 +42,10 @@ namespace steam { namespace engine { // using std::function; + using mobject::ModelPort; using vault::gear::JobParameter; using vault::gear::JobClosure; - using mobject::ModelPort; + using vault::out::Timings; // using lib::time::TimeSpan; // using lib::time::FSecs; // using lib::time::Time; @@ -52,23 +53,6 @@ namespace engine { using lib::HashVal; - /** - * Abstract definition of the environment - * hosting a given render activity (CalcStream). - * Exposes all the operations necessary to adjust the - * runtime behaviour of the render activity, like e.g. - * re-scheduling with modified playback speed. Since the - * CalcStream is an conceptual representation of "the rendering", - * the actual engine implementation is kept opaque this way. - */ - class RenderEnvironment - { - public: - virtual ~RenderEnvironment() { } ///< this is an interface - - virtual play::Timings& effectiveTimings() =0; - virtual Dispatcher& getDispatcher() =0; - }; /** @@ -115,7 +99,7 @@ namespace engine { { } - play::Timings const& + Timings const& getTimings() const { return engine_.effectiveTimings(); diff --git a/src/steam/engine/render-environment.cpp b/src/steam/engine/render-environment.cpp new file mode 100644 index 000000000..59ee8c953 --- /dev/null +++ b/src/steam/engine/render-environment.cpp @@ -0,0 +1,76 @@ +/* + RenderEnvironment - setup of the Lumiera Render Engine + + Copyright (C) + 2026, Hermann Vosseler + +  **Lumiera** is free software; you can redistribute it and/or modify it +  under the terms of the GNU General Public License as published by the +  Free Software Foundation; either version 2 of the License, or (at your +  option) any later version. See the file COPYING for further details. + +* *****************************************************************/ + + +/** @file render-environment.cpp + ** Base implementation of render engine operational service access. + ** This singleton implementation provides and configures the access point, + ** while the actual [»facilities«](engine-facilities.hpp), which are retrieved + ** through interface methods, can be configured with the help of the + ** [Dependency-Injection framework](depend-inject.hpp) + ** @warning as of 1/2026 a preliminary integration of Render-Engine underway ////////////////////////////////////////TICKET #1367 + */ + + +#include "steam/engine/render-environment.hpp" +#include "steam/engine/engine-facilities.hpp" +#include "vault/mem/naive-buffer-setup.hpp" +#include "vault/mem/buffer-provider.hpp" + + +using vault::mem::BufferProvider; +using vault::mem::NaiveBufferSetup; + +namespace steam { +namespace engine{ + + + namespace { + /** A simplistic buffer manager for tests + * (Meyer's Singleton) */ + BufferProvider& + getNaiveBufferManager() + { + static NaiveBufferSetup buffProvider; ///////////////////////////////////////////////////TICKET #1367 : only suitable for first prototype; should implement a bare bone memory manager instead + return buffProvider; + } + } + + /* ===== Render Environment Implementation ===== */ + + + ///////////////////////////////////////////////////////////////////////////////////////////////////////////TICKET #1409 : actually build that! right now (1/2026) it is a conceptual placeholder + /// The real Render Engine needs a well defined life cycle; + /// services must be booted and shut down. + /// Notably a dedicated serivce-like subclass of EngineCtx + /// must be created here and installed for lib::Depend + /// + + + /* ===== Access to Render Engine Facilities ===== */ + + BufferProvider& + EngineFacilities::setupBufferProvider() + { + return getNaiveBufferManager(); + } + + BufferProvider& + EngineFacilities::setupCacheProvider() ////////////////////////////////////////////////////////////////TICKET #1223 : in what ways does client code need to interact with the Frame Cache / Management? + { + return getNaiveBufferManager(); + } + + + +}} // namespace steam::engine diff --git a/src/steam/engine/render-environment.hpp b/src/steam/engine/render-environment.hpp new file mode 100644 index 000000000..445bc71bf --- /dev/null +++ b/src/steam/engine/render-environment.hpp @@ -0,0 +1,67 @@ +/* + RENDER-ENVIRONMENT.hpp - setup of the Lumiera Render Engine + + Copyright (C) + 2026, Hermann Vosseler + +  **Lumiera** is free software; you can redistribute it and/or modify it +  under the terms of the GNU General Public License as published by the +  Free Software Foundation; either version 2 of the License, or (at your +  option) any later version. See the file COPYING for further details. + +*/ + + +/** @file render-environment.hpp + ** A configuration context for processing within the Render Engine. + ** This context can be accessed from within the ongoing render processes to query + ** strategy settings; actually, all the global support facilities within the Render Engine + ** are attached here. Thus it might also be used to control the lifecycle (2026 not clear yet) + ** @todo 2026 the concept is quite old, but I have reestablished it now to use it as an anchor. + ** it is not clear yet if there will be several sub environments for different playback + ** configurations; effectively, this is a conceptual placeholder now and needs to be + ** put into actual use to see where this leads us to.... ////////////////////////////////////////////TICKET #1409 + ** + ** @todo 4/2023 »Playback Vertical Slice« -- effort towards first integration of render process ////////////TICKET #1221 + */ + + +#ifndef STEAM_ENGINE_RENDER_ENVIRONMENT_H +#define STEAM_ENGINE_RENDER_ENVIRONMENT_H + +#include "steam/common.hpp" +#include "steam/engine/dispatcher.hpp" +#include "vault/out/timings.hpp" +//#include "lib/nocopy.hpp" + + +namespace steam { +namespace engine{ + + using vault::out::Timings; + + + + /** + * Abstract definition of the environment + * hosting a given render activity (CalcStream). + * Exposes all the operations necessary to adjust the + * runtime behaviour of the render activity, like e.g. + * re-scheduling with modified playback speed. Since the + * CalcStream is an conceptual representation of "the rendering", + * the actual engine implementation is kept opaque this way. + */ + class RenderEnvironment + { + public: + virtual ~RenderEnvironment() { } ///< this is an interface + + virtual Timings& effectiveTimings() =0; + virtual Dispatcher& getDispatcher() =0; + }; + + + + +}} // namespace steam::engine +#endif /*STEAM_ENGINE_RENDER_ENVIRONMENT_H*/ diff --git a/src/steam/engine/tracking-heap-block-provider.cpp b/src/steam/engine/tracking-heap-block-provider.cpp deleted file mode 100644 index 996f082e5..000000000 --- a/src/steam/engine/tracking-heap-block-provider.cpp +++ /dev/null @@ -1,348 +0,0 @@ -/* - TrackingHeapBlockProvider - plain heap allocating BufferProvider implementation for tests - - Copyright (C) - 2011, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -* *****************************************************************/ - - -/** @file tracking-heap-block-provider.cpp - ** Implementation details of a mock engine::BufferProvider for unit testing - */ - - -#include "lib/error.hpp" -#include "include/logging.h" -#include "lib/scoped-ptrvect.hpp" -#include "lib/util-foreach.hpp" - -#include "steam/engine/tracking-heap-block-provider.hpp" - -#include -#include - -using util::and_all; -using std::vector; -using lib::ScopedPtrVect; - - - -namespace steam { -namespace engine { - - namespace error = lumiera::error; - - using Buff = StreamType::ImplFacade::DataBuffer; - - - namespace { // implementation helpers... - - inline Buff* - asBuffer(void* mem) - {// type tag to mark memory address as Buffer - return static_cast (mem); - } - - - using diagn::Block; - - /** helper to find Block entries - * based on their raw memory address */ - inline bool - identifyBlock (Block const& inQuestion, void* storage) - { - return storage == &inQuestion; - } - - /** build a searching predicate */ - inline function - search_for_block_using_this_storage (void* storage) - { - return bind (identifyBlock, _1, storage); - } - - template - inline Block* - pick_Block_by_storage (VEC& vec, void* blockLocation) - { - typename VEC::iterator pos - = std::find_if (vec.begin(),vec.end() - ,search_for_block_using_this_storage(blockLocation)); - if (pos!=vec.end()) - return &(*pos); - else - return NULL; - } - } - - - - namespace diagn { - - using PoolBlocks = ScopedPtrVect; - - /** - * @internal Pool of allocated buffer Blocks of a specific size. - * Helper for implementing a Diagnostic BufferProvider; actually does - * just heap allocations for the Blocks, but keeps a collection of - * allocated Blocks around. Individual entries can be retrieved - * and thus removed from the responsibility of BlockPool. - * - * The idea is that each buffer starts its lifecycle within some pool - * and later gets "emitted" to an output sequence, where it remains for - * later investigation and diagnostics. - */ - class BlockPool - { - uint maxAllocCount_; - size_t memBlockSize_; - PoolBlocks blockList_; - - public: - BlockPool() - : maxAllocCount_(0) // unlimited by default - , memBlockSize_(0) - , blockList_{} - { } - - void - initialise (size_t blockSize) - { - memBlockSize_ = blockSize; - } - // standard copy operations are valid, but will - // raise an runtime error, once BlockPool is initialised. - - ~BlockPool() - { - if (!verify_all_children_idle()) - ERROR (test, "Block actively in use while shutting down BufferProvider " - "allocation pool. This might lead to Segfault and memory leaks."); - } - - /** mark all managed blocks as disposed */ - void - discard() - { - for (Block& block : blockList_) - block.markReleased(); - } - - uint - prepare_for (uint number_of_expected_buffers) - { - if (maxAllocCount_ && - maxAllocCount_ < blockList_.size() + number_of_expected_buffers) - { - ASSERT (maxAllocCount_ >= blockList_.size()); - return maxAllocCount_ - blockList_.size(); - } - // currently no hard limit imposed - return number_of_expected_buffers; - } - - - Block& - createBlock() - { - return blockList_.manage (new Block(memBlockSize_)); - } - - - Block* - find (void* blockLocation) - { - return pick_Block_by_storage (blockList_, blockLocation); - } - - - Block* - transferResponsibility (Block* allocatedBlock) - { - return blockList_.detach (allocatedBlock); - } - - - size_t - size() const - { - return blockList_.size(); - } - - bool - isValid() const - { - return not blockList_.empty(); - } - - explicit - operator bool() const - { - return isValid(); - } - - private: - bool - verify_all_children_idle() - { - try { - return and_all (blockList_, is_in_sane_state); - } - ERROR_LOG_AND_IGNORE (test, "State verification of diagnostic BufferProvider allocation pool"); - return true; - } - - - static bool - is_in_sane_state (Block const& block) - { - return not block.was_used() or block.was_closed(); - } - }; - } - - - - namespace { // Details of allocation and accounting - - const uint MAX_BUFFERS = 50; - - diagn::Block emptyPlaceholder(0); - - } // (END) Details of allocation and accounting - - - - /** - * @internal create a memory tracking BufferProvider, - */ - TrackingHeapBlockProvider::TrackingHeapBlockProvider() - : BufferProvider ("Diagnostic_HeapAllocated") - , pool_(new diagn::PoolTable) - , outSeq_() - { } - - TrackingHeapBlockProvider::~TrackingHeapBlockProvider() - { - INFO (proc_mem, "discarding %zu diagnostic buffer entries", outSeq_.size()); - } - - - /* ==== Implementation of the BufferProvider interface ==== */ - - uint - TrackingHeapBlockProvider::prepareBuffers(uint requestedAmount, HashVal typeID) - { - diagn::BlockPool& responsiblePool = getBlockPoolFor (typeID); - return responsiblePool.prepare_for (requestedAmount); - } - - - BuffHandle - TrackingHeapBlockProvider::provideLockedBuffer(HashVal typeID) - { - diagn::BlockPool& blocks = getBlockPoolFor (typeID); - diagn::Block& newBlock = blocks.createBlock(); - return buildHandle (typeID, asBuffer(newBlock.accessMemory()), &newBlock); - } - - - void - TrackingHeapBlockProvider::mark_emitted (HashVal typeID, LocalTag const& specifics) - { - diagn::Block* block4buffer = locateBlock (typeID, specifics); - if (!block4buffer) - throw error::Logic ("Attempt to emit a buffer not known to this BufferProvider" - , LUMIERA_ERROR_BUFFER_MANAGEMENT); - diagn::BlockPool& pool = getBlockPoolFor (typeID); - Block* active = pool.transferResponsibility (block4buffer); - if (active) - outSeq_.manage (active); - else - if (block4buffer->was_closed()) - WARN (proc_mem, "Attempt to emit() an already closed buffer."); - else - WARN (proc_mem, "Attempt to emit() a buffer not found in active pool. " - "Maybe duplicate call to emit()?"); - } - - - /** mark a buffer as officially discarded */ - void - TrackingHeapBlockProvider::detachBuffer (HashVal typeID, LocalTag const& specifics, Buff& storage) - { - diagn::Block* block4buffer = locateBlock (typeID, specifics); - REQUIRE (block4buffer, "releasing a buffer not allocated through this provider"); - REQUIRE (util::isSameAdr (storage, block4buffer->accessMemory())); - block4buffer->markReleased(); - } - - - - /* ==== Implementation details ==== */ - - size_t - TrackingHeapBlockProvider::emittedCnt() const - { - return outSeq_.size(); - } - - void - TrackingHeapBlockProvider::markAllEmitted() - { - for (auto& [_, blockPool] : *pool_) - blockPool.discard(); - } - - diagn::Block& - TrackingHeapBlockProvider::access_emitted (uint bufferID) - { - if (!withinOutputSequence (bufferID)) - return emptyPlaceholder; ////////////////////////////////TICKET #856 - else - return outSeq_[bufferID]; - } - - bool - TrackingHeapBlockProvider::withinOutputSequence (uint bufferID) const - { - if (bufferID >= MAX_BUFFERS) - throw error::Fatal ("hardwired internal limit for test buffers exceeded"); - - return bufferID < outSeq_.size(); - } - - diagn::BlockPool& - TrackingHeapBlockProvider::getBlockPoolFor (HashVal typeID) - { - diagn::BlockPool& pool = (*pool_)[typeID]; - if (!pool) - pool.initialise(getBufferSize(typeID)); - return pool; - } - - diagn::Block* - TrackingHeapBlockProvider::locateBlock (HashVal typeID, void* storage) - { - diagn::BlockPool& pool = getBlockPoolFor (typeID); - diagn::Block* block4buffer = pool.find (storage); ////////////////////////////////TICKET #856 - return block4buffer? block4buffer - : searchInOutSeqeuence (storage); - } - - diagn::Block* - TrackingHeapBlockProvider::searchInOutSeqeuence (void* blockLocation) - { - return pick_Block_by_storage (outSeq_, blockLocation); ////////////////////////////////TICKET #856 - } - - - - -}} // namespace engine diff --git a/src/steam/engine/tracking-heap-block-provider.hpp b/src/steam/engine/tracking-heap-block-provider.hpp deleted file mode 100644 index 74a308b00..000000000 --- a/src/steam/engine/tracking-heap-block-provider.hpp +++ /dev/null @@ -1,181 +0,0 @@ -/* - TRACKING-HEAP-BLOCK-PROVIDER.hpp - plain heap allocating BufferProvider implementation for tests - - Copyright (C) - 2011, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -*/ - -/** @file tracking-heap-block-provider.hpp - ** Dummy implementation of the BufferProvider interface to support writing unit tests. - ** This BufferProvider is especially straight forward and brain dead: it just claims - ** more and more heap blocks and never releases any memory dynamically. This allows - ** to investigate additional tracking status flags for each allocated block after - ** the fact. - ** - ** The allocated buffers are numbered with a simple ascending sequence of integers, - ** used as LocalTag (see BufferMetadata). Clients can just request a Buffer with the - ** given number, causing that block to be allocated. There is a "backdoor", allowing - ** to access any allocated block, even if it is considered "released" by the terms - ** of the usual lifecycle. Only when the provider object itself gets destroyed, - ** all allocated blocks will be discarded. - ** - ** @see DiagnosticOutputSlot - ** @see DiagnosticBufferProvider - ** @see buffer-provider-protocol-test.cpp - */ - -#ifndef STEAM_ENGINE_TRACKING_HEAP_BLOCK_PROVIDER_H -#define STEAM_ENGINE_TRACKING_HEAP_BLOCK_PROVIDER_H - - -#include "lib/error.hpp" -#include "lib/hash-value.h" -#include "steam/engine/buffer-provider.hpp" -#include "lib/scoped-ptrvect.hpp" - -#include -#include - - -namespace steam { -namespace engine { - - namespace error = lumiera::error; - - using lib::ScopedPtrVect; - using lib::HashVal; - - namespace diagn { - - using std::unique_ptr; - - - /** - * Helper for implementing a diagnostic BufferProvider: - * A block of heap allocated storage, with the capability - * to store some additional tracking information. - */ - class Block - : util::NonCopyable - { - unique_ptr storage_; - - bool was_released_; - - public: - explicit - Block(size_t bufferSize) - : storage_(bufferSize? new char[bufferSize] : NULL) - , was_released_(false) - { } - - bool - was_used() const - { - return bool(storage_); - } - - bool - was_closed() const - { - return was_released_; - } - - void* - accessMemory() const - { - REQUIRE (storage_, "Block was never prepared for use"); - return storage_.get(); - } - - void - markReleased() - { - was_released_ = true; - } - }; - - class BlockPool; - - using PoolTable = std::unordered_map; - } - - - /** - * simple BufferProvider implementation with additional allocation tracking. - * @internal used as PImpl by DiagnosticBufferProvider and DiagnosticOutputSlot. - * - * This dummy implementation of the BufferProvider interface uses a linearly growing - * table of heap allocated buffer blocks, which will never be discarded, unless the object - * is discarded as a whole. There is an additional testing/diagnostics API to access the - * tracked usage information, even when blocks are already marked as "released". - */ - class TrackingHeapBlockProvider - : public BufferProvider - { - unique_ptr pool_; - ScopedPtrVect outSeq_; - - public: - /* === BufferProvider interface === */ - - virtual uint prepareBuffers (uint count, HashVal typeID) override; - virtual BuffHandle provideLockedBuffer (HashVal typeID) override; - virtual void mark_emitted (HashVal, LocalTag const&) override; - virtual void detachBuffer (HashVal, LocalTag const&, Buff&) override; - - public: - TrackingHeapBlockProvider(); - virtual ~TrackingHeapBlockProvider(); - - size_t emittedCnt() const; - - diagn::Block& access_emitted (uint bufferID); - - template - TY& accessAs (uint bufferID); - - void markAllEmitted(); - - private: - bool withinOutputSequence (uint bufferID) const; - diagn::BlockPool& getBlockPoolFor (HashVal typeID); - diagn::Block* locateBlock (HashVal typeID, void*); - diagn::Block* searchInOutSeqeuence (void* storage); - }; - - - - /** convenience shortcut: access the buffer with the given number, - * then try to convert the raw memory to the templated type. - * @throw error::Invalid if the required fame number is beyond - * the number of buffers marked as "emitted" - * @throw error::Fatal if conversion is not possible or the - * conversion path chosen doesn't work (which might - * be due to RTTI indicating an incompatible type). - */ - template - TY& - TrackingHeapBlockProvider::accessAs (uint bufferID) - { - if (!withinOutputSequence (bufferID)) - throw error::Invalid ("Buffer with the given ID not yet emitted"); - - diagn::Block& memoryBlock = access_emitted (bufferID); - TY* converted = std::launder (reinterpret_cast (memoryBlock.accessMemory())); - - REQUIRE (converted); - return *converted; - } - - - - -}} // namespace steam::engine -#endif diff --git a/src/steam/engine/weaving-pattern-builder.hpp b/src/steam/engine/weaving-pattern-builder.hpp index b99a09a16..eb2937a4e 100644 --- a/src/steam/engine/weaving-pattern-builder.hpp +++ b/src/steam/engine/weaving-pattern-builder.hpp @@ -135,12 +135,9 @@ #include "lib/several-builder.hpp" #include "steam/engine/proc-id.hpp" #include "steam/engine/engine-ctx.hpp" -#include "steam/engine/buffer-provider.hpp" -#include "steam/engine/buffhandle-attach.hpp" /////////////////OOO why do we need to include this? we need the accessAs() template function #include "steam/engine/media-weaving-pattern.hpp" +#include "vault/mem/buffer-provider.hpp" #include "lib/meta/tuple-closure.hpp" -#include "lib/meta/tuple-helper.hpp" -//#include "lib/test/test-helper.hpp" ////////////////////////////OOO TODO added for test #include "lib/format-string.hpp" #include "lib/iter-zip.hpp" #include "lib/util.hpp" @@ -156,6 +153,8 @@ namespace engine { namespace err = lumiera::error; using StrView = std::string_view; + using vault::mem::BufferProvider; + using vault::mem::BuffDescr; using std::forward; using lib::Depend; using lib::izip; diff --git a/src/steam/engine/worker/dummy-image-generator.cpp b/src/steam/engine/worker/dummy-image-generator.cpp index 1085194ec..4a75c0f3c 100644 --- a/src/steam/engine/worker/dummy-image-generator.cpp +++ b/src/steam/engine/worker/dummy-image-generator.cpp @@ -75,6 +75,10 @@ namespace node { } // (End) implementation details + using vessel::spine::DisplayerInput; + using vessel::spine::DISPLAY_NONE; + using vessel::spine::DISPLAY_YUV; + using vessel::spine::DISPLAY_RGB; @@ -86,14 +90,14 @@ namespace node { { } void - DummyImageGenerator::configure (lumiera::DisplayerInput displayFormat) + DummyImageGenerator::configure (DisplayerInput displayFormat) { INFO (steam, "ImageGen: use format %d", displayFormat); - REQUIRE ( displayFormat == lumiera::DISPLAY_NONE - or displayFormat == lumiera::DISPLAY_YUV - or displayFormat == lumiera::DISPLAY_RGB); + REQUIRE ( displayFormat == DISPLAY_NONE + or displayFormat == DISPLAY_YUV + or displayFormat == DISPLAY_RGB); - useRGB_ = (displayFormat == lumiera::DISPLAY_RGB); + useRGB_ = (displayFormat == DISPLAY_RGB); } void diff --git a/src/steam/engine/worker/dummy-image-generator.hpp b/src/steam/engine/worker/dummy-image-generator.hpp index a9f741b3f..fd89eeed4 100644 --- a/src/steam/engine/worker/dummy-image-generator.hpp +++ b/src/steam/engine/worker/dummy-image-generator.hpp @@ -33,7 +33,7 @@ #include "lib/error.hpp" -#include "include/display-handles.hpp" +#include "vessel/spine/display-handles.hpp" #include @@ -53,7 +53,7 @@ namespace node { static const uint H = 240; DummyImageGenerator(uint fps); - void configure (lumiera::DisplayerInput); + void configure (vessel::spine::DisplayerInput); /** generate the next frame and occupy the alternate buffer. diff --git a/src/steam/external/libgavl.cpp b/src/steam/external/libgavl.cpp index 3ca80e537..7eaedc5bd 100644 --- a/src/steam/external/libgavl.cpp +++ b/src/steam/external/libgavl.cpp @@ -32,7 +32,7 @@ namespace external { using control::STypeManager; using control::ON_STREAMTYPES_RESET; - using lumiera::LifecycleHook; + using vessel::LifecycleHook; void provide_GAVL_stream_implementation_types () diff --git a/src/steam/facade.cpp b/src/steam/facade.cpp index 7a5440b78..1c5db5253 100644 --- a/src/steam/facade.cpp +++ b/src/steam/facade.cpp @@ -20,12 +20,12 @@ ** - The »play out subsystem« is able to _perform_ such a render nodes network ** for video playback and rendering. ** - ** lumiera::Subsys and lumiera::SubsystemRunner together define a protocol for some + ** vessel::Subsys and vessel::SubsystemRunner together define a protocol for some ** large scale building blocks of the whole application to be started and terminated. ** Typically this entails to create a dedicated thread to manage the top level concerns ** for the given subsystem, and to create registration and services for public access: ** - to operate on the session, use the steam::control::SessionCommand facade - ** - playback and render operations can be operated by the lumiera::Play facade + ** - playback and render operations can be operated by the vessel::spine::Play facade ** ** @see SteamDispatcher ** @see OutputDirector @@ -47,8 +47,8 @@ namespace steam { using std::string; using std::unique_ptr; - using lumiera::Subsys; - using lumiera::Option; + using vessel::Subsys; + using vessel::Option; using steam::control::SteamDispatcher; diff --git a/src/steam/facade.hpp b/src/steam/facade.hpp index 578db2179..36e6797d4 100644 --- a/src/steam/facade.hpp +++ b/src/steam/facade.hpp @@ -28,7 +28,7 @@ #define STEAM_INTERFACE_FACADE_H -#include "common/subsys.hpp" +#include "vessel/subsys.hpp" @@ -51,18 +51,18 @@ namespace steam { */ struct Facade { - /** provide a descriptor for lumiera::AppState, + /** provide a descriptor for the vessel::Voyage, * wired accordingly to allow `main()` to boot * the support infrastructure for loading of * and working on the editing session. */ - static lumiera::Subsys& getSessionDescriptor(); + static vessel::Subsys& getSessionDescriptor(); - /** provide a descriptor for lumiera::AppState, + /** provide a descriptor for vessel::Voyage, * wired accordingly to allow `main()` to bring up * the render / playback coordination and * output management subsystem. */ - static lumiera::Subsys& getPlayOutDescriptor(); + static vessel::Subsys& getPlayOutDescriptor(); //////////////////TODO: define the global access interfaces for the Steam-Layer diff --git a/src/steam/mobject/builder/operationpoint.cpp b/src/steam/mobject/builder/operationpoint.cpp index 9be3c4695..4c9ce2e69 100644 --- a/src/steam/mobject/builder/operationpoint.cpp +++ b/src/steam/mobject/builder/operationpoint.cpp @@ -26,7 +26,7 @@ #include "steam/asset/proc.hpp" #include "steam/engine/node-factory.hpp" #include "steam/streamtype.hpp" -//#include "common/util.hpp" +//#include "vessel/util.hpp" //#include diff --git a/src/steam/mobject/builder/operationpoint.hpp b/src/steam/mobject/builder/operationpoint.hpp index af2f2e84b..992209454 100644 --- a/src/steam/mobject/builder/operationpoint.hpp +++ b/src/steam/mobject/builder/operationpoint.hpp @@ -28,7 +28,7 @@ #define MOBJECT_BUILDER_OPERATIONPOINT_H //#include "steam/engine/proc-node.hpp" /////TODO can we get rid of this header here? -//#include "common/query.hpp" +//#include "vessel/query.hpp" #include "lib/hash-standard.hpp" #include diff --git a/src/steam/mobject/output-designation.cpp b/src/steam/mobject/output-designation.cpp index 1d4c97272..7897116c4 100644 --- a/src/steam/mobject/output-designation.cpp +++ b/src/steam/mobject/output-designation.cpp @@ -31,7 +31,7 @@ #include "lib/error.hpp" #include "lib/symbol.hpp" #include "lib/query-util.hpp" -#include "common/query/query-resolver.hpp" +#include "vessel/query/query-resolver.hpp" #include "steam/mobject/mobject.hpp" #include "steam/mobject/placement-ref.hpp" #include "steam/mobject/output-designation.hpp" @@ -42,8 +42,9 @@ #include #include -using lumiera::Symbol; -using lumiera::query::QueryHandler; +using lib::Symbol; +using vessel::Query; +using vessel::query::QueryHandler; using lib::query::removeTerm; using lib::query::extractID; using steam::ConfigResolver; @@ -53,9 +54,9 @@ using util::uNum; namespace steam { namespace mobject { - typedef OutputDesignation::PPipe PPipe; - typedef OutputDesignation::PID PID; - typedef OutputDesignation::TargetSpec TargetSpec; + using PPipe = OutputDesignation::PPipe; + using PID = OutputDesignation::PID; + using TargetSpec = OutputDesignation::TargetSpec; struct AbsoluteSpec diff --git a/src/steam/mobject/output-mapping.hpp b/src/steam/mobject/output-mapping.hpp index 7e53cb440..be4e3f9aa 100644 --- a/src/steam/mobject/output-mapping.hpp +++ b/src/steam/mobject/output-mapping.hpp @@ -47,7 +47,7 @@ #include "lib/util.hpp" #include "lib/hash-value.h" #include "steam/asset/pipe.hpp" -#include "common/query.hpp" +#include "vessel/query.hpp" #include #include @@ -87,7 +87,7 @@ namespace mobject { }//(End) type rebinding helper namespace error = lumiera::error; - using lumiera::Query; + using vessel::Query; using lib::HashVal; diff --git a/src/steam/mobject/session.hpp b/src/steam/mobject/session.hpp index 919e66039..455a3d051 100644 --- a/src/steam/mobject/session.hpp +++ b/src/steam/mobject/session.hpp @@ -43,7 +43,7 @@ #include "steam/mobject/placement.hpp" #include "steam/mobject/mobject-ref.hpp" -#include "common/query/defs-manager.hpp" ////////////////////////////TICKET #643 forward declare this? +#include "vessel/query/defs-manager.hpp" ////////////////////////////TICKET #643 forward declare this? #include "lib/depend.hpp" #include "lib/symbol.hpp" #include "lib/p.hpp" @@ -101,7 +101,7 @@ namespace mobject { : util::NonCopyable { protected: - typedef lumiera::query::DefsManager& DefaultsAccess; + typedef vessel::query::DefsManager& DefaultsAccess; typedef session::ElementQuery& ElementsAccess; typedef std::vector const& TimelineAccess; typedef std::vector const& SequenceAccess; diff --git a/src/steam/mobject/session/lifecycle-advisor.hpp b/src/steam/mobject/session/lifecycle-advisor.hpp index d4a32cb04..b331838f1 100644 --- a/src/steam/mobject/session/lifecycle-advisor.hpp +++ b/src/steam/mobject/session/lifecycle-advisor.hpp @@ -30,7 +30,7 @@ ** ** @see SessManager ** @see LifecycleHook - ** @see lumiera::AppState + ** @see vessel::Voyage ** @see session.hpp ** @see sess-manager-impl.cpp concrete definition of Lifecycle ** @@ -42,7 +42,7 @@ #include "lib/error.hpp" #include "lib/nocopy.hpp" -#include "include/lifecycle.h" +#include "include/lifecycle.hpp" #include "steam/mobject/session.hpp" #include "lib/symbol.hpp" @@ -119,7 +119,7 @@ namespace session { void emitEvent (Symbol eventLabel) { - lumiera::LifecycleHook::trigger (eventLabel); + vessel::LifecycleHook::trigger (eventLabel); } diff --git a/src/steam/mobject/session/mobjectfactory.cpp b/src/steam/mobject/session/mobjectfactory.cpp index 2ce145e64..0a1c34f39 100644 --- a/src/steam/mobject/session/mobjectfactory.cpp +++ b/src/steam/mobject/session/mobjectfactory.cpp @@ -27,9 +27,9 @@ #include "steam/mobject/session/binding.hpp" #include "steam/asset/clip.hpp" #include "steam/asset/effect.hpp" -#include "common/query/defs-manager.hpp" +#include "vessel/query/defs-manager.hpp" -using lumiera::query::DefsManager; +using vessel::query::DefsManager; namespace steam { namespace mobject { diff --git a/src/steam/mobject/session/mobjectfactory.hpp b/src/steam/mobject/session/mobjectfactory.hpp index 6821eed13..29e057299 100644 --- a/src/steam/mobject/session/mobjectfactory.hpp +++ b/src/steam/mobject/session/mobjectfactory.hpp @@ -28,7 +28,7 @@ #include "lib/symbol.hpp" -namespace lumiera { +namespace vessel{ namespace query { class DefsManager; }} @@ -66,7 +66,7 @@ namespace session { public: - Placement operator() (lumiera::query::DefsManager&); + Placement operator() (vessel::query::DefsManager&); Placement operator() (asset::Clip const&, asset::Media const&); Placement operator() (asset::Clip const&, vector); Placement operator() (lib::idi::EntryID const&); diff --git a/src/steam/mobject/session/placement-index-query-resolver.cpp b/src/steam/mobject/session/placement-index-query-resolver.cpp index 1decae585..c9c8a3c62 100644 --- a/src/steam/mobject/session/placement-index-query-resolver.cpp +++ b/src/steam/mobject/session/placement-index-query-resolver.cpp @@ -36,10 +36,10 @@ namespace session { using std::unique_ptr; - using lumiera::Goal; - using lumiera::Query; - using lumiera::Resolution; - using lumiera::getResultTypeID; + using vessel::Goal; + using vessel::Query; + using vessel::Resolution; + using vessel::getResultTypeID; typedef PlacementIndex::ID PID; @@ -191,7 +191,7 @@ namespace session { * there is a smart-ptr managing this ResultSet. */ class ResultSet - : public lumiera::Resolution + : public vessel::Resolution { ContentFilter acceptable_; ExplorerBuilder buildExploartion_; diff --git a/src/steam/mobject/session/placement-index-query-resolver.hpp b/src/steam/mobject/session/placement-index-query-resolver.hpp index fda97815a..a4c3bc879 100644 --- a/src/steam/mobject/session/placement-index-query-resolver.hpp +++ b/src/steam/mobject/session/placement-index-query-resolver.hpp @@ -50,7 +50,7 @@ #include "steam/mobject/session/placement-index.hpp" #include "steam/mobject/session/scope-query.hpp" -#include "common/query.hpp" +#include "vessel/query.hpp" #include @@ -60,7 +60,7 @@ namespace mobject { namespace session { using std::function; - using lumiera::Goal; + using vessel::Goal; typedef PlacementIndex& IndexLink(void); @@ -79,7 +79,7 @@ namespace session { * * more generally, any ScopeQuery with these properties, in some variations */ class PlacementIndexQueryResolver - : public lumiera::QueryResolver + : public vessel::QueryResolver { function _getIndex; @@ -98,7 +98,7 @@ namespace session { void defineHandling(); template - lumiera::Resolution* resolutionFunction (Goal const& goal); + vessel::Resolution* resolutionFunction (Goal const& goal); public: diff --git a/src/steam/mobject/session/placement-index.cpp b/src/steam/mobject/session/placement-index.cpp index b473d3afc..f4198c8a1 100644 --- a/src/steam/mobject/session/placement-index.cpp +++ b/src/steam/mobject/session/placement-index.cpp @@ -395,7 +395,6 @@ namespace session { } - using ElementResolver = function const&)>; mutable ElementResolver elementResolver_; ElementResolver diff --git a/src/steam/mobject/session/placement-index.hpp b/src/steam/mobject/session/placement-index.hpp index 282cb4c51..9891b3f6d 100644 --- a/src/steam/mobject/session/placement-index.hpp +++ b/src/steam/mobject/session/placement-index.hpp @@ -106,6 +106,7 @@ #include "lib/util.hpp" #include +#include #include #include @@ -181,7 +182,9 @@ namespace session { using _PID = PlacementMO::ID; using ScopeIter = std::unordered_multimap<_PID,_PID>::const_iterator; using ScopeRangeIter = lib::RangeIter; - using _ID_TableIterator = lib::TransformIter; + using ElementResolver = std::function const&)>; + using _ID_TableIterator = decltype(lib::transformIter(std::declval() + ,std::declval())); diff --git a/src/steam/mobject/session/query/fake-configrules.cpp b/src/steam/mobject/session/query/fake-configrules.cpp index 48ad184ff..4e11aa1cc 100644 --- a/src/steam/mobject/session/query/fake-configrules.cpp +++ b/src/steam/mobject/session/query/fake-configrules.cpp @@ -34,7 +34,7 @@ #include "steam/asset/procpatt.hpp" #include "steam/asset/pipe.hpp" #include "lib/query-util.hpp" -#include "common/query.hpp" +#include "vessel/query.hpp" #include "include/logging.h" #include "lib/symbol.hpp" diff --git a/src/steam/mobject/session/query/fake-configrules.hpp b/src/steam/mobject/session/query/fake-configrules.hpp index 3e75238a3..4c05a6561 100644 --- a/src/steam/mobject/session/query/fake-configrules.hpp +++ b/src/steam/mobject/session/query/fake-configrules.hpp @@ -28,8 +28,8 @@ ** away, some day the real thing will be there, displacing this mess without further notice. ** @todo to be removed in Alpha, when integrating a real resolution engine /////////////////TICKET #710 ** - ** @see lumiera::Query - ** @see lumiera::ConfigRules + ** @see vessel::Query + ** @see vessel::ConfigRules ** */ @@ -64,9 +64,9 @@ namespace session { using lib::meta::InstantiateChained; using lib::P; - using lumiera::Query; - using lumiera::QueryKey; - using lumiera::query::isFakeBypass; /////////TODO placeholder until there is a real resolution engine + using vessel::Query; + using vessel::QueryKey; + using vessel::query::isFakeBypass; /////////TODO placeholder until there is a real resolution engine using util::contains; using util::isnil; diff --git a/src/steam/mobject/session/root.cpp b/src/steam/mobject/session/root.cpp index ee89fb7bb..d71ac75eb 100644 --- a/src/steam/mobject/session/root.cpp +++ b/src/steam/mobject/session/root.cpp @@ -19,9 +19,9 @@ #include "steam/mobject/session/root.hpp" -#include "common/query/defs-manager.hpp" +#include "vessel/query/defs-manager.hpp" -using lumiera::query::DefsManager; +using vessel::query::DefsManager; using lib::idi::EntryID; namespace steam { diff --git a/src/steam/mobject/session/root.hpp b/src/steam/mobject/session/root.hpp index 54075b50e..fcdec9d07 100644 --- a/src/steam/mobject/session/root.hpp +++ b/src/steam/mobject/session/root.hpp @@ -33,7 +33,7 @@ #include "lib/idi/entry-id.hpp" -namespace lumiera { +namespace vessel { namespace query { class DefsManager; }} @@ -59,7 +59,7 @@ namespace session { */ class Root : public Meta { - lumiera::query::DefsManager& defaults_; + vessel::query::DefsManager& defaults_; ///////////TODO: timespan fields here or already in class Meta?? ///////////TODO: any idea about the purpose of root's "timespan"?? ///////TICKET #448 @@ -73,7 +73,7 @@ namespace session { virtual bool isValid() const; public: - Root (lumiera::query::DefsManager&); + Root (vessel::query::DefsManager&); static lib::idi::EntryID getID(); static lib::idi::EntryID getAssetID(); diff --git a/src/steam/mobject/session/scope-locator.hpp b/src/steam/mobject/session/scope-locator.hpp index 1f13dd265..8e4d704dc 100644 --- a/src/steam/mobject/session/scope-locator.hpp +++ b/src/steam/mobject/session/scope-locator.hpp @@ -93,7 +93,7 @@ namespace session { friend class lib::DependencyFactory; private: - lumiera::QueryResolver const& theResolver(); + vessel::QueryResolver const& theResolver(); }; diff --git a/src/steam/mobject/session/scope-query.hpp b/src/steam/mobject/session/scope-query.hpp index 84b45ee5d..9f1a6f981 100644 --- a/src/steam/mobject/session/scope-query.hpp +++ b/src/steam/mobject/session/scope-query.hpp @@ -34,7 +34,7 @@ #include "steam/mobject/placement.hpp" -#include "common/query/query-resolver.hpp" +#include "vessel/query/query-resolver.hpp" #include "lib/format-string.hpp" #include @@ -48,8 +48,8 @@ namespace session { using std::function; using std::placeholders::_1; - using lumiera::Goal; - using lumiera::Query; + using vessel::Goal; + using vessel::Query; diff --git a/src/steam/mobject/session/scope.cpp b/src/steam/mobject/session/scope.cpp index 031bda956..f1f70f651 100644 --- a/src/steam/mobject/session/scope.cpp +++ b/src/steam/mobject/session/scope.cpp @@ -35,7 +35,7 @@ #include "steam/mobject/session/session-service-explore-scope.hpp" #include "steam/mobject/mobject.hpp" #include "lib/iter-source.hpp" ////////////////////TICKET #493 : only using the IterSource wrapIter adapter here -#include "common/query/query-resolver.hpp" +#include "vessel/query/query-resolver.hpp" #include @@ -115,7 +115,7 @@ namespace session { * of contents discovery queries to the PlacementIndex * managed within the session */ - lumiera::QueryResolver const& + vessel::QueryResolver const& ScopeLocator::theResolver() { return SessionServiceExploreScope::getResolver(); diff --git a/src/steam/mobject/session/sess-manager-impl.cpp b/src/steam/mobject/session/sess-manager-impl.cpp index 6e80e4508..0b55ea23b 100644 --- a/src/steam/mobject/session/sess-manager-impl.cpp +++ b/src/steam/mobject/session/sess-manager-impl.cpp @@ -36,8 +36,8 @@ #include "steam/control/steam-dispatcher.hpp" #include "steam/config-resolver.hpp" #include "steam/asset/timeline.hpp" -#include "common/query/defs-manager.hpp" -#include "common/query.hpp" +#include "vessel/query/defs-manager.hpp" +#include "vessel/query.hpp" #include #include @@ -119,7 +119,7 @@ namespace session { REQUIRE (0 == session_->timelines.size(), "injecting default timeline, but session isn't pristine"); // issue a default query to retrieve or create a Timeline and a default Sequence - asset::PTimeline initialTimeline = session_->defaults (lumiera::Query ("")); + asset::PTimeline initialTimeline = session_->defaults (vessel::Query ("")); // these got registered automatically ENSURE (1 == session_->timelines.size()); diff --git a/src/steam/mobject/session/session-interface-modules.hpp b/src/steam/mobject/session/session-interface-modules.hpp index 141a4c847..31afc3ce9 100644 --- a/src/steam/mobject/session/session-interface-modules.hpp +++ b/src/steam/mobject/session/session-interface-modules.hpp @@ -65,7 +65,7 @@ #include "steam/asset/timeline.hpp" #include "steam/asset/sequence.hpp" #include "steam/mobject/session/element-query.hpp" -#include "common/query/defs-manager.hpp" +#include "vessel/query/defs-manager.hpp" @@ -87,7 +87,7 @@ namespace session { struct SessionInterfaceModules : util::NonCopyable { - lumiera::query::DefsManager defaultsManager_; + vessel::query::DefsManager defaultsManager_; mobject::session::ElementQuery elementQueryAPI_; TimelineTracker timelineRegistry_; SequenceTracker sequenceRegistry_; diff --git a/src/steam/mobject/session/session-service-explore-scope.hpp b/src/steam/mobject/session/session-service-explore-scope.hpp index e6082aa4b..8883d7875 100644 --- a/src/steam/mobject/session/session-service-explore-scope.hpp +++ b/src/steam/mobject/session/session-service-explore-scope.hpp @@ -38,7 +38,7 @@ #define MOBJECT_SESSION_SESSION_SERVICE_EXPLORE_SCOPE_H #include "steam/mobject/placement.hpp" -#include "common/query/query-resolver.hpp" +#include "vessel/query/query-resolver.hpp" @@ -59,7 +59,7 @@ namespace session { */ struct SessionServiceExploreScope { - static lumiera::QueryResolver const& getResolver(); + static vessel::QueryResolver const& getResolver(); static PlacementMO& getScope (PlacementMO const&); static PlacementMO& getScope (PlacementMO::ID const&); diff --git a/src/steam/mobject/session/session.cpp b/src/steam/mobject/session/session.cpp index f72aecd42..b8f39bd88 100644 --- a/src/steam/mobject/session/session.cpp +++ b/src/steam/mobject/session/session.cpp @@ -25,7 +25,7 @@ #include "steam/mobject/session.hpp" #include "steam/mobject/session/session-impl.hpp" #include "steam/mobject/session/sess-manager-impl.hpp" -//#include "common/query/defs-manager.hpp" +//#include "vessel/query/defs-manager.hpp" #include "lib/symbol.hpp" #include "lib/depend.hpp" diff --git a/src/steam/mobject/session/specific-contents-query.hpp b/src/steam/mobject/session/specific-contents-query.hpp index b521103e7..ae7abf644 100644 --- a/src/steam/mobject/session/specific-contents-query.hpp +++ b/src/steam/mobject/session/specific-contents-query.hpp @@ -28,7 +28,7 @@ #include "steam/mobject/session/scope-query.hpp" #include "steam/mobject/placement.hpp" -#include "common/query/query-resolver.hpp" +#include "vessel/query/query-resolver.hpp" #include @@ -39,7 +39,7 @@ namespace session { using std::function; - using lumiera::QueryResolver; + using vessel::QueryResolver; /** diff --git a/src/steam/mobject/test-dummy-mobject.hpp b/src/steam/mobject/test-dummy-mobject.hpp index cccdefac7..aad699f33 100644 --- a/src/steam/mobject/test-dummy-mobject.hpp +++ b/src/steam/mobject/test-dummy-mobject.hpp @@ -31,7 +31,7 @@ #ifndef MOBJECT_TEST_DUMMY_MOBJECT_H #define MOBJECT_TEST_DUMMY_MOBJECT_H -#include "lib/test/run.hpp" +#include "test/run.hpp" #include "steam/mobject/builder/buildertool.hpp" #include "steam/mobject/session/abstractmo.hpp" diff --git a/src/steam/play/output-director.hpp b/src/steam/play/output-director.hpp index 93e402c89..fdf946ad1 100644 --- a/src/steam/play/output-director.hpp +++ b/src/steam/play/output-director.hpp @@ -30,16 +30,17 @@ #include "lib/depend.hpp" #include "lib/depend-inject.hpp" #include "steam/play/output-manager.hpp" -#include "common/subsys.hpp" +#include "vessel/subsys.hpp" #include "lib/nocopy.hpp" #include "lib/sync.hpp" #include -namespace lumiera { +namespace vessel { +namespace spine { class Play; -} +}} namespace steam { namespace play { @@ -61,8 +62,8 @@ namespace play { : util::NonCopyable , public lib::Sync<> { - using SigTerm = lumiera::Subsys::SigTerm; - using PlayServiceHandle = lib::DependInject::ServiceInstance; + using SigTerm = vessel::Subsys::SigTerm; + using PlayServiceHandle = lib::DependInject::ServiceInstance; PlayServiceHandle player_; ///////TODO more components and connections to manage here... diff --git a/src/steam/play/output-manager.hpp b/src/steam/play/output-manager.hpp index 88a6b4c18..135831681 100644 --- a/src/steam/play/output-manager.hpp +++ b/src/steam/play/output-manager.hpp @@ -24,7 +24,8 @@ #include "lib/error.hpp" #include "lib/nocopy.hpp" -#include "steam/play/output-slot.hpp" +#include "vault/out/timings.hpp" +#include "vault/out/output-slot.hpp" #include "steam/mobject/model-port.hpp" //#include @@ -42,6 +43,12 @@ namespace play { //using std::string; //using std::vector; + using vault::out::PlaybackUrgency; + using vault::out::TIMEBOUND; + using vault::out::NICE; + using vault::out::ASAP; + using vault::out::Timings; + using vault::out::OutputSlot; using std::shared_ptr; diff --git a/src/steam/play/output-slot-connection.hpp b/src/steam/play/output-slot-connection.hpp deleted file mode 100644 index 932962882..000000000 --- a/src/steam/play/output-slot-connection.hpp +++ /dev/null @@ -1,242 +0,0 @@ -/* - OUTPUT-SLOT-CONNECTION.hpp - implementation API for concrete output slots - - Copyright (C) - 2011, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -*/ - -/** @file output-slot-connection.hpp - ** Interface for concrete output implementations to talk to the OutputSlot frontend. - ** The OutputSlot concept helps to decouple the render engine implementation from the details - ** of handling external output connections. For this to work, a concrete implementation of such - ** an external output needs to integrate with the generic OutputSlot frontend, as used by the - ** engine. This generic frontend uses a PImpl, pointing to a ConnectionState object, which embodies - ** the actual implementation. Moreover, this actual implementation is free to use specifically crafted - ** OutputSlot::Connection elements to handle the ongoing output for individual channels. The latter - ** thus becomes the central implementation side API for providing actual output capabilities. - ** - ** @see OutputSlotProtocol_test - ** @see diagnostic-output-slot.hpp ////TODO - */ - - -#ifndef STEAM_PLAY_OUTPUT_SLOT_CONNECTION_H -#define STEAM_PLAY_OUTPUT_SLOT_CONNECTION_H - - -#include "lib/error.hpp" -#include "lib/nocopy.hpp" -#include "steam/play/output-slot.hpp" -#include "lib/scoped-collection.hpp" -#include "lib/iter-adapter-stl.hpp" -#include "lib/iter-source.hpp" -#include "lib/handle.hpp" -//#include "lib/time/timevalue.hpp" -//#include "steam/engine/buffer-provider.hpp" -//#include "steam/play/timings.hpp" -//#include "lib/sync.hpp" - -//#include -#include -#include -//#include - - -namespace steam { -namespace play { - - using steam::engine::BuffHandle; -//using steam::engine::BufferProvider; -//using lib::time::Time; -//using std::string; - using lib::iter_stl::eachElm; - -//using std::placeholders::_1; -//using std::bind; - using std::vector; -//using std::shared_ptr; - - - - - /** @internal represents the \em active - * point in each of the per-channel connections - * used when this OutputSlot is operational. - * - * # OutputSlot Core API - * - * Actually, this extension point towards the implementation - * of the actual output handling carries the core API of OutputSlot. - * Thus, the task of actually implementing an OutputSlot boils down - * to implementing this interface and providing a ConnectionState. - * - `lock()` announces this FrameID and the corresponding buffer - * to be in exclusive use by the client from now on - * - `transfer()` ends the client sided processing and initiates - * the outputting of the data found in the corresponding buffer. - * - `pushout()` actually pushes the denoted buffer to the output. - * Typically, `pushout()` is called from the `transfer()` - * implementation; yet it may as well be called from a separate - * service thread or some kind of callback. - * @note the meaning of FrameID is implementation defined. - * @note typically the concrete connection is non-copyable - */ - class OutputSlot::Connection - { - public: - virtual ~Connection(); - - virtual BuffHandle claimBufferFor(FrameID) =0; - virtual bool isTimely (FrameID, TimeValue) =0; - virtual void transfer (BuffHandle const&) =0; - virtual void pushout (BuffHandle const&) =0; - virtual void discard (BuffHandle const&) =0; - virtual void shutDown () =0; - }; - - - - /** - * Extension point for Implementation. - * The ConnectionState is where the concrete output - * handling implementation is expected to reside. - * OutputSlot is a frontend and accesses - * ConnectionState in the way of a PImpl. - */ - class OutputSlot::ConnectionState - : public OutputSlot::Allocation - , util::NonCopyable - { - public: - virtual ~ConnectionState() { } - - virtual Connection& access (uint) const =0; - }; - - - - - /* ===== Building blocks for OutputSlot implementation ===== */ - - /** Base for OutputSlot standard implementation */ - class OutputSlotImplBase - : public OutputSlot - { - protected: - template - class ConnectionManager; - }; - - - /** - * Maintaining a list of active connections. - * Base class for the typical implementation approach. - * Using this class is \em not mandatory. But obviously, - * we'd get to manage a selection of Connection objects - * representing the "active points" in several media channels - * connected through this OutputSlot. These Connection subclasses - * are what is referenced by the DataSink smart-ptrs handed out - * to the client code. As ConnectionState implements the Allocation - * API, it has the liability to create these DataSink smart-ptrs, - * which means to wire them appropriately and also provide an - * deleter function (here #shutdownConnection) to be invoked - * when the last copy of the smart-handle goes out of scope. - * - * The typical standard/base implementation provided here - * manages a collection of active Connection subclass objects. - */ - template - class OutputSlotImplBase::ConnectionManager - : public OutputSlot::ConnectionState - { - typedef lib::ScopedCollection Connections; - typedef OutputSlot::OpenedSinks OpenedSinks; - - Connections connections_; - - - /* == Allocation Interface == */ - - OpenedSinks - getOpenedSinks() - { - //////////////////////////TICKET #878 not re-entrant, lifecycle isn't clear - REQUIRE (this->isActive()); - return lib::iter_source::transform (eachElm(connections_), connectOutputSink); - } - - Timings - getTimingConstraints() - { - UNIMPLEMENTED ("find out about timing constraints"); //////////////////////////TICKET #831 - } - - bool - isActive() const - { - return 0 < connections_.size(); - } - - CON& - access (uint chanNr) const - { - return connections_[chanNr]; - } - - - protected: /* == API for OutputSlot-Impl == */ - void - init() ///< derived classes need to invoke this to build the actual connections - { - //////////////////////////TICKET #878 really build all at once? or on demand? - connections_.populate_by (&ConnectionManager::buildConnection, this); - } - - using ConnectionStorage = Connections::ElementHolder&; - - /** factory function to build the actual - * connection handling objects per channel */ - virtual void buildConnection(ConnectionStorage) =0; - - - ConnectionManager(uint numChannels) - : connections_(numChannels) - { } - - public: - virtual - ~ConnectionManager() - { } - - - - private: // Implementation details - - static DataSink - connectOutputSink (CON& connection) - { - TRACE (test, "activating Con=%p", &connection ); - - DataSink newSink; - newSink.activate(&connection, shutdownConnection); - return newSink; - } - - static void - shutdownConnection (OutputSlot::Connection* toClose) - { - REQUIRE (toClose); - toClose->shutDown(); - } - }; - - - - -}} // namespace steam::play -#endif diff --git a/src/steam/play/output-slot.cpp b/src/steam/play/output-slot.cpp deleted file mode 100644 index 7aed3ca3a..000000000 --- a/src/steam/play/output-slot.cpp +++ /dev/null @@ -1,114 +0,0 @@ -/* - OutputSlot - capability to transfer data to a physical output - - Copyright (C) - 2011, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -* *****************************************************************/ - - -/** @file output-slot.cpp - ** Generic implementation functionality common to all output connections. - */ - - -#include "lib/error.hpp" -#include "steam/play/output-slot.hpp" -#include "steam/play/output-slot-connection.hpp" - -#include - - -namespace steam { -namespace play { - - using std::vector; - - namespace error = lumiera::error; - - - - - OutputSlot::~OutputSlot() { } // emit VTables here.... - - OutputSlot::Allocation::~Allocation() { } - - OutputSlot::Connection::~Connection() { } - - - - - - /** whether this output slot is occupied - * @return true if currently unconnected and - * able to connect and handle output data - */ - bool - OutputSlot::isFree() const - { - return not this->state_; - } - - - /** claim this OutputSlot for active use as output sink(s). - * At any point, a given slot can only be used for a single - * ongoing output process (which may serve several channels though). - * The assumption is for the OutputSlot to be picked through a query - * to some OutputManater, so the parameters (resolution, sample rate...) - * should be suited for the intended use. Thus no additional configuration - * is necessary. - * @return Allocation representing the "connected state" from the client's POV. - * The client may retrieve the effectively required Timings from there, - * as well as the actual output sinks, ready for use. - * @remarks calls back into #buildState, where the concrete OutputSlot - * is expected to provide a private Connection implementation, - * subclassing OutputSlot::Allocation - */ - OutputSlot::Allocation& - OutputSlot::allocate() - { - if (not isFree()) - throw error::Logic ("Attempt to open/allocate an OutputSlot already in use."); - - state_.reset (this->buildState()); - return *state_; - } - - - void - OutputSlot::disconnect() - { - if (not isFree()) - state_.reset(0); - } - - - - /* === DataSink frontend === */ - - BuffHandle - DataSink::lockBufferFor(FrameID frameNr) - { - return impl().claimBufferFor(frameNr); - } - - - void - DataSink::emit (FrameID frameNr, BuffHandle const& data2emit, TimeValue currentTime) - { - OutputSlot::Connection& connection = impl(); - if (connection.isTimely(frameNr,currentTime)) - connection.transfer(data2emit); - else - connection.discard(data2emit); - } - - - - -}} // namespace steam::play diff --git a/src/steam/play/output-slot.hpp b/src/steam/play/output-slot.hpp deleted file mode 100644 index e1f10c447..000000000 --- a/src/steam/play/output-slot.hpp +++ /dev/null @@ -1,185 +0,0 @@ -/* - OUTPUT-SLOT.hpp - capability to transfer data to a physical output - - Copyright (C) - 2011, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -*/ - -/** @file output-slot.hpp - ** An (abstract) capability to send media data to an external output. - ** OutputSlot is the central metaphor for the organisation of actual (system level) outputs; - ** using this concept allows to separate and abstract the data calculation and the organisation - ** of playback and rendering from the specifics of the actual output sink. Actual output - ** possibilities can be added and removed dynamically from various components (vault, stage), - ** all using the same resolution and mapping mechanisms - ** - ** Each OutputSlot is an unique and distinguishable entity. It corresponds explicitly to an - ** external output, or a group of such outputs (e.g. left and right sound card output channels), - ** or an output file or similar capability accepting media content. Initially, an output slot - ** needs to be provided, configured and registered, using an implementation suitable for the - ** kind of media data to be sent (sound, video) and also suitable for the special circumstances - ** of the output capability (render a file, display video in a GUI widget, send video to some - ** full screen display, establish a Jack port, just use some kind of "sound out"). An output - ** slot is always limited to a single kind of media, and to a single connection unit, but - ** this connection may still be comprised of multiple channels - ** (e.g. stereoscopic video, multichannel sound). - ** - ** In order to be usable as output sink, an output slot needs to be \em allocated: At any time, - ** there may be only a single client using a given output slot this way. To stress this point: - ** output slots don't provide any kind of inherent mixing capability; any adaptation, mixing, - ** overlaying and sharing needs to be done within the nodes network producing the output data - ** to be fed into the slot. (in special cases, some external output capabilities -- e.g. the - ** Jack audio connection system -- may still provide additional mixing capabilities, - ** but that's beyond the scope of the Lumiera application) - ** - ** Once allocated, the output slot returns a set of concrete sink handles (one for each - ** physical channel expecting data). The calculating process feeds its results into those handles. - ** Size and other characteristics of the data frames are assumed to be suitable, which typically - ** won't be verified at that level anymore. Besides that, the allocation of an output slot reveals - ** detailed timing expectations. The client is required to comply to these timings when _emitting_ - ** data -- they are even required to provide a current time specification, alongside with the data. - ** Based on this information, the output slot has the ability to handle timing failures gracefully; - ** the concrete output slot implementation is expected to provide some kind of de-click or - ** de-flicker facility, which kicks in automatically when a timing failure is detected. - ** - ** @see OutputSlotProtocol_test - ** @see diagnostic-output-slot.hpp - */ - - -#ifndef STEAM_PLAY_OUTPUT_SLOT_H -#define STEAM_PLAY_OUTPUT_SLOT_H - - -#include "lib/error.hpp" -#include "lib/handle.hpp" -#include "lib/time/timevalue.hpp" -#include "steam/engine/buffer-provider.hpp" -#include "steam/play/timings.hpp" -#include "lib/iter-source.hpp" -#include "lib/nocopy.hpp" - -#include - - -namespace steam { -namespace play { - - using steam::engine::BuffHandle; - using steam::engine::BufferProvider; - using lib::time::FrameCnt; - using lib::time::TimeValue; - using lib::time::Time; - - using std::unique_ptr; - - - - class DataSink; - - typedef FrameCnt FrameID; - - - - /****************************************************************************//** - * Interface: Generic output sink. - * An OutputSlot represents the possibility to send data through multiple - * channels to some kind of external sink (video in GUI window, video full screen, - * sound, Jack, rendering to file). Clients are expected to retrieve a suitably - * preconfigured implementation from some OutputManager instance. An OutputSlot - * needs to be \em claimed for output by invoking #allocate, which returns a - * representation of the connection state. This operation is exclusive. - * The actual \link DataSink output sinks \endlink can be retrieved - * through the Allocation object returned from there. - */ - class OutputSlot - : util::NonCopyable - { - - protected: - - /** active connections through this OutputSlot */ - class ConnectionState; - - unique_ptr state_; - - /** build the \em connected state, - * based on the existing configuration - * within this concrete OutputSlot */ - virtual ConnectionState* buildState() =0; - - - public: - virtual ~OutputSlot(); - - using OpenedSinks = lib::IterSource::iterator; - - class Allocation - { - public: - virtual OpenedSinks getOpenedSinks() =0; - virtual bool isActive() const =0; - - virtual Timings getTimingConstraints() =0; - - protected: - ~Allocation(); ///< never to be managed by clients directly - }; - - /** established output channel */ - class Connection; - - - /** can this OutputSlot be allocated? */ - bool isFree() const; - - /** claim this slot for exclusive use */ - Allocation& allocate(); - - /** disconnect from this OutputSlot - * @warning may block until DataSinks are gone */ - void disconnect(); - }; - - - - /** - * denotes an opened connection ready to receive media data for output. - * Each DataSink (handle) corresponds to an OutputSlot::Connection entry. - * Data is handed over frame wise in a two-phase protocol: first, the client - * gets exclusive access to an output buffer, and then, when done, the buffer - * is handed over by an #emit call. - */ - class DataSink - : public lib::Handle - { - - public: - BuffHandle lockBufferFor(FrameID); - void emit(FrameID, BuffHandle const&, TimeValue currentTime = Time::ANYTIME); - - - friend bool operator== (DataSink const& sink1, DataSink const& sink2) - { - return not (sink1 and sink2) - or (sink1 and sink2 - and & sink1.impl() == & sink2.impl()); - } - friend bool operator!= (DataSink const& sink1, DataSink const& sink2) - { - return not (sink1 == sink2); - } - }; - - - - - -}} // namespace steam::play -#endif diff --git a/src/steam/play/play-controller.cpp b/src/steam/play/play-controller.cpp index a460b42ed..ce457e697 100644 --- a/src/steam/play/play-controller.cpp +++ b/src/steam/play/play-controller.cpp @@ -15,18 +15,17 @@ /** @file play-controller.cpp ** Implementation of the control- / management frontend(s) exposed ** to clients for talking to an ongoing PlayProcess. This includes - ** the implementation of lumiera::Play::Controller, exposed directly + ** the implementation of vessel::spine::Play::Controller, exposed directly ** from the Player facade interface. Besides that primary control ** interface, this translation unit also contains the implementation ** for several extended play controllers targeted at special usage ** situations ** - ** @see lumiera::DummyPlayer ** @see stage::PlaybackController usage example */ -#include "steam/play/play-controller.hpp" +#include "vessel/spine/play-facade.hpp" #include "steam/play/play-process.hpp" #include "lib/time/timevalue.hpp" @@ -34,7 +33,8 @@ //#include //#include -namespace lumiera { +namespace vessel { +namespace spine { using lib::time::Time; using lib::time::Duration; @@ -191,14 +191,14 @@ namespace lumiera { -} +}}// namespace vessel::spine namespace steam{ namespace play{ // using std::string; -// using lumiera::Subsys; +// using vessel::Subsys; // using std::bind; diff --git a/src/steam/play/play-controller.hpp b/src/steam/play/play-controller.hpp deleted file mode 100644 index ab9f19e1e..000000000 --- a/src/steam/play/play-controller.hpp +++ /dev/null @@ -1,60 +0,0 @@ -/* - PLAY-CONTROLLER.hpp - frontend handle to control an play process - - Copyright (C) - 2011, Hermann Vosseler - -  **Lumiera** is free software; you can redistribute it and/or modify it -  under the terms of the GNU General Public License as published by the -  Free Software Foundation; either version 2 of the License, or (at your -  option) any later version. See the file COPYING for further details. - -*/ - -/** @file play-controller.hpp - ** Functionality for driving and controlling an ongoing playback or render. - ** - ** @see lumiera::DummyPlayer - ** @see stage::PlaybackController usage example - */ - - -#ifndef STEAM_PLAY_PLAY_CONTROLLER_H -#define STEAM_PLAY_PLAY_CONTROLLER_H - - -#include "steam/play/play-service.hpp" -//#include "include/display-facade.h" -//#include "common/instancehandle.hpp" -//#include "lib/singleton-ref.hpp" -#include "lib/nocopy.hpp" - -//#include - - -namespace steam { -namespace play { - - using std::string; - - - /****************************************************************//** - * TODO do we still need that?????? - * Isn't lumiera::Play::Controller already sufficient? - */ - class PlayController - : util::NonCopyable - { - - public: - - private: - }; - - - - - - -}} // namespace steam::play -#endif diff --git a/src/steam/play/play-process.cpp b/src/steam/play/play-process.cpp index 5cfd2751a..ad96fa114 100644 --- a/src/steam/play/play-process.cpp +++ b/src/steam/play/play-process.cpp @@ -34,7 +34,7 @@ namespace steam { namespace play { // using std::string; -// using lumiera::Subsys; +// using vessel::Subsys; // using std::bind; diff --git a/src/steam/play/play-process.hpp b/src/steam/play/play-process.hpp index 8ac6c3386..01619ba72 100644 --- a/src/steam/play/play-process.hpp +++ b/src/steam/play/play-process.hpp @@ -33,7 +33,6 @@ ** the playback or rendering state and goals, which then causes the PlayProcess ** to reconfigure the ongoing or planned calculations. ** - ** @see lumiera::DummyPlayer ** @see stage::PlaybackController usage example */ @@ -46,7 +45,7 @@ #include "lib/nocopy.hpp" //#include "include/dummy-player-facade.h" //#include "include/display-facade.h" -//#include "common/instancehandle.hpp" +//#include "vessel/instancehandle.hpp" //#include "lib/singleton-ref.hpp" #include "steam/mobject/model-port.hpp" #include "steam/engine/calc-stream.hpp" @@ -62,9 +61,9 @@ namespace steam { namespace play { // using std::string; -// using lumiera::Subsys; -// using lumiera::Display; -// using lumiera::DummyPlayer; +// using vessel::Subsys; +// using vessel::Display; +// using vessel::DummyPlayer; using util::isnil; using steam::mobject::ModelPort; using std::function; @@ -114,7 +113,7 @@ namespace play { * * @note PlayProcess entities are never directly visible. * Client code handles and controls these processes - * through a lumiera::Play::Controller returned + * through a vessel::spine::Play::Controller returned * from the PlayService. */ class PlayProcess diff --git a/src/steam/play/play-service.cpp b/src/steam/play/play-service.cpp index 5be3aefd9..a28ed7a59 100644 --- a/src/steam/play/play-service.cpp +++ b/src/steam/play/play-service.cpp @@ -21,7 +21,8 @@ #include "lib/error.hpp" -#include "include/play-facade.hpp" +#include "lib/sync.hpp" +#include "vessel/spine/play-facade.hpp" #include "steam/play/play-service.hpp" #include "steam/play/play-process.hpp" #include "steam/play/render-configurator.hpp" @@ -35,8 +36,13 @@ #include - namespace lumiera { + namespace error { + LUMIERA_ERROR_DEFINE (CANT_PLAY, "unable to build playback or render process for this configuration"); +} } + +namespace vessel { +namespace spine { Play::~Play() { } // emit VTables here... @@ -44,18 +50,14 @@ namespace lumiera { /** static storage for the facade access front-end */ lib::Depend Play::facade; - - namespace error { - LUMIERA_ERROR_DEFINE (CANT_PLAY, "unable to build playback or render process for this configuration"); - } -}//(End) namespace lumiera +}}//(End) namespace vessel::spine namespace steam { namespace play { //using std::string; -//using lumiera::Subsys; +//using vessel::Subsys; //using std::bind; using lib::Sync; using lib::RecursiveLock_NoWait; @@ -77,7 +79,7 @@ namespace play { } // (End) hidden service impl details - using lumiera::Play; + using vessel::spine::Play; typedef POutputManager Output; @@ -155,7 +157,7 @@ namespace play { * This service allows to create individual PlayProcess instances * to \em perform a timeline or similar model object, creating * rendered data for output. Client code is assumed to access - * this service through the lumiera::Play facade. + * this service through the vessel::spine::Play facade. */ PlayService::PlayService() : pTable_(new ProcessTable) @@ -196,7 +198,8 @@ namespace play { }} // namespace steam::play -namespace lumiera { +namespace vessel { +namespace spine { /* ==== convenience shortcuts for creating a PlayProcess ==== */ @@ -265,4 +268,4 @@ namespace lumiera { } -} +}}// namespace vessel::spine diff --git a/src/steam/play/play-service.hpp b/src/steam/play/play-service.hpp index 9bdbd11de..b55c4b9d6 100644 --- a/src/steam/play/play-service.hpp +++ b/src/steam/play/play-service.hpp @@ -14,7 +14,7 @@ /** @file play-service.hpp ** Primary service of the Player subsystem: Starting and controlling render/playback. ** This is the implementation level service interface for the "Player". Client code - ** should access this service through the lumiera::Play facade interface. + ** should access this service through the vessel::spine::Play facade interface. ** ** The player and render control subsystem allows to start and control playback and ** rendering at any given collection of model ports (the conceptual exit points @@ -50,7 +50,7 @@ #include "lib/error.hpp" #include "lib/nocopy.hpp" -#include "include/play-facade.hpp" +#include "vessel/spine/play-facade.hpp" #include //#include @@ -65,9 +65,6 @@ namespace steam { namespace play { using std::string; -//using lumiera::Subsys; -//using lumiera::Display; -//using lumiera::DummyPlayer; // class DummyImageGenerator; @@ -91,10 +88,10 @@ namespace play { * by the OutputDirector, which acts as a central hub * for the Player subsystem. Clients should always * access this functionality through the - * lumiera::Play facade interface. + * vessel::spine::Play facade interface. */ class PlayService - : public lumiera::Play + : public vessel::spine::Play , util::NonCopyable { std::unique_ptr pTable_; diff --git a/src/steam/play/render-configurator.cpp b/src/steam/play/render-configurator.cpp index 58f5ed405..bb600e6f8 100644 --- a/src/steam/play/render-configurator.cpp +++ b/src/steam/play/render-configurator.cpp @@ -16,6 +16,8 @@ ** Implementation functionality to bridge between engine::PlayService and engine::EngineService. ** The former allows to start playback (or rendering) as a high-level activity, while the latter ** can manage and perform [calculation streams](\ref CalcStream) + ** + ** @todo 2026 : the concept seems valid, but needs to be refined and integrated with play processing ///////TICKET #868 */ @@ -37,7 +39,6 @@ namespace play { namespace error = lumiera::error; // using std::string; -// using lumiera::Subsys; using std::shared_ptr; using std::bind; using std::placeholders::_1; @@ -67,7 +68,7 @@ namespace play { { REQUIRE (outputResolver_); OutputSlot& slot = outputResolver_->getOutputFor (port); - if (!slot.isFree()) + if (not slot.isActive()) throw error::State{"unable to acquire a suitable output slot" /////////////////////TICKET #197 #816 --- could use util::_Fmt here , LERR_(CANT_PLAY)}; return slot; @@ -77,16 +78,16 @@ namespace play { engine::CalcStreams buildCalculationStreams (ModelPort port, OutputSlot& output) { - OutputSlot::Allocation& activeOutputConnection = output.allocate(); - Timings nominalTimings = activeOutputConnection.getTimingConstraints() - .constrainedBy(playbackTimings_); + Timings nominalTimings = output.timingConstraints() + .constrainedBy(playbackTimings_); - return activateEngine (port, nominalTimings, activeOutputConnection, renderQuality_); + //////////////////////////////////////////TICKET #878 consider to store a copy of the OutputSlot (≙ref-couting handle) somewhere in a »PlayProcess« to keep it alive + return activateEngine (port, nominalTimings, output, renderQuality_); } engine::CalcStreams - activateEngine (ModelPort port, Timings timings, OutputSlot::Allocation& activeOutputConnection, RenderQuality quality) + activateEngine (ModelPort port, Timings timings, OutputSlot& activeOutputConnection, RenderQuality quality) { return EngineService::instance().calculate (port, timings, activeOutputConnection, quality); } diff --git a/src/steam/play/render-configurator.hpp b/src/steam/play/render-configurator.hpp index de1c7b9e0..cc6f62cd7 100644 --- a/src/steam/play/render-configurator.hpp +++ b/src/steam/play/render-configurator.hpp @@ -31,13 +31,14 @@ #include "lib/nocopy.hpp" //#include "include/dummy-player-facade.h" //#include "include/display-facade.h" -//#include "common/instancehandle.hpp" +//#include "vessel/instancehandle.hpp" //#include "lib/singleton-ref.hpp" #include "steam/mobject/model-port.hpp" #include "steam/play/play-process.hpp" #include "steam/engine/calc-stream.hpp" -#include "steam/play/output-slot.hpp" #include "steam/play/output-manager.hpp" +#include "vault/out/output-slot.hpp" +#include "vault/out/timings.hpp" //#include "lib/iter-source.hpp" //#include "lib/util.hpp" @@ -50,11 +51,10 @@ namespace steam { namespace play { // using std::string; -// using lumiera::Subsys; -// using lumiera::Display; -// using lumiera::DummyPlayer; // using util::isnil; using std::function; + using vault::out::Timings; + using vault::out::OutputSlot; using steam::mobject::ModelPort; // typedef steam::play::POutputManager POutputManager; diff --git a/src/steam/play/sound/jack-output.hpp b/src/steam/play/sound/jack-output.hpp index 31e04b3a2..e8ccf63cc 100644 --- a/src/steam/play/sound/jack-output.hpp +++ b/src/steam/play/sound/jack-output.hpp @@ -15,8 +15,6 @@ ** ** @todo empty placeholder to underpin the claim: yes we will support Jack Audio! ** - ** @see lumiera::Display - ** @see lumiera::DummyPlayer ** @see stage::PlaybackController usage example */ @@ -26,7 +24,7 @@ //#include "include/display-facade.h" -//#include "common/instancehandle.hpp" +//#include "vessel/instancehandle.hpp" //#include "lib/singleton-ref.hpp" //#include "lib/scoped-ptrvect.hpp" // diff --git a/src/steam/streamtype.hpp b/src/steam/streamtype.hpp index affcd4fa5..358bee8d7 100644 --- a/src/steam/streamtype.hpp +++ b/src/steam/streamtype.hpp @@ -26,8 +26,9 @@ #define STEAM_STREAMTYPE_H #include "lib/symbol.hpp" -//#include "common/query.hpp" +//#include "vessel/query.hpp" #include "lib/idi/entry-id.hpp" +#include "vault/mem/buffhandle.hpp" @@ -104,12 +105,11 @@ namespace steam { class TypeTag ; /** - * placeholder type for the contents of a data buffer. - * The actual buffer will always be provided by a - * library implementation; throughout the engine, - * it's just hidden behind a DataBuffer pointer. + * placeholder marker for the contents of a data buffer. + * The actual buffer will always be provided by a library implementation; + * Access is coordinated by a BufferProvider. */ - struct DataBuffer { }; + using DataBuffer = vault::mem::Buff; virtual bool operator== (ImplFacade const& other) const =0; @@ -184,10 +184,4 @@ namespace steam { } // namespace steam - - -namespace lumiera { - using steam::StreamType; -} - #endif /*STEAM_STREAMTYPE_H*/ diff --git a/src/test/DIR_INFO b/src/test/DIR_INFO new file mode 100644 index 000000000..d51353d19 --- /dev/null +++ b/src/test/DIR_INFO @@ -0,0 +1 @@ +Testrunner and test support utilities -- but not the unit-tests themselves diff --git a/src/lib/test/diagnostic-output.hpp b/src/test/diagnostic-output.hpp similarity index 82% rename from src/lib/test/diagnostic-output.hpp rename to src/test/diagnostic-output.hpp index 4ab9f851b..d595f7127 100644 --- a/src/lib/test/diagnostic-output.hpp +++ b/src/test/diagnostic-output.hpp @@ -23,12 +23,12 @@ */ -#ifndef LIB_TEST_DIAGNOSTIC_OUTPUT_H -#define LIB_TEST_DIAGNOSTIC_OUTPUT_H +#ifndef TEST_DIAGNOSTIC_OUTPUT_H +#define TEST_DIAGNOSTIC_OUTPUT_H #include "lib/format-cout.hpp" -#include "lib/test/test-helper.hpp" +#include "test/test-helper.hpp" #include #include @@ -36,10 +36,9 @@ using std::string; -namespace lib { namespace test{ -}} // namespace lib::test +}// namespace test @@ -50,12 +49,12 @@ namespace test{ * Macro to print types and expressions to STDOUT, * using Lumiera's string conversion framework */ -#define SHOW_TYPE(_TY_) \ - cout << "typeof( " << STRINGIFY(_TY_) << " )= " << lib::test::showType<_TY_>() <()" << " ? = " << ::test::showType<_TY_>() <; @@ -103,7 +104,7 @@ namespace test{ buildSearchFilter(Log const& srcSeq) { using Cursor = lib::iter::CursorGear; - return iter::chainSearch (Cursor{srcSeq.begin(), srcSeq.end()}); + return lib::iter::chainSearch (Cursor{srcSeq.begin(), srcSeq.end()}); } } @@ -243,7 +244,7 @@ namespace test{ template EventMatch& on (const X *const targetObj) { - string targetID = idi::instanceTypeID (targetObj); + string targetID = instanceTypeID (targetObj); return this->on(targetID); } @@ -305,7 +306,7 @@ namespace test{ template explicit EventLog (const X *const obj) - : EventLog(idi::instanceTypeID (obj)) + : EventLog(instanceTypeID (obj)) { } @@ -337,7 +338,7 @@ namespace test{ EventLog& clear (const X *const obj) { - return clear (idi::instanceTypeID (obj)); + return clear (instanceTypeID (obj)); } @@ -387,7 +388,7 @@ namespace test{ EventLog& call (const X *const targetObj, string function, ARGS const& ...args) { - return call (idi::instanceTypeID (targetObj), function, args...); + return call (instanceTypeID (targetObj), function, args...); } template @@ -532,5 +533,5 @@ namespace test{ -}} // namespace lib::test -#endif /*LIB_TEST_EVENT_LOG_H*/ +} // namespace test +#endif /*TESTSUPPORT_EVENT_LOG_H*/ diff --git a/src/lib/test/microbenchmark-adaptor.hpp b/src/test/microbenchmark-adaptor.hpp similarity index 96% rename from src/lib/test/microbenchmark-adaptor.hpp rename to src/test/microbenchmark-adaptor.hpp index 3733d3a02..262783b2f 100644 --- a/src/lib/test/microbenchmark-adaptor.hpp +++ b/src/test/microbenchmark-adaptor.hpp @@ -27,14 +27,13 @@ */ -#ifndef LIB_TEST_MICROBENCHMARK_ADAPTOR_H -#define LIB_TEST_MICROBENCHMARK_ADAPTOR_H +#ifndef TEST_MICROBENCHMARK_ADAPTOR_H +#define TEST_MICROBENCHMARK_ADAPTOR_H #include "lib/meta/function.hpp" #include "lib/meta/util.hpp" -namespace lib { namespace test{ namespace microbenchmark { @@ -158,5 +157,5 @@ namespace microbenchmark { -}}} // namespace lib::test::microbenchmark -#endif /*LIB_TEST_MICROBENCHMARK_ADAPTOR_H*/ +}} // namespace test::microbenchmark +#endif /*TEST_MICROBENCHMARK_ADAPTOR_H*/ diff --git a/src/lib/test/microbenchmark.hpp b/src/test/microbenchmark.hpp similarity index 97% rename from src/lib/test/microbenchmark.hpp rename to src/test/microbenchmark.hpp index e8e4c5630..13c5ac6e2 100644 --- a/src/lib/test/microbenchmark.hpp +++ b/src/test/microbenchmark.hpp @@ -41,8 +41,8 @@ */ -#ifndef LIB_TEST_MICROBENCHMARK_H -#define LIB_TEST_MICROBENCHMARK_H +#ifndef TEST_MICROBENCHMARK_H +#define TEST_MICROBENCHMARK_H #include "lib/meta/function.hpp" @@ -50,13 +50,12 @@ #include "lib/sync-barrier.hpp" #include "lib/thread.hpp" -#include "lib/test/microbenchmark-adaptor.hpp" +#include "test/microbenchmark-adaptor.hpp" #include -namespace lib { namespace test{ namespace { @@ -146,6 +145,7 @@ namespace test{ inline auto threadBenchmark(FUN const& subject, const size_t repeatCnt = DEFAULT_RUNS) { + using lib::SyncBarrier; using std::chrono::steady_clock; using Dur = std::chrono::duration; @@ -195,5 +195,5 @@ namespace test{ -}} // namespace lib::test -#endif /*LIB_TEST_MICROBENCHMARK_H*/ +}// namespace test +#endif /*TESTSUPPORT_MICROBENCHMARK_H*/ diff --git a/src/lib/test/run.hpp b/src/test/run.hpp similarity index 94% rename from src/lib/test/run.hpp rename to src/test/run.hpp index 035633af9..c403100b3 100644 --- a/src/lib/test/run.hpp +++ b/src/test/run.hpp @@ -24,15 +24,15 @@ */ -#ifndef TESTHELPER_RUN_H -#define TESTHELPER_RUN_H +#ifndef TESTSUITE_RUN_H +#define TESTSUITE_RUN_H #include "lib/error.hpp" #include "lib/random.hpp" - -#include "lib/test/suite.hpp" #include "lib/util.hpp" +#include "test/suite.hpp" + #include #include @@ -57,7 +57,7 @@ namespace test { virtual void run(Arg arg) = 0; void seedRand(); ///< draw a new random seed from a common nucleus, and re-seed the default-Gen. - lib::Random makeRandGen(); ///< build a dedicated new RandomGen, seeded from the default-Gen + static lib::Random makeRandGen(); ///< build a dedicated new RandomGen, seeded from the default-Gen static string firstTok (Arg); ///< conveniently pick the first token from the argument line static uint firstVal (Arg, uint =1); ///< conveniently use some number given as argument, with optional default }; @@ -118,4 +118,4 @@ using lib::defaultGen; Launch<_TEST_CLASS_> run_##_TEST_CLASS_##_(STRINGIFY(_TEST_CLASS_), _GROUPS_); -#endif /*TESTHELPER_RUN_H*/ +#endif /*TESTSUITE_RUN_H*/ diff --git a/src/lib/test/suite.cpp b/src/test/suite.cpp similarity index 98% rename from src/lib/test/suite.cpp rename to src/test/suite.cpp index d0b6af2b4..eaa0557f3 100644 --- a/src/lib/test/suite.cpp +++ b/src/test/suite.cpp @@ -20,13 +20,15 @@ #include "lib/error.hpp" #include "lib/symbol.hpp" +#include "lib/random-reseed.hpp" #include "lib/format-cout.hpp" -#include "lib/test/suite.hpp" -#include "lib/test/run.hpp" #include "lib/cmdline.hpp" #include "lib/random.hpp" #include "lib/util.hpp" +#include "test/suite.hpp" +#include "test/run.hpp" + #include #include #include @@ -210,7 +212,7 @@ namespace test { void Test::seedRand() { - lib::defaultGen.reseed (suiteSeed); + lib::reseed_for_test (suiteSeed); } diff --git a/src/lib/test/suite.hpp b/src/test/suite.hpp similarity index 96% rename from src/lib/test/suite.hpp rename to src/test/suite.hpp index b7e7137e5..d072209f8 100644 --- a/src/lib/test/suite.hpp +++ b/src/test/suite.hpp @@ -28,8 +28,8 @@ */ -#ifndef TESTHELPER_SUITE_H -#define TESTHELPER_SUITE_H +#ifndef TEST_SUITE_H +#define TEST_SUITE_H #include #include @@ -75,4 +75,4 @@ namespace test { } // namespace test -#endif +#endif /*TEST_SUITE_H*/ diff --git a/src/lib/test/temp-dir.hpp b/src/test/temp-dir.hpp similarity index 92% rename from src/lib/test/temp-dir.hpp rename to src/test/temp-dir.hpp index 5c20aea88..ae2d995c6 100644 --- a/src/lib/test/temp-dir.hpp +++ b/src/test/temp-dir.hpp @@ -18,8 +18,8 @@ */ -#ifndef LIB_TEST_TEMP_DIR_H -#define LIB_TEST_TEMP_DIR_H +#ifndef TESTSUPPORT_TEMP_DIR_H +#define TESTSUPPORT_TEMP_DIR_H #include "lib/error.hpp" @@ -27,6 +27,7 @@ #include "lib/random.hpp" #include "include/limits.hpp" #include "lib/format-string.hpp" +#include "lib/format-obj.hpp" #include "lib/file.hpp" #include "lib/util.hpp" @@ -34,9 +35,7 @@ #include -namespace lib { namespace test{ - namespace error = lumiera::error; using util::_Fmt; @@ -44,7 +43,7 @@ namespace test{ using std::string; namespace { - Literal TEMPFILE_PREFIX = "Lux"; + lib::Literal TEMPFILE_PREFIX = "Lux"; } @@ -98,7 +97,7 @@ namespace test{ auto tmpDir = fs::temp_directory_path(); for (uint attempt=0; attempt -#include #include #include -namespace lib { -namespace test{ +namespace test { - typedef std::vector VecI; - template inline VEC getTestSeq_int (const uint NUM_ELMS) { VEC vec; for (uint i=0; i MapII; - typedef std::multimap MMapII; - typedef std::unordered_map HMapII; - typedef std::unordered_multimap HMMapII; + using VecI = std::vector; + + using MapII = std::map; + using MMapII = std::multimap; + + using HMapII = std::unordered_map; + using HMMapII = std::unordered_multimap; template @@ -82,10 +82,10 @@ namespace test{ pullOut (ITER const& i) { for (ITER ii(i); ii ; ++ii ) - std::cout << "::" << *ii; + cout << "::" << *ii; } - -}} // namespace lib::test -#endif /*LIB_TEST_TEST_COLL_H*/ + +}// namespace test +#endif /*TESTSUPPORT_TEST_COLL_H*/ diff --git a/tests/core/steam/engine/testframe.cpp b/src/test/test-frame.cpp similarity index 91% rename from tests/core/steam/engine/testframe.cpp rename to src/test/test-frame.cpp index 098d933b0..eea8d1e8b 100644 --- a/tests/core/steam/engine/testframe.cpp +++ b/src/test/test-frame.cpp @@ -11,7 +11,7 @@ * *****************************************************************/ -/** @file testframe.cpp +/** @file test-frame.cpp ** Implementation of fake data frames to support unit testing. ** The data generation is based on a _discriminator seed value,_ ** which is computed as a linear combination of a statically fixed anchor-seed @@ -33,9 +33,10 @@ #include "lib/error.hpp" #include "lib/random.hpp" +#include "lib/random-reseed.hpp" #include "lib/hash-standard.hpp" #include "lib/hash-combine.hpp" -#include "steam/engine/testframe.hpp" +#include "test/test-frame.hpp" #include "lib/nocopy.hpp" #include "lib/util.hpp" @@ -45,9 +46,7 @@ -namespace steam { -namespace engine{ -namespace test { +namespace test { namespace err = lumiera::error; using std::deque; @@ -133,11 +132,11 @@ namespace test { } - TestFrame& - accessAsTestFrame (void* memoryLocation) + TestFrame const& + accessAsTestFrame (void const* memoryLocation) { REQUIRE (memoryLocation); - return *reinterpret_cast (memoryLocation); + return *reinterpret_cast (memoryLocation); } @@ -192,11 +191,13 @@ namespace test { /** - * @remark this function should be invoked at the start of any test - * which requires reproducible data values in the TestFrame. + * @remark this function can be invoked whenever either completely new + * or reproducible data values are required in TestFrame instances. * It generates a new base seed to distinguish individual data frames. * The seed is drawn from the \ref lib::defaultGen, and thus will be * reproducible if the latter has been reseeded beforehand. + * @note A default registration setup ensures this function is invoked + * from each call of test::Test::seedRand() * @warning after invoking reseed(), the validity of previously generated * frames can no longer be verified. */ @@ -207,6 +208,11 @@ namespace test { dataSeed = drawSeed (lib::defaultGen); } + namespace { // ---------------automatic-registration-for-uint-tests----------------- + const uint RESEED_SLOT{1}; + static uint _registration_ = lib::register_at_seq (RESEED_SLOT, &TestFrame::reseed ); + } + @@ -304,9 +310,9 @@ namespace test { } bool - TestFrame::operator== (void* memLocation) const + TestFrame::operator== (void const* memLocation) const { - TestFrame& candidate (accessAsTestFrame (memLocation)); + TestFrame const& candidate (accessAsTestFrame (memLocation)); return candidate.isSane() and candidate == *this; } @@ -367,9 +373,9 @@ namespace test { } /** access current checksum without recomputing. */ HashVal - TestFrame::getChecksum() + TestFrame::getChecksum() const { - return accessHeader().checksum; + return unConst(this)->accessHeader().checksum; } bool @@ -420,16 +426,16 @@ namespace test { bool TestFrame::isAlive (void* memLocation) { - TestFrame& candidate (accessAsTestFrame (memLocation)); + TestFrame const& candidate (accessAsTestFrame (memLocation)); return candidate.isAlive(); } bool TestFrame::isDead (void* memLocation) { - TestFrame& candidate (accessAsTestFrame (memLocation)); + TestFrame const& candidate (accessAsTestFrame (memLocation)); return candidate.isDead(); } -}}} // namespace steam::engine::test +}// namespace test diff --git a/tests/core/steam/engine/testframe.hpp b/src/test/test-frame.hpp similarity index 94% rename from tests/core/steam/engine/testframe.hpp rename to src/test/test-frame.hpp index d3802a4b6..7604826f1 100644 --- a/tests/core/steam/engine/testframe.hpp +++ b/src/test/test-frame.hpp @@ -1,5 +1,5 @@ /* - TESTFRAME.hpp - test data frame (stub) for checking Render engine functionality + TEST-FRAME.hpp - test data frame (stub) for checking Render engine functionality Copyright (C) 2011, 2024 Hermann Vosseler @@ -11,7 +11,7 @@ */ -/** @file testframe.hpp +/** @file test-frame.hpp ** Unit test helper to generate fake test data frames. ** Each TestFrame holds a 1k buffer of byte data, which can be ** verified, accessed and manipulated to emulate media computations. @@ -30,8 +30,8 @@ */ -#ifndef STEAM_ENGINE_TESTFRAME_H -#define STEAM_ENGINE_TESTFRAME_H +#ifndef TEST_TEST_FRAME_H +#define TEST_TEST_FRAME_H #include "lib/integral.hpp" @@ -40,9 +40,7 @@ #include -namespace steam { -namespace engine { -namespace test { +namespace test { using lib::HashVal; @@ -67,11 +65,13 @@ namespace test { */ class TestFrame { + public: enum StageOfLife { CREATED, EMITTED, DISCARDED }; - static constexpr size_t BUFFSIZ = 1024; + + private: using _Arr = std::array; using _A64 = std::array; @@ -105,7 +105,7 @@ namespace test { /** recompute and store checksum based on current contents */ HashVal markChecksum(); - HashVal getChecksum(); + HashVal getChecksum() const; /** Helper to verify that a given memory location holds * an active TestFrame instance (created, not yet destroyed) @@ -124,7 +124,7 @@ namespace test { bool isValid() const; bool isPristine() const; - bool operator== (void* memLocation) const; + bool operator== (void const* memLocation) const; friend bool operator== (TestFrame const& f1, TestFrame const& f2) { return f1.contentEquals(f2); } friend bool operator!= (TestFrame const& f1, TestFrame const& f2) { return !f1.contentEquals(f2); } @@ -158,5 +158,5 @@ namespace test { TestFrame& testData (uint seqNr =0, uint chanNr =0); -}}} // namespace steam::engine::test -#endif +}// namespace test +#endif /*TEST_TEST_FRAME_H*/ diff --git a/src/lib/test/test-helper.cpp b/src/test/test-helper.cpp similarity index 76% rename from src/lib/test/test-helper.cpp rename to src/test/test-helper.cpp index 891a71cdf..e53aa86fd 100644 --- a/src/lib/test/test-helper.cpp +++ b/src/test/test-helper.cpp @@ -21,8 +21,8 @@ */ -#include "lib/test/test-helper.hpp" -#include "lib/test/tracking-dummy.hpp" +#include "test/test-helper.hpp" +#include "test/tracking-dummy.hpp" #include "lib/unique-malloc-owner.hpp" #include "lib/format-string.hpp" #include "lib/format-cout.hpp" @@ -33,7 +33,6 @@ using util::_Fmt; using std::string; -namespace lib { namespace test{ @@ -55,23 +54,6 @@ namespace test{ - /** @todo probably this can be done in a more clever way. Anyone...? - */ - string - randStr (size_t len) - { - static const string alpha{"aaaabbccddeeeeffgghiiiijjkkllmmnnooooppqqrrssttuuuuvvwwxxyyyyzz0123456789"}; - static const size_t MAXAL{alpha.size()}; - - string garbage(len,'\0'); - size_t p = len; - while (p) - garbage[--p] = alpha[rani (MAXAL)]; - return garbage; - } - - - /** * @internal check equality and print difference * @remark defined here to avoid inclusion of `` in header @@ -92,4 +74,4 @@ namespace test{ } -}} // namespace lib::test +}// namespace test diff --git a/src/lib/test/test-helper.hpp b/src/test/test-helper.hpp similarity index 65% rename from src/lib/test/test-helper.hpp rename to src/test/test-helper.hpp index 4f3a73515..7cb136b60 100644 --- a/src/lib/test/test-helper.hpp +++ b/src/test/test-helper.hpp @@ -19,22 +19,22 @@ ** reference kind (value, lvalue, rvalue) of a template parameter instantiation. ** For GNU compatible compilers, we expose also the interface to the internal ** ABI for [demangling type names](\ref demangleCxx). - ** - ** @note this header is included into a large number of tests. + ** @remark util::showType also uses this ABI and indicates also reference / pointer + ** @warning this header is included into a large number of tests. ** @see TestHelper_test ** @see TestHelperDemangling_test ** */ -#ifndef LIB_TEST_TEST_HELPER_H -#define LIB_TEST_TEST_HELPER_H +#ifndef TESTSUPPORT_TEST_HELPER_H +#define TESTSUPPORT_TEST_HELPER_H #include "lib/symbol.hpp" #include "lib/meta/trait.hpp" #include "lib/time/timevalue.hpp" -#include "lib/test/transiently.hpp" +#include "test/transiently.hpp" #include "lib/format-obj.hpp" #include "lib/random.hpp" @@ -48,22 +48,27 @@ -namespace lib { namespace test{ using lib::Literal; using std::string; + using lib::randStr; using lib::rani; + using lib::meta::enable_if; + using lib::meta::disable_if; using lib::meta::demangleCxx; + using lib::meta::humanReadableTypeID; + using util::showType; + using util::showTypes; - constexpr auto ROUGH_PRECISION = pow (10, -3); - constexpr auto EPSILON_ULP = 5; + constexpr auto ROUGH_PRECISION = pow (10, -3); + constexpr auto EPSILON_ULP = 5; template - constexpr inline meta::enable_if< std::is_floating_point, - bool > + constexpr inline enable_if< std::is_floating_point, + bool > roughEQ (F val, N target, F limit =ROUGH_PRECISION) { REQUIRE (0 < limit); @@ -72,8 +77,8 @@ namespace test{ template - constexpr inline meta::enable_if< std::is_floating_point, - F > + constexpr inline enable_if< std::is_floating_point, + F > ulp (F val) { val = fabs (val); @@ -86,8 +91,8 @@ namespace test{ } template - constexpr inline meta::enable_if< std::is_floating_point, - bool > + constexpr inline enable_if< std::is_floating_point, + bool > epsEQ (F val, N target, uint ulps =EPSILON_ULP) { return abs (val - target) < ulps * ulp (target); @@ -112,8 +117,8 @@ namespace test{ } template - inline meta::disable_if, - string > // note:: force invocations with pointer to the first overload + inline disable_if, + string > // note:: force invocations with pointer to the first overload showSizeof (T const& obj, CStr name =nullptr) { return showSizeof (&obj, name); @@ -133,9 +138,12 @@ namespace test{ string showRefKind() { - return std::is_lvalue_reference::value? "REF" - : std::is_rvalue_reference::value? "MOV" - : "VAL"; + using lib::meta::isLRef_v; + using lib::meta::isRRef_v; + + return isLRef_v? "REF" + : isRRef_v? "MOV" + : "VAL"; } @@ -162,116 +170,6 @@ namespace test{ - - namespace { // helper for printing type diagnostics - - template - struct TypeDiagnostics - { - using Type = X; - static constexpr auto prefix = ""; - static constexpr auto postfix = ""; - }; - template - struct TypeDiagnostics - { - using Type = X; - static constexpr auto prefix = "const "; - static constexpr auto postfix = ""; - }; - template - struct TypeDiagnostics - { - using Type = X; - static constexpr auto prefix = ""; - static constexpr auto postfix = "&"; - }; - template - struct TypeDiagnostics - { - using Type = X; - static constexpr auto prefix = ""; - static constexpr auto postfix = " &&"; - }; - template - struct TypeDiagnostics - { - using Type = X; - static constexpr auto prefix = ""; - static constexpr auto postfix = " const&"; - }; - template - struct TypeDiagnostics - { - using Type = X; - static constexpr auto prefix = "const "; - static constexpr auto postfix = " &&"; - }; - template - struct TypeDiagnostics - { - using Type = X; - static constexpr auto prefix = ""; - static constexpr auto postfix = " *"; - }; - template - struct TypeDiagnostics - { - using Type = X; - static constexpr auto prefix = "const "; - static constexpr auto postfix = " *"; - }; - template - struct TypeDiagnostics - { - using Type = X; - static constexpr auto prefix = "const "; - static constexpr auto postfix = " * const"; - }; - template - struct TypeDiagnostics - { - using Type = X; - static constexpr auto prefix = ""; - static constexpr auto postfix = " * const"; - }; - template - struct TypeDiagnostics - { - using Type = X; - static constexpr auto prefix = ""; - static constexpr auto postfix = " * const *"; - }; - } - - /** diagnostic type output, including const and similar adornments - * @warning operates after-the-fact and relies on mangled type names - * plus several heuristics. Output might thus not be entirely correct, - * especially when several levels of const, pointer and references are involved. - * If in doubt, place the TypeDebugger to reveal the type as the compiler sees it. - * @remarks the function lib::meta::typeStr removes adornments and does not work on all kinds - * of reference. This helper attempts to work around those limitations. - */ - template - inline string - showType() - { - using Case = TypeDiagnostics; - using Type = Case::Type; - - return Case::prefix - + meta::humanReadableTypeID (typeid(Type).name()) - + Case::postfix; - } - - template - string - showTypes() - { - return "<| " + ((showType()+", ") + ... + "|>"); - } - - /** helper for investigating a variadic argument pack * @remark you can pass arguments directly to this function, * but be sure to use `std::forward(args)...` when forwarding @@ -304,17 +202,13 @@ namespace test{ - /** create a random but not insane Time value between 1s ... 10min + 500ms */ + /** create a random yet plausible Time value between 1s ... 10min + 500ms */ inline lib::time::Time randTime () { return lib::time::Time (500 * rani(2), 1 + rani(600)); } - /** create garbage string of given length - * @return string containing arbitrary lower case letters and numbers - */ - string randStr (size_t len); /** @@ -360,7 +254,7 @@ namespace test{ bool verify (std::string const& actual) const; }; -}} // namespace lib::test +}// namespace test @@ -374,10 +268,10 @@ namespace test{ * CHECK (result23 == "[-100..100]"_expect); * \endcode */ -inline lib::test::ExpectString +inline ::test::ExpectString operator""_expect (CStr lit, size_t siz) { - return lib::test::ExpectString{lit, siz}; + return ::test::ExpectString{lit, siz}; } @@ -390,24 +284,24 @@ operator""_expect (CStr lit, size_t siz) * an assertion failure. In case of exception, the #lumiera_error * state is checked, cleared and verified. */ -#define VERIFY_ERROR(ERROR_ID, ERRONEOUS_STATEMENT) \ - try \ - { \ - ERRONEOUS_STATEMENT ; \ - NOTREACHED("expected »%s« failure in: %s", \ - #ERROR_ID, #ERRONEOUS_STATEMENT); \ - } \ - catch (lumiera::Error& ex) \ - { \ - CHECK (ex.getID() \ - == lib::test::ExpectString{LUMIERA_ERROR_##ERROR_ID} );\ - lumiera_error(); \ - } \ - catch (...) \ - { \ - CHECK (lumiera_error_peek() \ - == lib::test::ExpectString{LUMIERA_ERROR_##ERROR_ID} ); \ - lumiera_error(); \ +#define VERIFY_ERROR(ERROR_ID, ERRONEOUS_STATEMENT) \ + try \ + { \ + ERRONEOUS_STATEMENT ; \ + NOTREACHED("expected »%s« failure in: %s", \ + #ERROR_ID, #ERRONEOUS_STATEMENT); \ + } \ + catch (lumiera::Error& ex) \ + { \ + CHECK (ex.getID() \ + == ::test::ExpectString{LUMIERA_ERROR_##ERROR_ID} );\ + lumiera_error(); \ + } \ + catch (...) \ + { \ + CHECK (lumiera_error_peek() \ + == ::test::ExpectString{LUMIERA_ERROR_##ERROR_ID} ); \ + lumiera_error(); \ } /** @@ -443,4 +337,4 @@ operator""_expect (CStr lit, size_t siz) #define MARK_TEST_FUN \ cout << "|" << endl << "| »"<<__FUNCTION__<<"«" <