DOC: locating of dependencies and resources at application start-up
a long standing TODO to document the actual start-up sequence, which is implemented this way since a long time now. There was an unwritten section in the "Linking and Application Structure", which seems the apropriate place for this kind of intricate techincal details. Last week, Benny Lyons was here on visit in munich and he was pondering the idea of an experimental secondary build system, as a way to learn more about the source structure of Lumiera. This reminded me to fill some missing parts of the documentation. Possibly this is also the right moment to land the GTK-3 transition?
This commit is contained in:
parent
f952cad073
commit
f17b1c8428
6 changed files with 146 additions and 49 deletions
|
|
@ -1,9 +1,20 @@
|
|||
Startup and Shutdown of Subsystems
|
||||
==================================
|
||||
:Date: Dec 2008
|
||||
:Author: Ichthyo
|
||||
|
||||
Lumiera: Startup and Shutdown of Subsystems
|
||||
===========================================
|
||||
//MENU: label Subsystem start
|
||||
|
||||
There is now sort-of an "application realm", which doesn't belong strictly to
|
||||
one of the Layers. It serves the purpose of pullig up and tearing down the
|
||||
.copied from 'pipapo.org'
|
||||
NOTE: This page was moved from _Cehteh_'s MoinMoin-wiki,
|
||||
which at that time was the first home of Lumiera development +
|
||||
The design and techniques outlined here are in use without major
|
||||
changes as of 2015 (and can be expected to remain this way). The
|
||||
documentation needs rewording in a more neutral and descriptive
|
||||
way and probably an introductory paragraph might be helpful [yellow-background]#TODO#
|
||||
|
||||
...There is now sort-of an ``application realm'', which doesn't belong strictly to
|
||||
any one of the Layers. It serves the purpose of pulling up and tearing down the
|
||||
application in a controlled fashion. And additionally, it provides the Interface
|
||||
and Config core services. Within the application, we find a small number of
|
||||
_Subsystems_, which are more or less independent. These subsystems are
|
||||
|
|
@ -21,38 +32,38 @@ __Currently I've identified the following subsystems:__
|
|||
- Script runner
|
||||
- Renderfarm node server
|
||||
|
||||
To deal with those subsystems, I've created an Interface to definesthe
|
||||
To deal with those subsystems, I've created an Interface to define the
|
||||
operations and liabilities of a subsystem. Additionally, I assume that for each
|
||||
subsystem there is a _Facade_, which the subsystem is free to implement as it
|
||||
sees fit. Typically, this facade will load plugins, register and provide further
|
||||
subsystem there is a _Façade_, which the subsystem is free to implement as it
|
||||
sees fit. Typically, this façade will load plugins, register and provide further
|
||||
"business interfaces", and especially set up the _Layer separation interfaces_
|
||||
which canalise any communication going on between the layers.
|
||||
|
||||
The code from my "startup" branch has meanwhile been merged to master. Look
|
||||
TIP: The code from my `startup` branch has meanwhile been merged to master. Look
|
||||
http://git.lumiera.org/gitweb?p=LUMIERA;a=tree;h=a0a0e456a5b149df81b25a08358cd488631639fb;hb=a0a0e456a5b149df81b25a08358cd488631639fb[here]
|
||||
for the code mentioned in the following discussion.
|
||||
for the code referred in the following discussion.
|
||||
|
||||
- +common/subsys.hpp+ contains the mentioned subsystem interface.
|
||||
- +common/subsys.hpp+ contains the subsystem interface mentioned above.
|
||||
- +lumiera/main.cpp+ uses the subsystem instances provided by the facades, and
|
||||
additionally the services of +lumiera::AppState+ (+common/appstate.hpp+)
|
||||
- AppState represents the state as relevant for the "application realm", i.e.
|
||||
it performs global initialisation and shutdown. See especially +AppState::init()+
|
||||
- see +backend/enginefacade.hpp|cpp+ as an (unimplemented) facade skeleton. +backend::EngineFacade::getDescriptor()+
|
||||
yields the subsytem interface
|
||||
- see +backend/enginefacade.hpp|cpp+ as an (unimplemented) façade skeleton. +backend::EngineFacade::getDescriptor()+
|
||||
yields the subsystem interface
|
||||
- the GuiFacade is somewhat special, because we want to load the GUI from a
|
||||
shared libary. This facade is basically completed and working, but it currently
|
||||
shared library. This façade is basically completed and working, but it currently
|
||||
just loads a dummy plugin. The implementation of the GuiFacade needs to be in
|
||||
core (because it pulls the plugin); that's why I've put it into
|
||||
+common/guifacade.cpp+, while the interface is in +gui/guifacade.hpp+ as usual.
|
||||
- as an example for a _Layer separation interface_, I've implemented the
|
||||
GuiNotificationFacade, which will be used by the lower layers to push
|
||||
informations into the gui (and finally to request the GUI to shut down). Layer
|
||||
informations into the GUI (and finally to request the GUI to shut down). Layer
|
||||
separation interfaces are considered part of the public Lumiera API, thus the
|
||||
headers go into +src/include/**+
|
||||
* include/guinotification.h (C/C++ combined header) defines an C++ interface (abstract class) and a CLI interface.
|
||||
* embedded into the interface is a factory, i.e. by gui::GuiNotification::facade() you get an instance...
|
||||
* which actually is a proxy and routes any call through the instance of the
|
||||
accompaning CLI interface which is defined within the interface/plugin system
|
||||
accompanying CLI interface which is defined within the interface/plugin system
|
||||
* this in turn forwards to the implementation class in
|
||||
gui/guinotificationfacade.cpp, which is assumed to live within the GUI
|
||||
(shared lib)
|
||||
|
|
@ -64,8 +75,9 @@ Actually this system builds on the assumption, that starting each subsystem
|
|||
doesn't block the overall start/main/stop thread. I.e. any subsystem is supposed
|
||||
to spawn its control/event threads if necessary. Not every subsystem needs to
|
||||
spawn threads though (for example, the session doesn't). In the current
|
||||
implementation *no spawning of threads happens*. Similarily, I've commented out
|
||||
the synchronisation primitives.
|
||||
implementation _no spawning of threads happens_. Likewise, I've commented out
|
||||
the synchronisation primitives. +
|
||||
[yellow-background]#TODO 2015# _meanwhile we do spawn threads and perform synchronisation_
|
||||
|
||||
|
||||
Initialisation and Lifecycle
|
||||
|
|
@ -78,23 +90,23 @@ a coupling between anything which needs to be done in a certain initialisation
|
|||
function. Actually, I prefer the usual approach of lifecycle events (or signals)
|
||||
used in many application frameworks, i.e. the publisher-subscriber model. This
|
||||
allows to keep the registration immediately within the implementation of a
|
||||
facility, and it allows to add an arbitrary numer of additional lifecycle
|
||||
facility, and it allows to add an arbitrary number of additional lifecycle
|
||||
events, like *ON_SESSION_CLOSE*, *ON_BUILD*, *ON_EMERGENCY_EXIT*.
|
||||
|
||||
Basically we've now the following steps and events happening
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
Basically we have now the following steps and events happening
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
* *ON_BASIC_INIT* runs "somewhere" in the static initialisation phase before
|
||||
main(). To schedule something there, you need to place a statc C++ variable.
|
||||
This should be reseverd for very basic initialisation tasks which need to be
|
||||
done even prior to any global initialisiation, e.g. the NOBUG_INIT is done this
|
||||
This should be reserved for very basic initialisation tasks which need to be
|
||||
done even prior to any global initialisation, e.g. the NOBUG_INIT is done this
|
||||
way, the config system, maybe setting up a dispatcher table based on (compile
|
||||
time) type information (for the rules engine).
|
||||
* +AppState::init()+ cares for bringing up the pluginloader and opens the config-interface.
|
||||
* ...followed by triggereing *ON_GLOBAL_INIT*
|
||||
* +AppState::init()+ cares for bringing up the plugin loader and opens the config-interface.
|
||||
* ...followed by triggering *ON_GLOBAL_INIT*
|
||||
* +main()+ pulls up the subsystems according to the command line options.
|
||||
* within each subsystem, facade interfaces shall be opened with the interface
|
||||
system. Any initialisation of components should be tied to them.
|
||||
* within each subsystem, façade interfaces shall be opened with the interface
|
||||
system. Any initialisation of components should be tied to these.
|
||||
* as a rule and if possible, _any service should be written such as to come up on demand._
|
||||
* shutdown or failure of _any given subsystem_ initiates the shutdown sequence
|
||||
by requesting all other running subsystems to terminate
|
||||
|
|
@ -139,7 +151,7 @@ Demo Run
|
|||
00000643: interface.c:230: TRACE: thread_1: lumiera_interface_close:
|
||||
00000646: interface.c:258: TRACE: thread_1: lumiera_interfacenode_close: lumieraorg_interface 1 ()
|
||||
------
|
||||
* similarily, running... `lumiera \--help`
|
||||
* incidentally, running... `lumiera --help` produces the following output
|
||||
------
|
||||
00000392: configfacade.cpp:74: INFO: thread_1: Config: Config system ready.
|
||||
00000394: main.cpp:55: NOTICE: thread_1: main: *** Lumiera NLE for Linux ***
|
||||
|
|
|
|||
|
|
@ -105,17 +105,17 @@ the lower layers, the GUI is _optional_ and the application is fully operational
|
|||
GTK Gui is built and loaded as Lumiera a plug-in.
|
||||
|
||||
.unit tests
|
||||
Since we're developing test-driven, about half of the overall code can be found in unit- and integration
|
||||
tests, residing below `test/`. There is a separate SConscript file, to define the various kinds of test
|
||||
artefacts to be created.
|
||||
Since our development is test-driven, about half of the overall code can be found in unit- and integration
|
||||
tests, residing below `test/`. There is a separate SConscript file, to define the various
|
||||
link:{ldoc}/technical/infra/TestSupport.html[kinds of test artefacts] to be created.
|
||||
|
||||
- plain-C tests are defined in _test-collections_, grouped thematically into several subdirectories.
|
||||
Here, each translation unit provides a separate +main()+ function and is linked into a stand-alone
|
||||
executable (yet still linked against the appropriate shared libraries of the main application layers)
|
||||
- the tests covering C++ components are organised into test-suites, residing in separate sub-trees.
|
||||
Currently (as of 10/2014), there is the *library suite* and the *proc components suite*. Here
|
||||
individual translation units define individual test case classes, which are linked together with
|
||||
a testrunner `main()` function.
|
||||
Currently (as of 5/2015), we link each sub-tree into a shared test library. Here
|
||||
individual translation units define individual test case classes. At the end, all these unit tests
|
||||
are linked together with a testrunner `main()` into the `test-suite` executable.
|
||||
|
||||
.research
|
||||
There is a separate subtree for research and experiments. The rationale being to avoid re-building most
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ Linking and Application Structure
|
|||
:Date: Autumn 2014
|
||||
:Author: Ichthyostega
|
||||
:toc:
|
||||
:toclevels: 3
|
||||
|
||||
This page focusses on some quite intricate aspects of the code structure,
|
||||
the build system organisation and the interplay of application parts on
|
||||
|
|
@ -90,9 +91,9 @@ Each piece of code incurs cost of various kinds
|
|||
produces debug information in each and every translation unit referring it.
|
||||
|
||||
Thus, for every piece of code we must ask ourselves how much _visible_ this
|
||||
code is. And we must consider the dependencies the code incurs. It pays off to
|
||||
turn something into a detail and ``push it into the backyard''. This explains
|
||||
why we're using the frontend - backend split so frequently.
|
||||
code is, need to be. And we must consider the dependencies the code incurs.
|
||||
It pays off to turn something into a detail and ``push it into the backyard''.
|
||||
This explains why we're using the frontend - backend split so frequently.
|
||||
|
||||
|
||||
Source and binary dependencies
|
||||
|
|
@ -150,12 +151,12 @@ layer operative, standalone, without the upper layer(s). The key is to introduce
|
|||
and then to _segregate_ along the realm of this abstraction, which needs to be chosen large enough
|
||||
in scope to cast the service and its contract entirely in terms of this abstraction, but at the same
|
||||
time it needs to be kept tight enough to prevent details of the client to leak into the abstraction.
|
||||
When this is achieved (which is the hard part), then any operations dealing with the abstraction solely
|
||||
When this is achieved (which is the hard part), then any operations dealing with the abstraction _solely_
|
||||
can be migrated into the entity offering the service, while the client hides the extended knowledge about
|
||||
the nature of the manipulated data behind a builder function footnote:[frequently this leads to the
|
||||
``type erasure'' pattern, where specific knowledge about the nature of the fabricated entities -- thus
|
||||
a specific type -- is relinquished and dropped once fabrication is complete], but retains ownership
|
||||
on these entities, passing just a reference to the service implementation. This move ties the binary
|
||||
a specific type -- is relinquished and dropped once fabrication is complete]. This way, the client retains
|
||||
ownership on these entities, passing just a reference to the service implementation. This move ties the binary
|
||||
dependency on the client implementation to this factory function -- as long as _this factory_ remains
|
||||
within the client, the decoupling works and eliminates binary cross dependencies.
|
||||
|
||||
|
|
@ -164,9 +165,93 @@ strict dependency checking (Link flag `--no-undefined`), so every violation of t
|
|||
hierarchical dependency order of our shared modules is spotted immediately during build.
|
||||
|
||||
|
||||
Finding dependencies at start-up
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
[yellow-background]#to be written#
|
||||
Locating dependencies at start-up
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
We hope for Lumiera to be not only installed on desktop systems, but also used in a studio
|
||||
or production context, where you'll use a given system just for the duration of one project.
|
||||
This might even be specific hardware booted with a Live-System, or it might be a ``headless''
|
||||
render-farm node. For this reason, we impose the explicit requirement that Lumiera must be
|
||||
fully *usable without installation*. Unzip the application into some folder, launch,
|
||||
start working. There might be some problems with required media handling libraries,
|
||||
but the basic idea is to use a self-contained bundle or sub-tree, and the application
|
||||
needs to locate all the further required resources actively on start-up.
|
||||
|
||||
On the other hand, we want Lumiera to be a good citizen, packaged in the usual way,
|
||||
compliant to the __F__ile__S__ystem __H__ierarchy standard. It turns out these two
|
||||
rather conflicting goals can be reconciled by leveraging some of the advanced features
|
||||
of the GNU dynamic linker: The application will figure out the whereabouts relatively,
|
||||
starting from the location of the executable, and with the help of some search paths
|
||||
and symlinks, the same mechanism can be made to work in the usual FSH compliant
|
||||
installation into `/usr/lib/lumiera` and `/usr/share/lumiera`
|
||||
|
||||
This way, we end up with a rather elaborate start-up sequence, where the application
|
||||
works out it's own installation location and establishes all the further resources
|
||||
step by step
|
||||
|
||||
. the first challenge are all the parts of the application built as dynamic libraries;
|
||||
effectively most of the application code resides in some shared modules. Since we
|
||||
most definitively do want a global link step in the build process, where unresolved
|
||||
symbols will be spotted, and we want a coherent application core, so we use
|
||||
dynamic linking right at start-up and thus need a way to make the linker locate
|
||||
our further modules and components relative to the executable. Fortunately, the
|
||||
GNU linker supports some extended attributes in the `.dynamic` section of ELF
|
||||
executables (known as the ``new style d-tags'')
|
||||
|
||||
* any executable may define an extended search path through the `RUNPATH` tag,
|
||||
which is searched to locate dynamic libraries and modules before looking
|
||||
into the standard directories
|
||||
footnote:[the linker also supports the old-style `RPATH` tag. In the glorious
|
||||
old days of ancient Unix, it was considered good practice to compile the
|
||||
installation location hard wired into the `RPATH` of each executable and library.
|
||||
Meanwhile, in the age of multi-arch installation and virtual machines, this practice
|
||||
is frowned upon and discouraged by many distributors. For the time being, our
|
||||
build system sets both `RPATH` and the new-style, more secure `RUNPATH` to the
|
||||
same value relative to the executable. We will cease using `RPATH` at
|
||||
some point in the future]
|
||||
* and this search patch may contain _relative_ entries, using the special
|
||||
magic token `$ORIGIN` to point at the directory holding the executable
|
||||
+
|
||||
By convention, the Lumiera buildsystem bakes in the search location `$ORIGIN/modules`
|
||||
-- so this subdirectory below the location of the executable is where all the dynamic
|
||||
modules of the application will be placed by default
|
||||
|
||||
. after the core application has been loaded and all direct dependencies are resolved,
|
||||
but still before entering `main()`, the class `lumiera::AppState` will be initialised,
|
||||
which in turn holds a member of type `lumiera::BasicSetup`. The latter will figure out
|
||||
the location of the executable footnote:[this is a Linux-only trick, using `/proc/self/exe`]
|
||||
and require a 'setup.ini' file in the same directory. This setup file is mandatory.
|
||||
|
||||
. from there, the _search paths_ are retrieved to locate the extended resources of the
|
||||
application. All these search paths are a colon separated list; the entries may
|
||||
optionally also use the token `$ORIGIN` to refer to the location of the main executable.
|
||||
The default version of 'setup.ini' is outfitted with search paths to cover both the
|
||||
situation of a self-contained bundle, but also the situation of a FSH compliant
|
||||
installation.
|
||||
|
||||
Lumiera.modulepath:: this is where the plugin loader looks for additional extensions and
|
||||
plug-ins, most notably the *GUI plugin*
|
||||
|
||||
Lumiera.gui:: defines the name of this GUI plugin, which is loaded and activated from
|
||||
`main()` -- unless Lumiera starts in ``headless'' mode
|
||||
|
||||
Lumiera.configpath:: all the extended application configuration will be picked up from
|
||||
these directories (_not yet implemented as of 2015_)
|
||||
|
||||
Gui.iconpath:: root of the folder structure used to load icons and similar graphical
|
||||
elements for the GTK-UI. Below, several subdirectories for various icon sizes are
|
||||
recognised. Actually, most of our icons are defined as SVG and rendered using
|
||||
libCairo during the build process.
|
||||
|
||||
Gui.resourcepath:: the place where the GTK-UI looks for further resources, most notably...
|
||||
|
||||
Gui.stylesheet:: the name of the CSS-stylesheet for GTK-3, which defines the
|
||||
application specific look, link:{ldoc}/technical/gui/guiTheme.html[skinning and theme].
|
||||
|
||||
While the first two steps, the relative locations `$ORIGIN/modules` and `$ORIGIN/setup.ini`
|
||||
are hard-wired, the further resolution steps rely on the contents of 'setup.ini' and are
|
||||
open for adjustments and reconfiguration, both for the packager or the advanced user.
|
||||
Any failure or mismatch during this start-up sequence will be considered fatal and abort
|
||||
the application execution.
|
||||
|
||||
|
||||
Transitive binary dependencies
|
||||
|
|
@ -194,7 +279,7 @@ Static registration magic
|
|||
|
||||
Relative dependency location
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Locating binary dependencies relative to the executable (as described above) is complicated when several
|
||||
Locating binary dependencies relative to the executable (as described above) gets complicated when several
|
||||
of _our own dynamically linked modules_ depend on each other transitively. For example, a plug-in might
|
||||
depend on `liblumierabackend.so`, which in turn depends on `liblumierasupport.so`. Now, when we link
|
||||
`--as-needed`, the linker will add the direct dependency, but omit the transitive dependency on the
|
||||
|
|
|
|||
|
|
@ -23,8 +23,8 @@
|
|||
/** @file appstate.hpp
|
||||
** Registering and managing primary application-global services.
|
||||
** This can be considered the "main" object of the Lumiera application
|
||||
** Besides encapsulating the logic for starting up the fundamental parts
|
||||
** of the application, there is a mechanism for registering \em subsystems
|
||||
** Besides encapsulating the logic to start up the fundamental parts of
|
||||
** the application, there is a mechanism for registering \em subsystems
|
||||
** to be brought up and shut down in order. AppState will issue the global
|
||||
** application lifecycle events (where other parts may have registered
|
||||
** callbacks) and provides the top-level catch-all error handling.
|
||||
|
|
|
|||
|
|
@ -77,7 +77,7 @@ namespace lumiera {
|
|||
"name of the Lumiera GUI plugin to load")
|
||||
("Lumiera.modulepath", opt::value<string>(),
|
||||
"search path for loadable modules. "
|
||||
"May us $ORIGIN to refer to the EXE location")
|
||||
"May use $ORIGIN to refer to the EXE location")
|
||||
("Lumiera.configpath", opt::value<string>(),
|
||||
"search path for extended configuration. "
|
||||
"Extended Config system not yet implemented "
|
||||
|
|
|
|||
|
|
@ -61,13 +61,13 @@ namespace lib {
|
|||
/**
|
||||
* Helper: Access a path Specification as a sequence of filesystem Paths.
|
||||
* This iterator class dissects a ':'-separated path list. The individual
|
||||
* components may use the symbol \c $ORIGIN to denote the directory holding
|
||||
* the current executable.
|
||||
* components may use the symbol \c $ORIGIN to refer to the directory
|
||||
* holding the current executable.
|
||||
* @note #next picks the current component and advances the iteration.
|
||||
*/
|
||||
class SearchPathSplitter
|
||||
: public BoolCheckable<SearchPathSplitter
|
||||
, boost::noncopyable>
|
||||
, boost::noncopyable>
|
||||
{
|
||||
string pathSpec_;
|
||||
sregex_iterator pos_,
|
||||
|
|
|
|||
Loading…
Reference in a new issue