diff --git a/doc/design/application/ApplicationStart.txt b/doc/design/application/ApplicationStart.txt index 6b48e57fa..41206a4c5 100644 --- a/doc/design/application/ApplicationStart.txt +++ b/doc/design/application/ApplicationStart.txt @@ -5,155 +5,144 @@ The Application : Start-up and Subsystems //MENU: label Start-up +.the purpose of »Application-main« +Lumiera is envisioned as a heavyweight yet coherent »**Application**« -- not so much +as platform, framework or operating system. And, in accordance with this focus, we +place no emphasis on possible configuration changes and state transitions within +the running application _as a whole._ And, while in fact it is possible to close +and load a new Session, most of the time we assume the application just to be +``up and running'' -- all required services to be available and interfaces to +be accessible. Without doubt, these are simplifications, but serve us well +to cut down complexity -- yet still there needs to be one dedicated realm +to deal with these specific concerns of configuration, dependency and lifecyle. -...There is now sort-of an ``application realm'', which doesn't belong strictly to -any one of the Layers. It serves the purpose of pulling up and tearing down the -application in a controlled fashion. And additionally, it provides the Interface -and Config core services. Within the application, we find a small number of -_Subsystems_, which are more or less independent. These subsystems are -conceptual entities and don't correspond 1:1 to a layer, an interface, a class -or a plugin. These subsystems are _of no relevance_ outside the mentioned -"application realm". When the application is in normal operational mode, we have -the usual components, interfaces, services, aggregated into the three Layers. -__Currently I've identified the following subsystems:__ +The Application Realm +--------------------- +So we treat all these concerns within a small and self contained structure, clearly +set apart from all the other layers, services and subsystems. This dedicated +_Application Realm_ is organised around the ``Application main object''.footnote:[ +This is the singleton `lumiera::AppState`, which is triggered by the `main` function +of the Lumiera Application. The sourcecode is kept in a separate folder 'src/common' +and linked into the shared library 'liblumieracommon.so'] +It serves the purpose of pulling up and tearing down the application in a controlled +fashion. And additionally, it provides the Interface and Config core services. +The act of building or tearing down this core realm and main object is what creates +the *Lifecycle* of the application. This is a succession of ``lifecycle phases'' -- +and almost all activities happen within the _operational phase,_ when everything +is ``up and running'' or ``just available''. -- Engine -- Builder -- Session -- Lumiera GUI -- Script runner -- Renderfarm node server +Subsystems +~~~~~~~~~~ +However, initially the application needs to be brought up, and at the end, all +parts need to be unwound cleanly. To organise this process, we identify a limited +number of *Subsystems* within the Application, which are more or less independent. +Each link:{ldoc}/design/architecture/Subsystems.html[Subsystem] is self contained +and groups some other parts and services, which typically work together and may +be mutually dependent. These subsystems represent a grouping, applied for the purpose +of starting and stopping the application in a regular way; they rather do not +correspond 1:1 to a layer, an interface, a class or a plugin. As a matter of fact, +they are _rather irrelevant_ outside the mentioned »Application realm«. A subsystem +may depend on other subsystems, which comprises a clear startup- and shutdown-ordering. +However, once the application is in normal operational mode, the subsystems turn +into a passive, guarding and managing role; the activities relevant for the +application's purpose rather rely on components, interfaces, services, all +aggregated into the three Layers »Stage«, »Steam« and »Vault«. -To deal with those subsystems, I've created an Interface to define the -operations and liabilities of a subsystem. Additionally, I assume that for each -subsystem there is a _Façade_, which the subsystem is free to implement as it +__We expect the following subsystems to be built eventually:__ + +Engine, Session, PlayOut, GUI, Script runner, Renderfarm node. + +Organisation of Subsystems +^^^^^^^^^^^^^^^^^^^^^^^^^^ +Not all subsystems need to be started for any use of the application. A script-driven +use, or a renderfarm node does not need a GUI. So there is an overall global operation +mode of the application, controlled through the launching options, and determined during +the startup phase. It is the responsibility of the _Application main object_ to +pull up required functionality, which in turn might result in pulling up +further subsystems as dependencies. + +Subsystems are defined by implementing the interface `lumiera::Subsys`, which acts +as façade to conduct the lifecycle, find out about dependencies and shut down +the subsystem in the end. So this interface, together with the _Subsystem Runner,_ +define a lifecycle protocol; each subsystem is free to implement this as it sees fit. Typically, this façade will load plugins, register and provide further -"business interfaces", and especially set up the _Layer separation interfaces_ +_business interfaces,_ and especially set up the _Layer separation interfaces_ which canalise any communication going on between the layers. -TIP: The code from my `startup` branch has meanwhile been merged to master. Look -http://git.lumiera.org/gitweb?p=LUMIERA;a=tree;h=a0a0e456a5b149df81b25a08358cd488631639fb;hb=a0a0e456a5b149df81b25a08358cd488631639fb[here] -for the code referred in the following discussion. - -- +common/subsys.hpp+ contains the subsystem interface mentioned above. -- +lumiera/main.cpp+ uses the subsystem instances provided by the facades, and - additionally the services of +lumiera::AppState+ (+common/appstate.hpp+) -- AppState represents the state as relevant for the "application realm", i.e. - it performs global initialisation and shutdown. See especially +AppState::init()+ -- see +vault/enginefacade.hpp|cpp+ as an (unimplemented) façade skeleton. +vault::EngineFacade::getDescriptor()+ - yields the subsystem interface -- the GuiFacade is somewhat special, because we want to load the GUI from a - shared library. This façade is basically completed and working, but it currently - just loads a dummy plugin. The implementation of the GuiFacade needs to be in - core (because it pulls the plugin); that's why I've put it into - +common/guifacade.cpp+, while the interface is in +gui/guifacade.hpp+ as usual. -- as an example for a _Layer separation interface_, I've implemented the - GuiNotificationFacade, which will be used by the lower layers to push - informations into the GUI (and finally to request the GUI to shut down). Layer - separation interfaces are considered part of the public Lumiera API, thus the - headers go into +src/include/**+ - * include/guinotification.h (C/C++ combined header) defines an C++ interface (abstract class) and a CLI interface. - * embedded into the interface is a factory, i.e. by stage::GuiNotification::facade() you get an instance... - * which actually is a proxy and routes any call through the instance of the - accompanying CLI interface which is defined within the interface/plugin system - * this in turn forwards to the implementation class in - gui/guinotificationfacade.cpp, which is assumed to live within the GUI - (shared lib) - +The *GUI Façade* is special, while in compliance with this protocol. The actual +UI is loaded from a plug-in at runtime,footnote:[This corresponds to the vision +to allow for different Lumiera UI's -- maybe to support different working styles +or target audiences. If such is actually feasible remains to be clarified as of +2020; even while decoupled on a technical level, the session still needs to make +a lot of assumptions regarding the UI's capabilities and needs.] +and so the implementation of this façade needs to reside in the application core +realm; it will start a `GuiRunner` to load and activate the GUI plug-in, which +then in turn has to open the public _GUI Notification_ façade. The latter is +one of the _Layer separation interfaces_ and comprises the actual way for the +lower layers to activate and interact with the user interface. Parallelism ------------ -Actually this system builds on the assumption, that starting each subsystem -doesn't block the overall start/main/stop thread. I.e. any subsystem is supposed -to spawn its control/event threads if necessary. Not every subsystem needs to -spawn threads though (for example, the session doesn't). In the current -implementation _no spawning of threads happens_. Likewise, I've commented out -the synchronisation primitives. + -[yellow-background]#TODO 2015# _meanwhile we do spawn threads and perform synchronisation_ +~~~~~~~~~~~ +Actually this scheme builds on the assumption that starting each subsystem will +not block the overall start/main/shutdown thread. Any subsystem is supposed +to spawn its own control/event threads if necessary. The Lumiera application +works fundamentally asynchronous. The user interface operates single threaded, +in accordance to long established best practices of UI programming. However, +any user interaction is translated into commands, sent down into the session +and handled there one by one. The result of processing such commands will be +pushed back up into the UI later and detached from the immediate interaction. +Likewise, the re-rendering caused by changes in the session is carried out +within the engine independently, relying on worker jobs and a thread pool. Initialisation and Lifecycle ---------------------------- -Basically, everything workes as discussed last spring in the -link:{ldoc}/devel/rfc/GlobalInitialization.html[GlobalInitialisation] -design entry. I've considered to switch to simple functions +pre_init(), init(), -...+ as proposed by Cehteh, but I'm not happy with this idea, because it creates -a coupling between anything which needs to be done in a certain initialisation -function. Actually, I prefer the usual approach of lifecycle events (or signals) -used in many application frameworks, i.e. the publisher-subscriber model. This -allows to keep the registration immediately within the implementation of a -facility, and it allows to add an arbitrary number of additional lifecycle -events, like *ON_SESSION_CLOSE*, *ON_BUILD*, *ON_EMERGENCY_EXIT*. +After some discussion,footnote:[See the +link:{ldoc}/devel/rfc/GlobalInitialization.html[GlobalInitialisation] RfC +from spring 2008. In the beginning, we all agreed to ``keep matters simple'' +and build an `init()` function within one central piece of code everyone knows +and hooks into. However, while the outline of the application emerged, there +was a tension between the concern about _over-engineering_ versus the concern +about _tangled and unmanageable complexity._ At some point, an alternative +implementation based on lifecycle callbacks was elaborated, which then turned +into the solution described here. Lumiera then ceased to be the typical UI +application started by GTK, and the existing GTK code was retrofitted to +launch from within a plug-in.] +the design leaned toward loosely coupled parts and a formal lifecycle; which +saves us from investigating and coding up the various interdependencies +explicitly. Rather, the parts of the application have to comply to +link:{ldoc}/design/architecture/Subsystems.html#lifecycle[Lifecycle Phases], +and each part has to care for its own state transitions, invoked through +_lifecycle callbacks._ We can distinguish two distinct models how to deal +with lifecycle, and both are equally acceptable: -Basically we have now the following steps and events happening -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +- Assuming that operations happen in response to some client's request, + this activation should go through a _service interface._ Interfaces + can be opened and closed in Lumiera, and this is accomplished by + hooking them up below some subsystem. +- However, some parts carry out continuous activities, and in that case + a _lifecycle hook_ should be registered, to limit activities to the + appropriate lifecycle phase. - * *ON_BASIC_INIT* runs "somewhere" in the static initialisation phase before - main(). To schedule something there, you need to place a statc C++ variable. - This should be reserved for very basic initialisation tasks which need to be - done even prior to any global initialisation, e.g. the NOBUG_INIT is done this - way, the config system, maybe setting up a dispatcher table based on (compile - time) type information (for the rules engine). - * +AppState::init()+ cares for bringing up the plugin loader and opens the config-interface. +Application Start +~~~~~~~~~~~~~~~~~ + * some fundamental language-level facilities will be prepared during + _static initialisation._ At some point, execution enters `main(argc,arvv)`. + * `AppState::init()` brings up the plugin loader and opens the config-interface. * ...followed by triggering *ON_GLOBAL_INIT* - * +main()+ pulls up the subsystems according to the command line options. - * within each subsystem, façade interfaces shall be opened with the interface - system. Any initialisation of components should be tied to these. - * as a rule and if possible, _any service should be written such as to come up on demand._ + * the main thread then pulls up the subsystems (`AppState::maybeStart(subsystem)`), + according to the command line options. + * within each subsystem, façade interfaces will be opened through the + interface/plug-in system. + * At this point, the GUI plug-in is loaded and launched, the windows created, + the UI event loop starts and the application becomes live. * shutdown or failure of _any given subsystem_ initiates the shutdown sequence - by requesting all other running subsystems to terminate - * there is an *ON_GLOBAL_SHUTDOWN* event, which can be used for normal cleanup. - In case of an emergency exit, this hook may be skipped - * alternatively, the *ON_EMERGENCY_EXIT* event is triggered. In case of nested - exceptions, this may happen twice. - * the AppState destructor tries to bring down the core systems (config-interface and pluginloader). - - -Demo Run --------- - * building with scons (default target) now yields a bin/guistart.so, besides the bin/lumiera exe the scons built created since day one. - * we need to set the LUMIERA_PLUGIN_PATH from the commandline to point to this bin directory, e.g. - * +++LUMIERA_PLUGIN_PATH=/home/hiv/devel/lumi/.libs NOBUG_LOG='test:TRACE,lumiera:TRACE,config:TRACE' bin/lumiera 2>&1 | egrep -v '(TODO)|(FIXME)'+++ ------- -00000392: configfacade.cpp:74: INFO: thread_1: Config: Config system ready. -00000394: main.cpp:55: NOTICE: thread_1: main: *** Lumiera NLE for Linux *** -00000395: appstate.cpp:114: TRACE: thread_1: init: initialising application core... -00000405: interface.c:54: TRACE: thread_1: lumiera_interface_open: lumieraorg_interface -00000406: interface.c:55: WARNING: thread_1: lumiera_interface_open: opening experimental interface: lumieraorg_interface_0_lumieraorg_interface -00000412: interface.c:168: TRACE: thread_1: lumiera_interface_open_interfacenode: lumieraorg_interface 0 () -00000437: config.c:199: NOTICE: thread_1: lumiera_config_get: envvar override for config LUMIERA_PLUGIN_PATH = /home/hiv/devel/lumi/ -00000444: config.c:199: NOTICE: thread_1: lumiera_config_get: envvar override for config LUMIERA_PLUGIN_PATH = /home/hiv/devel/lumi/ -00000467: appstate.cpp:132: TRACE: thread_1: init: Lumiera core started successfully. -00000469: appstate.cpp:140: TRACE: thread_1: maybeStart: maybe startup Builder...? -00000472: appstate.cpp:140: TRACE: thread_1: maybeStart: maybe startup Renderfarm node...? -00000475: appstate.cpp:140: TRACE: thread_1: maybeStart: maybe startup Lumiera GTK GUI...? -00000476: subsystemrunner.hpp:144: INFO: thread_1: triggerStartup: Starting subsystem "Lumiera GTK GUI" -00000477: interface.c:54: TRACE: thread_1: lumiera_interface_open: lumieraorg_GuiStarterPlugin -00000483: interface.c:168: TRACE: thread_1: lumiera_interface_open_interfacenode: lumieraorg_GuiStarterPlugin 0 () - *** Ha Ha Ha - this is the GuiStarterPlugin speaking! - now, the Lumiera GUI should be spawned.... - but actually nothing happens!!!!!!!!!!!!!! -00000491: appstate.cpp:140: TRACE: thread_1: maybeStart: maybe startup Script runner...? -00000494: appstate.cpp:171: NOTICE: thread_1: maybeWait: Shutting down Lumiera... -00000495: interface.c:230: TRACE: thread_1: lumiera_interface_close: -00000498: interface.c:258: TRACE: thread_1: lumiera_interfacenode_close: lumieraorg_GuiStarterPlugin 1 () -00000634: configfacade.cpp:83: TRACE: thread_1: ~Config: config system closed. -00000635: appstate.cpp:245: TRACE: thread_1: ~AppState: shutting down basic application layer... -00000643: interface.c:230: TRACE: thread_1: lumiera_interface_close: -00000646: interface.c:258: TRACE: thread_1: lumiera_interfacenode_close: lumieraorg_interface 1 () ------- - * incidentally, running... `lumiera --help` produces the following output ------- -00000392: configfacade.cpp:74: INFO: thread_1: Config: Config system ready. -00000394: main.cpp:55: NOTICE: thread_1: main: *** Lumiera NLE for Linux *** -Lumiera, the non linear video editor. Supported parameters: - -h [ --help ] produce help message - -f [ --session ] arg session file to load (UNIMPLEMENTED) - -s [ --script ] arg execute the given LUA script (UNIMPLEMENTED) - --headless start without GUI - -p [ --port ] arg open renderfarm node at given port (UNIMPLEMENTED) - -d [ --define ] arg enter definition into config system (UNIMPLEMENTED) ------- + by requesting all other running subsystems to terminate. In the typical case, + the UI subsystem will trigger this shutdown sequence, in response to closing + the main window. + * there is an *ON_GLOBAL_SHUTDOWN* event, which can be used for normal cleanup; + In case of an emergency exit, the *ON_EMERGENCY_EXIT* event is triggered alternatively. + * the AppState destructor tears down the core systems (config-interface and pluginloader). + * we establish a policy to _prohibit any non-local and non-trivial activities_ during the + tear-down phase after leaving the `main()` function. diff --git a/doc/design/architecture/Subsystems.txt b/doc/design/architecture/Subsystems.txt index 578892709..3184ed146 100644 --- a/doc/design/architecture/Subsystems.txt +++ b/doc/design/architecture/Subsystems.txt @@ -168,6 +168,8 @@ for link:{ldoc}/technical/library/Dependencies.html[lazily initialised dependenc establish the policy that *destructors must not rely on dependencies*. In fact, they should not do any tangible work at all, beyond releasing other resources. +anchor:lifecycle[] + Lifecycle Events ~~~~~~~~~~~~~~~~ The Application as a whole conducts a well defined lifecycle; whenever transitioning to the next phase,