From c7528bbe8add54150f6753c9136fc5b7f43b9ea6 Mon Sep 17 00:00:00 2001 From: Ichthyostega Date: Thu, 18 Sep 2025 23:13:55 +0200 Subject: [PATCH] clean-up: some RfCs became effectively final The purpose of RfCs is to channel and document discussions among a group of developers regarding questions of design. While there is unfortunately no longer a group of developers discussing matters and working together on the code base, some questions can be considered more or less settled, by being implemented and validated. - the Scheduler - structure of the Developer Documentation - Application installation as relocatable bundle --- doc/devel/meeting_summary/2011-04-13.txt | 1 + doc/devel/rfc/ApplicationInstall.txt | 36 ++++++++----- .../rfc/DeveloperDocumentationStructure.txt | 30 ++++++++++- doc/devel/rfc/SchedulerRequirements.txt | 51 ++++++++++++++----- .../ApplicationInstall.txt | 0 .../DeveloperDocumentationStructure.txt | 0 .../SchedulerRequirements.txt | 0 7 files changed, 91 insertions(+), 27 deletions(-) rename doc/devel/{rfc_pending => rfc_final}/ApplicationInstall.txt (100%) rename doc/devel/{rfc_pending => rfc_final}/DeveloperDocumentationStructure.txt (100%) rename doc/devel/{rfc_pending => rfc_final}/SchedulerRequirements.txt (100%) diff --git a/doc/devel/meeting_summary/2011-04-13.txt b/doc/devel/meeting_summary/2011-04-13.txt index 8d388c9c3..21fd0d915 100644 --- a/doc/devel/meeting_summary/2011-04-13.txt +++ b/doc/devel/meeting_summary/2011-04-13.txt @@ -355,6 +355,7 @@ revisit the relevant design process entries in each developer meeting. -> link:{rfc}/ApplicationInstall.html[RfC: Application Install] +[[application_install_proposal]] .-- the Application Install proposal -- [caption="☉Transcript☉ "] ---------------------------- diff --git a/doc/devel/rfc/ApplicationInstall.txt b/doc/devel/rfc/ApplicationInstall.txt index 60fa8990d..70ea3460e 100644 --- a/doc/devel/rfc/ApplicationInstall.txt +++ b/doc/devel/rfc/ApplicationInstall.txt @@ -5,20 +5,20 @@ ApplicationInstall [options="autowidth"] |==================================== -|*State* | _Draft_ +|*State* | _Final_ |*Date* | _Di 11 Jan 2011 17:00:55 CET_ |*Proposed by* | Ichthyostega |==================================== [abstract] ********************************************************************************* -Lumiera should be a _freely relocatable_ application bundle. +Lumiera should be a _freely relocatable_ application bundle. + Relying only on the relative folder structure within this bundle, the application will be fully functional at any location, provided that the external library dependencies are resolvable using the standard mechanisms of the platform. -The setup mechanism must be obvious, self-explanatory and must not rely -on compiled in magic or buildsystem trickery. Yet packaging into a FSH conforming -installation location should be supported by the same mechanisms. +This setup mechanism should be obvious and self-explanatory and must not rely +on compiled-in hidden magic or buildsystem trickery. Yet packaging into a fully +FSH conforming installation location should be supported by the same mechanisms. ********************************************************************************* Description @@ -66,7 +66,7 @@ In former days, it was common habit to compile-in a hard wired absolute policy forbids doing so. This is the result from numerous maintainability problems in the past. On the other hand, the GNU linker and other modern linkers support a relative resolution of shared modules directly accompanying -an specific executable. The Debian policy allows this, if and only if these +a specific executable. The Debian policy allows this, if and only if these shared modules are installed with the same binary package and only used by this specific executable(s). Together, this is exactly what we need to solve our requirement. @@ -76,7 +76,7 @@ and sets the +DT_RUNPATH+ with an value relative to +$ORIGIN+, which resolves to the path of the currently executing binary. Moreover, it is _sufficient_ to set this on the initial executable _only,_ because this creates a common searchpath for all lib resolution events in the scope of that loaded executable. -Besides that, we need to care that our private libraries have a unique +SONAME+, +Besides that, we need to care that our private libraries have an unique +SONAME+, in this case all starting with the prefix +liblumiera*+. Note moreover that this new-style +DT_RUNPATH+ indeed _can_ be overridden by an +LD_LIBRARY_PATH+ in the environment, should there be the need for very special experiments. @@ -172,10 +172,10 @@ up some very specific problems and constraints, which can be a serious threat when discovered late in the development process. The second alternative isn't really applicable IMHO. The original UNIX philosophy -breeds on an academic setup and really excels with small nifty commandline utils +draws from an academic setup and really excels with small nifty commandline utils combined by pipes, each specialised to do a single thing very well. These utils are more like the objects within our implementation. The concept of large -application software bundles and desktop software was always a bit alien +application software bundles and desktop software was always slightly alien within the classic UNIX environment. @@ -191,17 +191,20 @@ trickery, inline code compiled on-the-fly, relying on very specific and un-obvious behaviour of some build script, configuration via environment variables and a lot of similar idioms. These practices might be adequate in a quickly moving Research & Development setup, but turned out to be -not so helpful when it comes to industrial strength development, +not so helpful when it used within large-scale development efforts, as they are known to lead to maintenance problems. -//Conclusion -//---------- +Conclusion +---------- //conclusion: When approved (this proposal becomes a Final) // write some conclusions about its process: - +All the indicated clean-up was done and the bootstrap sequence could be implemented. + +The link:{ldoc}/devel/meeting_summary/2011-04-13.html#application_install_proposal[Apr.2011 developer meeting] +shortly touched on this proposal without raising any objections. +``... for example this application install .. is boring .. we need it, you did it well'' (_Cehteh_) @@ -221,5 +224,12 @@ try to bring these points to discussion separately. Ichthyostega:: 'So 13 Feb 2011 20:04:00 CET' +.State -> Final +After being established in 2011, this structure was used ever since. There was once +a problem with a bug in `ld.so` related to the transitive use of the `$origin` token; +this bug had been resolved quickly. Other than that, packaging and application start +always worked as intended -- so I consider this topic as settled. + +Ichthyostega:: '2025-09-18' //endof_comments: diff --git a/doc/devel/rfc/DeveloperDocumentationStructure.txt b/doc/devel/rfc/DeveloperDocumentationStructure.txt index 57ab7cbf3..041873654 100644 --- a/doc/devel/rfc/DeveloperDocumentationStructure.txt +++ b/doc/devel/rfc/DeveloperDocumentationStructure.txt @@ -5,7 +5,7 @@ Developer Documentation Structure [options="autowidth"] |==================================== -|*State* | _Draft_ +|*State* | _Final_ |*Date* | _Mon Aug 2 18:03:25 2010_ |*Proposed by* | Cehteh + Ichthyo |==================================== @@ -125,7 +125,21 @@ developers. //---------- //conclusion: When approvedd (this proposal becomes a Final) // write some conclusions about its process: +This RfC had been started initially by _Cehteh_ to propose a _simple structure_ -- +the ensuing discussion however showed that our documentation is already structured +in a mostly consistent way, which just happens to be not that simple. Some confusing +redundancies could be removed, as result from that discussion. _Ichthyo_ then edited +and extended the original RfC to reflect the existing structure. +A serious problem is posed by the rather tedious procedure to define cross-links in the +Asciidoc based web content. link:WebsiteSupportMarkup.html[Another RfC] was created to +discuss these problems in detail, leading to a solution proposal, which unfortunately +was never implemented (due to lack of developer capacity). This _somehow transitory_ +situation had the (adverse) effect that most detailed technical content was authored +within the Development TiddlyWiki. + +These remaining problems do not seem to be related to the _structure_ of the documentation +however, which seems to fulfil its purpose. So this RfC now counts as de-facto standard. @@ -175,4 +189,18 @@ documentation in textual form, which is inherentely structured. Ichthyostega:: 'Mo 09 Sep 2013 01:01:25 CEST' ~~ +.State -> Final +I am using this structure now since several years, without encountering any +serious mismatch or incompatibility. The situation with the cross-linking +generator, which still remains to be implemented, is obviously some kind +of impediment, since it prevents me from migrating more mature content +from the TiddlyWiki into the documentation section of the website. + +Furthermore, I should notice, as a limitation, that I am the _only one_ to +use this structure actively -- so the fact that I feel comfortable with +the arrangement should be taken with a grain of salt. Yet, on the other +hand, we have meanwhile a massive body of information integrated +into this structure, so that it can be considered more than +just a ``proposal for discussion'' + //endof_comments: diff --git a/doc/devel/rfc/SchedulerRequirements.txt b/doc/devel/rfc/SchedulerRequirements.txt index 5266a44fd..e3a90503d 100644 --- a/doc/devel/rfc/SchedulerRequirements.txt +++ b/doc/devel/rfc/SchedulerRequirements.txt @@ -5,7 +5,7 @@ SchedulerRequirements [options="autowidth"] |==================================== -|*State* | _Draft_ +|*State* | _Final_ |*Date* | _Mi 09 Jan 2013 12:04:03 CET_ |*Proposed by* | Ichthyostega |==================================== @@ -15,7 +15,7 @@ SchedulerRequirements Define the expected core properties and requirements of the Scheduler service. ******************************************************************************** -The rendering and playback subsystem relies on a Scheduler component to invoke +The rendering and playback subsystem relies on a *Scheduler* component to invoke individual frame rendering and resource fetching jobs. This RfC summarises the general assumptions and requirements other parts of the application are relying on @@ -77,13 +77,14 @@ knowledge of some specific job's termination. More precisely, we need to find ou when a ``stream of calculations'' has left a well defined domain -- and this can be modelled by the activation of specific marker jobs. It is possible to obtain that knowledge with some timing leeway, but in the end, this information needs -to arrive with absolutely reliability (violations leading to segfault). +to arrive with absolutely reliability (memory management will rely on them, +and thus failure might lead to out-of.memory or segfault). job scheduling modes ~~~~~~~~~~~~~~~~~~~~ The scheduler offers various modes of treatment on a per job base. The default is to -handle jobs time based with a moderate latency. Alternatively jobs can be handled as +handle jobs as _time-bound_, with a moderate latency. Alternatively jobs can be handled as _background_ jobs, as _freewheeling_ jobs (maximum usage of performance and bandwidth), or as _low-latency timed_ jobs. @@ -115,9 +116,9 @@ Tasks // List what needs to be done to implement this Proposal: // * first step [yellow-background]#WIP# * define the job interface ([green]#✔ done#) - * define a protocol for job state handling [red yellow-background]#TBD# - * define the representation of dependencies and the notifications in practice [red yellow-background]#TBD# - * verify the proposed requirements by an scheduler implementation sketch [red yellow-background]#TBD# + * define a protocol for job state handling ([green]#✔ done#) + * define the representation of dependencies and the notifications in practice ([green]#✔#) + * verify the proposed requirements by an scheduler implementation sketch ([green]#✔#) @@ -138,7 +139,7 @@ Pros Cons ^^^^ // fact list of the known/considered bad implications: - * there is no ``for-loop'' to base any playback control structures on + * there is no ``while-loop'' to base any playback control structures on * compliance to externally imposed deadlines and memory management are challenging. Alternatives @@ -150,8 +151,8 @@ Alternatives We do not want (1), since it is tied to an obsolete hardware model and lacks the ability to be adapted to the new kinds of hardware available today or to be expected in near future. We do not -want (2) since it essentially doesn't solve any problem, but rather pushes complexity into the -higher layers (Session, Stage), which are lacking the information about individual jobs and timing. +want (2) since it essentially doesn't solve any problem, but rather pushes complexity into the higher +layers (Session, Stage), which however are lacking information about individual jobs and timing. @@ -161,17 +162,20 @@ Rationale We use a scheduler to gain flexibility in controlling various aspects of computation and I/O usage. Moreover, we turn the scheduler into an interface between the Vault and Steam-Layer; while the exact outfitting of the individual jobs highly depends on internals of the Session and Engine models, -the properties of _actual job execution_, closely related to system programming are akin +the properties of _actual job execution_, closely related to system programming, are linked to the Vault. The actual requirements outlined in this RfC are derived from the internals of the player implementation, while _the way_ these requirements are defined, and especially the aspects _omitted_ from specification are derived from knowledge regarding the possible scheduler and vault layer implementation. -//Conclusion -//---------- +Conclusion +---------- //conclusion: When approved (this proposal becomes a Final) // write some conclusions about its process: +A solution closely following this proposal was implemented and became a +centrepiece of the Lumiera Render Engine. There was no further formal +decision, since _Ichthyo_ worked alone during the past years. @@ -189,6 +193,27 @@ can be developed independently of the scheduler implementation Ichthyostega:: 'Do 19 Sep 2013 21:31:07 CEST' ~~ +.State -> Final +After an extended intermission, where development was focused at GUI topics, the integration +work since 2023 strives at achieving an initial integration of the Render Engine. As part of the +https://issues.lumiera.org/ticket/1221[#1221 »PlaybackVerticalSlice«], an implementation of the +Scheduler, as proposed by this RfC, was +link:/x/imp/Scheduler.html[built from scratch and tested extensively]. Practical experience +during this effort leads to the following refinements: + +- the Scheduler does not invoke Jobs, + + but processes _verbs_ from an _Activity Language_ rather. +- this language in turn embodies a primitive to check a dependency count +- an external event may re-trigger such a chain of Activities within the Scheduler +- an extent-based memory manager can be based on deadlines of these Activities. +- the Scheduler becomes a passive management task, pulled by a active Workers. +- Workers manage their capacity through a statistical distribution of _sleep times._ + +In this form, the proposal can be considered as implemented. + +^(see -> https://git.lumiera.org/?p=LUMIERA;a=commit;h=d71eb37[d71eb37] +and https://issues.lumiera.org/ticket/1344[#1344])^ + +Ichthyostega:: '2024-04-20 01:55 CEST' //endof_comments: diff --git a/doc/devel/rfc_pending/ApplicationInstall.txt b/doc/devel/rfc_final/ApplicationInstall.txt similarity index 100% rename from doc/devel/rfc_pending/ApplicationInstall.txt rename to doc/devel/rfc_final/ApplicationInstall.txt diff --git a/doc/devel/rfc_pending/DeveloperDocumentationStructure.txt b/doc/devel/rfc_final/DeveloperDocumentationStructure.txt similarity index 100% rename from doc/devel/rfc_pending/DeveloperDocumentationStructure.txt rename to doc/devel/rfc_final/DeveloperDocumentationStructure.txt diff --git a/doc/devel/rfc_pending/SchedulerRequirements.txt b/doc/devel/rfc_final/SchedulerRequirements.txt similarity index 100% rename from doc/devel/rfc_pending/SchedulerRequirements.txt rename to doc/devel/rfc_final/SchedulerRequirements.txt