clean-up: some RfCs became effectively final
The purpose of RfCs is to channel and document discussions among a group of developers regarding questions of design. While there is unfortunately no longer a group of developers discussing matters and working together on the code base, some questions can be considered more or less settled, by being implemented and validated. - the Scheduler - structure of the Developer Documentation - Application installation as relocatable bundle
This commit is contained in:
parent
67695c97b8
commit
c7528bbe8a
7 changed files with 91 additions and 27 deletions
|
|
@ -355,6 +355,7 @@ revisit the relevant design process entries in each developer meeting.
|
|||
|
||||
-> link:{rfc}/ApplicationInstall.html[RfC: Application Install]
|
||||
|
||||
[[application_install_proposal]]
|
||||
.-- the Application Install proposal --
|
||||
[caption="☉Transcript☉ "]
|
||||
----------------------------
|
||||
|
|
|
|||
|
|
@ -5,20 +5,20 @@ ApplicationInstall
|
|||
|
||||
[options="autowidth"]
|
||||
|====================================
|
||||
|*State* | _Draft_
|
||||
|*State* | _Final_
|
||||
|*Date* | _Di 11 Jan 2011 17:00:55 CET_
|
||||
|*Proposed by* | Ichthyostega <prg@ichthyostega.de>
|
||||
|====================================
|
||||
|
||||
[abstract]
|
||||
*********************************************************************************
|
||||
Lumiera should be a _freely relocatable_ application bundle.
|
||||
Lumiera should be a _freely relocatable_ application bundle. +
|
||||
Relying only on the relative folder structure within this bundle, the application
|
||||
will be fully functional at any location, provided that the external library
|
||||
dependencies are resolvable using the standard mechanisms of the platform.
|
||||
The setup mechanism must be obvious, self-explanatory and must not rely
|
||||
on compiled in magic or buildsystem trickery. Yet packaging into a FSH conforming
|
||||
installation location should be supported by the same mechanisms.
|
||||
This setup mechanism should be obvious and self-explanatory and must not rely
|
||||
on compiled-in hidden magic or buildsystem trickery. Yet packaging into a fully
|
||||
FSH conforming installation location should be supported by the same mechanisms.
|
||||
*********************************************************************************
|
||||
|
||||
Description
|
||||
|
|
@ -66,7 +66,7 @@ In former days, it was common habit to compile-in a hard wired absolute
|
|||
policy forbids doing so. This is the result from numerous maintainability
|
||||
problems in the past. On the other hand, the GNU linker and other modern
|
||||
linkers support a relative resolution of shared modules directly accompanying
|
||||
an specific executable. The Debian policy allows this, if and only if these
|
||||
a specific executable. The Debian policy allows this, if and only if these
|
||||
shared modules are installed with the same binary package and only used by
|
||||
this specific executable(s). Together, this is exactly what we need to
|
||||
solve our requirement.
|
||||
|
|
@ -76,7 +76,7 @@ and sets the +DT_RUNPATH+ with an value relative to +$ORIGIN+, which resolves
|
|||
to the path of the currently executing binary. Moreover, it is _sufficient_
|
||||
to set this on the initial executable _only,_ because this creates a common
|
||||
searchpath for all lib resolution events in the scope of that loaded executable.
|
||||
Besides that, we need to care that our private libraries have a unique +SONAME+,
|
||||
Besides that, we need to care that our private libraries have an unique +SONAME+,
|
||||
in this case all starting with the prefix +liblumiera*+. Note moreover that this
|
||||
new-style +DT_RUNPATH+ indeed _can_ be overridden by an +LD_LIBRARY_PATH+ in the
|
||||
environment, should there be the need for very special experiments.
|
||||
|
|
@ -172,10 +172,10 @@ up some very specific problems and constraints, which can be a serious
|
|||
threat when discovered late in the development process.
|
||||
|
||||
The second alternative isn't really applicable IMHO. The original UNIX philosophy
|
||||
breeds on an academic setup and really excels with small nifty commandline utils
|
||||
draws from an academic setup and really excels with small nifty commandline utils
|
||||
combined by pipes, each specialised to do a single thing very well. These utils
|
||||
are more like the objects within our implementation. The concept of large
|
||||
application software bundles and desktop software was always a bit alien
|
||||
application software bundles and desktop software was always slightly alien
|
||||
within the classic UNIX environment.
|
||||
|
||||
|
||||
|
|
@ -191,17 +191,20 @@ trickery, inline code compiled on-the-fly, relying on very specific and
|
|||
un-obvious behaviour of some build script, configuration via environment
|
||||
variables and a lot of similar idioms. These practices might be adequate
|
||||
in a quickly moving Research & Development setup, but turned out to be
|
||||
not so helpful when it comes to industrial strength development,
|
||||
not so helpful when it used within large-scale development efforts,
|
||||
as they are known to lead to maintenance problems.
|
||||
|
||||
|
||||
|
||||
|
||||
//Conclusion
|
||||
//----------
|
||||
Conclusion
|
||||
----------
|
||||
//conclusion: When approved (this proposal becomes a Final)
|
||||
// write some conclusions about its process:
|
||||
|
||||
All the indicated clean-up was done and the bootstrap sequence could be implemented. +
|
||||
The link:{ldoc}/devel/meeting_summary/2011-04-13.html#application_install_proposal[Apr.2011 developer meeting]
|
||||
shortly touched on this proposal without raising any objections.
|
||||
``... for example this application install .. is boring .. we need it, you did it well'' (_Cehteh_)
|
||||
|
||||
|
||||
|
||||
|
|
@ -221,5 +224,12 @@ try to bring these points to discussion separately.
|
|||
|
||||
Ichthyostega:: 'So 13 Feb 2011 20:04:00 CET'
|
||||
|
||||
.State -> Final
|
||||
After being established in 2011, this structure was used ever since. There was once
|
||||
a problem with a bug in `ld.so` related to the transitive use of the `$origin` token;
|
||||
this bug had been resolved quickly. Other than that, packaging and application start
|
||||
always worked as intended -- so I consider this topic as settled.
|
||||
|
||||
Ichthyostega:: '2025-09-18'
|
||||
|
||||
//endof_comments:
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ Developer Documentation Structure
|
|||
|
||||
[options="autowidth"]
|
||||
|====================================
|
||||
|*State* | _Draft_
|
||||
|*State* | _Final_
|
||||
|*Date* | _Mon Aug 2 18:03:25 2010_
|
||||
|*Proposed by* | Cehteh + Ichthyo
|
||||
|====================================
|
||||
|
|
@ -125,7 +125,21 @@ developers.
|
|||
//----------
|
||||
//conclusion: When approvedd (this proposal becomes a Final)
|
||||
// write some conclusions about its process:
|
||||
This RfC had been started initially by _Cehteh_ to propose a _simple structure_ --
|
||||
the ensuing discussion however showed that our documentation is already structured
|
||||
in a mostly consistent way, which just happens to be not that simple. Some confusing
|
||||
redundancies could be removed, as result from that discussion. _Ichthyo_ then edited
|
||||
and extended the original RfC to reflect the existing structure.
|
||||
|
||||
A serious problem is posed by the rather tedious procedure to define cross-links in the
|
||||
Asciidoc based web content. link:WebsiteSupportMarkup.html[Another RfC] was created to
|
||||
discuss these problems in detail, leading to a solution proposal, which unfortunately
|
||||
was never implemented (due to lack of developer capacity). This _somehow transitory_
|
||||
situation had the (adverse) effect that most detailed technical content was authored
|
||||
within the Development TiddlyWiki.
|
||||
|
||||
These remaining problems do not seem to be related to the _structure_ of the documentation
|
||||
however, which seems to fulfil its purpose. So this RfC now counts as de-facto standard.
|
||||
|
||||
|
||||
|
||||
|
|
@ -175,4 +189,18 @@ documentation in textual form, which is inherentely structured.
|
|||
Ichthyostega:: 'Mo 09 Sep 2013 01:01:25 CEST' ~<prg@ichthyostega.de>~
|
||||
|
||||
|
||||
.State -> Final
|
||||
I am using this structure now since several years, without encountering any
|
||||
serious mismatch or incompatibility. The situation with the cross-linking
|
||||
generator, which still remains to be implemented, is obviously some kind
|
||||
of impediment, since it prevents me from migrating more mature content
|
||||
from the TiddlyWiki into the documentation section of the website.
|
||||
|
||||
Furthermore, I should notice, as a limitation, that I am the _only one_ to
|
||||
use this structure actively -- so the fact that I feel comfortable with
|
||||
the arrangement should be taken with a grain of salt. Yet, on the other
|
||||
hand, we have meanwhile a massive body of information integrated
|
||||
into this structure, so that it can be considered more than
|
||||
just a ``proposal for discussion''
|
||||
|
||||
//endof_comments:
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ SchedulerRequirements
|
|||
|
||||
[options="autowidth"]
|
||||
|====================================
|
||||
|*State* | _Draft_
|
||||
|*State* | _Final_
|
||||
|*Date* | _Mi 09 Jan 2013 12:04:03 CET_
|
||||
|*Proposed by* | Ichthyostega <prg@ichthyostega.de>
|
||||
|====================================
|
||||
|
|
@ -15,7 +15,7 @@ SchedulerRequirements
|
|||
Define the expected core properties and requirements of the Scheduler service.
|
||||
********************************************************************************
|
||||
|
||||
The rendering and playback subsystem relies on a Scheduler component to invoke
|
||||
The rendering and playback subsystem relies on a *Scheduler* component to invoke
|
||||
individual frame rendering and resource fetching jobs. This RfC summarises the
|
||||
general assumptions and requirements other parts of the application are relying
|
||||
on
|
||||
|
|
@ -77,13 +77,14 @@ knowledge of some specific job's termination. More precisely, we need to find ou
|
|||
when a ``stream of calculations'' has left a well defined domain -- and this can
|
||||
be modelled by the activation of specific marker jobs. It is possible to obtain
|
||||
that knowledge with some timing leeway, but in the end, this information needs
|
||||
to arrive with absolutely reliability (violations leading to segfault).
|
||||
to arrive with absolutely reliability (memory management will rely on them,
|
||||
and thus failure might lead to out-of.memory or segfault).
|
||||
|
||||
|
||||
job scheduling modes
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
The scheduler offers various modes of treatment on a per job base. The default is to
|
||||
handle jobs time based with a moderate latency. Alternatively jobs can be handled as
|
||||
handle jobs as _time-bound_, with a moderate latency. Alternatively jobs can be handled as
|
||||
_background_ jobs, as _freewheeling_ jobs (maximum usage of performance and bandwidth),
|
||||
or as _low-latency timed_ jobs.
|
||||
|
||||
|
|
@ -115,9 +116,9 @@ Tasks
|
|||
// List what needs to be done to implement this Proposal:
|
||||
// * first step [yellow-background]#WIP#
|
||||
* define the job interface ([green]#✔ done#)
|
||||
* define a protocol for job state handling [red yellow-background]#TBD#
|
||||
* define the representation of dependencies and the notifications in practice [red yellow-background]#TBD#
|
||||
* verify the proposed requirements by an scheduler implementation sketch [red yellow-background]#TBD#
|
||||
* define a protocol for job state handling ([green]#✔ done#)
|
||||
* define the representation of dependencies and the notifications in practice ([green]#✔#)
|
||||
* verify the proposed requirements by an scheduler implementation sketch ([green]#✔#)
|
||||
|
||||
|
||||
|
||||
|
|
@ -138,7 +139,7 @@ Pros
|
|||
Cons
|
||||
^^^^
|
||||
// fact list of the known/considered bad implications:
|
||||
* there is no ``for-loop'' to base any playback control structures on
|
||||
* there is no ``while-loop'' to base any playback control structures on
|
||||
* compliance to externally imposed deadlines and memory management are challenging.
|
||||
|
||||
Alternatives
|
||||
|
|
@ -150,8 +151,8 @@ Alternatives
|
|||
|
||||
We do not want (1), since it is tied to an obsolete hardware model and lacks the ability to be
|
||||
adapted to the new kinds of hardware available today or to be expected in near future. We do not
|
||||
want (2) since it essentially doesn't solve any problem, but rather pushes complexity into the
|
||||
higher layers (Session, Stage), which are lacking the information about individual jobs and timing.
|
||||
want (2) since it essentially doesn't solve any problem, but rather pushes complexity into the higher
|
||||
layers (Session, Stage), which however are lacking information about individual jobs and timing.
|
||||
|
||||
|
||||
|
||||
|
|
@ -161,17 +162,20 @@ Rationale
|
|||
We use a scheduler to gain flexibility in controlling various aspects of computation and I/O usage.
|
||||
Moreover, we turn the scheduler into an interface between the Vault and Steam-Layer; while the exact
|
||||
outfitting of the individual jobs highly depends on internals of the Session and Engine models,
|
||||
the properties of _actual job execution_, closely related to system programming are akin
|
||||
the properties of _actual job execution_, closely related to system programming, are linked
|
||||
to the Vault. The actual requirements outlined in this RfC are derived from the internals
|
||||
of the player implementation, while _the way_ these requirements are defined, and especially
|
||||
the aspects _omitted_ from specification are derived from knowledge regarding the possible
|
||||
scheduler and vault layer implementation.
|
||||
|
||||
|
||||
//Conclusion
|
||||
//----------
|
||||
Conclusion
|
||||
----------
|
||||
//conclusion: When approved (this proposal becomes a Final)
|
||||
// write some conclusions about its process:
|
||||
A solution closely following this proposal was implemented and became a
|
||||
centrepiece of the Lumiera Render Engine. There was no further formal
|
||||
decision, since _Ichthyo_ worked alone during the past years.
|
||||
|
||||
|
||||
|
||||
|
|
@ -189,6 +193,27 @@ can be developed independently of the scheduler implementation
|
|||
|
||||
Ichthyostega:: 'Do 19 Sep 2013 21:31:07 CEST' ~<prg@ichthyostega.de>~
|
||||
|
||||
.State -> Final
|
||||
After an extended intermission, where development was focused at GUI topics, the integration
|
||||
work since 2023 strives at achieving an initial integration of the Render Engine. As part of the
|
||||
https://issues.lumiera.org/ticket/1221[#1221 »PlaybackVerticalSlice«], an implementation of the
|
||||
Scheduler, as proposed by this RfC, was
|
||||
link:/x/imp/Scheduler.html[built from scratch and tested extensively]. Practical experience
|
||||
during this effort leads to the following refinements:
|
||||
|
||||
- the Scheduler does not invoke Jobs, +
|
||||
but processes _verbs_ from an _Activity Language_ rather.
|
||||
- this language in turn embodies a primitive to check a dependency count
|
||||
- an external event may re-trigger such a chain of Activities within the Scheduler
|
||||
- an extent-based memory manager can be based on deadlines of these Activities.
|
||||
- the Scheduler becomes a passive management task, pulled by a active Workers.
|
||||
- Workers manage their capacity through a statistical distribution of _sleep times._
|
||||
|
||||
In this form, the proposal can be considered as implemented. +
|
||||
^(see -> https://git.lumiera.org/?p=LUMIERA;a=commit;h=d71eb37[d71eb37]
|
||||
and https://issues.lumiera.org/ticket/1344[#1344])^
|
||||
|
||||
Ichthyostega:: '2024-04-20 01:55 CEST'
|
||||
|
||||
//endof_comments:
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue