clean-up: some RfCs became effectively final
The purpose of RfCs is to channel and document discussions among a group of developers regarding questions of design. While there is unfortunately no longer a group of developers discussing matters and working together on the code base, some questions can be considered more or less settled, by being implemented and validated. - the Scheduler - structure of the Developer Documentation - Application installation as relocatable bundle
This commit is contained in:
parent
67695c97b8
commit
c7528bbe8a
7 changed files with 91 additions and 27 deletions
|
|
@ -355,6 +355,7 @@ revisit the relevant design process entries in each developer meeting.
|
||||||
|
|
||||||
-> link:{rfc}/ApplicationInstall.html[RfC: Application Install]
|
-> link:{rfc}/ApplicationInstall.html[RfC: Application Install]
|
||||||
|
|
||||||
|
[[application_install_proposal]]
|
||||||
.-- the Application Install proposal --
|
.-- the Application Install proposal --
|
||||||
[caption="☉Transcript☉ "]
|
[caption="☉Transcript☉ "]
|
||||||
----------------------------
|
----------------------------
|
||||||
|
|
|
||||||
|
|
@ -5,20 +5,20 @@ ApplicationInstall
|
||||||
|
|
||||||
[options="autowidth"]
|
[options="autowidth"]
|
||||||
|====================================
|
|====================================
|
||||||
|*State* | _Draft_
|
|*State* | _Final_
|
||||||
|*Date* | _Di 11 Jan 2011 17:00:55 CET_
|
|*Date* | _Di 11 Jan 2011 17:00:55 CET_
|
||||||
|*Proposed by* | Ichthyostega <prg@ichthyostega.de>
|
|*Proposed by* | Ichthyostega <prg@ichthyostega.de>
|
||||||
|====================================
|
|====================================
|
||||||
|
|
||||||
[abstract]
|
[abstract]
|
||||||
*********************************************************************************
|
*********************************************************************************
|
||||||
Lumiera should be a _freely relocatable_ application bundle.
|
Lumiera should be a _freely relocatable_ application bundle. +
|
||||||
Relying only on the relative folder structure within this bundle, the application
|
Relying only on the relative folder structure within this bundle, the application
|
||||||
will be fully functional at any location, provided that the external library
|
will be fully functional at any location, provided that the external library
|
||||||
dependencies are resolvable using the standard mechanisms of the platform.
|
dependencies are resolvable using the standard mechanisms of the platform.
|
||||||
The setup mechanism must be obvious, self-explanatory and must not rely
|
This setup mechanism should be obvious and self-explanatory and must not rely
|
||||||
on compiled in magic or buildsystem trickery. Yet packaging into a FSH conforming
|
on compiled-in hidden magic or buildsystem trickery. Yet packaging into a fully
|
||||||
installation location should be supported by the same mechanisms.
|
FSH conforming installation location should be supported by the same mechanisms.
|
||||||
*********************************************************************************
|
*********************************************************************************
|
||||||
|
|
||||||
Description
|
Description
|
||||||
|
|
@ -66,7 +66,7 @@ In former days, it was common habit to compile-in a hard wired absolute
|
||||||
policy forbids doing so. This is the result from numerous maintainability
|
policy forbids doing so. This is the result from numerous maintainability
|
||||||
problems in the past. On the other hand, the GNU linker and other modern
|
problems in the past. On the other hand, the GNU linker and other modern
|
||||||
linkers support a relative resolution of shared modules directly accompanying
|
linkers support a relative resolution of shared modules directly accompanying
|
||||||
an specific executable. The Debian policy allows this, if and only if these
|
a specific executable. The Debian policy allows this, if and only if these
|
||||||
shared modules are installed with the same binary package and only used by
|
shared modules are installed with the same binary package and only used by
|
||||||
this specific executable(s). Together, this is exactly what we need to
|
this specific executable(s). Together, this is exactly what we need to
|
||||||
solve our requirement.
|
solve our requirement.
|
||||||
|
|
@ -76,7 +76,7 @@ and sets the +DT_RUNPATH+ with an value relative to +$ORIGIN+, which resolves
|
||||||
to the path of the currently executing binary. Moreover, it is _sufficient_
|
to the path of the currently executing binary. Moreover, it is _sufficient_
|
||||||
to set this on the initial executable _only,_ because this creates a common
|
to set this on the initial executable _only,_ because this creates a common
|
||||||
searchpath for all lib resolution events in the scope of that loaded executable.
|
searchpath for all lib resolution events in the scope of that loaded executable.
|
||||||
Besides that, we need to care that our private libraries have a unique +SONAME+,
|
Besides that, we need to care that our private libraries have an unique +SONAME+,
|
||||||
in this case all starting with the prefix +liblumiera*+. Note moreover that this
|
in this case all starting with the prefix +liblumiera*+. Note moreover that this
|
||||||
new-style +DT_RUNPATH+ indeed _can_ be overridden by an +LD_LIBRARY_PATH+ in the
|
new-style +DT_RUNPATH+ indeed _can_ be overridden by an +LD_LIBRARY_PATH+ in the
|
||||||
environment, should there be the need for very special experiments.
|
environment, should there be the need for very special experiments.
|
||||||
|
|
@ -172,10 +172,10 @@ up some very specific problems and constraints, which can be a serious
|
||||||
threat when discovered late in the development process.
|
threat when discovered late in the development process.
|
||||||
|
|
||||||
The second alternative isn't really applicable IMHO. The original UNIX philosophy
|
The second alternative isn't really applicable IMHO. The original UNIX philosophy
|
||||||
breeds on an academic setup and really excels with small nifty commandline utils
|
draws from an academic setup and really excels with small nifty commandline utils
|
||||||
combined by pipes, each specialised to do a single thing very well. These utils
|
combined by pipes, each specialised to do a single thing very well. These utils
|
||||||
are more like the objects within our implementation. The concept of large
|
are more like the objects within our implementation. The concept of large
|
||||||
application software bundles and desktop software was always a bit alien
|
application software bundles and desktop software was always slightly alien
|
||||||
within the classic UNIX environment.
|
within the classic UNIX environment.
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -191,17 +191,20 @@ trickery, inline code compiled on-the-fly, relying on very specific and
|
||||||
un-obvious behaviour of some build script, configuration via environment
|
un-obvious behaviour of some build script, configuration via environment
|
||||||
variables and a lot of similar idioms. These practices might be adequate
|
variables and a lot of similar idioms. These practices might be adequate
|
||||||
in a quickly moving Research & Development setup, but turned out to be
|
in a quickly moving Research & Development setup, but turned out to be
|
||||||
not so helpful when it comes to industrial strength development,
|
not so helpful when it used within large-scale development efforts,
|
||||||
as they are known to lead to maintenance problems.
|
as they are known to lead to maintenance problems.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
//Conclusion
|
Conclusion
|
||||||
//----------
|
----------
|
||||||
//conclusion: When approved (this proposal becomes a Final)
|
//conclusion: When approved (this proposal becomes a Final)
|
||||||
// write some conclusions about its process:
|
// write some conclusions about its process:
|
||||||
|
All the indicated clean-up was done and the bootstrap sequence could be implemented. +
|
||||||
|
The link:{ldoc}/devel/meeting_summary/2011-04-13.html#application_install_proposal[Apr.2011 developer meeting]
|
||||||
|
shortly touched on this proposal without raising any objections.
|
||||||
|
``... for example this application install .. is boring .. we need it, you did it well'' (_Cehteh_)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -221,5 +224,12 @@ try to bring these points to discussion separately.
|
||||||
|
|
||||||
Ichthyostega:: 'So 13 Feb 2011 20:04:00 CET'
|
Ichthyostega:: 'So 13 Feb 2011 20:04:00 CET'
|
||||||
|
|
||||||
|
.State -> Final
|
||||||
|
After being established in 2011, this structure was used ever since. There was once
|
||||||
|
a problem with a bug in `ld.so` related to the transitive use of the `$origin` token;
|
||||||
|
this bug had been resolved quickly. Other than that, packaging and application start
|
||||||
|
always worked as intended -- so I consider this topic as settled.
|
||||||
|
|
||||||
|
Ichthyostega:: '2025-09-18'
|
||||||
|
|
||||||
//endof_comments:
|
//endof_comments:
|
||||||
|
|
|
||||||
|
|
@ -5,7 +5,7 @@ Developer Documentation Structure
|
||||||
|
|
||||||
[options="autowidth"]
|
[options="autowidth"]
|
||||||
|====================================
|
|====================================
|
||||||
|*State* | _Draft_
|
|*State* | _Final_
|
||||||
|*Date* | _Mon Aug 2 18:03:25 2010_
|
|*Date* | _Mon Aug 2 18:03:25 2010_
|
||||||
|*Proposed by* | Cehteh + Ichthyo
|
|*Proposed by* | Cehteh + Ichthyo
|
||||||
|====================================
|
|====================================
|
||||||
|
|
@ -125,7 +125,21 @@ developers.
|
||||||
//----------
|
//----------
|
||||||
//conclusion: When approvedd (this proposal becomes a Final)
|
//conclusion: When approvedd (this proposal becomes a Final)
|
||||||
// write some conclusions about its process:
|
// write some conclusions about its process:
|
||||||
|
This RfC had been started initially by _Cehteh_ to propose a _simple structure_ --
|
||||||
|
the ensuing discussion however showed that our documentation is already structured
|
||||||
|
in a mostly consistent way, which just happens to be not that simple. Some confusing
|
||||||
|
redundancies could be removed, as result from that discussion. _Ichthyo_ then edited
|
||||||
|
and extended the original RfC to reflect the existing structure.
|
||||||
|
|
||||||
|
A serious problem is posed by the rather tedious procedure to define cross-links in the
|
||||||
|
Asciidoc based web content. link:WebsiteSupportMarkup.html[Another RfC] was created to
|
||||||
|
discuss these problems in detail, leading to a solution proposal, which unfortunately
|
||||||
|
was never implemented (due to lack of developer capacity). This _somehow transitory_
|
||||||
|
situation had the (adverse) effect that most detailed technical content was authored
|
||||||
|
within the Development TiddlyWiki.
|
||||||
|
|
||||||
|
These remaining problems do not seem to be related to the _structure_ of the documentation
|
||||||
|
however, which seems to fulfil its purpose. So this RfC now counts as de-facto standard.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -175,4 +189,18 @@ documentation in textual form, which is inherentely structured.
|
||||||
Ichthyostega:: 'Mo 09 Sep 2013 01:01:25 CEST' ~<prg@ichthyostega.de>~
|
Ichthyostega:: 'Mo 09 Sep 2013 01:01:25 CEST' ~<prg@ichthyostega.de>~
|
||||||
|
|
||||||
|
|
||||||
|
.State -> Final
|
||||||
|
I am using this structure now since several years, without encountering any
|
||||||
|
serious mismatch or incompatibility. The situation with the cross-linking
|
||||||
|
generator, which still remains to be implemented, is obviously some kind
|
||||||
|
of impediment, since it prevents me from migrating more mature content
|
||||||
|
from the TiddlyWiki into the documentation section of the website.
|
||||||
|
|
||||||
|
Furthermore, I should notice, as a limitation, that I am the _only one_ to
|
||||||
|
use this structure actively -- so the fact that I feel comfortable with
|
||||||
|
the arrangement should be taken with a grain of salt. Yet, on the other
|
||||||
|
hand, we have meanwhile a massive body of information integrated
|
||||||
|
into this structure, so that it can be considered more than
|
||||||
|
just a ``proposal for discussion''
|
||||||
|
|
||||||
//endof_comments:
|
//endof_comments:
|
||||||
|
|
|
||||||
|
|
@ -5,7 +5,7 @@ SchedulerRequirements
|
||||||
|
|
||||||
[options="autowidth"]
|
[options="autowidth"]
|
||||||
|====================================
|
|====================================
|
||||||
|*State* | _Draft_
|
|*State* | _Final_
|
||||||
|*Date* | _Mi 09 Jan 2013 12:04:03 CET_
|
|*Date* | _Mi 09 Jan 2013 12:04:03 CET_
|
||||||
|*Proposed by* | Ichthyostega <prg@ichthyostega.de>
|
|*Proposed by* | Ichthyostega <prg@ichthyostega.de>
|
||||||
|====================================
|
|====================================
|
||||||
|
|
@ -15,7 +15,7 @@ SchedulerRequirements
|
||||||
Define the expected core properties and requirements of the Scheduler service.
|
Define the expected core properties and requirements of the Scheduler service.
|
||||||
********************************************************************************
|
********************************************************************************
|
||||||
|
|
||||||
The rendering and playback subsystem relies on a Scheduler component to invoke
|
The rendering and playback subsystem relies on a *Scheduler* component to invoke
|
||||||
individual frame rendering and resource fetching jobs. This RfC summarises the
|
individual frame rendering and resource fetching jobs. This RfC summarises the
|
||||||
general assumptions and requirements other parts of the application are relying
|
general assumptions and requirements other parts of the application are relying
|
||||||
on
|
on
|
||||||
|
|
@ -77,13 +77,14 @@ knowledge of some specific job's termination. More precisely, we need to find ou
|
||||||
when a ``stream of calculations'' has left a well defined domain -- and this can
|
when a ``stream of calculations'' has left a well defined domain -- and this can
|
||||||
be modelled by the activation of specific marker jobs. It is possible to obtain
|
be modelled by the activation of specific marker jobs. It is possible to obtain
|
||||||
that knowledge with some timing leeway, but in the end, this information needs
|
that knowledge with some timing leeway, but in the end, this information needs
|
||||||
to arrive with absolutely reliability (violations leading to segfault).
|
to arrive with absolutely reliability (memory management will rely on them,
|
||||||
|
and thus failure might lead to out-of.memory or segfault).
|
||||||
|
|
||||||
|
|
||||||
job scheduling modes
|
job scheduling modes
|
||||||
~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
The scheduler offers various modes of treatment on a per job base. The default is to
|
The scheduler offers various modes of treatment on a per job base. The default is to
|
||||||
handle jobs time based with a moderate latency. Alternatively jobs can be handled as
|
handle jobs as _time-bound_, with a moderate latency. Alternatively jobs can be handled as
|
||||||
_background_ jobs, as _freewheeling_ jobs (maximum usage of performance and bandwidth),
|
_background_ jobs, as _freewheeling_ jobs (maximum usage of performance and bandwidth),
|
||||||
or as _low-latency timed_ jobs.
|
or as _low-latency timed_ jobs.
|
||||||
|
|
||||||
|
|
@ -115,9 +116,9 @@ Tasks
|
||||||
// List what needs to be done to implement this Proposal:
|
// List what needs to be done to implement this Proposal:
|
||||||
// * first step [yellow-background]#WIP#
|
// * first step [yellow-background]#WIP#
|
||||||
* define the job interface ([green]#✔ done#)
|
* define the job interface ([green]#✔ done#)
|
||||||
* define a protocol for job state handling [red yellow-background]#TBD#
|
* define a protocol for job state handling ([green]#✔ done#)
|
||||||
* define the representation of dependencies and the notifications in practice [red yellow-background]#TBD#
|
* define the representation of dependencies and the notifications in practice ([green]#✔#)
|
||||||
* verify the proposed requirements by an scheduler implementation sketch [red yellow-background]#TBD#
|
* verify the proposed requirements by an scheduler implementation sketch ([green]#✔#)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -138,7 +139,7 @@ Pros
|
||||||
Cons
|
Cons
|
||||||
^^^^
|
^^^^
|
||||||
// fact list of the known/considered bad implications:
|
// fact list of the known/considered bad implications:
|
||||||
* there is no ``for-loop'' to base any playback control structures on
|
* there is no ``while-loop'' to base any playback control structures on
|
||||||
* compliance to externally imposed deadlines and memory management are challenging.
|
* compliance to externally imposed deadlines and memory management are challenging.
|
||||||
|
|
||||||
Alternatives
|
Alternatives
|
||||||
|
|
@ -150,8 +151,8 @@ Alternatives
|
||||||
|
|
||||||
We do not want (1), since it is tied to an obsolete hardware model and lacks the ability to be
|
We do not want (1), since it is tied to an obsolete hardware model and lacks the ability to be
|
||||||
adapted to the new kinds of hardware available today or to be expected in near future. We do not
|
adapted to the new kinds of hardware available today or to be expected in near future. We do not
|
||||||
want (2) since it essentially doesn't solve any problem, but rather pushes complexity into the
|
want (2) since it essentially doesn't solve any problem, but rather pushes complexity into the higher
|
||||||
higher layers (Session, Stage), which are lacking the information about individual jobs and timing.
|
layers (Session, Stage), which however are lacking information about individual jobs and timing.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -161,17 +162,20 @@ Rationale
|
||||||
We use a scheduler to gain flexibility in controlling various aspects of computation and I/O usage.
|
We use a scheduler to gain flexibility in controlling various aspects of computation and I/O usage.
|
||||||
Moreover, we turn the scheduler into an interface between the Vault and Steam-Layer; while the exact
|
Moreover, we turn the scheduler into an interface between the Vault and Steam-Layer; while the exact
|
||||||
outfitting of the individual jobs highly depends on internals of the Session and Engine models,
|
outfitting of the individual jobs highly depends on internals of the Session and Engine models,
|
||||||
the properties of _actual job execution_, closely related to system programming are akin
|
the properties of _actual job execution_, closely related to system programming, are linked
|
||||||
to the Vault. The actual requirements outlined in this RfC are derived from the internals
|
to the Vault. The actual requirements outlined in this RfC are derived from the internals
|
||||||
of the player implementation, while _the way_ these requirements are defined, and especially
|
of the player implementation, while _the way_ these requirements are defined, and especially
|
||||||
the aspects _omitted_ from specification are derived from knowledge regarding the possible
|
the aspects _omitted_ from specification are derived from knowledge regarding the possible
|
||||||
scheduler and vault layer implementation.
|
scheduler and vault layer implementation.
|
||||||
|
|
||||||
|
|
||||||
//Conclusion
|
Conclusion
|
||||||
//----------
|
----------
|
||||||
//conclusion: When approved (this proposal becomes a Final)
|
//conclusion: When approved (this proposal becomes a Final)
|
||||||
// write some conclusions about its process:
|
// write some conclusions about its process:
|
||||||
|
A solution closely following this proposal was implemented and became a
|
||||||
|
centrepiece of the Lumiera Render Engine. There was no further formal
|
||||||
|
decision, since _Ichthyo_ worked alone during the past years.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -189,6 +193,27 @@ can be developed independently of the scheduler implementation
|
||||||
|
|
||||||
Ichthyostega:: 'Do 19 Sep 2013 21:31:07 CEST' ~<prg@ichthyostega.de>~
|
Ichthyostega:: 'Do 19 Sep 2013 21:31:07 CEST' ~<prg@ichthyostega.de>~
|
||||||
|
|
||||||
|
.State -> Final
|
||||||
|
After an extended intermission, where development was focused at GUI topics, the integration
|
||||||
|
work since 2023 strives at achieving an initial integration of the Render Engine. As part of the
|
||||||
|
https://issues.lumiera.org/ticket/1221[#1221 »PlaybackVerticalSlice«], an implementation of the
|
||||||
|
Scheduler, as proposed by this RfC, was
|
||||||
|
link:/x/imp/Scheduler.html[built from scratch and tested extensively]. Practical experience
|
||||||
|
during this effort leads to the following refinements:
|
||||||
|
|
||||||
|
- the Scheduler does not invoke Jobs, +
|
||||||
|
but processes _verbs_ from an _Activity Language_ rather.
|
||||||
|
- this language in turn embodies a primitive to check a dependency count
|
||||||
|
- an external event may re-trigger such a chain of Activities within the Scheduler
|
||||||
|
- an extent-based memory manager can be based on deadlines of these Activities.
|
||||||
|
- the Scheduler becomes a passive management task, pulled by a active Workers.
|
||||||
|
- Workers manage their capacity through a statistical distribution of _sleep times._
|
||||||
|
|
||||||
|
In this form, the proposal can be considered as implemented. +
|
||||||
|
^(see -> https://git.lumiera.org/?p=LUMIERA;a=commit;h=d71eb37[d71eb37]
|
||||||
|
and https://issues.lumiera.org/ticket/1344[#1344])^
|
||||||
|
|
||||||
|
Ichthyostega:: '2024-04-20 01:55 CEST'
|
||||||
|
|
||||||
//endof_comments:
|
//endof_comments:
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue