Many years ago, I integrated the IconSvgRenderer (written by Joel Holdsworth)
directly into the SCons build by means of a custom builder with an ''emitter function''.
The build as such works fine and automatically determines which icons can be
generated from a given SVG source. However, those SVG icons tend to be rebuilt
very frequently, even while none of the SVG sources has changed.
Basically this was more of an annoyance, since we have now about 15 icons
and the rendering is really fast ... it is just ugly, especially on
incremental builds (and it could become a problem once we have a
massive amount of graphics to process.
So I took the opportunity to take close look now, since I am doing
uninspiring clean-up work since several weeks now. But that problem
turned out to be quite insidious and hard to spot.
First, AI set me off into a completely wrong angle, since it is ''not''
caused by a missing string representation of the custom Action class.
However, from there I went to watching the target checks with the debugger,
and this eventually got me to realise, that SCons mistakenly detects
a change in the Executor / the Action class.
The root cause is, that we invoke Python code from an external Module,
IconSvgRenderer.py, and this is »materialised« by SCons automatically
into a string representation, which includes memory addresses of functions
in that module. And that yields a signature, that is, quite obviously,
not stable, even while you end up sometimes with loading the code to
the same memory location.
As a solution / workaround, we now subclass the standard implementation
from SCons and override the signature function; instead of fingerprinting
the binary code, we just compute a MD5 over the python source file,
which can be easily achieved with the help of the File-Node from SCons.
Essential resources:
https://scons.org/doc/4.8.0/HTML/scons-user.html#chap-builders-writing
...and the Reference / Manpage of SCons
https://scons.org/doc/4.8.0/HTML/scons-man.html#action_objects
SCons/Action.py
SCons/Node/__init__.py
* Simplify some constructs from the Python-3 migration
* update copyright, since I did maintain the SCons build continuosly
* remove unused method from Buildhelper
Furthermore, the documentation page for our build system
https://lumiera.org/documentation/technical/build/SCons.html
has been reworked, to add a synopsis, some further background
information about the internal structure of SCons and about
the specific conventions and definitions used for Lumiera
Lumiera is built such that it can be invoked from a ''bundle directory''
without the need to install it into the system; this requires to perform
a relative lookup of shared modules loaded as dependencies.
Since Lumiera has a layered architecture, and each layer is built into a shared object,
we get transitive dependencies and thus transitive relative lookup.
There was a bug in binutils 2.23, which caused that transitive relative lookup to fail.
* as of 2025, Debian/Trixie is defined as ''Reference Platform''
* in Trixie the binutils version is v2.44-3
* after removing the workaround, Lumiera and all tests can be launched
* thus ''considering this workaround as obsolete now''
This is a Debian policy, which collides with the default behaviour
of GCC rsp. the plattform linker. The latter creates all ELF files as
executable, possibly because some platforms require that, and also
because some libraries provide a main() function for diagnostics.
The argument by the Debian people is that most libraries don't provide
such a main() and that this is an esoteric feature which should not be
supported by default.
We can fix that in our SCons build, since we use a specialised Builder
to also define the install targets systematically; it suffices to add
a post-action to these install targets for shared objects.
Note: currently Debian/Trixie provides SCons 4.8, but v4.10 will provide
a shorthand notation with the env.Chmod Action factory.
Some recherche regarding the requirements and conditions of the GPL
made it clear that we are obliged to have at least an summary statement
of all authors and the years of their contributions embedded somewhere
in the source itself. The Git history alone is not sufficient to fulfil
this requirement, since a code base could be packaged as Tarball
and passed-on outside of any Git repository.
Thus I propose the following policy
- add yourself to the copyright header whenever you made
some ''significant'' contribution to a code file
- maintain a list of relevant authors at top-level,
indicating a time range of their contribution
- use the same information also in the debian/copyright (DEB package)
- clarify in the LICENSE that authorship can be extracted from Git
__Remark__: Anton Yakovlev made a single code contribution in 2009 (87e528bd),
which was never used anywhere and removed as part of a general clean-up in 2023.
Thus I do not list him as a code author, but he is mentioned in credits.txt
- use a current copy of the GPL2 text, downloaded from fsf.org
- reformat the AUTHORS and get rid of the table formatting
- Stackoverflow has upgraded its Subscriber Content license to CC-By-SA 4.0
A handwritten changelog is considered important and
helps to single out the overall relevant achievements.
Following a proposal from the Debian policy, this summary
of development steps will now be maintained in the NEWS file.
It will be marked with the version number of Lumiera.
...and split-out the description of preview-releases,
since that rather belongs into some kind of changelog (NEWS).
Furthermore, check the links to the build dependencies
See corresponding commit in the website repository.
- Verified all authors via Git
- simplify the list and group by core/contributors
- list only significant contributors here
__Note__: the page "project/contributors" on the website
contains a much more extensive list with all helpers,
casual contributors and concept discussion participants
The XDG Desktop spec is accepted universally, and thus
the old Debian-specific 'menu' system is deprecated and
no longer engaged automatically.
See: https://lists.debian.org/debian-devel-announce/2015/09/msg00000.html
Thus we'll now provide a Desktop file and install that already from out
SCons build system, together with a suitable variation of the Lumiera icon.
TODO: not sure if everything was done the correct way
* do we need to ''register'' the new file in some way (preinst script?)
* the menu entry shows up, but not the icon
* but if we put an absolute path for the lumiera.svg into the desktop file, it shows up
Remark: in later experiments with package building,
the menu entry and the icon showed up in the menu.
Not sure if this requires a reboot or some similar
trigger (like restart of the destkop)
...funny enough I never noticed this obvious mistake,
since I never install software directly into my system,
while the DEB-build does not use absolute paths...
Solution:
* SCons has this speical convention that a path prefixed with '#'
is resolved relative to the root of the build (where the SConstruct resides)
* now we apply this automatically to the two relevant settings
** INSTALLDIR
** TARGDIR
* but only (conditionally) if the configured path is relative, not absolute
As a consequence, most other hard-coded usages of the '#'-prefix can then be dropped
Debian-Docbase allows to register some HTML documentation;
My old package definition added placeholder config, which renders
the documentation configuration invalid (as pointed out by Lintian).
However, I still think it is a good idea to have the anchor point
already defined, and thus I came up with the idea of in fact
providing some usable placeholder content...
As it turns out, we also have a placeholder page at the Lumiera website,
where the User Manual is assumed to be located later — so why not extend
this one and then provide the HTML-rendering for the DEB package?
To allow for this setup
* I have now extended the placeholder page for the Website
to include some generic description about Lumiera (from the 'about' page)
* Furthermore, I added the screenshot (from the »Outer Space« page)
* and I use this a an opportunity to document the various test / demo
facilities currently available in the GUI, since these are rather obscure.
While only intended for the developer, it seems still worthwhile
to describe the possible effects — it may well be that we retain
some of that test/demo functionality and in that case, we have
now already some starting point for a documentation
* Then, to include that page as stand-alone HTML, I used the 'Print Edit WE'-plugin
from Firefox, to encode the images as inline-base64 URLs (which are restored
by a tiny JavaScript embedded into that page)
* and last but not least, our SCons buildsystem needs the ability
to install such a documentation file, since it seems most adequate
to handle this requirement as part of the generic installation (and
not hidden in some Debian scripting)
Yes, I know...
Programmers absolutely LOVE to sneak-in various nifty toggles via environment.
Yet this is an anti-pattern!
And, by extension of that verdict, a "build interface" which relies
on the implicit convention that some magical variables are set,
is **not a proper interface** but a hack.
Thus we abandon that bad practice and handle the build in a clean and explicit way.
* the DEB-Build in `debian/rules` now invokes SCons explicitly, passing arguments
* the Lumiera Build-System is FSH aware and knows the proper installation locations
* the setup of the application uses a setup configuration, shipped with the package
* there is no need to ''compile-in any configuration''
** `LUMIERA_PLUGIN_PATH` is obsolete and unused since several years now
** `LUMIERA_CONFIG_PATH` was never used at all
** consequently, we also do not need `PKGLIBDIR` and `PKGDATADIR`
A long-standing unnecessary mutual dependency caused rebuild
of most tests in the standard target, because the ''valgrind suppression''
executable was classified as »tool«, while also depending on the libraies
with the test code.
...this bug was spotted as the test failed reproducibly
when built with `-fstack-protector-strong` — which adds
additional "canary" markers to some kinds of problematic
stack storage; this seemed to have the effect that now
the second test uses exactly the same location as the
preceding test, and thus finds valid data in the newly
created work buffers.
There is no reason for using uninitialised storage in this
test (I can recall that I wanted to build a generic helper
and intended to use that as ''virtual overlay'' over existing
memory — but that usage never took place, and the `struct Buffer`
is not a general-purpose tool now, but only made for this
specific test. Thus memory can be easily zero-initialised.
Turns out that in practice there will be (at least temporarily)
some version-tags including a suffix: the RC-versions!
Now there is the special twist, that Git does not allow '~' in Tag names,
and thus `git-buildpackage` introduced an additional layer of translation.
So we are forced to revert that translation, which is possible,
since the basic Debian version syntax disallows '_' in version numbers
(because '_' is used to separate the package name from the version number).
It seems prudent to implement that as an preprocessing step
and thus keep it out of the regular version number syntax.
Furthermore we need the ability to handle existing suffixes,
which (as we know now) can be picked up from the Git history.
* my decision is to allow both pass-through and suppressing them
* use `--suffix=False` to suppress / remove any existing suffix
* the latter now allows us also to automate the setting of
the final Release version
Based on the additional insight gained through the FrOSCon talk,
an attempt was made to investigate and fix the garbled display of the
docking panel header directly above our VideoDisplayWidget.
However, that turned out to be a problem with insufficeint support
of the XVideo standard with my (very old, 2011) NVidia graphic card.
Indeed, the hardware+software stack needs to support this Auto-Keying
extension of XVideo, so that the video display can be integrated
seamlessly into the rest of the UI. In my case, the driver seemingly
just fills the complete XWindow with the key-marker and thus the
decorations of the docking panel, which happens to use the same XWindow,
will just not be flushed to the display.
This unsuccessful research indicates that upholding support for XVideo
could become problematic...
- reorganise the navigation tab structure
- place all further index lists into a common "Index" tab
- include a list of concepts
- remove the various "member" sub-tabs, these are not helpful
- the "modules" tab is now called "topics" (since C++ has now Modules!)
- generally re-sync DoxygenLayout.xml with a current pristine template
- comb though the Doxygen Warnings and fix a lot of small problems
The »Outer Space« and »Inner Core« documents present a comprehensive
overview of the vision, architecture and essential parts of the implementation.
In the light of changes to policy, design and implementation approaches,
some updates were necessary to align these crucial texts with the current
state of planning and implementation. Notably I have added a recent Screenshot
of the UI, showing a nested track structure pushed up by Diff from the core.
Over the last weeks, I conducted an extended analysis of the inception time
of the Lumiera project, which gave me a sharpened understanding of the goals,
going forward. I have also written an in-depth essay "Complexity and Flexibility".
See the corresponding commits in the Website repository.
The RfC documents were written to complement discussions of the Lumiera developers;
yet since the time where ''Ichthyo'' is working basically alone on the project,
this kind of discussions have ceased. During the following years, some ideas
promoted by the existing RfC documents became rather detached from the
actual state of development in the code base.
Many of the existing RfC documents require some commentary to place them
into context, and some of the decisions taken in the early stage of the
project should be **re-assessed**. This includes the decision to reject
some proposals, which initially might have seemed desirable, yet could not
be reconciled with the understanding of the matter and topic in question,
as was gained through the ongoing analysis and development.
The purpose of RfCs is to channel and document discussions
among a group of developers regarding questions of design.
While there is unfortunately no longer a group of developers
discussing matters and working together on the code base,
some questions can be considered more or less settled,
by being implemented and validated.
- the Scheduler
- structure of the Developer Documentation
- Application installation as relocatable bundle
Some sections of the Lumiera website document meeting minutes,
discussion protocols and design proposals from the early days
of the project; these pages were initially authored in the
»Moin Moin Wiki« operated by Cehteh on pipapo.org at that time;
this wiki backed the first publications of the »Cinelerra-3«
initiative, which turned into the Lumiera project eventually.
Some years later, those pages were transliterated into Asciidoc
semi-automatically, resulting in a lot of broken markup and links.
This is a long standing maintenance problem problem plaguing the
Lumiera website, since those breakages cause a lot of warnings
and flood the logs of any linkchecker run.
`BuilderGCH` was added as an experimental feature many years ago;
this is code found ''somewhere'' on the net and was use ''as-is'' —
however, precompiled headers turned out as a feature of GCC with
a lot of quality problems and, furthermore seems of questionable
usability — we thus decided to adhere strictly to the »Borland model«
of template instantiation and recommend against using precompiled headers.
Rather, in the Lumiera code base, much care is taken to avoid unnecessary
header includes — this, together with the incremental build feature of SCons
largely obsoletes the necessity to resort to precompiled headers and
similar facilities (as CCache).
Notably some links on the **Gnome documentation** pages are problematic,
since there was seemingly an infrastructure change -- which unfortunately
leads to loosing several deep-links into the GTK-3 related documentation
and tutorials.
While the differences to GTK-4 are often rather minimal, I would still prefer
to link to those documentation pages used as reference at implementation time
of our related library and GUI functionality.
In several cases I have thus looked up the old URLs at Archive.org
- indicate clearly that a ''past discussion'' is documented here
- point out what is the primary focus for UI design ''currently''
- arrange some links better
- use cross-links / Linkfarm; especially cross-link to
the discussion regarding Wouter's Workflow proposals (2025)
Create a new subcategory "design/architecture/time"
and rearrange several pages related to time handling and time codes.
NOTE: starting with this changeset, a ''Link-Farm'' is required for cross-links;
since we don't have an automatic solution for this task yet, I have created
the necessary forwarding pages manually in the website repository.
Christian started a series of reworks in 2018;
the result was considered experimental and was parked on website-staging ever since.
As part of the current release- and clean-up activities I now reviewed
and integrated those changes into the current website as far as applicable.
...fine-tuned a few small bits here and there in the
current chapter of the workflow document
===========================================================
Remark from Ichthyo(committer):
Converted from PDF to raw text with
pdftotext wouter250819v3.pdf WorkflowProposals.txt
Images extracted with:
pdfimages -png -j wouter250819v3.pdf wouter/wouter250819v3
Again adding the images separately to the Website-repository,
in order to save storage space in the main repository....
- add markup to match formatting from PDF
- link to the images extracted into the website Git-repository
- adjust image sizes to fit into the text
- add some cross references
(incidentally: TimelineDiscussion.txt -- store image locally)
Chapter 2 (about the timeline) is now complete.
- extend discussion regarding trackless
- add section about Tools / Modes / Views
- section: Adding clips to the timeline
- section: Selecting clips
- section: Arranging clips
- section: Trimming clips
- section: Splitting and merging clips
- section: Removing clips
- add subchapter regarding sections of the timeline
- Adding and editing transitions
- Changing timeline clip properties
===========================================================
Remark from Ichthyo(committer):
Converted from PDF to raw text with
pdftotext wouter250629v2.pdf WorkflowProposals.txt
Images extracted with
pdfimages -png -j wouter250629v2.pdf wouter/wouter250629v2
Images required no cropping;
the white background images were omitted.
Again adding the images separately to the Website-repository,
in order to save storage space in the main repository....
This is the version from 4.Apr
- add a subchapter about the current NLE landscape
- add a subchapter on tracks vs. trackless
- add a bit more information in the navigation subchapter.
===========================================================
Remark from Ichthyo(committer):
Converted from PDF to raw text with
pdftotext wouter250404v1.pdf WorkflowProposals.txt
Images extracted with
pdfimages -png wouter250404v1.pdf
...and then cropped in Gimp ("crop to content");
images 06 and 11 needed to be split in two parts
cropped in Gimp ("crop to content")
NOTE: Adding the images separately to the Website-repository,
in order to save storage space in the main repository,
because content, once added, can not be removed from Git....
- included the list of "Personas", quoting from Wouter's document
- add some details from the ensuing discussion
- close the document with a section "Conclusions" to list the
most relevant open points
Topic: proposals for Lumiera Workflow
Present:
- Wouter Verweijlen
- Benny Lyons
- Hermann Voßeler
Note: This commit creates a new subsection
for the discussion related to Wouter's »Lumiera Workflow Proposals«....
The topic of this talk was ''Video output from a Linux desktop application''
after extended research we built a demo application in four flavours
(XVideo, SDL, OpenGL legacy / modern).
This research project was set off by our immediate needs for a setup
to show computed video pixel data in a viewer window in the GUI; the
investigated technologies should work at present and in the near term
future (yet leaving out Vulkan and Wayland)
Starting with ''preview release'' `v0.pre.04`, branch and version tags
will be handled in accordance to the **Git-flow** naming scheme.
Notably this implies that from now on the version in-tree will indicate
the ''next expected release,'' adorned by a suffix to mark the preview.
To accommodate this transition to Git-flow
- the new branch `integration` will be introduced
- the version number will once (and the last time for this release)
be adjusted ''before'' forking the release branch
- branch `master` will transition to reflect the latest released state
- several existing branches will be discontinued, notably
`gui`, `steam`, `vault`, `release`, `play`
Starting with the upcoming ''preview release'', branches, branch names and tags
will be rearranged to follow the Git-flow pattern instead of the existing
ad-hoc organisation with a release branch.
The documentation provided here defines the actual naming conventions
and some fine points regarding the version number upgrades
and placement of release tags.
Furthermore, two helper-scripts are provided to automate version number updates
- `buildVersion.py` : extract current version from git tag and allow to bump version
- `setVersion` : manipulate all relevant files with `sed` to update the version info
