193 lines
11 KiB
Text
193 lines
11 KiB
Text
SCons Build-System
|
|
==================
|
|
:author: Ichthyo
|
|
:date: 2012
|
|
|
|
//MENU: label SCons Build
|
|
|
|
Lumiera uses a build system based on link:http://scons.org[SCons]
|
|
|
|
SCons is an open source software construction tool based on Python build definition scripts.
|
|
Within these build scripts, we define a data structure to describe the parts and dependencies
|
|
of our software. When executed, SCons evaluates those definitions and the actual files in
|
|
the source tree to derive a build strategy, which is then performed.
|
|
|
|
SCons core concepts
|
|
-------------------
|
|
^_this section is based on the introductory pages on the link:http://www.scons.org/wiki/BasicConcepts[SCons Wiki]_^
|
|
|
|
.SCons Environment
|
|
When SCons starts building the project, it creates its own environment with dependency trees,
|
|
helper functions, builders and other stuff. The SCons environment is created in memory and some parts of it
|
|
are saved to disk to speed up things on the next start. The definition of the build happens within this
|
|
artificial build environment. This often confuses people who used Makefiles, where environment is actually
|
|
the System Environment.
|
|
|
|
.System Environment
|
|
the familiar operating system container with environment variables such as PATH, HOME etc.
|
|
It is usually accessible via os.environ mapping in Python and therefore in SCons too.
|
|
SCons doesn't import any settings from System Environment automatically (like flags for compilers,
|
|
or paths for tools), because it's designed to be a cross platform tool with _predictable behaviour._
|
|
That's why, if you rely on any system PATHs or environment variables -- you need to extract
|
|
those settings explicitly in your build definition.
|
|
|
|
.SConstruct
|
|
when SCons executes, it performs a build definition python script written by the user.
|
|
By convention, this main script is called 'SConstruct' and is located in the root of the source tree.
|
|
It is a full featured Python module executed within a specifically prepared environment.
|
|
|
|
.SConscript
|
|
these files are also SCons scripts, but they are placed in subdirectories of the project.
|
|
Typically they are used to organize hierarchical builds and are included from the main SConstruct file
|
|
from the project root. Often, all of the actual build definitions reside in SConscript files in
|
|
the sub-trees of the project.
|
|
|
|
.Builder
|
|
The SCons buildsystem revolves around the metaphor of a _Builder_. This is a SCons object that you explicitly
|
|
invoke from the scripts to _define_ that there is something to build, transforming a _source_ into a _target_.
|
|
So the target depends on the sources, and typically those _source nodes_ were created by previous builder invocations.
|
|
The power of SCons is in the fact that dependencies are tracked automatically. When source files change, the system
|
|
is able to derive which targets to rebuild.
|
|
|
|
.Scanner
|
|
when defining a builder, SCons relies on modular scanner components to ``understand'' the source of the build step.
|
|
They may scan source files to discover additional dependencies referenced inside. Thus, SCons comes with built-in
|
|
knowledge about the source files and artefacts to be created by a typical build, and further types can be added
|
|
through plug-ins.
|
|
|
|
.Tool
|
|
any further, external component that adds Builders, Scanners and other helpers to SCons environments
|
|
for use within scripts. There are special tools for _configuring the platform_ to detect libraries and
|
|
further requirements. Relying on those tools. the build environment will be outfitted to reflect the
|
|
needs of the specific build. Sub-environments with special configuration may be created.
|
|
|
|
.Target
|
|
any _node_ or ``build step'' encountered through the definition of the build is a _target_. The actual build
|
|
will be triggered by requesting a target, which typically might be just an executable known to reside at some
|
|
location in the tree. Special _alias targets_ may be defined, based on other targets, to trigger off standard
|
|
build situations. Especially, a _default_ target may be defined.
|
|
|
|
''''
|
|
|
|
Organisation of the Lumiera SCons build
|
|
---------------------------------------
|
|
Within our build system, we build upon the power of the Python programming language to create abstractions
|
|
tailored to the needs of our project. Located in the `admin/scons` subdirectory, you'll find a collection
|
|
of Python modules to provide these building blocks.
|
|
|
|
- the *LumieraEnvironment* is created as a subclass of the standard SCons build environment; it is
|
|
outfitted with pre-configured custom builders for executables, libraries, extension module, Lumiera plug-in
|
|
and icon resources.
|
|
- all these *custom builders* implement a set of conventions and directory locations within the tree.
|
|
These are defined (and can be adjusted) in the *Setup.py* module. This way, each builder automatically
|
|
places the generated artefacts at standard build and installation locations.
|
|
- for defining individual targets and builder invocations, we rely on *build helpers* to process whole
|
|
*source sub trees* rather than individual files. Mostly, just placing a source file into the appropriate
|
|
sub tree is sufficient to get it compiled, linked and installed in a standard way.
|
|
|
|
Sub-trees
|
|
~~~~~~~~~
|
|
.the source tree
|
|
All sourcecode of the core application resides below `src/`. Building these components is controlled by
|
|
the SConscript residing in this application source root. By convention, this is also the root for header
|
|
includes -- _all headers should be included relative_ to `src/`.
|
|
|
|
.the three layers
|
|
Within this application core tree, there are sub-trees for the main layers comprising the application.
|
|
Each of these sub-trees will be built into a shared library and then linked against the application framework
|
|
and common services residing in `src/common`. These common services in turn are also built into a shared
|
|
library `liblumieracommon.so`, as is the collection of helper classes and support facilities, known as
|
|
our 'support library' `liblumierasupport.so`. Besides, there is a sub-tree for core plug-ins and helper tools.
|
|
|
|
.the GTK Gui
|
|
one of the sub-trees, residing in `src/stage` forms the _upper layer_ or _user-interaction layer_. Contrary to
|
|
the lower layers, the Stage Layer (GUI) is _optional_ and the application is fully operational _without GUI._
|
|
Thus, the GTK Gui is built and loaded as Lumiera a plug-in.
|
|
|
|
.unit tests
|
|
Since our development is test-driven, about half of the overall code can be found in unit- and integration
|
|
tests, residing below `test/`. There is a separate SConscript file, to define the various
|
|
link:{ldoc}/technical/infra/TestSupport.html[kinds of test artefacts] to be created.
|
|
|
|
- plain-C tests are defined in _test-collections_, grouped thematically into several subdirectories.
|
|
Here, each translation unit provides a separate +main()+ function and is linked into a stand-alone
|
|
executable (yet still linked against the appropriate shared libraries of the main application layers)
|
|
- the tests covering C++ components are organised into test-suites, residing in separate sub-trees.
|
|
Currently (as of 5/2015), we link each sub-tree into a shared test library. Here
|
|
individual translation units define individual test case classes. At the end, all these unit tests
|
|
are linked together with a testrunner `main()` into the `test-suite` executable.
|
|
|
|
.research
|
|
There is a separate subtree for research and experiments. The rationale being to avoid re-building most
|
|
of the core application when it comes to experimenting and trying out new technologies.
|
|
|
|
.icons and resources
|
|
the +data/+ subtree holds resources, configuration files and icons for the GUI. Most of our icons
|
|
are defined as SVG graphics. The build process creates a helper executable (+rsvg_convert+) to render
|
|
these vector graphics with the help of lib Cairo into icon collections of various sizes.
|
|
|
|
.documentation
|
|
Largely, the documentation is written in Asciidoc and provided online at link:{ldoc}[the documentation section]
|
|
of our website. The plain-text sources of this documentation tree is shipped alongside with the code.
|
|
Besides, we build *Doxygen* API documentation there, and we create design and technical specs and drawings
|
|
in SVG and in UML.
|
|
|
|
.the target directory
|
|
This is where the results of the build process are created. Lumiera is organised into a
|
|
_self contained folder structure_. As long as the relative locations, as found within +target/+,
|
|
are kept intact, the Application will be able to start up and find all its resources. Consequently,
|
|
there is no need to ``install'' Lumiera (and the ``install'' target just copies this folder structure
|
|
into the standard installation locations of a typical Unix system)
|
|
|
|
Unfortunately SCons is a bit weird regarding the object files created during the build process.
|
|
So, for the time being, we're just building in-tree. Apologies for that.
|
|
|
|
|
|
Invoking the Build
|
|
~~~~~~~~~~~~~~~~~~
|
|
All of the build process is launched through the `scons` python script, usually installed into
|
|
`/usr/bin` when installing the SCons package onto the system. Just invoking
|
|
|
|
scons -h
|
|
|
|
prints a summary of all custom options, targets and toggles defined for our build.
|
|
|
|
Targets
|
|
^^^^^^^
|
|
- *build* is the default target: it creates the shared libs, the application, core plug-ins and the GUI.
|
|
- *testcode* additionally builds the research and unit test code
|
|
- *check* builds test code and runs our test-suites
|
|
- *research* builds just the research tree
|
|
- *doc* builds documentation (currently just Doxygen)
|
|
- *all* builds the Application, test-suites and documentation
|
|
- *install* builds and installs the Lumiera Application
|
|
|
|
By convention, invoking `scons -c` <TARGET> will _clean up_ everything the given target _would_ build.
|
|
Thus, invoking `scons -c /` is the most global clean operation: it will clean up al build artefacts and
|
|
will un-install Lumiera (recall: every defined node, or directory is also a target).
|
|
|
|
Configure checks
|
|
^^^^^^^^^^^^^^^^
|
|
SCons does not support the concept of a separate ``configure'' step. The necessary dependency detection is
|
|
performed before each build. Currently, we expect _all dependencies to be installed first-class_ into
|
|
the system. We do not (yet) support custom libraries in strange locations. Please use your package manager.
|
|
|
|
Caching and MD5 sums
|
|
^^^^^^^^^^^^^^^^^^^^
|
|
SCons stores MD5 sums of all source files, all configure checks and all the command lines used
|
|
to invoke compilers and external tools. The decision, what needs to be rebuilt is based entirely
|
|
on these checksums. For one, this means that configure checks are re-run only when necessary.
|
|
It also means that changes to some compiler switches will automatically cause all affected parts
|
|
of the application to be re-built. And of course it means, that you only ever compile what is
|
|
necessary.
|
|
|
|
With SCons, there is no need for the usual ``cleaning paranoia''. Similarly, there is no need
|
|
for CCache (but using DistCC rocks !). Unfortunately, calculating those MD5 sums requires some
|
|
time on each build, even if the net result is that nothing will happen at all.
|
|
|
|
Configuration options
|
|
^^^^^^^^^^^^^^^^^^^^^
|
|
We provide several custom configuration options (run `scons -h` to get a summary). All of these
|
|
options are *sticky*: once set, the build system will recall them in a file '.optcache' and apply
|
|
them the same way in subsequent builds. It is fine to edit '.optcache' with a text editor.
|
|
|