From 51b3f0defae55e7067c0850040e6cc79ae5b9f91 Mon Sep 17 00:00:00 2001 From: Ichthyostega Date: Sun, 22 May 2011 00:50:23 +0200 Subject: [PATCH 1/3] start a page to collect technical notes about Scheduler and Jobs (Backend) Signed-off-by: Ichthyostega --- doc/technical/backend/index.txt | 3 ++- doc/technical/backend/scheduler.txt | 40 +++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+), 1 deletion(-) create mode 100644 doc/technical/backend/scheduler.txt diff --git a/doc/technical/backend/index.txt b/doc/technical/backend/index.txt index 58fd1a5e9..4e7c2bda9 100644 --- a/doc/technical/backend/index.txt +++ b/doc/technical/backend/index.txt @@ -1,8 +1,9 @@ Technical Documentation: Backend ================================ -Eventually, this will have technical documentation for the Backend. +Here we collect bits of technical documentation for the Backend. For now, we have: * link:ConfigLoader.html[Config Loader brainstorming from 2008] + * link:scheduler.html[Scheduler and Jobs] diff --git a/doc/technical/backend/scheduler.txt b/doc/technical/backend/scheduler.txt new file mode 100644 index 000000000..f62da23dd --- /dev/null +++ b/doc/technical/backend/scheduler.txt @@ -0,0 +1,40 @@ +Scheduler and Job handling +========================== + +The purpose of the _Scheduler_ is to run small self contained _Jobs_ +ordered by priority and observing specific timing constraints. + +Scheduler implementation ideas +------------------------------ + +Use multiple priority queues + +- background work +- foreground high-priority +- soft-realtime actions + +About Jobs +---------- +A job is a closure to run a small and limited action or operation, which +in itself _should not block_. Job may depend on each other and on resources +to be provided. A job may be conained in multiple queues and may be marked +as _canceled_ -- in which case the job function will never run and the job +will be discarded on occasion. + +Job States +~~~~~~~~~~ + +[source,C] +-------------- +enum job_state +{ + done, // already done, nothing to do + running, // job is running + waiting, // waits for some resource (annother job) + rejected, // sorry, cant do that dave, time will run out + expired, // time expired + aborted // got aborted +} +-------------- + + From c6f4f9bf25035cfe0ac37e5b4354e99b4c62bf50 Mon Sep 17 00:00:00 2001 From: Ichthyostega Date: Sun, 22 May 2011 05:45:59 +0200 Subject: [PATCH 2/3] create pages and sections for the design of output handling and player subsystem Signed-off-by: Ichthyostega --- doc/design/architecture/playRender.txt | 20 ++++++------- doc/design/engine/OutputHandling.txt | 15 ++++++++++ doc/design/engine/index.txt | 3 +- doc/user/intro/intro.txt | 39 ++++++++++++++------------ 4 files changed, 48 insertions(+), 29 deletions(-) create mode 100644 doc/design/engine/OutputHandling.txt diff --git a/doc/design/architecture/playRender.txt b/doc/design/architecture/playRender.txt index afcf2b61f..2d76f716e 100644 --- a/doc/design/architecture/playRender.txt +++ b/doc/design/architecture/playRender.txt @@ -70,8 +70,8 @@ proceeding along a timeline). Reconfiguration ^^^^^^^^^^^^^^^ -Some of these operation modes need to be prepared to an unpredictable live reconfiguration, -driven by user interactions: +Some of these operation modes need to be prepared to encounter an unpredictable live +reconfiguration, driven by user interactions: - any part of background rendering can be invalidated and restarted, while other parts should be re-integrated, possibly with adjusted position @@ -108,7 +108,7 @@ playback location, and it can be hooked up with a play-control GUI widget Each play-controller in turn gets associated with several *play/render-processes*, one for each independent media stream (channel) to be produced. Of course this -isn't an operating system process; rather, ach such process is a compound of entries +isn't an operating system process; rather, each such process is a compound of entries in a registration table, which serve the purpose of tying several other services together, which we initiate and use in order to make that render process happen. Most notably, we'll use the services of the actual engine, which provides us with kind of @@ -126,7 +126,7 @@ Viewer and Output connection ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Creating a player instance binds together three partners: a _timeline_, a _viewer_ and _the engine_. While the timeline provides the content to play, the _viewer connection_ -is crutial for working out the actual output sink(s) and thus the output format to use. +is crucial for working out the actual output sink(s) and thus the output format to use. Thus, a viewer connection is prerequisite for creating a player instance. Viewer connections exist as associations in the session/model -- as entities separate @@ -135,13 +135,13 @@ such a connection is (still) missing, building a player instance recurs to the s to get a suitable viewer _allocated_. The viewer connection can't be broken during the lifetime of that player instance (or putting it the other way: breaking that viewer connection, e.g. by forcing a different connection or by shutting down the viewer, -immediately terminates the player. This detaching works synchroneously, i.e. it -blocks untlil all the allocated _output slots_ could be released. +immediately terminates the player. This detaching works synchronously, i.e. it +blocks until all the allocated _output slots_ could be released. Live switching ^^^^^^^^^^^^^^ While the viewer connection can be treated as fixed during the lifespan of a player -instance, several life switching and reconfiguration operations might happen anytime: +instance, several life switching and reconfiguration operations might happen any time: The _model port_ (place where data is retrieved from calculation), the output characteristics (framerate, direction) and the delivery goals (playback position, loop playing, scrubbing) all may be changed during playback -- we need a way for the player to ``cancel'' and @@ -164,7 +164,7 @@ the quantisation, which leaves us with just a few possible junction points where to place quantisation: The backend, the GUI, the player, the session. - putting it into the backend seems to be the most reasonable at first sight: - We can ``do away'' with nasty things soon, especially if they are technicallities, + We can ``do away'' with nasty things soon, especially if they are technicalities, ``get a clean state soon'' -- and hasn't frame quantisation something to do with media data, which is handled in the backend? + @@ -174,7 +174,7 @@ amount of degraded information flows throughout the whole system; thus the general rule to do it as late as possible. Uncrippled information is enablement. And last but not least: the frame quantisation is connected to the _output_ format -- and the backend is likely within the whole -application the subsytem most remote and unaware of output requirements. +application the subsystem most remote and unaware of output requirements. - rounding/quantising in the GUI is extremely common within media applications; unfortunately there seems to be not a single rational argument supporting that habit. @@ -184,7 +184,7 @@ Which leaves us with the player and the session. Both positions could arguably be supported. Here, a more careful consideration shows, that the ``act of frame rounding'' can be decomposed: into the _act of quantisation_ and the _frame grid:. Basically its the session which has the ability -to form the *frame grid*, but it is lacking crutial information about +to form the *frame grid*, but it is lacking crucial information about the output. Only when connecting both -- which is the essence of the player -- frame quantisation can actually be performed. Thus, the player is the natural location to perform that quantisation operation. diff --git a/doc/design/engine/OutputHandling.txt b/doc/design/engine/OutputHandling.txt new file mode 100644 index 000000000..fa3d5e268 --- /dev/null +++ b/doc/design/engine/OutputHandling.txt @@ -0,0 +1,15 @@ +Design: Output Handling +======================= +:Date: June 2011 +:Author: Ichthyostega + +//Menu: label Output handling + +Some ideas.... + +- abstract away the actual technology used for output +- have generic *output designations* and translate them into an *output slot* +- the OutputSlot interface can be designed to match the requirements of the Engine +- assume a mechanism to handle timeouts, glitches and skips within each concrete OutputSlot implementation + + diff --git a/doc/design/engine/index.txt b/doc/design/engine/index.txt index 7d9b5ea39..0852b6bac 100644 --- a/doc/design/engine/index.txt +++ b/doc/design/engine/index.txt @@ -1,5 +1,6 @@ Design Documents: Renderengine ============================== -Eventually, this will have design documentation for the Engine. +This section contains design documents regarding the overall workings of the Render Engine, +and the handling of output generation and output connections. diff --git a/doc/user/intro/intro.txt b/doc/user/intro/intro.txt index 0e32a454b..5617dd44a 100644 --- a/doc/user/intro/intro.txt +++ b/doc/user/intro/intro.txt @@ -1,6 +1,7 @@ Lumiera (as seen) from Outer Space ================================== :Author: Christian Thäter ct@pipapo.org +:Author: Hermann Voßeler Ichthyostega@web.de :Date: Summer 2010 @@ -18,13 +19,10 @@ they get to work with. It is aimed for workflow designers any anyone who wants to know how the program works in general. ****************************************************************************** - -About this Document -------------------- - // all things starting with '//' are asciidoc comments and drafts/notes while // working on this document +.About this Document This document is meant to be read electronically, it contains a lot hyper-links between explanations denoted by an arrow ->. Lumiera is still in development, we describe here planned features without explicitly tagging them; @@ -38,8 +36,13 @@ Vision // objective and goals of the project -Lumiera claims to be `professional', this is quite a vague term and needs -some explanation what it means to us: +Lumiera claims to be a _professional non-linear video editor_. To start with, we should +point out that ``professional'' does not necessarily mean ``commercial'' or ``industrial''. +It's more of an attitude or mindset -- doing work seriously, and to be subject to any +kind of wider goal, demand, or purpose. When it comes to editing film, this might be +artistry, a narration or meaning to convey, a political message or something to show +to your audience. Anyhow, for the tools, the editing software used to this end, +we can identify several properties and requirements, to be labeled ``professional'': Reliability:: Whatever happens, your work must be safe, protected against software @@ -47,34 +50,34 @@ some explanation what it means to us: never crash, in practice even crashes or power outages should not result in lost work. - Productivity:: + Quality:: + If you work with high quality, cinema grade digital video material you + want to be sure that you can deliver crisp quality without compromise, + throughout the whole workflow to your final product. All rendering + must be reproducible to the bit. + + Performance and Productivity:: Professionals want to get things done, in time, but optionally with control over every aspect. Balancing these goals should be the central concern for workflow design and usability. - Quality:: - If you work with high quality, cinema grade digital video material you - want to be sure that you can deliver this crisp quality without - compromise throughout you workflow to your final product. All rendering - must be reproducible to the bit. - - Scalability:: + Scalability and Adaptability:: Projects and budgets differ, hardware advances, Lumiera must scale in different dimensions and use the available resources as best as it - can. From small Laptops to multicore Computers and Renderfarms. + can. From small Laptops to multi core computers and Renderfarms. - Future Proofness:: + Durability:: Soft and Hardware advances at a fast pace. We must not lock into the current state of technology but being flexible to extend the System without breaking compatibility. Projects you create nowadays with Lumiera should be usable in foreseeable future, at least there needs to be a guaranteed upgrade path. -< + Fundamental Forces ------------------ -// the basic ideas which drive the lumiera design +// the basic ideas which drive the Lumiera design The Lumiera design is guided by a small number of basic principles. Keeping these in mind will help to understand how actually more interesting things can From 64cf3c9d34715f1fb915714f057f2fb25f78cf90 Mon Sep 17 00:00:00 2001 From: Ichthyostega Date: Thu, 25 Aug 2011 03:03:43 +0200 Subject: [PATCH 3/3] update GDB-pretty-printer HOWTO Eclipse now supports structured display of STL containers --- doc/technical/howto/DebugGdbPretty.txt | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/doc/technical/howto/DebugGdbPretty.txt b/doc/technical/howto/DebugGdbPretty.txt index b46c70a72..e1a2c6f97 100644 --- a/doc/technical/howto/DebugGdbPretty.txt +++ b/doc/technical/howto/DebugGdbPretty.txt @@ -5,16 +5,24 @@ Since some time, GDB supports Python written extension modules, especially for pretty printing of data structures. A selection of pre-defined pretty printers for STL containers is part of the standard distribution. The graphical debugger frontend provided by the Eclipse CDT automatically uses this debugger provided presentation -to show the value of variables in the detail pane of the variables view, while the -individual variable entries always show the expandable structure view. +to show the value of variables in the detail pane of the variables view. The most recent +version of CDT (Version 8 for Eclipe 3.7 »Indigo«) is even able to populate the structure view +based on the python pretty printer's output, which is a big step ahead towards easy debugging +of more elaborate data structures based on STL containers. + Installation notes ------------------ -This feature requires an python enabled GDB (>6.8.50). Debian/Lenny isn't enough, -but compiling the GDB package from Debian/Squeeze (GDB-7.1) is straightforward. -Moreover, you need to check out and install the python pretty-printers and -you need to activate them through your +~/.gdbinit+ +This feature requires an python enabled GDB (>6.8.50). Actually, even the most recent stable +GDB (Version 7.2) is recommended, because it contains some relevant bugfixes. Debian users +might want to backport the the GDB package from Debian/Wheezy (GDB-7.2). +Moreover, you need to check out and install a recent version of the python pretty-printers +from the GNU Subversion repository: + +* 'svn checkout' + http://gcc.gnu.org/viewcvs/trunk/libstdc%2B%2B-v3/python/[svn://gcc.gnu.org/svn/gcc/trunk/libstdc++-v3/python stlPrettyPrinter] +* you need to activate them explicitly through your +~/.gdbinit [source,python] ---- @@ -61,4 +69,8 @@ When selecting the string or the vector in the Eclipse variables view (or when issuing "print str" at the GDB prompt), the GDB python pretty-printer should be activated. +NOTE: to get the full support in _Eclipse Indigo + CDT-8_, you need to activate + the setting ``enable pretty printers in variable/expression tree'', which + can be accessed as Window/Preferences > C++ / debug / GDB +