ichthyo/LUMIERA.clone
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

move in and adjust some of the asciidoced design documents

Browse source
This commit is contained in:
Fischlurch 2010-11-15 02:21:13 +01:00
parent 3bd7c610fc
commit 414f8a066d
22 changed files with 1447 additions and 41 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
doc/design/application/config.txt → doc/design/application/Config.txt
View file

@ -1,6 +1,9 @@
Lumiera Design: Application and Session Configuration
=====================================================
*TODO* : 'write design document'
-> see also the link:{ldoc}/technical/backend/ConfigLoader.html[Config Loader brainstorming from 2008] (implementation details)
[icon="warning.png"]
WARNING: Website under construction

154
doc/design/application/SubsystemLifecycle.txt Normal file
View file

@ -0,0 +1,154 @@
Lumiera: Startup and Shutdown of Subsystems
===========================================
There is now sort-of an "application realm", which doesn't belong strictly to
one of the Layers. It serves the purpose of pullig up and tearing down the
application in a controlled fashion. And additionally, it provides the Interface
and Config core services. Within the application, we find a small number of
_Subsystems_, which are more or less independent. These subsystems are
conceptual entities and don't correspond 1:1 to a layer, an interface, a class
or a plugin. These subsystems are _of no relevance_ outside the mentioned
"application realm". When the application is in normal operational mode, we have
the usual components, interfaces, services, aggregated into the three Layers.
__Currently I've identified the following subsystems:__
- Engine
- Builder
- Session
- Lumiera GUI
- Script runner
- Renderfarm node server
To deal with those subsystems, I've created an Interface to definesthe
operations and liabilities of a subsystem. Additionally, I assume that for each
subsystem there is a _Facade_, which the subsystem is free to implement as it
sees fit. Typically, this facade will load plugins, register and provide further
"business interfaces", and especially set up the _Layer separation interfaces_
which canalise any communication going on between the layers.
The code from my "startup" branch has meanwhile been merged to master. Look
http://git.lumiera.org/gitweb?p=LUMIERA;a=tree;h=a0a0e456a5b149df81b25a08358cd488631639fb;hb=a0a0e456a5b149df81b25a08358cd488631639fb[here]
for the code mentioned in the following discussion.
- +common/subsys.hpp+ contains the mentioned subsystem interface.
- +lumiera/main.cpp+ uses the subsystem instances provided by the facades, and
additionally the services of +lumiera::AppState+ (+common/appstate.hpp+)
- AppState represents the state as relevant for the "application realm", i.e.
it performs global initialisation and shutdown. See especially +AppState::init()+
- see +backend/enginefacade.hpp|cpp+ as an (unimplemented) facade skeleton. +backend::EngineFacade::getDescriptor()+
yields the subsytem interface
- the GuiFacade is somewhat special, because we want to load the GUI from a
shared libary. This facade is basically completed and working, but it currently
just loads a dummy plugin. The implementation of the GuiFacade needs to be in
core (because it pulls the plugin); that's why I've put it into
+common/guifacade.cpp+, while the interface is in +gui/guifacade.hpp+ as usual.
- as an example for a _Layer separation interface_, I've implemented the
GuiNotificationFacade, which will be used by the lower layers to push
informations into the gui (and finally to request the GUI to shut down). Layer
separation interfaces are considered part of the public Lumiera API, thus the
headers go into +src/include/**+
* include/guinotification.h (C/C++ combined header) defines an C++ interface (abstract class) and a CLI interface.
* embedded into the interface is a factory, i.e. by gui::GuiNotification::facade() you get an instance...
* which actually is a proxy and routes any call through the instance of the
accompaning CLI interface which is defined within the interface/plugin system
* this in turn forwards to the implementation class in
gui/guinotificationfacade.cpp, which is assumed to live within the GUI
(shared lib)
Parallelism
-----------
Actually this system builds on the assumption, that starting each subsystem
doesn't block the overall start/main/stop thread. I.e. any subsystem is supposed
to spawn its control/event threads if necessary. Not every subsystem needs to
spawn threads though (for example, the session doesn't). In the current
implementation *no spawning of threads happens*. Similarily, I've commented out
the synchronisation primitives.
Initialisation and Lifecycle
----------------------------
Basically, everything workes as discussed last spring in the
link:{ldoc}/devel/rfc/GlobalInitialization.html[GlobalInitialisation]
design entry. I've considered to switch to simple functions +pre_init(), init(),
...+ as proposed by Cehteh, but I'm not happy with this idea, because it creates
a coupling between anything which needs to be done in a certain initialisation
function. Actually, I prefer the usual approach of lifecycle events (or signals)
used in many application frameworks, i.e. the publisher-subscriber model. This
allows to keep the registration immediately within the implementation of a
facility, and it allows to add an arbitrary numer of additional lifecycle
events, like *ON_SESSION_CLOSE*, *ON_BUILD*, *ON_EMERGENCY_EXIT*.
Basically we've now the following steps and events happening
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* *ON_BASIC_INIT* runs "somewhere" in the static initialisation phase before
main(). To schedule something there, you need to place a statc C++ variable.
This should be reseverd for very basic initialisation tasks which need to be
done even prior to any global initialisiation, e.g. the NOBUG_INIT is done this
way, the config system, maybe setting up a dispatcher table based on (compile
time) type information (for the rules engine).
* +AppState::init()+ cares for bringing up the pluginloader and opens the config-interface.
* ...followed by triggereing *ON_GLOBAL_INIT*
* +main()+ pulls up the subsystems according to the command line options.
* within each subsystem, facade interfaces shall be opened with the interface
system. Any initialisation of components should be tied to them.
* as a rule and if possible, _any service should be written such as to come up on demand._
* shutdown or failure of _any given subsystem_ initiates the shutdown sequence
by requesting all other running subsystems to terminate
* there is an *ON_GLOBAL_SHUTDOWN* event, which can be used for normal cleanup.
In case of an emergency exit, this hook may be skipped
* alternatively, the *ON_EMERGENCY_EXIT* event is triggered. In case of nested
exceptions, this may happen twice.
* the AppState destructor tries to bring down the core systems (config-interface and pluginloader).
Demo Run
--------
* building with scons (default target) now yields a bin/guistart.so, besides the bin/lumiera exe the scons built created since day one.
* we need to set the LUMIERA_PLUGIN_PATH from the commandline to point to this bin directory, e.g.
* +++LUMIERA_PLUGIN_PATH=/home/hiv/devel/lumi/.libs NOBUG_LOG='test:TRACE,lumiera:TRACE,config:TRACE' bin/lumiera 2>&1 | egrep -v '(TODO)|(FIXME)'+++
------
00000392: configfacade.cpp:74: INFO: thread_1: Config: Config system ready.
00000394: main.cpp:55: NOTICE: thread_1: main: *** Lumiera NLE for Linux ***
00000395: appstate.cpp:114: TRACE: thread_1: init: initialising application core...
00000405: interface.c:54: TRACE: thread_1: lumiera_interface_open: lumieraorg_interface
00000406: interface.c:55: WARNING: thread_1: lumiera_interface_open: opening experimental interface: lumieraorg_interface_0_lumieraorg_interface
00000412: interface.c:168: TRACE: thread_1: lumiera_interface_open_interfacenode: lumieraorg_interface 0 ()
00000437: config.c:199: NOTICE: thread_1: lumiera_config_get: envvar override for config LUMIERA_PLUGIN_PATH = /home/hiv/devel/lumi/
00000444: config.c:199: NOTICE: thread_1: lumiera_config_get: envvar override for config LUMIERA_PLUGIN_PATH = /home/hiv/devel/lumi/
00000467: appstate.cpp:132: TRACE: thread_1: init: Lumiera core started successfully.
00000469: appstate.cpp:140: TRACE: thread_1: maybeStart: maybe startup Builder...?
00000472: appstate.cpp:140: TRACE: thread_1: maybeStart: maybe startup Renderfarm node...?
00000475: appstate.cpp:140: TRACE: thread_1: maybeStart: maybe startup Lumiera GTK GUI...?
00000476: subsystemrunner.hpp:144: INFO: thread_1: triggerStartup: Starting subsystem "Lumiera GTK GUI"
00000477: interface.c:54: TRACE: thread_1: lumiera_interface_open: lumieraorg_GuiStarterPlugin
00000483: interface.c:168: TRACE: thread_1: lumiera_interface_open_interfacenode: lumieraorg_GuiStarterPlugin 0 ()
*** Ha Ha Ha
this is the GuiStarterPlugin speaking!
now, the Lumiera GUI should be spawned....
but actually nothing happens!!!!!!!!!!!!!!
00000491: appstate.cpp:140: TRACE: thread_1: maybeStart: maybe startup Script runner...?
00000494: appstate.cpp:171: NOTICE: thread_1: maybeWait: Shutting down Lumiera...
00000495: interface.c:230: TRACE: thread_1: lumiera_interface_close:
00000498: interface.c:258: TRACE: thread_1: lumiera_interfacenode_close: lumieraorg_GuiStarterPlugin 1 ()
00000634: configfacade.cpp:83: TRACE: thread_1: ~Config: config system closed.
00000635: appstate.cpp:245: TRACE: thread_1: ~AppState: shutting down basic application layer...
00000643: interface.c:230: TRACE: thread_1: lumiera_interface_close:
00000646: interface.c:258: TRACE: thread_1: lumiera_interfacenode_close: lumieraorg_interface 1 ()
------
* similarily, running... `lumiera \--help`
------
00000392: configfacade.cpp:74: INFO: thread_1: Config: Config system ready.
00000394: main.cpp:55: NOTICE: thread_1: main: *** Lumiera NLE for Linux ***
Lumiera, the non linear video editor. Supported parameters:
-h [ --help ] produce help message
-f [ --session ] arg session file to load (UNIMPLEMENTED)
-s [ --script ] arg execute the given LUA script (UNIMPLEMENTED)
--headless start without GUI
-p [ --port ] arg open renderfarm node at given port (UNIMPLEMENTED)
-d [ --define ] arg enter definition into config system (UNIMPLEMENTED)
------

4
doc/design/application/index.txt
View file

@ -1,9 +1,9 @@
Design Documents: Application Framework
=======================================
* link:config.html[Configuration]
* link:Config.html[Configuration]
* link:plugin_loader.html[Plugin Loader]
* Application startup- and shutdown
* link:SubsystemLifecycle.html[Application Subsystems and Lifecycle]
[icon="warning.png"]
WARNING: Website under construction

200
doc/design/builddrone/BuildDroneConcept.txt Normal file
View file

@ -0,0 +1,200 @@
A Build Drone for Lumiera
=========================
:Author: Christian Thäter
:Date: 2008
This is a brainstorming page for what a build system on the server would need.
First of all, it should be simple and extensible, preferably by simple shell
scripts (note that *anything* else is a subset of a shell script!). We should
use existing good ol' unix utilities when applicable (batch, mail,...).
In my opinion this all can be a lot shell code glued together. Things should be
isolated to single tasks and well refactored to make it maintainable.
Lets go:
Whats needed?
-------------
* A description/configuration what needs to be done, these may be recursive.
* a batch/scheduling system (batch)
* triggers/events which are passed along (initiate a build by a git commit, ...)
* actions to perform
* handlers for success and failure cases which act on the outcome of a action
(scp, mail, rsync, filters to produce asciidoc output...)
* maybe some dependency handing (only build lumiera after installing the recent nobug version successfully)
Top Down scratch
----------------
Lets say the configs are shell scripts.
first a verbose one:
------------------------------------------------------------
cd project_dir
if git pull; then
# call handlers for success case
sendmail foo@bar.com <<EOF
blablah
EOF
sendmail other@blah.net <<EOF
blahblah
EOF
# success event which tells the caller to proceed
Huh?
else
# call handlers for the fail case
sendmail boss@bar.com <<EOF
blablah
EOF
# fail event which tells the caller to abort (or whatever)
Hah!
fi
------------------------------------------------------------
Ok that above doesnt look good, but would do the job with some efforts
Now lets factor this to some shell functions:
------------------------------------------------------------
function link:AddToReport[]()
{
cat >>,build_drone_report
}
function link:GitPull[]()
{
local dir="$1"
local repo="${2:-origin}"
local branch="${3:-master}"
ACTION="GitPull $*"
RESULT=FAILURE
cd "$dir" 2>&1 | link:AddToReport[] , return 1
git pull "$repo" "$branch" 2>&1 | link:AddToReport[] , return 1
RESULT=SUCCESS
return 0
}
function link:SendMail[]
{
cat ,build_drone_mail <<EOF
Subject: Build Drone Report
$ACTION $RESULT
blah blah
EOF
cat ,build_drone_report >>,build_drone_mail
for i in "$@"; do
sendmail "$i" <,build_drone_mail
done
}
------------------------------------------------------------
now we can do following: (looks much nicer or?)
------------------------------------------------------------
if link:GitPull[] project_dir; then
link:SendMail[] foo@bar.com other@blah.net
else
link:SendMail[] boss@bar.com
fi
------------------------------------------------------------
the above snippet can be a file in some config dir and will be processed by sourcing it with proper setup
lets place it in /build_drone/project/10GitPull
------------------------------------------------------------
for i in "/build_drone/project/*"; do
(
source "$i"
)
link:CallTrigger[]
done
------------------------------------------------------------
... ok enough for now, whats needed is the link:CallTrigger[] implementation,
stopping processes waiting (either reading on a pipe or wait for signal (-CONT
for success, -TERM for failure))
setup of the function scriptlets:
for i in link:/usr/local/lib/BuildDrone/[]*; do source $i; done ...
and so on
..
Conventions
-----------
. one drone runs always in a well defined directory, if commands have to go into subdirs, use subshells (good idea anyways)
. all jobs need to run in serial order, nothing parallel. What the job then does itself (make -j3 ..) can be parallelized.
. we need a well defined environment and some file where it is registered ,bd_env or so
. all jobs append write their output to some well defined logfile (logfiles ..)
. the status must be recorded in a file too
Example:
------------------------------------------------------------
function link:BatchJob[]()
{
batch <<EOF
(
$(cat)
) >&log
echo $? >,bd_exitcode
kill -CONT $$
EOF
# wait for signal
suspend
return $(<,bd_exitcode)
}
------------------------------------------------------------
This of course needs some more work, trap handlers for signals etc. I am not
sure if the idea with signals is good, maybe better use a pipe, lets see.
Actions and Handlers Brainstorming
----------------------------------
Ideas, not in order
link:SendMail[]::
send report as email
link:GitPull[]::
updates a git checkout +
Split this in
* link:GitFetch[]
fetch from remote
* link:GitHasChanges[]
are there changes between HEAD and remote?
* link:GitCheckout[]
reset to the remote head and checkout
Bootstrap::
autoreconf and configure
Make::
runs 'make'
Check::
runs 'make check'
Distcheck::
runs 'make distcheck'
Doxygen::
runs doxygen
link:StyleCheck[]::
checks if files violate the style rules, something like:
'astyle --style=gnu <$FILE.c | diff -u FILE.c - ...'
link:ToDos[]::
greps for TODO (with some -A context) and produces a report
* beautify (asciidoc) reports
* publish reports, packages etc to the webserver (scp, rsync)

1
doc/design/builddrone/index.txt
View file

@ -1,6 +1,7 @@
Lumiera Builddrone
==================
* link:BuildDroneConcept.html[initial concept from 2008]
[icon="warning.png"]
WARNING: Website under construction

28
doc/design/gui/GuiBrainstorming/Proposal.RichardSpindler.txt
View file

@ -1,28 +0,0 @@
GUI Brainstorming Proposal
==========================
:Author: Richard Spindler
:Date: 2008-30-03
Node Graph proposal
-------------------
on 2/6/2008 Richard Spindler wrote:
I've made a little Picture about how I think the User Interface of a
yet to be named Video Editor could be presented. There is no central
notion of the camera/projector Model, which I think could be added
anywhere in the Graphs as a Node with an unlimited Number of inputs.
Automations are not yet part of the Mockup, but could be added.
Cross Bus connections would likely not be represented by Arrows in a
real UI, but rather the Bus would be adressed by Name.
The dotted Lines mean that one element expands to a different View,
for example in a new Window.
The same goes for the rectangles with the slashed outline, they
represent another View, and the connection from the timeline to the
compositing nodes will be by Name, and not by arrows. Default Names
would be "Video Track #1", etc.
image::RichardSpindler.proposal.png[]

2
doc/design/gui/GuiBrainstormingReviewed.txt → doc/design/gui/GuiDiscussion/GuiBrainstormingReviewed.txt
View file

@ -313,5 +313,5 @@ Window configurations can be stored/restored in customizable presets and are par
Won't Implement
~~~~~~~~~~~~~~~
[[Include(/WontImplement, , editlink)]]
-> link:GuiBrainstormingWontImplement.html[Review of features we won't implement...]

0
doc/design/gui/GuiBrainstormingReviewed/WontImplement.txt → doc/design/gui/GuiDiscussion/GuiBrainstormingWontImplement.txt
View file

0
doc/design/gui/GuiBrainstorming/MenuAndShortcuts.txt → doc/design/gui/GuiDiscussion/MenuAndShortcuts.txt
View file

7
doc/design/gui/GuiBrainstorming/Proposal.AkhiL.txt → doc/design/gui/GuiDiscussion/Proposal.AkhiL.txt
View file

@ -1,7 +1,7 @@
GUI Brainstorming Proposal
==========================
:Author: AkhiL
:Date: 2008-30-03
:Author: IL'dar AKHmetgaleev (»AkhIL«)
Date: 2008-30-03
Wiring nodes within the timeline
@ -10,4 +10,5 @@ Wiring nodes within the timeline
In fall 2007, rather then using an external node editor, AkhiL proposed to do wiring
of effect nodes directly within the timeline. He did the following Mockup:
image::AkhiL.proposal.png[]
image::{imgg}/AkhiL.proposal.png[]

28
doc/design/gui/GuiDiscussion/Proposal.RichardSpindler.txt Normal file
View file

@ -0,0 +1,28 @@
GUI Brainstorming Proposal
==========================
:Author: Richard Spindler
:Date: 2008-30-03
Node Graph proposal
-------------------
'on 2/6/2008 Richard Spindler wrote:'
I've made a little Picture about how I think the User Interface of a
yet to be named Video Editor could be presented. There is no central
notion of the camera/projector Model, which I think could be added
anywhere in the Graphs as a Node with an unlimited Number of inputs.
Automations are not yet part of the Mockup, but could be added.
Cross Bus connections would likely not be represented by Arrows in a
real UI, but rather the Bus would be adressed by Name.
The dotted Lines mean that one element expands to a different View,
for example in a new Window.
The same goes for the rectangles with the slashed outline, they
represent another View, and the connection from the timeline to the
compositing nodes will be by Name, and not by arrows. Default Names
would be "Video Track #1", etc.
image::{imgg}/RichardSpindler.proposal.png[]

381
doc/design/gui/GuiDiscussion/TimelineDiscussion.txt Normal file
View file

@ -0,0 +1,381 @@
Timeline: Discussion
====================
:author: Joel Holdsworth
This is Joel Holdsworth's attempt to amalgamate all the ideas from all the brainstorming to one consistent idea that makes sense, can be implemented, and seems like it would be nice to use. Comments are welcome.
* Avid vs FCP 2006 - http://www.fini.tv/articles/avidfcp2006.html[]
* Final Cut Pro 5 - A First Look - http://www.kenstone.net/fcp_homepage/fcp_5_new_martin.html[]
* SpeedEdit Tutorials - http://www.newtek.com/speededit/tutorials/index.php[]
Brainstorming
-------------
Akhil
~~~~~
-> link:Proposal.AkhiL.html[AkhIL's Proposal]
.Things I like
* The roots of group tracks show what's being rendered inside (Ichthyo calls these root tracks "group bars"). This makes sense - I think it might do more harm than good to let the user edit in group roots.
* The power of manual wiring. I like the way you can wire things together on the timeline.
.Things I don't like
* Tracks seem to be generic - all tracks are the same: either a group track or a normal "track". Yet you've got some tracks with just filters, some with just automation curves, and some with just clips. The problem here is that it gets confusing to know what to do if we allow the user to mix and match. I think that tracks 1-4 and 5-6 in your diagram need to lumped together in a single complex track.
* Some of your links seem to go from clips, and some from full tracks. This behaviour needs to be consistent. It's difficult to know what to suggest. Either links go from clips only - in which case 1hour long effect applied to 100 clips becomes a pain in the arse (maybe you'd use busses in this case). The alternative is only applying constant links between tracks, but this is quite inflexible, and would force you to make lots of meta-clips, anytime you wanted the linkage to change in the middle of the timeline.
Richard Spindler
~~~~~~~~~~~~~~~~
-> link:Proposal.RichardSpindler.html[Richard's Proposal]
.Things I like
* You've got a concept of sub-EDLs and metaclips. We're definitely going to do that. If a clip is a metaclip, you can open up the nested timeline in another tab on the timeline view.
* Lots of flexibility by having filter graphs on the input stage of clips, and on the output stage of the timeline.
* Your scratch bus provokes the idea of busses in general - these are going to be really useful for routing video around.
.Things I don't like
* Filter graphs on the input stage: This concept might be better expressed with "compound effects", which would be a filter-graph-in-a-box. These would work much like any effects, and could be reused, rather than forcing the user to rebuild the same graph for 100s of clips which need the same processing. (See Alcarinque's suggestion for "Node Layouts").
* Output stage filter graph: It's a good idea, but I think we need to re-express it in terms of busses and the track tree. We should be able to give the user the same results, but it reduces the number of distinct "views" that the user has to deal with in the normal workflow. I believe we can find a way of elegantly expressing this concept through the views that we already have.
Alcarinque
~~~~~~~~~~
-> http://telniratha.atspace.com/spec.html[]
.Things I like
* Node Layouts in the media library. This isn't a page about the ML, but I like your ideas of having stored node layouts in the library. My own name for these is compound effects - but it's the same idea. Your idea of having collecting a library of these on the website is pretty cool as well.
* Being able to connect the view port to different plugs in the compositing system - rather like the PFL buttons on a mixing desk.
* Keyable parameters, automatable on the timeline
* Having a transform trajectory applied to every track
* Tabbed timeline for sub-EDLs for having more than one timeline in a project - e.g. metaclips: scne 1, scene 2 etc.
.Things I don't like
* I'm not sure, but I think some of these ideas stumble on the problems of
reconciling fixed node layouts with the variable nature of a timeline. Either
you make connections "per track", in which case there's not enough flexibility,
or "per clip", in which case I think there's too much if you have 100s of
clips.
Hermann Vosseler (Ichthyo)
~~~~~~~~~~~~~~~~~~~~~~~~~~
-> http://ichthyostega.de/cin3/drafts/GUI-draft1.html[]
.Things I like
* All clips are compound audio/video clips.
* Groups can be expanded and collapsed.
... more here ...
Clay Barnes (rcbarnes)
~~~~~~~~~~~~~~~~~~~~~~
-> http://www.hci-matters.com/blog/2008/05/21/lumiera-timeline-first-draft/[]
.Things I like
* Nudge buttons are shown when the user hovers over the clip. This could be quite a timesaver.
* Video filters are shown above the top track in the set of inputs that the effect is applied to. This is useful for displaying filters which have multiple inputs. The highlighting of other attached inputs is cool and useful.
* Automation curves are show in effects just the same as everything else.
* Split and heal buttons. Heal would be hard to do well - but still, it's a great idea. Might requre extra metadata to be stored in the clip fragments so they can be recombined.
* ichthyo: "you create sort-of “sub-tracks” within each track for each of the contained media types (here video and audio). To me this seems a good idea, as I am myself pushing the Idea (and implementing the necessary infrastructure) of the clips being typically multi-channel. So, rather than having a Audio and Video section, separate Video and audio tracks, and then having to solve the problem how to “link” audio to video, I favour to treat each clip as a compound of related media streams."
.Things I don't like
* If a filter has multiple inputs, then how does the user control which track in the tree actually sees the output of the effect? cehteh says: "You showing that effects and tracks are grouped by dashed boxes, which is rather limiting to the layout how you can organize things on screen and how things get wired together". Perhaps it would be better to require all input tracks to an effect to be part of a group track, rather than being able to spaghetti lots of tracks together. The down side of this is it would make it difficult to to use a group twice i.e. have a track that is both an input to an effect, and a normal track - part of the another tree.
* I'm not sure I understand the (+) feature. Can you explain it more for me?
* cehteh: "curves need to be way bigger". Maybe the tracks could be sizable.
* cehteh: "You show that some things (audio) tracks can be hidden, for tree like organization this would be just a tree collapse."
Joel Conclusions:
-----------------
Timeline like this:
image::http://www.kenstone2.net/fcp_homepage/images_fcp_5_new_martin/07_fc_studio_new_martin.jpg[]
Tracks
''''''''''
Busses
''''''''''
Outputs
- Multicamera Angles
- Track Tree?
- Mixed Mode Tracks? __such a good idea?__
- Scalable UI
- Small text
- Storyboard view?
- "M" key to add markers during playback
- Hovering nudge buttons.
- Compound Effects/Stored Node Layouts
- Keyable parameters, automatable on the timeline
- Tabbed timeline
- All clips are compound audio/video clips
- Groups can be expanded and collapsed.
- Extra Headers to display video and audio headings - multi-part headers.
- Groups are read only tracks, as are the outputs from busses.
- Every group a bus?
- Pointers instead of wiring?
Every group would actually be a multi-input effect. If no effect is explicitly
applied to the track, the default effect could be "overlay N-tracks". If a 1 or
2 or 3 input effect is applied, then only 1 or 2 or 3 child tracks would be
allowed. Some effects allow N-inputs. Note that we'd need an effect/effect-stack
for every type of media in the group - audio and video.
I want to say effects apply to not to clips, or to the whole length of tracks,
apply to the periods of a timeline, and in the UI as you change the period of an
effect it snaps to clips. However, I think effects have to applied upstream of
transitions. This means that effects have to be per-clip. One way to make long
effects would be to put the whole track in a group, and apply the effect to the
whole group.
What happens if you put a 4 channel audio clip in a timeline with 2 channel
clips? I guess we display the stereo and 4 channels are equal citizens, then
upmix all audio into sound space, then down-mix somewhere downstream.
.Node Stuff:
* In a compound plugin (stored node layout), I guess some of the parameters
would be fixed, but I guess you'd want to present params to the user. The
question is how. The most obvious way is to load the params of individual effect
nodes with mathematical expressions that relate their params to a list of
user-params for the compound node. These user-params would of course be
automatable. Mathematical expressions are pretty unfriendly to users though. I
wonder if it would be if these node-layouts were simply expressed with AviSynth
style script.
* See Open Movie Editor's node editor: http://www.openmovieeditor.org/nodefilters.html[]
Ichthyo's Comments
------------------
Hi Joel, I'll summarize some of my thoughts here. Maybe we should split of this
discussion to a separate page? Maybe we should carry over more detailed
discussions to the Mailinglist? First of all -- I really like your work and the
effort to get the GUI reasonable complete but simple. Next, when I choose to
take a certain approach for implementing a basic feature in the Proc-Layer, this
doesn't bind the GUI to present it exactly the way it's implemented. Of course,
in many cases the GUI will be reasonable close to the actual underlying
implementation, but it's an independent layer after all. The Proc I am building
is very flexible, and I take it for guaranteed that the GUI has the freedom of
using some of those possibilities always in one certain limited way. And,
finally, please don't take my arguments as general criticism -- I am just
questioning some of the "usual" approaches at places where I have the feeling or
the experience that it would be beneficial, i.e.where re-thinking the solution
could lead to a smoother treatment of the problems at hand.
Signal processing, signal routing and control of parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Most existing solutions we know or have worked with are more or less derived
conceptually from hardware sound mixing consoles and multitrack tape machines.
Most noticeable, a mixer strip, (track, bus) means at the same time actual data
processing, routing of signals and controlling of parameters. Actual hardware
works this way, but in software this is only a metaphor. The hard wiring is
rather within our brains. We think it's most reasonable and simple this way, but
actually it creates a bunch of artificial problems. That's why I am stressing
this point all the time.
You pointed out that my placement concept may go a bit too far, especially when
it attempts to think of sound panning or output routing also as a kind of
placement of an object in a configuraton space. And, on the other hand,
Christian asked "why then not also controlling plugin parameters by placement?".
I must admit, both of you have a point. Christians proposal goes even beyond
what I proposed. It is completely straight forward when considered structurally,
but it is also clearly beyond the meaning of the term/idea/concept "placement".
But what, if we re-adjust the concepts and think in terms of ''Advice'' ?
Thus, a placement would bind an media object into the session/sequence and also attach additional advice?
As an example, to apply this re-adjusted Idea to the gain control: When a track
group head has an gain/fade control this would just denote an advice to set the
gain. It does ''not'' mean that signal data has to flow "through" this
track/track group, it does ''not'' mean the gain is actually applied at this
point and it does ''not'' mean that any routing decision is tied to this gain.
Even signal chains being routed to quite different master busses can receive
this advice, given the source clips are placed within the scope of this track
group. The only difference needed on the GUI to implement this approach would be
an additional setting on the fader, which allows to disable it (no advice at
this point), to apply it to audio or video only, or to specify an custom
condition (like applying only to media with a certain tag).
Obviously, the concept similarly applies to output destination and routing.
Actually, by doing this switch-of-concept, most of the problems you and other
people noted in the various drafts simply go away. For example the scalability
problem which plagues all explicit node editor GUIs. Instead of doing
connections manually, just by putting an media object into a given context
(sequence, track, track group), it receives any routing advice available in this
context. Only those cases really in need of a special wiring would necessitate
an additional routing advice directly at the media object ("send to...").
Similar to the example with the fader control, the change needed in the GUI
would be an additional setting on the output chooser, allowing to disable it (no
advice here), applying it just to audio, video (text, midi or whatever media
type we get), or an custom condition or tag.
Thus, generally the price to pay would be a more elaborate track head area,
which actually is a collection of advice. Besides, of course it causes a lot
more work in the Proc Layer (which obviously I am willing to take or already try
to implement). And, finally, one price to pay is a steeper learning curve for
anyone accustomed to the conventional approach.
kinds of effects
~~~~~~~~~~~~~~~~
I am mulling this question over and over again, but up to now I didn't come up
with an unifying idea. Maybe we should face the fact that there are some
fundamentally different ways you need to apply an effect. It is very common for
sound applications to overlook the "per object" effects, thus forcing you to
create a separate track/bus just because you need some EQ for a single clip, but
you need it non-destructive and automatable. On the other hand, solely having
effects on individual objects doesn't scale. Overall, we have
* effects on individual clips/objects, often even only on parts of them
* effects applied to a certain time range and typically just to a group of tracks
* global effects on the master/bus level
Besides, there are things like "pre/post fader", but I take it that in our model
the user just can control the order in which effects are attached. Basically,
the mentioned global level could be unified with the track/time range level
effects. But especially for Lumiera, with our considerations of using multiple
timelines, meta-clips and nested sequences, we've probably killed this possible
simplification. Because now, an effect applied to the root track in a given
sequence indeed is something different than an effect applied to the master
output. Because the latter is something global, it's associated with the
timeline you currently render/playback, while the former sticks with the
sequence and thus could also go within a meta-clip or sit within a sub-sequence
which is used just as one scene in a larger movie. Anyway, I'm doubtful if a
global colour correction or a global sound balance mix could be emulated by
using a master sequence and effects placed on the root tracks there. Actually, I
can't see what would be better, in terms of handling or workflow. Is this
situation so common (probably yes) as to justify building a specific facility?
Or would it be enough just to build a specific GUI for this case (which then
would necessitate to define a binding between the specific GUI and some
otherwise just normal effects in a master sequence)?
Regarding effects, I should point out that in our concept, mask, "camera" and
"projector" (Cinelerra's terms) are just some specific effects. Given that, it
is clear that effects need to be applicable ''both'' pre transition and post
transition. But the following limitation seems reasonable to me and seems to
match the actual use:
* all per-object effects are pre-transition. Including a local source viewport
transformation ("camera"), flip and mirror, reframing (changing playback speed
of just one clip) and a masking of just this media object. Thus, the realm of
the individual media object would end at the transition(s)
* all per time/track-range effects are post-transition. Meaning, they are
placed to a time range and track group, and not attached at an individual
object. They use the individual objects on the track as source, already tied
together by transitions. This includes colour changes, visual effects like
blur, maybe again masking (track-level mask) and again a viewport change
("projector")
Btw. I very much agree with you that putting clips on group head tracks would be
rather confusing, while I think placing an effect at a group head track is OK
and feels natural. It will just be applied post-transition to all media within
the covered range. And, note again the power of the placement+advice concept:
this does ''not'' mean the data need to be merged together and piped through the
effect at this point. Rather, it means that each signal processing chain in the
covered range will get an instance of this plugin wired in at a position
equivalent to the place the effect is found on the group track with respect to
the track hierarchy. I.e. after wiring in the transition(s), instances for
effects placed at tracks are wired in order and ascending the track tree, and
finally the connection will be made to the mixing step or output destination
according to the output advice found for ''this individual'' object (but in most
cases this output destination advice will be inherited from the context, i.e.
some root track)
type free tracks and multichannel clips
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Actually, what I am proposing is not to have multi-mode tracks, but rather to
have type-free tracks. Type is figured out on a per-connection base while
building (or on demand). The rationale is, we should prefer to think in media
types, not so much in channel numbers etc. Each media type incurs some specific
limitations on how it needs to be handled. E.g, even if two audio clips both
have two channels doesn't mean they are compatible. One of them could be
binaural audio, meaning, that the other one need to be spatialized with an HRTF
prior to mixing (assumed your intention is to create an binaural sound track).
Thus, when trying to get the type handling correct based on tracks, we'll
quickly end up with lots of tracks, with the consequence that closely related
objects are teared apart in the GUI display, just because you need to put them
on different tracks based on their type or some other specific handling. That's
why I am going through the hassle of creating a real type system, so we can stop
(ab)using the tracks as "poor man's type system"
Now, what are the foreseeable UI problems we create with such an approach?
* getting the display/preview steamlined
* having to handle L-cuts as real transitions
* the need for an switching advice on the clip to control the streams actually to be used
* how to control which effect applies where
I think, all of those problems are solvable, and my feeling is that the benefits
of such an approach outweigh the problems.
The problems with the preview could be solved by introducing a preview renderer
component, instead of doing it hard-wired in code as most applications do. To
elaborate on this, what exactly is the benefit of showing 8 microscopically
small waveforms for an 7.1 sound track? It would be far better to use a preview
renderer, which could visualize a sum waveform, an sound energy average, maximum
or even an spectral display. Similarly, a video preview renderer could either
produce a strip of images taken from the footage, or just a single image (pivot
image), or a colouring derived from motion detection etc. This way, all clip
displays could be made to fit the same space in the GUI.
Regarding transitions, the Proc-Layer (with the use of the type information) has
to find out if an transition is applicable to a given pair of clips. If it is
not, it will be flagged as error. Besides, we need an editor for tweaking the
transition curve. Now we could consider differentiating this transition curve
for several media kinds contained the same clip (audio crossfade goes
differently than the video crossfade). But I think, this isn't worth the hassle,
as the user is always free to split off the sound channels into a separate clip
and put it on a neighbouring track (and by using a relative placement, the link
between both could be kept intact). Just, only the L-cut is such an common
editing technique, that it's worth just supporting it as a special kind of
transition.
Having type-free tracks and compound clips allows us to handle multicam
natively, without the need of a special "multicam-feature". Some thoughts:
* we need an additional advice to specify which channel from the clip should be
sent to output. (Implementation-wise, this advice is identical to the advice
used to select the output on a per-track base; but we need an UI to set and
control it at the clip). Of course, different channels (e.g different sound
pickups) could be sent to different destinations (mixing subgroup busses).
For multicam, this advice would select the angle to be used.
* now we could think of allowing such angle switching to happen in the middle
of an clip. Meaning, the advice would be attached like an label. Personally, I
am rather against it, as it seems to create a lot of secondary problems. The
same effect can be easily achieved by splitting into multiple clips and then
using rolling edits. Plus, this way you also could attach transitions or
deliberately use slide and shuffle edits (creating temporal jumps in the
overall flow)
When a clip contains multiple media streams, we build an pipe for every stream
actually sent to some output. Now the question is, what happens when attaching
an effect (either attaching it directly to the clip, or placing the effect on a
track/trackgroup in a timerange touching this clip). Basically, Proc-Layer will
be able to determine if an effect is applicable at all. It won't try to wire in
an sound effect into a video pipe. It won't wire an stereo reverb into a mono
sound pipe. Thus, I would propose a pragmatic approach to this problem: Let the
effect apply to those processing pipes he is applicable and ignore the others.
Plus having the ability to limit by an additional advice with an condition or
tag (a possibility which I take for granted anyway). We could go for a much more
elaborate solution, like being able to expand the clip and then specify
explicitly which effect to attach where. But I think, as we have real nested
sequences which appear as meta-clips when used, limiting the effects to just
apply to "all applicable" is not actually a problem. Again, in the rare cases
where more is needed, the user is free to split off a channel as a separate
clip.

0
doc/design/gui/GuiBrainstorming/Workspaces.txt → doc/design/gui/GuiDiscussion/Workspaces.txt
View file

18
doc/design/gui/GuiDiscussion/index.txt Normal file
View file

@ -0,0 +1,18 @@
GUI Design and Feature Discussion
=================================
In the early stages of the project, there was a lot of debate regarding
GUI concepts and -features and proposals.
- link:GuiBrainstormingReviewed.html[GUI Brainstorming page] 'TODO: explain whats this all about'
- several Proposals
* link:Proposal.AkhiL.html[AkhIL]
* link:Proposal.RichardSpindler.html[Richard Spindler]
* link:Workspaces.html[about Workspace organsiation]
* link:scrolling.html[about scrolling..]
* link:MenuAndShortcuts.html[Ideas for menues and shortcuts]
- link:TimelineDiscussion.html[Conclusions by Joel Holdsworth]
[icon="warning.png"]
WARNING: Website under construction

0
doc/design/gui/GuiBrainstorming/scrolling.txt → doc/design/gui/GuiDiscussion/scrolling.txt
View file

14
doc/design/gui/index.txt
View file

@ -1,6 +1,20 @@
Design Documents: GUI
=====================
In the early stages of the project, there was a lot of debate regarding
GUI concepts and -features and proposals.
link:GuiDiscussion/index.html[here] is a collection of documents from these discussions.
In our discussions, *Workflow* denotes all considerations how certain tasks
can be achieved within the application in the most suitable and stringent fashion.
This effort is closely related to GUI and interface design, but at the same quite
integral, so its rather treated separately of individual GUI features.
-> link:../workflow/index.html[more here...]
.*TODO* more contributions please... ;-)
We had several additional discussions and contributions on the mailinglist,
several people did even concept drawings. Collect these and add them here
[icon="warning.png"]
WARNING: Website under construction

7
doc/design/gui/workflow/index.txt
View file

@ -1,7 +0,0 @@
Design Documents: Workflow
==========================
[icon="warning.png"]
WARNING: Website under construction

21
doc/design/plugins/PluginBrainstorm.txt Normal file
View file

@ -0,0 +1,21 @@
Plugin Brainstorm
=================
:Author: Christian Thäter
:Date: 2008-09-15
Raw Version
-----------
[cpp]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
LUMIERA_INTERFACE_DECLARE (interface_descriptor, 0,
/* The following slots are some human-readable descriptions of certain properties */
LUMIERA_INTERFACE_SLOT (const char*, name, (LumieraInterface)),
LUMIERA_INTERFACE_SLOT (const char*, version, (LumieraInterface)),
LUMIERA_INTERFACE_SLOT (const char*, author, (LumieraInterface)),
LUMIERA_INTERFACE_SLOT (const char*, copyright, (LumieraInterface)),
LUMIERA_INTERFACE_SLOT (const char*, license, (LumieraInterface))
/* TODO add more things here, dependencies, provisions etc */
);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

1
doc/design/plugins/index.txt
View file

@ -1,6 +1,7 @@
Design Documents: Plug-In
=========================
* link:PluginBrainstorm.html[early draft for the plugin interface]
[icon="warning.png"]
WARNING: Website under construction

14
doc/design/workflow/index.txt Normal file
View file

@ -0,0 +1,14 @@
Design Documents: Workflow
==========================
In our discussions, *Workflow* denotes all considerations how certain tasks
can be achieved within the application in the most suitable and stringent fashion.
This effort is closely related to GUI and interface design, but at the same quite
integral, so it's rather treated separately of individual GUI features.
-> see also link:../gui/index.html[general GUI discussion...]
[icon="warning.png"]
WARNING: Website under construction

603
doc/technical/backend/ConfigLoader.txt Normal file
View file

@ -0,0 +1,603 @@
ConfigLoader
============
:Author: ct
:Date: 2008-08-07
link:Lumiera/ConfigLoader[] is a brainstorming about our configuration file loader.
Table of Contents
-----------------
. xref:anchor_api[API]
.. xref:anchor_config[lumiera_configfile]
.. xref:anchor_types[provided TYPES]
.. xref:anchor_syntax[Syntax/Semantics]
* xref:anchor_proposal[Syntax Proposal by cehteh]
.. xref:anchor_links[links]
.. xref:anchor_internals[internals/implementation sketch]
. xref:anchor_discussion[Discussion on IRC]
. xref:anchor_brainstorm[Brainstorming]
anchor:anchor_api[]API
----------------------
* Namespace is lumiera_config_*
* `struct lumiera_config_struct` for the config system, only one object needed
* `lumiera_config_init (...)`
- does only initialize the variables, so that they get valid values, but does not allocate them as they will be allocated before as they are singleton.
- lumiera_config_init (const char* searchpath) searchpath is a buildin-default, can be changed via configure and can be appended and overridden by using a flag, e.g. `--config-path-append=""` or `--config-path=""`
* `lumiera_config_destroy(...)`
- frees all space allocated by the link:ConfigLoader[].
* `lumiera_config_load(...)`
- reads *one* single configuration file that will include all settings from other files.
- does not read itself but give delegates reading. The actual reading and parsing will be done in configfile object. s.later.
* `lumiera_config_save(...)`
- saves all the changed settings to user's configuration files, but recognizes where settings came from and will write them to an appropriate named file. Example: *changed* values from `'/usr/local/share/lumiera/plugins/blur.conf'` will be saved into `'~/.lumiera/plugins/blur.conf'`
- finds out which files are dirty and which settings have to be written to user's config files.
- does initiate the actual saving procedure by delegating the save to the actual configfile objects, see below.
- empty user configuration files in RAM will be deleted from disk on write.
- checks whether the file has changed since last read, and will print out an error if necessary instead of overriding it without notification.
+
`lumiera_config_save () { LLIST_FOREACH(config_singleton.files, f) { LumieraFile file = (LumieraFile) f; if(lumiera_configfile_isdirty (file)) lumiera_configfile_save(file); } }`
* `lumiera_config_purge(const char* filename)` removes all configs loaded from filename * `lumiera_config_get(...)`
- get a value by key
- handles internally everything as string:string key:value pair.
- lowlevel function
- lumiera_config_integer_get (const char* key, int *value) will return integers instead of strings and return 0 if succeeded and -1 if it failed.
- 'int lumiera_config_TYPE_get(const char* key, TYPE* value, const char* default)'
High level config interface for different types.
if default is given (!NULL) then value is set to default in case key was not found or any other error occured.
error code is still set and -1 (fail) is returned in case of an error, but it might be cleared with no ill effects.
NOTE: errors are persistent in our error handler, they must still be cleared, even when ignored.
if default is given then `KEY_NOT_FOUND` is not a error here, if default is NULL then it is
NOTE2: default values are given as strings, the config loader remembers a given default value and checks if it got changed
when it is _set(). Thus a default value can be suppressed when set/written
* `lumiera_config_set(...)`
- set a value by key
- handles internally everything as string:string key:value pair.
- lowlevel function
- tag file as dirty
- set will create a new user configuration file if it does not exist yet or will append a line to the existing one in RAM. These files, tagged as \'dirty', will be only written if save() is called.
- `lumiera_config_TYPE_set (const char* key, TYPE*value, const char* fmt)`
Highlevel interface for different types, fmt is a printf format specifier for the desired format, when NULL, defaults apply.
* `lumiera_config_reset(...)`
- reset a value by key to the system default values, thus removes a user's configuration line.
* `lumiera_config_info (const char* key, const char** filename, unsigned* line)`
- Find exact place of a setting.
[NOTE]
=======================================
* multiple config items with the same key stack (to allow undo/removing custom configs)
* same keys in the same file stack also, the _get api returns the last value the _set api sets the last value
* we might need to implement some cursor api to retrieve all values with the same key, this should allow to filter keys from single files!
* if one queries a setting with a default, this default should be stored in the configuration system without an backing file on disk. Rationale is, that when one later _set's a value and this value is found to be equivalent to the default, this _set option is a no-op
=======================================
.foo.bar
=======================================
`integer_set("foo.bar", 123)`
looks up "foo.bar"
this returns as well file and line
it will be decided whether it's a system file or a user's config file.
if systemfile: loop for user's config file, if necessary create it in our RAM but not yet on disk
set the value in the line.
=======================================
anchor:anchor_config[]lumiera_configfile
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* `lumiera_configfile_struct`
- contains a `llist` with all lines stored as `lumiera_configline_struct`
- contains an integer that indicates how \'dirty' the file is, i.e. how many settings were changed. (one increment per _set() call)
- contains a stat struct for comparison while saving if the file was altered since last read
* `lumiera_configline_struct`
- stores the line exactly as read from file, but holds as well pointers to the key and the value.
* lumiera_configfile_load
* lumiera_configfile_save
* lumiera_configfile_new
* lumiera_configfile_delete
* lumiera_configfile_init
* lumiera_configfile_destroy
anchor:anchor_types[]provided TYPES
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[grid="all"]
`--------------.--------------------------------------'------------------------------------------------------------
config type C type semantic (example)
-------------------------------------------------------------------------------------------------------------------
number signed long long also hex and octal (foo.bar = 12345 #comment)
real long double differerent formats as well (foo.bar=1.2345 #comment)
string const char* optionaly quoted string (foo.bar="this string" #this not)
word const char* first optionaly quoted word of the config (foo.bar= first this # not)
bool int common representations for bools (0,1,true,false,yes,no,on,off,set,clear)
-------------------------------------------------------------------------------------------------------------------
... add more types on demand (ichthyo mentioned a tuple type for color triples etc.)
config files/strings are utf-8!
anchor:anchor_syntax[]Syntax/Semantics
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...to be defined...
in short:
* configfiles allow comments,
* some kind of include directive
* includes happen only \'once', its impossible to create include recursions
* config settings are key:value pairs where the key is a hierachical string as in "foo.bar.baz"
anchor:anchor_proposal[]Syntax Proposal by cehteh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
simple example:
------------------------------------------------------------
@directive options
key = value
[prefix]
[prefix suffix]
------------------------------------------------------------
ebnf:
------------------------------------------------------------
configline ::= [ whitespace ] , { comment | directive | configentry | section } , ? end of line ? ;
whitespace ::= ? one to many tab or blank ? ;
comment ::= "#" , ? arbitary text ? ;
directive ::= "@", ? commandword ? , whitespace, ? arguments ? ;
configentry ::= key, { whitespace } , assignop, value ;
key ::= ? lowercase characters and _ ? , [ "." , key ] ;
assignop ::= " " | ":" | "=" ;
value ::= ? arbitary text ? ;
section ::= "link:, [ whitespace ] , prefix, [ whitespace, suffix ] , [ whitespace ] []" | emptysection ;
emptysection ::= "[]" ;
prefix ::= key ;
suffix ::= key ;
------------------------------------------------------------
semantics notes:
* The format is strictly line-based
* directives inside non empty sections are not yet defined
* sections can't nest so far (idea: thinking about `[ .extend.with.leading.dot ]`)
* section semantics: a key inside a section `[foo bar]` as in `key=value` is treated as `foo.key.bar=value`
anchor:anchor_links[]links
~~~~~~~~~~~~~~~~~~~~~~~~~~
http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html[]
anchor:anchor_internals[]internals/implementation sketch
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* `config` *singleton*
- has a cuckoo hash for lookups
- holds many `configfile` in a linked list
* `configfile` is a `configitem`
- holds `lines` as `section` or `directive` starting with the empty section
- there is one special unnamed (non-disk) file for _defaults_
* `configitem`
- just an abstraction, we prolly dont need an implementation
* `section` is a `line`
- parses prefix/suffix
- holds list of childs of `configline` or `other` lines
* `line` is a `configitem`
- holds the allocated string (char*)
- backpointer to parent item (section or file)
- 2 pointers+length to parsed data, key/value for `configline`, prefix/suffix for `section`
* `configline` is a `line`
- parses key/value as raw strings
- maintains lookup hash in ctor/dtor
* `other` is a `line`
- comment or empty line, just preserved nothing special
* `directive` is a `line`
- must appear in a empty section, else this is an error
anchor:anchor_discussion[]discussion on irc
-------------------------------------------
------------------------------------------------------------
[01:21] <cehteh> joelholdsworth_: you seen the upcoming config interface i planned with simav?
[01:21] <mno> what about using ardours icons to start with?
[01:21] <joelholdsworth_> *least
[01:22] <joelholdsworth_> cehteh: not yeat, can I see
[01:22] <joelholdsworth_> mno: ardour icons arn't that useful
[01:22] <joelholdsworth_> there aren't that many
[01:22] <cehteh> http://www.pipapo.org/pipawiki/Lumiera/ConfigLoader[]
[01:22] <cehteh> question for you: do you need any more 'types'
[01:22] <joelholdsworth_> and those that are there are probably among the gtk stock set anywhere
[01:22] <cehteh> ichthyo suggested color triples
[01:23] <cehteh> anythinfg you have in mind, just suggest it
[01:23] <joelholdsworth_> so is the this structured like an ini file
[01:23] <joelholdsworth_> sections and keys?
[01:23] <joelholdsworth_> or is it treelike?
[01:24] <cehteh> little unordered, you basically just need to care for lumiera_config_TYPE_get ... and *_set
[01:24] <cehteh> like ini, not nested
[01:24] <joelholdsworth_> flat is good
[01:24] <joelholdsworth_> simple
[01:24] <cehteh> the namespaces are nested
[01:24] <cehteh> but the sectioning is not
[01:24] <cehteh> you can have
[01:24] <cehteh> [gui]
[01:24] <__nasa__> Have you all see Agave (for colorscheme generation)?
[01:24] <cehteh> foo = bar
[01:25] <cehteh> [gui.baz]
[01:25] <cehteh> barf= 123
[01:25] <cehteh> its not exactly ini style but close
[01:25] <joelholdsworth_> cehteh: ooh - does mean we can levarge an ini parser?
[01:25] <joelholdsworth_> that would save some work and maintenance effore
[01:25] <cehteh> hey simav want to write the parser himself :P
[01:25] <joelholdsworth_> __nasa__: agave: interesting
[01:26] <joelholdsworth_> cehteh: would he though
[01:26] <cehteh> well i think thats fun to do by ourself :)
[01:26] <joelholdsworth_> it is just like ini
[01:26] <joelholdsworth_> yeah
[01:26] <cehteh> and i plan some special things
[01:26] <joelholdsworth_> but reduction of volume is good
[01:26] <cehteh> like for example setting values from the gui preserves the rest of the data
[01:26] <cehteh> when you have
[01:26] <__nasa__> cehteh: suffixes?
[01:27] <cehteh> [gui.foo bar]
[01:27] <cehteh> dada = 1234 #comment
[01:27] <__nasa__> I just saw the bottom...
[01:27] <cehteh> that is gui.foo.dada.bar = 1234 # comment
[01:27] <cehteh> see suffix
[01:28] <cehteh> more in a moment
[01:28] <joelholdsworth_> still an ini parser would be happy with that
[01:28] <cehteh> what i want to tell to joel
[01:28] <joelholdsworth_> of course you have to add some extra code to the end to make the tree
[01:28] <joelholdsworth_> but that would make our lives simpler
[01:29] <cehteh> when you now have a gui which does gui_config_number_set ("gui.foo.dada.bar", 6666)
[01:29] <cehteh> then
[01:29] <cehteh> [gui.foo bar]
[01:29] <cehteh> dada = 6666 #comment
[01:29] <cehteh> appears in the configfile
[01:29] <joelholdsworth_> did you mean...
[01:29] <cehteh> meaning preserving everything manually set up
[01:29] <joelholdsworth_> [gui.foo.dada]
[01:29] <__nasa__> cehteh: How does it know not to do
[01:29] <joelholdsworth_> bar = 6666 #comment
[01:29] <joelholdsworth_> ?
[01:30] <__nasa__> [gui dada.bar]
[01:30] <__nasa__> foo = 6666 #comment
[01:30] <__nasa__> joelholdsworth_: that works as well :)
[01:31] <cehteh> it just selects best fit sections which already exist
[01:31] <joelholdsworth_> but isn't it supposed to be this way...
[01:31] <cehteh> best fit .. first prefix, then suffix
[01:31] <joelholdsworth_> so [foo.bar] is different from [bar.foo]
[01:31] <__nasa__> Ok.
[01:31] <cehteh> if nothing fits then it places it at a non sectioned part in verbose way
[01:31] <cehteh> joelholdsworth_: yes
[01:31] <__nasa__> (I just wanted to make sure we had a decision on that, as I agree that is the best way)
[01:31] <cehteh> so rationale about sections:
[01:32] <cehteh> rather rarely used but you can do things like
[01:32] <cehteh> [plugin radius]
[01:32] <cehteh> blur = 1
[01:32] <cehteh> sharpen = 2
[01:33] <cehteh> or same for key shortcuts what simav initiall intended
[01:33] <joelholdsworth_> wouldn't it be simpler to have simpler systaxx
[01:34] <joelholdsworth_> and have it always with dots or with spaces
[01:34] <cehteh> well the prefix stuff is completely optinal
[01:34] <joelholdsworth_> that way you can't have 1 section in twice
[01:34] <cehteh> eh?
[01:34] <joelholdsworth_> well is [plugin radius] the same as [plugin.radius] ?
[01:34] <cehteh> no
[01:35] <joelholdsworth_> good
[01:35] <joelholdsworth_> ok
[01:35] <cehteh> [plugin radius] would be suffix prefix
[01:35] <joelholdsworth_> seems overkill to me
[01:35] <cehteh> err opposite
[01:35] <joelholdsworth_> I'd just stick with one or the other
[01:35] <cehteh> <tired
[01:36] <cehteh> [plugin radius] would be prefix suffix
[01:36] <cehteh> [plugin.radius] is just prefix
[01:36] <mno> i'm trying to understand all this hehe
[01:37] <cehteh> the idea is that with just prefix you have to settle with a quite rigid hierachy where you know that things will always be ordered by the leaves
[01:37] <cehteh> that doesnt fit for everyone
[01:38] <joelholdsworth_> "ordered by the leaves" ?
[01:38] <cehteh> one might want to configure his plugins one affter each other
[01:38] <__nasa__> Here is what I think is cool about the setup: it means that we can treat builtins and plugins the same.
[01:38] <cehteh> imagine we have blur and sharben
[01:38] <cehteh> sharpen
[01:38] <__nasa__> [plugin radius]
[01:39] <__nasa__> cinelerra.sharpen = 3
[01:39] <cehteh> both have a radius and a algorihtm. lets say horizontal or vertical
[01:39] <__nasa__> rotate = 2
[01:39] <cehteh> so then you can either first configure blur
[01:39] <cehteh> [plugin.blur]
[01:39] <cehteh> radius = 1
[01:39] <cehteh> algo = horiz
[01:40] <joelholdsworth_> oooh! - have you thought of simply using libconfig?
[01:40] <cehteh> # and then sharpen
[01:40] <cehteh> [plugin.sharpen]
[01:40] <cehteh> radius = 2
[01:40] <cehteh> algo = horiz
[01:40] <cehteh> ..
[01:40] <__nasa__> joelholdsworth_: that format looks a little painful, but it could work.
[01:41] <joelholdsworth_> it's like JSON I think
[01:41] <cehteh> hey so far thats just like ini
[01:41] <cehteh> well now about suffix
[01:41] <joelholdsworth_> ok
[01:41] <cehteh> imagine you want to configure things by 'topics' ..
[01:42] <cehteh> # first define radius for our plugins
[01:42] <cehteh> [plugin radius]
[01:42] <cehteh> blur = 1
[01:42] <cehteh> sharpen = 2
[01:42] <cehteh> # then algo
[01:43] <cehteh> [plugin algo]
[01:43] <joelholdsworth_> yes that's quite clever
[01:43] <cehteh> sharpen = horiz
[01:43] <cehteh> blur = horiz
[01:43] <joelholdsworth_> but still it seems overkill
[01:43] <__nasa__> joelholdsworth_: It does look similar to JSON, but more C-like. For example, there are semi-colons instead of commas at the ends of single values.
[01:43] <cehteh> well its completely optional and we can leave it out
[01:43] <joelholdsworth_> these files are not primarily for humans
[01:44] <cehteh> yeah
[01:44] <joelholdsworth_> and also that prefix thing while clever is quite non obvious when you first see it in the file
[01:44] <__nasa__> Yes, but I am a big proponent of trying to make things human readable
[01:44] <cehteh> git works the same way
[01:44] <joelholdsworth_> sure it should be human readable
[01:44] <cehteh> .git/config
[01:44] <__nasa__> So if anything does go wrong, the user can fix it.
[01:45] <joelholdsworth_> well I'm not sure the prefixes thing is obvious enough
[01:45] <cehteh> well and we will have plenty of values which dont need a gui
[01:45] <cehteh> see about:config for firefox
[01:45] <__nasa__> I think that INI is much easier to read than libconfig, even though I have been programming in C for years and I have never edited a windows INI file.
[01:46] <joelholdsworth_> I think I agree
[01:46] <cehteh> they have all defaults and only when altered they are recorded to a config file but they dont need any special gui
[01:46] <__nasa__> about:config is so nasty.
[01:46] <cehteh> well its mighty
[01:46] <joelholdsworth_> but yes all of about:config could be stored in ini format
[01:46] <joelholdsworth_> [accessibility.typeaheadfind]
[01:46] <cehteh> and anything which is commonly changed should have a preferences gui
[01:46] <joelholdsworth_> linksonly=false
[01:47] <cehteh> btw our configs are not enumerateable
[01:47] <cehteh> i dont want the rest of the system to register there, that will be pita
[01:47] <cehteh> you can freely introduce configs (within your namespace)
[01:48] <-- benG has quit (Remote closed the connection)
[01:48] <cehteh> i'd rather make a grep script which goes through the sources and searched for lumiera_config_.*_get (".*", .*)
[01:49] <joelholdsworth_> yeah that's ok
[01:49] <cehteh> to have some idea what configs are actually used
[01:49] <joelholdsworth_> well maybe we can use libini
[01:49] <cehteh> so a about:config like ui wont be possible
[01:49] <__nasa__> which libini? There are quite a few.
[01:49] <joelholdsworth_> ahh
[01:49] <joelholdsworth_> that's a shame
[01:49] <cehteh> he forget about some libs .. thats so trivial to write by ourself :P
[01:49] <__nasa__> I assume the SF one is the best, as it is GPL.
[01:50] <cehteh> and simav wants to do something :P
[01:50] <joelholdsworth_> yeah
[01:50] <joelholdsworth_> anyway... time for bed for me
[01:50] <cehteh> well and what i rate high is that one can retrieve typed values and that one can add/set values
[01:51] <cehteh> and this setting should not destroy the existing file structure
[01:51] <joelholdsworth_> yeah
[01:51] <cehteh> just read and write values is no magic .. but preserving human structured files is
[01:52] <joelholdsworth_> that is hard, yes
[01:52] <joelholdsworth_> anyway... got to go
[01:52] <__nasa__> Bye!
[01:52] <cehteh> n8 :)
[01:52] <-- joelholdsworth_ has quit ("Ex-Chat")
[01:52] <__nasa__> It looks like libini uses [plugin/blur] instead, but that is an easy fix.
[01:52] <cehteh> url?
[01:52] <__nasa__> Or we just let simav do the whole thing
[01:53] <cehteh> well i dont know how much time he has
[01:53] <__nasa__> http://sourceforge.net/project/showfiles.php?group_id=25464[]
[01:54] <cehteh> last change 3 year ago :P
[01:56] * cehteh looks at the source
[01:57] <cehteh> fails standard C++ conformance :P
[01:58] <cehteh> and no @include like directive
[01:58] * __nasa__ thinks we should look elsewhere...
[01:58] * cehteh things we should just do ourself :P
[01:58] * __nasa__ agrees.
[01:59] <cehteh> the syntax i typed there is not very special and can be replaced
[01:59] <cehteh> but some other features are important
[01:59] * __nasa__ thinks it should PRE. That way we don't have to totally agree.
[01:59] <cehteh> pre?
[01:59] <__nasa__> (On syntax)
[01:59] <cehteh> no suffix?
[02:00] <cehteh> well as saied its just an idea and completely optional, we can leave it out
[02:00] <__nasa__> sorry -- PCRE.
[02:00] <cehteh> ah
[02:00] <cehteh> i dont see how PCRE helps for a syntax
[02:00] <__nasa__> I always think of it as "Perl Regular Expressions" and forget the "Compatible"
[02:01] <cehteh> anyways: features matter
[02:01] <__nasa__> I was thinking in terms of plugins/blend vs. plugins.blend.
[02:01] <cehteh> that is @include which follows some LUMIERA_CONFIG_PATH
[02:02] <__nasa__> We should have some way to only include certain sections.
[02:02] <__nasa__> Like have the default config be
[02:02] <cehteh> yes
[02:02] <__nasa__> [lumiera.core]
[02:02] <__nasa__> ...
[02:02] <__nasa__> then
[02:02] <cehteh> but thats setup by the administrator/user
[02:02] <__nasa__> @include<plugin> ~/.lumi/plugins/
[02:02] <cehteh> if the user changes lumiera.core.somevalue
[02:03] <cehteh> it will find the file where it is already defined
[02:03] <__nasa__> and it adds all preferences for plugins from <dir>
[02:03] <cehteh> then check if thats a file in the users config space (~/.lumiera/*)
[02:03] <cehteh> and edit it there
[02:03] <__nasa__> Ok.
[02:04] <cehteh> if it is a system wide file it will place a correspondending file in the users config space
[02:04] <cehteh> if the value is not configured already it will search for [lumiera.core] and then [lumiera] and then []
[02:04] <cehteh> best fit the section where it would apply
[02:05] <cehteh> and then do the same write or create into correspondending user config
[02:05] <cehteh> thats why i saied 'features matter'
[02:06] <__nasa__> There should be a callback for a plugin to add a config file for that plugin.
[02:06] <__nasa__> Ooh
[02:06] <__nasa__> cool idea
[02:06] <cehteh> this is not really complicated to do but would give a big usage experience link:JustWorks[][TM]
[02:06] <__nasa__> ~/.lumi/config is the main configuration directory
[02:06] <__nasa__> inside, there is a ~/.lumi/config/config.ini which is main configuration
[02:07] <cehteh> no callbacks .. the idea is that the config system is independent and nothing else need to register there, that makes it much easier
[02:07] <__nasa__> Then, for example accessing plugin.shade.
[02:07] <cehteh> if you install a plugin this plugin should come with a system config file
[02:07] <cehteh> that can be a empty one
[02:07] <__nasa__> either find it in ~/.lumi/config/config.ini or ~/.lumi/config/plugin/config.ini.
[02:08] <cehteh> /usr/local/share/lumiera/plugin/shade
[02:08] <cehteh> containing just
[02:08] <cehteh> [plugin.shade]
[02:08] <cehteh> when the user edits some value
[02:09] <cehteh> plugin.shade.transparency = 209
[02:09] <cehteh> then this will automatically create a correspondending ~/.lumiera/plugin/shade in the user config space
[02:09] <__nasa__> yeah, so what I am saying is that instead of [plugin.shade] only [] is needed
[02:10] <cehteh> using the system one as template
[02:10] <cehteh> well if its []
[02:10] <__nasa__> since the file is already in config/PLUGIN/SHADE
[02:10] <cehteh> or that
[02:10] <cehteh> this is not yet decided
[02:10] <cehteh> we can do that too
[02:10] <__nasa__> so automatically add [plugin.shade] to the prefix of files in config/plugin/shade, etc
[02:11] <cehteh> but i would be careful to tie filenames and internal namespaces together
[02:11] <cehteh> yes doable
[02:11] <cehteh> well should still have [plugin.shade]
[02:12] <cehteh> else we have a nested section mess and files loose their identity
[02:12] <__nasa__> Yeah, I just realized it would be a pain to have default configs
[02:12] <__nasa__> Since where does the file go? you have to look inside to see if it had [plugin.shade], tec
[02:12] <__nasa__> etc*
[02:12] <cehteh> anyways nothing is etched in stone yet, we are just brainstorming
[02:13] <cehteh> most important thing is the interface the applciation can use
[02:13] <cehteh> then we can make a mockup for that which only returns defaults
[02:13] <cehteh> then we can use it already even when the config loader isnt finished
[02:14] <__nasa__> We could have a warning if a file in e.g. config/plugin/blur had a section that was [plugin.shade] or something
[02:14] <cehteh> actually the inital plan was to bootstrap the config system by one single site config file
[02:14] <__nasa__> to prevent hackery
[02:14] <cehteh> but plugins are a good point .. they might have their own configs
[02:15] <cehteh> which is not @included by the site config
[02:15] <__nasa__> lumiera_config_t *plugin_shade.request_default_config() or something
[02:15] <cehteh> but the plugin itself loads it when the plugin is used
[02:15] <__nasa__> Yeah
[02:16] <cehteh> nah
[02:16] <cehteh> lumiera_config is just singleton
[02:16] <cehteh> after you loaded plugin.shade it becomes visible there
[02:16] <cehteh> thats way easier
[02:16] <__nasa__> that is lumiera_config_struct
[02:16] <__nasa__> ok
[02:16] <cehteh> not even the struct
[02:17] <cehteh> thats an implementation detail
[02:17] <__nasa__> yeah -- to deal with later
[02:17] <cehteh> the programmers use only lumiera_coonfig_TYPE_get or _set
[02:17] <cehteh> where TYPE is to be defined .. see my notes
[02:18] <cehteh> number real string word ... and whatever else we need
[02:19] <cehteh> so you get typed data back .. when you provide a default then you can be even sure that it is always valid
[02:19] <__nasa__> Yeah, I do like that structure.
[02:19] <__nasa__> It also makes mass configuration setup very easy.
[02:21] <cehteh> btw file must have identity ...
[02:21] <__nasa__> meaning?
[02:21] <__nasa__> like they are not all config.ini?
[02:22] <cehteh> i mean a config file has to define for what kinds of values it applies
[02:22] <cehteh> [plugin.shade] for the plugin/shade file
[02:22] <__nasa__> Ok, I see.
[02:22] <cehteh> then changing a values there will create the correspondending user file
[02:23] <cehteh> *thinking*
[02:23] <cehteh> the user could write [plugin.shade]
[02:23] <cehteh> into some other config file which then becomes target for that
[02:23] <cehteh> (best fit)
[02:24] <cehteh> just wondering if the user just writes [pluin]
[02:24] <cehteh> [plugin]
[02:24] <cehteh> well i am sure we solve that
[02:25] <__nasa__> [plugin]
[02:26] <__nasa__> shade.radius = 3
[02:26] <__nasa__> in like the plugin/blur folder?
[02:28] <cehteh> no in the users config somewher
[02:28] <__nasa__> oh
[02:28] <cehteh> ~/.lumiera/myconfig
[02:29] <cehteh> shall it pick and mirror /usr/share/lumiera/plugin/blur to the user config space
[02:29] <cehteh> or write it to the [plugin] section in myconfig
[02:30] <cehteh> well maybe we add a [plugin*] syntax which hints the best fit
[02:30] <__nasa__> I would think it makes more sense to add a plugin/blur folder
[02:30] <__nasa__> or else lumiera/myconfig would be huge after configuring one project ;-)
[02:30] <cehteh> yes .. thats what you think .. i just wondering how to give the user the control to fit it to his needs
[02:31] <cehteh> ~/.lumiera/all_my_pluginmyconfig
[02:31] <cehteh> [plugin*]
[02:31] <cehteh> hints the best fit algo
[02:32] <cehteh> well just thinking about optional things
[02:32] <__nasa__> I think I am confused -- how is [plugin*] different from [plugin]
[02:33] <cehteh> when you change plugin.blur.radius then the best fit algo will search the section which covers it best
[02:33] <cehteh> that would be [plugin.blur] in /usr/share/lumiera/plugin/blur
[02:33] <__nasa__> Oh, unless you have [plugin*]
[02:34] <cehteh> so it will create a correspondending ~/.lumiera/plugin/blur file for you
[02:34] <cehteh> yes exactly
[02:34] <__nasa__> Ok, that makes sense. Good idea!
[02:34] <cehteh> [plugin*] will take precedence
[02:34] <cehteh> when you have that then you can hint it not to create many tiny files
[02:35] <__nasa__> so that way the config is in one file
[02:35] <__nasa__> yeah
[02:35] <cehteh> hehe the suffux thing comes in mind here
[02:35] <cehteh> [plugin* radius] in ~/.lumiera/plugin_radius_conf
[02:35] <__nasa__> I was thinking suffixed are only allowed at the e.g. plugin/ level if they effect multiple plugins.
[02:35] <cehteh> well or as trivial as
[02:36] <__nasa__> Ok.
[02:36] <cehteh> [plugin* enable] in ~/.lumiera/plugins_enabled
[02:36] <__nasa__> Fun!
[02:36] <cehteh> blur = on
[02:36] <cehteh> sharpen = off
[02:36] <cehteh> ah .. we need a bool TYPE
[02:36] <cehteh> *blink*
[02:37] <__nasa__> on/off or true/false?
[02:37] <__nasa__> Well, either way.
[02:38] <cehteh> bool
[02:38] <cehteh>
[02:38] <cehteh> int
[02:38] <cehteh>
[02:38] <cehteh> common representations for bools (0,1,yes,no,on,off,set,clear)
[02:38] <cehteh> we can add more
[02:38] <__nasa__> It should be reasonably good at guessing
[02:40] <cehteh> yeah .. i would like to make this little intelligent
[02:40] <cehteh> for example when you want to set something it might look at the existing thing before doing so and then flavor the set with that
[02:41] <cehteh> when a string was quoted, the new one will be quoted to
[02:41] <__nasa__> "Weaken the blur effect"
[02:41] <cehteh> if there was a bool with 'yes' then it will choose 'no' ..
[02:41] <cehteh> and so on
[02:42] <cehteh> this are generally just a few extra lines of code, nothing dramatic .. but that makes it much nicer in detail
[02:43] <__nasa__> Yeah, I think that it will improve the user experience
[02:43] <cehteh> ok i add this irclog to the wiki
[02:43] <__nasa__> for most people, editing a config file is scary
[02:43] <__nasa__> so making it as easy and friendly as possible is good
[02:44] <__nasa__> All right
------------------------------------------------------------
anchor:anchor_brainstorm[]Brainstorming
---------------------------------------
* support for successive dereferencing foo.bar > baz.barf > blah.boo = final value
* support for alternatives, query a set of keys, return the first existing
* provide a @readonly directive which marks a file to be handled readonly (even if writing would be permitted)

2
doc/technical/backend/index.txt
View file

@ -1,6 +1,8 @@
Technical Documentation: Backend
================================
* link:ConfigLoader.html[Config Loader brainstorming from 2008]
[icon="warning.png"]
WARNING: Website under construction