Ichthyostega
b556d2b770
Some sections of the Lumiera website document meeting minutes, discussion protocols and design proposals from the early days of the project; these pages were initially authored in the »Moin Moin Wiki« operated by Cehteh on pipapo.org at that time; this wiki backed the first publications of the »Cinelerra-3« initiative, which turned into the Lumiera project eventually. Some years later, those pages were transliterated into Asciidoc semi-automatically, resulting in a lot of broken markup and links. This is a long standing maintenance problem problem plaguing the Lumiera website, since those breakages cause a lot of warnings and flood the logs of any linkchecker run.
242 lines
10 KiB
Text
242 lines
10 KiB
Text
Design Process : Placement Metaphor
|
|
====================================
|
|
|
|
[options="autowidth"]
|
|
|====================================
|
|
|*State* | _Final_
|
|
|*Date* | _2008-03-06_
|
|
|*Proposed by* | Ichthyostega
|
|
|====================================
|
|
|
|
|
|
Placement Metaphor used within the high-level view of Steam-Layer
|
|
-----------------------------------------------------------------
|
|
|
|
besides the link:{rfc}/ProcBuilder.html[Builder], one of the core ideas of
|
|
the Lumiera application, as envisioned and implemented currently, is to utilise
|
|
_Placement_ as a single central metaphor for object association, location and
|
|
configuration within the high-level model. The intention is to prefer _rules_
|
|
over fixed _values._ Instead of ``having'' a property for this and that, we
|
|
query for information when it is needed.
|
|
|
|
The proposed use of *Placement* within the steam layer spans several,
|
|
closely related ideas:
|
|
|
|
* using the placement as a universal means to stick the ``media objects'' together
|
|
and put them on some location in the timeline, with the consequence of a
|
|
unified and simplified processing.
|
|
* recognise that various _location-like_ degrees of freedom actually form a
|
|
single _``configuration space''_ with multiple (more than 3) dimensions.
|
|
* distinguish between _properties_ of an object and qualities, which are
|
|
caused by ``placing'' or ``locating'' the object in _configuration space_
|
|
- _properties_ belong to the object, like the blur value, the media source
|
|
file, the sampling/frame rate of a source
|
|
- _location qualities_ exist only because the object is ``at'' a given
|
|
location in the graph or space, most notably the start time, the output
|
|
connection, the layering order, the stereoscopic window depth, the sound
|
|
pan position, the MIDI instrument
|
|
* introduce a _way of placement_ independent of properties and location
|
|
qualities, describing if the Placement _itself_ is _absolute, relative or
|
|
even derived_
|
|
* open especially the possibility to _derive_ parts of the Placement from
|
|
the context by searching over connected objects and then up the _Fork_ (``tree of tracks'');
|
|
this includes the possibility of having rules for resolving unspecified
|
|
qualities.
|
|
|
|
|
|
|
|
Description
|
|
~~~~~~~~~~~
|
|
The basic idea is to locate _Media Objects_ of various kinds within a
|
|
_configuration space_. Any object can have a lot of different qualities,
|
|
which may partially depend on one another, and partially may be chosen freely.
|
|
All these various choices are considered as _degrees of freedom_ -- and
|
|
defining a property can be seen as _placing_ the object to a specific
|
|
parameter value on one of these dimensions. While this view may be bewildering
|
|
at first sight, the important observation is that in many cases we don't want
|
|
to lock down any of those parameters completely to one fixed value. Rather, we
|
|
just want to _limit_ some parameters.
|
|
|
|
To give an example, most editing applications let the user place a video clip
|
|
at a fixed time and track. They do so by just assigning fixed values, where the
|
|
track number determines the output and the layering order. While this may seem
|
|
simple, sound and pragmatic, indeed this puts down way to much information in a
|
|
much to rigid manner for many common use cases of editing media. More often
|
|
than not it's not necessary to ``nail down'' a video clip -- rather, the user
|
|
wants it to start immediately after the end of another clip, it should be sent
|
|
to some generic output and it should stay in the layering order above some
|
|
other clip. But, as the editing system fails to provide the means for
|
|
expressing such relationships, we are forced to work with hard values, resort
|
|
to a bunch of macro features or even compensate for this lack by investing
|
|
additional resources in production organisation (the latter is especially true
|
|
for building up a movie sound track).
|
|
|
|
On the contrary, using the *Placement* metaphor has the implication of
|
|
switching to a query-driven approach.
|
|
|
|
* it gives us one single instrument to express the various kinds of relations
|
|
* the _kind of placement_ becomes an internal value of the _Placement_ (as
|
|
opposed to the object)
|
|
* some kinds of placement can express rule-like relations in a natural fashion
|
|
* while there remains only one single mechanism for treating a bunch of
|
|
features in a unified manner
|
|
* plugins could provide exotic and advanced kinds of placement, without the
|
|
need of massively reworking the core.
|
|
|
|
When interpreting the high-level model and creating the low-level model,
|
|
Placements need to be _resolved_, resulting in a simplified and completely
|
|
nailed-down copy of the session contents, which this design calls »the
|
|
_Fixture_«
|
|
|
|
Media Objects can be placed
|
|
|
|
* fixed at a given time
|
|
* relative to some reference point given by another object (clip, label,
|
|
timeline origin)
|
|
* as plugged into a specific output pipe (destination port)
|
|
* as attached directly to another media object
|
|
* to a fixed layer number
|
|
* layered above or below another reference object
|
|
* fixed to a given pan position in virtual sound space
|
|
* panned relative to the pan position of another object
|
|
|
|
|
|
|
|
|
|
Tasks
|
|
^^^^^
|
|
* currently just the simple standard case is drafted in code.
|
|
* the mechanism for treating placements within the builder is drafted in code,
|
|
but needs to be worked out to see the implications more clearly
|
|
* while this design opens endless possibilities, it is not clear how much of
|
|
it should be visible through the GUI
|
|
|
|
|
|
|
|
|
|
|
|
Discussion
|
|
~~~~~~~~~~
|
|
|
|
Pros
|
|
^^^^
|
|
* with just one concept, we get a lot of issues right, which many conventional
|
|
approaches fail to solve satisfactory
|
|
* one grippy metaphor instead of several special treatments
|
|
* includes the simple standard case
|
|
* unified treatment
|
|
* modular and extensible
|
|
* allows much more elaborate handling of media objects then the conventional
|
|
approach, while both the simple standard case and the elaborate special case
|
|
are ``first class citizens'' and completely integrated in all object
|
|
treatment.
|
|
|
|
|
|
|
|
Cons
|
|
^^^^
|
|
* difficult to grasp, breaks with some habits
|
|
* requires a separate resolution step
|
|
* requires to _query_ for object properties instead of just looking up a
|
|
fixed value
|
|
* forces the GUI to invent means for handling object placement which may go
|
|
beyond the conventional
|
|
* can create quite some surprises for the user, especially if he doesn't care
|
|
to understand the concept up front
|
|
|
|
|
|
|
|
Alternatives
|
|
^^^^^^^^^^^^
|
|
Use the conventional approach
|
|
|
|
* media objects are assigned with fixed time positions
|
|
* they are stored directly within a grid (or tree) of tracks
|
|
* layering and pan are hard wired additional properties
|
|
* implement an additional auto-link macro facility to attach sound to a video clip
|
|
* implement a magnetic snap-to for attaching clips seamless after each other
|
|
* implement a splicing/sliding/shuffling mode in the UI
|
|
* provide a output wiring tool in the GUI
|
|
* provide macro features for this and that.... +
|
|
^(hopefully I made clear by now _why_ I don't want to take the conventional approach)^
|
|
|
|
|
|
|
|
Rationale
|
|
~~~~~~~~~
|
|
There is a conventional way of dealing with those properties, which stems of
|
|
the use of analogue hardware, especially multitrack tape machines. This
|
|
conventional approach constantly creates practical problems, which could be
|
|
avoided by using the placement concept. This is due to the fact, that the
|
|
placement concept follows the natural relations of the involved concepts, while
|
|
the conventional approach was dictated by technological limitations.
|
|
|
|
* the ususal layering based on tracks constantly forces the user to place
|
|
clips in a unnatural and unrelated fashion and tear apart clips which belong
|
|
closely together
|
|
* the conventional approach of having a fixed ``pan control'' in specialized
|
|
``audio tracks'' constantly hinders the development of more natural and
|
|
convincing sound mixing. It favors a single sound system (intensity based
|
|
stereophony) for no good reason.
|
|
* handling of stereoscopic (3D) video/film is notoriously difficult within the
|
|
conventional, hard wired approach
|
|
* building more elaborate soundscapes and sound design is notoriously
|
|
difficult to maintain, because the user is forced to use hidden
|
|
``side chains'', magic rules and re-build details in external applications,
|
|
because of the lack of flexible integration of control data alongside
|
|
with the main data.
|
|
|
|
The high-level model is close to the problem domain, it should provide means to
|
|
express the (naturally complex) relationships between media objects. Using an
|
|
abstract and unified concept is always better then having a bunch of seemingly
|
|
unrelated features, even if they may be more easy to grasp for beginners.
|
|
Moreover, the Placement concept deliberately leads to a _rule based approach_,
|
|
which well fits into the problem domain. Finally, there is sort-of a visionary
|
|
aspect involved here: _Ichthyo_ thinks that nowadays, after image and sound are
|
|
no longer bound to physical media, there is potential for new workflows to be
|
|
discovered, and the Placement concept could be an extension point for such
|
|
undertakings.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Comments
|
|
--------
|
|
|
|
|
|
Placement Metaphor
|
|
~~~~~~~~~~~~~~~~~~
|
|
[quote]
|
|
``Finally, there is sort-of a visionary aspect involved here:
|
|
Ichthyo thinks that nowadays, after image and sound are no longer bound to
|
|
physical media, there is potential for _new workflows_ to be
|
|
_discovered_, and the _Placement concept_ could be an
|
|
extension point for such undertakings.''
|
|
|
|
New workflows will not just be _discovered_, but they will be able to be
|
|
_recorded, analysed, templated, automated, and integrated_ into the full
|
|
workflow process. This will free up a greater proportion of time for the
|
|
»finishing« processes of projects.
|
|
|
|
``The Placement concept _could be_ an extension for such undertakings'' is very
|
|
likely to be an understatement as it is this which *will be* what
|
|
makes these undertakings possible, because it enables the gathering,
|
|
use, and decision rules based on these parameters.
|
|
|
|
This feature/capability is likely to stamp the Lumiera project as a flagship
|
|
benchmark in more ways than one, for some time.
|
|
|
|
Tree:: '2008-08-23T12:54:00NZ'
|
|
|
|
|
|
Final
|
|
~~~~~
|
|
Discussed at the link:{ldoc}/devel/meeting_summary/2011-04-13.html#_proc_high_level_model_and_placement_concept[Apr.2011 developer meeting]. +
|
|
|
|
Christian Thaeter:: 'Thu 14 Apr 2011 03:06:42 CEST'
|
|
|
|
|
|
''''
|
|
Back to link:/x/DesignProcess.html[Lumiera Design Process overview]
|