[grid="all"]
`------------`-----------------------
*State*         _Final_
*Date*          _2008-03-06_
*Proposed by*   link:Ichthyostega[]
-------------------------------------


Placement Metaphor used within the high-level view of Steam-Layer
-----------------------------------------------------------------
besides the [wiki:self:../ProcBuilder Builder], one of the core ideas of the
``Proc-Layer'' (as being currently implemented by Ichthyo) is to utilize
_Placement_ as a single central metaphor for object association, location and
configuration within the high-level model. The intention is to prefer _rules_
over fixed _values._ Instead of ``having'' a property for this and that, we
query for information when it is needed.

The proposed use of *Placement* within the steam layer spans several,
closely related ideas:

 * use the placement as a universal means to stick the "media objects" together
   and put them on some location in the timeline, with the consequence of a
   unified and simplified processing.
 * recognize that various _location-like_ degrees of freedom actually form a
   single _``configuration space''_ with multiple (more than 3) dimensions.
 * distinguish between _properties_ of an object and qualities, which are
   caused by ``placing'' or ``locating'' the object in _configuration space_
   - _propetries_ belong to the object, like the blur value, the media source
       file, the sampling/frame rate of a source
   - _location qualities_ exist only because the object is ``at'' a given
       location in the graph or space, most notably the start time, the output
       connection, the layering order, the stereoscopic window depth, the sound
       pan position, the MIDI instrument
 * introduce a _way of placement_ independent of properties and location
   qualities, describing if the placement _itself_ is _absolute, relative or
   even derived_
 * open especially the possibility to _derive_ parts of the placement from
   the context by searching over connected objects and then up the fork (``tree of tracks'');
   this includes the possibility of having rules for resolving unspecified
   qualities.



Description
~~~~~~~~~~~
The basic idea is to locate _Media Objects_ of various kinds within a
_configuration space_. Any object can have a lot of different qualities,
which may partially depend on one another, and partially may be chosen freely.
All these various choices are considered as _degrees of freedom_ -- and
defining a property can be seen as _placing_ the object to a specific
parameter value on one of these dimensions. While this view may be bewildering
at first sight, the important observation is that in many cases we don't want
to lock down any of those parameters completely to one fixed value. Rather, we
just want to _limit_ some parameters.

To give an example, most editing applications let the user place a video clip
at a fixed time and track. They do so by just assigning fixed values, where the
track number determines the output and the layering order. While this may seem
simple, sound and pragmatic, indeed this puts down way to much information in a
much to rigid manner for many common use cases of editing media. More often
than not it's not necessary to ``nail down'' a video clip -- rather, the user
wants it to start immediately after the end of another clip, it should be sent
to some generic output and it should stay in the layering order above some
other clip. But, as the editing system fails to provide the means for
expressing such relationships, we are forced to work with hard values, resort
to a bunch of macro features or even compensate for this lack by investing
additional resources in production organisation (the latter is especially true
for building up a movie sound track).

On the contrary, using the *Placement* metaphor has the implication of
switching to a query-driven approach.

 * it gives us one single instrument to express the various kinds of relations
 * the _kind of placement_ becomes an internal value of the _placement_ (as
   opposed to the object)
 * some kinds of placement can express rule-like relations in a natural fashion
 * while there remains only one single mechanism for treating a bunch of
   features in a unified manner
 * plugins could provide exotic and advanced kinds of placement, without the
   need of massively reworking the core.

When interpreting the high-level model and creating the low-level model,
Placements need to be _resolved_, resulting in a simplified and completely
nailed-down copy of the session contents, which this design calls »the
_Fixture_«

Media Objects can be placed

 * fixed at a given time
 * relative to some reference point given by another object (clip, label,
   timeline origin)
 * as plugged into a specific output pipe (destination port)
 * as attached directly to another media object
 * to a fixed layer number
 * layered above or below another reference object
 * fixed to a given pan position in virtual sound space
 * panned relative to the pan position of another object




Tasks
^^^^^
 * currently just the simple standard case is drafted in code.
 * the mechanism for treating placements within the builder is drafted in code,
   but needs to be worked out to see the implications more clearly
 * while this design opens endless possibilities, it is not clear how much of
   it should be visible through the GUI





Discussion
~~~~~~~~~~

Pros
^^^^
 * with just one concept, we get a lot of issues right, which many conventional
   approaches fail to solve satisfactory
 * one grippy metaphor instead of several special treatments
 * includes the simple standard case
 * unified treatment
 * modular and extensible
 * allows much more elaborate handling of media objects then the conventional
   approach, while both the simple standard case and the elaborate special case
   are "first class citizens" and completely integrated in all object
   treatment.



Cons
^^^^
 * difficult to grasp, breaks with some habits
 * requires a separate resolution step
 * requires to ''query'' for object properties instead of just looking up a
   fixed value
 * forces the GUI to invent means for handling object placement which may go
   beyond the conventional
 * can create quite some surprises for the user, especially if he doesn't care
   to understand the concept up front



Alternatives
^^^^^^^^^^^^
Use the conventional approach

 * media objects are assigned with fixed time positions
 * they are stored directly within a grid (or tree) of tracks
 * layering and pan are hard wired additional properties
 * implement an additional auto-link macro facility to attach sound to video
 * implement a magnetic snap-to for attaching clips seamless after each other
 * implement a splicing/sliding/shuffling mode in the UI
 * provide a output wiring tool in the GUI
 * provide macro features for this and that....

^hopefully I made clear by now _why_ I don't want to take the conventional approach^



Rationale
~~~~~~~~~
There is a conventional way of dealing with those properties, which stems of
the use of analogue hardware, especially multitrack tape machines. This
conventional approach constantly creates practical problems, which could be
avoided by using the placement concept. This is due to the fact, that the
placement concept follows the natural relations of the involved concepts, while
the conventional approach was dictated by technological limitations.

 * the ususal layering based on tracks constantly forces the user to place
   clips in a unnatural and unrelated fashion and tear apart clips which belong
   closely together
 * the conventional approach of having a fixed "pan control" in specialized
   "audio tracks" constantly hinders the development of more natural and
   convincing sound mixing. It favors a single sound system (intensity based
   stereophony) for no good reason.
 * handling of stereoscopic (3D) video/film is notoriously difficult within the
   conventional, hard wired approach
 * building more elaborate sound scapes and sound design is notoriously
   difficult to maintain, because the user is forced to use hidden "side
   chains", magic rules and re-build details in external applications, because
   of the lack of flexible integration of control data alongside with the main
   data.

The high-level model is close to the problem domain, it should provide means to
express the (naturally complex) relationships between media objects. Using an
abstract and unified concept is always better then having a bunch of seemingly
unrelated features, even if they may be more easy to grasp for beginners.
Moreover, the Placement concept deliberately brings in an rule based approach,
which well fits into the problem domain. Finally, there is sort-of a visionary
aspect involved here: _Ichthyo_ thinks that nowadays, after image and sound are
no longer bound to physical media, there is potential for new workflows to be
discovered, and the Placement concept could be an extension point for such
undertakings.






Comments
--------


Placement Metaphor
~~~~~~~~~~~~~~~~~~
Re:
``Finally, there is sort-of a visionary aspect involved here:
Ichthyo thinks that nowadays, after image and sound are no longer bound to
physical media, there is potential for _new workflows_ to be
_discovered_,  and the _Placement concept_ could be an
extension point for such undertakings.''

New workflows will not just be _discovered_, but they will be able to be
_recorded, analysed, templated, automated, and integrated_ into the full
workflow process. This will free up a greater proportion of time for the
»finishing« processes of projects.

``The Placement concept _could be_ an extension for such undertakings'' is very
likely to be an understatement as it is this which *will be* what
makes these undertakings possible, because it enables the gathering,
use, and decision rules based on these parameters.

This feature/capability is likely to stamp the Lumiera project as a flagship
benchmark in more ways than one, for some time.

 . --link:Tree[][[DateTime(2008-08-23T12:54:00NZ)]].


Final
~~~~~
    Do 14 Apr 2011 03:06:42 CEST Christian Thaeter



Back to link:/documentation/devel/rfc.html[Lumiera Design Process overview]
