Design: decide on the conventions for representing a timeline in the GUI
Actually this is nothing new, just making explicit what is evident from the definition of the Proc-Layer model entities. By following these conventions, it should be possible to come up with a clearer structure for the custom timeline widget(s).
This commit is contained in:
parent
994a0e718b
commit
718d544d8e
1 changed files with 40 additions and 9 deletions
|
|
@ -2285,6 +2285,9 @@ Probably the most important aspect regarding the GUI integration is how to get [
|
|||
Access point for the lower layers to push information and state changes (aynchronously) to the GUI. Actually, most operations within Lumiera are initiated by the user through the GUI. In the course of such actions, the GUI uses the services of the lower layer and typically recieves an synchronous response. In some exceptional cases, these operations may cause additional changes to happen asynchronously from the GUI's perspective. For example, an edit operation might trigger a re-build of the low-level model, which then detects an error.
|
||||
</pre>
|
||||
</div>
|
||||
<div title="GuiPattern" creator="Ichthyostega" modifier="Ichthyostega" created="201410160054" tags="overview" changecount="1">
|
||||
<pre>While the HighLevelModel is self-contained and forms an autonomous »Universe«, the Lumiera GUI uses a well defined set of Metaphors, structural patterns and conventions to represent the user's view upon the model within the session. </pre>
|
||||
</div>
|
||||
<div title="GuiStart" modifier="Ichthyostega" created="200812050525" modified="200902080720" tags="GuiIntegration">
|
||||
<pre>Starting up the GUI is optional and is considered part of the Application start/stop and lifecycle.
|
||||
* main and AppState activate the lifecyle methods on the ~GuiSubsysDescriptor, accessible via the GuiFacade
|
||||
|
|
@ -2310,11 +2313,39 @@ Now, when invoking an operation on some public interface, the code in the lower
|
|||
<pre>A specially configured LumieraPlugin, which actually contains or loads the complete code of the (GTK)GUI, and additionally is linked dynamically against the application core lib. During the [[UI startup process|GuiStart]], loading of this Plugin is triggered from {{{main()}}}. Actually this causes spawning of the GTK event thread and execution of the GTK main loop.
|
||||
</pre>
|
||||
</div>
|
||||
<div title="HighLevelModel" modifier="Ichthyostega" created="200808152311" modified="201003210021" tags="Model spec design discuss img">
|
||||
<div title="GuiTimelineView" creator="Ichthyostega" modifier="Ichthyostega" created="201410160100" modified="201410160235" tags="GuiPattern design decision draft" changecount="4">
|
||||
<pre>Within the Lumieara GUI, the [[Timeline]] structure(s) from the HighLevelModel are arranged and presented according to the following principles and conventions.
|
||||
Several timeline views may be present at the same time -- and there is not necessarily a relation between them, since »a Timeline« is the top-level concept within the [[Session]]. Obviously, there can also be several //views// based on the same »Timeline« model element, and in this latter case, these view behave according to a linked common state. An entity »Timeline« as represented through the GUI, emerges from the combination of several model elements
|
||||
* a root level [[Binding|BindingMO]] acts as framework
|
||||
* this binding in turn ties a [[Sequence]]
|
||||
* and the sequence provides a [[tree of Tracks|Track]]
|
||||
* within the scope of these tracks, there is content
|
||||
* and this content implies output designations
|
||||
* which are resolved to the [[global Pipes|GlobalPipe]] belonging to //this specific Timeline.//
|
||||
* after establishing a ViewerPlayConnection, a running PlayProcess exposes a PlayController
|
||||
Session, Binding and Sequence are the mandatory ingredients.
|
||||
|
||||
!Basic layout
|
||||
The representation is split into a ''Header pane'' exposing structure and configuration, and a ''Content pane'' extending in time. A ''Time ruler'' running alongside the content pane represents the //position in time.// Beyond this temporal dimension, the content area is conceived as a flexible working space. This working space //can// be structured hierarchically -- when interacting with the GUI, hierarchical nesting will be created and collapsed on demand. Contrast this with conventional editing applications which are built upon the rigid notion of "Tracks": Lumiera is rather based on //Pipes// than Tracks.
|
||||
In the temporal dimension, there is the usual scrolling and zooming of content, and possibly a selected time range, and after establishing a ViewerPlayConnection, there is an effective playback location featured as a "Playhead"
|
||||
The workspace dimension (vertical layout) is more like a ''Fork'', which can be expanded recursively. More specifically, each strip or layer or "track" can be featured in //collapsed// or //expanded state.//
|
||||
* the collapsed state features a condensed representation ("the tip of the iceberg"). It exposes just the topmost entity, and might show a rendered (pre)view. Elements might be stacked on top, but any element visible here //is accessible.//
|
||||
* when expanding, the content unfolds into...
|
||||
** a ''scope ruler'' to represent the whole sub-scope. This is rendered as a small pane, holding any locally attached labels and track-wide or temporally scoped effects
|
||||
** the content stack, comprised of media, attached effects and transitions
|
||||
** a stack of nested sub-scopes (recursive).
|
||||
This collapsed, expanded and possibly nested workspace structure is always exactly paralleled in the header pane.
|
||||
|
||||
!!!nesting
|
||||
By principle, this workspace structure is //not a list of "Tracks"// -- it is a system of ''nested scopes''. The nesting emerges on demand.
|
||||
In the most general case, there can be pr-track content and nested content at the same point in time. The GUI is able to represent this state. But, due to the semantics of Lumiera's HighLevelModel, top-level content and nested content are siblings //within the same scope.// Thus, at a suitable point {{red{to be defined}}}, an equivalence transformation is applied to the GUI model, by prepending a new sibling track and moving top-level content there.
|
||||
</pre>
|
||||
</div>
|
||||
<div title="HighLevelModel" modifier="Ichthyostega" created="200808152311" modified="201410160108" tags="Model spec design discuss img" changecount="1">
|
||||
<pre>While the low-level model holds the data used for carrying out the actual media data processing (=rendering), the high-level model is what the user works upon when performing edit operations through the GUI (or script driven in &raquo;headless mode&laquo;). Its building blocks and combination rules determine largely what structures can be created within the [[Session]].
|
||||
On the whole, it is a collection of [[media objects|MObjects]] stuck together and arranged by [[placements|Placement]].
|
||||
|
||||
Basically, the structure of the high-level model is is a very open and flexible one &mdash; every valid connection of the underlying object types is allowed &mdash; but the transformation into a low-level node network for rendering follows certain patterns and only takes into account any objects reachable while processing the session data in accordance to these patterns. Taking into account the parameters and the structure of these objects visited when building, the low-level render node network is configured in detail.
|
||||
Basically, the structure of the high-level model is is a very open and flexible one &mdash; every valid connection of the underlying object types is allowed &mdash; but the transformation into a low-level node network for rendering follows certain patterns and only takes into account any objects reachable while processing the session data in accordance to these patterns. Taking into account the parameters and the structure of these objects visited when building, the low-level render node network is configured in detail. In a similar vein, the [[representation within the GUI|GuiPattern]] is based on distinct patterns and conventions -- any object not in line with these conventions remains //hidden// and is //silently ignored.//
|
||||
|
||||
The fundamental metaphor or structural pattern is to create processing ''pipes'', which are a linear chain of data processing modules, starting from an source port and providing an exit point. [[Pipes|Pipe]] are a //concept or pattern,// they don't exist as objects. Each pipe has an input side and an output side and is in itself something like a Bus treating a single [[media stream|StreamType]] (but this stream may still have an internal structure, e.g. several channels related to a spatial audio system). Other processing entities like effects and transitions can be placed (attached) at the pipe, resulting them to be appended to form this chain. Optionally, there may be a ''wiring plug'', requesting the exit point to be connected to another pipe. When omitted, the wiring will be figured out automatically.
|
||||
Thus, when making an connection //to// a pipe, output data will be sent to the //source port// (input side) of the pipe, wheras when making a connection //from// a pipe, data from it's exit point will be routed to the destination. Incidentally, the low-level model and the render engine employ //pull-based processing,// but this is rather of no relevance for the high-level model.
|
||||
|
|
@ -4569,11 +4600,11 @@ Besides, they provide an __inward interface__ for the [[ProcNode]]s, enabling th
|
|||
|
||||
</pre>
|
||||
</div>
|
||||
<div title="ProcLayer and Engine" modifier="Ichthyostega" created="200706190056" modified="201402162030" tags="overview" changecount="2">
|
||||
<div title="ProcLayer and Engine" modifier="Ichthyostega" created="200706190056" modified="201410160050" tags="overview" changecount="3">
|
||||
<pre>The Render Engine is the part of the application doing the actual video calculations. Relying on system level services and retrieving raw audio and video data through [[Lumiera's Backend|Backend]], its operations are guided by the objects and parameters edited by the user in [[the session|Session]]. The middle layer of the Lumiera architecture, known as the Proc-Layer, spans the area between these two exteremes, providing the the (abstract) edit operations available to the user, the representation of [["editable things"|MObjects]] and the translation of those into structures and facilities allowing to [[drive the rendering|Rendering]].
|
||||
|
||||
!About this wiki page
|
||||
|background-color:#e3f3f1;width:96ex;padding:2ex; This TiddlyWiki is the central location for design, planning and documentation of the Lumiera Proc-Layer. Some parts are used as //extended brain// &mdash; collecting ideas, considerations and conclusions &mdash; while other tiddlers contain the decisions and document the planned or implemented facilities. The intention is to move over the more mature parts into the emerging technical documentation section on the [[Lumiera website|http://www.lumiera.org]] eventually. <br/><br/>Besides cross-references, content is largely organised through [[Tags|TabTags]], most notably <br/><<tag overview>> &middot; <<tag def>> &middot; <<tag decision>> &middot; <<tag Concepts>> <br/> <<tag Model>> &middot; <<tag SessionLogic>> &middot; <<tag GuiIntegration>> &middot; <<tag Builder>> &middot; <<tag Rendering>> &middot; <<tag Player>> &middot; <<tag Rules>> &middot; <<tag Types>> |
|
||||
|background-color:#e3f3f1;width:96ex;padding:2ex; This TiddlyWiki is the central location for design, planning and documentation of the Lumiera Proc-Layer. Some parts are used as //extended brain// &mdash; collecting ideas, considerations and conclusions &mdash; while other tiddlers contain the decisions and document the planned or implemented facilities. The intention is to move over the more mature parts into the emerging technical documentation section on the [[Lumiera website|http://www.lumiera.org]] eventually. <br/><br/>Besides cross-references, content is largely organised through [[Tags|TabTags]], most notably <br/><<tag overview>> &middot; <<tag def>> &middot; <<tag decision>> &middot; <<tag Concepts>> &middot; <<tag GuiPattern>> <br/> <<tag Model>> &middot; <<tag SessionLogic>> &middot; <<tag GuiIntegration>> &middot; <<tag Builder>> &middot; <<tag Rendering>> &middot; <<tag Player>> &middot; <<tag Rules>> &middot; <<tag Types>> |
|
||||
|
||||
!~Proc-Layer Summary
|
||||
When editing, the user operates several kinds of //things,// organized as [[assets|Asset]] in the AssetManager, like media, clips, effects, codecs, configuration templates. Within the context of the [[Project or Session|Session]], we can use these as &raquo;[[Media Objects|MObjects]]&laquo; &mdash; especially, we can [[place|Placement]] them in various kinds within the session and relative to one another.
|
||||
|
|
@ -7433,8 +7464,8 @@ Currently (1/11), the strategy is implemented according to (1) and (4) above, le
|
|||
Implementation of this strategy is still broken: it doesn't work properly when actually the change passing over the zero point happens by propagation from lower digits. Because then -- given the way the mutators are implemented -- the //new value of the wrapping digit hasn't been stored.// It seems the only sensible solution is to change the definition of the functors, so that any value will be changed by side-effect {{red{Question 4/11 -- isn't this done and fixed by now??}}}
|
||||
</pre>
|
||||
</div>
|
||||
<div title="Timeline" modifier="Ichthyostega" created="200706250721" modified="201105221835" tags="def">
|
||||
<pre>Timeline is the top level element within the [[Session (Project)|Session]]. It is visible within a //timeline view// in the GUI and represents the effective (resulting) arrangement of media objects, to be rendered for output or viewed in a Monitor (viewer window). A timeline is comprised of:
|
||||
<div title="Timeline" modifier="Ichthyostega" created="200706250721" modified="201410160111" tags="def" changecount="2">
|
||||
<pre>Timeline is the top level element within the [[Session (Project)|Session]]. It is visible within a [[timeline view in the GUI|GuiTimelineView]] and represents the effective (resulting) arrangement of media objects, to be rendered for output or viewed in a Monitor (viewer window). A timeline is comprised of:
|
||||
* a time axis in abolute time ({{red{WIP 1/10}}}: not clear if this is an entity or just a conceptual definition)
|
||||
* a list of [[global Pipes|GlobalPipe]] representing the possible outputs (master busses)
|
||||
* //exactly one// top-level [[Sequence]], which in turn may contain further nested Sequences.
|
||||
|
|
@ -7443,7 +7474,7 @@ Implementation of this strategy is still broken: it doesn't work properly when a
|
|||
Please note especially that following this design //a timeline doesn't define tracks.// [[Tracks form a Tree|Track]] and are part of the individual sequences, together with the media objects placed to these tracks. Thus sequences are independent entities which may exist stand-alone within the model, while a timeline is //always bound to hold a sequence.// &rarr; see ModelDependencies
|
||||
[>img[Fundamental object relations used in the session|uml/fig136453.png]]
|
||||
|
||||
Within the Project, there may be ''multiple timelines'', to be viewed and rendered independently. But, being the top-level entities, multiple timelines may not be combined further. You can always just render (or view) one specific timeline. A given sequence may be referred directly or indirectly from multiple timelines though.
|
||||
Within the Project, there may be ''multiple timelines'', to be viewed and rendered independently. But, being the top-level entities, multiple timelines may not be combined further. You can always just render (or view) one specific timeline. A given sequence may be referred directly or indirectly from multiple timelines though. A given timeline is represented within the GUI according to [[distinct principles and conventions|GuiTimelineView]]
|
||||
|
||||
''Note'': in early drafts of the design (2007) there was an entity called "Timeline" within the [[Fixture]]. This entity seems superfluous and has been dropped. It never got any relevance in existing code and at most was used in some code comments.
|
||||
|
||||
|
|
@ -7453,7 +7484,7 @@ Actually, Timeline is both an interface and acts as façade. Its an interface, b
|
|||
Besides building on the asset management, implementing Timeline (and Sequence) as StructAsset yields another benefit: ~StructAssets can be retrieved by query, allowing to specify more details of the configuration immediately on creation. //But on the short term, this approach causes problems:// there is no real inference engine integrated into Lumiera yet (as of 2/2010 the plan is to get an early alpha working end to end first). For now we're bound to use the {{{fake-configrules}}} and to rely on a hard wired simulation of the intended behaviour of a real query resolution. Just some special magic queries will work for now, but that's enough to get ahead.
|
||||
</pre>
|
||||
</div>
|
||||
<div title="TimelineSequences" modifier="Ichthyostega" created="200811011836" modified="201003160202" tags="design decision img">
|
||||
<div title="TimelineSequences" modifier="Ichthyostega" created="200811011836" modified="201410160057" tags="design decision img" changecount="2">
|
||||
<pre>There is a three-level hierarchy: [[Project|Session]], [[Timeline]], [[Sequence]]. Each project can contain ''multiple timelines'', to be viewed and rendered independently. But, being the top-level entities, these timelines may not be combined further. You can always just render (or view) one specific timeline. Each of those timelines refers to a Sequence, which is a bunch of [[media objects|MObject]] placed to a tree of [[tracks|Track]]. Of course it is possible to use ~sub-sequences within the top-level sequence within a timeline to organize a movie into several scenes or chapters.
|
||||
|
||||
[>img[Relation of Timelines, Sequences and MObjects within the Project|uml/fig132741.png]]
|
||||
|
|
@ -7464,7 +7495,7 @@ This is because the top-level entities (Timelines) are not permitted to be combi
|
|||
The Timeline owns a list of global [[pipes (busses)|Pipe]] which are used to collect output. If the track tree of a sequence doesn't contain specific routing advice, then connections will be done directly to these global pipes in order and by matching StreamType (i.e. typically video to video master, audio to stereo audio master). When a monitor (viewer window) is attached to this timeline, similar output connections are made from those global pipes, i.e. the video display will take the contents of the first video (master) bus, and the first stereo audio pipe will be pulled and sent to system audio out. The timeline owns a ''play control'' shared by all attached viewers and coordinating the rendering-for-viewing. Similarly, a render task may be attached to the timeline to pull the pipes needed for a given kind of generated output. The actual implementation of the play controller and the coordination of render tasks is located in the Backend, which uses the service of the Proc-Layer to pull the respective exit nodes of the render engine network.
|
||||
|
||||
!Timeline versus Timeline View
|
||||
Actually, what the GUI creates and uses is the //view// of a given timeline. This makes no difference to start with, as the view is modelled to be a sub-concept of "timeline" and thus can stand-in. All different views of the //same// timeline also share one single play control instance, i.e. they all have one single playhead position. Doing it this way should be the default, because it's the least confusing. Anyway, it's also possible to create multiple //independent timelines// &mdash; in an extreme case even so when referring to the same top-level sequence. This configuration gives the ability to play the same arrangement in parallel with multiple independent play controllers (and thus independent playhead positions)
|
||||
Actually, what the [[GUI creates and uses|GuiTimelineView]] is the //view// of a given timeline. This makes no difference to start with, as the view is modelled to be a sub-concept of "timeline" and thus can stand-in. All different views of the //same// timeline also share one single play control instance, i.e. they all have one single playhead position. Doing it this way should be the default, because it's the least confusing. Anyway, it's also possible to create multiple //independent timelines// &mdash; in an extreme case even so when referring to the same top-level sequence. This configuration gives the ability to play the same arrangement in parallel with multiple independent play controllers (and thus independent playhead positions)
|
||||
|
||||
To complement this possibilities, I'd propose to give the //timeline view// the possibility to be re-linked to a sub-sequence. This way, it would stay connected to the main play control, but at the same time show a sub-sequence //in the way it will be treated as embedded// within the top-level sequence. This would be the default operation mode when a meta-clip is opened (and showed in a separate tab with such a linked timeline view). The reason for this proposed handling is again to give the user the least surprising behaviour. Because, when &mdash; on the contrary &mdash; the sub-sequence would be opened as //separate timeline,// a different absolute time position and a different signal routing may result; doing such should be reserved for advanced use, e.g. when multiple editors cooperate on a single project and a sequence has to be prepared in isolation prior to being integrated in the global sequence (featuring the whole movie).
|
||||
</pre>
|
||||
|
|
|
|||
Loading…
Reference in a new issue