more detailed planning of architecture for clip presentation
...first UML diagram created with Umbrello!
This commit is contained in:
parent
25328865fa
commit
3757a56ac9
4 changed files with 1698 additions and 18 deletions
BIN
doc/devel/uml/Timeline-clip-display.png
Normal file
BIN
doc/devel/uml/Timeline-clip-display.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 7.9 KiB |
1587
uml/Lumiera.xmi
1587
uml/Lumiera.xmi
File diff suppressed because it is too large
Load diff
|
|
@ -1432,6 +1432,17 @@ Lumiera agrees to this common understanding (of most film editing and sound hand
|
|||
→ PipeHandling
|
||||
</pre>
|
||||
</div>
|
||||
<div title="ClipPresenter" creator="Ichthyostega" modifier="Ichthyostega" created="201611200101" modified="201611200204" tags="def GuiPattern img" changecount="7">
|
||||
<pre>//mediating entity used to guide and control the presentation of a clip in the UI.//
|
||||
|
||||
The clip representation in the UI links together two distinct realms and systems of concerns. For one, there are the properties and actions performed on the clip for sake of editing a movie. Everything which has a tangible effect on the resulting render. These information and operations are embodied into the HighLevelModel and can be manipulated script driven, without relying on an UI. But beyond that, there is also the local mechanics of the interface, everything which makes working on the edit and interacting with the model into a smooth experience and workflow. With respect to this concerns, a clip is a self-contained entity which owns its own state and behaviour. The ClipPresenter is the pivotal element to link together those two realms.
|
||||
|
||||
[>img[Clip presentation control|uml/Timeline-clip-display.png]]
|
||||
Regarding the global angle, the ClipPresenter is an UI-Element connected to the UI-Bus, and it is added as child element to some parent entity, typically the UI representation of a [[fork ("track")|Fork]]. Commands manipulating the clip can be sent via the embedded bus terminal, and status regarding clip properties and nested child elements (effects, transitions) is received as messages via the bus, which insofar plays the role of model and controller.
|
||||
|
||||
But regarding the local UI behaviour, the ClipPresenter acts autonomous. It controls an actual {{{ClipWidget}}} for presentation, negotiating the display strategy with some overarching presentation manager. Active parts of the widget are wired back to handling methods of the presenter.
|
||||
</pre>
|
||||
</div>
|
||||
<div title="ColorPalette" modifier="Ichthyostega" created="200706190033" modified="200807131329" tags="excludeMissing">
|
||||
<pre>Background: #fefefd
|
||||
Foreground: #000
|
||||
|
|
@ -2441,7 +2452,7 @@ we need a test setup for this investigation.
|
|||
* realistic: shall reflect the situation in our actual UI
|
||||
</pre>
|
||||
</div>
|
||||
<div title="GuiClipWidget" creator="Ichthyostega" modifier="Ichthyostega" created="201611180114" modified="201611191841" tags="GuiPattern design draft" changecount="15">
|
||||
<div title="GuiClipWidget" creator="Ichthyostega" modifier="Ichthyostega" created="201611180114" modified="201611200138" tags="GuiPattern design draft" changecount="17">
|
||||
<pre>//The representation of a [[media clip|Clip]] for manipulation by the user within the UI.//
|
||||
Within Lumiera, a clip is conceived as a //chunk of media,// which can be handled in compound. Clip as such is an abstract concept, which is treated with minimal assumptions...
|
||||
* we know that a clip has //media content,// which need not be uniform and can be inherently structured (e.g. several media, several channels)
|
||||
|
|
@ -2470,11 +2481,22 @@ This is unfortunate, since it defeats a self-contained control structure, where
|
|||
|
||||
This creates a tension related to the kind of architecture we want to build with the help of the UI-Bus. The intention is to split and separate between the application global concerns, like issuing an editing command or retrieving changed model contents, and the //local UI mechanics.// This separation can only work, if -- with respect to the global concerns -- the UI-Bus embodies (actually mediates) the roles of model //and// controler, while the UI widgets take on the role of a "mere view". While, for the local concerns, the same widgets are to act self-contained. But now, already the existence of aforementioned widgets becomes dependent from another, obviously local concern, which unfortunately cross-cuts the 1:1 relation between model entities and view counterpart.
|
||||
|
||||
Starting from the architecture as predetermined by the UI-Bus, it is clear that we have to add a UI-Element for every tangible child entity represented in the (abstracted) model. And the parent entity receiving a diff and adding such a child element is also in charge of that element, controlling its lifecycle. Yet this child element is not necessarily bound to be a widget. This is our chance here: what "adding" means is completely within the scope of the parent element and basically an implementation detail. So the parent may create and own a tracker or controller object to stand for the clip, and the latter may create and adjust the actual widget in negotiation with a display strategy.
|
||||
Starting from the architecture as predetermined by the UI-Bus, it is clear that we have to add a UI-Element for every tangible child entity represented in the (abstracted) model. And the parent entity receiving a diff and adding such a child element is also in charge of that element, controlling its lifecycle. Yet this child element is not necessarily bound to be a widget. This is our chance here: what "adding" means is completely within the scope of the parent element and basically an implementation detail. So the parent may create and own a tracker or controller object to stand for the clip, and the latter may create and adjust the actual widget in negotiation with a display strategy. So effectively we're introducing a //mediator,// which represents the clip (as far as the modelling is concerned) and which at the same time communicates with a global display manager. We call this mediating entity a ClipPresenter
|
||||
|
||||
!!!how to carry out the clip appearances
|
||||
Initially, one could think that we'd need to build several widgets to realise the wide variety of clip appearances. But in fact it turns out that we're able to reshape a single base widget to encompass all the necessary presentation styles. This base widget is a simple, one-element container, with {{{Gtk::Frame}}} being the most obvious pick. This gives us already a rectangular covered space, and the ability to add a label widget, optionally with controlled alignment of the label. All the more elaborate presentation styles can be achieved by adding a canvas widget into this frame and then placing additional stuff on top of that.
|
||||
Initially, one could think that we'd need to build several widgets to realise the wide variety of clip appearances. But in fact it turns out that we're able to reshape a single base widget to encompass all the necessary presentation styles. This base widget is a simple, one-element container, with {{{Gtk::Frame}}} being the most obvious pick. This gives us already a rectangular covered space, and the ability to add a label widget, optionally with controlled alignment of the label. All the more elaborate presentation styles can be achieved by adding a canvas widget into this frame and then placing additional stuff on top of that. The only tricky part arises in overview display, when just some clip rectangle can stand-in for a whole series of clips, which themselves remain hidden as UI elements.
|
||||
|
||||
A prime consideration regarding this whole clip presentation strategy is the performance concern. It is quite common for movie edits to encompass several hundred individual clips. Combined with several tracks and an elaborate audio edit, it may well happen that we end up with thousands of individual UI objects. If treated naively, this load might seriously degrade the responsiveness of the interface. Thus we need to care for the relevant infrastructure to enable optimisation of the display. For that reason, the memory footprint of the ClipPresenter and the basic widget has to be kept as small as possible. And moreover, since we do our own layout management in the timeline display, in theory it is possible any time just to add those widgets to the enclosing GTK container, which are actually about to become visible. (if we follow this approach, a problem yet to be solved is how to remove widgets falling out of sight, since removing N widgets easily turns into a quadratic operation).
|
||||
|
||||
!clip content rendering
|
||||
In a typical editing application, the user can expect to get some visual clue regarding the media content of a clip. For example, sound clips can be visualised as waveform, while movie clips might feature a sequence of images taken from the video. Our intention is to ''use our own rendering engine'' to produce these thumbnails. In fact, our engine is perfectly suited for this task: it has precisely the necessary media decoding and rendering abilities, plus it offers an elaborate system of priorities and deadlines, allowing to throttle the load produced by thumbnail generation. In addition to all those qualities, our engine is planned to be complemented by an "intelligent" frame cache, which, given proper parametrisation, ensures the frequently requested thumbnails will be available for quick display. For this approach to work, we need to provide some infrastructure
|
||||
* we need to configure and maintain a //preview rendering strategy.//
|
||||
* the request for rendering has to be cast "just somewhere" as message, obviously via the UI-Bus
|
||||
* actually rendered content will likewise arrive asynchronously as message via UI-Bus.
|
||||
* we still need to work out how buffer management for this task will be handled; it should be a derivative of typical buffer management for display rendering.
|
||||
* the clip widget needs to provide a simple placeholder drawing to mark the respective space in the interface, until the actual preview arrives.
|
||||
|
||||
To start with, mostly this means to avoid a naive approach, like having code in the UI to pull in some graphics from media files. We certainly won't just render every media channel blindly. Rather, we acknowledge that we'll have a //strategy,// depending on the media content and some further parameters of the clip. This might well just be a single ''pivot image'' chosen explicitly by the editor to represent a given take. And the actual implementation of content preview rendering will largely be postponed until we get our rendering engine into a roughly working state.
|
||||
</pre>
|
||||
</div>
|
||||
<div title="GuiCommandBinding" creator="Ichthyostega" modifier="Ichthyostega" created="201511272246" modified="201609091641" tags="design decision operational GuiPattern GuiIntegration draft discuss" changecount="26">
|
||||
|
|
|
|||
|
|
@ -99,8 +99,7 @@
|
|||
</node>
|
||||
<node CREATED="1476376943985" HGAP="22" ID="ID_1422206856" MODIFIED="1479434895083" TEXT="Viewer" VSHIFT="10"/>
|
||||
<node CREATED="1479434763643" HGAP="25" ID="ID_1572413636" MODIFIED="1479434887744" TEXT="Clip" VSHIFT="31">
|
||||
<node CREATED="1479434774138" ID="ID_912862507" MODIFIED="1479442348920" TEXT="Grundlagen">
|
||||
<icon BUILTIN="pencil"/>
|
||||
<node CREATED="1479434774138" ID="ID_912862507" MODIFIED="1479601717907" TEXT="Grundlagen">
|
||||
<node CREATED="1479434780105" ID="ID_239720378" MODIFIED="1479434829990" TEXT="apperances">
|
||||
<richcontent TYPE="NOTE"><html>
|
||||
<head>
|
||||
|
|
@ -117,19 +116,21 @@
|
|||
Verwende das als Leitgedanke, um das Layout zu entwickeln
|
||||
</p>
|
||||
</body>
|
||||
</html>
|
||||
</richcontent>
|
||||
</html></richcontent>
|
||||
<icon BUILTIN="idea"/>
|
||||
<node CREATED="1479434831825" ID="ID_975727645" MODIFIED="1479434836781" TEXT="abridged form">
|
||||
<node CREATED="1479434845992" ID="ID_609815513" MODIFIED="1479434860330" TEXT="media bin, collapsed fork"/>
|
||||
<node CREATED="1479441548335" ID="ID_150470698" MODIFIED="1479441564712" TEXT="icon + ID label"/>
|
||||
<node CREATED="1479601640961" ID="ID_191706335" MODIFIED="1479601645229" TEXT="Gtk::Frame"/>
|
||||
</node>
|
||||
<node CREATED="1479441503085" ID="ID_386209482" MODIFIED="1479441506119" TEXT="compact form">
|
||||
<node CREATED="1479441530689" ID="ID_1057853337" MODIFIED="1479441539364" TEXT="timeline default"/>
|
||||
<node CREATED="1479441567268" ID="ID_1413366977" MODIFIED="1479441582078" TEXT="rendered content strip + decorations"/>
|
||||
<node CREATED="1479601649615" ID="ID_417588278" MODIFIED="1479601662578" TEXT="Gtk::Frame + Gtk::Layout"/>
|
||||
</node>
|
||||
<node CREATED="1479441519834" ID="ID_1669486876" MODIFIED="1479441522374" TEXT="expanded form">
|
||||
<node CREATED="1479441586057" ID="ID_64386373" MODIFIED="1479441597593" TEXT="window with embedded sub-clips"/>
|
||||
<node CREATED="1479601667734" ID="ID_1877644742" MODIFIED="1479601683095" TEXT="dto, with recursive embedded sub-clips"/>
|
||||
</node>
|
||||
</node>
|
||||
<node CREATED="1479442296443" ID="ID_911884433" MODIFIED="1479442304498" TEXT="graceful display degradation">
|
||||
|
|
@ -138,6 +139,95 @@
|
|||
<node CREATED="1479442322359" ID="ID_492785140" MODIFIED="1479442337761" TEXT="Fall2: in diesem Pixel ist content"/>
|
||||
</node>
|
||||
</node>
|
||||
<node CREATED="1479601690675" ID="ID_542607961" MODIFIED="1479601720226" TEXT="Struktur">
|
||||
<icon BUILTIN="pencil"/>
|
||||
<node CREATED="1479601699057" ID="ID_1130364446" MODIFIED="1479601707854" TEXT="Konflikt mit der UI-Bus-Architektur">
|
||||
<icon BUILTIN="messagebox_warning"/>
|
||||
<node CREATED="1479601736116" ID="ID_891600328" MODIFIED="1479601748486" TEXT="Bus == Rolle des Controler/Model"/>
|
||||
<node CREATED="1479601749946" ID="ID_887287956" MODIFIED="1479601757981" TEXT="Widget == View-Rolle"/>
|
||||
</node>
|
||||
<node CREATED="1479601765896" ID="ID_1632476975" MODIFIED="1479601784408">
|
||||
<richcontent TYPE="NODE"><html>
|
||||
<head>
|
||||
|
||||
</head>
|
||||
<body>
|
||||
<p>
|
||||
UI-Bus gilt nur für <b>globale Belange</b>
|
||||
</p>
|
||||
</body>
|
||||
</html>
|
||||
</richcontent>
|
||||
<node CREATED="1479601788237" ID="ID_1871715779" MODIFIED="1479601795259" TEXT="wichtige neue Unterscheidung">
|
||||
<icon BUILTIN="idea"/>
|
||||
</node>
|
||||
<node CREATED="1479601796075" ID="ID_1177606400" MODIFIED="1479601807006" TEXT="Global == in der Session aufgezeichnet"/>
|
||||
<node CREATED="1479601807666" ID="ID_1374495669" MODIFIED="1479601812781" TEXT="vs. lokale UI-Mechanik"/>
|
||||
<node CREATED="1479601828799" ID="ID_1797519191" MODIFIED="1479601837377" TEXT="Ziel: Subsidiarität für lokale Belange"/>
|
||||
</node>
|
||||
<node CREATED="1479601884288" HGAP="34" ID="ID_1371609613" MODIFIED="1479602442561" TEXT="Lösungs-Idee" VSHIFT="18">
|
||||
<icon BUILTIN="idea"/>
|
||||
<node CREATED="1479601889567" ID="ID_1123973572" MODIFIED="1479601899475">
|
||||
<richcontent TYPE="NODE"><html>
|
||||
<head>
|
||||
|
||||
</head>
|
||||
<body>
|
||||
<p>
|
||||
es geht nur um <i>Rollen</i>
|
||||
</p>
|
||||
</body>
|
||||
</html>
|
||||
</richcontent>
|
||||
</node>
|
||||
<node CREATED="1479601907221" ID="ID_286625648" MODIFIED="1479601914101" TEXT="es ist ein Design-Pattern">
|
||||
<icon BUILTIN="yes"/>
|
||||
</node>
|
||||
<node CREATED="1479601925938" ID="ID_22286545" MODIFIED="1479601944896">
|
||||
<richcontent TYPE="NODE"><html>
|
||||
<head>
|
||||
|
||||
</head>
|
||||
<body>
|
||||
<p>
|
||||
das lokale Element muß nur als View <i>fungieren</i>
|
||||
</p>
|
||||
</body>
|
||||
</html>
|
||||
</richcontent>
|
||||
</node>
|
||||
<node CREATED="1479601945839" ID="ID_254470029" MODIFIED="1479601954903" TEXT="Model-View-Presenter">
|
||||
<icon BUILTIN="forward"/>
|
||||
</node>
|
||||
<node CREATED="1479602306718" ID="ID_359464754" MODIFIED="1479602311385" TEXT="ClipPresenter">
|
||||
<node CREATED="1479602311740" ID="ID_963961994" MODIFIED="1479602314256" TEXT="Mittelsmann"/>
|
||||
<node CREATED="1479602314676" ID="ID_944757170" MODIFIED="1479602318896" TEXT="steht für den Clip"/>
|
||||
<node CREATED="1479602319555" ID="ID_1557047586" MODIFIED="1479602327854" TEXT="steuert ein ClipWidget"/>
|
||||
<node CREATED="1479602338609" ID="ID_444253818" MODIFIED="1479602340884" TEXT="ClipWidget">
|
||||
<node CREATED="1479602341777" ID="ID_1610807382" MODIFIED="1479602381582" TEXT="empfängt DisplayStrategy"/>
|
||||
<node CREATED="1479602382195" ID="ID_1504834509" MODIFIED="1479602405257">
|
||||
<richcontent TYPE="NODE"><html>
|
||||
<head>
|
||||
|
||||
</head>
|
||||
<body>
|
||||
<p>
|
||||
kann sich selbst
|
||||
</p>
|
||||
<p>
|
||||
transformieren
|
||||
</p>
|
||||
</body>
|
||||
</html>
|
||||
</richcontent>
|
||||
<node CREATED="1479602412095" ID="ID_832808487" MODIFIED="1479602415154" TEXT="Frame"/>
|
||||
<node CREATED="1479602415654" ID="ID_1358720" MODIFIED="1479602418642" TEXT="Frame + Layout"/>
|
||||
<node CREATED="1479602419134" ID="ID_1788706818" MODIFIED="1479602425033" TEXT="sub-Clips im Layout"/>
|
||||
</node>
|
||||
</node>
|
||||
</node>
|
||||
</node>
|
||||
</node>
|
||||
</node>
|
||||
<node CREATED="1476376927660" HGAP="35" ID="ID_688318446" MODIFIED="1479434903774" TEXT="Docks" VSHIFT="12"/>
|
||||
<node CREATED="1477342616175" HGAP="37" ID="ID_954058801" MODIFIED="1477342623660" TEXT="Workspace" VSHIFT="33">
|
||||
|
|
@ -9681,8 +9771,7 @@
|
|||
Kind-Widgets noch gar nicht festgelegt war (denn das passiert erst beim draw).
|
||||
</p>
|
||||
</body>
|
||||
</html>
|
||||
</richcontent>
|
||||
</html></richcontent>
|
||||
<icon BUILTIN="ksmiletris"/>
|
||||
</node>
|
||||
<node CREATED="1477703741232" ID="ID_942790225" MODIFIED="1477703768906" TEXT="get_children returns copy">
|
||||
|
|
|
|||
Loading…
Reference in a new issue