From e219dcbb8cffc2a4c642c36249c517fc7f657049 Mon Sep 17 00:00:00 2001 From: Ichthyostega Date: Wed, 29 Jul 2026 20:48:38 +0200 Subject: [PATCH] Workflow: revised the discussion summary I went through the entire digest extracted by Claude, revalidated the central statements, reworded most summaries to capture the meaning adequately and added quotes from the original mails verbatim at several instances. I consider this version now ready for review. --- .../workflow/Verwijlen/Discussion25Digest.txt | 1521 ++++++++++------- .../workflow/Verwijlen/WorkflowProposals.txt | 3 +- 2 files changed, 945 insertions(+), 579 deletions(-) diff --git a/doc/design/workflow/Verwijlen/Discussion25Digest.txt b/doc/design/workflow/Verwijlen/Discussion25Digest.txt index 53884194c..dc6bcdb6f 100644 --- a/doc/design/workflow/Verwijlen/Discussion25Digest.txt +++ b/doc/design/workflow/Verwijlen/Discussion25Digest.txt @@ -1,8 +1,9 @@ Workflow Discussion 2025 -- a Digest ==================================== :Date: 2026-07-27 -:Author: Compilation: Hermann Voßeler +:Author: Hermann Voßeler (Compilation) :toc: +:toclevels: 3 //MENU: label Discussion 2025 @@ -10,29 +11,39 @@ Workflow Discussion 2025 -- a Digest // eMail archive, the IRC log from 2025-04-16, the FrOSCon-25 meeting summary and // the »Lumiera Workflow Proposals« document -- to be reviewed and reworked. -************************************************************************ -This page condenses an extended design discussion, conducted throughout -2025 (and continuing), between *Wouter Verwijlen* -- documentary filmmaker -from Utrecht -- and *Hermann Voßeler* (_Ichthyo_) -- lead developer of -Lumiera. The goal of this digest is to preserve the findings, agreements -and open questions of that discussion in a form that remains readable -and useful long-term. -************************************************************************ + +[abstract] +This page is based on the extended design discussion, conducted throughout +2025 (and continuing), between *Wouter Verwijlen* and *Hermann Voßeler* (_Ichthyo_). +The goal of this discussion is to establish the foundations of a _Workflow Concept_ +for the Lumiera video editing application. A digest and an summary is presented here, +based on the original eMail messages, IRC logs and published documents, +and was compiled to present the outcomes of the exchange, the insights, +agreements and outstanding issues identified during this discussion in +a format that will remain accessible in the long term. + +CAUTION: Knowledge-Technology (»AI«) was used to identify topics of the discussion + and to create an initial clustering of quotes and arrangement of arguments. + These were then rearranged, reweighed and reformulated by recourse to + the original messages and IRC logs. + Overview -------- -The basis of this discussion is the link:WorkflowProposals.html[»Lumiera Workflow Proposals«] -document, in which Wouter analyses how contemporary editing applications (Avid Media Composer, -Adobe Premiere, Final Cut Pro, DaVinci Resolve, Lightworks, ...) handle the central tasks of -film editing -- and develops proposals how this handling might be improved. This document grew -in parallel to the discussion and was treated by both participants as _part_ of the discussion: -the proposals it contains are contributions by Wouter, and several of them were picked up, -challenged and refined in the exchange documented here. +The starting point and anchor for this discussion is the +link:WorkflowProposals.html[»Lumiera Workflow Proposals«] document contributed by _Wouter_, +to analyse how contemporary editing applications handle the central tasks of film editing -- +leading to proposals how this handling might be improved. This document grew +in parallel to the discussion and was treated by both participants as _part of the discussion:_ +the proposals it contains are contributions by Wouter, based on his yearlong experience as +a documentary filmmaker -- and several of them were picked up, challenged and refined in +the ensuing exchange of thoughts and insights, that is documented here. The discussion itself was conducted mostly through long eMail threads, complemented by -IRC chat sessions and video calls (only partially recorded), and by an in-person meeting -at link:FrosconMeeting.html[FrOSCon 2025] in St.Augustin, where Benny Lyons also joined. +IRC chat sessions (only partially recorded), and by an in-person +link:FrosconMeeting.html[meeting at FrOSCon-2025] in St.Augustin, +where Benny Lyons also participated. .Chronology [options="header",cols="2,3,5"] @@ -41,46 +52,55 @@ at link:FrosconMeeting.html[FrOSCon 2025] in St.Augustin, where Benny Lyons also | March 2025 | re-establishing contact | Wouter offers concept work; first version of the Proposals document | April 2025 | »Workflow and Structure« | first deep dive: Placements, keybindings, grouping, gear switch | 2025-04-16 | IRC chat | placements in practice, sticky selection, trimming, keyboard navigation -| June 2025 | document update | Proposals chapter 2 (timeline) completed +| June 2025 | document update | Workflow Proposals [underline]#chapter 2# (timeline) completed | 2025-08-17 | FrOSCon-25 meeting | in-person discussion, summarised in link:FrosconMeeting.html[a page of its own] | Aug--Dec 2025 | »Workflow Discussion« | second deep dive: placement prototypes, moving and adding clips -| March 2026 | document update | Proposals chapter 1 (source material) completed +| March 2026 | document update | Proposals [underline]#chapter 1# (source material) completed |==================================================================== -How to read this digest -~~~~~~~~~~~~~~~~~~~~~~~ +Structure of this digest +~~~~~~~~~~~~~~~~~~~~~~~~ -The material is arranged by _topic_, not chronologically -- although within each topic thread, +The material is _arranged by topic_, not chronologically -- although within each topic thread, the argument is presented in the order it actually developed, through quotes attributed with author and date. Each topic thread closes with a status box stating what was _agreed_, what -remains _open_, and what was consciously _parked_. Topics touch upon each other in many places; +remains _open_, and what was _postponed_ deliberately. Topics touch upon each other in many places; such connections are given as cross-references. Quotes from eMails and chat have been lightly edited (typos, punctuation) for readability; the substance has been preserved verbatim. Some agreements were reached in unrecorded meetings and are marked as such. -Hermann proposed (eMail 2025-08-24) to organise the topic field into four areas, which -also provide the structure of this digest -- preceded by a section of cross-cutting themes: +In August 2025, Hermann proposed to organise the topic field into four areas, +which also form the structure of this overview -- preceded here by a section that +explains the fundamentals and addresses overarching topics. * *Control* -- control systems, command structure, navigation * *Routing* -- default rules, layering, explicit wiring * *Grouping* -- placements, grouping devices, ripple effects * *Editing* -- build-up, (re)arrangement, trimming +''''' [quote, Hermann, eMail 2025-08-24] ________________________________________________________________________ The next question however is a tricky one: how can we structure the further -discussion. This reminds me of Sergej Eisenstein's "spherical book". As you +discussion. This reminds me of Sergej Eisenstein's »spherical book«. As you probably know, he always wanted to write a book about film montage, but he could not quite see how to accomplish this task, because he noticed that -such a book must be "spherical", because everything is connected to everything. +such a book must be ``spherical'', because everything is connected to everything. ________________________________________________________________________ -Measured against this fourfold organisation, the discussion so far covered *Grouping* -almost exhaustively and *Editing* substantially, *Control* in one extended pass -- while -*Routing* was barely touched. This asymmetry is a finding in itself: the uncovered ground -marks natural starting points for the continuation of the discussion. +Measured against this fourfold organisation, the discussion so far covered _Grouping_ +and _Editing_ substantially, _Control_ in one extended pass -- while _Routing_ was barely touched. +This asymmetry originates from the way in which the discussion unfolded, +and provides natural starting points for a follow-up. + +· [big navy]#🙞 🙜# · + +​{nbsp} + +​{nbsp} + Cross-cutting Themes @@ -90,7 +110,7 @@ Cross-cutting Themes Vision and common ground ~~~~~~~~~~~~~~~~~~~~~~~~ -The very first exchange already established a striking degree of agreement on the goals. +The very first exchange already established a remarkable convergence of views regarding the aims. [quote, Hermann, eMail 2025-03-31] ________________________________________________________________________ @@ -102,8 +122,8 @@ editor's angle, seems to be spot on. ________________________________________________________________________ Both participants share the desire to get away from conventional _track management_ --- the constant enabling, disabling and locking of tracks -- while remaining pragmatic -about the outcome: +-- the constant enabling, disabling and locking of tracks -- whilst maintaining +flexibility regarding the choice of method: [quote, Wouter, eMail 2025-04-06] ________________________________________________________________________ @@ -124,13 +144,18 @@ order [...] to free up our usage of screen space within the arrangement of the edit in the timeline. ________________________________________________________________________ -A terminological note from Wouter frames this aspiration: if we speak of “tracks”, -we mean layers that carry enable/disable buttons and other track controls -- which is -why Final Cut Pro rather speaks of “layers”, a term Wouter also prefers, even though -there is no technical difference (eMail 2025-04-06). +A terminological note from Wouter also underpins this aspiration: +[quote, Wouter, eMail 2025-04-06] +________________________________________________________________________ +``Tracks'' we associate with the likes of Avid and Premiere, which is why FCP rather speaks +of ``layers'', a term I also prefer to use, even if there's no technical difference between the two. +________________________________________________________________________ + +In other words: Wouter tends to use the term ``tracks'', to imply not only the layering, +but also the UI feature with buttons to enable/disable them, and further track controls. Another shared conviction concerns the _level of abstraction_. Lumiera aims neither -at a rigid high-level feature set, nor at exposing raw building blocks: +at a ``magical'' high-level feature set, nor at exposing raw building blocks: [quote, Hermann, eMail 2025-04-07] ________________________________________________________________________ @@ -138,7 +163,7 @@ If you ask people, how a really cool (computer based) drawing tool should work, then you'll get two kinds of answers: - I want a tool which can read my mind and create a world-class drawing -- I want a digital paintbrush / crayon / pencil, so that I can be "creative" +- I want a digital paintbrush / crayon / pencil, so that I can be ``creative'' Yet in fact, a good digital drawing tool should rather be somewhere in the middle, like vector graphics with some additional supportive features. So @@ -146,33 +171,35 @@ that it is indeed still *you* who does the drawing, but the tool can amplify and support you, allowing you to move faster. ________________________________________________________________________ -This “medium level of abstraction” is a foundational principle from the -link:/documentation/user/intro/intro.html[Lumiera vision document]; and it connects -to a deliberate strategic choice: Lumiera is explicitly _not_ conceived as a clone -of existing commercial applications, and does not attempt to compete with +This ``medium level of abstraction'' is a foundational principle from the +link:{ldoc}/user/intro/intro.html#_medium_level_abstraction_and_project_specific_conventions [Lumiera vision document]; +and it connects to a deliberate strategic choice: Lumiera is explicitly _not conceived_ +as a clone of existing commercial applications, and does not attempt to compete with industry-backed development on their terms: [quote, Hermann, eMail 2025-10-04] ________________________________________________________________________ Personally, I think it is a strategically important insight that a community-backed project should try to avoid competing with an industrial -entity. [...] Even if that means, that it can not look that "slick" than an +entity. [...] Even if that means, that it can not look that ``slick'' than an industry backed solution can. I think, we need to find a way -to be *not* industry-backed, and be proud of it. +_to be not industry-backed,_ and be proud of it. ________________________________________________________________________ Wouter accepted this reasoning explicitly for the case discussed there (GPU-based -live preview rendering, see <>), while insisting that immediacy of -feedback is not mere polish but a genuine timesaver -- a tension both participants +live preview rendering, see -> xref:gesture-concept[gestures]), while insisting that immediacy of +feedback is not mere polish but a genuine time-saver -- a tension both participants agreed to resolve by clever design rather than raw computing power. +[role="summary"] ********************************************************************** *Agreed* -- shared goals: away from track management; tracks as fluid working space; medium level of abstraction; no clone, no feature race with industry. + *Open* -- how far the “trackless” aspiration can be realised in practice; -this question permeates the topics <>, <> -and <>. +this question permeates the topics -> xref:layering[layering], +xref:grouping-devices[grouping-devices] +and xref:moving[moving of clips]. ********************************************************************** @@ -180,14 +207,14 @@ and <>. Flow and organic editing ~~~~~~~~~~~~~~~~~~~~~~~~ -The notion of _flow_ became the central yardstick of the whole discussion: +The notion of »*Flow*« became the central yardstick of the whole discussion: UI concepts are judged by whether they sustain or interrupt the editor's flow. -Wouter formulated this most clearly: +Wouter formulated this pointedly: [quote, Wouter, eMail 2025-04-08] ________________________________________________________________________ I would like to share a few concerns that we might want to look into. The first -one is 'flow'. We previously talked about pursuing an organic way to edit. To me +one is ``flow''. We previously talked about pursuing an organic way to edit. To me that means a number of things, for example, on a physical level: how do you use your hands and fingers while editing, to work smoothly, and to have a sense of tactility? On another level it means that I'd like most of my actions to be @@ -208,27 +235,29 @@ tracks before performing an action (which is why I'm interested in trackless timelines). ________________________________________________________________________ -Hermann's corresponding formulation approaches the same goal from the side of +Hermann's corresponding formulation approaches a similar idea from the side of perception and spatial cognition: [quote, Hermann, eMail 2025-04-02] ________________________________________________________________________ Especially I mean the space metaphor, i.e. thinking in locations, rooms, -"close by", "here and there", up and down, left and right. As every editor +``close by'', ``here and there'', up and down, left and right. As every editor knows, humans are quite good at picking up spatial cues and relations. And it's often much easier to recall the context where you find some tool, than memorising it in an abstract, top-down classification scheme. [...] And the overarching idea is that we want the software to be an extension -of our body, so that we're able to be "in" the material we're working with. +of our body, so that we're able to be ``in'' the material we're working with. ________________________________________________________________________ The flow criterion recurs throughout: expand/collapse cycles are interruptions -(see <>); modes must be entered and exited fluidly (see <>); -and a coherent gesture vocabulary should let the work become “similar to kind of -a dance” (Hermann, eMail 2025-04-23; see <>). +(see -> xref:sections[timeline sections]); modes must be entered and exited fluidly +(see -> xref:tools-modes[tools and modes]); +and a coherent gesture vocabulary should let the work become ``similar to kind of +a dance'' (Hermann, eMail 2025-04-23; see -> xref:keybindings[keybindings]). +[role="summary"] ********************************************************************** -*Agreed* -- flow, understood as uninterrupted engagement with the material +Agreed:: flow, understood as uninterrupted engagement with the material and increased focus on storytelling, is the central design yardstick. ********************************************************************** @@ -239,7 +268,7 @@ Target audience: the personas In the »Workflow Proposals«, Wouter identified six groups of users who would most likely be attracted to Lumiera and its goals. This topic carried considerable weight -in the in-person discussion at FrOSCon-25, even while agreement was reached quickly: +in the discussion at the FrOSCon-25 conference, even while agreement was reached quickly: - the *highly specialised editor* working in an environment where the various parts of post-production are handled by different people (assistant editors, colourists, @@ -259,15 +288,16 @@ social media creators as the more challenging audiences, since both require rath specialised tools. The groupings are understood as provisional -- a working instrument to focus feature discussions and eventually order the feature list by priority. -The personas actually _do work_ within the discussion: for instance, the keybinding -debate (<>) explicitly distinguishes the specialist, who commands a rich -vocabulary of shortcuts, from the returning casual user, who will have forgotten -anything that is not obvious -- and the free-flowing editor is the natural constituency -for the sections proposal (<>). +The personas actually _do work_ within the discussion: for instance, the +-> xref:keybindings[keybinding debate] explicitly distinguishes the specialist, +who commands a rich vocabulary of shortcuts, from the returning casual user, +who will have forgotten anything that is not obvious -- and the free-flowing editor +is the natural constituency for the -> xref:sections[sections proposal]. +[role="summary"] ********************************************************************** -*Agreed* -- the six personas as provisional instrument for prioritisation. + -*Open* -- actually applying them: ordering the feature list, and checking +Agreed:: the six personas as provisional instrument for prioritisation. +Open:: actually applying them: ordering the feature list, and checking proposed interactions against each persona systematically. ********************************************************************** @@ -277,9 +307,9 @@ Working method ~~~~~~~~~~~~~~ Right at the beginning, Hermann proposed to conduct the exchange as a kind of -_workshop_: “to set aside a limited time budget [...] and try to discuss some of +_workshop_: ``to set aside a limited time budget [...] and try to discuss some of the aspects and proposals you raise and attempt to create a kind of design document, -or at least in some way document the ensuing discussion” (eMail 2025-03-31). +or at least in some way document the ensuing discussion'' (eMail 2025-03-31). Wouter later underlined the documentation aspect from his side: [quote, Wouter, eMail 2025-04-18] @@ -294,19 +324,20 @@ The method of the discussion itself was made explicit at one point: [quote, Hermann, eMail 2025-10-04] ________________________________________________________________________ -My train of thought is that I use an "explorative" approach. I try to determine, +My train of thought is that I use an ``explorative'' approach. I try to determine, first by logical reasoning, what is possible. But then we'd need to narrow that down, because we want something that feels _plausible_. What we're looking for -is some kind of "mechanics" -- quite similar to a game mechanic -- i.e. something -the user can pick up intuitively and apply it to similar situations. Something -that feels tactile, in a way. +is some kind of ``mechanics'' -- quite similar to a game mechanic -- i.e. +something the user can pick up intuitively and apply it to similar situations. +Something that feels tactile, in a way. ________________________________________________________________________ -Complementing this deductive side, Wouter contributes the empirical side: a growing -*catalogue of editing scenarios*, collected as screenshots from his actual work, -documenting typical situations which tend to be challenging -- to check any proposed -mechanics against reality. Hermann encouraged this explicitly: descriptions can be -terse, need not be encyclopedic, and need not come with solutions; what matters is +Complementing this _inductive method_, Wouter considers to contributes empirical +findings: a growing _catalogue of editing scenarios_, that might be collected as +screenshots from his actual work, documenting typical situations which tend to be +challenging -- to check any proposed mechanics against real-world usage. Hermann +encouraged this explicitly: descriptions can be terse, need not be encyclopedic, +and need not come with solutions; what matters is “the special twist that makes it difficult to handle” (eMail 2025-10-05). Both agree on the limits of armchair reasoning: @@ -317,34 +348,36 @@ built and user tested before we can really say whether something is an improvement or not. ________________________________________________________________________ +[role="summary"] ********************************************************************** -*Agreed* -- workshop-style discussion; documented reasoning; explorative +Agreed:: workshop-style discussion; documented reasoning; explorative concept work checked against a scenario catalogue; final validation only -through building and user testing. + -*Open* -- the scenario catalogue is work in progress and not yet published; +through building and user testing. +Open:: the scenario catalogue is work in progress and not yet published; the design document proper remains to be written -- this digest is a step on that path. ********************************************************************** [[background]] -Conceptual background: Placements, symbolic representation, Event Sourcing -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Conceptual background +~~~~~~~~~~~~~~~~~~~~~ Early in the discussion, Hermann gave a series of explanations of Lumiera's foundational concepts, which the later topic threads constantly build upon. They are collected here as reference. -.The Placement concept -The whole Session model is connected by _Placements_ as a universal “glue”. +The Placement concept +^^^^^^^^^^^^^^^^^^^^^ +The whole Session model is connected by _Placements_ as a ``universal glue''. [quote, Hermann, eMail 2025-04-02] ________________________________________________________________________ -The user arranges "things" in the GUI, but actually this leads to an arrangement -of element descriptions in the *Session Model*. [...] However, this arrangement is -very loosely coupled and flexible. In fact, it is a collection of symbolic -representations, which are stitched together by a common "glue": the Placements. -[...] each of these symbolic "entities" is connected to the model through a +The user arranges ``things'' in the GUI, but actually this leads to an arrangement +of element descriptions in the _Session Model_. [...] However, this arrangement is +very loosely coupled and flexible. In fact, it is a collection of *symbolic +representations*, which are stitched together by a common ``glue'': the Placements. +[...] each of these symbolic _entities_ is connected to the model through a Placement, which describes _relations_. The placement has an (open, unlimited) list of conditions and constrictions. There is only one mandatory aspect: each placement is attached below another parent-Placement. @@ -359,43 +392,108 @@ to its parent. ________________________________________________________________________ Among the motivations for this approach, one stands out as directly workflow-related: -the notorious problem of _unintended modifications_ to already settled parts of an -edit. Instead of protecting an arrangement after the fact by “locking”, the Placement -concept expresses directly the constraints, links and connections that should not be -broken (eMail 2025-04-02). Further motivations: Lumiera must be able to operate -without GUI (so concepts like “snap-to” are first-class in the core model, not UI -gadgets); and the model must not be a fixed data structure with pre-determined -meaning, since such fixed models notoriously paint an application into a corner. +the notorious problem of *unintended modifications* to already settled parts of an +edit. Instead of protecting an arrangement after the fact by ``locking'' (prohibiting) +any modifications, the Placement concept expresses directly the constraints, links and +connections that should not be broken (eMail 2025-04-02). +Further motivations: Lumiera must be able to operate +without GUI (so concepts like »snap-to« are first-class in the core model, not just +UI features); and the model must not be a fixed data structure with a pre-determined +meaning, as such fixed models are notorious for leading an application into a dead end. -The general scheme of arrangement: at top level there are several *Timelines*; -each Timeline is connected through a Binding with a *Sequence*; a Sequence provides -a *Fork* (a tree of tracks); *Clips* live within such forks. And crucially: -a clip either holds a binding to a media asset -- or to another sequence, whose -contents thereby become a new _virtual medium_, basically indistinguishable from -real media. This is the foundation for nested sequences, “virtual clips” and the -grouping discussion (<>). +[quote, Hermann, eMail 2025-04-02] +________________________________________________________________________ +[...] a description how virtual clips or +nested timelines are intended to work in Lumiera. +Both the GUI and the Render Engine assume a general scheme +of arrangement (and ignore any elements which are present +for whatever reason but do not fit into this scheme): -.Symbolic representation -Responding to Wouter's question what “symbolic representation” actually means, -Hermann contrasted two styles of building an IT system: the conventional approach -fixes a mental model up-front and only represents parameters within that model --- whereas the symbolic approach represents _the model itself_ by connected symbols, -writes code to interpret this structure, and thereby gains the ability to also -_transform_ the structure (eMail 2025-04-07). The cautionary example is Lumiera's -own ancestor: +- at top level we have several Timelines +- each Timeline is connected through a Binding with a Sequence +- a Sequence provides a Fork (tree of tracks) +- Clips will be considered if found in such a fork/track +- Effects will be considered if found attached either to a fork/track + or to an individual clip +- a clip either holds a _Binding_ to a Media Asset, or + _to another Sequence_. + +This means: + +- if we bind a sequence to a Timeline, we connect content + to a time/frame grid, and as a result, we get a set of media outputs. + These can then be ``performed'' (played back, rendered) + +- however, if we bind a sequence to a Clip, the contents of the sequence + become a new, _virtual medium_, which is basically indistinguishable + from real media (video on disk). + +Each clip is attached by a Placement. This in turn can point to +an absolute time position, or be attached with offset behind the end +of an predecessor, or a relative offset to the parent track, or it +can be attached relative to some point in another medium (say a +sync-point in some audio) + +By default, if nothing is qualified, data output is traced from +the clip through the fork and the binding into the timeline; at +the fork, an overlayer / combiner / mixer is added automatically. +The Engine knows which kinds of media it can connect. + +However, if you add a »plug« qualification to some placement, then +we get the functionality of nodes: the designation indicated in the +plug will determine how the pipeline is built. +________________________________________________________________________ + +So placements are the foundation for nested sequences and ``virtual clips'', +but also control aspects of the routing and can connect several clips to +move together, which becomes relevant later for the +-> xref:grouping-devices[grouping discussion]. + + +Symbolic representation +^^^^^^^^^^^^^^^^^^^^^^^ +Responding to Wouter's question what ``symbolic representation'' actually means, +Hermann contrasted two styles of building an IT system: [quote, Hermann, eMail 2025-04-07] ________________________________________________________________________ -In Cinelerra, the author created a session object, which has an "EditDecisionList", -where each entry is a clip, which has a start frame number. Then later, he also -added a layer number to each clip. Fast forward through some quite nasty code -in the render engine: the net result is that in Cinelerra, a transition can only -join source clips, and the result is sent through a flat compositing / effect -pipeline. You'd have to rewrite the complete engine in order to have transitions -after the effects. +First to mention is a conventional, or maybe also obvious or naive approach: + +- you agree on a mental model how to structure the matter to be treated +- in the actual data, you only represent the parameters within this model +- then you write actual code for the treatment, against the backdrop of + the fixed (mental) model, yet controlled by those parameters + +However, since the early days of computing, a different approach was developed +in the realm of language processing, process automation and simulation + +- you represent the model itself by symbols, which are connected +- then you write code to interpret this structure and control the processing +- but in addition, you can now also write code that transforms the structure. + +To give you a practical example: let's assume someone has tasked us to +implement a ``programme'' to calculate a balance sheet for a small business. +With the naive approach, you ask your customer how data has to be processed, +and the customer tells you: it has to be summed up. So this sets the mental +model. Thus you implement a list of positions, and then you add code to +iterate over that list and accumulate a sum. And when you're done, your +customer will tell you, that on some occasions, also a statistical mean +is required. Thus you go back, add a flag to toggle if you'd want to +compute a sum, or also to calculate the average. Then you ask your +customer, how to set that flag, and here we reach the point where most +typical projects start to go downhill. + +With the symbolic / language processing approach, you'd rather encode a +formula, and use code from a library to compute algebraic expressions. +And you add rules when to use which formula. This makes the implementation +more demanding, but also more flexible, since you can adapt to your customers +requirements by changing the configuration, instead of having to write new +code whenever the customer comes up with a new explanation of what they +actually need. ________________________________________________________________________ -.Event Sourcing and CQRS +Event Sourcing and CQRS +^^^^^^^^^^^^^^^^^^^^^^^ Lumiera's processing of user actions follows the _Event Sourcing_ pattern: instead of mutating a central application state, the application captures a log of things that _happened_. Every part of the application listens to these events @@ -408,17 +506,20 @@ interaction -> command -> logged event -> the various parts of the application adjust their state in response (eMail 2025-04-02). How a UI gesture actually travels through this architecture is spelled out in -<>. Note also the resonance with Wouter's proposal of _timeline -versioning_ (snapshots of timelines as a core feature, see <>): an event -log with save points provides exactly the substrate for such functionality. +the context of the xref:gesture-concept[Gestures]. +Note also the resonance with Wouter's proposal of _timeline versioning_ +(snapshots of timelines as a core feature, see -> xref:tagging[tagging]): +an event log with save points _provides the substrate_ for such functionality. +[role="summary"] ********************************************************************** This thread is reference material rather than controversy; the discussion of consequences happens in the topic threads. Wouter's caution deserves -record though: novel concepts like Placements must not create “surprising -or non-deterministic (suddenly jumping) behaviour” (a risk Hermann named +record though: novel concepts like Placements must not create ``surprising +or non-deterministic (suddenly jumping) behaviour'' (a risk Hermann named himself, eMail 2025-04-02) -- the answer to which is worked out in -<> and <>. +the disccussion pertaining -> xref:prototypes[Placement prototypes] +and -> xref:adding[adding of source material]. ********************************************************************** @@ -445,18 +546,18 @@ right and providing ways to export the timeline to other apps for finishing seems a bit more manageable. ________________________________________________________________________ -Hermann's answer: “I agree and I disagree at the same time, because that indicates -a dilemma” (eMail 2025-04-07) -- if a project starts simple and pragmatic, it gets -locked into a fixed model (see the Cinelerra example in <>); yet a small -project cannot cover the whole ground. Lumiera's resolution is a two-layered -approach: form a core model that _can_ cover the larger ground, while structuring -it such that plain editing-and-rendering functionality can be finished first. -Interestingly, both participants share the assessment that “nodes as-we-know-them” +Hermann's answer: ``I agree and I disagree at the same time, because that indicates +a dilemma'' (eMail 2025-04-07) -- if a project starts simple and pragmatic, it runs +danger of getting locked into a fixed model; yet a small project cannot cover the +whole ground. Lumiera's resolution is a two-layered approach: form a core model that +_can_ cover the larger ground, while structuring it such that plain editing-and-rendering +functionality can be finished first. +Interestingly, both participants share the assessment that ``nodes as-we-know-them'' do not work well in larger editing projects -- but draw different conclusions: Wouter wants the topic out of scope, Hermann wants to re-think what is good about nodes, so that traditional editing can be smoothly _extended_ into compositing territory when and where needed (eMail 2025-04-06). In the Lumiera model, node-like -wiring exists as a capability of Placements (“plug” qualifications), without a +wiring exists as a capability of Placements (»plug« qualifications), without a node-graph UI. .Internationalisation @@ -468,32 +569,38 @@ bidirectional support would cause a complexity explosion precisely in the massiv custom-drawing code of a video editor (does time run right-to-left? the track heads, the tool palettes, the timecode formats...?), effort a small team cannot shoulder (eMail 2025-08-29). He also flagged the meta-question -- whether it is honest -to “give the impression that you're open towards an idea when in fact you have made -up your mind otherwise” -- and enumerated similar priority questions likely to +to ``give the impression that you're open towards an idea when in fact you have made +up your mind otherwise'' -- and enumerated similar priority questions likely to come up: MS Windows portability, tablet platforms, GPU-based realtime previews, beginner-friendliness, AI assistance. .The UI toolkit (GTK vs. Qt) -Started “in the garden” at FrOSCon-25: Wouter pointed out Qt's track record for -big timeline applications (DaVinci Resolve, Ableton Live), the poor Mac situation -of GTK, and easier availability of Qt developers; he observed sluggishness in -several GTK-3/4 applications (eMail 2025-08-26). Hermann laid out the other side: -any project doing non-standard custom drawing is effectively “married” to its -toolkit; the GTK choice was historical (Joel Holdsworth), but meanwhile rests on -deep inside knowledge and a way of using GTK-3's toolkit aspects while pushing -aside the framework aspects; Qt in turn is a massive application framework whose -abstraction layers carry their own price tag, and pushing those aside is equally -not the intended usage. “Since we intend to do very specific things that are -certainly not mainstream, we need to walk a thin line here.” (eMail 2025-08-29) +This discussion started ``in the garden at FrOSCon-25'': Wouter pointed out Qt's +track record for big timeline applications (DaVinci Resolve, Ableton Live), the +poor Mac situation of GTK, and easier availability of Qt developers; he observed +sluggishness in several GTK-3/4 applications (eMail 2025-08-26). Hermann laid out +the other side: any project doing non-standard custom drawing is effectively +``married'' to its toolkit; the GTK choice was historical (Joel Holdsworth), yet +meanwhile rests on deep inside knowledge and a way of using GTK-3's toolkit aspects +while pushing aside the framework aspects; Qt in turn is a massive application framework +whose abstraction layers carry their own price tag, and pushing those aside is equally +not the intended usage. ``Since we intend to do very specific things that are +certainly not mainstream, we need to walk a thin line here.'' (eMail 2025-08-29) +[role="summary"] ********************************************************************** -*Agreed* -- editing first; no node-graph UI, but the core model retains -node-like wiring capability; UI language English, no bidirectional -support planned. + -*Open* -- toolkit question: no change of course, but the concern is on -record and the direction of GTK development is being watched. + -*Parked* -- Windows/tablet ports, GPU preview engineering, AI assistance: -consciously deferred priority questions. +Agreed:: +- editing takes centre stage +- no node-graph based primary UI +- yet the core model retains node-like wiring capability +- UI language English, some terms not to be translated +- no support for bidirectional language UI planned +Open:: toolkit question: no change of course, but the concern is on +record and the direction of GTK development is being watched. +Postponed:: deliberately de-prioritised and taken out of focus +- Windows / tablet ports +- GPU preview engineering +- AI assistance ********************************************************************** @@ -505,41 +612,42 @@ The gesture concept: command architecture and immediate feedback ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The single most load-bearing concept to emerge from the Control discussion is -the _gesture_. The crispest definition was given in the IRC chat, starting from +the *Gesture*. The crispest definition was given in the IRC chat, starting from the question how a trim edit should be performed: [quote, Hermann, IRC 2025-04-16] ________________________________________________________________________ -"Trimming" is a gesture, and results in a "trimming command" sent to the core. +``Trimming'' is a _Gesture_, and results in a ``trimming command'' sent to the core. But that gesture can have multiple implementations. So basically, what I call -a "gesture" has the structure of a sentence in language. It has a subject, +a ``gesture'' has the structure of _a sentence in language_. It has a subject, a predication (here: trimming) and some qualifications. E.g. in mouse based editing, you would enter into that gesture with a modifier key and then by dragging at one side of the cut -- while in a keyboard-only control system, you would need a selection, and then orient that selection towards one cut, and then use a further key or combination to enter into the adjustment, -and maybe an enter key to complete the "sentence". +and maybe an enter key to complete the ``sentence''. ________________________________________________________________________ A gesture is thus an _abstract interaction_, which can be bound to different *control systems* (mouse, keyboard, pen, hardware controllers) -- the vision being that all control systems are supported on an equal footing. Concrete gesture -designs are discussed within their functional topics (e.g. the hooking gesture in -<>, navigation gestures in <>, trim gestures in <>); -this thread records the underlying machinery and the question of immediate feedback. +designs are discussed within their functional topics (e.g. the hooking gesture for +-> xref:adding[adding clips], navigation gestures in the +-> xref:keybindings[keybinding discussion], trim gestures in -> xref:trimming[trimming edits]); +so this thread here records the underlying machinery and the question of immediate feedback. .How a gesture becomes a change in the model On the technical level, dragging and its visual feedback happen entirely within the UI, handled by a _gesture controller_ with pre-canned, quick-response logic. Once the gesture completes (e.g. the mouse is released), the controller assembles -a *command sentence* -- a symbolic representation of the kind “this Subject with +a _command sentence_ -- a symbolic representation of the kind ``this Subject with ID 12345 was moved by GUI-coordinates of Δ −205 and hit another Subject with -ID 9876 in constellation III”. This sentence is sent through the UI-Bus down into -the Session, where it is processed asynchronously and with much more flexibility: -placements are queried, clips re-arranged, knockout conditions checked. The outcome -is placed as an *event* into the global session log, broadcast, and “projected”: -the GUI receives a diff message and re-arranges the visible widgets -(Hermann, eMail 2025-10-04; cf. <>). +ID 9876 in constellation III''. This sentence, encoded as a symbolic term, is sent +through the UI-Bus down into the Session, where it is processed asynchronously and with +much more flexibility: placements are queried, clips re-arranged, knockout conditions +checked. The outcome is placed as an *event* into the global session log, broadcast, +and ``projected'': the GUI receives a diff message and re-arranges the visible widgets +(Hermann, eMail 2025-10-04; see also -> xref:_event_sourcing_and_cqrs[Event Sourcing]). [quote, Hermann, eMail 2025-10-04] ________________________________________________________________________ @@ -550,30 +658,30 @@ ________________________________________________________________________ .Immediate feedback -- and its economics Wouter raised the obvious concern: Final Cut, Resolve and to some degree Avid -show the result of an action _while you are performing it_ -- “That's a fantastic -way to see what kind of impact your action is having” (eMail 2025-10-04). +show the result of an action _while you are performing it_ -- ``That's a fantastic +way to see what kind of impact your action is having'' (eMail 2025-10-04). Hermann's analysis: UI event streams deliver several events per millisecond, while resolving a network of placement constraints can take between a tenth of a second and seconds; full live resolution would require the kind of GPU-backed -engineering that only industry can sustain (see <>). Wouter conceded the -strategic point -- “it might be better to not even try; Adobe Premiere doesn't -have this either” -- but insisted that immediacy is a genuine timesaver, and that -Lumiera's placement concept makes outcomes _harder_ to predict, so the user needs -guidance all the more (eMail 2025-10-06). +engineering that only industry can sustain (see also -> xref:vision[vision section]). +Wouter conceded the strategic point -- ``it might be better to not even try; +Adobe Premiere doesn't have this either'' -- but insisted that immediacy is a genuine +timesaver, and that Lumiera's placement concept makes outcomes _harder_ to predict, +so the user needs guidance all the more (eMail 2025-10-06). The agreed middle ground was formulated as two priorities: [quote, Hermann, eMail 2025-10-08] ________________________________________________________________________ -Priority-1: get the immediate feedback to reflect the gesture as such, but as +Priority-1:: get the immediate feedback to reflect the gesture as such, but as complete as possible, preferably with simple graphical means: you are dragging _this_ and it engages with _that_ bounds/targets. -Priority-2: enrich this with additional cues about consequences, even if these +Priority-2:: enrich this with additional cues about consequences, even if these are not completely accurate, but indicate the overall tendency correctly. ________________________________________________________________________ -Priority-2 requires the GUI to receive “a little help” from the session model +Priority-2 requires the GUI to receive ``a little help'' from the session model _beforehand_: connectivity hints attached to the elements (e.g. a clip carrying an index-hint naming the later clip that attaches to it magnetically), so the gesture controller can draw connection lines as a simple overlay without latency. @@ -592,15 +700,16 @@ That should give us enough information about how clips on the timeline will react to our gesture. ________________________________________________________________________ +[role="summary"] ********************************************************************** -*Agreed* -- gestures as sentence-like abstract interactions, mappable onto +Agreed:: gestures as sentence-like abstract interactions, mappable onto all control systems; command-sentence / event-log architecture; immediate feedback follows the two-priority scheme; full live-resolution rendering -is not pursued. + -*Open* -- feasibility of the Priority-2 connectivity hints (“sound out how -deep the rabbit hole goes”); the concrete visual language of the cues. + -*Left over from FrOSCon-25* -- it remains to be shown that a generic -gesture concept can be mapped onto the different control systems coherently. +is not pursued. +Open:: feasibility of the Priority-2 connectivity hints (“sound out how +deep the rabbit hole goes”); the concrete visual language of the cues. +Left over from FrOSCon-25:: it remains to be shown that a generic gesture +concept can be mapped onto the different control systems coherently. ********************************************************************** @@ -608,16 +717,16 @@ gesture concept can be mapped onto the different control systems coherently. Keybindings and keyboard navigation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This topic opened the first deep dive, with Hermann's critique of the “almost -universal pattern” of key bindings in contemporary applications: a flat mapping +This topic opened the first deep dive, with Hermann's critique of the ``almost +universal pattern'' of key bindings in contemporary applications: a flat mapping space, scopes that are obvious to the programmer but opaque to the user and do not mesh with the workflow, bewildering function names, and de-facto un-documentability caused by the very configurability of the bindings. The -opposite pole -- the “opinionated GUI” -- surprisingly yields a small, well -structured, documentable set of bindings, plus the ability to give active cues. -And overshadowing both: “we can only recall and associate a very small number +opposite pole -- the ``opinionated GUI'' -- surprisingly yields a small, well +structured, well documentable set of bindings, plus the ability to give active cues. +And overshadowing both: ``we can only recall and associate a very small number of things at any instant [...] even while the software gives you hundreds of -keybindings, most people will only ever be able to use one hand full” +keybindings, most people will only ever be able to use one hand full'' (eMail 2025-04-02). Wouter countered with the professional editor's reality: @@ -631,22 +740,37 @@ editors are very keen on customizing the keys to their liking. ________________________________________________________________________ ...and later supplied the ergonomic argument why no default can fit everyone: -his own most efficient layout clustered essential functions around ASD for the -left hand -- until a (left-handed) switch to a Wacom tablet changed the entire -configuration. “It will be impossible to come up with ergonomic defaults as we -will never know the specific setup of a user.” (eMail 2025-04-25) At the same +his own most efficient layout clustered essential functions around the `ASD` keys +for the left hand -- until a (left-handed) switch to a Wacom tablet changed the +entire configuration. ``It will be impossible to come up with ergonomic defaults +as we will never know the specific setup of a user'' (eMail 2025-04-25). At the same time he confirmed Hermann's core observation from his own experience: flat bindings only stick when used identically across applications and daily -- with the notable exception of Blender, whose keys survive years of disuse because -its gesture patterns build muscle memory quickly. +its gesture patterns are so well structured and plausible so that they tend +to build muscle memory quickly. + +Incidentally, both agree to reject the ``opinionated'' style, the »Gnome style« +of user interface design -- as Hermann put it (eMail 2025-04-02): ``we know best +what you want and need and we give you exactly that and will not confuse you +with any configurability''... Hermann drew the two approaches together with an observation about what actually succeeds: [quote, Hermann, eMail 2025-04-24] ________________________________________________________________________ -Many really successful interaction paradigms fall into the second category -[a small set of interaction primitives, mapped depending on context]: +One common approach is to have a broad set of shortcuts, +be it key bindings, or be it a set of buttons on a toolbar. + +The complimentary approach is to rely on context and +navigation rather, and to use the same set of bindings +or interaction primitives, but mapped depending on the context. + +The first approach seems to be very popular and widely used. +It is easy to understand and it is even more simple to build. +However, many really successful interaction paradigms +fall into the second category: - the mouse - the "ubiquitous" keys (spacebar, cursor keys, ESC, TAB, enter) @@ -654,75 +778,96 @@ Many really successful interaction paradigms fall into the second category - playing music on the piano keyboard (you use only 10 fingers, and achieve everything with context and posture of your hands) -[...] So I'd rather put 20% into the generic thing [a flat keybinding framework -plus editor] and 80% into trying to come up with specific stuff, like building -something for media playback and something for trimming/rolling etc. and -something for coherent navigation. +Here, with *successful* I mean that I can take it on and get +accustomed to it _and get back into using it_ when returning +to the same application, say, one year later. ________________________________________________________________________ +He would thus prefer to put only 20% of development effort into a conventional +generic keybinding framework, while allocate 80% towards development of broad +generic handling patterns, that can be controlled by a small set of common +gestures. Notable examples would be navigation, controlling playback, or an +attempt to connect editing actions like trimming/rolling to navigation gestures. + + The synthesis both sides subscribed to: build the coherent structure, but let it _map down_ to common expectations -- or more precisely, let it be *simplified down*, while the more elaborate structure remains underneath (unlike conventional -applications, where no such structure exists). A newcomer gets click-and-drag -behaviour; the expert configuration uses explicit triggers Blender-style -(“g”/“m”), which as a side effect makes selections _sticky_. +applications, where no such elaborate backing structure exists). A newcomer would +get the expected ``click-and-drag'' behaviour; the expert configuration might use +explicit triggers Blender-style (`g`/`m`), which would also lead to making +selections _sticky_. [quote, Wouter, eMail 2025-04-25] ________________________________________________________________________ I think trying to come up with something that works consistently across the entire application is worth pursuing. I also think that having a system that -can be simplified to something that people will instantly 'get' because it +can be simplified to something that people will instantly ``get'' because it uses common paradigms found in other apps is essential [...] it will be very hard to reach both these goals with one system, but let's see if it's possible! ________________________________________________________________________ -The same principle was formulated memorably in the IRC chat: “if we want to do -something unconventional, there must be a path leading to it from known terrain” -(Hermann); with Wouter adding the reason: “people carry their knowledge of other +The same principle was formulated memorably in the IRC chat: ``if we want to do +something unconventional, there must be a path leading to it from known terrain'' +(Hermann); with Wouter adding the reason: ``people carry their knowledge of other apps into new apps, expecting them to respect certain conventions -- when a program does things wildly different, people will be very uncomfortable, and -this becomes an obstacle for them to invest time into learning it” +this becomes an obstacle for them to invest time into learning it'' (IRC 2025-04-16). -.Selection +Selection +^^^^^^^^^ Selection emerged as the structural backbone of keyboard control: there is -_always_ a selection, inherently structured as a focal point plus a path of +_always a selection_, inherently structured as a focal point plus a path of nested scopes; navigation moves this focal point through a tree -- to siblings, up and down, and across dedicated cross-links (into an effect's property pane, into the media bin a clip was taken from). In the IRC chat this became the image -of a “2½-dimensional” space: two dimensions of neighbourhood, plus _portals_ -where you step into an object. Related agreements and proposals: +of a ``2½-dimensional'' space: two dimensions of neighbourhood, plus _portals_ +where you step into an object or contextually connected setting. +Related agreements and proposals: - losing a selection unintentionally must not happen (Blender's sticky selection - as reference; Premiere's ‘selection follows playhead’ as counter-example); -- Wouter's proposal: include select/deselect operations in the *undo stack*, + as reference; Premiere's ``selection follows playhead'' as counter-example); +- Wouter's proposal: include select/deselect operations in the _undo stack_, so an accidentally lost selection is one Ctrl-Z away (eMail 2025-04-18); - trim-side selections are especially precious and must be recoverable - (IRC 2025-04-16, cf. <>); + (IRC 2025-04-16, -> see xref:trimming[trim editing]); - clip selection should stay intact when trim sides are selected (eMail 2025-04-18). -.Keyboard navigation gestures -Wouter proposed the base vocabulary (eMail 2025-04-18): one track up/down, one -clip left/right; with Shift, jumps of 25--35% of the timeline _viewport_ -- sized -relative to zoom level, and thus independent of how many clips of which length -are on the timeline; plus ‘add to selection’ and ‘select from start to end -position’. He prefers a single playhead, if workable: “simpler will be better”, -with the selected track indicated by two outward-facing triangles on the playhead. -Later additions: a shortcut to *cycle* through placement options -- workable only -if there are at most three (see <>) -- or, alternatively, small -Blender-style context menus on plain keys: “p” pops up placement options, -“e” lists everything that can be enabled... (eMail 2025-10-21). An -“expand selection” shortcut following the scope hierarchy was discussed in a -meeting and found promising. +Keyboard navigation gestures +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Wouter proposed a base vocabulary (eMail 2025-04-18): +- one track up/down, one clip left/right; with Shift +- jumps of 25--35% of the timeline _viewport_ -- sized relative to zoom level, + and thus independent of how many clips of which length are on the timeline +- ``add to selection'' and ``select from start to end position''. + +He prefers a single playhead, if workable: ``simpler will be better'', +with the selected track indicated by two outward-facing triangles on the playhead. +Hermann however would not rule out prematurely the idea of an edit location to be +kept separate from the playhead. + +Later additions to the base vocabulary: + +- a shortcut to _cycle_ through tool-palette variations (e.g. the different + xref:prototypes[Placement prototypes] +- or, alternatively, small Blender-style context menus on plain keys: + "`p`" pops up placement options, "`e`" lists everything that can be enabled... + (eMail 2025-10-21). +- An ``expand selection'' shortcut following the scope hierarchy was discussed + in a meeting and found promising. + +[role="summary"] ********************************************************************** -*Agreed* -- both systems will exist: a configurable flat binding layer as +Agreed:: both systems will coexist: a configurable flat binding layer as necessary baseline, and a small coherent gesture vocabulary as the actual -investment (“20/80”); the coherent structure must be simplifiable down to +investment (``20/80''); the coherent structure must be simplifiable down to conventional behaviour; selections must be robust, possibly undo-tracked. + -*Open* -- the concrete default bindings; cycling vs. key-popup-menus for +Open:: the concrete default bindings; cycling vs. key-popup-menus for placement selection; how far selection focus navigation can span the -whole UI (FrOSCon-25 left open whether effect properties and mixer strips +whole UI. +FrOSCon-25 left open:: whether effect properties and mixer strips integrate into an overarching navigation scheme). ********************************************************************** @@ -731,29 +876,30 @@ integrate into an overarching navigation scheme). Tools, modes, views and the contextual tool palette ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Wouter's document analyses how existing NLEs switch timeline interactions: -tool-based (Premiere, FCP), mode-based (Avid, Lightworks, Resolve), or view-based -(FCP's precision editor) -- and finds that no NLE uses one method exclusively; +Wouter's »Lumiera Workflow Proposals« document analyses how existing NLEs switch +timeline interactions: tool-based (Premiere, FCP), mode-based (Avid, Lightworks, Resolve), +or view-based (FCP's precision editor) -- and finds that no NLE uses one method exclusively; hybrids are the norm. His conclusion sets the direction -(link:WorkflowProposals.html#\_tools_modes_views[»Tools + modes + views«]): -use whichever combination is appropriate, avoid all three where possible -- -and where a mode is used, design it as a *contextual mode* that is entered and -exited fluidly, without dedicated user actions. +(link:WorkflowProposals.html#_tools_modes_views[»Tools + modes + views«]): +use whichever combination is appropriate, avoid all three whenever a simple, +direct and obvious interaction is possible -- and where a mode is used, design it as +a *contextual mode* that is entered and exited fluidly, without dedicated user actions. -The centrepiece of his proposal is the *contextual bar* (also: contextual tool -palette): an overlay appearing over the bottom part of the timeline whenever a -contextual mode is active, offering exactly the options of that mode -- e.g. for -clip selection (group, cut, duplicate, nudge, ripple- and snap-toggles), for -trimming (trim/slip/slide, ripple toggle), and for adding clips (insert, +The centrepiece of his proposal is the +link:WorkflowProposals.html#contextual_bar[»contextual bar«] (also termed the +»*contextual tool palette*«): an overlay appearing over the bottom part of the timeline +whenever a contextual mode is active, offering exactly the options of that mode -- e.g. +for clip selection (group, cut, duplicate, nudge, ripple- and snap-toggles), for +trimming (trim/roll/slip/slide/ripple toggle), and for adding clips (insert, overwrite, replace...). Colours can indicate which contextual mode is active. -In the mail discussion, the mode question first appeared as a point of friction -- -“in textbook UI design, modes are frowned upon [...] however, in our domain, I am -convinced that modes, when used well, are a valuable tool” (Hermann, -eMail 2025-04-07); Wouter concurred, listing the many modes hiding in every -serious NLE (eMail 2025-04-08). At FrOSCon-25 the direction was then settled: -*tools* are the preferable system -- provided a handling mechanism can be found -that works naturally across all control systems. Hermann proposed, taking +In the mail discussion, the _question of modes_ first appeared as a point of friction -- +``in textbook UI design, modes are frowned upon [...] however, in our domain, I am +convinced that modes, when used well, are a valuable tool'' (Hermann, eMail 2025-04-07); +Wouter concurred, listing the many modes hiding in every serious NLE (eMail 2025-04-08). +At FrOSCon-25 the direction was then settled: +*tools* (tool-modes) are the preferable system -- provided a handling mechanism can +be found that works naturally across all control systems. Hermann proposed, taking inspiration from Blender, to extend tool usage to the entire UI, with a top-level *navigation tool* as default; moving of clips would become a sub-mode of this navigation tool rather than a separate tool. Wouter's contextual palette @@ -785,15 +931,20 @@ phase of an edit may need a different configuration than the tweaking phase; a switchable sub-mode which can also be overlaid temporarily by a modifier can serve both (Hermann, eMail 2025-10-04). +[role="summary"] ********************************************************************** -*Agreed* -- tools as primary system, extended over the whole UI with a +Agreed:: tools as primary system, extended over the whole UI with a default navigation tool; contextual modes with fluid entry/exit; the contextual tool palette as central device, carrying sub-modes and placement choices; the Gimp pattern: palette shows standard + alternative, -sticky by click, temporary via the common modifier (Ctrl). + -*Open* -- how effect parameters and mixer strips are reached under an -overarching navigation tool (explicitly unresolved since FrOSCon-25); -palette layout details, avoiding occlusion of timeline content. +sticky by click, temporary via the common modifier (Ctrl). +Open:: most aspects of the tangible manifestation: ++ +- what actual modes to provide and how they mesh up +- how effect parameters and mixer strips are reached under an +overarching navigation tool (explicitly unresolved since FrOSCon-25) +- palette layout details +- avoiding occlusion of timeline content. ********************************************************************** @@ -807,8 +958,17 @@ in media work span several orders of magnitude, and every level matters at times [quote, Hermann, eMail 2025-04-23] ________________________________________________________________________ So in my dreams I'd like to have a well-delineated gesture for a gear switch, -similar to what I have on my bicycle handlebar. And I would be able to "drive" +similar to what I have on my bicycle handlebar. And I would be able to ``drive'' any graduated movement and setting with the same patterns and gestures. [...] +Domain knowledge plays an important role here, and presumably the actual +gear-level-scale would need to be context dependent. Because working on a +sound fade requires a different stepping than moving a playhead in a frame +based medium, or when moving around in a timeline. [...] + +I would rather not try to come up with a dedicated key binding solely +for movement in the timeline. And maybe another one for fine-tuning sound +levels. Rather I'd propose to aim at a single uniform set of handles, +which work likewise for media playback, moving the viewport, +trimming a clip or any parameter adjustment - nudge one step plus / minus - nudge on the next higher gear level @@ -817,39 +977,48 @@ any graduated movement and setting with the same patterns and gestures. [...] - presumably we also need an (unobtrusive) overlay to indicate the gear scale. ________________________________________________________________________ -The point of reference: mouse acceleration is in fact a hidden, built-in gear -switch nobody can control; media applications add ad-hoc fine-tuning modifiers -for a few “especially relevant” knobs. Instead, one uniform set of handles should -work identically for media playback, viewport movement, trimming, and any -parameter adjustment -- with the gear-level scale being context dependent (a sound -fade needs different stepping than frame-wise playhead movement or “warp speed” -travel across a large project). Like every part of the interaction concept, it -must map down: configured simply, it degenerates to two gears -- ordinary and -accelerated. +The point of reference is the mouse acceleration feature of most desktop environments. +This is in fact a hidden, built-in gear switch nobody can control; media applications +tend to add ad-hoc fine-tuning modifiers for a few ``especially relevant'' knobs. +Instead, one uniform set of handles and a gesture should be usable whenever some +value or setting must be adjusted. Wouter found the idea clear and worth considering, but contributed two concerns -from hardware experience (eMail 2025-04-25): misremembering the current gear -means gestures overshoot or undershoot, so the active gear needs very clear -visualisation; and the DaVinci Speed Editor's search dial -- effectively two -useful gears (jog, scroll) -- taught him that _dedicated_ controls per -sensitivity can beat switching: “I would have rather had a bigger device with -two dedicated wheels, so that I would always know which one to use.” His open -question: should Lumiera offer both approaches -- gears with few keys, _and_ -the option to map each gear's movement to dedicated keys? +from hardware usage experience + +[quote, Wouter, eMail 2025-04-25] +________________________________________________________________________ +[...] I was wondering whether constantly having to change this mode/gear +would be experienced as unpleasant. I can imagine, for example, that you simply +want to navigate a certain amount to the right, but if you were to incorrectly +remember the gear, then that gesture would not take you as far as you wanted, +and you'd have to switch gears first, then try again. + +The gears make me think of DaVinci's Speed Editor. +It has a wheel that can be set in three modes: shuttle, jog and scroll [...] +Shuttle is useless, but jog and scroll are two different sensitivities for navigation. [...] +So these could be seen as two gears. It does work and it is efficient, but: +I would have rather have a bigger device with two dedicated wheels [...] +so that I would always know which one to use and immediately turn the right one. +________________________________________________________________________ At FrOSCon-25 the gear switch was integrated into the tool concept as a sub-mode entered when manipulating any setting value; Wouter afterwards questioned exactly -this point: “I'm not entirely sure about the gear switch being a sub-mode. Maybe -it's something we would like to always work for whoever uses keyboard shortcuts.” +this point: ``I'm not entirely sure about the gear switch being a sub-mode. Maybe +it's something we would like to always work for whoever uses keyboard shortcuts.'' (eMail 2025-08-26) +[role="summary"] ********************************************************************** -*Agreed* -- the gear switch as uniform concept for graduated movement, +Agreed:: the gear switch as uniform concept for graduated movement, with context-dependent gear scales and an unobtrusive indicator overlay; -simplifiable down to plain two-level acceleration. + -*Open* -- activation model: sub-mode of tools vs. always available; -whether dedicated per-gear bindings should exist in parallel; the actual -gear scales per context (“the spread, as any cyclist knows”). +simplifiable down to plain two-level acceleration. +Open:: the actual handling +- activation model: sub-mode of tools vs. always available +- whether dedicated per-gear bindings should exist in parallel +- to what degree and in which scope a gear setting is remembered +- the actual gear scales per context (``the spread matters, as any cyclist knows''). +- presumably requires prototyping to determine its feasibility ********************************************************************** @@ -871,32 +1040,36 @@ the track order to control layering; rather, this ordering is just the default rule, and can be overridden in the placement of some enclosing track. ________________________________________________________________________ -Tracks in Lumiera form a _tree_ (the “fork”), and each sequence carries its own +Tracks in Lumiera form a _tree_ (»the Fork«), and each sequence carries its own local track tree -- which rules out one homogeneous global track order, but turns -tracks into a cheap, flexible resource. Sub-tracks then become “a fluid local -working space”: if moving a clip creates an overlap, a rule can resolve it by +tracks into a cheap, flexible resource. Sub-tracks then become ``a fluid local +working space'': if moving a clip creates an overlap, a rule can resolve it by moving the clip to an adjacent sub-track; sub-tracks carry no meaning by default, unless a placement explicitly ties to them (eMail 2025-04-07). In the same spirit, a collision while dragging could _shear_ a clip aside: two sibling sub-tracks are created on the fly, each overlapping clip put on one of them, inheriting the original track's placement settings such as panning or fading -(eMail 2025-10-04, cf. <>). The track head, incidentally, is understood -as the visual representation of the fork's placement -- common settings like fade -levels or mixing mode live there. +(eMail 2025-10-04, -> xref:moving[moving clips]). The track head, incidentally, +is understood as the visual representation of the fork's placement -- common +settings like fade levels or mixing mode live there. + +.Resolving a collision by _shearing aside_ +image::{imgg}/wflow/groupClip-2.2025-04-16.jpg[width="55%", alt="Collision resolution"] What is lost by giving up global tracks: the ability to toggle or mute whole groups of content by track. The replacement follows an audio-workflow insight -- -routing by _tagging_ the material (see <>). +routing by -> xref:tagging[_tagging_ the material]. At FrOSCon-25, a concrete mechanism for controlled layering was discussed: allow the user to configure, per track, that all content stays fixed _above a reference clip_ -- promising greater freedom of track arrangement, with the caveat recorded in the meeting summary that practical tests are needed to determine feasibility. -.Vertical screen real estate -The hard practical problem attached to all of this is _vertical space_. Wouter, -who dedicates one monitor entirely to the timeline, formulated the concern for -expanded nested sequences: +Vertical screen real estate +^^^^^^^^^^^^^^^^^^^^^^^^^^^ +The hard practical problem inherent in this is the use of _vertical screen space_. +Wouter, who dedicates one monitor exclusively to the timeline in his personal work setup, +has expressed his concerns regarding extended nested sequences as follows: [quote, Wouter, eMail 2025-04-08] ________________________________________________________________________ @@ -908,24 +1081,38 @@ keeping the overview and at the cost of another gesture that needs to be performed. ________________________________________________________________________ -During the IRC chat, Hermann shared quick sketches indicating a possible -reconciliation: a “group clip” _is_ a fork sub-tree in the session model, but +During the IRC chat, Hermann shared quick sketches indicating ways to approach +these problems: a ``group clip'' _is_ a fork sub-tree in the session model, but its contents could be rendered inline, sharing the surrounding track space rather than opening a separate lane. -// EDITORIAL: the hand-drawn sketches (clip1..4.jpg, 2025-04-16) exist in the -// project archive; clip3/clip4 illustrate this point -- decide whether to -// publish them alongside this digest. -The discussion has *not yet fully expanded* into this topic; it stands as one -of the most consequential open areas. +.A »virtual clip« with a nested sub-fork +image::{imgg}/wflow/groupClip-1.2025-04-16.jpg[width="60%",alt="Sub-fork in clip"] + +.A group-clip (virtual clip) can sit anywehere in the fork +image::{imgg}/wflow/groupClip-3.2025-04-16.jpg[width="50%",alt="Group clip in the fork"] + +.It _might be possible_ to establish common use of sub-tracks... +image::{imgg}/wflow/groupClip-4.2025-04-16.jpg[width="65%", alt="using common tracks"] + +...however, many aspects of such a feature still need clarification; the +merely technical representation within the Session model likely being +_not_ the most difficult part... +The further ongoing discussion has *not yet fully expanded* into this topic; +it stands as one of the most consequential open areas. + +[role="summary"] ********************************************************************** -*Agreed* -- layering order decoupled from track order (track sequence as -default rule); tracks as tree and cheap resource; sub-tracks as fluid -working space without inherent meaning. + -*Open* -- the UI mechanics of vertical space: inline rendering of nested -content, stepwise expansion, balanced timeline (cf. <>); the -above-reference-clip layering mechanism needs a practical test; controls -for explicitly overriding layering order. +Agreed:: layering order decoupled from track order (using the track sequence +as default rule); tracks as tree and cheap resource; sub-tracks as fluid +working space without inherent meaning. +Open:: +- the UI mechanics of vertical space +- inline rendering of nested content +- stepwise expansion, +- balanced -> xref:sections[global timeline sections] +- the above-reference-clip layering mechanism needs a practical test +- controls for explicitly overriding layering order. ********************************************************************** @@ -933,11 +1120,13 @@ for explicitly overriding layering order. Organising footage: bins, tags, and tag-based routing ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This thread connects Wouter's chapter 1 (source material) with the routing -question. The pivotal idea arose from combining two observations. Hermann, -from audio work experience: music, ambience, foley arrive from distinct sources -anyway, so routing to sub-mixes can be established automatically if the material -is _tagged_ (eMail 2025-04-07). Wouter then drew the radical conclusion: +This topic connects Wouter's [underline]#chapter 1# (source material) with the +routing question. The pivotal idea arose from combining two observations. +Hermann, drawing from his audio work experience: music, ambience, +https://en.wikipedia.org/wiki/Foley_(sound_design)[foleys] +will be integrated from distinct sources anyway, so routing to sub-mixes +can be established automatically if the material is _tagged_ (eMail 2025-04-07). +Wouter then took that to an even more radical conclusion: [quote, Wouter, eMail 2025-04-18] ________________________________________________________________________ @@ -948,30 +1137,33 @@ clips organised in the project view and we know how to route them as well in a single action. It could be useful for video clips too. ________________________________________________________________________ -Organising and routing thus collapse into one act -- a striking example of the -“consistent scheme solving additional problems en passant” pattern that recurs -in this discussion. +Organising and routing thus collapse into one act -- so this yet again connects +to the recurring theme of using a ``consistent scheme solving additional problems +en passant'', that resurfaced time and again during that discussion. -From the Proposals document, chapter 1 contributes further points treated as part -of the discussion (link:WorkflowProposals.html#\_proposals[»Proposals«]): support -_most_ organising systems side by side, since they don't interfere (source -timelines, rating, tagging, marking); metadata-based search and filtering as -must-haves; a native 1--5 star rating working on clip sections as well as whole -clips; bins-in-bins rather than bins-in-folders; automatic source reels based on -user-defined rules (“show me all clips from the second day of the shoot from -camera A”); watch-folder auto-import; a built-in metadata browser for sound -libraries; and *timeline versioning* -- snapshots without manual duplicate-and- -rename archaeology, which resonates directly with Lumiera's event-log -architecture (see <>). +From the _Workflow Proposals_ document, [underline]#chapter 1# contributes further +link:WorkflowProposals.html#\_proposals[conclusions and proposals], which have +been woven into the discussion thereafter: support _most_ organising systems side by side, +since they don't interfere (source timelines, rating, tagging, marking); metadata-based +search and filtering as must-haves; a native 1...5 star rating working on clip sections +as well as whole clips; bins-in-bins rather than bins-in-folders; automatic source reels +based on user-defined rules (``show me all clips from the second day of the shoot from +camera A''); watch-folder auto-import; a built-in metadata browser for sound +libraries; and *timeline versioning* -- snapshots without manual +duplicate-and-rename archaeology, which would directly tie into Lumiera's +->xref:_event_sourcing_and_cqrs[event-log based architecture]. +[role="summary"] ********************************************************************** -*Agreed* -- tagging as the organising principle, potentially replacing +Agreed:: _Tagging_ as organising principle, potentially replacing bins; tags double as routing directives (sub-mixes, groups); metadata -search, rating, source timelines to be supported. + -*Open* -- the whole Routing area (default rules, explicit wiring/“plugs”, +search, rating, source timelines to be supported. +Open:: the whole Routing area (default rules, explicit wiring/»plugs«, mixing groups) has not really been discussed yet; the tag-based scheme needs elaboration: tag taxonomy, manual effort, interplay with automatic -detection. +detection. Furthermore, all the functions for organising material within the +session should be regarded more as indications of ``features we'd like to have'', +as the handling of »assets« in Lumiera is still at a very early, conceptual stage. ********************************************************************** @@ -983,74 +1175,129 @@ Grouping devices: groups, linked clips, nested sequences ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The question how to group and encapsulate parts of an edit was identified early -as _the_ pivotal decision -- “the choices made here will very much influence all -other aspects of working in the timeline” (Wouter, eMail 2025-04-02) -- and its +as _the_ pivotal decision -- ``the choices made here will very much influence all +other aspects of working in the timeline'' (Wouter, eMail 2025-04-02) -- and its treatment shows the discussion at its most productive: two independently developed models converged step by step. .Wouter's proposal -In his document and in the mail of 2025-04-06, Wouter proposed grouping as a -first-class citizen of the timeline: every clip is part of a group -- groups -spring into existence automatically, since “if there's no automatic way for -group creation then few people will put in the effort [...] they will just start -to cut and not think about it”. The operational hierarchy: sections > groups > -clips. Three modes govern the interplay (“connect: none | clips | everything”, -or equivalently “sync: none | local | global”): from familiar destructive -Premiere/Resolve behaviour, through FCP-like magnetic behaviour _per group_, -up to global sync where the entire timeline ripples together. Group boundaries -are drawn in the timeline itself, so the active mode is always visible -- born -from a real annoyance: “what disrupts my workflow in Premiere and Resolve is -when I have to take my eyes off the timeline to look for buttons like ‘linked -selection’”. +In his document and in the ensuing discussion in April 2025, Wouter proposed grouping as a +first-class citizen of the timeline: every clip is part of a group. + +[quote, Wouter, eMail 2025-04-06] +________________________________________________________________________ +[...] I'd like to point out that the way I imagine it now, clips on layers within a group will occupy +(or ``share'') the same layers on the timeline as the clips from other groups. + +In order for this proposal to work, each clip on a timeline should be part of a group. +This way there will always be at least one group and the user can choose to append new clips to this group, +or to create a new group. If there's no automatic way for group creation then I think few people will +put in the effort to create groups manually (they will just start to cut and not think about it). + +Then, within one timeline, the operational hierarchy could be: + +- sections + + * groups + + ** clips + +Where clips, as you pointed out, could either contain source clips, or titles, +but also other groups/sequences, which can be opened in a timeline of their own +(if these too were to be displaying directly in the parent timeline I'm afraid +the complexity will get out of hand.) + +Moving on to how this could actually work within the GUI: at the moment I think +we will need to provide three modes of operation within the timeline: + +[underline]#connect: none | clips | everything# + +Or a different name for the same could be: + +[underline]#sync: none | local | global# + +[underline]#No sync / connect none#: this will practically turn off group boundaries and make +the timeline behave similar to Premiere or Resolve (in their default modes/tools). +Destructive, yes, but familiar to most editors and it will give users the freedom +to move clips wherever they like. + +[underline]#Local sync / connect clips#: this will enable ripple trimming and easy reordering +of clips within groups. Pretty much like FCP's magnetic timeline, but per group +and actually only per layer within a group (so probably no clip connections here). +If the group you're working in gets shorter or longer, all other groups +on the timeline will stay in place. + +[underline]#Global sync / connect everything#: this mode will keep everything in sync. +If you ripple trim a single clip, the entire timeline, including other groups, +will move with it and stay in sync. + +About the visual presentation of these modes: what disrupts my workflow in Premiere and Resolve +is when I have to take my eyes off the timeline to look for buttons like ``linked selection'', +to check if this mode is toggled on or off. +I would suggest to make it visually clear in the timeline which mode we're in. +No sync would mean not displaying group boundaries. +Local sync would mean displaying them as in my mockup picture, and global sync... +not sure yet, maybe change the colors of the boundaries? + +Groups can be selected by either clicking an empty space within the group +(if there is any), or clicking the group header. [...]] + +I'm thinking we need to have the option for groups to overlap each other +without erasing the group or clips that fall underneath. +________________________________________________________________________ .Hermann's counter-position -Hermann recognised his own plans in the proposal -- “what you describe as a -'group' would be modelled in Lumiera as a Sequence, with its own track tree, -but bound as virtual medium within a clip” (eMail 2025-04-05) -- but challenged +Hermann recognised his own plans in the proposal -- ``what you describe as a +`group' would be modelled in Lumiera as a Sequence, with its own track tree, +but bound as virtual medium within a clip'' (eMail 2025-04-05) -- but challenged the fixed three-level hierarchy: [quote, Hermann, eMail 2025-04-07] ________________________________________________________________________ -What I rather like to question / criticise is that we have three "kinds of -things" here, which are all grouping devices. Sections, groups, clips. Also, +What I rather like to question / criticise is that we have three ``kinds of +things'' here, which are all grouping devices. Sections, groups, clips. Also, the fact that we define a fixed hierarchy bears the danger to go into the direction of a fixed model structure. Could we instead get away with only a single grouping device in the model? [...] If we _conceptually_ define our model such that each clip is again a *sequence*, then we attain full -self-similarity and can throw away lots of "special features" present in many +self-similarity and can throw away lots of ``special features'' present in many audio or video editing software. On the implementation level, the key trick is that you do not need to _represent_ things which are not relevant. ________________________________________________________________________ -Wouter defended the fixed structure as _roles_ rather than kinds -- “on a +Wouter defended the fixed structure as _roles_ rather than kinds -- ``on a technical level, all three of these could be sequences [...] it keeps things -simple enough that many actions can be performed fast” -- while acknowledging -the appeal: “I'm intrigued by your quest for everything to be flexible, but I -also wonder if it comes at a cost to the user.” (eMail 2025-04-08) +simple enough that many actions can be performed fast'' -- while acknowledging +the appeal: ``I'm intrigued by your quest for everything to be flexible, but I +also wonder if it comes at a cost to the user.'' (eMail 2025-04-08) .Convergence -Two weeks later, Wouter proposed the resolution -- groups and nested sequences +Two weeks later, Wouter proposed a resolution -- groups and nested sequences are different things serving different purposes: [quote, Wouter, eMail 2025-04-18] ________________________________________________________________________ +[...] you briefly spoke about possibly uniting our approaches to grouping clips, +and I am beginning to think that they might actually safely co-exist. +Many NLE's have a feature called ``linked clips'', usually used to keep a video +and audio clip together through a toggle called ``linked selection''. +But often you can manually link any combination of clips to treat them as a group. My proposal for group clips might actually be closer related to this linking -[the 'linked clips' feature of many NLEs] than to nested sequences, and could -perhaps be thought of as an evolution of the linked clips concept. [...] In my -proposal every clip always belongs to a group. The moment you add a clip to the -timeline your action will decide whether it will be added to an existing group, +than to nested sequences, and could perhaps be thought of as an evolution of +the linked clips concept. [...] + +In my proposal every clip always belongs to a group. The moment you add a clip +to the timeline your action will decide whether it will be added to an existing group, or if a new group will be created. So I'd like to immediately establish the relations between clips when they are added, and not leave it as a thing to do later. [...] This way, the groups and nested sequences can both exist and have their own purposes. ________________________________________________________________________ -The second half of the convergence came from the placement side: once clips are -connected by placements (see <>), grouping _emerges_ from these -relations -- clips with relative placement act as “group leaders”, and an -implicit grouping scheme arises from the direct relationships of neighbouring -clips (Hermann, eMail 2025-08-29). Wouter, initially wary of unintended automatic -connections, arrived at the same point: +The second half of the convergence arose from exploring the consequences of +the placement concept: once clips are -> xref:prototypes[connected by placements], +grouping _emerges_ from these relations -- clips with relative placement act as +``group leaders'', and an implicit grouping scheme arises from the direct relationships +of neighbouring clips (Hermann, eMail 2025-08-29). Wouter, initially wary of +unintended automatic connections, arrived at the same point: [quote, Wouter, eMail 2025-10-03] ________________________________________________________________________ @@ -1060,25 +1307,31 @@ also see how we might not need too many other grouping devices when groups are built using clip connections. ________________________________________________________________________ -At FrOSCon-25 this was consolidated: rather than introducing a dedicated -light-weight grouping device, placements achieve the same effect; what remains -unresolved is the _visual representation_. On that front the discussion has -concrete warnings and leads: no NLE offers a model to copy -- “we need to write -down several ideas and just create mockups” -- and colour is largely spent -already, since clip colourisation is a vital organisational tool for editors; -lines and borders between or around clips are the more promising direction +The discussion ensuing after FrOSCon-25 consolidated this line of thought: +rather than introducing a dedicated light-weight grouping device, placements +can achieve the same effect; what remains unresolved is the _visual representation_. +On that front the discussion has concrete warnings and leads: no NLE offers a model +to copy -- ``we need to write down several ideas and just create mockups'' -- and +colour as visual cue is largely spent already, since clip colourisation is a vital +organisational tool for editors; thus lines and borders between or around clips +are the more promising direction (Wouter, eMail 2025-08-26). +[role="summary"] ********************************************************************** -*Agreed* -- no heavy dedicated group object: grouping arises from clip +Agreed:: no heavy dedicated group object: grouping arises from clip connections (placements), complemented by nested sequences (a sequence bound as virtual medium in a clip) for true encapsulation; relations are established at add-time, not as an afterthought; the group leader must be -clearly visible. + -*Open* -- the visual language for connections and groups (mockups needed); -interplay of implicit groups with explicit multi-selection; how group -selection works across control systems (click empty space / group header / -step-in-out navigation were sketched in eMail 2025-04-06). +clearly visible. +Open:: + - the visual language for connections and groups (mockups needed) + - interplay of implicit groups with explicit multi-selection + - how group selection works across control systems + (click empty space / group header / step-in-out navigation were + sketched in eMail 2025-04-06). + - the actual ``UI mechanics'' to ensure that clips added to the timeline + indeed get connected into clusters when applicable. ********************************************************************** @@ -1089,7 +1342,7 @@ Placement prototypes and their behaviour To make placements workable in the UI, the discussion (from FrOSCon-25 onward) settled on offering a small set of *placement prototypes* -- pre-configured placement setups, chosen per clip, adjustable through the contextual tool -palette. Hermann prefers the term “prototype” over “archetype”, referencing the +palette. Hermann prefers the term ``prototype'' over ``archetype'', referencing the prototype-based type systems of JavaScript and Python: a new instance is created by cloning the prototype (eMail 2025-08-29). @@ -1100,18 +1353,18 @@ The set was worked out in Hermann's mail of 2025-08-29 and condensed by Wouter with adjustable offset; behaves for all practical purposes like absolute positioning, and can replicate traditional NLE behaviour or truly lock clips in time. -2. *Magnetic* -- anchored at the end of “the preceding element”, without naming +2. *Magnetic* -- anchored at the end of ``the preceding element'', without naming it explicitly; the default for a continuous series of clips. A non-zero offset creates a gap -- or an overlap, which can be a transition. 3. *Content-anchored* -- anchored to a specific point _in the media content_ of another element (a beat in the music, an action in another clip); technically - a point on a “synthetic local time axis”, which extends the concept to any + a point on a ``synthetic local time axis'', which extends the concept to any element with a temporal dimension. -Wouter's design maxim was accepted as guiding principle: “the fewer, the better. +Wouter's design maxim was accepted as guiding principle: ``the fewer, the better. Few prototypes will make it easier to understand what is going on, and we'll need fewer visual cues on the timeline. [...] The prototypes you proposed seem -sufficient to me, and if not, we can always expand later.” (eMail 2025-10-01) +sufficient to me, and if not, we can always expand later.'' (eMail 2025-10-01) .Behaviour under editing operations The mail of 2025-08-29 analysed systematically what happens to _attached_ clips @@ -1119,7 +1372,11 @@ when their anchor is edited: [options="header",cols="3,4,4,4"] |==================================================================== -| anchor is... | moved | trimmed | slipped +| anchor is... +| moved +| trimmed +| slipped + | relative (anchor = start) | attached clips shift by the same delta | nothing happens (start unaffected) @@ -1144,7 +1401,7 @@ defined -- [quote, Hermann, eMail 2025-08-29] ________________________________________________________________________ So, for example, if the anchor point coincided with the clash of the cymbals in -"The Man Who Knew Too Much", and we edit away that clash from the music track -- +»The Man Who Knew Too Much«, and we edit away that clash from the music track -- then, in a strictly mathematical sense, the anchor point as such remains well defined [...] so the attached clips should remain anchored to that very point where the cymbals will clash and the murder is expected to happen. @@ -1155,51 +1412,70 @@ The chat session settled ground rules that held throughout: - Manipulating a *multi-selection* with mixed placement types never silently changes the placement kinds -- it adjusts _offset values_ only, with the - minimum of necessary adjustment, so that “no clip jumps around just wild - and uncontrolled” (Hermann). + minimum of necessary adjustment, so that ``no clip jumps around just wild + and uncontrolled'' (Hermann). - Test case: clips attached to a music clip are box-selected (without the music) and dragged. Comply and adjust their offsets -- or refuse to move them? Both agreed on complying; in Wouter's words, the refusing variant - “would lead to agitation”. + ``would lead to agitation''. - Locking a clip in place is a placement property, reachable via a properties panel with a visual indication on the clip -- rather than many clickable - decorations on the clip itself, which Wouter warned against. + decorations on the clip itself, which Wouter cautioned against. - And the reassuring perspective for newcomers: with sane automatic setup and - offset-only dragging, “you'd probably not even notice this placement concept - at start” (Hermann) -- “yes that would be fine” (Wouter). + offset-only dragging, ``you'd probably not even notice this placement concept + at start'' (Hermann) -- ``yes that would be fine'' (Wouter). .Requirements for the UI -From Hermann's mail of 2025-04-07, a requirements list that FrOSCon-25 confirmed: -a visual indicator for the prevalent placement modes; a well-made placement -editor (wizard-like shortcuts plus direct control); capturing the _derivation_ -when a position is resolved, so the user can be told *why* a clip sits where it -sits; detecting and visually indicating collisions in the resolution process; -and capturing explicit collision resolutions as events -- permanent, reviewable, -alterable. +Early on, Hermann outlined a list of requirements for the representation of +placements in the UI, that were elaborated further and confirmed at the +FrOSCon-25 meeting: + +[quote, Hermann, eMail 2025-04-07] +________________________________________________________________________ +Just to indicate some (for me obvious) relevant further points to consider: + +- we need a visual indicator for the most prevalent placement modes +- we need a really well-made placement editor element, which should include + some wizzard-like shortcuts, but also allow the user to exert direct control +- we need to capture the reasoning / the so called »derivation« when + resolving a position, so that we can give the user the necessary diagnostics +- we need to detect when some kind of collision happens in the resolution + process and also indicate that visually +- we need to capture any explicit collision resolution as event, so that + it is permanent and can also be reviewed and altered later. +________________________________________________________________________ .Known open edge cases -Wouter's catalogue (eMail 2025-10-01) stands largely unresolved: +Wouter's catalogue of problematic situations (eMail 2025-10-01) stands largely unresolved: - Clips moving out of the way to prevent overwrites may _break_ magnetic connections. - Swapping a relative-placed clip with a magnetic clip can fling the magnetic clip towards its distant anchor; should placement types swap along -- and - “would users accept that the software changes their clip settings?” + ``would users accept that the software changes their clip settings?'' - Does a magnetic clip dragged across its neighbour inherit that neighbour's offset? - Track time offsets (as DAWs use for latency compensation): should tracks - ever carry them? + ever carry them? And what would be the effect of marking such an offset + to its nested contents? - Deleting an anchor: what happens to attached clips (relative case)? A magnetic clip whose predecessor disappears reattaches to the previous one -- but what if there is none? +[role="summary"] ********************************************************************** -*Agreed* -- three prototypes (relative / magnetic / content-anchored); -“the fewer, the better”; offset-only adjustment for selections; prototype -choice via contextual tool palette; placement diagnostics (“why is this -clip here”) as requirement; discoverability via properties panel. + -*Open* -- the edge-case semantics listed above; the visual language for -placement types (colour is largely unavailable -- see <>). +Agreed:: + - three prototypes (relative / magnetic / content-anchored) + - ``the fewer, the better'' + - offset-only adjustment for selections + - prototype choice via contextual tool palette + - placement diagnostics (``why is this clip here'') as requirement + - discoverability via properties panel +Open:: + - the edge-case semantics listed above + - the visual language for placement types + - the interplay with -> xref:grouping-devices[grouping devices] + - how to establish the prototype when -> xref:adding[adding clips] ********************************************************************** @@ -1207,9 +1483,10 @@ placement types (colour is largely unavailable -- see <>). Sections and the narrative structure of the timeline ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Wouter's document proposes to divide the timeline into named *sections* -(link:WorkflowProposals.html#\_organising_the_timeline_sections[»Organising the -timeline: sections«]): headers in the ruler, background colours for orientation, +In the chapter titled +link:WorkflowProposals.html#\_organising_the_timeline_sections[»Organising the timeline: sections«], +Wouter's document proposes to divide the timeline into named *sections*: +mark their headers in the ruler, use background colours for orientation, keyboard navigation between sections -- and, crucially, encapsulation for the sake of _sync safety_: working inside one section cannot throw other sections out of sync; sections can be time-locked (e.g. when cut to music); their order @@ -1217,19 +1494,18 @@ can be changed by drag and drop, so scenes can be rearranged freely -- a feature squarely aimed at the free-flowing editor persona; sections could even carry version alternatives of a scene. This proposal carries particular weight for Wouter, and its motivation -- a top-level handle on the _narrative_ structure -- -was emphasised repeatedly in the discussion. +was emphasised repeatedly throughout the discussion. -At FrOSCon-25, the counterpart position was worked out: implementing sections -as a dedicated timeline feature would require considerable effort while not -fitting the Lumiera core design as a homogeneous building block. Hermann -proposed *nested sequences* as the implementing mechanism -- already planned for -the session model: every nested sequence has its own track tree; a transition -can be built between adjacent chapters while retaining the ability to reshuffle -them -- “we can thus solve two problems with the same feature”. The sync-safety -aspect of sections, meanwhile, is covered by placement connections -(<>) -- and Wouter's own “4th mode” consideration had already pointed -that way: sections as the means to keep some parts locked while others ripple -(eMail 2025-04-06). +At and after the FrOSCon-25 meeting, the alternative approach was explored: +implementing sections as a _dedicated timeline feature_ would require considerable +effort while not fitting the Lumiera core design as a homogeneous building block. +Hermann thus proposed *nested sequences* as the implementing mechanism -- these +are already planned for the session model: every such nested sequence has its own +track tree; a transition can be built between adjacent chapters while retaining the +ability to reshuffle them -- ``we can thus solve two problems with the same feature''. +Since each nested sequence is self-contained, the sync-safety aspect of sections would +be achieved as a by-product, while each such nested sequence could again be connected +through a suitable placement to some overall global music track. The cost of encapsulation, however, is *opacity* -- and this is where Wouter's gravest reservations live: @@ -1245,42 +1521,49 @@ Nested sequences exist in Premiere and Resolve, yet I rarely use them for exactly this reason. ________________________________________________________________________ -(His two exceptions: compositing stacks meant to be seen as one clip, and -grouped music stems.) +Incidentally, Wouter mentions two exceptions: compositing stacks that are +meant to be seen as one clip, and grouped music stems. The elements of an answer, collected across the discussion: - Three graded gestures -- *expand* (reveal internals inline), *collapse* (show a condensed overview rendering, with thumbnails, produced by Lumiera's - own render engine and cache), and *open* (a dedicated tab with a timeline - created on the fly for the nested sequence); cutting such a clip yields two - clips referring to the _same_ underlying sequence (Hermann, eMail 2025-04-07). -- FrOSCon-25: stepwise expansion, so the overall timeline remains balanced, and - a condensed preview of a virtual clip's internal structure, to avoid having to - open it in many situations. + own render engine and cache), and *open* (to show the contents as a dedicated + tab with a timeline created on the fly for the nested sequence) +- a nested session would be attached into a »virtual clip« -- cutting such a clip + yields two clips referring _to the same underlying sequence_ (Hermann, eMail 2025-04-07). +- FrOSCon-25: use a stepwise expansion, so the overall timeline remains balanced, + and a condensed preview of a virtual clip's internal structure, to avoid having + to open it in many situations. - Wouter's *popup portal* idea: hovering a nested clip (or dwelling on it with the keyboard focus) opens, after a brief delay, a popup that acts as a window into the nested sequence -- with the mouse, one could even edit its contents directly, without stepping in; though he flagged the misclick risk himself when the popup appears exactly where the user intends to click (eMails 2025-04-18, 2025-08-26). -- Hermann's inversion of the question: “in which kind of editing situations do +- Hermann's inversion of the question: ``in which kind of editing situations do we actually want to _collapse_ a virtual nested clip, as opposed to have it all open for editing like a segment, maybe just slightly condensed - vertically?” (eMail 2025-08-29) -- which leads directly into the vertical - space problem (<>). + vertically?'' (eMail 2025-08-29) -- which leads directly into the + -> xref:layering[vertical space problem]. +[role="summary"] ********************************************************************** -*Agreed* -- the narrative-structure need is accepted as important; nested -sequences are the intended implementing mechanism, doubling as chapter -transition device; sync-safety comes from placement connections; condensed -content previews instead of black boxes. + -*Open* -- whether nested sequences can actually deliver the lightweight, -always-visible quality of Wouter's sections (drag-to-reorder scenes, -section headers in the ruler, per-section versioning); the gradual -expand/collapse steps; the popup portal; vertical screen real estate. -This cluster is the least-resolved of the Grouping area and a prime -candidate for the next in-person discussion. +Agreed:: + - the narrative-structure need is accepted as important + - nested sequences are the intended implementing mechanism, + also doubling as chapter transition device + - sync-safety comes from placement connections + - condensed content previews instead of black boxes +Open:: + - whether nested sequences can actually deliver the lightweight, + always-visible quality of Wouter's sections (drag-to-reorder scenes, + section headers in the ruler, per-section versioning) + - precise definition of the gradual expand/collapse steps + - the popup portal idea needs elaboration + - vertical screen real estate. +Agenda:: This cluster is the least-resolved of the Grouping area +and a prime candidate for the next in-person discussion. ********************************************************************** @@ -1292,24 +1575,24 @@ Adding clips and automatic placement setup ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If placements are to carry the editing experience, the decisive moment is when -a clip _enters_ the timeline: which placement prototype does it get, and how much +a clip _enters the timeline:_ which placement prototype will it select, and how much does the user have to do about it? This question dominated the last months of the -2025 discussion, and is the point where it paused. +2025 discussion, and is the point where that discussion eventually paused. Wouter framed the option space (eMail 2025-10-06): setup *by context* (the surroundings determine the new clip's placement), *last-used setting*, or a -*fixed default* -- his least favourite, “because it encourages users to stick -with one type as much as possible”. All agree on the goal: “we want whichever -option results in the least amount of user actions to get a desired result.” +*fixed default* -- his least favourite, ``because it encourages users to stick +with one type as much as possible''. All agree on the goal: ``we want whichever +option results in the least amount of user actions to get a desired result.'' Hermann invoked the mainstream UI guideline -- avoid explicit options if an automatic rule covers most cases and can be adjusted after the fact -- and sketched the automatic rule via *proximity* (eMail 2025-10-08): close to a clip -on the left ⇒ magnetic; close to a clip above/below ⇒ content-anchored; else -parent-relative. The tuning spectrum ranges from “engage whenever any clip lies -in that direction” (if the auto mode itself is gated by a gesture -- like -switching on a magnet and letting things fall into place) to “engage only when -really close” -- a *hooking* feature. +on the left => magnetic; close to a clip above/below => content-anchored; else +parent-relative. The tuning spectrum ranges from ``engage whenever any clip lies +in that direction'' (which would be acceptable only if the auto mode itself is +_gated by a gesture -- like switching on a magnet toggle icon and letting things +fall into place) to ``engage only when really close'' -- a *hooking* feature. Wouter accepted proximity for the magnetic case but drew a line for content-anchoring: @@ -1324,13 +1607,13 @@ or below that you do not (immediately) want to anchor to. Hooking by hovering will probably be engaged many times by accident. ________________________________________________________________________ -His fallback suggestion -- make parent-relative the global default, giving +His fallback suggestion -- make ``parent-relative'' the global default, giving near-traditional NLE behaviour for everyone -- provoked Hermann's power-of-defaults argument: [quote, Hermann, eMail 2025-11-09] ________________________________________________________________________ -If we make the "parent relative" the default [...] then we can expect that most +If we make the ``parent relative'' the default [...] then we can expect that most people will setup most of their clips with that mode, and then they need indeed a traditional ripple trim feature. Thus I'm inclined that we should rather try to work out a very good solution for an automatic mode of placement selection, @@ -1341,54 +1624,105 @@ cases with B-Roll and sound effects better. ________________________________________________________________________ The B-Roll build-up scenario served as recurring proof of concept: the first -B-Roll clip becomes a group anchor; further B-Roll snaps magnetically to its -end; sound effects and subtitles are content-anchored on neighbouring tracks; +B-Roll clip becomes a group anchor; further B-Roll material snaps magnetically +to its end; sound effects and subtitles are content-anchored on neighbouring tracks; the whole compound can then be dragged as a single entity (eMail 2025-08-29). .The lifetime of a clip -Hermann's summary of 2025-11-09 organised the accumulated mechanics into a state -model: when first added, a clip is set up either by the *automatic mode* (1) or, -via modifier/toggle, as a *traditional* separate clip (2). Once dropped, it has -a placement, and manipulation happens either *with placements engaged* (3) -- -grabbing moves a whole compound -- or as *offset adjustment* (4) -- the placement -kind stays, only the offset changes (a magnetic clip dragged away from its -neighbour remains magnetically linked, now with an offset). Multi-selections -combine with either (move several clusters together, or offset several clips by -the same amount). A dedicated function re-enters placement-kind selection -- -cycling through the possibilities, or via the tool palette. And the stages -connect: a clip _lifted out_ of the timeline is again an isolated clip in hand, -“as if we'd picked it from the media bins” (eMail 2025-12-07). +Hermann's summarised the interaction mechanics discussed thus far, and +integrated their use into a state model: + +[quote, Hermann, eMail 2025-11-09] +________________________________________________________________________ +We have defined now several different ``mechanics'' how clips interact +when being manipulated. But we have not decided yet upon the way how +to choose between these different possible "mechanics". + +Thus I'd like to summarise or list those different mechanics +we discussed thus far: + +1. the automatic setup mode. You drag a clip and it connects somehow + to neighbouring clips. This is also the mode where the idea of + ``hooking by proximity'' could be integrated, if desired + +2. the traditional behaviour: each clip is separate and thus + gets the ``relative to parent track'' placement, which behaves + effectively as if the clip was just positioned at a fixed point + within the track + +3. dragging (and even trimming / editing) with placements engaged. + If we grab one clip, a whole compound moves alongside. We have + discussed several possible flavours, like that you have to grab + the group leader, and effects work ``hierarchically down'' or, + alternatively that a compound of clips is glued together, + so that you can grab any part to move the whole compound + +4. adjusting offsets. When I grab and drag a clip in that mode, + it does not change its placement mode, rather it adjusts an + offset relative to the existing kind of placement. For example, + I could drag away a ``magnetic'' clip from its left neighbour, + and it would still remain magnetically linked, just now with + an offset. + +5. what happens if we have a multi-selection (either of several + clips added explicitly to a multi-selection, or by a box-select)? + In theory, we could combine that both with mode (3) or mode (4) + + ** (5+3) means that we can mark several clusters and move them together + ** (5+4) means that we can offset several clips by the same amount + +Expanding onto these ideas and reasoning: a clip would go through several +states during its lifetime. When added first, we would be either in mode (1) +or when using a modifier/toggle, in mode (2). + +Once the clip was dropped onto the timeline space, it has got its placement, +and now it would transition either into (3) or (4), again selected by a +toggle or modifier system. + +And we could have a modifier or function, which brings a clip back +into the selection of the placement kind. And then it would probably +perform some kind of cycle through the possibilities, as you proposed. +And of course, since we show those possibilities also on the tool +palette, the user could always just click onto another one. +________________________________________________________________________ + + +And those stages will mesh up: a clip _lifted out_ of the timeline is again +an isolated clip in hand, ``as if we'd picked it from the media bins'' (eMail 2025-12-07). .The distinctive gesture -Both agree plain hovering cannot trigger anchoring -- “it must be a distinctive -gesture” (Hermann, eMail 2025-12-07). Three candidate designs are on the table: +Both agree that plain hovering cannot trigger anchoring -- ``it must be a distinctive +gesture'' (Hermann, eMail 2025-12-07). Three candidate designs are on the table: 1. *Context rule at drop location only* (Hermann): very close left predecessor ⇒ snap magnetically; clip on the immediately adjacent track ⇒ content-anchor; correct afterwards via the palette. 2. *The hooking gesture* (Hermann, preferred by him): default is parent-relative; while dragging, moving _over_ a target clip merely highlights it -- but a - marked change of the horizontal movement direction right above it (“hook it - in there!”) triggers the attachment, confirmed by an overlay, after which the + marked change of the horizontal movement direction right above it (``hook it + in there!'') triggers the attachment, confirmed by an overlay, after which the clip is dragged on to its destination. Additionally: dropping a clip directly onto the _seam_ between two clips inserts it in-between, connecting all three - magnetically. Wouter: “This is an interesting idea, although quite + magnetically. Wouter: ``This is an interesting idea, although quite unconventional. I would like to try this (in whichever way possible), to see - how that would feel.” (eMail 2025-12-11) + how that would feel.'' (eMail 2025-12-11) 3. *Arrow connect* (Wouter): after dropping a clip onto another clip, an arrow appears between them, the dropped clip sticks to the mouse, and a click places it at its intended position -- a two-step process, but triggered only in that - specific constellation (eMail 2025-12-11; refining his earlier idea of an - arrow anchored to the new clip, from eMail 2025-11-25). + specific constellation (eMail 2025-12-11; thereby refining his earlier idea of + an arrow anchored to the new clip, from eMail 2025-11-25). +[role="summary"] ********************************************************************** -*Agreed* -- placement setup must happen without explicit user choice in -the common cases; automatic magnetic snapping to a close predecessor; -content-anchoring only by deliberate act; the clip lifetime state model; -correction always possible via the contextual tool palette. + -*Open* -- the decisive design choice between (and possible combination of) +Agreed:: + - placement setup must happen without explicit user choice in the common cases + - automatic magnetic snapping to a close predecessor + - content-anchoring only by deliberate act + - correction always possible via the contextual tool palette + - the clip lifetime state model +Open:: the decisive design choice between (and possible combination of) the three candidate gestures; this is where the discussion paused in -December 2025 and should resume -- ideally with something tangible to try. +December 2025 and should resume. ********************************************************************** @@ -1415,35 +1749,35 @@ simply move the group leader. Everything else will move with it. It just needs to be visually clear which clip is that group leader. ________________________________________________________________________ -Hermann adopted it on the spot: “Hey, that's brilliant, I didn't see this as a +Hermann adopted it on the spot: ``Hey, that's brilliant, I didn't see this as a possible concept [...] Hierarchical is more generic and encompasses that case, -you just need to grab the right thing.” What remains from the modifier idea is +you just need to grab the right thing.'' What remains from the modifier idea is the *temporary detach*: a modifier (or toggle) to move a clip or selection as an isolated case, ignoring its connections -- analogous to Final Cut's tilde key and -the classic ‘linked selection’ toggle (Wouter, eMails 2025-10-01, 2025-10-04). +the classic ``linked selection'' toggle (Wouter, eMails 2025-10-01, 2025-10-04). .Collisions: block or make way? Enumerating the logical possibilities for one clip moved into another -- block; push aside; overlap above/below; shear aside into adjacent track space; repel -and flip order -- Hermann proposed “no magic” as default: move within free space, -block on obstacles, and never reconfigure things or create chains of ripple -effects without an explicit indication by the user (snapping-resistance and -tear-away being the familiar analogy). Wouter pushed back hard: +and flip order -- Hermann proposed ``no magic'' as default: move within free space, +block on obstacles, and never reconfigure things or create chains of secondary +ripple effects without an explicit indication by the user (snapping-resistance +and tear-away being the familiar analogy). Wouter pushed back hard: [quote, Wouter, eMail 2025-10-04] ________________________________________________________________________ I don't think blocking is a good idea. Let clips move to another track. At the very least, let a user push through a blockade. Adobe Premiere has blocking behavior in its trimming system and it's annoying. There's a reason I'm trying -to do something, so the app shouldn't tell me 'no'. DaVinci Resolve never -blocks. It 'understands' what I'm trying to do, and it figures out a way to +to do something, so the app shouldn't tell me ``_no_''. DaVinci Resolve never +blocks. It ``understands'' what I'm trying to do, and it figures out a way to let me do it in a way that makes sense in relation to the rest of the timeline. This saves a lot of hassle with turning sync locks on and off. I strongly prefer that. ________________________________________________________________________ -Hermann acknowledged the concern without conceding the default (“A very valid -point. I do not know if I agree to it, really. Could be!”); Wouter suggested +Hermann acknowledged the concern without conceding the default (``A very valid +point. I do not know if I agree to it, really. Could be!''); Wouter suggested trying Kdenlive's default mode as a specimen of blocking behaviour. The question -- and the related one whether clips move out of the way during a drag (risking broken magnetic connections when they do) -- remains open. @@ -1453,54 +1787,69 @@ A load-bearing workflow requirement still lacks its mechanism: easy reordering of clips, FCP's signature move (drag a clip elsewhere; the rest makes space; the gap closes). Under pure placement semantics, dragging a magnetic clip moves everything attached behind it -- useful for trimming, but not what reordering -wants: “While a user might want magnetic placement for trimming, they might not +wants: ``While a user might want magnetic placement for trimming, they might not want it for moving clips. So we need to distinguish: dragging while respecting -all placements -- and dragging while ignoring the selected clip's children.” +all placements -- and dragging while ignoring the selected clip's children.'' (Wouter, eMail 2025-11-25) His earlier question marks the same spot: if moving -the anchor moves the attached clips, “wouldn't moving them reorder their +the anchor moves the attached clips, ``wouldn't moving them reorder their positions? [...] If not done by moving magnetic clips, how else could we -reorder?” (eMail 2025-10-01) +reorder?'' (eMail 2025-10-01) .The modifier budget -Modes and modifiers are “a very limited resource” (Hermann, eMail 2025-10-04); -Wouter began the bookkeeping (eMail 2025-12-11): keep *Shift* = add to selection -and *Alt* = duplicate for convention's sake; *Ctrl* becomes “the common -modifier” switching to the alternative behaviour shown on the contextual palette -(the Gimp pattern, see <>); Avid-style snapping can be a toggle -instead; Premiere's Ctrl-box-select-of-cuts is probably dispensable. +Modes and modifiers are ``a very limited resource'' (Hermann, eMail 2025-10-04); +Wouter began the bookkeeping (eMail 2025-12-11): keep *`Shift`* ≙ add to selection +and *`Alt`* ≙ duplicate for convention's sake; *`Ctrl`* becomes ``the common modifier'' +switching to the alternative behaviour shown on the contextual palette +(the Gimp pattern, -> xref:tools-modes[tools and modes]); Avid-style snapping can be +a toggle instead; Premiere's Ctrl-box-select-of-cuts is probably dispensable. .Terminology and adjacent decisions -Settled: *Lift* = take a clip out and leave a gap; *Extract* = take it out and -close the gap (Wouter, eMail 2025-12-11) -- with Wouter defending the usefulness -of lift-with-gap in track-based contexts (protecting timings until the gap is -closed deliberately). Also settled en passant: dropping a clip from the bin onto -an existing timeline clip _replaces_ it, taking over position and connections, -with the length adjusting to the new clip (eMails 2025-10-05/06). And: no brick -wall at timecode zero -- Lumiera has no internal timecode (opaque 64-bit -µs-granularity time; timecode is mere presentation), so negative positions are -unproblematic and the user can relocate the zero point on the time ruler. +Some terms were explained and settled (Wouter, eMail 2025-12-11): + +Lift:: take a clip out and leave a gap +Extract:: take it out and close the gap + +-- with Wouter defending the usefulness of lift-with-gap in track-based contexts +(protecting timings until the gap is closed deliberately). + +[underline]#Also settled en passant#: _dropping a clip from the bin onto an existing timeline clip_ +will *replace the target*, taking over position and connections, with the length +adjusting to the new clip (eMails 2025-10-05/06). + +[underline]#Furthermore#: no brick wall _at timecode zero_ -- Lumiera is not based on _timecode_ +internally (rather on an opaque 64-bit µs-granularity time -- timecode is mere presentation), +so negative positions are unproblematic and the user can relocate +the zero point on the time ruler when required. .Ripple trim -- parked -Wouter repeatedly noted that a general ripple-trim function would be “handy in -any case”, to serve users replicating traditional behaviour. Hermann sees -classic ripple trim as a workaround “for lack of a better alternative, since the -user needs a way to keep sync between adjacent tracks, without having proper -means to define the relationship between these tracks” -- and as a second -behaviour pattern subtly deviating from what placements provide -(eMail 2025-11-09). Outcome: “I understand your reasoning for not wanting a -ripple function, so let's park that for now.” (Wouter, eMail 2025-11-25) +Wouter repeatedly noted that a general ripple-trim function would be ``handy in any case'', +to serve users replicating traditional behaviour. Hermann sees classic ripple trim as a +workaround ``for lack of a better alternative, since the user needs a way to keep sync +between adjacent tracks, without having proper means to define the relationship +between these tracks'' -- and thus implementing ripple-trim would introduce a +duplicated behaviour pattern, yet subtly deviating from what placements provide +(eMail 2025-11-09). Outcome: ``I understand your reasoning for not wanting a +ripple function, so let's park that for now.'' (Wouter, eMail 2025-11-25) +[role="summary"] ********************************************************************** -*Agreed* -- hierarchy principle: parents move children, behaviour follows -placement, not mode; visible group leader; temporary-detach modifier; -Ctrl as the universal alternative-action modifier; lift/extract -terminology; replace-by-drop semantics; no timecode brick wall. + -*Open* -- blocking vs. never-blocking as default collision behaviour; -whether/when clips move out of the way during drags; the reordering -mechanism under magnetic placement; drag horizontal vs. vertical: -identical connection semantics were proposed (Wouter, eMail 2025-12-11) -but overlap reactions remain to be defined. + -*Parked* -- a general ripple-trim function. +Agreed:: + - hierarchy principle: parents move children, + behaviour follows placement, not mode + - visible group leader + - temporary-detach modifier + - Ctrl as the universal alternative-action modifier + - lift/extract terminology + - replace-by-drop semantics + - no timecode brick wall +Open:: + - blocking vs. never-blocking as default collision behaviour + - whether/when clips move out of the way during drags + - the reordering mechanism under magnetic placement + - drag horizontal vs. vertical: identical connection semantics + were proposed (Wouter, eMail 2025-12-11) + but overlap reactions remain to be defined +Parked:: discussion regarding a general ripple-trim function. ********************************************************************** @@ -1508,9 +1857,9 @@ but overlap reactions remain to be defined. + Trimming ~~~~~~~~ -“Trimming is a topic in itself, one of the most complex indeed” (Wouter) -- -“yes, but it is the very heart of an editor's work. Thus for me it is the litmus -test.” (Hermann, IRC 2025-04-16). Both also agree on its systematic priority: +``Trimming is a topic in itself, one of the most complex indeed'' (Wouter) -- +``yes, but it is the very heart of an editor's work. Thus for me it is the litmus +test.'' (Hermann, IRC 2025-04-16). Both also agree on its systematic priority: [quote, Wouter, eMail 2025-04-25] ________________________________________________________________________ @@ -1519,76 +1868,92 @@ topic by itself. Because once we understand how we would like trimming, navigation, arranging, grouping, adding and removing clips will work, we will know which gestures we need for those, and then see how similar gestures will work in other panels and contexts. [...] I'm inclined to have the timeline -operations take the lead here, and then adapt to other panels (or 'extract' +operations take the lead here, and then adapt to other panels (or ``extract'' general gestures out of these), because the timeline is the one area that can not be compromised. ________________________________________________________________________ The IRC chat explored the mechanics of trim-side selection: most NLEs select a _cut_ (nearest the playhead) and pick its A or B side; only Lightworks selects -the side of an underlying _clip_. Wouter prefers the clip-based approach -- -“since we already have a mechanism for navigating clips, we should probably not -add another mechanism just for navigating cuts” -- and his document turns this -into a concrete proposal -(link:WorkflowProposals.html#\_analysis_and_proposal[»Analysis and proposal«]): +the side of an underlying _clip_. + +Wouter prefers the clip-based approach -- ``since we already have a mechanism +for navigating clips, we should probably not add another mechanism just for +navigating cuts'' -- and his document turns this into a +link:WorkflowProposals.html#_analysis_and_proposal[concrete proposal]: three commands on the selected clip (select in point / out point as trim side; select in point for a roll edit, toggling to the out point on second press), with repeated presses switching between ripple and non-ripple trims. Trimming -itself “doesn't need to be reinvented”: frame-wise, by fixed amounts, or +itself ``doesn't need to be reinvented'': frame-wise, by fixed amounts, or dynamically through playback. Since the trim keys collide with clip-nudge keys, -a *trim mode* is accepted -- but designed as a contextual mode (see -<>): entered by selecting a trim side, exited by clicking empty -timeline space or the mode key; playback in trim mode always previews the -selected cut, Avid-style. At FrOSCon-25 the sub-mode arrangement was agreed: -switching between trim, roll, slide and slip happens on the tool palette after -activating the edit tool. +a *trim mode* is accepted -- yet would be designed as a contextual mode +(-> xref:tools-modes[tools and modes]): entered by selecting a trim side, +exited by clicking empty timeline space or the mode key; playback in trim mode +always previews the selected cut, Avid-style. At FrOSCon-25 the sub-mode arrangement +was agreed upon: switching between trim, roll, slide and slip happens on the +tool palette after activating the edit tool. -Two connections to the placement world deserve emphasis: +Two connections to the placement concept deserve emphasis: -- Wouter's document proposal *renounces sync locks* -- “we previously established +- Wouter's document proposal *renounces sync locks* -- ``we previously established that clips that have no relation to each other might share a track, so it makes little sense to provide track based operations. We should instead take - the actual clip connections that the user establishes into account.” This is - the trimming-side mirror of the placement concept, and quietly removes the - need for multi-trim-side asymmetric operations. -- The behaviour of connected clips _under_ trimming is exactly the prototype - behaviour matrix of <> -- e.g. trimming a magnetic chain ripples - by construction, no sync-lock required. + the actual clip connections that the user establishes into account.'' This + effectively applies the placement patterns onto the trimming behaviour, and + quietly removes most of the need for multi-trim-side asymmetric operations. +- The behaviour of connected clips _under trimming_ is what was outlined in the + -> xref:prototypes[prototype behaviour matrix] -- e.g. trimming a magnetic chain + ripples by construction, no no sync-lock is required. -In the mail discussion proper, trimming was touched but never given its own deep -dive -- Wouter's mutually-exclusive-selections question is still standing: should -clip selection and trim-side selection exclude each other (saving precious keys), -and how does that interact with sticky selections without introducing yet -another mode (eMail 2025-04-25)? +In the mail discussion proper, trimming was touched frequently, but never given its +own deep dive -- Wouter's mutually-exclusive-selections question is still standing: +should clip selection and trim-side selection exclude each other (saving precious keys), +and how does that interact with sticky selections without introducing yet another mode? +(eMail 2025-04-25) +[role="summary"] ********************************************************************** -*Agreed* -- timeline operations take the lead in gesture design; clip-based +Agreed:: timeline operations take the lead in gesture design; clip-based trim-side selection; trim mode as fluid contextual mode with palette sub-modes (trim/roll/slide/slip); no sync locks -- clip connections govern -ripple behaviour. + -*Open* -- a dedicated deep dive is still missing: the interplay of trim +ripple behaviour. + +Open:: a dedicated deep dive is still missing: the interplay of trim gestures with placement connections in detail, keyboard bindings, the clip-selection vs. trim-side-selection question, dynamic (playback) trimming. Together with Routing, this is the least-covered core area. ********************************************************************** +{nbsp} + State of the discussion ----------------------- Looking back over the year, the discussion has produced a remarkably coherent body of design: a shared vision (flow, trackless aspiration, medium abstraction), -a settled conceptual core (three placement prototypes, grouping through -connections plus nested sequences, the contextual tool palette, the gesture -concept with its two-priority feedback scheme), a set of working principles -(map down / simplify down, the Gimp pattern, the modifier budget, power of -defaults) -- and a well-delimited frontier of open questions. +a settled conceptual core (three placement prototypes, grouping through connections +plus nested sequences, the contextual tool palette, the gesture concept with its +two-priority feedback scheme), and a set of working principles -The discussion paused in December 2025 at a well-defined point: the choice of -the concrete mechanism for attaching clips while adding and arranging them -(<>). The largest uncharted areas are Routing (<>, <>), -the vertical-space / sections cluster (<>), and a systematic treatment -of Trimming (<>). +- minimise the number of design elements +- map down / simplify down to known software behavior +- the Gimp modifier pattern +- modifiers as limited resource +- power of defaults -A separate page compiles these open points into a working agenda for the -planned meeting at FrOSCon 2026: link:Froscon26Agenda.html[FrOSCon-26 Agenda]. + +The discussion paused in December 2025 when discussing the choice of +the concrete mechanism for attaching clips while +-> xref:adding[adding and arranging them] -- +and this opens a well-delimited frontier of further questions to explore. + +The largest uncharted areas are xref:_routing[Routing] +(xref:tagging[tagging], xref:layering[layering]), +the xref:sections[vertical-space, grouping and sections] cluster of topics, +and a systematic treatment of xref:trimming[trimming]. + +'''' + +TIP: A separate page compiles these open points into a selection for a possible +working agenda for the planned link:Froscon26Agenda.html[meeting at FrOSCon 2026]. diff --git a/doc/design/workflow/Verwijlen/WorkflowProposals.txt b/doc/design/workflow/Verwijlen/WorkflowProposals.txt index cd778d74d..e615d8ea5 100644 --- a/doc/design/workflow/Verwijlen/WorkflowProposals.txt +++ b/doc/design/workflow/Verwijlen/WorkflowProposals.txt @@ -711,6 +711,7 @@ mode/tool system. FCP's Select and Position tools also change the behaviour of t can therefore indeed also be considered modes, which gives us a hybrid design as well. Specialised views can be combined with tools or modes. +[[contextual_bar]] To conclude: all methods have pros and cons. Tools, modes and views aren't mutually exclusive, so I think we should use whichever method or combination of methods is appropriate for different actions in Lumiera. Whenever we can, we should try to avoid all three and design the app to be as @@ -721,7 +722,7 @@ side automatically enters trim mode. However, a user does still need to explicit Regardless of the method, immediate visual differentiation of tools/modes/views is important and I'd like this to extend to any timeline option that can be toggled on/off. Therefore I'd like to propose -the use of a _contextual bar_ that appears over the bottom part of the timeline whenever a contextual +the use of a *contextual bar* that appears over the bottom part of the timeline whenever a contextual mode is active. .Contextual bar with options for the mode/tool