# Engine - Building a Render Nodes Network from Objects in the EDL

loading Proc-Layer devel doku ...
Requires Javascript.
Welcome to TiddlyWiki created by Jeremy Ruston, Copyright © 2007 UnaMesa Association

    Background: #fff
    Foreground: #000
    PrimaryPale: #8cf
    PrimaryLight: #18f
    PrimaryMid: #04b
    PrimaryDark: #014
    SecondaryPale: #ffc
    SecondaryLight: #fe8
    SecondaryMid: #db4
    SecondaryDark: #841
    TertiaryPale: #eee
    TertiaryLight: #ccc
    TertiaryMid: #999
    TertiaryDark: #666
    Error: #f88

    /*{{{*/
    body {background:[[ColorPalette::Background]]; color:[[ColorPalette::Foreground]];}

    a {color:[[ColorPalette::PrimaryMid]];}
    a:hover {background-color:[[ColorPalette::PrimaryMid]]; color:[[ColorPalette::Background]];}
    a img {border:0;}

    h1,h2,h3,h4,h5,h6 {color:[[ColorPalette::SecondaryDark]]; background:transparent;}
    h1 {border-bottom:2px solid [[ColorPalette::TertiaryLight]];}
    h2,h3 {border-bottom:1px solid [[ColorPalette::TertiaryLight]];}

    .button {color:[[ColorPalette::PrimaryDark]]; border:1px solid [[ColorPalette::Background]];}
    .button:hover {color:[[ColorPalette::PrimaryDark]]; background:[[ColorPalette::SecondaryLight]]; border-color:[[ColorPalette::SecondaryMid]];}
    .button:active {color:[[ColorPalette::Background]]; background:[[ColorPalette::SecondaryMid]]; border:1px solid [[ColorPalette::SecondaryDark]];}

    .header {background:[[ColorPalette::PrimaryMid]];}
    .headerShadow {color:[[ColorPalette::Foreground]];}
    .headerShadow a {font-weight:normal; color:[[ColorPalette::Foreground]];}
    .headerForeground {color:[[ColorPalette::Background]];}
    .headerForeground a {font-weight:normal; color:[[ColorPalette::PrimaryPale]];}

    .tabSelected{color:[[ColorPalette::PrimaryDark]];
    	background:[[ColorPalette::TertiaryPale]];
    	border-left:1px solid [[ColorPalette::TertiaryLight]];
    	border-top:1px solid [[ColorPalette::TertiaryLight]];
    	border-right:1px solid [[ColorPalette::TertiaryLight]];
    }
    .tabUnselected {color:[[ColorPalette::Background]]; background:[[ColorPalette::TertiaryMid]];}
    .tabContents {color:[[ColorPalette::PrimaryDark]]; background:[[ColorPalette::TertiaryPale]]; border:1px solid [[ColorPalette::TertiaryLight]];}
    .tabContents .button {border:0;}

    #sidebar {}
    #sidebarOptions input {border:1px solid [[ColorPalette::PrimaryMid]];}
    #sidebarOptions .sliderPanel {background:[[ColorPalette::PrimaryPale]];}
    #sidebarOptions .sliderPanel a {border:none;color:[[ColorPalette::PrimaryMid]];}
    #sidebarOptions .sliderPanel a:hover {color:[[ColorPalette::Background]]; background:[[ColorPalette::PrimaryMid]];}
    #sidebarOptions .sliderPanel a:active {color:[[ColorPalette::PrimaryMid]]; background:[[ColorPalette::Background]];}

    .wizard {background:[[ColorPalette::PrimaryPale]]; border:1px solid [[ColorPalette::PrimaryMid]];}
    .wizard h1 {color:[[ColorPalette::PrimaryDark]]; border:none;}
    .wizard h2 {color:[[ColorPalette::Foreground]]; border:none;}
    .wizardStep {background:[[ColorPalette::Background]]; color:[[ColorPalette::Foreground]];
    	border:1px solid [[ColorPalette::PrimaryMid]];}
    .wizardStep.wizardStepDone {background::[[ColorPalette::TertiaryLight]];}
    .wizardFooter {background:[[ColorPalette::PrimaryPale]];}
    .wizardFooter .status {background:[[ColorPalette::PrimaryDark]]; color:[[ColorPalette::Background]];}
    .wizard .button {color:[[ColorPalette::Foreground]]; background:[[ColorPalette::SecondaryLight]]; border: 1px solid;
    	border-color:[[ColorPalette::SecondaryPale]] [[ColorPalette::SecondaryDark]] [[ColorPalette::SecondaryDark]] [[ColorPalette::SecondaryPale]];}
    .wizard .button:hover {color:[[ColorPalette::Foreground]]; background:[[ColorPalette::Background]];}
    .wizard .button:active {color:[[ColorPalette::Background]]; background:[[ColorPalette::Foreground]]; border: 1px solid;
    	border-color:[[ColorPalette::PrimaryDark]] [[ColorPalette::PrimaryPale]] [[ColorPalette::PrimaryPale]] [[ColorPalette::PrimaryDark]];}

    #messageArea {border:1px solid [[ColorPalette::SecondaryMid]]; background:[[ColorPalette::SecondaryLight]]; color:[[ColorPalette::Foreground]];}
    #messageArea .button {color:[[ColorPalette::PrimaryMid]]; background:[[ColorPalette::SecondaryPale]]; border:none;}

    .popupTiddler {background:[[ColorPalette::TertiaryPale]]; border:2px solid [[ColorPalette::TertiaryMid]];}

    .popup {background:[[ColorPalette::TertiaryPale]]; color:[[ColorPalette::TertiaryDark]]; border-left:1px solid [[ColorPalette::TertiaryMid]]; border-top:1px solid [[ColorPalette::TertiaryMid]]; border-right:2px solid [[ColorPalette::TertiaryDark]]; border-bottom:2px solid [[ColorPalette::TertiaryDark]];}
    .popup hr {color:[[ColorPalette::PrimaryDark]]; background:[[ColorPalette::PrimaryDark]]; border-bottom:1px;}
    .popup li.disabled {color:[[ColorPalette::TertiaryMid]];}
    .popup li a, .popup li a:visited {color:[[ColorPalette::Foreground]]; border: none;}
    .popup li a:hover {background:[[ColorPalette::SecondaryLight]]; color:[[ColorPalette::Foreground]]; border: none;}
    .popup li a:active {background:[[ColorPalette::SecondaryPale]]; color:[[ColorPalette::Foreground]]; border: none;}
    .popupHighlight {background:[[ColorPalette::Background]]; color:[[ColorPalette::Foreground]];}
    .listBreak div {border-bottom:1px solid [[ColorPalette::TertiaryDark]];}

    .tiddler .defaultCommand {font-weight:bold;}

    .shadow .title {color:[[ColorPalette::TertiaryDark]];}

    .title {color:[[ColorPalette::SecondaryDark]];}
    .subtitle {color:[[ColorPalette::TertiaryDark]];}

    .toolbar {color:[[ColorPalette::PrimaryMid]];}
    .toolbar a {color:[[ColorPalette::TertiaryLight]];}
    .selected .toolbar a {color:[[ColorPalette::TertiaryMid]];}
    .selected .toolbar a:hover {color:[[ColorPalette::Foreground]];}

    .tagging, .tagged {border:1px solid [[ColorPalette::TertiaryPale]]; background-color:[[ColorPalette::TertiaryPale]];}
    .selected .tagging, .selected .tagged {background-color:[[ColorPalette::TertiaryLight]]; border:1px solid [[ColorPalette::TertiaryMid]];}
    .tagging .listTitle, .tagged .listTitle {color:[[ColorPalette::PrimaryDark]];}
    .tagging .button, .tagged .button {border:none;}

    .footer {color:[[ColorPalette::TertiaryLight]];}
    .selected .footer {color:[[ColorPalette::TertiaryMid]];}

    .sparkline {background:[[ColorPalette::PrimaryPale]]; border:0;}
    .sparktick {background:[[ColorPalette::PrimaryDark]];}

    .error, .errorButton {color:[[ColorPalette::Foreground]]; background:[[ColorPalette::Error]];}
    .warning {color:[[ColorPalette::Foreground]]; background:[[ColorPalette::SecondaryPale]];}
    .lowlight {background:[[ColorPalette::TertiaryLight]];}

    .zoomer {background:none; color:[[ColorPalette::TertiaryMid]]; border:3px solid [[ColorPalette::TertiaryMid]];}

    .imageLink, #displayArea .imageLink {background:transparent;}

    .annotation {background:[[ColorPalette::SecondaryLight]]; color:[[ColorPalette::Foreground]]; border:2px solid [[ColorPalette::SecondaryMid]];}

    .viewer .listTitle {list-style-type:none; margin-left:-2em;}
    .viewer .button {border:1px solid [[ColorPalette::SecondaryMid]];}
    .viewer blockquote {border-left:3px solid [[ColorPalette::TertiaryDark]];}

    .viewer table, table.twtable {border:2px solid [[ColorPalette::TertiaryDark]];}
    .viewer th, .viewer thead td, .twtable th, .twtable thead td {background:[[ColorPalette::SecondaryMid]]; border:1px solid [[ColorPalette::TertiaryDark]]; color:[[ColorPalette::Background]];}
    .viewer td, .viewer tr, .twtable td, .twtable tr {border:1px solid [[ColorPalette::TertiaryDark]];}

    .viewer pre {border:1px solid [[ColorPalette::SecondaryLight]]; background:[[ColorPalette::SecondaryPale]];}
    .viewer code {color:[[ColorPalette::SecondaryDark]];}
    .viewer hr {border:0; border-top:dashed 1px [[ColorPalette::TertiaryDark]]; color:[[ColorPalette::TertiaryDark]];}

    .highlight, .marked {background:[[ColorPalette::SecondaryLight]];}

    .editor input {border:1px solid [[ColorPalette::PrimaryMid]];}
    .editor textarea {border:1px solid [[ColorPalette::PrimaryMid]]; width:100%;}
    .editorFooter {color:[[ColorPalette::TertiaryMid]];}

    #backstageArea {background:[[ColorPalette::Foreground]]; color:[[ColorPalette::TertiaryMid]];}
    #backstageArea a {background:[[ColorPalette::Foreground]]; color:[[ColorPalette::Background]]; border:none;}
    #backstageArea a:hover {background:[[ColorPalette::SecondaryLight]]; color:[[ColorPalette::Foreground]]; }
    #backstageArea a.backstageSelTab {background:[[ColorPalette::Background]]; color:[[ColorPalette::Foreground]];}
    #backstageButton a {background:none; color:[[ColorPalette::Background]]; border:none;}
    #backstageButton a:hover {background:[[ColorPalette::Foreground]]; color:[[ColorPalette::Background]]; border:none;}
    #backstagePanel {background:[[ColorPalette::Background]]; border-color: [[ColorPalette::Background]] [[ColorPalette::TertiaryDark]] [[ColorPalette::TertiaryDark]] [[ColorPalette::TertiaryDark]];}
    .backstagePanelFooter .button {border:none; color:[[ColorPalette::Background]];}
    .backstagePanelFooter .button:hover {color:[[ColorPalette::Foreground]];}
    #backstageCloak {background:[[ColorPalette::Foreground]]; opacity:0.6; filter:'alpha(opacity:60)';}
    /*}}}*/

    /*{{{*/
    * html .tiddler {height:1%;}

    body {font-size:.75em; font-family:arial,helvetica; margin:0; padding:0;}

    h1,h2,h3,h4,h5,h6 {font-weight:bold; text-decoration:none;}
    h1,h2,h3 {padding-bottom:1px; margin-top:1.2em;margin-bottom:0.3em;}
    h4,h5,h6 {margin-top:1em;}
    h1 {font-size:1.35em;}
    h2 {font-size:1.25em;}
    h3 {font-size:1.1em;}
    h4 {font-size:1em;}
    h5 {font-size:.9em;}

    hr {height:1px;}

    a {text-decoration:none;}

    dt {font-weight:bold;}

    ol {list-style-type:decimal;}
    ol ol {list-style-type:lower-alpha;}
    ol ol ol {list-style-type:lower-roman;}
    ol ol ol ol {list-style-type:decimal;}
    ol ol ol ol ol {list-style-type:lower-alpha;}
    ol ol ol ol ol ol {list-style-type:lower-roman;}
    ol ol ol ol ol ol ol {list-style-type:decimal;}

    .txtOptionInput {width:11em;}

    #contentWrapper .chkOptionInput {border:0;}

    .externalLink {text-decoration:underline;}

    .indent {margin-left:3em;}
    .outdent {margin-left:3em; text-indent:-3em;}
    code.escaped {white-space:nowrap;}

    .tiddlyLinkExisting {font-weight:bold;}
    .tiddlyLinkNonExisting {font-style:italic;}

    /* the 'a' is required for IE, otherwise it renders the whole tiddler in bold */
    a.tiddlyLinkNonExisting.shadow {font-weight:bold;}

    #mainMenu .tiddlyLinkExisting,
    	#mainMenu .tiddlyLinkNonExisting,
    	#sidebarTabs .tiddlyLinkNonExisting {font-weight:normal; font-style:normal;}
    #sidebarTabs .tiddlyLinkExisting {font-weight:bold; font-style:normal;}

    .header {position:relative;}
    .header a:hover {background:transparent;}
    .headerShadow {position:relative; padding:4.5em 0em 1em 1em; left:-1px; top:-1px;}
    .headerForeground {position:absolute; padding:4.5em 0em 1em 1em; left:0px; top:0px;}

    .siteTitle {font-size:3em;}
    .siteSubtitle {font-size:1.2em;}

    #mainMenu {position:absolute; left:0; width:10em; text-align:right; line-height:1.6em; padding:1.5em 0.5em 0.5em 0.5em; font-size:1.1em;}

    #sidebar {position:absolute; right:3px; width:16em; font-size:.9em;}
    #sidebarOptions {padding-top:0.3em;}
    #sidebarOptions a {margin:0em 0.2em; padding:0.2em 0.3em; display:block;}
    #sidebarOptions input {margin:0.4em 0.5em;}
    #sidebarOptions .sliderPanel {margin-left:1em; padding:0.5em; font-size:.85em;}
    #sidebarOptions .sliderPanel a {font-weight:bold; display:inline; padding:0;}
    #sidebarOptions .sliderPanel input {margin:0 0 .3em 0;}
    #sidebarTabs .tabContents {width:15em; overflow:hidden;}

    .wizard {padding:0.1em 1em 0em 2em;}
    .wizard h1 {font-size:2em; font-weight:bold; background:none; padding:0em 0em 0em 0em; margin:0.4em 0em 0.2em 0em;}
    .wizard h2 {font-size:1.2em; font-weight:bold; background:none; padding:0em 0em 0em 0em; margin:0.4em 0em 0.2em 0em;}
    .wizardStep {padding:1em 1em 1em 1em;}
    .wizard .button {margin:0.5em 0em 0em 0em; font-size:1.2em;}
    .wizardFooter {padding:0.8em 0.4em 0.8em 0em;}
    .wizardFooter .status {padding:0em 0.4em 0em 0.4em; margin-left:1em;}
    .wizard .button {padding:0.1em 0.2em 0.1em 0.2em;}

    #messageArea {position:fixed; top:2em; right:0em; margin:0.5em; padding:0.5em; z-index:2000; _position:absolute;}
    .messageToolbar {display:block; text-align:right; padding:0.2em 0.2em 0.2em 0.2em;}
    #messageArea a {text-decoration:underline;}

    .tiddlerPopupButton {padding:0.2em 0.2em 0.2em 0.2em;}
    .popupTiddler {position: absolute; z-index:300; padding:1em 1em 1em 1em; margin:0;}

    .popup {position:absolute; z-index:300; font-size:.9em; padding:0; list-style:none; margin:0;}
    .popup .popupMessage {padding:0.4em;}
    .popup hr {display:block; height:1px; width:auto; padding:0; margin:0.2em 0em;}
    .popup li.disabled {padding:0.4em;}
    .popup li a {display:block; padding:0.4em; font-weight:normal; cursor:pointer;}
    .listBreak {font-size:1px; line-height:1px;}
    .listBreak div {margin:2px 0;}

    .tabset {padding:1em 0em 0em 0.5em;}
    .tab {margin:0em 0em 0em 0.25em; padding:2px;}
    .tabContents {padding:0.5em;}
    .tabContents ul, .tabContents ol {margin:0; padding:0;}
    .txtMainTab .tabContents li {list-style:none;}
    .tabContents li.listLink { margin-left:.75em;}

    #contentWrapper {display:block;}
    #splashScreen {display:none;}

    #displayArea {margin:1em 17em 0em 14em;}

    .toolbar {text-align:right; font-size:.9em;}

    .tiddler {padding:1em 1em 0em 1em;}

    .missing .viewer,.missing .title {font-style:italic;}

    .title {font-size:1.6em; font-weight:bold;}

    .missing .subtitle {display:none;}
    .subtitle {font-size:1.1em;}

    .tiddler .button {padding:0.2em 0.4em;}

    .tagging {margin:0.5em 0.5em 0.5em 0; float:left; display:none;}
    .isTag .tagging {display:block;}
    .tagged {margin:0.5em; float:right;}
    .tagging, .tagged {font-size:0.9em; padding:0.25em;}
    .tagging ul, .tagged ul {list-style:none; margin:0.25em; padding:0;}
    .tagClear {clear:both;}

    .footer {font-size:.9em;}
    .footer li {display:inline;}

    .annotation {padding:0.5em; margin:0.5em;}

    * html .viewer pre {width:99%; padding:0 0 1em 0;}
    .viewer {line-height:1.4em; padding-top:0.5em;}
    .viewer .button {margin:0em 0.25em; padding:0em 0.25em;}
    .viewer blockquote {line-height:1.5em; padding-left:0.8em;margin-left:2.5em;}
    .viewer ul, .viewer ol {margin-left:0.5em; padding-left:1.5em;}

    .viewer table, table.twtable {border-collapse:collapse; margin:0.8em 1.0em;}
    .viewer th, .viewer td, .viewer tr,.viewer caption,.twtable th, .twtable td, .twtable tr,.twtable caption {padding:3px;}
    table.listView {font-size:0.85em; margin:0.8em 1.0em;}
    table.listView th, table.listView td, table.listView tr {padding:0px 3px 0px 3px;}

    .viewer pre {padding:0.5em; margin-left:0.5em; font-size:1.2em; line-height:1.4em; overflow:auto;}
    .viewer code {font-size:1.2em; line-height:1.4em;}

    .editor {font-size:1.1em;}
    .editor input, .editor textarea {display:block; width:100%; font:inherit;}
    .editorFooter {padding:0.25em 0em; font-size:.9em;}
    .editorFooter .button {padding-top:0px; padding-bottom:0px;}

    .fieldsetFix {border:0; padding:0; margin:1px 0px 1px 0px;}

    .sparkline {line-height:1em;}
    .sparktick {outline:0;}

    .zoomer {font-size:1.1em; position:absolute; overflow:hidden;}
    .zoomer div {padding:1em;}

    * html #backstage {width:99%;}
    * html #backstageArea {width:99%;}
    #backstageArea {display:none; position:relative; overflow: hidden; z-index:150; padding:0.3em 0.5em 0.3em 0.5em;}
    #backstageToolbar {position:relative;}
    #backstageArea a {font-weight:bold; margin-left:0.5em; padding:0.3em 0.5em 0.3em 0.5em;}
    #backstageButton {display:none; position:absolute; z-index:175; top:0em; right:0em;}
    #backstageButton a {padding:0.1em 0.4em 0.1em 0.4em; margin:0.1em 0.1em 0.1em 0.1em;}
    #backstage {position:relative; width:100%; z-index:50;}
    #backstagePanel {display:none; z-index:100; position:absolute; margin:0em 3em 0em 3em; padding:1em 1em 1em 1em;}
    .backstagePanelFooter {padding-top:0.2em; float:right;}
    .backstagePanelFooter a {padding:0.2em 0.4em 0.2em 0.4em;}
    #backstageCloak {display:none; z-index:20; position:absolute; width:100%; height:100px;}

    .whenBackstage {display:none;}
    .backstageVisible .whenBackstage {display:block;}
    /*}}}*/

    /***
    StyleSheet for use when a translation requires any css style changes.
    This StyleSheet can be used directly by languages such as Chinese, Japanese and Korean which use a logographic writing system and need larger font sizes.
    ***/

    /*{{{*/
    body {font-size:0.8em;}

    #sidebarOptions {font-size:1.05em;}
    #sidebarOptions a {font-style:normal;}
    #sidebarOptions .sliderPanel {font-size:0.95em;}

    .subtitle {font-size:0.8em;}

    .viewer table.listView {font-size:0.95em;}

    .htmlarea .toolbarHA table {border:1px solid ButtonFace; margin:0em 0em;}
    /*}}}*/

    /*{{{*/
    @media print {
    #mainMenu, #sidebar, #messageArea, .toolbar, #backstageButton {display: none ! important;}
    #displayArea {margin: 1em 1em 0em 1em;}
    /* Fixes a feature in Firefox 1.5.0.2 where print preview displays the noscript content */
    noscript {display:none;}
    }
    /*}}}*/

    <!--{{{-->
    <div class='header' macro='gradient vert [[ColorPalette::PrimaryLight]] [[ColorPalette::PrimaryMid]]'>
    <div class='headerShadow'>
    <span class='siteTitle' refresh='content' tiddler='SiteTitle'></span>&nbsp;
    <span class='siteSubtitle' refresh='content' tiddler='SiteSubtitle'></span>
    </div>
    <div class='headerForeground'>
    <span class='siteTitle' refresh='content' tiddler='SiteTitle'></span>&nbsp;
    <span class='siteSubtitle' refresh='content' tiddler='SiteSubtitle'></span>
    </div>
    </div>
    <div id='mainMenu' refresh='content' tiddler='MainMenu'></div>
    <div id='sidebar'>
    <div id='sidebarOptions' refresh='content' tiddler='SideBarOptions'></div>
    <div id='sidebarTabs' refresh='content' force='true' tiddler='SideBarTabs'></div>
    </div>
    <div id='displayArea'>
    <div id='messageArea'></div>
    <div id='tiddlerDisplay'></div>
    </div>
    <!--}}}-->

    <!--{{{-->
    <div class='toolbar' macro='toolbar closeTiddler closeOthers +editTiddler > fields syncing permalink references jump'></div>
    <div class='title' macro='view title'></div>
    <div class='subtitle'><span macro='view modifier link'></span>, <span macro='view modified date'></span> (<span macro='message views.wikified.createdPrompt'></span> <span macro='view created date'></span>)</div>
    <div class='tagging' macro='tagging'></div>
    <div class='tagged' macro='tags'></div>
    <div class='viewer' macro='view text wikified'></div>
    <div class='tagClear'></div>
    <!--}}}-->

    <!--{{{-->
    <div class='toolbar' macro='toolbar +saveTiddler -cancelTiddler deleteTiddler'></div>
    <div class='title' macro='view title'></div>
    <div class='editor' macro='edit title'></div>
    <div macro='annotations'></div>
    <div class='editor' macro='edit text'></div>
    <div class='editor' macro='edit tags'></div><div class='editorFooter'><span macro='message views.editor.tagPrompt'></span><span macro='tagChooser'></span></div>
    <!--}}}-->

    To get started with this blank TiddlyWiki, you'll need to modify the following tiddlers:
    * SiteTitle & SiteSubtitle: The title and subtitle of the site, as shown above (after saving, they will also appear in the browser title bar)
    * MainMenu: The menu (usually on the left)
    * DefaultTiddlers: Contains the names of the tiddlers that you want to appear when the TiddlyWiki is opened
    You'll also need to enter your username for signing your edits: <<option txtUserName>>

    These InterfaceOptions for customising TiddlyWiki are saved in your browser

    Your username for signing your edits. Write it as a WikiWord (eg JoeBloggs)

    <<option txtUserName>>
    <<option chkSaveBackups>> SaveBackups
    <<option chkAutoSave>> AutoSave
    <<option chkRegExpSearch>> RegExpSearch
    <<option chkCaseSensitiveSearch>> CaseSensitiveSearch
    <<option chkAnimate>> EnableAnimations

    ----
    Also see AdvancedOptions

    PageTemplate
    |>|SiteTitle - SiteSubtitle|
    |>|MainMenu|
    |DefaultTiddlers<<br>><<br>><<br>>ViewTemplate<<br>><<br>>EditTemplate|SideBarOptions|
    |~|OptionsPanel|
    |~|SideBarTabs|
    |~|AdvancedOptions|
    |~|<<tiddler Configuration.SideBarTabs>>|

    ''StyleSheet:'' StyleSheetColors - StyleSheetLayout - StyleSheetPrint

    ColorPalette

    SiteUrl

    Memory management facility for the low-level model (render nodes network). The model is organised into temporal segments, which are considered to be structurally constant and uniform. The objects within each segment are strongly interconnected, and thus each segment is being built in a single build process and is replaced or released as a whole. __~AllocationCluster__ implements memory management to support this usage pattern. He owns a number of object families of various types.[>img[draw/AllocationCluster.png]]
    * [[processing nodes|ProcNode]] &mdash; probably with several subclasses (?)
    * [[wiring descriptors|WiringDescriptor]]
    * the input/output descriptor arrays used by the latter

    To Each of those families we can expect an initially undetermined (but rather large) number of individual objects, which can be expected to be allocated within a short timespan and which are to be released cleanly on destruction of the AllocationCluster.

    ''Problem of calling the dtors''
    Even if the low-level memory manager(s) may use raw storage, we require that the allocated object's destructors be called. This means keeping track at least of the number of objects allocated (without wasting too much memory for bookkeeping). Besides, as the objects are expected to be interconnected, it may be dangerous to destroy a given family of objects while another family of objects may rely on the former in its destructor. //If we happen do get into this situation,// we need to define a priority order on the types and assure the destruction sequence is respected.

    &rarr; see MemoryManagement

    Asset management is a subsystem on its own. Assets are "things" that can be loaded into a session, like Media, Clips, Effects, Transitions. It is the "bookkeeping view", while the EDL is the "manipulation and process view". Some Assets can be //loaded// and a collection of Assets is saved with each Session. Besides, there is a collection of basic Assets always available by default.

    The Assets are important reference points holding the information needed to access external resources. For example, an Clip asset can reference a Media asset, which in turn holds the external filename from which to get the media stream. For Effects, the situation is similar. Assets thus serve two quite distinct purposes. One is to load, list, group search and browse them, and to provide an entry point to create new or get at existing MObjects in the EDL, while the other purpose is to provide attribute and property informations to the inner parts of the engine, while at the same time isolating and decoupling them from environmental details.

    We can distinguish several different Kinds of Assets, each one with specific properties. While all these Kinds of Assets implement the basic Asset interface, they in turn are the __key abstractions__ of the asset management view. Mostly, their interfaces will be used directly, because they are quite different in behaviour. Thus it is common to see asset related operations being templated on the Asset Kind.
    &rarr; see also [[Creating and registering Assets|AssetCreation]]
    [img[Asset Classess|uml/fig130309.png]]

    !Media Asset
    Some piece of Media Data accessible at some external Location and able to be processed by Lumiera. A Media File on Harddisk can be considered as the most basic form of Media Asset, with some important derived flavours, like a Placeholder for a currently unavailable Source, or Media available in different Resolutions or Formats.
    * __outward interface operations__ include querying properties, creating an Clip MObject, controlling processing policy (low res proxy placeholders, interlacing and other generic pre- and postprocessing)
    * __inward interface operations__ include querying filename, codec, offset and any other informations necessary for creating a source render node, getting additional processing policy decisions (handling of interlacing, aspect ratio).
    &rarr; MediaAsset

    !Processing Asset
    Some software component able to work on media data in the Lumiera Render engine Framework. This includes all sorts of loadable effects, as well as some of the standard, internal facilities (Mask, Projector). Note that Processing Assets typically provide some attachment Point or means of communication with GUI facilities.
    * __outward interface operations__ include...
    * __inward interface operations__ include...
    &rarr; ProcAsset {{red{to be defined}}}

    !Structural Asset
    Some of the building blocks providing the framework for the objects placed into the current Session. Notable examples are [[processing pipes|Pipe]] within the high-level-model, Viewer attachment points, Tracks, etc.
    * __outward interface operations__ include...
    * __inward interface operations__ include...
    &rarr; StructAsset {{red{to be defined}}}

    !Meta Asset
    Some additional, virtual facilities created in the course of the editing process. Examples are Automation data sets, Labels and reference points, Meta Clips (nested sub-~EDLs)
    * __outward interface operations__ include...
    * __inward interface operations__ include...
    &rarr; MetaAsset {{red{to be defined}}}

    !!!!still to be worked out..
    is how to implement the relationship between [[MObject]]s and Assets. Do we use direct pointers, or do we prefer an ID + central registry approach? And how to handle the removal of an Asset.
    &rarr; see also [[analysis of mem management|ManagementAssetRelation]]
    &rarr; see also [[Creating Objects|ObjectCreation]], especially [[Assets|AssetCreation]]

      //9/07: currently implementing it as follows: use a refcounting-ptr from Clip-~MObject to asset::Media while maintaining a dependency network between Asset objects. We'll see if this approach is viable//

    Assets are created by a Factories returning smart pointers; the Asset creation is bound to specific use cases and //only available// for these specific situations. There is no generic Asset Factory.

    For every Asset we generate a __Ident tuple__ and a long ID (hash) derived from this Ident tuple. The constructor of the abstract base class {{{Asset}}} takes care of this step and automatically registeres the new Asset object with the AssetManager. Typically, the factory methods for concrete Asset classes provide some shortcuts providing sensible default values for some of the Ident tuple data fields. They may take additional parameters &mdash; for example the factory method for creating {{{asset::Media}}} takes a filename (and may at some point in the future aply "magic" based on examination of the file &rarr; LoadingMedia)

    The Asset Manager provides an Interface to some internal Database holding all Assets in the current Session and System state. It may be a real Database at some point (and for the moment it's a Hashtable). Each [[Asset]] is registered automatically with the Asset Manager; it can be queried either by it's //identification tuple// or by it's unique ID.

    Placing an MObject relatively to another object such that it should be handled as //attached//&nbsp; to the latter results in several design and implementation challenges. Actually, such an attachment creates a cluster of objects. The typical use case is that of an effect attached to a clip or processing pipe.
    * attachment is not a globally fixed relation between objects, rather, it typically exists only for some limited time span (e.g. the duration of the basic clip the effect is attached to)
    * the order of attachment is important and the attached placement may create a fork in the signal flow, so we need a way for specifying reproducibly how the resulting wiring should be
    * when building, we access the information in reversed direction: we have the target object and need to query for all attachments

    The first step towards an solution is to isolate the problem; obviously we don't need to store the objects differently, we just need //information about attached objects//&nbsp; for some quite isolated tasks (namely for creating a GUI representation and for combining attached objects into a [[Pipe]] when building). Resorting to a query (function call) interface should turn the rest of the problem into an implementation detail. Thus
    * for an __attachment head__ (= {{{Placement<MObject>}}} to which other objects have been attached) get the ordered list of attachments
    * for an __attached placement__ (member of the cluster) get the placement of the corresponding attachment head
    * retrieve and break the attachment when //deleting.//

    !!Implementation notes
    Attachment is managed within the participating placements, mostly by special [[locating pins|LocatingPin]]. Attachment doesn't necessarily nail down an attached object to a specific position, rather the behaviour depends on the type of the object and the locating pins actually involved, especially on their order and priority. For example, if an {{{Placement<Effect>}}} doesn't contain any locating pin defining a temporal position, then the attachment will result in the placement inheriting the temporal placement of the //attachment head// (i.e. the clip this effect has been attached to). But, if on the contrary the effect in question //does// have an additional locating pin, for example relative to another object or even to a fixed time position, this one will "win" and determine the start position of the effect &mdash; it may even move the effect out of the time interval covered by the clip, in which case the attachment has no effect on the clip's processing pipe.
    The attachment relation is hierarchical and has a clearly defined //active// and //passive// side: The attachment head is the parent node in a tree, but plays the role of the passive partner, to which the child nodes attach. But note, this does not mean we are limited to a single attachment head. Actually, each placement has a list of locating pins and thus can attach to several other placements. For example, a transition attaches to at least two local pipes (clips).

    !!!!Relation to memory management
    Attachment on itself does //not// keep an object alive. Rather, it's a direct reference and has to be taken into account when deleting an object. On the other hand, processing nodes in the render engine never depend on placements &mdsash; they always refer directly to the MObject instance or even the underlying asset. In the case of MObject instances, the pointer from within the engine will //share ownership// with the placement (remember: both are derived from {{{boost::shared_ptr}}}).

    Automation is treated as a function over time. It is always tied to a specific Parameter (which can thus be variable over the course of the timeline). All details //how// this function is defined are completely abstracted away. The Parameter uses a ParamProvider to get the value for a given Time (point). Typically, this will use linear or bezier interpolation over a set of keyframes internally. Parameters can be configured to have different value ranges and distribution types (on-off, stepped, continuous, bounded)

    [img[how to implement Automation|uml/fig129669.png]]

    While generally automation is treated as a function over time, defining and providing such a function requires some //Automation Data.// The actual layout and meaning of this data is deemed an implementation detail of the [[parameter provider|ParamProvider]] used, but nevertheless an automation data set has object characteristics within the [[EDL (high-level-model)|EDL]], allowing it to be attached, moved and [[placed|Placement]] by the user.

    Starting out from the concepts of Objects, Placement to Tracks, render Pipes and connection properties (&rarr; see [[here|TrackPipeEDL]]) within the EDL, we can identify the elementary operations occuring within the Builder. Overall, the Builder is organized as application of //visiting tools// to a collection of objects, so finally we have to consider some object kind appearing in the working function of the given builder tool, which holds at this moment some //context//. The job now is to organize this context such as to create a predictable build process from this //event driven// approach.
    &rarr;see also: BuilderPrimitives for the elementary situations used to cary out the building operations

    !Builder working Situations
    # any ''Clip'' (which at this point has been reduced already to a part of a simple elementary media stream &rarr; see [[Fixture]])
    ## yields a source reading node
    ## which needs to be augmented by the underlying media's [[processing pattern|ProcPatt]]
    ##* thus inserting codec(s) and source transformations
    ##* effectively this is an application of effects
    ## at this point we have to process (and maybe generate on-the-fly) the [[source port of this clip|ClipSourcePort]]
    ##* the output of the source reading and preprocessing defined thus far is delivered as input to this port, which is done by a ~WiringRequest (see below)
    ##* as every port, it is the entry point to a [[processing pipe|Pipe]], thus the source port has a processing pattern, typically inserting the camera (transformation effect) at this point
    ## followed by the application of effects
    ##* separately for every effect chain rooted (placed) directly onto the clip
    ##* and regarding the chaining order
    ## next we have to assess the [[pipes|Pipe]] to which the clip has been placed
    ## producing a [[wiring request|WiringRequest]] for every pair {{{(chainEndpoint, pipe)}}}
    # [>img[draw/Proc.builder1.png]]   attaching an ''Effect'' is actually always an //insertion operation// which is done by //prepending// to the previously built nodes. Effects may be placed as attached to clips and pipes, which causes them to be included in the processing chain at the given location. Effects may as well be placed at an absolute time, which means they are to be applied to every clip that happens to be at this time &mdash; but this usecase will be reolved when creating the Fixture, causing the effect to be attached to the clips in question. The same holds true for Effects put on tracks.
    # treating an ''wiring request'' means
    ## detecting possible and impossible connections
    ## deriving additional possible "placement dimensions" generated by executing such an connection (e.g. connecting a mono source to a spatial sound system bus creates panning possibilities)
    ##* deriving parameter sources for this additional degrees of freedom
    ##* fire off insertion of the necessary effects to satisfy this connection request and implement the additional "placement dimensions" (pan, layer order, overlay mode, MIDI channel selection...)
    # processing the effects and further placements ''attached to a Pipe'' is handled identical to the processing done with all attachments to individual clips.
    # ''Transitions'' are to be handled differently according to their placement (&rarr; more on [[Transitions|TransitionsHandling]])
    #* when placed normally to two (or N) clips, they are inserted at the exit node of the clip's complete effect chain.
    #* otherwise, when placed to the source port(s) or when placed to some other pipes they are inserted at the exit side of those pipe's effect chains. (Note: this puts additional requirements on the transition processor, so not every transition can be placed this way)
    After consuming all input objects and satisfying all wiring requests, the result is a set of [[exit nodes|ExitNode]] ready for pulling data. We call the network reachable from such an exit node a [[Processor]], together all processors of all segments and output data types comprise the render engine.

    !!!dependencies
    Pipes need to be there first, as everything else will be plugged (placed) to a pipe at some point. The same holds true for tracks. But, on the other hand, both are optional. We can have ~EDLs with ~MObjects without configuring pipes (but won't be able to build any render processor of course). And we could have an EDL without any track, if we place every ~MObject within this EDL directly to some pipe.

    Effects can be attached only to already existing pipelines, starting out at some pipes entry port or the source port of some clip. Besides that, all further parts can be built in any order and independent of each other. This is made possible by using [[wiring requests|WiringRequest]], which can be resolved later on. So, as long as we start out with the tracks (to resolve any pipe they are placed to), and further, if we manage to get any effect placed to some clip-MO //after// setting up and treating the clip, we are fine and can do the building quasi event driven.

    !!!building and resolving
    Building the network for the individual objects thus creates a queue of wiring requests. Some of them may be immediately resolvable, but detecting this correctly can be nontrivial, and so it seems better to group all wiring requests based on the pipe and treat them groupwise. Because &mdash; in the most general case &mdash; connecting includes the use of transforming and joining nodes, which can create additional wiring requests (e.g. for automation parameter data connections). Finally, if the network is complete, we could perform [[optimisations|RenderNetworkOptimisation]]

    /***
    |Name|BetterTimelineMacro|
    |Created by|SaqImtiaz|
    |Location|http://tw.lewcid.org/#BetterTimelineMacro|
    |Version|0.5 beta|
    |Requires|~TW2.x|
    !!!Description:
    A replacement for the core timeline macro that offers more features:
    *list tiddlers with only specfic tag
    *exclude tiddlers with a particular tag
    *limit entries to any number of days, for example one week
    *specify a start date for the timeline, only tiddlers after that date will be listed.

    !!!Installation:
    Copy the contents of this tiddler to your TW, tag with systemConfig, save and reload your TW.
    Edit the ViewTemplate to add the fullscreen command to the toolbar.

    !!!Syntax:
    {{{<<timeline better:true>>}}}
    ''the param better:true enables the advanced features, without it you will get the old timeline behaviour.''

    additonal params:
    (use only the ones you want)
    {{{<<timeline better:true  onlyTag:Tag1 excludeTag:Tag2 sortBy:modified/created firstDay:YYYYMMDD maxDays:7 maxEntries:30>>}}}

    ''explanation of syntax:''
    onlyTag: only tiddlers with this tag will be listed. Default is to list all tiddlers.
    excludeTag: tiddlers with this tag will not be listed.
    sortBy: sort tiddlers by date modified or date created. Possible values are modified or created.
    firstDay: useful for starting timeline from a specific date. Example: 20060701 for 1st of July, 2006
    maxDays: limits timeline to include only tiddlers from the specified number of days. If you use a value of 7 for example, only tiddlers from the last 7 days will be listed.
    maxEntries: limit the total number of entries in the timeline.

    !!!History:
    *28-07-06: ver 0.5 beta, first release

    !!!Code
    ***/
    //{{{
    // Return the tiddlers as a sorted array
    TiddlyWiki.prototype.getTiddlers = function(field,excludeTag,includeTag)
    {
              var results = [];
              this.forEachTiddler(function(title,tiddler)
              {
              if(excludeTag == undefined || tiddler.tags.find(excludeTag) == null)
                            if(includeTag == undefined || tiddler.tags.find(includeTag)!=null)
                                          results.push(tiddler);
              });
              if(field)
                       results.sort(function (a,b) {if(a[field] == b[field]) return(0); else return (a[field] < b[field]) ? -1 : +1; });
              return results;
    }

    //this function by Udo
    function getParam(params, name, defaultValue)
    {
              if (!params)
              return defaultValue;
              var p = params[0][name];
              return p ? p[0] : defaultValue;
    }

    window.old_timeline_handler= config.macros.timeline.handler;
    config.macros.timeline.handler = function(place,macroName,params,wikifier,paramString,tiddler)
    {
              var args = paramString.parseParams("list",null,true);
              var betterMode = getParam(args, "better", "false");
              if (betterMode == 'true')
              {
              var sortBy = getParam(args,"sortBy","modified");
              var excludeTag = getParam(args,"excludeTag",undefined);
              var includeTag = getParam(args,"onlyTag",undefined);
              var tiddlers = store.getTiddlers(sortBy,excludeTag,includeTag);
              var firstDayParam = getParam(args,"firstDay",undefined);
              var firstDay = (firstDayParam!=undefined)? firstDayParam: "00010101";
              var lastDay = "";
              var field= sortBy;
              var maxDaysParam = getParam(args,"maxDays",undefined);
              var maxDays = (maxDaysParam!=undefined)? maxDaysParam*24*60*60*1000: (new Date()).getTime() ;
              var maxEntries = getParam(args,"maxEntries",undefined);
              var last = (maxEntries!=undefined) ? tiddlers.length-Math.min(tiddlers.length,parseInt(maxEntries)) : 0;
              for(var t=tiddlers.length-1; t>=last; t--)
                      {
                      var tiddler = tiddlers[t];
                      var theDay = tiddler[field].convertToLocalYYYYMMDDHHMM().substr(0,8);
                      if ((theDay>=firstDay)&& (tiddler[field].getTime()> (new Date()).getTime() - maxDays))
                         {
                         if(theDay != lastDay)
                                   {
                                   var theDateList = document.createElement("ul");
                                   place.appendChild(theDateList);
                                   createTiddlyElement(theDateList,"li",null,"listTitle",tiddler[field].formatString(this.dateFormat));
                                   lastDay = theDay;
                                   }
                      var theDateListItem = createTiddlyElement(theDateList,"li",null,"listLink",null);
                      theDateListItem.appendChild(createTiddlyLink(place,tiddler.title,true));
                      }
                      }
              }

              else
                  {
                  window.old_timeline_handler.apply(this,arguments);
                  }
    }
    //}}}

    All decisions on //how // the RenderProcess has to be carried out are concentrated in this rather complicated Builder Subsystem. The benefit of this approach is, besides decoupling of subsystems, to keep the actual performance-intensive video processing code as simple and transparent as possible. The price, in terms of increased complexity &mdash; to pay in the Builder &mdash; can be handled by making the Build Process generic to a large degree. Using a Design By Contract approach we can decompose the various decisions into small decision modules without having to trace the actual workings of the Build Process as a whole.

    [>img[Outline of the Build Process|uml/fig129413.png]]
    The building itself will be broken down into several small tool application steps. Each of these steps has to be mapped to the MObjects found on the [[Timeline]]. Remember: the idea is that the so called "[[Fixture]]" contains only [[ExplicitPlacement]]s which in turn link to MObjects like Clips, Effects and [[Automation]]. So it is sufficient to traverse this list and map the build tools to the elements. Each of these build tools has its own state, which serves to build up the resulting Render Engine. So far I see two steps to be necessary:
    * find the "Segments", i.e. the locations where the overall configuration changes
    * for each segment: generate a ProcNode for each found MObject and wire them accordingly
    Note, //we still have to work out how exactly building, rendering and playback work// together with the backend-design. The build process as such doesn't overly depend on these decisions. It is easy to reconfigure this process. For example, it would be possible as well to build for each frame separately (as Cinelerra2 does), or to build one segment covering the whole timeline (and handle everything via [[Automation]]

    &rarr;see also: [[Builder Overview|Builder]]
    &rarr;see also: BasicBuildingOperations
    &rarr;see also: BuilderStructures
    &rarr;see also: BuilderMechanics
    &rarr;see also: PlanningBuildFixture
    &rarr;see also: PlanningSegementationTool
    &rarr;see also: PlanningNodeCreatorTool

    [img[Colaborations in the Build Process|uml/fig128517.png]]

    Actually setting up and wiring a [[processing node|ProcNode]] involves several issues and is carried out at the lowest level of the build process.
    It is closely related to &rarr; [[the way nodes are operated|NodeOperationProtocol]] and the &rarr; [[mechanics of the render process|RenderMechanics]]

    !!!object creation
    The Nodes are small polymorphic objects, carrying configuration data, but no state. They are [[specially allocated|ManagementRenderNodes]], and the object creation is accessible by means of the NodeFactory solely. They //must not be deallocated manually.// The decision of what concrete node type to create depends on the actual build situation and is worked out by the combination of [[mould|BuilderMould]] and [[processing pattern|ProcPatt]] at the current OperationPoint, issuing a call to one of NodeFactory's {{{operator()}}}

    !!!node, plugin and processing function
    Its a good idea to distinguish clearly between those concepts. A plugin is a piece of (possibly external) code we use to carry out operations. We have to //discover its properties and capabilities.// We don't have to discover anything regarding nodes, because we (Lumiera builder and renderengine) are creating, configuring and wiring them to fit the specific purpose. Both are to be distinguished from processing functions, which do the actual calculations on the media data. Every node typically encompasses at least one processing function, which may be an internal function in the node object, a library function from Lumiera or GAVL, or external code loaded from a plugin.

    !!!node interfaces
    As a consequence of this distinctions, in conjunction with a processing node, we have to deal with three different interfaces
    * the __build interface__ is used by the builder to set up and wire the nodes. It can be full blown C++ (including templates)
    * the __operation interface__ is used to run the calculations, which happens in cooperation of Proc-Layer and Backend. So a function-style interface is preferable.
    * the __inward interface__ is accessed by the processing function in the course of the calculations to get at the necessary context, including in/out buffers and param values.

    !!!wiring data connections
    A node //knows its predecessors, but not its successors.// When being //pulled//&nbsp; in operation, it can expect to get a frame provider for accessing the in/out buffer locations (some processing functions may be "in-place capable", but that's only a special case of the former). At this point, the ''pull principle'' comes into play: the node may request input frames from the frame provider, passing its predecessors as a ''continuation''.
    With regard to the build process, the wiring of data connections translates into providing the node with its predecessors and preconfiguring the possible continuations. While in the common case, a node has just one input/output and pulls from its predecessor a frame for the same timeline position, the general case can be more contrived. A node may process N buffers in parallel and may require several different time positions for it's input, even at a differing framerate. So the actual source specification is (predNode,time,frameType). The objective of the wiring done in the build process is to factor out the parts known in advance, while in the render process only the variable part need to be filled in. Or to put it differently: wiring builds a higher order function (time)->(continuation), where continuation can be invoked to get the desired input frame.

    !!!wiring control conections
    In many cases, the parameter values provided by these connections aren't frame based data, rather, the processing function needs a call interface to get the current value (value for a given time), which is provided by the parameter object. Here, the wiring needs to link to the suitable parameter instance, which is located within the high-level model (!). As an additional complication, calculating the actual parameter value may require a context data frame (typically for caching purposes to speed up the interpolation). While these parameter context data frames are completely opaque for the render node, they have to be passed in and out similar to the state needed by the node itself, and the wiring has to prepare for accessing these frames too.

    The Builder takes some MObject/[[Placement]] information (called Timeline) and generates out of this a Render Engine configuration able to render this Objects. It does all decisions and retrieves the current configuration of all objects and plugins, so the Render Engine can just process them stright forward.

    The Builder is the central part of the [[Builder Pattern|http://en.wikipedia.org/wiki/Builder_pattern]]
    <br/>
    As the builder has to create a render node network implementing most of the features and wiring possible with the various MObject kinds and placement types, it is a rather complicated piece of software. In order to keep it manageable, it is broken down into several specialized sub components:
    * clients access builder functionality via the BuilderFacade
    * the [[Proc-Layer-Controller|Controller]] initiates the BuildProcess and does the overall coordination of scheduling edit operations, rebuilding the fixture and triggering the Builder
    * to carry out the building, we use several primary tools (SegmentationTool, NodeCreaterTool,...),  together with a BuilderToolKit to be supplied by the [[tool factory|BuilderToolFactory]]
    * //operating the Builder// can be viewed at from two different angles, either emphasizing the [[basic building operations|BasicBuildingOperations]] employed to assemble the render node network, or focussing rather at the [[mechanics|BuilderMechanics]] of cooperating parts while processing.
    * besides, we can identify a small set of elementary situations we call [[builder primitives|BuilderPrimitives]], to be covered by the mentioned BuilderToolKit; by virtue of [[processing patterns|ProcPatt]] they form an [[interface to the rule based configuration|BuilderRulesInterface]].
    * the actual building (i.e. the application of tools to the timeline) is done by the [[Assembler|BuilderAssembler]], which is basically a collection of functions (but has a small amount of global configuration state)
    * any non-trivial wiring of render nodes, tracks, pipes and [[automation|Automation]] is done by the services of the [[connection manager|ConManager]]

    The cooperation of several components creates a context of operation for the primary builder working tool, the [[node creator|PlanningNodeCreatorTool]]:
    * the BuilderToolFactory acts as the "builder for the builder tools", i.e. we can assume to be able to retrive all needed primary tools and elementary tools from this factory, completely configured and ready to use.
    * the [[Assembler|BuilderAssembler]] has the ability to consume objects from the high level model and feed them to the node creator (which translates into a dispatch of individual operations suited to the objects to be treated). This involves some sort of scheduling or ordering of the operaions, which is the only means to direct the overall process such as to create a sensible and usable result. //This is an fundamental design decision:// the actual working tools have no hard wired knowledge of the "right process", which makes the whole Builder highly configurable ("open").
    * the [[connection manager|ConManager]] on the contrary is a passive service provider. Fed with [[wiring requests|WiringRequest]], he can determine if a desired connection is possible, and what steps to take to implement it; the latter recursively creates further building requests to satisfy by the assembler, and possibly new wiring requests.

    !!pattern of operation
    The working pattern of this builder mechanics can be described as triggering, enqueuing, priorizing, recursing and exhausting. Without the priorizing part, it would be a depth-first call graph without any context state, forcing us to have all cross reference information available at every node or element to be treated. We prefer to avoid this overhead by ordering the operations into several phases and within these phases into correlated entities with the help of a ''weighting function'' and scheduling with a ''priority queue''

    The [[Builder]] uses different kinds of tools for creating a network of render nodes from a given high-level model. When breaking down this (necessarily complex) process into small manageable chunks, we arrive at [[elementary building situations|BuilderPrimitives]]. For each of these there is a specialized tool. We denote these tools as "moulds" because they are a rather passive holder for the objects to be attached and wired up. They are shaped according to the basic form the connections have to follow for each of these basic situations:
    * attaching an effect to a pipe
    * combining pipes via a transition
    * starting out a pipe from a source reader
    * general connections from the exit node of a pipe to the port of another pipe
    In all those cases, the active part is provided by [[processing patterns|ProcPatt]] &mdash; sort of micro programs executed within the context of a given mould: the processing pattern defines the steps to take (in the standard/basic case this is just "attach"), while the mould holds and provides the location where these steps will operate. Actually, this location is represented as a OperationPoint, provided by the mould and abstracting the details of making multi-channel connections.

    While assembling and building up the render engines node network, a small number of primitive building situations is encountered repeatedly. The BuilderToolKit provides a "[[mould|BuilderMould]]" for each of these situations, typically involving parametrisation and the application of a [[processing pattern|ProcPatt]].

    The ''Lifecycle'' of such a mould starts out by arming it with the object references involved into the next building step. After conducting this building step, the resulting render nodes can be found &mdash; depending on the situation &mdash; attached either to the same mould, or to another kind of mould, but in any case ready to be included in the next building step. Thus, //effectively//&nbsp; the moulds are //used to handle the nodes being built,// due to the fact that the low-level model (nodes to be built) and the high-level model (objects directing what is to be built) are //never connected directly.//

    !List of elementary building situations
    !!!inserting an Effect or Plugin
    [>img[draw/builder-primitives1.png]]
    The __~PipeMould__ is used to chain up the effects attached to a clip (=local pipe) or global pipe (=bus)
    * participating: a Pipe and an Effect
    * point of reference: current exit node of the pipe
    * result: Effect appended at the pipe's exit node
    * returns: ~PipeMould holding onto the new exit node

    @@clear(right):display(block):@@

    !!!attaching a transition
    [>img[draw/builder-primitives2.png]]
    After having completed N pipe's node chains, a __~CombiningMould__ can be used to join them into a [[transition|TransitionsHandling]]
    * participating: N pipe's exit nodes, transition
    * point of reference: N exit nodes corresponding to (completed) pipes
    * result: transition has been attached with the pipe's exit nodes, new wiring requests created attached to the transition's exit node(s)
    * returns: ~WiringMould, connected with the created wiring request
    Using this mould implicitly "closes" the involved pipes, which means that we give up any reference to the exit node and can't build any further effect attached to this pipes. Generally speaking, "exit node" isn't a special kind of node, rather it's a node we are currently holding on. Similarly, there is nothing directly correlated to a pipe within the render nodes network after we are done with building the part of the network corresponding to the pipe; the latter serves rather as a blueprint for building, but isn't an entity in the resulting low-level model.
    Actually, there is {{red{planned}}} a more general (and complicated) kind of transition, which can be inserted into N data connections without joining them together into one single output, as the standard transitions do. The ~CombiningMould can handle this case too by just returning N wiring moulds as a result.

    @@clear(right):display(block):@@

    !!!building a source connection
    [>img[draw/builder-primitives3.png]]
    The __~SourceChainMould__ is used as a starting point for any further building, as it results in a local pipe (=clip) rooted at the clip source port. This reflects the fact that the source readers (=media access points) are the //leaf nodes// in the node graph we are about to build.
    * participating: source port of a clip, media access point, [[processing pattern|ProcPatt]]
    * point of reference: //none//
    * result: processing pattern has been //executed//, resulting in a chain of nodes from the source reader to the clip source port
    * returns: ~PipeMould holding onto the new exit node (of a yet-empty pipe)

    @@clear(right):display(block):@@

    !!!wiring a general connection
    Any wiring (outside the chain of effects within a pipe) is always done from exit nodes to the port of another pipe, requiring an [[wiring request|WiringRequest]] already checked and deemed resolvable. Within the __~WiringMould__ the actual wiring is conducted, possibly adding a summation node (called "overlayer" in case of video) and typically a fader element (the specific setup to be used is subject to configuration by processing patterns)
    * participating: already verified connection request, providing a Pipe and an exit node; a processing pattern and a Placement
    * points of reference: exit node and (optionally) starting point of a pipe's chain (in case there are already other connections)
    * result: summation node prepended to the port of the pipe, processing pattern has been //executed// for building the connection from the exit node to the pipe's port, ParamProvider has been setup in [[accordance|PlacementDerivedDimension]] to the Placement.
    * returns: ~PipeMould holding onto the destination pipe's exit node, ~WiringMould holding onto the port side of the same pipe, i.e. the destination where further connections will insert summation nodes. {{red{TODO how to handle the //empty//-case?}}}
    [>img[draw/builder-primitives4.png]]

    @@clear(right):display(block):@@

    * the MObjects implement //Buildable//
    * each Buildable can "receive" a Tool object and apply it
    * the different Tool objects are iterated/mapped onto the list of MObjects in the [[Timeline]]
    * __Rationale__
    ** the MObject class hierarchy is rather fixed (it is unlikely the we will be adding much new MObject subclasses)
    ** so this design makes it easy to add new Tool subclasses, and within each Tool subclass, all operations on the different MObject classes are grouped together, so it is easy to see what is going on.
    ** a given Tool instance can carry state while being iterated, so we don't need any global (or object-global) variables to hold the result of the build process

    This programming technique is often referred to as [["double dispatch" or "visitor"|VisitorUse]]. We use a specialized library implementation of this pattern &mdash; heavily inspired by the [[Loki library|http://loki-lib.sourceforge.net/]]. We use this approach not only for the builder, but also for carrying out operations on the objects in the EDL in a typesafe manner.
    It is the low level foundation of the actual [[building operations|BasicBuildingOperations]] necessary to create render nodes starting from the given high level model.
    [img[Entities cooperating in the Builder|uml/fig129285.png]]

    !Colaborations

    While building, the application of such a visiting tool (especially the [[NodeCreatorTool|PlanningNodeCreatorTool]]) is embedded into an execution context formed by the BuilderToolFactory providing our BuilderToolKit, the [[Assembler|BuilderAssembler]] and [[connection manager|ConManager]]. The colaboration of these parts can be seen as the [[mechanics of the builder|BuilderMechanics]] &mdash; sort of the //outward view//, contrary to the //invard aspects// visible when focussing on how the nodes are put together.

    [img[Colaborations in the Build Process|uml/fig128517.png]]

    Besides the primary working tool within the builder (namely the [[Node Creator Tool|PlanningNodeCreatorTool]]), on a lower level, we encounter several [[elementary building situations|BuilderPrimitives]] &mdash; and for each of these elementary situations we can retrieve a suitable "fitting tool" or [[mould|BuilderMould]]. The palette of these moulds is called the ''tool kit'' of the builder. It is subject to configuration by rules.

    !!addressing a mould
    All mould instances are owned and managed by the [[tool factory|BuilderToolFactory]], and can be referred to by their type (PipeMould, CombiningMould, SourceChainMould, WiringMould) and a concrete object instance (of suitable type). The returned mould (instance) acts as a handle to stick together the given object instance (from the high-level model) with the corresponding point in the low-level node network under construction. As consequence of this approach, the tool factory instance holds a snapshot of the current building state, including all the active spots in the build process. As the latter is driven by objects from the high-level model appearing (in a sensible order &rarr; see BuilderMechanics) within the NodeCreatorTool, new moulds will be created and fitted as necessary, and existing moulds will be exhausted when finished, until the render node network is complete.

    !!configuring a mould
    As each mould kind is different, it has a {{{prepare(...)}}} function with suitably typed parameters. The rest is intended to be  self-configuring (for example, a ~CombiningMould will detect the actual kind of Transition and select the internal mode of operation), so that it's sufficient to just call {{{operate()}}}

    !!sequence of operations
    When {{{operate()}}} doesn't throw, the result is a list of //successor moulds// &mdash; you shouldn't use the original mould after triggering its operation, because it may have been retracted as a result and reused for another purpose by the tool factory. It is not necessary to store these resulting moulds either (as they can be retrieved as described above), but they can be used right away for the next building step if applicable. In the state they are returned from a successful building step (mould operation = execution of a contained [[processing pattern|ProcPatt]]), they are usually already holding a reference to the part of the network just created and need to be configured only with the next high-level object (effect, placement, pipe, processing pattern or similar, depending on the concrete situation) in order to carry out the next step.

    !!single connection step
    at the lowest level within the builder there is the step of building a //connection.// This step is executed by the processing pattern with the help of the mould. Actually, making such a connection is more complicated, because in the standard case it will connect N media streams simultaneously (N=2 for stereo sound or 3D video, N=6 for 5.1 Surround, N=9 for 2nd order Ambisonics). These details are encapsulated within the OperationPoint, which is provided by the mould and exhibits a common interface for the processing pattern to express the connecting operation.

    &rarr;see also: BuilderPrimitives for the elementary working situations corresponding to each of these [[builder moulds|BuilderMould]]

    Background: #fefefd
    Foreground: #000
    PrimaryPale: #8fb
    PrimaryLight: #4dc9a7
    PrimaryMid: #16877a
    PrimaryDark: #0f3f56
    SecondaryPale: #ffc
    SecondaryLight: #fe8
    SecondaryMid: #db4
    SecondaryDark: #841
    TertiaryPale: #eef
    TertiaryLight: #ccd
    TertiaryMid: #99a
    TertiaryDark: #667
    Error: #f88

    The Connection Manager is a service for wiring connections and for querying information and deriving decisions regarding various aspects of data streams and the possibility of connections. The purpose of the Connection Manager is to isolate the [[Builder]], which is client of this information and decision services, from the often convoluted details of type information and organizing a connection.

    !control connections
    my intention was that it would be sufficient for the builder to pass an connection request, and the Connection Manager will handle the details of establishing a control/parameter link.
    {{red{TODO: handling of parameter values, automation and control connections still need to be designed}}}

    !data connections
    Connecting data streams of differing type involves a StreamConversion. Mostly, this aspect is covered by the [[stream type system|StreamType]]. The intended implementation will rely partially on [[rules|ConfigRules]] to define automated conversions, while other parts need to be provided by hard wired logic. Thus, regarding data connections, the ConManager can be seen as a specialized Facade and will delegate to the &rarr; [[stream type manager|STypeManager]]
    * retrieve information about capabilities of a stream type given by ID
    * decide if a connection is possible
    * retrieve a //strategy// for implementing a connection

    Configuration Queries are requests to the system to "create or retrieve an object with //this and that // capabilities". They are resolved by a rule based system ({{red{planned feature}}}) and the user can extend the used rules for each Session. Syntactically, they are stated in ''prolog'' syntax as a conjunction (=logical and) of ''predicates'', for example {{{stream(mpeg), pipe(myPipe)}}}. Queries are typed to the kind of expected result object: {{{Query<Pipe> ("stream(mpeg)")}}} requests a pipe excepting/delivering mpeg stream data &mdash; and it depends on the current configuration what "mpeg" means. If there is any stream data producing component in the system, which advertises to deliver {{{stream(mpeg)}}}, and a pipe can be configured or connected with this component, then the [[defaults manager|DefaultsManagement]] will create/deliver a [[Pipe|PipeHandling]] object configured accordingly.
    &rarr; [[Configuration Rules system|ConfigRules]]
    &rarr; [[accessing and integrating configuration queries|ConfigQueryIntegration]]

    * planning to embed a YAP Prolog engine
    * currently just integrated by a table driven mock
    * the baseline is a bit more clear by now (4/08)

    &rarr; see also ConfigRules
    &rarr; see also DefaultsManagement

    !Use cases
    [<img[when to run config queries|uml/fig131717.png]]

    The key idea is that there is a Rule Base &mdash; partly contained in the session (building on a stock of standard rules supplied with the application). Now, whenever there is the need to get a new object, for adding it to the session or for using associated with another object &mdash; then instead of creating it by a direct hard wired ctor call, we issue a ConfigQuery requesting an object of the given type with some //capabilities// defined by predicates. The same holds true when loading an existing session: some objects won't be loaded back blindly, rather they will be re-created by issuing the config queries again. Especially important is the case of (re)creating a [[processing pattern|ProcPatt]] guiding how to wire up the processing pipeline for some given media.

    At various places, instead of requiring a fixed set of capabilities, it is possible to request a "default configured" object instead, specifying just those capabilities we really need to be configured in a specific way. This is done by using the [[Defaults Manager|DefaultsManagement]] accessible on the [[Session]] interface. Such a default object query may either retrieve an already existing object instance, run further config queries, and finally result in the invocation of a factory for creating new objects &mdash; just as necessary and guarded by the rules.

    @@clear(left):display(block):@@

    !Components and Relations
    [>img[participating classes|uml/fig131461.png]]

    <br/>
    <br/>
    <br/>
    <br/>
    <br/>
    <br/>
    <br/>

    Access point is the interface {{{ConfigRules}}}, which allowes to resolve a ConfigQuery resulting in an object with properties condigured such as to fulfill the query. This whole subsystem employes quite some generic programming, because actually we don't deal with "objects", but rather with similar instantiations of the same functionality for a collection of different object types. For the purpose of resolving these queries, the actual kind of object is not so much of importance, but on the caller sinde, of course we want to deal with the result of the queries in a typesafe manner.
    Examples for //participating object kinds// are [[pipes|Pipe]], [[processing patterns|ProcPatt]], effect instances, [[tags|Tag]], [[labels|Label]], [[automation data sets|AutomationData]],...
    @@clear(right):display(block):@@
    For this to work, we need each of the //participating object types// to implement a generic interface {{{TypeHandler}}}, which will provide actual C/C++ implementations for the predicates usable on objects of this type within the Prolog rules. The implementation has to make sure that alongside with each config query, there are additional //type constraints// to be regarded. For example, if the client code runs a {{{Query<Pipe>}}}, an additional //type guard// (implemented by a predicate {{{type(pipe)}}} has to be inserted, so only rules and facts in accordance with this type will be used for resolution.

    !when querying for a [["default"|DefaultsManagement]] object
    [<img[colaboration when issuing a defaults query|uml/fig131845.png]]

    @@clear(left):display(block):@@

    Many features can be implemented by specifically configuring and wiring some unspecific components. Rather than tie the client code in need of some given feature to these configuration internals, in Lumiera the client can //query // for some kind of object providing the //needed capabilities. // Right from start (summer 2007), Ichthyo had the intention to implement such a feature using sort of a ''declarative database'', e.g. by embedding a Prolog system. By adding rules to the basic session configuration, users should be able to customize the semi-automatic part of Lumiera's behaviour to great extent.

    [[Configuration Queries|ConfigQuery]] are used at various places, when creating and adding new objects, as well when building or optimizing the render engine node network.
    * Creating a [[pipe|PipeHandling]] queries for a default pipe or a pipe with a certain stream type
    * Adding a new [[track|TrackHandling]] queries for some default placement configuration, e.g. the pipe it will be plugged to.
    * when processing a [[wiring request|WiringRequest]], connection possibilities have to be evaluated.
    * actually building such a connection may create additional degrees of freedom, like panning for sound or layering for video.

    !anatomy of a Configuration Query
    The query is given as a number of logic predicates, which are required to be true. Syntactically, it is a string in prolog syntax, e.g. {{{stream(mpeg)}}}, where "stream" is the //predicate, // meaning here "the stream type is...?" and "mpeg" is a //term // denoting an actual property, object, thing, number etc &mdash; the actual kind of stream in the given example. Multible comma separated predicates are combined with logical "and". Terms may be //variable // at start, which is denoted syntactically by starting them with a uppercase letter. But any variable term need to be //bound //  to some known value while computing the solution to the query, otherwise the query fails. A failed query is treated as a local failure, which may cause some operation being aborted or just some other possibility being chosen.
    Queries are represented by instantiations of the {{{Query<TYPE>}}} template, because their actual meaning is "retrieve or create an object of TYPE, configured such that...!". At the C++ side, this ensures type safety and fosters programming against interfaces, while being implemented rule-wise by silently prepending the query with the predicate "{{{object(TYPE)}}}"

    !executing a Configuration Query
    Actually posing such an configuration query, for example to the [[Defaults Manager in the Sessison|DefaultsManagement]], may trigger several actions: First it is checked against internal object registries (depending on the target object type), which may cause the delivery of an already existing object (as reference, clone, or smart pointer). Otherwise, the system tries to figure out an viable configuration for a newly created object instance, possibly by issuing recursive queries. In the most general case this may silently impose additional decisions onto the //execution context // of the query &mdash; by default the session.

    !Implementation
    At start and for debugging/testing, there is an ''dummy'' implementation using a map with predefined queries and answers. But for the real system, the idea is to embed a ''YAP Prolog'' engine to run the queries. This includes the task of defining and loading a set of custom predicates, so the rule system can interact with the object oriented execution environment, for example by transforming some capability predicate into virtual calls to a corresponding object interface. We need a way for objects to declare some capability predicates, together with a functor that can be executed on an object instance (and further parameters) in the cause of the evaluation of some configuration query. Type safety and diagnostics play an important role here, because effectively the rule base is a pool of code open for arbitray additions from the user session.
    &rarr; [[considerations for a Prolog based implementation|QueryImplProlog]]
    &rarr; [[accessing and integrating configuration queries|ConfigQueryIntegration]]
    &rarr; see {{{src/common/query/mockconfigrules.cpp}}} for the table with the hard wired (mock) answers

    {{red{WARN}}} there is an interference with the (planned) Undo function: a totally correct implementation of "Undo" would need to remember and restore the internal state of the query system (similar to backtracking). But, more generally, such an correct implementation is not achievable, because we are never able to capture and control the whole state of a real world system doing such advanced things like video and sound processing. Seemingly we have to accept that after undoing an action, there is no guarantee we can re-do it the same way as it was first time.

    Here, in the context of the Render Engine, the Controller component is responsible for triggering and coordinating the build process and for activating the backend and the Render Engine configuration created by the Builder to carry out the actual rendering. There is another Controller in the backend, the ~PlaybackController, which is in charge of the playback/rendering/display state of the application, and consequently will call this (Proc-Layer) Controller to get the necessary Render Engine.

    !Facade
    This is an very important external Interface, because it links together all three Layers of our current architecture. It can be used by the backend to initiate [[Render Processes (=StateProxy)|StateProxy]] and it will probably be used by the Dispatcher for GUI actions as well...

    The question is where to put all the state-like information [[associated with the current session|SessionOverview]]. Because this is certainly "global", but may depend on the session or need to be configured differently when loading another session. At the moment (9/07) Ichthyo considers the following solution:
    * represent all configuration as [[Asset]]s
    * find a way {{red{TODO}}} how to reload the contents of the [[AssetManager]].
    * completely hide the Session object behind a ''~PImpl'' smart pointer, so the session object can be switched when reloading.
    * the [[Fixture]] acts as isolation layer, and all objects refered from the Fixture are refcounting smart pointers. So, even when the session gets switched, the old objects remain valid as long as needed.

    [[ProcLayer and Engine]]

    As detailed in the [[definition|DefaultsManagement]], {{{default(Obj)}}} is sort of a Joker along the lines "give me a suitable Object and I don't care for further details". Actually, default objects are implemented by the {{{mobject::session::DefsManager}}}, which remembers and keeps track of anything labeled as "default". This defaults manager is a singleton and can be accessed via the [[Session]] interface, meaning that the memory track regarding defaults is part of the session state. Accessing an object via the query for an default actually //tagges// this object (storing a weak ref in the ~DefsManager). Alongside with each object successfully queried via "default", the degree of constriction is remembered, i.e. the number of additional conditions contained in the query. This enables us to search for default objects starting with the most unspecific.

    !Skeleton
    # ''search'': using the predicate {{{default(X)}}} enumerates existing objects of suitable type
    #* candidates are delivered starting with the least constrained default
    #* the argument is unified
    #** if the rest of the query succeeds we've found our //default object// and are happy.
    #** otherwise, if all enumerated solutions are exhausted without success, we enter
    # ''default creation'': try to get an object fulfilling the conditions and remember this situation
    #* we issue an ConfigQuery with the query terms //minus// the {{{default(X)}}} predicate
    #* it depends on the circumstances how this query is handled. Typically the query resolution first searches existing objects and then creates a new instance to match the required capabilities. Usually, this process succeeds, but there can be configurations leading to failure.
    #** failing the ~ConfigQuery is considered an (non-critical) exception (throws), as defaults queries are supposed to succeed
    #** otherwise, the newly created object is remembered (tagged) as new default, together with the degree of constriction

    !!!Implementation details
    Taken precisely, the "degree of constriction" yields only a partial ordering &mdash; but as the "default"-predicate is sort of a existential quantification anyways, its sole purpose is to avoid polluting the session with unnecessary default objects, and we don't need to care for absolute precision. A suitable approximation is to count the number of predicates terms in the query and use a (sorted) set (separate for each Type) to store weak refs to the the objects tagged as "default"
    {{red{WARN}}} there is an interference with the (planned) Undo function. This is a general problem of the config queries; just ignoring this issue seems reasonable.

    !!!Problems with the (preliminary) mock implementation
    As we don't have a Prolog interpreter on board yet, we utilize a mock store with preconfigured answers. (see MockConfigQuery). As this preliminary solution is lacking the ability to create new objects, we need to resort to some trickery here (please look away). The overall logic is quite broken, because the system isn't capable to do any real resolution &mdash; if we ignore this fact, the rest of the algorithm can be implemented, tested and used right now.

    For several components and properties there is an implicit default value or configuration; it is stored alongside with the session. The intention is that defaults never create an error, instead, they are to be extended silently on demand. Objects configured according to this defaults can be retrieved at the [[Session]] interface by a set of overloaded functions {{{Session::current->default(Query<TYPE> ("query string"))}}}, where the //query string // defines a capability query similar to what is employed for pipes, stream types, codecs etc. This query mechanism is implemented by [[configuration rules|ConfigRules]]

    !!!!what is denoted by {{{default}}}?
    {{{default(Obj)}}} is a predicate expressing that the object {{{Obj}}} can be considered the default setup under the given conditions. Using the //default// can be considered as a shortcut for actually finding a exact and unique solution. The latter would require to specify all sorts of detailed properties up to the point where only one single object can satisfy all conditions. On the other hand, leaving some properties unspecified would yield a set of solutions (and the user code issuing the query had to provide means for selecting one soltution from this set). Just falling back on the //default// means that the user code actually doesn't care for any additional properties (as long as the properties he //does// care for are satisfied). Nothing is said specifically on //how//&nbsp; this default gets configured; actually there can be rules //somewhere,// and, additionally, anything encountered once while asking for a default can be re-used as default under similar circumstances.
    &rarr; [[implementing defaults|DefaultsImplementation]]

    Along the way of working out various [[implementation details|ImplementationDetails]], decisions need to be made on how to understand the different facilities and entities and how to tackle some of the problems. This page is mainly a collection of keywords, summaries and links to further the discussion. And the various decisions should allways be read as proposals to solve some problem at hand...

    ''Everything is an object'' &mdash; of course, that's a //no-brainer // todays. Rather, important is what is not "an object", meaning it can't be arranged arbitrarily
    * we have one and only one global [[Session]] which directly contains a collection of multiple [[EDLs|EDL]] and is associated with a globally managed collection of [[assets|Asset]] (no, we don't have scoped variables here; if e.g. a media has been "opened", it is just plain globally known as asset.)
    * we have some global [[pipes|Pipe]], which are treated in a unique manner and don't behave like other objects (e.g. effects)
    * we have a [[Fixture]] which acts as isolation layer towards the render engine and is (re)built automatically.

    We ''separate'' processing (rendering) and configuration (building). We have a [[Builder]] which creates a network of [[render nodes|ProcNode]], to be processed by //pulling data // from some [[Pipe]]

    ''Objects are [[placed|Placement]] rather'' than assembled, connected, wired, attached. This is more of a rule-based approach and gives us one central metaphor and abstraction, allowing us to treat everything in an uniform manner. You can place it as you like, and the builder tries to make sense out of it, silently disabling what doesn't make sense.
    An [[EDL]] is just a collection of configured and placed objects (and has no additional, fixed structure). [[Tracks|Track]] form a mere organisational grid, they are grouping devices not first-class entities (a track doesn't "have" a pipe or "is" a video track and the like; it can be configured to behave in such manner by using placements though). [[Pipes|Pipe]] are hooks for making connections and are the only facility to build processing chains. We have global pipes, and each clip is built around a lokal [[source port|ClipSourcePort]] &mdash; and that's all. No special "media viewer" and "arranger", no special role for media sources, no commitment to some fixed media stream types (video and audio). All of this is sort of pushed down to be configuration, represented as asset of some kind. For example, we have [[processing pattern|ProcPatt]] assets to represent the way of building the source network for reading from some media file (including codecs treated like effect plugin nodes)

    ''State'' is rigorously ''externalized'' and operations are to be ''scheduled'', to simplify locking and error handling. State is either treated similar to media stream data (as addressable and cacheable data frame), or is represented as "parameter" to be served by some [[parameter provider|ParamProvider]]. Consequently, [[Automation]] is just another kind of parameter, i.e. a function &mdash; how this function is calculated is an encapsulated implementation detail (we don't have "bezier automation", and then maybe a "linear automation", a "mask automation" and yet another way to handle transitions)

    Deliberately there is an limitaion on the flexibility of what can be added to the system via Plugins. We allow configuration and parametrisation to be extended and we allow processing and data handling to be extended, but we disallow extensions to the fundamental structure of the system by plugins. They may provide new implementations for already known subsystems, but they can't introduce new subsystems not envisioned in the general design of the application.
    * thus fixed assortments include: the possbile kinds of MObject and [[Asset]], the possible automation parameter data types, the supported kinds of plugin systems
    * while plugins may extend: the supported kinds of media, the [[media handling libraries|MediaImplLib]] used to deal with those media, the session storage backends, the behaviour of placments

    This __proc-Layer__ and ~Render-Engine implementation started out as a design-draft by [[Ichthyo|mailto:Ichthyostega@web.de]] in summer 2007. The key idea of this design-draft is to use the [[Builder Pattern|http://en.wikipedia.org/wiki/Builder_pattern]] for the Render Engine, thus separating completely the //building// of the Render Pipeline from //running,// i.e. doing the actual Render. The Nodes in this Pipeline should process Video/Audio and do nothing else. No more decisions, tests and conditional operations when running the Pipeline. Move all of this out into the configuration of the pipeline, which is done by the Builder.

    !Why doesn't the current Cinelerra-2 Design succeed?
    The design of Cinelerra-2 basically follows a similar design, but __fails because of two reasons__
    # too much differentiation is put into the class hierarchy instead of configuring Instances differently.<br>This causes overly heavy use of virtual functions and -- in order to ameliorate this -- falling back to hard wired branching
    # far too much coupling and back-coupling to the internals of the EDL, forcing a overly rigid structure on the latter

    !Try to learn from the Problems of the current Cinelerra-2 codebase
    * build up an [[Node Abstraction|ProcNode]] powerful enough to express //all necessary Operations// without the need to recure on the actual object type
    * need to redesign the internals of the EDL in a far more open manner. Everything is an [[M-Object|MObjects]] which is [[placed|Placement]] in some manner
    * strive at a StrongSeparation between EDL and Render Engine, encapsulate and allways implement against interfaces

    !Design Goals
    As always, the main goal is //to cut down complexity// by the usual approach to separate into small manageable chunks.

    To achieve this, here we try to separate ''Configuration'' from ''Processing''. Further, in Configuration we try to separate the ''high level view'' (users view when editing) from the ''low level view'' (the actual configuration effective for the calculations). Finally, we try to factor out and encapsulate ''State'' in order to make State explicit.

    The main tool used to implement this separation is the [[Builder Pattern|http://en.wikipedia.org/wiki/Builder_pattern]]. Here especially we move all decisions and parametrization into the BuildProcess. The Nodes in the render pipeline should process Video/Audio and do nothing else. All decisions, tests and conditional operations are factored out of the render process and handled as configuration of the pipeline, which is done by the Builder. The actual color model and number of ports is configured in by a pre-built wiring descriptor. All Nodes are of equal footing with each other, able to be connected freely within the limitations of the necessary input and output. OpenGL and renderfarm support can be configured in as an alternate implementation of some operations together with an alternate signal flow (usable only if the whole Pipeline can be built up to support this changed signal flow), thus factoring out all the complexities of managing the data flow between core and hardware accelerated rendering. We introduce separate control data connections for the [[automation data|Automation]], separating the case of true multi-channel-effects from the case where one node just gets remote controlled by another node (or the case of two nodes using the same automation data).

    Another pertinent theme is to make the basic building blocks simpler, while on the other hand gaining much more flexibility for combining these building blocks. For example we try to unfold any "internal-multi" effects into separate instances (e.g. the possibility of having an arbitrary number of single masks at any point of the pipeline instead of having one special masking facility encompassing multiple sub-masks. Similarly, we treat the Objects in the EDL in a more uniform manner and gain the possibility to [[place|Placement]] them in various ways.

    <<<
    {{red{WARNING: Naming is currently being discussed (11/08)}}}
    * [[EDL]] probably will be called ''Sequence'' (or maybe ''Arrangement'')
    * [[Session]] maybe renamed to ''Project''
    * there seems to be a new entity called [[Timeline]] which holds the global Pipes
    <<<

    ''EDL'' is a short-hand for __E__dit __D__ecision __L__ist. The use of this term can be confusing; for the usual meaning see the definition in [[Wikipedia|http://en.wikipedia.org/wiki/Edit_decision_list]]

    Cinelerra uses this term in a related manner but with a somewhat shifted focus (and we just stick to this usage here): In Lumiera the EDL is comprised of the whole set of clips and other media objects parametrized and placed onto the tracks by the user. It is the result of the user's //editing efforts.//

    In this usage, the EDL in most cases will be almost synonymous to the [[Session|SessionOverview]], just the latter emphasizes more the state aspect, as it can be thought as the current EDL contents contained in a file or data structure together with additional Option values and settings for the GUI. The Session is what you save and load, while the EDL rather denotes a structured collection of Objects placed in time.

    The EDL within the Session is just a //logical grouping//, allowing for lots of flexibility. Actually, we can have several ~EDLs within one Session, and these ~EDLs can be linked together or not, they may be in sequence or may constitue a logical grouping of clips used simultanously in compositional work etc. Multiple ~EDLs can use the same or different tracks, and tracks as well are only an organisational (grouping) device. But at any time, we have exactly one [[Fixture]], derived automatically from all ~EDLs and containing the content actually to be rendered.

    &rarr; see considerations about [[the role of Tracks and Pipes in conjunction with the EDL|TrackPipeEDL]]

    These are the tools provided to any client of the Proc layer for handling and manipulating the entities in the EDL. When defining such operations, //the goal should be to arrive at some uniformity in the way things are done.// Ideally, when writing client code, one should be able to guess how to achieve some desired result.

    !guiding principle
    The approach taken to define any operation is based primarily on the ''~OO-way of doing things'': entities operate themselfs. You don't advise some manager, session or other &raquo;god class&laquo; to manipulate them. And, secondly, the scope of each operation will be as large as possible, but not larger. This often means performing quite some elementary operations &mdash; sometimes a convenience shortcut provided by the higher levels of the application may come in handy &mdash; and basically this gives rise to several different paths of doing the same thing, all of which need to be equivalent.

    !main tasks
    * you ''create a clip'' either from a source media or from another clip (copy, maybe clone?). The new clip always reflects the full size (and other properties) of the source used for creating.
    * you can request a clip to ''resize'', which always relates to its current dimensions.
    * you can ''place or attach'' the clip to some point or other entity by creating a [[Placement]] from the clip. (&rarr; [[handling of Placements|PlacementHandling]])
    * you can ''adjust'' a placement relative to some other placement, meta object (i.e. selection), label or media, and this may cause the placement to resize or even delete the clip as necessary
    * you can perform ''adjustments'' on the whole EDL.
    All these operations propagate to directly dependant objects and may schedule global reconfigurations.

    !state, locking, policies
    While performing any manipulative task, the state of the object is considered inconsistent, but it is guaranteed to be consistent after any such operation, irrespective of the result. There is no automatic atomicity and, consequently each propagation is considered a separate operation.

    !!parallelism
    At the level of fine grained operations on individual entities, there is ''no support for parallelism''. Individual entities don't lock themselves. Tasks perform locking and are to be scheduled. Thus, we need an isolation layer towards all inherently multithreaded parts of the system, like the GUI or the renderengine.

    This has some obvious and some subtle consequences. Of course, manipulating //tasks// will be queued and scheduled somewhere. But as we rely on object identity and reference semantics for the entities in the EDL, some value changes are visible to  operations going on in parallel threads (e.g. rendering) and each operation on the EDL entities //has to be aware of this.//
    Consequently, sometimes there needs to be done sort of a ''read-copy-update'', i.e. self-replacement by copy followed by manipulation of the new copy, while ongoing processes use the unaltered original object until they receive some sort of reset.

    !!undo
    Basically, each elementary operation has to record the informations necessary to be undone. It does so by registering a Memento with some central UndoManager facility. This Memento object contains a functor pre-bound with the needed parameter values. (Besides, the UndoManager is free to implement a second level of security by taking independent state snapshots).
    {{red{to be defined in more detail later...}}}

    The &raquo;Timeline&laquo; is a sequence of ~MObjects -- here clips -- together with an ExplicitPlacement, locating each clip at a given time and track. (Note: I simplified the time format and wrote frame numbers to make it more clear)
    [img[Example1: Objects in the EDL/Fixture|uml/fig128773.png]]

    ----
    After beeing processed by the Builder, we get the following Render Engine configuration
    {{red{note: please take this only as a "big picture", the implementation details got a lot more complicated as of 6/08}}}
    [img[Example1: generated Render Engine|uml/fig129029.png]]

    {{red{TODO: seemingly this example is slightly outdated, as the implementation of placements is now indirect via LocatingPin objects}}}
    This Example showes the //high level// EDL as well. This needs to be transformed into a Fixture by some facility still to be designed. Basically, each [[Placement]] needs to be queried for this to get the corresponding ExplicitPlacement. The difficult part is to handle possible Placement constraints, e.g. one clip can't be placed at a timespan covered by another clip on the same track. In the current Cinelerra2, all of this is done directly by the GUI actions.

    The &raquo;Timeline&laquo; is a sequence of ~MObjects -- note: using the same Object instances -- but now with the calculated ExplicitPlacement, locating the clip at a given time and track. The effect is located absolutely in time as well, but because it is the same Instance, it has the pointer to the ~RelativePlacement, wich basically attaches the effect to the clip. This structure may look complicated, but is easy to process if we go "backward" and just rely on the information contained in the ExplicitPlacement.
    [img[Example2: Clip with Effect and generated Fixture for this EDL|uml/fig128901.png]]

    ----
    After beeing processed by the Builder, we get a Render Engine configuration.<br>
    It has to be segmented at least at every point with changes in the configuration, but some variations are possible, e.g. we could create a Render Engine for every Frame (as Cinelerra2 does) or we could optimize out some configurations (for example the effect extended beyond the end of the clip)
    {{red{note: as of 6/08 this can be taken only as the "big picture". Implementation will differ in details, and is more complicated than showed here}}}
    [img[Example2: generated Render Engine|uml/fig129157.png]]

    !MObject assembly
    To make the intended use of the classes more clear, consider the following two example Object graphs:
    * a video clip and a audio clip placed (explicitly) on two tracks &rarr;[[Example1]]
    * a video clip placed relatively, with an attached HUE effect &rarr;[[Example2]]

    a special ProcNode which is used to pull the finished output of one Render Pipeline (Tree or Graph). This term is already used in the Cinelerra2 codebase. I am unsure at the moment if it is a distinct subclass or rahter a specially configured ProcNode (a general design rule tells us to err in favour of the latter if in doubt).

    A special kind (subclass) of [[Placement]]. As such it is always linked to a //Subject//, i.e. a MObject. In addition to the properties of a (unspecific) Placement, the ExplicitPlacement specifies a absolute time and track position for locating the Subject

    We use Factories
    * for centralizing [[memory management|MemoryManagement]]
    * to support polymorphism (of course...)

    !Requirements
    * request the actual placement/allocation from the backend
    * allways hand out a smart-pointer
    * encapsulate / make configurable the smart-pointer type
    * install a callback into the smart-pointer for destroying the resource.
    * redirect the destroying request to the backend

    !Implementation Questions
    * how much genericity? (Ichthyo is rather inclined not to overdo this one. Often it is preferable to have repeated implementations follow a well known pattern, if this leads to short and simple implementations, while the complete general solution will be much more contrived).
    * how to specify the actual type needed?
    * how to implement the cases where a subtype needs to be selected (abstract factory pattern). Embody this into the Factory, pass it in as a Strategy or treat the Factory just as a simple service taking an explicit type-ID and providing the new object?

    !!chosen design
    My main goal is to have an easy-to-use interface for the implementer of individual classes using this factory mechanism. The intended use should mimic the standard use of operator new, and at the same time there should be a possibility to configure »real« Factory behaviour in.

    For this reason I make Factory a Functor, so it can be incorporated as a member into another class, while still looking like a function call to the client code. The users of this factory template typically either parametrize it with existing smart-pointer types, or they may chose to create a specialized Factory derived from this Factory template. After typically hiding this configuration behind a typedef, the user adds a static field to the class intended to use the Factory and initializes this field with the concrete Factory (this may pass internal ~IDs to the constructor of the Factory and from there on to the underlying Allocator).
    {{{
    #include "common/factory.hpp"

      class Product
        {
          int wow_;

        public:
          typedef lumiera::factory::RefcountPtr<Product> Factory;
          static Factory create;

          Product() : wow_(42) {} ;
        };

      /** storage for a static Factory instance
       *  for creating refcounting Ptrs to Product objects
       */
      Product::Factory Product::create;  // <----note this is a ctor call
    }}}
    Now, the clients of {{{class Product}}} can create ref-counting pointers to Product-objects by doing a fully qualified {{{create()}}} functor call:
    {{{
      std::tr1::shared_ptr<Product> huii = Product::create ();  // <----will invoke the default Product() ctor
      std::tr1::shared_ptr<Product> pfuii = huii;
    }}}
    Some further details
    * the product class (or base class) cares for using an custom allocator via an overloaded {{{operator new(size_t)}}} if applicable.
    * thus, when Functor or any derived class issues a new XX(), our custom Allocator gains control
    * the Functor-behaviour relies on a custom {{{operator()}}} which can be overridden in derived classes to take various parameter lists.
    * additionally, such a Factory class derived from Functor can do specific magic and e.g. create some subclass
    * and, as the created smart-pointer is a template parameter, such a custom Functor can create all sorts of Proxies, wrappers and the like

    a specially configured EDL
     * all MObjects have their position, length and configuration set up ready for rendering.
     * compound objects (e.g. multichannel clips) have been resolved to single non-compound basic objects.
     * every MObject is associated with an ExplicitPlacement, which declares a fixed position (Time, Track)
     * this ~ExplicitPlacements are contained in a ordered List called the Timeline

    As the builder and thus render engine //only consults the fixture,// while all editing operations finally propagate to the fixture as well, we get an isolation layer between the high level part of the Proc layer (editing, object manipulation) and the render engine. Creating the Fixture can be seen as a preprocessing step to simplify the build process. For this reason, the process of [[(re)building the fixture|PlanningBuildFixture]] has been designed together with the [[Builder]]

    This term has //two meanings, //so care has to be taken for not confusing them.
    # in general use, a Frame means one full image of a video clip, i.e an array of rows of pixels. For interlaced footage, one Frame contains two halfimages, commonly called Fields. (Cinelerra2 confuses this terms)
    # here in this design, we use Frame as an abstraction for a buffer of raw media data to be processed. If in doubt, we should label this "Dataframe".
    #* one video Dataframe contains a single video frame
    #* one audio Dataframe contains a block of raw audio samples
    #* one OpenGL Dataframe could contain raw texture data (but I am lacking expertise for this topic)

    /***
    |Name|FullScreenPlugin|
    |Created by|SaqImtiaz|
    |Location|http://tw.lewcid.org/#FullScreenPlugin|
    |Version|1.1|
    |Requires|~TW2.x|
    !Description:
    Toggle between viewing tiddlers fullscreen and normally. Very handy for when you need more viewing space.

    !Demo:
    Click the ↕ button in the toolbar for this tiddler. Click it again to turn off fullscreen.

    !Installation:
    Copy the contents of this tiddler to your TW, tag with systemConfig, save and reload your TW.
    Edit the ViewTemplate to add the fullscreen command to the toolbar.

    !History:
    *25-07-06: ver 1.1
    *20-07-06: ver 1.0

    !Code
    ***/
    //{{{
    var lewcidFullScreen = false;

    config.commands.fullscreen =
    {
                text:" ↕ ",
                tooltip:"Fullscreen mode"
    };

    config.commands.fullscreen.handler = function (event,src,title)
    {
                if (lewcidFullScreen == false)
                   {
                    lewcidFullScreen = true;
                    setStylesheet('#sidebar, .header, #mainMenu{display:none;} #displayArea{margin:0em 0 0 0 !important;}',"lewcidFullScreenStyle");
                   }
                else
                   {
                    lewcidFullScreen = false;
                    setStylesheet(' ',"lewcidFullScreenStyle");
                   }
    }

    config.macros.fullscreen={};
    config.macros.fullscreen.handler =  function(place,macroName,params,wikifier,paramString,tiddler)
    {
            var label = params[0]||" ↕ ";
            var tooltip = params[1]||"Fullscreen mode";
            createTiddlyButton(place,label,tooltip,config.commands.fullscreen.handler);
    }

    var lewcid_fullscreen_closeTiddler = Story.prototype.closeTiddler;
    Story.prototype.closeTiddler =function(title,animate,slowly)
    {
               lewcid_fullscreen_closeTiddler.apply(this,arguments);
               if (story.isEmpty() && lewcidFullScreen == true)
                  config.commands.fullscreen.handler();
    }

    Slider.prototype.lewcidStop = Slider.prototype.stop;
    Slider.prototype.stop = function()
    {
               this.lewcidStop();
               if (story.isEmpty() && lewcidFullScreen == true)
                  config.commands.fullscreen.handler();
    }
    //}}}

    The ''Gmerlin Audio Video Library''. &rarr; see [[homepage|http://gmerlin.sourceforge.net/gavl.html]]
    Used within Lumiera as a foundation for working with raw video and audio media data

    __G__roup __of__ __P__ictures: several compressed video formats don't encode single frames. Normally, such formats are considered mere //delivery formates// but it was one of the key strenghts of Cinelrra from start to be able to do real non linear editing on such formats (like the ~MPEG2-ts unsed in HDV video). The problem of course is that the data backend needs to decode the whole GOP to be serve  single raw video frames.

    For this Lumiera design, we could consider making GOP just another raw media data frame type and integrate this decoding into the render pipeline, similar to an effect based on several source frames for every calculated output frame.

    &rarr;see in [[Wikipedia|http://en.wikipedia.org/wiki/Group_of_pictures]]

    All communication between Proc-Layer and GUI has to be routed through the respective LayerSeparationInterfaces. Following a fundamental design decision within Lumiera, these interface are //intended to be language agnostic// &mdash; forcing them to stick to the least common denominator. Which creates the additional problem of how to create a smooth integration without forcing the architecture into functional decomposition style. To solve this problem, we rely on the well known solution of using a __business facade__ and delegation proxies.
    Thus, the Proc-Layer exposes (one or several) facade interfaces for the GUI to use it's functionality, and similarily the GUI provides a [[notification facade|GuiNotificationFacade]] for pushing back status information created as result of the edit operations, the build process and the render tasks.

    !anatomy of the Proc/GUI interface
    * the GuiFacade is used as a general lifecycle facade to start up the GUI and to set up the LayerSeparationInterfaces.
    * GuiFacade is implemented by a class //in core// and exposes a notification proxy implementing GuiNotificationFacade, which actually is wired through the InterfaceSystem to forward to the corresponding implementation facade object within the GUI
    * similarly, starting the fundamental subsystems within proc installs Interfaces into the InterfaceSystem and exposes them through proxy objects

    Considering how to interface to and integrate with the GUI Layer. Running the GUI is //optional,// but it requires to be [[started up|GuiStart]], installing the necessary LayerSeparationInterfaces.

    Starting up the GUI is optional and is considered part of the Application start/stop and lifecycle.
    * main and AppState activate the lifecyle methods on the ~GuiSubsysDescriptor
    * loading a GuiStarterPlugin creates an instance of the notification interface within the InterfaceSystem
    * its implementation (within GuiStarterPlugin) is linked against the GuiNotificationFacade, i.e. the actual implementation. This link is hard wired.
    * the process of loading this GuiStarterPlugin creates an interface ref on the calling side, which is made available through the usual singleton factory embedded within the GuiNotificationFacade interface. {{red{TODO current solution doesn't handle shutdown correctly}}}
    * activating this singleton factory yields the (single) instance of the forwarding proxy, which calls through this interface ref.

    While the low-level model holds the data used for carrying out the actual media data processing (=rendering), the high-level model is what the user works upon when performing edit operations through the GUI (or script driven in &raquo;headless mode&laquo;). Its building blocks and combination rules determine largely what structures can be created within the [[Session]].
    On the whole, it is a collection of [[media objects|MObjects]] stuck together and arranged by [[placements|Placement]].

    Basically, the structure of the high-level model is is a very open and flexible one &mdash; every valid connection of the underlying object types is allowed &mdash; but the transformation into a low-level node network for rendering follows certain patterns and only takes into account any objects reachable while processing the session data in accordance to these patterns. Taking into account the parameters and the structure of these objects visited when building, the low-level render node network is configured in detail.

    The fundamental metaphor or structural pattern is to create processing ''pipes'', which are a linear chain of data processing modules, starting from an source port and providing an exit point. [[Pipes|Pipe]] are a //concept or pattern,// they don't exist as objects. Each pipe has an input side and an output side and is in itself something like a Bus treating a single [[media stream|StreamType]] (but this stream may still have an internal structure, e.g. several channels related to a spatial audio system). Other processing entities like effects and transitions can be placed (attached) at the pipe, resulting them to be appended to form this chain. Optionally, there may be a ''wiring plug'', requesting the exit point to be connected to another pipe. When omitted, the wiring will be figured out automatically.
    Thus, when making an connection //to// a pipe, output data will be sent to the //source port// (input side) of the pipe, wheras when making a connection //from// a pipe, data from it's exit point will be routed to the destination. Incidentally, the low-level model and the render engine employ //pull-based processing,// but this is rather of no relevance for the high-level model.

    [img[draw/high-level1.png]]
    Normally, pipes are limited to a //strictly linear chain// of data processors ("''effects''") working on a single data stream type, and consequently there is a single ''exit point'' which may be wired to an destination. As an exception to this rule, you may insert wire tap nodes (probe points), which explicitly may send data to an arbitrary input port; they are never wired automatically. It is possible to create cyclic connections by such arbitrary wiring, which will be detected by the builder and flagged as an error.

    While pipes have a rather rigid and limited structure, it is allowed to make several connections to and from any pipe &mdash; even connections requiring an stream type conversion. It is not even necessary to specify //any// output destination, because then the wiring will be figured out automatically by searching the context and finally using some general rule. Connecting multiple outputs to the input of another pipe automatically creates a ''mixing step'' (which optionally can be controlled by a fader). Several pipes may be joined together by a ''transition'', which in the general case simultaneously treats N media streams. Of course, the most common case is to combine two streams into one output, thereby also mixing them. Most available transition plugins belong to this category, but, as said, the model isn't limited to this simple case, and moreover it is possible to attach several overlapping transitions covering the same time interval.

    Individual Media Objects are attached, located or joined together by ''Placements''. A [[Placement]] is a handle for a single MObject (implemented as a refcounting smart-ptr) and contains a list of placement specifications, called LocatingPin. Adding an placement to the session acts as if creating an //instance.// (it behaves like a clone in case of multiple placements of the same object). Besides absolute and relative placement, there is also the possibility of a placement to stick directly to another MObject's placement, e.g. for attaching an effect to a clip or to connect an automation data set to an effect. This //stick-to placement// creates sort of a loose clustering of objects: it will derive the position from the placement it is attached to. Note that while the length and the in/out points are a //property of the ~MObject,// it's actual location depends on how it is //placed// and thus can be maintained quite dynamically. Note further that effects can have an length on their own, thus by using these attachement mechaics, the wiring and configuration within the high-level model can be quite time dependant.

    [>img[draw/high-level2.png]]

    Actually a ''clip'' is handled as if it was comprised of local pipe(s). In the example shown here, a two-channel clip has three effects attached, plus a wiring plug. Each of those attachments is used only if applicable to the media stream type the respective pipe will process. As the clip has two channels (e.g. video and audio), it will have two ''source ports'' pulling from the underlying media. Thus, as showed in the drawing to the right, by chaining up any attached effect applicable to the respective stream type defined by the source port, effectively each channel (sub)clip gets its own specifically adapted processing pipe.

    @@clear(right):display(block):@@
    !!Example of an complete Session
    [img[draw/high-level3.png]]
    The Session contains several independent [[EDL]]s plus an output bus section (''global Pipes''). Each EDL holds a collection of ~MObjects placed within a ''tree of tracks''.
    Within Lumiera, tracks are a rather passive means for organizing media objects, but aren't involved into the data processing themselves. The possibility of nesting tracks allows for easy grouping. Like the other objects, tracks are connected together by placements: A track holds the list of placements of its child tracks. Each EDL holds a single placement pointing to the root track.

    As placements have the ability to cooperate and derive any missing placement specifications, this creates a hierarchical structure throughout the session, where parts on any level behave similar if applicable. For example, when a track is anchored to some external entity (label, sync point in sound, etc), all objects placed relatively to this track will adjust and follow automatically. This relation between the track tree and the individual objects is especially important for the wiring, which, if not defined locally within an ~MObject's placement, is derived by searching up this track tree and utilizing the wiring plug locating pins found there, if applicable. In the default configuration, the placement of an EDL's root track contains a wiring plug for video and another wiring plug for audio. This setup is sufficient for getting every object within this EDL wired up automatically to the correct global output pipe. Moreover, when adding another wiring plug to some sub track, we can intercept and reroute the connections of all objects creating output of this specific stream type within this track and on all child tracks.

    Besides routing to a global pipe, wiring plugs can also connect to the source port of an ''meta-clip''. In this example session, the outputs of ~EDL-2 as defined by locating pins in it's root track's placement, are directed to the source ports of a [[meta-cllip|VirtualClip]] placed within ~EDL-1. Thus, within ~EDL-1, the contents of ~EDL-2 appear like a pseudo-media, from which the (meta) clip has been taken. They can be adorned with effects and processed further completely similar to a real clip.

    Finally, this example shows an ''automation'' data set controlling some parameter of an effect contained in one of the global pipes. From the effect's POV, the automation is simply a ParamProvider, i.e a function yielding a scalar value over time. The automation data set may be implemented as a bézier curve, or by a mathematical function (e.g. sine or fractal pseudo random) or by some captured and interpolated data values. Interestingly, in this example the automation data set has been placed relatively to the meta clip (albeit on another track), thus it will follow and adjust when the latter is moved.

    This wiki page is the entry point to detail notes covering some technical decisions, details and problems encountered in the course of the implementation of the Lumiera Renderengine, the Builder and the related parts.

    * [[Packages, Interfaces and Namespaces|InterfaceNamespaces]]
    * [[Memory Management Issues|MemoryManagement]]
    * [[Creating and registering Assets|AssetCreation]]
    * [[Creating new Objects|ObjectCreation]]
    * [[Multichannel Media|MultichannelMedia]]
    * [[Editing Operations|EditingOperations]]
    * [[Handling of the current Session|CurrentSession]]
    * [[collecting Ideas for Implementation Guidelines|ImplementationGuidelines]]
    * [[using the Visitor pattern?|VisitorUse]] &mdash; resulting in [[»Visiting-Tool« library implementation|VisitingToolImpl]]
    * [[Handling of Tracks and render Pipes in the EDL|TrackPipeEDL]]. [[Handling of Tracks|TrackHandling]] and [[Pipes|PipeHandling]]
    * [[getting default configured|DefaultsManagement]] Objects relying on [[rule-based Configuration Queries|ConfigRules]]
    * [[integrating the Config Query system|ConfigQueryIntegration]]
    * [[identifying the basic Builder operations|BasicBuildingOperations]] and [[planning the Implementation|PlanningNodeCreatorTool]]
    * [[how to handle »attached placement«|AttachedPlacementProblem]]
    * working out the [[basic building situations|BuilderPrimitives]] and [[mechanics of rendering|RenderMechanics]]
    * how to classify and [[describe media stream types|StreamType]] and how to [[use them|StreamTypeUse]]
    * the relation of [[Project, Timelines and Sequences|TimelineSequences]]
    * how to [[start the GUI|GuiStart]] and how to [[communicate|GuiCommunication]]

    !Observations, Ideas, Proposals
    ''this page is a scrapbook for collecting ideas'' &mdash; please don't take anything noted here too literal. While writing code, I observe that I (ichthyo) follow certain informal guidelines, some of which I'd like to note down because they could evolve into general style guidelines for the proc layer code.
    * avoid doing anything non-local during the startup phase or shutdown phase of the application. consequently, allways prefer using an explicit lumiera::Singleton<T> over using an static instance directly, thus yielding better control when the ctor and dtor will be invoked.
    * write error handling code only if the error situation can be actually //handled// at this place. Otherwise, be prepared for exceptions just passing by and thus handle any resources by "resource acquisition is initialisation". Remember: error handling defeats decoupling and encapsulation
    * (almost) never {{{delete}}} an object directly, use {{{new}}} only when some smart pointer is at hand.
    * when user/client code is inteded to create objects, make the ctor protected and provide a factory member called {{{create}}} instead, returning a smart pointer
    * avoid asuming anything that can't be enforced by types, interfaces or signatures; this means: be prepared for open possibilities
    * prefer {{{const}}} and initialisation code over assignment and active changes (inspired by functional programming)

    /***
    ''InlineJavascriptPlugin for ~TiddlyWiki version 1.2.x and 2.0''
    ^^author: Eric Shulman - ELS Design Studios
    source: http://www.TiddlyTools.com/#InlineJavascriptPlugin
    license: [[Creative Commons Attribution-ShareAlike 2.5 License|http://creativecommons.org/licenses/by-sa/2.5/]]^^

    Insert Javascript executable code directly into your tiddler content. Lets you ''call directly into TW core utility routines, define new functions, calculate values, add dynamically-generated TiddlyWiki-formatted output'' into tiddler content, or perform any other programmatic actions each time the tiddler is rendered.
    !!!!!Usage
    <<<
    When installed, this plugin adds new wiki syntax for surrounding tiddler content with {{{<script>}}} and {{{</script>}}} markers, so that it can be treated as embedded javascript and executed each time the tiddler is rendered.

    ''Deferred execution from an 'onClick' link''
    By including a label="..." parameter in the initial {{{<script>}}} marker, the plugin will create a link to an 'onclick' script that will only be executed when that specific link is clicked, rather than running the script each time the tiddler is rendered.

    ''External script source files:''
    You can also load javascript from an external source URL, by including a src="..." parameter in the initial {{{<script>}}} marker (e.g., {{{<script src="demo.js"></script>}}}). This is particularly useful when incorporating third-party javascript libraries for use in custom extensions and plugins. The 'foreign' javascript code remains isolated in a separate file that can be easily replaced whenever an updated library file becomes available.

    ''Defining javascript functions and libraries:''
    Although the external javascript file is loaded while the tiddler content is being rendered, any functions it defines will not be available for use until //after// the rendering has been completed. Thus, you cannot load a library and //immediately// use it's functions within the same tiddler. However, once that tiddler has been loaded, the library functions can be freely used in any tiddler (even the one in which it was initially loaded).

    To ensure that your javascript functions are always available when needed, you should load the libraries from a tiddler that will be rendered as soon as your TiddlyWiki document is opened. For example, you could put your {{{<script src="..."></script>}}} syntax into a tiddler called LoadScripts, and then add {{{<<tiddler LoadScripts>>}}} in your MainMenu tiddler.

    Since the MainMenu is always rendered immediately upon opening your document, the library will always be loaded before any other tiddlers that rely upon the functions it defines. Loading an external javascript library does not produce any direct output in the tiddler, so these definitions should have no impact on the appearance of your MainMenu.

    ''Creating dynamic tiddler content''
    An important difference between this implementation of embedded scripting and conventional embedded javascript techniques for web pages is the method used to produce output that is dynamically inserted into the document:
    * In a typical web document, you use the document.write() function to output text sequences (often containing HTML tags) that are then rendered when the entire document is first loaded into the browser window.
    * However, in a ~TiddlyWiki document, tiddlers (and other DOM elements) are created, deleted, and rendered "on-the-fly", so writing directly to the global 'document' object does not produce the results you want (i.e., replacing the embedded script within the tiddler content), and completely replaces the entire ~TiddlyWiki document in your browser window.
    * To allow these scripts to work unmodified, the plugin automatically converts all occurences of document.write() so that the output is inserted into the tiddler content instead of replacing the entire ~TiddlyWiki document.

    If your script does not use document.write() to create dynamically embedded content within a tiddler, your javascript can, as an alternative, explicitly return a text value that the plugin can then pass through the wikify() rendering engine to insert into the tiddler display. For example, using {{{return "thistext"}}} will produce the same output as {{{document.write("thistext")}}}.

    //Note: your script code is automatically 'wrapped' inside a function, {{{_out()}}}, so that any return value you provide can be correctly handled by the plugin and inserted into the tiddler. To avoid unpredictable results (and possibly fatal execution errors), this function should never be redefined or called from ''within'' your script code.//

    ''Accessing the ~TiddlyWiki DOM''
    The plugin provides one pre-defined variable, 'place', that is passed in to your javascript code so that it can have direct access to the containing DOM element into which the tiddler output is currently being rendered.

    Access to this DOM element allows you to create scripts that can:
    * vary their actions based upon the specific location in which they are embedded
    * access 'tiddler-relative' information (use findContainingTiddler(place))
    * perform direct DOM manipulations (when returning wikified text is not enough)
    <<<
    !!!!!Examples
    <<<
    an "alert" message box:
    {{{
    <script>alert('InlineJavascriptPlugin: this is a demonstration message');</script>
    }}}
    <script>alert('InlineJavascriptPlugin: this is a demonstration message');</script>

    dynamic output:
    {{{
    <script>return (new Date()).toString();</script>
    }}}
    <script>return (new Date()).toString();</script>

    wikified dynamic output:
    {{{
    <script>return "link to current user: [["+config.options.txtUserName+"]]";</script>
    }}}
    <script>return "link to current user: [["+config.options.txtUserName+"]]";</script>

    dynamic output using 'place' to get size information for current tiddler
    {{{
    <script>
     if (!window.story) window.story=window;
     var title=story.findContainingTiddler(place).id.substr(7);
     return title+" is using "+store.getTiddlerText(title).length+" bytes";
    </script>
    }}}
    <script>
     if (!window.story) window.story=window;
     var title=story.findContainingTiddler(place).id.substr(7);
     return title+" is using "+store.getTiddlerText(title).length+" bytes";
    </script>

    creating an 'onclick' button/link that runs a script
    {{{
    <script label="click here">
     if (!window.story) window.story=window;
     alert("Hello World!\nlinktext='"+place.firstChild.data+"'\ntiddler='"+story.findContainingTiddler(place).id.substr(7)+"'");
    </script>
    }}}
    <script label="click here">
     if (!window.story) window.story=window;
     alert("Hello World!\nlinktext='"+place.firstChild.data+"'\ntiddler='"+story.findContainingTiddler(place).id.substr(7)+"'");
    </script>

    loading a script from a source url
    {{{
    <script src="demo.js">return "loading demo.js..."</script>
    <script label="click to execute demo() function">demo()</script>
    }}}
    where http://www.TiddlyTools.com/demo.js contains:
    >function demo() { alert('this output is from demo(), defined in demo.js') }
    >alert('InlineJavascriptPlugin: demo.js has been loaded');
    <script src="demo.js">return "loading demo.js..."</script>
    <script label="click to execute demo() function">demo()</script>
    <<<
    !!!!!Installation
    <<<
    import (or copy/paste) the following tiddlers into your document:
    ''InlineJavascriptPlugin'' (tagged with <<tag systemConfig>>)
    <<<
    !!!!!Revision History
    <<<
    ''2006.01.05 [1.4.0]''
    added support 'onclick' scripts. When label="..." param is present, a button/link is created using the indicated label text, and the script is only executed when the button/link is clicked. 'place' value is set to match the clicked button/link element.
    ''2005.12.13 [1.3.1]''
    when catching eval error in IE, e.description contains the error text, instead of e.toString(). Fixed error reporting so IE shows the correct response text. Based on a suggestion by UdoBorkowski
    ''2005.11.09 [1.3.0]''
    for 'inline' scripts (i.e., not scripts loaded with src="..."), automatically replace calls to 'document.write()' with 'place.innerHTML+=' so script output is directed into tiddler content
    Based on a suggestion by BradleyMeck
    ''2005.11.08 [1.2.0]''
    handle loading of javascript from an external URL via src="..." syntax
    ''2005.11.08 [1.1.0]''
    pass 'place' param into scripts to provide direct DOM access
    ''2005.11.08 [1.0.0]''
    initial release
    <<<
    !!!!!Credits
    <<<
    This feature was developed by EricShulman from [[ELS Design Studios|http:/www.elsdesign.com]]
    <<<
    !!!!!Code
    ***/
    //{{{
    version.extensions.inlineJavascript= {major: 1, minor: 4, revision: 0, date: new Date(2006,1,5)};

    config.formatters.push( {
     name: "inlineJavascript",
     match: "\\<script",
     lookahead: "\\<script(?: src=\\\"((?:.|\\n)*?)\\\")?(?: label=\\\"((?:.|\\n)*?)\\\")?\\>((?:.|\\n)*?)\\</script\\>",

     handler: function(w) {
     var lookaheadRegExp = new RegExp(this.lookahead,"mg");
     lookaheadRegExp.lastIndex = w.matchStart;
     var lookaheadMatch = lookaheadRegExp.exec(w.source)
     if(lookaheadMatch && lookaheadMatch.index == w.matchStart) {
     if (lookaheadMatch[1]) { // load a script library
     // make script tag, set src, add to body to execute, then remove for cleanup
     var script = document.createElement("script"); script.src = lookaheadMatch[1];
     document.body.appendChild(script); document.body.removeChild(script);
     }
     if (lookaheadMatch[2] && lookaheadMatch[3]) { // create a link to an 'onclick' script
     // add a link, define click handler, save code in link (pass 'place'), set link attributes
     var link=createTiddlyElement(w.output,"a",null,"tiddlyLinkExisting",lookaheadMatch[2]);
     link.onclick=function(){try{return(eval(this.code))}catch(e){alert(e.description?e.description:e.toString())}}
     link.code="function _out(place){"+lookaheadMatch[3]+"};_out(this);"
     link.setAttribute("href","javascript:;"); link.setAttribute("title",""); link.style.cursor="pointer";
     }
     else if (lookaheadMatch[3]) { // run inline script code
     var code="function _out(place){"+lookaheadMatch[3]+"};_out(w.output);"
     code=code.replace(/document.write\(/gi,'place.innerHTML+=(');
     try { var out = eval(code); } catch(e) { out = e.description?e.description:e.toString(); }
     if (out && out.length) wikify(out,w.output);
     }
     w.nextMatch = lookaheadMatch.index + lookaheadMatch[0].length;
     }
     }
    } )
    //}}}

    /***
    |''Name:''|InlineJavascriptPlugin|
    |''Source:''|http://www.TiddlyTools.com/#InlineJavascriptPlugin|
    |''Author:''|Eric Shulman - ELS Design Studios|
    |''License:''|[[Creative Commons Attribution-ShareAlike 2.5 License|http://creativecommons.org/licenses/by-sa/2.5/]]|
    |''~CoreVersion:''|2.0.10|

    Insert Javascript executable code directly into your tiddler content. Lets you ''call directly into TW core utility routines, define new functions, calculate values, add dynamically-generated TiddlyWiki-formatted output'' into tiddler content, or perform any other programmatic actions each time the tiddler is rendered.
    !!!!!Usage
    <<<
    When installed, this plugin adds new wiki syntax for surrounding tiddler content with {{{<script>}}} and {{{</script>}}} markers, so that it can be treated as embedded javascript and executed each time the tiddler is rendered.

    ''Deferred execution from an 'onClick' link''
    By including a label="..." parameter in the initial {{{<script>}}} marker, the plugin will create a link to an 'onclick' script that will only be executed when that specific link is clicked, rather than running the script each time the tiddler is rendered.

    ''External script source files:''
    You can also load javascript from an external source URL, by including a src="..." parameter in the initial {{{<script>}}} marker (e.g., {{{<script src="demo.js"></script>}}}). This is particularly useful when incorporating third-party javascript libraries for use in custom extensions and plugins. The 'foreign' javascript code remains isolated in a separate file that can be easily replaced whenever an updated library file becomes available.

    ''Display script source in tiddler output''
    By including the keyword parameter "show", in the initial {{{<script>}}} marker, the plugin will include the script source code in the output that it displays in the tiddler.

    ''Defining javascript functions and libraries:''
    Although the external javascript file is loaded while the tiddler content is being rendered, any functions it defines will not be available for use until //after// the rendering has been completed. Thus, you cannot load a library and //immediately// use it's functions within the same tiddler. However, once that tiddler has been loaded, the library functions can be freely used in any tiddler (even the one in which it was initially loaded).

    To ensure that your javascript functions are always available when needed, you should load the libraries from a tiddler that will be rendered as soon as your TiddlyWiki document is opened. For example, you could put your {{{<script src="..."></script>}}} syntax into a tiddler called LoadScripts, and then add {{{<<tiddler LoadScripts>>}}} in your MainMenu tiddler.

    Since the MainMenu is always rendered immediately upon opening your document, the library will always be loaded before any other tiddlers that rely upon the functions it defines. Loading an external javascript library does not produce any direct output in the tiddler, so these definitions should have no impact on the appearance of your MainMenu.

    ''Creating dynamic tiddler content''
    An important difference between this implementation of embedded scripting and conventional embedded javascript techniques for web pages is the method used to produce output that is dynamically inserted into the document:
    * In a typical web document, you use the document.write() function to output text sequences (often containing HTML tags) that are then rendered when the entire document is first loaded into the browser window.
    * However, in a ~TiddlyWiki document, tiddlers (and other DOM elements) are created, deleted, and rendered "on-the-fly", so writing directly to the global 'document' object does not produce the results you want (i.e., replacing the embedded script within the tiddler content), and completely replaces the entire ~TiddlyWiki document in your browser window.
    * To allow these scripts to work unmodified, the plugin automatically converts all occurences of document.write() so that the output is inserted into the tiddler content instead of replacing the entire ~TiddlyWiki document.

    If your script does not use document.write() to create dynamically embedded content within a tiddler, your javascript can, as an alternative, explicitly return a text value that the plugin can then pass through the wikify() rendering engine to insert into the tiddler display. For example, using {{{return "thistext"}}} will produce the same output as {{{document.write("thistext")}}}.

    //Note: your script code is automatically 'wrapped' inside a function, {{{_out()}}}, so that any return value you provide can be correctly handled by the plugin and inserted into the tiddler. To avoid unpredictable results (and possibly fatal execution errors), this function should never be redefined or called from ''within'' your script code.//

    ''Accessing the ~TiddlyWiki DOM''
    The plugin provides one pre-defined variable, 'place', that is passed in to your javascript code so that it can have direct access to the containing DOM element into which the tiddler output is currently being rendered.

    Access to this DOM element allows you to create scripts that can:
    * vary their actions based upon the specific location in which they are embedded
    * access 'tiddler-relative' information (use findContainingTiddler(place))
    * perform direct DOM manipulations (when returning wikified text is not enough)
    <<<
    !!!!!Examples
    <<<
    an "alert" message box:
    ><script show>
     alert('InlineJavascriptPlugin: this is a demonstration message');
    </script>
    dynamic output:
    ><script show>
     return (new Date()).toString();
    </script>
    wikified dynamic output:
    ><script show>
     return "link to current user: [["+config.options.txtUserName+"]]";
    </script>
    dynamic output using 'place' to get size information for current tiddler:
    ><script show>
     if (!window.story) window.story=window;
     var title=story.findContainingTiddler(place).id.substr(7);
     return title+" is using "+store.getTiddlerText(title).length+" bytes";
    </script>
    creating an 'onclick' button/link that runs a script:
    ><script label="click here" show>
     if (!window.story) window.story=window;
     alert("Hello World!\nlinktext='"+place.firstChild.data+"'\ntiddler='"+story.findContainingTiddler(place).id.substr(7)+"'");
    </script>
    loading a script from a source url:
    >http://www.TiddlyTools.com/demo.js contains:
    >>{{{function demo() { alert('this output is from demo(), defined in demo.js') } }}}
    >>{{{alert('InlineJavascriptPlugin: demo.js has been loaded'); }}}
    ><script src="demo.js" show>
     return "loading demo.js..."
    </script>
    ><script label="click to execute demo() function" show>
     demo()
    </script>
    <<<
    !!!!!Installation
    <<<
    import (or copy/paste) the following tiddlers into your document:
    ''InlineJavascriptPlugin'' (tagged with <<tag systemConfig>>)
    <<<
    !!!!!Revision History
    <<<
    ''2006.06.01 [1.5.1]'' when calling wikify() on script return value, pass hightlightRegExp and tiddler params so macros that rely on these values can render properly
    ''2006.04.19 [1.5.0]'' added 'show' parameter to force display of javascript source code in tiddler output
    ''2006.01.05 [1.4.0]'' added support 'onclick' scripts. When label="..." param is present, a button/link is created using the indicated label text, and the script is only executed when the button/link is clicked. 'place' value is set to match the clicked button/link element.
    ''2005.12.13 [1.3.1]'' when catching eval error in IE, e.description contains the error text, instead of e.toString(). Fixed error reporting so IE shows the correct response text. Based on a suggestion by UdoBorkowski
    ''2005.11.09 [1.3.0]'' for 'inline' scripts (i.e., not scripts loaded with src="..."), automatically replace calls to 'document.write()' with 'place.innerHTML+=' so script output is directed into tiddler content. Based on a suggestion by BradleyMeck
    ''2005.11.08 [1.2.0]'' handle loading of javascript from an external URL via src="..." syntax
    ''2005.11.08 [1.1.0]'' pass 'place' param into scripts to provide direct DOM access
    ''2005.11.08 [1.0.0]'' initial release
    <<<
    !!!!!Credits
    <<<
    This feature was developed by EricShulman from [[ELS Design Studios|http:/www.elsdesign.com]]
    <<<
    !!!!!Code
    ***/
    //{{{
    version.extensions.inlineJavascript= {major: 1, minor: 5, revision: 1, date: new Date(2006,6,1)};

    config.formatters.push( {
     name: "inlineJavascript",
     match: "\\<script",
     lookahead: "\\<script(?: src=\\\"((?:.|\\n)*?)\\\")?(?: label=\\\"((?:.|\\n)*?)\\\")?( show)?\\>((?:.|\\n)*?)\\</script\\>",

     handler: function(w) {
     var lookaheadRegExp = new RegExp(this.lookahead,"mg");
     lookaheadRegExp.lastIndex = w.matchStart;
     var lookaheadMatch = lookaheadRegExp.exec(w.source)
     if(lookaheadMatch && lookaheadMatch.index == w.matchStart) {
     if (lookaheadMatch[1]) { // load a script library
     // make script tag, set src, add to body to execute, then remove for cleanup
     var script = document.createElement("script"); script.src = lookaheadMatch[1];
     document.body.appendChild(script); document.body.removeChild(script);
     }
     if (lookaheadMatch[4]) { // there is script code
     if (lookaheadMatch[3]) // show inline script code in tiddler output
     wikify("{{{\n"+lookaheadMatch[0]+"\n}}}\n",w.output);
     if (lookaheadMatch[2]) { // create a link to an 'onclick' script
     // add a link, define click handler, save code in link (pass 'place'), set link attributes
     var link=createTiddlyElement(w.output,"a",null,"tiddlyLinkExisting",lookaheadMatch[2]);
     link.onclick=function(){try{return(eval(this.code))}catch(e){alert(e.description?e.description:e.toString())}}
     link.code="function _out(place){"+lookaheadMatch[4]+"};_out(this);"
     link.setAttribute("href","javascript:;"); link.setAttribute("title",""); link.style.cursor="pointer";
     }
     else { // run inline script code
     var code="function _out(place){"+lookaheadMatch[4]+"};_out(w.output);"
     code=code.replace(/document.write\(/gi,'place.innerHTML+=(');
     try { var out = eval(code); } catch(e) { out = e.description?e.description:e.toString(); }
     if (out && out.length) wikify(out,w.output,w.highlightRegExp,w.tiddler);
     }
     }
     w.nextMatch = lookaheadMatch.index + lookaheadMatch[0].length;
     }
     }
    } )
    //}}}

    Because we rely on strong decoupling and separation into self contained components, there is not much need for a common quasi-global namespace. Operations needing the cooperation of another subsystem will be delegated or even dispatched, consequently implementation code needs only the service acces points from "direct cooperation partner" subsystems. Hierarchical scopes besides classes are needed only when multiple subsystems share a set of common abstractions. Interface and Implementation use separate namespaces.

    !common definitions
    # surely there will be the need to use some macros (albeit code written in C++ can successfully avoid macros to some extent)
    # there will be one global ''Application State'' representation (and some application services)
    # we have some lib facilities, especially a common [[Time]] abstraction
    For these we have one special (bilingual) header file __lumiera.h__, which places in its C++ part some declarations into __namespace lumiera__. These declarations should be pulled into the specific namespaces or (still better) into the implementation //on demand//. There is no nesting (we deliberately don't want an symbol appearing //automatically// in every part of the system).

    !subsystem interface and facade
    These large scale interfaces reside in special namespaces "~XXX_interface" (where XXX is the subsystem). The accompanning //definitions// depend on the implementation namespace and are placed in the top-level source folder of the corresponding subsystem.

     &rarr; Example: [[Interfaces/Namespaces of the Session Subsystem(s)|InterfacesSession]]

    !contract checks and test code
    From experiences with other middle scale projects, I prefer having the test code in a separate tree (because test code easily doubles the number of source files). But of course it should be placed into the same namespace as the code being checked, or (better?) into a nested namespace "test". It is esp. desirable to have good coverage on the contracts of the subsystem interfaces and mayor components (while it is not always feasible or advisable to cover every implementation detail).

    &rarr; see also [[testsuite documentation in the main wiki|index.html#TestSuite]]

    * Subdir src/proc contains Interface within namespace proc_interface
    * Subdir src/proc/mobject contains commonly used entities (namespace mobject)
    ** nested namespace controller
    ** nested namespace builder
    ** nested namespace session
    * Subdir src/proc/engine (namespace engine) uses directly the (local) interface components StateProxy and ParamProvider; triggering of the render process is initiated by the controller and can be requested via the controller facade. Normally, the playback/render controller located in the backend will just use this interface and won't be aware of the build process at all.

    [img[Example: Interfaces/Namespaces of the ~Session-Subsystems|uml/fig130053.png]]

    Lumiera uses a 3-layered architecture. Separation between layers is crucial. Any communication between the layers regarding the normal operation of the application should //at least be initiated// through ''Layer abstraction interfaces''. This is a low-impact version of layering, because, aside from this triggering, direct cooperation of parts within the single Lumiera process is allowed, under the condition that is is implemented using additional abstractions (interfaces with implementation level granularity). We stick to the policy of //disallowing direct coupling of implementations located in different layers.//

    To implement such a structure, each layer separation interface actually is comprised of several parts:
    * an CLI interface to be installed into the InterfaceSystem
    * a facade interface defining the respective abstractions in terms of the implementation language (C or C++)
    * a implementation object on the service proiding side which is directly addressed by the implementation instantiated within the InterfaceSystem
    * a proxy object on the client side, which usually is given inline alongside with the CLI interface definition.

    !opening and closing
    Handling the lifecycle can be tricky, because client- and service-side need to carry out the opening and closing operations in sync and observing a specific call order to ensure calls get blocked already on the client side unless the whole interface compound is really up and running. To add to this complexity, plugins and built-in interfaces are handled differently regarding the question who is in charge of the lifecycle: interfaces are installed on the service side, whereas the loading of plugins is triggered from client side by requesting the plugin from the loader.
    Anyway, interfaces are resources which best should be managed automatically. At least within the C++ part of the application we can ensure this by using a handle class. This way the handling of plugins and interfaces can be unified; the only remaining difference is on what side (client or service provider side) the handle is instantiated, with the appropriate ctor parameters.

    Opening and accessing media files on disk poses several problems, most of which belong to the domain of Lumiera's data backend. Here, we focus on the questions related to making media data available to the EDL and the render engine. Each media will be represented by an MediaAsset object, which indeed could be a compound object (in case of MultichannelMedia). Building this asset object thus includes getting informations from the real file on disk. For delegating this to the backend, we use the following query interface:
    * {{{queryFile(char* name)}}} requests accessing the file and yields some (opaque) handle when successful.
    * {{{queryChannel(fHandle, int)}}} will then be issued in sequence with ascending index numbers, until it returns {{{NULL}}}.
    * the returned struct (pointer) will provide the following information:
    ** some identifier which can be used to create a name for the corresponding media (channel) asset
    ** some identifier characterizing the access method (codec) needed to get at the media data. This should be rather a high level description of the media stream type, e.g. "H264"
    ** some (opaque) handle usable for accessing this specific stream. When the render engine later on pulls data for this channel, it will pass this handle down to the backend.

    {{red{to be defined in more detail later...}}}

    &rarr; see "~MediaAccessFacade" for a (preliminary) interface definitioin
    &rarr; see "~MediaAccessMock" for a mock/test implementaion

    Used to actually implement the various kinds of [[Placement]] of ~MObjects. ~LocatingPin is the root of a hierarchy of different kinds of placing, constraining and locating a Media Object. Basically, this is an instance of the ''state pattern'': The user sees one Placement object with value semantics, but when the properties of the Placement are changed, actually a ~LocatingPin object (or rather a chain of ~LocatingPins) is changed within the Placement. Subclasses of ~LocatingPin implement different placing/constraining behaviour:
    * {{{FixedLocation}}} places a MObject to a fixed temporal position and track
    * {{{RelativeLocation}}} is used to atach the MObject to some other anchor MObject
    * //additional constraints, placement objectives, range restrictions, pattern rules will follow...//

    All sorts of "things" to be placed and manipulated by the user in the EDL. This interface abstracts the details and just supposes
    * the media object has a duration
    * it is allways //placed// in some manner, i.e. it is allways accessed via a [[Placement]]
    * {{red{and what else?}}}

    The ~MObjects Subsystem contains everything related to the [[EDL]] and the various Media Objects placed within. It is complemented by the Asset Management (see &rarr; [[Asset]]). Examples for [[MObjects|MObject]] being:
    * audio/video clips
    * effects and plugins
    * special facilities like mask and projector
    * [[Automation]] sets
    * labels and other (maybe functional) markup

    This Design strives to achieve a StrongSeparation between the low-level Structures used to carry out the actual rendering and the high level Entities living in the EDL and being manipulated by the user. In this high level view, the Objects are grouped and located by [[Placements|Placement]], providing a flexible and open way to express different groupings, locations and ordering constraints between the Media Objects.
    &rarr; EditingOperations
    &rarr; PlacementHandling
    &rarr; SessionOverview

    [img[Classess related to the EDL|uml/fig128133.png]]

    ''[[Lumiera|index.html]]''
    [[Proc-Layer|ProcLayer and Engine]]
    [[MObjects]]
    [[Implementation|ImplementationDetails]]
    [[Admin]]
    <<fullscreen>>

    Problem is: when removing an Asset, all corresponding MObjects need to disappear. This means, besides the obvious ~Ref-Link (MObject referring to an asset) we need backlinks or a sort of registry. And still worse: we need to remove the affected MObject from the object network in the EDL and rebuild the Fixture...

    &rarr; for a general design discussion see [[Relation of Clip and Asset|RelationClipAsset]]

    //Currently// Ichthyo considers the following approach:
    * all references between MObjects and Assets are implemented as __refcounting__ boost::shared_ptr
    * the opposite direction is also a __strong reference__, effectively keeping the clip-MO alive even if it is no longer in use in the session (note this is a cyclic dependency that needs to be actively broken on deletion). This design decision is based on  logical considerations (&rarr; see "deletions, Model-2" [[here|RelationClipAsset]]). This back-link is implemented by a Placement which is stored internally within the asset::Clip, it is needed for clean deletion of Assets, for GUI search functions and for adding the Clip to the EDL (again after been removed, or multiple times as if cloned).
    * MObjects and Assets implement an {{{unlink()}}} function releasing any internal links causing circular dependencies. It is always implemented such as to drop //optional// associations while retaining those associations mandatory for fulfilling the objects contract.
    * Instead of a delete, we call this unlink() function and let the shared_ptr handle the actual deletion. Thus, even if the object is already unlinked, it is still valid and usable as long as some other entity holds a smart-ptr. An ongoing render process for example can still use a clip asset and the corresponding media asset linked as parent, but this media asset's link to the dependant clip has already been cleared (and the media is no longer registered with the AssetManager of course).
    * so the back-link from dependant asset up to the parent asset is mandatory for the child asset to be functional, but preferably it should be {{{const}}} (only used for information retrieval)
    * the whole hierarchy has to be crafted accordingly, but this isn't much of a limitation
    * we don't use a registry, rather we model the real dependencies by individual dependency links. So a MediaAsset gets links to all Clips created from this Asset and by traversing this tree, we can handle the deletion
    * after the deletion, the Fixture needs to be rebuilt.
    * but any render processes still can have pointers to the Asset to be removed, and the shared_ptr will ensure, that the referred objects stay alive as long as needed.

    {{red{let's see if this approach works...}}}

    Contrary to the &rarr;[[Assets and MObjects|ManagementAssetRelation]], the usage pattern for [[render nodes|ProcNode]] is quite simple: All nodes are created together every time a new segment of the network is being build and are all needed together until this segment is re-built, at which point they can be thrown away altogether. While it would be easy to handle the nodes automatically by smart-ptr (the creation is accessible only by use of the {{{NodeFactory}}} anyways), it //seems advisable to care for a bulk allocation/deallocation here.// The reason being not so much the amount of memory (which is expected to be moderate), but the fact the build process can be triggered repeatedly several times a second when tweaking the EDL, which could lead to fragmentation and memory pressure.

    __10/2008__: the allocation mechanism can surely be improved later, but for now I am going for a simple implementation based on heap allocated objects owned by a vector or smart-ptrs. For each segment of the render nodes network, we have several families of objects, each of with will be maintained by a separate low-level memory manager (as said, for now implemented as vector of smart-ptrs). Together, they form an AllocationCluster; all objects contained in such a cluster will be destroyed together.

    <!--{{{-->
    <link rel='alternate' type='application/rss+xml' title='RSS' href='index.xml'/>
    <!--}}}-->

    <style type="text/css">#contentWrapper {display:none;}</style><div id="SplashScreen" style="border: 3px solid #ccc; display: block; text-align: center; width: 320px; margin: 100px auto; padding: 50px; color:#000; font-size: 28px; font-family:Tahoma; background-color:#eee;">loading <b>Proc-Layer</b> devel doku<blink> ...</blink><br><br><span style="font-size: 14px; color:red;">Requires Javascript.</span></div>

    The Interface asset::Media is a //key abstraction// It ties together several concepts and enables to deal with them on the interfaces in a uniform manner. Besides, as every Asset kind it belongs rather to the bookkeeping view: an asset::Media holds the specific properties and parametrisation of the media source it stands for. Regarding the __inward interface__ &mdash; as used from within the [[EDL]] or the [[Render Nodes|ProcNode]], it is irrelevant if any given asset::Media object stands for a complete media source, just a clip taken from this source or if a placeholder version of the real media source is used instead.
    [img[Asset Classess|uml/fig130437.png]]

    &rarr; see also LoadingMedia

    The Proc-Layer is designed such as to avoid unnecessary assumptions regarding the properties of the media data and streams. Thus, for anything which is not completely generic, we rely on an abstract [[type description|StreamTypeDescriptor]], which provides a ''Facade'' to an actual library implementation. This way, the fundamental operations can be invoked, like allocating a buffer to hold media data.

    In the context of Lumiera and especially in the Proc-Layer, __media implementation library__ means
    * a subsystem which allows to work with media data of a specific kind
    * such as to provide the minimal set of operations
    ** allocating a frame buffer
    ** describing the type of the data within such a buffer
    * and this subsystem or external library has been integrated to be used through Lumiera by writing adaptation code for accessing these basic operations through the [[implementation facade interface|StreamTypeImplFacade]]
    * such a link to an type implementation is registered and maintained by the [[stream type manager|STypeManager]]

    !Problem of the implementation data types
    Because we deliberately won't make any asumptions about the implementation library (besides the ones imposed indirectly by the facade interface), we can't integrate the data types of the library first class into the type system. All we can do is to use marker types and rely on the builder to have checked the compatibility of the actual data beforehand.
    It would be possible to circumvent this problem by requiring all supported implementation libraries to be known at compile time, because then the actual media implementation type could be linked to a facade type by generic programming. Indeed, Lumiera follows this route with regards to the possible kinds of MObject or [[Asset]] &mdash; but to the contraty, for the problem in question here, being able to include support for a new media data type just by adding a plugin by far outweights the benefits of compile-time checked implementation type selection. So, as a consequence of this design decision we //note the possibility of the media file type discovery code to be misconfigured// and select the //wrong implementation library at runtime.// And thus the render engine needs to be prepared for the source reading node of any pipe to flounder completely, and protect the rest of the system accordingly

    Of course: Cinelerra currently leaks memory and crashes regularilly. For the newly written code, besides retaining the same level of performance, a main goal is to use methods and techniques known to support the writing of quality code. So, besides the MultithreadConsiderations, a solid strategy for managing the ownership of allocated memory blocks is necessary right from start.

    !Problems
    # Memory management needs to work correct in a //fault tolerant environment//. That means that we need to be prepared to //handle on a non-local scale// some sorts of error conditions (without aborting the application). To be more precise: some error condition arises locally, which leads to a local abort and just the disabling/failing of some subsystem without affecting the application as a whole. This can happen on a regular base (e.g. rendering fails) and thus is __no excuse for leaking memory__
    # Some (not all) parts of the core application are non-deterministic. That means, we can't tie the memory management to any assumptions on behalf of the execution path

    !C++ solution
    First of all -- this doesn't concern //every// allocation. It rather means there are certain //dangerous areas// which need to be identified. Anyhow, instead of carrying inherent complexities of the problem into the solution, we should rather look for  common solution pattern(s) which help factoring out complexity.

    For the case here in question this seems to be the ''resource allocation is construction'' pattern. Which boils down to basically never using bare pointers when concerned with ownership. Instead, ownership should be handled by smart-pointers.

    !!usage scenarios
    # __existence is being used__: Objects just live for being referred to in a object network. In this case, use refcounting smart-pointers for every ref. (note: problem with cyclic refs)
    # __entity bound ownership__: Objects can be tied to some long living entity in the program, which holds the smart-pointer
    #* if the existence of these ref-holding entity can be //guaranteed// (as if by contract), then the other users can build a object network with conventional pointers
    #* otherwise, when the ref-holding entity //can disappear// in a regular program state, we need weak-refs and checking (because by our postulate the controlled resource needs to be destructed immediately, otherwise we would have the first case, existence == being used)

    !!!dangerous uses
    * the render nodes &rarr; [[detail analysis|ManagementRenderNodes]] {{red{TODO}}}
    * the MObjects in the EDL &rarr; [[detail analysis|ManagementMObjects]] {{red{TODO}}}
    * Asset - MObject relationship. &rarr; [[detail analysis|ManagementAssetRelation]] {{red{TODO}}}

    !!!rather harmless
    * Frames (buffers), because they belong to a given [[RenderProcess (=StateProxy)|StateProxy]] and are just passed in into the individual [[ProcNode]]s. This can be handled consistently with conventional methods.
    * each StateProxy belongs to one top-level call to the [[Controller-Facade|Controller]]
    * similar for the builder tools, which belong to a build process. Moreover, they are pooled and reused.
    * the EDL and the defined [[Asset]]s belong together to one Session. If the Session is closed, this means a internal shutdown of the whole ProcLayer, i.e. closing of all GUI representations and terminating all render processes. If these calles are implemented as blocking operations, we can assert that as long as any GUI representation or any render process is running, there is a valid Session and EDL.

    !using Factories
    And, last but not least, doing large scale allocations is the job of the backend. Exceptions being long-lived objects, like the Session or the EDL, which are created once and don't bear the danger of causing memory pressure. Besides, the ProcLayer code shouldn't issue "new" and "delete" when it comes in hand, rather it should use some centralized [[Factories]] for all allocation and freeing, so we can redirect these calls down to the backend, which may use pooling or special placement allocators or the like. The rationale is, for modern hardware/architectures, care has to be taken with heap allocations, esp. with many small objects and irregular usage patterns.

    Based on practical experiences, Ichthyo tends to consider Multichannel Media as the base case, while counting media files providing just one single media stream as exotic corner cases. This may seem counter intuitive at first sight; you should think of  it as an attempt to avoid right from start some of the common shortcomings found in many video editors, especially
    * having to deal with keeping a "link" between audio and video clips
    * silly limitations on the supported audio setups (e.g. "sound is mono, stereo or Dolby-5.1")
    * unnecessary complexity when dealing with more realistic setups, esp. when working on dialogue scenes
    * inability to edit stereoscopic (3D) video in a natural fashion

    !Compound Media
    [>img[Outline of the Build Process|uml/fig131333.png]]
    Basically, each [[media asset|MediaAsset]] is considered to be a compound of several elementary media (tracks), possibly of various different media kinds. Adding support for placeholders (''proxy clips'') at some point in future will add still more complexity (because then there will be even dependencies between some of these elementary media). To handle, edit and render compound media, we need to impose some structural limitations. But anyhow, we try to configure as much as possible already at the "asset level" and make the rest of the proc layer behave just according to the configuration given with each asset.

    So, when creating a clip out of such a compound media asset, the clip has to be a compound of elementary clips mirroring the given media asset's structure. Besides, it should be possible to //detach// and //attach// elementary clips from a compound clip. On the other hand, the [[Fixture]] created from the current state of the [[EDL]] is explicit to a great extent. So, in the Fixture we deal only with elementary clips placed to absolute positions, and thus the builder will see only simple non-compound clips and translate them into the corresponding source reading nodes.

    !Handling
    * from a Media asset, we can get a [[Processing Pattern (ProcPatt)|ProcPatt]] describing how to build a render pipeline for this media
    * we can create a Clip (MObject) from each Media, which will be linked back to the media asset internally.
    * moreover, creating a Clip will create and register a Clip asset as well, and this Clip asset will be tied to the original Clip and will show up in some special Category
    * media can be compound and the created Clips will mirror this compound structure
    * we distinguish elementay (non-compound) Clips from compound clips by concrete subtype. The builder can only handle elementary clips, because he needs to build a separate pipeline for every output channel. So the work of splitting common effect stacks for clips with several channels needs to be done when calculating the Fixture for the current EDL. The Builder expects to be able to build the render nodes corresponding to each entity found in the Fixture one by one.
    * the Builder gets at the ProcPatt (descriptor) of the underlying media for each clip and uses this description as a template to build the render pipeline. That is, the ProcPatt specifies the codec asset and maybe some additional effect assets (deinterlace, scale) necessary for feeding media data corresponding to this clip/media into the render nodes network.

    NodeCreaterTool is a [[visiting tool|VisitorUse]] used as second step in the [[Builder]]. Starting out from a [[Fixture]], the builder first [[divides the Timeline into segments|SegmentationTool]] and then processes each segment with the NodeCreaterTool to build a render nodes network (Render Engine) for this part of the timeline. While visiting individual Objects and Placements, the NodeCreaterTool creates and wires the necessary [[nodes|ProcNode]]

    The [[nodes|ProcNode]] are wired to form a "Directed Acyclic Graph"; each node knows its predecessor(s), but not its successor(s).  The RenderProcess is organized according to the ''pull principle'', thus we find an operation {{{pull()}}} at the core of this process. There is no such thing as an "engine object" calling nodes iteratively or table driven, rather, the nodes themselves issue recursive calls to their predecessor(s). For this to work, we need the nodes to adhere to a specific protocol:
    # Node is pulled, with a StateProxy object as parameter (encapsulating the access to the frames or buffers)
    # Node may now access current parameter values, using the state accessible via the StateProxy
    # using it's //input-output and wiring descriptor,// the Node creates a StateAdapter wrapping the StateProxy for allocating buffers and accessing the required input
    # StateAdapter first tries to get the output frames from the Cache in the Backend. In case of failure, a {{{process()}}} call is prepared by generating {{{pull()}}} call(s) for the input
    # as late as possible, typically on return, these recursive pull-calls have allocated a buffer containing the input data.
    # when input is ready prior to the {{{process()}}} call, output buffers will be allocated, either from the cache, or (if not caching) from the "parent" StateAdapter up the callstack.
    # after all buffers are available, the StateAdapter issues the {{{process()}}} call back to the originating node, which now may dereference the frame pointers and do its calculations
    # finally, when the {{{pull()}}} call returns, "parent" state originating the pull holds onto the buffers containing the calculated output result.
    some points to note:
    * the WiringDescriptor is {{{const}}} and precalculated while building (remember another thread may call in parallel)
    * when a node is "inplace-capable", input and output buffer may actually point to the same location
    * but there is no guarantee for this to happen, because the cache may be involved (and we can't overwrite the contents of a cache frame)
    * generally, a node may have N inputs and M output frames, which are expected to be processed in a single call

    &rarr; the [["mechanics" of the render process|RenderMechanics]]
    &rarr; more fine grained [[implementation details|RenderImplDetails]]

    We have to consider carefully how to handle the Creation of new class instances. Because, when done naively, it can defeat all efforts of separating subsystems, or &mdash; the other extreme &mdash; lead to a //switch-on-typeID//  programming style. We strive at a solution somewhere in the middle by utilizing __Abstract Factories__ on Interface or key abstraction classes, but providing specialized overloads for the different use cases. So in each use case we have to decide if we want to create a instance of some general concept (Interface), or if we have a direct collaboration and thus need the Factory to provide a more specific sub-Interface or even a concrete type.

    !Object creation use cases
    !![[Assets|Asset]]
    |!Action|>|!creates |
    |loading a media file|asset::Media, asset::Codec| |
    |viewing media|asset::Clip, session::Clip and Placement (on hold)| for the whole Media, if not already existent|
    |mark selection as clip|asset::Clip, session::Clip, Placement with unspec. LocatingPin| doesn't add to EDL|
    |loading Plugin|asset::Effect| usually at program startup|
    |create Session|asset::Track, asset::Pipe| |
    &rarr; [[creating and registering Assets|AssetCreation]]
    &rarr; [[loading media|LoadingMedia]]

    !![[MObjects|MObject]]
    |add media to EDL|asset::Clip, session::Clip, Placement with unspecified LocatingPin| creating whole-media clip on-the-fly |
    |add Clip to EDL|copy of Placement| creates intependent Placement of existing ~Clip-MO|
    |attach Effect|session::Effect, Placement with RelativeLocation| |
    |start using Automation|session::Auto, asset::Dataset, RelativeLocation Placement| |

    !Invariants
    when creating Objects, certain invariants have to be maintained. Because creating an Object can be considered an atomic operation and must not leave any related objects in an inconsistent state. Each of our interfaces implies some invariants:
    * every Placement has a Subject it places
    * MObjects are always created to be placed in some way or the other
    * [[Assets|Asset]] manage a dependency graph. Creating a derived Object (e.g. a Clip from a Media) implies a new dependency. (&rarr; [[memory management|ManagementAssetRelation]] relies on this)

    Cinelerra2 introduced OpenGL support for rendering previews. I must admit, I am very unhappy with this, because
    * it just supports some hardware
    * it makes building difficult
    * it can't handle all color models Cinelerra is capable of
    * it introduces a separate codepath including some complicated copying of video data into the textures (and back?)
    * it can't be used for rendering

    So my judgement would be: in contrary to a realtime/gaming application, for quality video editing it is not worth the effort implementing OpenGL support in all details and with all its complexity. I would accept ~OpenGL as an option, if it could be pushed down into a Library, so it can be handled and maintained transparently and doesnt bind our limited developer manpower.

    But because I know the opinions on this topc are varying (users tend to be delighted if they hear "~OpenGL", because it carries notion of "fast" and "power" todays) &mdash; I try to integrate ~OpenGL as apossibility into this design of the Render Engine. Obviousely, I have the hard requirement that it //must not jeopardize the code structure.//

    My proposed aproach is to treat OpenGL as a separate video raw data type, requiring separete and specialized [[Processing Nodes|ProcNode]] for all calculations. Thus the Builder could connect OpenGL nodes if it is possible to cover the whole render path for preview and fall back to the normal ~ProcNodes for all relevant renders

    A low-level abstraction within the [[Builder]] &mdash; it serves to encapsulate the details of making multi-channel connections between the render nodes: In some cases, a node can handle N channels internally, while in other cases we need to replicate the node N times and wire each channel individually.

    The operation point is provided by the current BuilderMould and used by the [[processing pattern|ProcPatt]] executing within this mould and conducting the current build step. The operation point's interface allows //to abstract//&nbsp; these details, as well as to //gain additional control//&nbsp; if necessary (e.g. addressing only one of the channels). The most prominent build instruction used within the processing patterns (which is the instruction {{{"attach"}}}) relies on the aforementioned //approach of abstracted handling,// letting the operation point determine automatically how to make the connection.

    This is possible because the operation point has been provided (by the mould) with informations about the media stream type to be wired, which, together with information accessible at the the [[render node interface|ProcNode]] and from the [[referred processing assets|ProcAsset]], with the help of the [[connection manager|ConManager]] allows to figure out what's possible and how to do the desired connections.

    The Lumiera Processing Layer is comprised of various subsystems and can be separated into a low-level and a high-level part. At the low-level end is the [[Render Engine|OverviewRenderEngine]] which basically is a network of render nodes cooperating closely with the Backend Layer in order to carry out the actual playback and media transforming calculations. Whereas on the high-level side we find several different [[Media Objects|MObjects]] that can be placed into the [[EDL]], edited and manipulated. This is complemented by the [[Asset Management|Asset]], which is the "bookkeeping view" of all the different "things" within each [[Session|SessionOverview]].

    There is rather strong separation between these two levels, and &mdash; <br/>correspondingly you'll encounter the data held within the Processing Layer organized in two different views, the ''high-level-model'' and the ''low-level-model''
    * from users (and GUI) perspective, you'll see a [[Session|SessionOverview]] with a timeline-like structure, where various [[Media Objects|MObjects]] are arranged and [[placed|Placement]]. By looking closer, you'll find that there are data connections and all processing is organized around processing chains or [[pipes|Pipe]], which can be either global (in the Session) or local (in real or [[virtual|VirtualClip]] clips)
    * when dealing with the actual calculations in the Engine (&rarr; see OverviewRenderEngine), you won't find any Tracks, Media Objects or Pipes &mdash; rather you'll find a network of interconnected [[render nodes|ProcNode]] forming the low level model. Each structurally constant segment of the timeline corresponds to a separate node network providing an ExitNode corresponding to each of the global pipes; pulling frames from them means running the engine.
    * it is the job of the [[Builder]] create and wire up this render nodes network when provided with a given hig-level-model. So, actually the builder (together with the so called [[Fixture]]) form an isolation layer in the middle, separating the //editing part&nbsp;// from the //processing part.//
    [img[Block Diagram|uml/fig128005.png]]

    Render Engine, [[Builder]] and [[Controller]] are closely related Subsystems. Actually, the [[Builder]] //creates// a newly configured Render Engine //for every// RenderProcess. Before doing so, it queries from the Session (or, to be more precise, from the [[Fixture]] within the current Session) all necessary Media Object Placement information. The [[Builder]] then derives from this information the actual assembly of [[Processing Nodes|ProcNode]] comprising the Render Engine. Thus:
     * the source of the build process is a sequence of absolute (explicit) [[Placements|Placement]] called the [[Playlist]]
     * the [[build process|BuildProcess]] is driven, configured and controlled by the [[Controller]] subsystem component. It encompasses the actual playback configuration and State of the System.
     * the resulting Render Engine is a list of [[Processors]], each configured to calculate a segment of the timeline with uniform properties. Each of these Processors in turn is a graph of interconnected ProcNode.s.

    see also: RenderEntities, [[two Examples (Object diagrams)|Examples]]

    [img[Overview: Components of the Renderengine|uml/fig128261.png]]

    <!--{{{-->
    <div class='header' macro='gradient vert [[ColorPalette::PrimaryLight]] [[ColorPalette::PrimaryMid]]'>
    	<div class='headerShadow'>
    		<span class='siteTitle' refresh='content' tiddler='SiteTitle'></span>&nbsp;
    		<span class='siteSubtitle' refresh='content' tiddler='SiteSubtitle'></span>
    	</div>
    	<div class='headerForeground'>
    		<span class='siteTitle' refresh='content' tiddler='SiteTitle'></span>&nbsp;
    		<span class='siteSubtitle' refresh='content' tiddler='SiteSubtitle'></span>
    	</div>
    </div>
    <!-- horizontal MainMenu -->
    <div id='topMenu' refresh='content' tiddler='MainMenu'></div>
    <!-- original MainMenu menu -->
    <!-- <div id='mainMenu' refresh='content' tiddler='MainMenu'></div> -->
    <div id='sidebar'>
    	<div id='sidebarOptions' refresh='content' tiddler='SideBarOptions'></div>
    	<div id='sidebarTabs' refresh='content' force='true' tiddler='SideBarTabs'></div>
    </div>
    <div id='displayArea'>
    	<div id='messageArea'></div>
    	<div id='tiddlerDisplay'></div>
    </div>
    <!--}}}-->

    A ParamProvider is the counterpart for (one or many) [[Parameter]] instances. It implements the value access function made available by the Parameter object to its clients.

    To give a concrete example:
    * a Fade Plugin needs the actual fade value for Frame t=xxx
    * the Plugin has a Parameter Object (from which we could query the information of this parameter being a continuous float function)
    * this Parameter Object provides a getValue() function, which is internally linked (i.e. by configuration) to a //Parameter Provider//
    * the actual object implementing the ParamProvider Interface could be a Automation MObject located somewhere in the EDL and would do bezier interpolation on a given keyframe set.
    * while building the Render Engine configuration actually at work, the Builder would have to setup this link between the Plugin Parameter Object and the ParamProvider; he can do so, because he sees the link between the Automation MObject and the corresponding Effect MObject

    &rarr; see the class diagram for [[Automation]]

    Parameters are all possibly variable control values used within the Render Engine. Contrast this with configuration values, which are considered to be fixed and need an internal reset of the application (or session) state to take effect.

    A ''Parameter Object'' provides a descriptor of the kind of parameter, together with a function used to pull the //actual value// of this parameter. Here, //actual// has a two-fold meaning:
    * if called without a time specification, it is either a global (but variable) system or session parameter or a default value for automated Parameters. (the intention is to treat this cases uniformly)
    * if called with a time specification, it is the query for an &mdash; probably interpolated &mdash; [[Automation]] value at this absolute time. The corresponding ParamProvider should fall back transparently to a default or session value if no time varying data is available

    {{red{TODO: define how Automation works}}}

    /***
    |<html><a name="Top"/></html>''Name:''|PartTiddlerPlugin|
    |''Version:''|1.0.6 (2006-11-07)|
    |''Source:''|http://tiddlywiki.abego-software.de/#PartTiddlerPlugin|
    |''Author:''|UdoBorkowski (ub [at] abego-software [dot] de)|
    |''Licence:''|[[BSD open source license]]|
    |''TiddlyWiki:''|2.0|
    |''Browser:''|Firefox 1.0.4+; InternetExplorer 6.0|
    !Table of Content<html><a name="TOC"/></html>
    * <html><a href="javascript:;" onclick="window.scrollAnchorVisible('Description',null, event)">Description, Syntax</a></html>
    * <html><a href="javascript:;" onclick="window.scrollAnchorVisible('Applications',null, event)">Applications</a></html>
    ** <html><a href="javascript:;" onclick="window.scrollAnchorVisible('LongTiddler',null, event)">Refering to Paragraphs of a Longer Tiddler</a></html>
    ** <html><a href="javascript:;" onclick="window.scrollAnchorVisible('Citation',null, event)">Citation Index</a></html>
    ** <html><a href="javascript:;" onclick="window.scrollAnchorVisible('TableCells',null, event)">Creating "multi-line" Table Cells</a></html>
    ** <html><a href="javascript:;" onclick="window.scrollAnchorVisible('Tabs',null, event)">Creating Tabs</a></html>
    ** <html><a href="javascript:;" onclick="window.scrollAnchorVisible('Sliders',null, event)">Using Sliders</a></html>
    * <html><a href="javascript:;" onclick="window.scrollAnchorVisible('Revisions',null, event)">Revision History</a></html>
    * <html><a href="javascript:;" onclick="window.scrollAnchorVisible('Code',null, event)">Code</a></html>
    !Description<html><a name="Description"/></html>
    With the {{{<part aPartName> ... </part>}}} feature you can structure your tiddler text into separate (named) parts.
    Each part can be referenced as a "normal" tiddler, using the "//tiddlerName//''/''//partName//" syntax (e.g. "About/Features"). E.g. you may create links to the parts, use it in {{{<<tiddler...>>}}} or {{{<<tabs...>>}}} macros etc.

    ''Syntax:''
    |>|''<part'' //partName// [''hidden''] ''>'' //any tiddler content// ''</part>''|
    |//partName//|The name of the part. You may reference a part tiddler with the combined tiddler name "//nameOfContainerTidder//''/''//partName//.|
    |''hidden''|When defined the content of the part is not displayed in the container tiddler. But when the part is explicitly referenced (e.g. in a {{{<<tiddler...>>}}} macro or in a link) the part's content is displayed.|
    |<html><i>any&nbsp;tiddler&nbsp;content</i></html>|<html>The content of the part.<br>A part can have any content that a "normal" tiddler may have, e.g. you may use all the formattings and macros defined.</html>|
    |>|~~Syntax formatting: Keywords in ''bold'', optional parts in [...]. 'or' means that exactly one of the two alternatives must exist.~~|
    <html><sub><a href="javascript:;" onclick="window.scrollAnchorVisible('Top',null, event)">[Top]</sub></a></html>

    !Applications<html><a name="Applications"/></html>
    !!Refering to Paragraphs of a Longer Tiddler<html><a name="LongTiddler"/></html>
    Assume you have written a long description in a tiddler and now you want to refer to the content of a certain paragraph in that tiddler (e.g. some definition.) Just wrap the text with a ''part'' block, give it a nice name, create a "pretty link" (like {{{[[Discussion Groups|Introduction/DiscussionGroups]]}}}) and you are done.

    Notice this complements the approach to first writing a lot of small tiddlers and combine these tiddlers to one larger tiddler in a second step (e.g. using the {{{<<tiddler...>>}}} macro). Using the ''part'' feature you can first write a "classic" (longer) text that can be read "from top to bottom" and later "reuse" parts of this text for some more "non-linear" reading.

    <html><sub><a href="javascript:;" onclick="window.scrollAnchorVisible('Top',null, event)">[Top]</sub></a></html>

    !!Citation Index<html><a name="Citation"/></html>
    Create a tiddler "Citations" that contains your "citations".
    Wrap every citation with a part and a proper name.

    ''Example''
    {{{
    <part BAX98>Baxter, Ira D. et al: //Clone Detection Using Abstract Syntax Trees.//
    in //Proc. ICSM//, 1998.</part>

    <part BEL02>Bellon, Stefan: //Vergleich von Techniken zur Erkennung duplizierten Quellcodes.//
    Thesis, Uni Stuttgart, 2002.</part>

    <part DUC99>Ducasse, Stéfane et al: //A Language Independent Approach for Detecting Duplicated Code.//
    in //Proc. ICSM//, 1999.</part>
    }}}

    You may now "cite" them just by using a pretty link like {{{[[Citations/BAX98]]}}} or even more pretty, like this {{{[[BAX98|Citations/BAX98]]}}}.

    <html><sub><a href="javascript:;" onclick="window.scrollAnchorVisible('Top',null, event)">[Top]</sub></a></html>

    !!Creating "multi-line" Table Cells<html><a name="TableCells"/></html>
    You may have noticed that it is hard to create table cells with "multi-line" content. E.g. if you want to create a bullet list inside a table cell you cannot just write the bullet list
    {{{
    * Item 1
    * Item 2
    * Item 3
    }}}
    into a table cell (i.e. between the | ... | bars) because every bullet item must start in a new line but all cells of a table row must be in one line.

    Using the ''part'' feature this problem can be solved. Just create a hidden part that contains the cells content and use a {{{<<tiddler >>}}} macro to include its content in the table's cell.

    ''Example''
    {{{
    |!Subject|!Items|
    |subject1|<<tiddler ./Cell1>>|
    |subject2|<<tiddler ./Cell2>>|

    <part Cell1 hidden>
    * Item 1
    * Item 2
    * Item 3
    </part>
    ...
    }}}

    Notice that inside the {{{<<tiddler ...>>}}} macro you may refer to the "current tiddler" using the ".".

    BTW: The same approach can be used to create bullet lists with items that contain more than one line.

    <html><sub><a href="javascript:;" onclick="window.scrollAnchorVisible('Top',null, event)">[Top]</sub></a></html>

    !!Creating Tabs<html><a name="Tabs"/></html>
    The build-in {{{<<tabs ...>>}}} macro requires that you defined an additional tiddler for every tab it displays. When you want to have "nested" tabs you need to define a tiddler for the "main tab" and one for every tab it contains. I.e. the definition of a set of tabs that is visually displayed at one place is distributed across multiple tiddlers.

    With the ''part'' feature you can put the complete definition in one tiddler, making it easier to keep an overview and maintain the tab sets.

    ''Example''
    The standard tabs at the sidebar are defined by the following eight tiddlers:
    * SideBarTabs
    * TabAll
    * TabMore
    * TabMoreMissing
    * TabMoreOrphans
    * TabMoreShadowed
    * TabTags
    * TabTimeline

    Instead of these eight tiddlers one could define the following SideBarTabs tiddler that uses the ''part'' feature:
    {{{
    <<tabs txtMainTab
     Timeline Timeline SideBarTabs/Timeline
     All 'All tiddlers' SideBarTabs/All
     Tags 'All tags' SideBarTabs/Tags
     More 'More lists' SideBarTabs/More>>
    <part Timeline hidden><<timeline>></part>
    <part All hidden><<list all>></part>
    <part Tags hidden><<allTags>></part>
    <part More hidden><<tabs txtMoreTab
     Missing 'Missing tiddlers' SideBarTabs/Missing
     Orphans 'Orphaned tiddlers' SideBarTabs/Orphans
     Shadowed 'Shadowed tiddlers' SideBarTabs/Shadowed>></part>
    <part Missing hidden><<list missing>></part>
    <part Orphans hidden><<list orphans>></part>
    <part Shadowed hidden><<list shadowed>></part>
    }}}

    Notice that you can easily "overwrite" individual parts in separate tiddlers that have the full name of the part.

    E.g. if you don't like the classic timeline tab but only want to see the 100 most recent tiddlers you could create a tiddler "~SideBarTabs/Timeline" with the following content:
    {{{
    <<forEachTiddler
     sortBy 'tiddler.modified' descending
     write '(index < 100) ? "* [["+tiddler.title+"]]\n":""'>>
    }}}
    <html><sub><a href="javascript:;" onclick="window.scrollAnchorVisible('Top',null, event)">[Top]</sub></a></html>

    !!Using Sliders<html><a name="Sliders"/></html>
    Very similar to the build-in {{{<<tabs ...>>}}} macro (see above) the {{{<<slider ...>>}}} macro requires that you defined an additional tiddler that holds the content "to be slid". You can avoid creating this extra tiddler by using the ''part'' feature

    ''Example''
    In a tiddler "About" we may use the slider to show some details that are documented in the tiddler's "Details" part.
    {{{
    ...
    <<slider chkAboutDetails About/Details details "Click here to see more details">>
    <part Details hidden>
    To give you a better overview ...
    </part>
    ...
    }}}

    Notice that putting the content of the slider into the slider's tiddler also has an extra benefit: When you decide you need to edit the content of the slider you can just doubleclick the content, the tiddler opens for editing and you can directly start editing the content (in the part section). In the "old" approach you would doubleclick the tiddler, see that the slider is using tiddler X, have to look for the tiddler X and can finally open it for editing. So using the ''part'' approach results in a much short workflow.

    <html><sub><a href="javascript:;" onclick="window.scrollAnchorVisible('Top',null, event)">[Top]</sub></a></html>

    !Revision history<html><a name="Revisions"/></html>
    * v1.0.6 (2006-11-07)
    ** Bugfix: cannot edit tiddler when UploadPlugin by Bidix is installed. Thanks to José Luis González Castro for reporting the bug.
    * v1.0.5 (2006-03-02)
    ** Bugfix: Example with multi-line table cells does not work in IE6. Thanks to Paulo Soares for reporting the bug.
    * v1.0.4 (2006-02-28)
    ** Bugfix: Shadow tiddlers cannot be edited (in TW 2.0.6). Thanks to Torsten Vanek for reporting the bug.
    * v1.0.3 (2006-02-26)
    ** Adapt code to newly introduced Tiddler.prototype.isReadOnly() function (in TW 2.0.6). Thanks to Paulo Soares for reporting the problem.
    * v1.0.2 (2006-02-05)
    ** Also allow other macros than the "tiddler" macro use the "." in the part reference (to refer to "this" tiddler)
    * v1.0.1 (2006-01-27)
    ** Added Table of Content for plugin documentation. Thanks to RichCarrillo for suggesting.
    ** Bugfix: newReminder plugin does not work when PartTiddler is installed. Thanks to PauloSoares for reporting.
    * v1.0.0 (2006-01-25)
    ** initial version
    <html><sub><a href="javascript:;" onclick="window.scrollAnchorVisible('Top',null, event)">[Top]</sub></a></html>

    !Code<html><a name="Code"/></html>
    <html><sub><a href="javascript:;" onclick="window.scrollAnchorVisible('Top',null, event)">[Top]</sub></a></html>
    ***/
    //{{{
    //============================================================================
    // PartTiddlerPlugin

    // Ensure that the PartTiddler Plugin is only installed once.
    //
    if (!version.extensions.PartTiddlerPlugin) {

    version.extensions.PartTiddlerPlugin = {
     major: 1, minor: 0, revision: 6,
     date: new Date(2006, 10, 7),
     type: 'plugin',
     source: "http://tiddlywiki.abego-software.de/#PartTiddlerPlugin"
    };

    if (!window.abego) window.abego = {};
    if (version.major < 2) alertAndThrow("PartTiddlerPlugin requires TiddlyWiki 2.0 or newer.");

    //============================================================================
    // Common Helpers

    // Looks for the next newline, starting at the index-th char of text.
    //
    // If there are only whitespaces between index and the newline
    // the index behind the newline is returned,
    // otherwise (or when no newline is found) index is returned.
    //
    var skipEmptyEndOfLine = function(text, index) {
     var re = /(\n|[^\s])/g;
     re.lastIndex = index;
     var result = re.exec(text);
     return (result && text.charAt(result.index) == '\n')
     ? result.index+1
     : index;
    }

    //============================================================================
    // Constants

    var partEndOrStartTagRE = /(<\/part>)|(<part(?:\s+)((?:[^>])+)>)/mg;
    var partEndTagREString = "<\\/part>";
    var partEndTagString = "</part>";

    //============================================================================
    // Plugin Specific Helpers

    // Parse the parameters inside a <part ...> tag and return the result.
    //
    // @return [may be null] {partName: ..., isHidden: ...}
    //
    var parseStartTagParams = function(paramText) {
     var params = paramText.readMacroParams();
     if (params.length == 0 || params[0].length == 0) return null;

     var name = params[0];
     var paramsIndex = 1;
     var hidden = false;
     if (paramsIndex < params.length) {
     hidden = params[paramsIndex] == "hidden";
     paramsIndex++;
     }

     return {
     partName: name,
     isHidden: hidden
     };
    }

    // Returns the match to the next (end or start) part tag in the text,
    // starting the search at startIndex.
    //
    // When no such tag is found null is returned, otherwise a "Match" is returned:
    // [0]: full match
    // [1]: matched "end" tag (or null when no end tag match)
    // [2]: matched "start" tag (or null when no start tag match)
    // [3]: content of start tag (or null if no start tag match)
    //
    var findNextPartEndOrStartTagMatch = function(text, startIndex) {
     var re = new RegExp(partEndOrStartTagRE);
     re.lastIndex = startIndex;
     var match = re.exec(text);
     return match;
    }

    //============================================================================
    // Formatter

    // Process the <part ...> ... </part> starting at (w.source, w.matchStart) for formatting.
    //
    // @return true if a complete part section (including the end tag) could be processed, false otherwise.
    //
    var handlePartSection = function(w) {
     var tagMatch = findNextPartEndOrStartTagMatch(w.source, w.matchStart);
     if (!tagMatch) return false;
     if (tagMatch.index != w.matchStart || !tagMatch[2]) return false;

     // Parse the start tag parameters
     var arguments = parseStartTagParams(tagMatch[3]);
     if (!arguments) return false;

     // Continue processing
     var startTagEndIndex = skipEmptyEndOfLine(w.source, tagMatch.index + tagMatch[0].length);
     var endMatch = findNextPartEndOrStartTagMatch(w.source, startTagEndIndex);
     if (endMatch && endMatch[1]) {
     if (!arguments.isHidden) {
     w.nextMatch = startTagEndIndex;
     w.subWikify(w.output,partEndTagREString);
     }
     w.nextMatch = skipEmptyEndOfLine(w.source, endMatch.index + endMatch[0].length);

     return true;
     }
     return false;
    }

    config.formatters.push( {
     name: "part",
     match: "<part\\s+[^>]+>",

     handler: function(w) {
     if (!handlePartSection(w)) {
     w.outputText(w.output,w.matchStart,w.matchStart+w.matchLength);
     }
     }
    } )

    //============================================================================
    // Extend "fetchTiddler" functionality to also recognize "part"s of tiddlers
    // as tiddlers.

    var currentParent = null; // used for the "." parent (e.g. in the "tiddler" macro)

    // Return the match to the first <part ...> tag of the text that has the
    // requrest partName.
    //
    // @return [may be null]
    //
    var findPartStartTagByName = function(text, partName) {
     var i = 0;

     while (true) {
     var tagMatch = findNextPartEndOrStartTagMatch(text, i);
     if (!tagMatch) return null;

     if (tagMatch[2]) {
     // Is start tag

     // Check the name
     var arguments = parseStartTagParams(tagMatch[3]);
     if (arguments && arguments.partName == partName) {
     return tagMatch;
     }
     }
     i += tagMatch[0].length;
     }
    }

    // Return the part "partName" of the given parentTiddler as a "readOnly" Tiddler
    // object, using fullName as the Tiddler's title.
    //
    // All remaining properties of the new Tiddler (tags etc.) are inherited from
    // the parentTiddler.
    //
    // @return [may be null]
    //
    var getPart = function(parentTiddler, partName, fullName) {
     var text = parentTiddler.text;
     var startTag = findPartStartTagByName(text, partName);
     if (!startTag) return null;

     var endIndexOfStartTag = skipEmptyEndOfLine(text, startTag.index+startTag[0].length);
     var indexOfEndTag = text.indexOf(partEndTagString, endIndexOfStartTag);

     if (indexOfEndTag >= 0) {
     var partTiddlerText = text.substring(endIndexOfStartTag,indexOfEndTag);
     var partTiddler = new Tiddler();
     partTiddler.set(
     fullName,
     partTiddlerText,
     parentTiddler.modifier,
     parentTiddler.modified,
     parentTiddler.tags,
     parentTiddler.created);
     partTiddler.abegoIsPartTiddler = true;
     return partTiddler;
     }

     return null;
    }

    // Hijack the store.fetchTiddler to recognize the "part" addresses.
    //

    var oldFetchTiddler = store.fetchTiddler ;
    store.fetchTiddler = function(title) {
     var result = oldFetchTiddler.apply(this, arguments);
     if (!result && title) {
     var i = title.lastIndexOf('/');
     if (i > 0) {
     var parentName = title.substring(0, i);
     var partName = title.substring(i+1);
     var parent = (parentName == ".")
     ? currentParent
     : oldFetchTiddler.apply(this, [parentName]);
     if (parent) {
     return getPart(parent, partName, parent.title+"/"+partName);
     }
     }
     }
     return result;
    };

    // The user must not edit a readOnly/partTiddler
    //

    config.commands.editTiddler.oldIsReadOnlyFunction = Tiddler.prototype.isReadOnly;

    Tiddler.prototype.isReadOnly = function() {
     // Tiddler.isReadOnly was introduced with TW 2.0.6.
     // For older version we explicitly check the global readOnly flag
     if (config.commands.editTiddler.oldIsReadOnlyFunction) {
     if (config.commands.editTiddler.oldIsReadOnlyFunction.apply(this, arguments)) return true;
     } else {
     if (readOnly) return true;
     }

     return this.abegoIsPartTiddler;
    }

    config.commands.editTiddler.handler = function(event,src,title)
    {
     var t = store.getTiddler(title);
     // Edit the tiddler if it either is not a tiddler (but a shadowTiddler)
     // or the tiddler is not readOnly
     if(!t || !t.abegoIsPartTiddler)
     {
     clearMessage();
     story.displayTiddler(null,title,DEFAULT_EDIT_TEMPLATE);
     story.focusTiddler(title,"text");
     return false;
     }
    }

    // To allow the "./partName" syntax in macros we need to hijack
    // the invokeMacro to define the "currentParent" while it is running.
    //
    var oldInvokeMacro = window.invokeMacro;
    function myInvokeMacro(place,macro,params,wikifier,tiddler) {
     var oldCurrentParent = currentParent;
     if (tiddler) currentParent = tiddler;
     try {
     oldInvokeMacro.apply(this, arguments);
     } finally {
     currentParent = oldCurrentParent;
     }
    }
    window.invokeMacro = myInvokeMacro;

    // Scroll the anchor anchorName in the viewer of the given tiddler visible.
    // When no tiddler is defined use the tiddler of the target given event is used.
    window.scrollAnchorVisible = function(anchorName, tiddler, evt) {
     var tiddlerElem = null;
     if (tiddler) {
     tiddlerElem = document.getElementById(story.idPrefix + tiddler);
     }
     if (!tiddlerElem && evt) {
     var target = resolveTarget(evt);
     tiddlerElem = story.findContainingTiddler(target);
     }
     if (!tiddlerElem) return;

     var children = tiddlerElem.getElementsByTagName("a");
     for (var i = 0; i < children.length; i++) {
     var child = children[i];
     var name = child.getAttribute("name");
     if (name == anchorName) {
     var y = findPosY(child);
     window.scrollTo(0,y);
     return;
     }
     }
    }

    } // of "install only once"
    //}}}

    /***
    <html><sub><a href="javascript:;" onclick="scrollAnchorVisible('Top',null, event)">[Top]</sub></a></html>

    !Licence and Copyright
    Copyright (c) abego Software ~GmbH, 2006 ([[www.abego-software.de|http://www.abego-software.de]])

    Redistribution and use in source and binary forms, with or without modification,
    are permitted provided that the following conditions are met:

    Redistributions of source code must retain the above copyright notice, this
    list of conditions and the following disclaimer.

    Redistributions in binary form must reproduce the above copyright notice, this
    list of conditions and the following disclaimer in the documentation and/or other
    materials provided with the distribution.

    Neither the name of abego Software nor the names of its contributors may be
    used to endorse or promote products derived from this software without specific
    prior written permission.

    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
    EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
    OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT
    SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
    INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
    TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
    BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
    CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
    ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
    DAMAGE.

    <html><sub><a href="javascript:;" onclick="scrollAnchorVisible('Top',null, event)">[Top]</sub></a></html>
    ***/

    Pipes play an central role within the Proc Layer, because for everything placed and handled within the EDL, the final goal is to get it transformed into data which can be retrieved at some pipe's exit port. Pipes are special facilities, rather like inventory, separate and not treated like all the other objects.
    We don't distinguish between "input" and "output" ports &mdash; rather, pipes are thought to be ''hooks for making connections to''. By following this line of thought, each pipe has an input side and an output side and is in itself something like a ''Bus'' or ''processing chain''. Other processing entities like effects and transitions can be placed (attached) at the pipe, resulting them to be appended to form this chain. Likewise, we can place [[wiring requests|WiringRequest]] to the pipe, meaning we want it connected so to send it's output to another destination pipe. The [[Builder]] may generate further wiring requests to fulfil the placement of other entities.
    Thus //Pipes are the basic building blocks// of the whole render network. We distinguish ''global available'' Pipes, which are like the sum groups of a mixing console, and the ''lokal pipe'' or [[source ports|ClipSourcePort]] of the individual clips, which exist only within the duration of the corresponding clip. The design //limits the possible kinds of pipes // to these two types &mdash; thus we can build local processing chains at clips and global processing chains at the global pipes of the session and that's all we can do. (because of the flexibility which comes with the concept of [[placements|Placement]], this is no real limitation)

    The GUI can connect the viewer(s) to some pipe (and moreover can use [[probe points|ProbePoint]] placed like effects and connected to some pipe), and likewise, when starting a ''render'', we get the opportunity to specify the pipes to pull the data from. Pulling data from some pipe is the (only) way to activate the render nodes network reachable from this pipe.

    &rarr; [[Handling of Tracks|TrackHandling]]
    &rarr; [[Handling of Pipes|PipeHandling]]

    !Identification
    Pipes are distinct objects and can be identified by their asset ~IDs. Besides, as for all [[structural assets|StructAsset]] there are extended query capabilities, including a symbolic pipe-id and a media (stream) type id. Any pipe can accept and deliver exactly one media stream kind (which may be inherently structured though, e.g. spatial sound systems or stereoscopic video)

    !creating pipes
    Pipe assets are created automatically by being used and referred. The [[Session]] holds a collection of global pipes ({{red{todo: implementation missing as of 5/08}}}), and further pipes can be created by using a new pipe reference in some placement. Moreover, every clip has an (implicit) [[source port|ClipSourcePort]], which will appear as pipe asset when first used (referred) while [[building|BuildProcess]]. Note that creating a new pipe implies using a [[processing pattern|ProcPatt]], which will be queried from the [[Defaults Manager|DefaultsManagement]] (resulting in the use of some preconfigured pattern or maybe the creation of a new ProcPatt object if necessary)

    !removal
    Deleting a Pipe is an advanced operation, because it includes finding and "detaching" all references, otherwise the pipe will leap back into existence immediately. Thus, global pipe entries in the Session and pipe references in [[locating pins|LocatingPin]] within any placement have to be removed, while clips using a given source port will be disabled. {{red{todo: implementation deferred}}}

    !using Pipes
    there is not much you can do directly with a pipe asset. It is an point of reference, after all. Any connection to some pipe is only temporarily done by a placement in some part of the timeline, so it isn't stored with the pipe. You can edit the (user visible) description an you can globally disable a pipe asset. The pipe's ID and media stream type of course are fixed, because any connection and referral (via the asset ID) is based on them. Later on, we should provide a {{{rewire(oldPipe, newPipe)}}} to search any ref to the {{{oldPipe}}} and try to rewrite it to use the {{{newPipe}}}, possibly with a new media stream type.
    Pipes are integrated with the [[management of defaults|DefaultsManagement]]. For example, any pipe uses implicitly some [[processing pattern|ProcPatt]] &mdash; it may default to the empty pattern. This feature enables to apply some standard wiring to the pipes (e.g a fader for audio, similar to the classic mixing consoles). This //is // a global property of the pipe, but &mdash; contrary to the stream type &mdash; this pattern may be switched

    A Placement represents a //relation:// it is always linked to a //Subject// (this being a [[Media Object|MObject]]) and has the meaning to //place// this Subject in some manner, either relatively to other Media Objects, or by some Constraint or simply absolute at (time, output). The latter case is especially important for the build process and thus represented by a special [[Sub-Interface ExplicitPlacement|ExplicitPlacement]]. Besides this simple cases, Placements can also express more specific kinds of "locating" an object, like placing a sound source at a pan position or placing a video clip at a given layer (above or below another video clip)

    So basically placements represent a query interface: you can allways ask the placement to find out about the position of the related object in terms of (time, output), and &mdash; depending on the specific object and situation &mdash; also about these additional [[placement derived dimensions|PlacementDerivedDimension]] like sound pan or layer order or similar things which also fit into the general concept of "placing" an object.

    The fact of being placed in the [[Session|SessionOverview]]/[[EDL]]is constitutive for all sorts of [[MObject]]s, without Placement they make no sense. Thus &mdash; technically &mdash; Placements act as ''smart pointers''. Of course, there are several kinds of Placements and they are templated on the type of MObject they are refering to. Placements can be //aggregated// to increasingly constrain the resulting "location" of the refered ~MObject. See &rarr; [[handling of Placements|PlacementHandling]] for more details

    For any [[media object|MObject]] within the session, we can allways at least query the time (reference/start) point and the output destination from the [[Placement]], by which the object is being handled. But the simple act of placing an object in some way, can &mdash; depending on the context &mdash; create additional degrees of freedom. To list some important examples:
    * placing a video clip overlapping with other clips on other tracks creates the possibility for the clip to be above another clip or to be combined in various other ways with the other clips at the same time position
    * placing a mono sound object plugged to a stereophoic output destination creates the freedom to define the pan position
    The Placement interface allows to query for these additional //parameter values derived from the fact of being placed.//

    !defining additional dimensions
    probably a LocatingPin but... {{red{TODO any details are yet unknown as of 5/08}}}
    !querying additional dimensions
    basically you resolve the Placement, yielding an ExplicitPlacement... {{red{TODO but any details of how additional dimensions are resolved is still undefined as of 5/08}}}

    [[Placement]]s are at the very core of all [[editing operations|EditingOperations]], because they act as handles (smart pointers) to access the [[media objects|MObject]] to be manipulated. Moreover, Placements are the actual content of the EDL(s) and Fixture and thus are small objects with //value semantics//. Many editing tasks include finding some Placement in the EDL or directly take a ref to some Placement. By acting on the Placement object, we can in some cases change parameters of the way the media object is placed (e.g. adjust an offset), while by dereferencing the Placement object, we access the "real" media object (e.g. for trimming its length). Placements are ''templated'' on the type of the actual ~MObject they refer to, thus defining the interface/methods usable on this object.

    Actually, the way each Placement locates its subject is implemented by one or several small LocatingPin objects, where subclasses of LocatingPin implement the various different methods of placing and resolving the final location. Notably, we can give a ~FixedLocation or we can atach to another ~MObject to get a ~RelativeLocation, etc. In the typical use case, these ~LocatingPins are added to the Placement, but never retrieved directly. Rather the Placement acts as a ''query interface'' for determining the location of the related object. Here, "location" can be thought of as encompassing multiple dimenstions at the same time. An object can be
    * located at a specific point in time
    * related to and plugged into a specific output or global bus
    * defined to have a position within some [[context-dependant additional dimensions|PlacementDerivedDimension]] like
    ** the pan position, either on the stereophoic base, or within a fully periphoic (spatial) sound system
    ** the layer order and overlay mode for video (normal, additive, subtractive, masking)
    ** the stereoscopic window position (depth parameter) for 3D video
    ** channel and parameter selection for MIDI data

    Placements have //value semantics,// i.e. we don't stress the identity of a placement object (~MObjects on the other hand //do have// a distinguishable identity): initially, you create a Placement parametrized to some specific kind by adding [[LocatingPin]]s (fixed, relative,...) and possibliy you use a subclass of {{{Placement<MObject>}}} to encode additional type information, say {{{Placement<Clip>}}}, but later on, you treat the placement polymorphically and don't care about its kind. The sole purpose of the placement's kind is to select some virtual function implementing the desired behaviour. There is no limitation to one single Placement per ~MObject, indeed we can have several different Placements of the same MObject (from a users point of view, these behave like being clones). Besides, we can ''aggregate'' additional [[LocatingPin]]s to one Placements, resulting in their properties and constraints being combined to yield the actual position of the referred ~MObject.

    !design decisions
    * the actual way of placing is implemented similar to the ''State Pattern'' by small embedded LocatingPin objects.
    * these LocatingPin objects form a ''decorator'' like chain
    * resolving into an ExplicitPlacement traverses this chain
    * //overconstraining// a placement is not an error, we just stop traversing the chain (ignoring the remaining additional Placements) at the moment the position is completely defined.
    * we provide subclasses to be able to form collections of e.g. {{{Placement<Effect>}}}, but we don't stress polymorphism here. Instead we stick to value semantics and explicitly ''allow slicing'' (all subclasses have the same size and layout and differ only in their vtable contents). This is justified, because for the polymorphical treating of ~MObjects the visitor mechanism used by the builder and the EDL is sufficiant, and I don't want to add another layer of complexity by making Placement something like {{{boost::variant}}}.
    * Why is the question how to access a ~MObject subinterface a Problem?
    *# we want the EDL/Fixture to be a collection of Placements. This means, either we store pointers, or Placement needs to be //one// unique type!
    *# but if Placement is //a single type//, then we can get only MObjects from a Placement.
    *# then either we had to do everything by a visitor (which gets the concrete subtype dynamically), or we'd end up switching on type.

    //This page is a scrapbook for working out the implementation of how to (re)build the [[Fixture]]//
    Structurally, (re)building the Fixture rather belongs to [[Session]]/[[EDLs|EDL]], but it is implemented very similar to the render engine build process: by treating all ~MObjects found in the various ~EDLs with a common [[visiting tool|VisitorUse]], this tool collects a simplified view with everyting made explicit, which can be pulled of as Fixture, i.e. (special kind of) EDL
    afterwards.
    * there is a //gathering phase// and a //solving phase//, the gathering is done by visiting.
    * during the gathering phase, there ''need to be a lock'' preventing any other edit operation.
    * the solving is delegated to the individual ~Placements. It is effectively a {{{const}}} operation creating a ExplicitPlacement (copy)
    * thus the Fixture contains these newly created ~ExplicitPlacements, refering to ~MObjects shared with the original Placements within the ~EDLs

    !!!prerequisites
    * Session and ~EDLs exist.
    * Pipes exist and are configured

    !!!postconditions
    * the Fixture contains one sorted timeline of ExplicitPlacement instances
    * Anything in this list is actually to be rendered
    * {{red{TODO: how to store and group the effects?}}}
    * any meta-clips or other funny things have been resolved to normal clips with placement
    * any multichannel clips has been broken down to elementary clips {{red{TODO: what is "elementary". e.g. stereo sound streams?}}}
    * any globally or otherwise strangely placed effects have been attached either to a clip or to some pipe
    * we have one unified list of tracks

    <<tasksum start>>
    <<taskadder below>>
    <<task >> work out how to get the processing of effects chained to some clip right
    <<task >> work out how to handle multichannel audio (and stereo video)

    !gathering phase
    !!preparing
    <<task>>what data collections to build?

    !!treating a Track
    <<task>>work out how to refer to pipes and do other config
    <<task>>get some uniqe identifier and get relevant properties

    !!treating a {{{Placement<Clip>}}}
    <<task>>check the direct enablement status
    <<task>>asses the compound status, maybe process recursively

    !!treating an {{{Placement<Effect>}}}
    <<task>>find out the application point {{red{really?}}}

    !solving phase
    <<task>>trigger solving on all placements
    <<task>>sort the resulting ~ExplicitPlacements

    <<tasksum end>>

    //This page is a scrapbook for working out the implementation of the builder//

    * NodeCreaterTool is a [[visiting tool|VisitorUse]]
    * the render engine to be built is contained as state within this tool object while it is passed around
    !!!prerequisites
    * Session and ~EDLs exist.
    * Pipes exist and are configured
    * Fixture contains ExplicitPlacement for every MObject to be rendered, and nothing else

    <<tasksum start>>
    <<taskadder>>

    !!preparing
    We need a way of addressing existing [[pipes|Pipe]]. Besides, as the Pipes and Tracks are referred by the Placements we are processing, they are guaranteed to exist.

    !!treating a Pipe
    <<task>>get the [[processing pattern|ProcPatt]] of the pipe by accessing the underlying pipe asset.
    <<task>>process this ProcPatt recursively

    !!treating a processing pattern
    <<task>>{{red{finally go ahead and define what a ProcPatt need to be...}}}

    !!treating a {{{Placement<Clip>}}}
    <<task>>get the ProcPatt of the underlying media (asset)
    <<task>>process the ProcPatt recursively
    <<task>>access the ClipSourcePort (which may be created on-the-fly)
    <<task>>enqueue an WiringRequest for connecting the source pipeline to the source port
    <<task>>process the clip's render pipe recursively (thus adding the camera etc.)
    <<task>>enqueue an WiringRequest for any placement to some pipe for this clip.
    * __note__: we suppose
    ** all wiring requests will be done after the processing of entities
    ** all effects placed to this clip will be processed after this clip (but before the wiring requests)

    !!treating an {{{Placement<Effect>}}}
    <<task>>{{red{how to assure that effecs are processed after clips/pipes??}}}
    <<task>>find out the application point
    <<task>>build a transforming node for the effect and insert it there

    !!postprocessing
    <<task>>sort and group the assembled list of [[wiring requests|WiringRequest]] by pipes

    <<tasksum end>>

    Playlist is a sequence of individual Render Engine Processors able to render a segment of the timeline. So, together these Processors are able to render the whole timeline (or part of the timeline if only a part has to be rendered).

    //Note, we have yet to specify how exactly the building and rendering will work together with the backend. There are several possibilities how to structure the Playlist//

    Open issues, Things to be worked out, Problems still to be solved...

    !!Parameter Handling
    The requirements are not quite clear; obviously Parameters are the foundation for getting automation right and for providing effect editing interfaces, so it seems to me we need some sort of introspection, i.e. Parameters need to be discovered, enumerated and described at runtime. (&rarr; see [[tag:automation|automation]])

    ''Automation Type'': Directly connected is the problem of handling the //type// of parameters sensible, including the value type of automation data. My first (somewhat naive) approach was to "make everything a double". But this soon leads into quite some of the same problems haunting the automation solution implemented in the current Cinelerra codebase. What makes the issue difficult is the fact we both need static diversity as well as dynamic flexibility. Usually, when combining hierarchies and templates, one has to be very careful; so I just note the problem down at the moment and will revisit it later, when I have a more clear understanding of the demands put onto the [[ProcNode]]s

    !!Treatment of Time (points) and Intervals
    At the moment we have no clear picture what is needed and what problems we may face in that domain.
    From experience, mainly with other applications, we can draw the following conclusions
    * drift and rounding errors are dangerous, because time in our context usually is understood as a fixed grid (Frames, samples...)
    * fine grained time values easily get very large
    * Cinelerra currently uses the approach of simply counting natural values for each media type separately. In an environment mixing several different media types freely, this seems a bit too simplistic (because it actually brings in the danger of rounding errors, just think at drop frame TC)

    !!Organizing of Output Channels
    How to handle the simultaneous rendering of several output streams (video, audio channels). Shall we treat the EDL as one entity containing different output channels, or should it rather be seen as a composite of several sub-~EDLs, each for only one output channel? This decision will be reflected in the overall structure of the network of render nodes: We could have a list of channel-output generating pipelines in each processor (for every segment), or we could have independently segmented lists of Processors for every output channel/type. The problem is, //it is not clear what approach to prefer at the moment//  because we are just guessing.

    !!Tracks, Channels, Layers
    Closely related to this is the not-so-obvious problem how to understand the common global structures found in most audio and video editing applications. Mostly, they stem from imitating hardware recording and editing solutions, thus easing the transition for professionals grown up with analogue hardware based media. But as digital media are the de-facto standard nowadays, we could rethink some of this accidental complexity introduced by sticking to the hardware tool metaphor.
    * is it really necessary to have fixed global tracks?
    * is it really helpful to feed "source tracks" into global processing busses/channels?
    Users accustomed with modern GUI applications typically expect that //everything is a object//  and can be pulled around and  manipulated individually. This seems natural at start, but raises the problem of providing a efficient workflow for handling larger projects and editing tasks. So, if we don't have a hard wired multitrack+bus architecture, we need some sort of templating to get the standard editing use case done efficiently.

    !!Compound and Multiplicity
    Simple relations can be hard wired. But, on the contrary, it would be as naive to define a Clip as having a Video track and two audio tracks, as it would be naive to overlook the problem of holding video and corresponding audio together. And, moreover, the default case has to be processed in a //straight forward// fashion, with as few tests and decisions as possible. So, basically each component participating in getting the core processing done has to mirror the structure pattern of the other parts, so that  processing can be done without testing and forking. But this leaves us with the problem where to put the initial knowledge about the structural pattern used for building up the compound structures and &mdash; especially &mdash; the problem how to treat different kinds of structural patterns, how to detect the pattern to be applied and how to treat multiple instances of the same structural pattern.

    One example of this problem is the [[handling of multichannel media|MultichannelMedia]]. Following the above reasoning, we end with having a [["structural processing pattern"|ProcPatt]], typically one video stream with MPEG decoder and a pair of audio streams which need either to be routed to some "left" and "right" output pipes, or have to be passed through a panning filter accordingly. Now the problem is: //create a new instance of this structure for each new media, or detect which media to subsume under a existing pattern instance.//

    !!Parallelism
    We need to work out guidelines for dealing with operations going on simultaneously. Certainly, this will divide the application in several different regions. As always, the primary goal is to avoid multithread problems altogether. Typically, this can be achieved by making matters explicit: externalizing state, make the processing subsystems stateless, queue and schedule tasks, use isolation layers.
    * the StateProxy is a key for the individual render processes state, which is managed in separate [[StateFrame]]s in the backend. The [[processing network|ProcNode]] is stateless.
    * the [[Fixture]] provides an isolation layer between the renderengine and the Session/EDL
    * all EditingOperations are not threadsafe intentionally, because they are [[scheduled|ProcLayerScheduler]]

    The current Lumiera architecture separates functionality into three Layers: __GUI__, __Proc__ and __Backend__.

    While the Backend is responsible for Data access and management and for carrying out the computation intensive media opteratons, the middle Layer or ~Proc-Layer contains [[assets|Asset]] and [[Session]], i.e. the user-visible data model and provides configuration and behaviour for these entities. Besides, he is responsible for [[building and configuring|Builder]] the [[render engine|RenderEngine]] based on the current Session state.

    All Assets of kind asset::Proc represent //processing algorithms// in the bookkeeping view. They enable loading, browsing and maybe even parametrizing all the Effects, Plugins and Codecs available for use within the Lumiera Session.

    Besides, they provide an important __inward interface__ for the [[ProcNode]]s, which will use these asset entries to dispatch the actual processing call when rendering.

    {{red{todo: the naming scheme??}}}

    [img[Asset Classess|uml/fig131077.png]]

    The middle Layer of our current Architecture plan, i.e. the layer managing all processing and manipulation, while the actual data handling is done in the backend and the user interaction belongs to the GUI Layer.

    &rarr; see the [[Overview]]

    The Render Engine is the part of the application doing the actual video calculations. Its operations are guided by the Objects and Parameters edited by the user in [[the EDL|EDL]] and it retrieves the raw audio and video data from the [[Data backend|backend.html]]. Because the inner workings of the Render Engine are closely related to the structures used in the EDL, this design covers [[the aspect of objects placed into the EDL|MObjects]] as well.
    <<<
    ''Status'': started out as design draft in summer '07, Ichthyo is now in the middle of a implementing the foundations and main structures in C++
    * basic [[AssetManager]] working
    * [[Builder]] implementation is on the way (&rarr;[[more|BuilderPrimitives]])
    * made a first draft of how to wire and operate procesing nodes (&rarr;[[more|RenderMechanics]])
    * working out the relation of [[Project, Timelines and Sequences|TimelineSequences]]
    * first attempts to [[integrate with the GUI|GuiIntegration]]
    <<<

    !Summary
    We have several kinds of "things" organized as [[assets|Asset]] in the AssetManager, like media, clips, effects, codecs, configuration templates. Within the context of the [[EDL]], we can use these as [["Media Objects"|MObjects]] &mdash; especially, we can [[place|Placement]] them in various kinds within the EDL and relative to one another. Basically, this is the [[editing work|EditingOperations]] done by the user.

    Now, from any given configuration within the EDL, we create sort or a frozen- and tied-down snapshot, here called [["Fixture"|Fixture]], containing all currently active ~MObjects, broken down to elementary parts and made explicit if necessary. This Fixture acts as a isolation layer towards the Render Engine. We will hand it over to the  [[Builder]], which in turn will transform it into a network of connected [[render nodes|ProcNode]]. This network //is// the [[Render Engine|OverviewRenderEngine]].

    The system is ''open'' inasmuch every part mirrors the structure of corresponding parts in adjacent subsystems, and the transformation of any given structure from one subsystem (e.g. Asset) to another (e.g. Render Engine) is done with minimal "magic". So the whole system should be able to handle completely new structures mostly by adding new configurations and components, without much need of rewriting basic workings.

    !!see also
    &rarr; [[Overview]] of Subsystems and Components, and DesignGoals
    &rarr; [[An Introduction|WalkThrough]] discussing the central points of this design
    &rarr; [[Overview Session (high level model)|SessionOverview]]
    &rarr; [[Overview Render Engine (low level model)|OverviewRenderEngine]]
    &rarr; BuildProcess and RenderProcess
    &rarr; [[Two Examples|Examples]] (Object diagrams)
    &rarr; how [[Automation]] works
    &rarr; [[Problems|ProblemsTodo]] to be solved and notable [[design decisions|DesignDecisions]]
    &rarr; [[Implementation Details|ImplementationDetails]] {{red{WIP}}}

    A data processing node within the Render Engine. Its key feature is the possibility to pull from it one (freely addressable) [[Frame]] of calculated data. Further, each ~ProcNode has the ability to be wired with other nodes and [[Parameter Providers|ParamProvider]]

    !! {{red{open questions}}}
    * how to address a node
    * how to type them
    * how to discover the number and type of the ports
    * how to discover the possible parameter ports
    * how to define and query for additional capabilities

    &rarr; see also the [[open design process draft|http://www.pipapo.org/pipawiki/Lumiera/DesignProcess/DesignRenderNodesInterface]]
    &rarr; see [[mem management|ManagementRenderNodes]]
    &rarr; see RenderProcess

    This special type of [[structural Asset|StructAsset]] represents information how to build some part of the render engine's processing nodes network. Processing patterns can be thought of as a blueprint or micro program for construction. Most notably, they are used for creating nodes reading, decoding and delivering source media material to the render network, and they are used for building the output connection via faders, summation or overlay nodes to the global pipes (busses). Each [[media Asset|MediaAsset]] has associated processing patterns describing the codecs and other transformations needed to get at the media data of this asset. (and because media assets are typically compound objects, the referred ~ProcPatt will be compound too). Similarily, for each stream kind, we can retrieve a processing pattern for making output connections. Obviously, the possibilities opened by using processing patterns go far beyond.

    Technically, a processing pattern is a list of building instructions, which will be //executed// by the [[Builder]] on the render node network under construction. This implies the possibility to define further instruction kinds when needed in future; at the moment the relevant sorts of instructions are
    * attach the given sequence of nodes to the specified point
    * recursively execute a nested ~ProcPatt
    More specifically, a sequence of nodes is given by a sequence of prototypical effect and codec assets (and from each of them we can create the corresponding render node). And the point to attach these nodes is given by an identifier &mdash; in most cases just "{{{current}}}", denoting the point the builder is currently working at, when treating some placed ~MObject which in turn yielded this processing pattern in question.

    Like all [[structural assets|StructAsset]], ~ProcPatt employs a special naming scheme within the asset name field, which directly mirrors its purpose and allows to bind to existing processing pattern instances when needed. {{red{TODO: that's just the general idea, but really it will rather use some sort of tags. Yet undefined as of 5/08}}} The idea is letting all assets in need of a similar processing pattern refer to one shared ~ProcPatt instance. For example, within a MPEG video media asset, at some point there will be a ~ProcPatt labeled "{{{stream(mpeg)}}}". In consequence, all MPEG video will use the same pattern of node wiring. And, of course, this pattern could be changed, either globally, or by binding a single clip to some other processing pattern (for making a punctual exception from the general rule)

    !!defining Processing Patterns
    The basic working set of processing patterns can be expected to be just there (hard wired or default configuration, mechanism for creating an sensible fallback). Besides, the idea is that new processing patterns can be added via rules in the session and then referred to by other rules controlling the build process. Any processing pattern is assembled by adding individual build instructions, or by including another (nested) processing pattern.

    !!retrieving a suitable Processing Pattern
    For a given situation, the necessary ProcPatt can be retrieved by issuing a [[configuration query|ConfigQuery]]. This query should include the needed capabilities in predicate form (technically this query is a Prolog goal), but it can leave out informations by just requesting "the default" &rarr; see DefaultsManagement

    !!how does this actually work?
    Any processing pattern needs the help of a passive holder tool suited for a specific [[building situation|BuilderPrimitives]]; we call these holder tools [[building moulds|BuilderMould]]. Depending on the situation, the mould has been armed up by the builder with the involved objects to be connected and extended. So, just by issuing the //location ID// defined within the individual build instruction (in most cases simply {{{"current"}}}), the processing pattern can retrieve the actual render object to use for building from the mould it is executed in.

    !!errors and misconfiguration
    Viewed as a micro program, the processing patterns are ''weak typed'' &mdash; thus providing the necessary flexibility within an otherwise strong typed system. Consequently, the builder assumes they are configured //the right way// &mdash; and will just bail out when this isn't the case, marking the related part of the high-level model as erroneous.
    &rarr; see BuilderErrorHandling for details

    a given Render Engine configuration is a list of Processors. Each Processor in turn contains a Graph of ProcNode.s to do the acutal data processing. In order to cary out any calculations, the Processor needs to be called with a StateProxy containing the state information for this RenderProcess

    //obviously, getting this one to work requires quite a lot of technical details to be planned and implemented.// This said...
    The intention is to get much more readable ("declarative") and changeable configuration as by programming the decision logic literately within the implementation of some object.

    !Draft
    As an example, specifying how a Track can be configured for connecting automatically to some "mpeg" bus (=pipe)
    {{{
    resolve(O, Cap) :- find(O), capabilities(Cap).
    resolve(O, Cap) :- make(O), capabilities(Cap).
    capabilities(Q) :- call(Q).

    stream(T, mpeg) :- type(T, track), type(P, pipe), resolve(P, stream(P,mpeg)), place_to(P, T).
    }}}

    Then, running the goal {{{:-resolve(T, stream(T,mpeg)).}}} would search a Track object, try to retrieve a pipe object with stream-type=mpeg and associate the track with this pipe. This relies on a predicate "stream(P,mpeg)" implemented (natively) for the pipe object. So, "Cap" is the query issued from calling code &mdash; here {{{stream(T,mpeg)}}}, the type guard {{{type(T, track)}}} will probably be handled or inserted automatically, while the predicate implementations for find/1, make/1, stream/2, and place_to/2 are to be provided by the target types.
    * __The supporting system__ had to combine several code snippets into one rule system to be used for running queries, with some global base rules, rules injected by each individual participating object kind and finally user provided rules added by the current session. The actual query is bound to "Cap" (and consequently run as a goal by {{{call(Q)}}}). The implementation needs to provide a symbol table associating variable terms (like "T" or "P") to C/C++ object types, enabling the participating object kinds to register their specific predicate implementations. This is crucial, because there can be no general scheme of object-provided predicates (for each object kind different predicates make sense, e.g. [[pipes|PipeHandling]] have other possibilities than [[wiring requests|WiringRequest]]). Basically, a query issues a Prolog goal, which in turn evaluates domain specific predicates provided by the participating objects and thus calls back into C/C++ code. The supporting system maintains the internal connection (via the "type" predicate) such that from Prolog viewpoint it looks as if we were binding Variables directly to object instances. (there are some nasty technical details because of the backtracking nature of Prolog evaluations which need to be hidden away)
    * Any __participating object kind__ needs a way to declare domain specific predicates, thus triggering the registration of the necessary hooks within the supporting system. Moreover, it should be able to inject further prolog code (as shown in the example above with the {{{strem(T, mpeg)}}} predicate. For each of these new domain specific predicates, there needs to be a functor which can be invoked when the C implementation of the predicate is called from Prolog (in some cases even later, when the final solution is "executed", e.g. a new instance has been created and now some properties need to be set).

    !!a note on Plugins
    In the design of the Lumiera Proc Layer done thus far, we provide //no possibility to introduce a new object kind// into the system via plugin interface. The system uses a fixed collection of classes intended to cover all needs (Clip, Effect, Track, Pipe, Label, Automation, ~Macro-Clips). Thus, plugins will only be able to provide new parametrisations of existing classes. This should not be any real limitation, because the whole system is designed to achieve most of its functionality by freely combining rather basic object kinds. As a plus, it plays nicely with any plain-C based plugin interface. For example, we will have C++ adapter classes for the most common sorts of effect plugin (pull system and synchronous frame-by-frame push with buffering) with a thin C adaptation layer for the specific external plugin systems used. Everything beyond this point can be considered "condiguration data" (including the actual plugin implementation to be loaded)

    /***
    |''Name:''|RSSReaderPlugin|
    |''Description:''|This plugin provides a RSSReader for TiddlyWiki|
    |''Version:''|1.1.1|
    |''Date:''|Apr 21, 2007|
    |''Source:''|http://tiddlywiki.bidix.info/#RSSReaderPlugin|
    |''Documentation:''|http://tiddlywiki.bidix.info/#RSSReaderPluginDoc|
    |''Author:''|BidiX (BidiX (at) bidix (dot) info)|
    |''Credit:''|BramChen for RssNewsMacro|
    |''License:''|[[BSD open source license|http://tiddlywiki.bidix.info/#%5B%5BBSD%20open%20source%20license%5D%5D ]]|
    |''~CoreVersion:''|2.2.0|
    |''OptionalRequires:''|http://www.tiddlytools.com/#NestedSlidersPlugin|
    ***/
    //{{{
    version.extensions.RSSReaderPlugin = {
    	major: 1, minor: 1, revision: 1,
    	date: new Date("Apr 21, 2007"),
    	source: "http://TiddlyWiki.bidix.info/#RSSReaderPlugin",
    	author: "BidiX",
    	coreVersion: '2.2.0'
    };

    config.macros.rssReader = {
    	dateFormat: "DDD, DD MMM YYYY",
    	itemStyle: "display: block;border: 1px solid black;padding: 5px;margin: 5px;", //useed  '@@'+itemStyle+itemText+'@@'
    	msg:{
    		permissionDenied: "Permission to read preferences was denied.",
    		noRSSFeed: "No RSS Feed at this address %0",
    		urlNotAccessible: " Access to %0 is not allowed"
    	},
    	cache: [], 	// url => XMLHttpRequest.responseXML
    	desc: "noDesc",

    	handler: function(place,macroName,params,wikifier,paramString,tiddler) {
    		var desc = params[0];
    		var feedURL = params[1];
    		var toFilter = (params[2] ? true : false);
    		var filterString = (toFilter?(params[2].substr(0,1) == ' '? tiddler.title:params[2]):'');
    		var place = createTiddlyElement(place, "div", "RSSReader");
    		wikify("^^<<rssFeedUpdate "+feedURL+" [[" + tiddler.title + "]]>>^^\n",place);
    		if (this.cache[feedURL]) {
    			this.displayRssFeed(this.cache[feedURL], feedURL, place, desc, toFilter, filterString);
    		}
    		else {
    			var r = loadRemoteFile(feedURL,config.macros.rssReader.processResponse, [place, desc, toFilter, filterString]);
    			if (typeof r == "string")
    				displayMessage(r);
    		}

    	},

    	// callback for loadRemoteFile
    	// params : [place, desc, toFilter, filterString]
    	processResponse: function(status, params, responseText, url, xhr) { // feedURL, place, desc, toFilter, filterString) {
    		if (window.netscape){
    			try {
    				if (document.location.protocol.indexOf("http") == -1) {
    					netscape.security.PrivilegeManager.enablePrivilege("UniversalBrowserRead");
    				}
    			}
    			catch (e) { displayMessage(e.description?e.description:e.toString()); }
    		}
    		if (xhr.status == httpStatus.NotFound)
    		 {
    			displayMessage(config.macros.rssReader.noRSSFeed.format([url]));
    			return;
    		}
    		if (!status)
    		 {
    			displayMessage(config.macros.rssReader.noRSSFeed.format([url]));
    			return;
    		}
    		if (xhr.responseXML) {
    			// response is interpreted as XML
    			config.macros.rssReader.cache[url] = xhr.responseXML;
    			config.macros.rssReader.displayRssFeed(xhr.responseXML, params[0], url, params[1], params[2], params[3]);
    		}
    		else {
    			if (responseText.substr(0,5) == "<?xml") {
    				// response exists but not return as XML -> try to parse it
    				var dom = (new DOMParser()).parseFromString(responseText, "text/xml");
    				if (dom) {
    					// parsing successful so use it
    					config.macros.rssReader.cache[url] = dom;
    					config.macros.rssReader.displayRssFeed(dom, params[0], url, params[1], params[2], params[3]);
    					return;
    				}
    			}
    			// no XML display as html
    			wikify("<html>" + responseText + "</html>", params[0]);
    			displayMessage(config.macros.rssReader.msg.noRSSFeed.format([url]));
    		}
    	},

    	// explore down the DOM tree
    	displayRssFeed: function(xml, place, feedURL, desc, toFilter, filterString){
    		// Channel
    		var chanelNode = xml.getElementsByTagName('channel').item(0);
    		var chanelTitleElement = (chanelNode ? chanelNode.getElementsByTagName('title').item(0) : null);
    		var chanelTitle = "";
    		if ((chanelTitleElement) && (chanelTitleElement.firstChild))
    			chanelTitle = chanelTitleElement.firstChild.nodeValue;
    		var chanelLinkElement = (chanelNode ? chanelNode.getElementsByTagName('link').item(0) : null);
    		var chanelLink = "";
    		if (chanelLinkElement)
    			chanelLink = chanelLinkElement.firstChild.nodeValue;
    		var titleTxt = "!![["+chanelTitle+"|"+chanelLink+"]]\n";
    		var title = createTiddlyElement(place,"div",null,"ChanelTitle",null);
    		wikify(titleTxt,title);
    		// ItemList
    		var itemList = xml.getElementsByTagName('item');
    		var article = createTiddlyElement(place,"ul",null,null,null);
    		var lastDate;
    		var re;
    		if (toFilter)
    			re = new RegExp(filterString.escapeRegExp());
    		for (var i=0; i<itemList.length; i++){
    			var titleElm = itemList[i].getElementsByTagName('title').item(0);
    			var titleText = (titleElm ? titleElm.firstChild.nodeValue : '');
    			if (toFilter && ! titleText.match(re)) {
    				continue;
    			}
    			var descText = '';
    			descElem = itemList[i].getElementsByTagName('description').item(0);
    			if (descElem){
    				try{
    					for (var ii=0; ii<descElem.childNodes.length; ii++) {
    						descText += descElem.childNodes[ii].nodeValue;
    					}
    				}
    				catch(e){}
    				descText = descText.replace(/<br \/>/g,'\n');
    				if (desc == "asHtml")
    					descText = "<html>"+descText+"</html>";
    			}
    			var linkElm = itemList[i].getElementsByTagName("link").item(0);
    			var linkURL = linkElm.firstChild.nodeValue;
    			var pubElm = itemList[i].getElementsByTagName('pubDate').item(0);
    			var pubDate;
    			if (!pubElm) {
    				pubElm = itemList[i].getElementsByTagName('date').item(0); // for del.icio.us
    				if (pubElm) {
    					pubDate = pubElm.firstChild.nodeValue;
    					pubDate = this.formatDateString(this.dateFormat, pubDate);
    					}
    					else {
    						pubDate = '0';
    					}
    				}
    			else {
    				pubDate = (pubElm ? pubElm.firstChild.nodeValue : 0);
    				pubDate = this.formatDate(this.dateFormat, pubDate);
    			}
    			titleText = titleText.replace(/\[|\]/g,'');
    			var rssText = '*'+'[[' + titleText + '|' + linkURL + ']]' + '' ;
    			if ((desc != "noDesc") && descText){
    				rssText = rssText.replace(/\n/g,' ');
    				descText = '@@'+this.itemStyle+descText + '@@\n';
    				if (version.extensions.nestedSliders){
    					descText = '+++[...]' + descText + '===';
    				}
    				rssText = rssText + descText;
    			}
    			var story;
    			if ((lastDate != pubDate) && ( pubDate != '0')) {
    				story = createTiddlyElement(article,"li",null,"RSSItem",pubDate);
    				lastDate = pubDate;
    			}
    			else {
    				lastDate = pubDate;
    			}
    			story = createTiddlyElement(article,"div",null,"RSSItem",null);
    			wikify(rssText,story);
    		}
    	},

    	formatDate: function(template, date){
    		var dateString = new Date(date);
    		// template = template.replace(/hh|mm|ss/g,'');
    		return dateString.formatString(template);
    	},

    	formatDateString: function(template, date){
    		var dateString = new Date(date.substr(0,4), date.substr(5,2) - 1, date.substr(8,2)
    			);
    		return dateString.formatString(template);
    	}

    };

    config.macros.rssFeedUpdate = {
    	label: "Update",
    	prompt: "Clear the cache and redisplay this RssFeed",
    	handler: function(place,macroName,params) {
    		var feedURL = params[0];
    		var tiddlerTitle = params[1];
    		createTiddlyButton(place, this.label, this.prompt,
    			function () {
    				if (config.macros.rssReader.cache[feedURL]) {
    					config.macros.rssReader.cache[feedURL] = null;
    			}
    			story.refreshTiddler(tiddlerTitle,null, true);
    		return false;});
    	}
    };

    //}}}

    What is the Role of the asset::Clip and how exactly are Assets and (Clip)-MObjects related?

    First of all: ~MObjects are the dynamic/editing/manipulation view, while Assets are the static/bookkeeping/searching/information view of the same entities. Thus, the asset::Clip contains the general configuration, the ref to the media and descriptive properties, while all parameters being "manipulated" belong to the session::Clip (MObject). Besides that, the practical purpose of asset::Clip is that you can save and remember some selection as a Clip (Asset), maybe even attach some informations or markup to it, and later be able to (re)create a editable representation in the Session (the GUI could implement this by allowing to drag from the asset::Clip GUI representation to the timeline window)

    !!dependencies
    The session::Clip (frequently called "clip-MO", i.e. the MObject) //depends on the Asset.// It can't exist without the Asset, because the Asset is needed for rendering. The other direction is different: the asset::Clip knows that there is a dependant clip-MO, there could be //at most one// such clip-MO depending on the Asset, but the Asset can exist without the clip-MO (it gives the possibility to re-create the clip-MO).

    !!deletions
    When the Asset or the corresponding asset::Media is deleted, the dependant clip-MO has to disappear. And the opposite direction?
    * Model-1: asset::Clip has a weak ref to the clip-MO. Consequently, the clip-MO can go out of scope and disappear, so the asset::Clip has to maintain the information of the clip's dimensions (source position and length) somewhere. Because of MultichannelMedia, this is not so simple as it may look at first sight.
    * Model-2: asset::Clip holds a smart ptr to the clip-MO, thus effectively keeping it alive. __obviously the better choice__
    In either case, we have to solve the ''problem of clip asset proliferation''

    !!multiplicity and const-ness
    The link between ~MObject and Asset should be {{{const}}}, so the clip can't change the media parameters. Because of separation of concerns, it would be desirable that the Asset can't //edit// the clip either (meaning {{{const}}} in the opposite direction as well). But unfortunately the asset::Clip is in power to delete the clip-MO and, moreover, handles out a smart ptr ([[Placement]]) referring to the clip-MO, which can (and should) be used to place the clip-MO within the EDL and to manipulate it consequently...
    [>img[Outline of the Build Process|uml/fig131333.png]]
    At first sight the link between asset and clip-MO is a simple logical relation between entities, but it is not strictly 1:1 because typical media are [[multichannel|MultichannelMedia]]. Even if the media is compound, there is //only one asset::Clip//, because in the logical view we have only one "clip-thing". On the other hand, in the Session/EDL, we have a compound clip ~MObject comprised of several elementary clip objects, each of which will refer to its own sub-media (channel) within the compound media (and don't forget, this structure can be tree-like)
    {{red{open question:}}} do the clip-MO's of the individual channels refer directly to asset::Media? does this mean the relation is different from the top level, where we have a relation to a asset::Clip??

    Conceptually, the Render Engine is the core of the application. But &mdash; surprisingly &mdash; we don't even have a distinct »~RenderEngine« component in our design. Rather, the engine is formed by the cooperation of several components spread over two layers (Backend and Proc-Layer): The [[Builder]] creates a network of [[render nodes|ProcNode]], which is used by the Backend to pull individual Frames.
    &rarr; OverviewRenderEngine

    The [[Render Engine|Rendering]] only carries out the low-level and performance critical tasks. All configuration and decision concerns are to be handled by [[Builder]] and [[Controller]]. While the actual connection of the Render Nodes can be highly complex, basically each Segment of the Timeline with uniform characteristics is handled by one Processor, which is a graph of [[Processing Nodes|ProcNode]] discharging into a ExitNode. The Render Engine Components as such are //stateless// themselves; for the actual calculations they are combined with a StateProxy object generated by and connected internally to the [[Controller]], while at the same time holding the Data Buffers (Frames) for the actual calculations.

    [img[Entities comprising the Render Engine|uml/fig128389.png]]

    Below are some notes regarding details of the actual implementation of the render process and processing node operation. In the description of the [[render node operation protocol|NodeOperationProtocol]] and the [[mechanics of the render process|RenderMechanics]], these details were left out deliberately.

    !Layered structure of State
    State can be seen as structured like an onion. All the [[StateAdapter]]s in one call stack are supposed to be within one layer: they all know of a "current state", which in turn is a StateProxy (and thus may refer yet to another state, maybe accros the network or in the backend or whatever). The actual {{{process()}}} function "within" the individual nodes just sees a single StateAdapter and thus can be thought to be a layer below.

    !Buffer identification
    For the purpose of node operation, Buffers are identified by a //Buffer-handle,// which contains both the actual buffer pointer and an internal indes and classification of the source providing the buffer; the latter information is used for deallocation. Especially for calling the {{{process()}}} function (which is supposed to be plain C) the respective StateAdapter provides an array containing just the output and input buffer pointers

    !Problem of multi-channel nodes
    Some data processors simply require to work on multiple channels simultanously, while others work just on a single channel and will be replicated by the builder for each channel invoved. Thus, we are struck with the nasty situation that the node graph may go through some nodes spanning the chain of several channels. Now the decision is //not to care for this complexity within a single chain calculating a single channel.// We rely solely on the cache to avoid duplicated calculations. When a given node happens to produce multiple output buffers, we are bound to allocate them for the purpose of this nodes {{{process()}}} call, but we just "let go" the buffers not needed immediately for the channel acutally to be processed. For this to work, it is supposed that the builder has wired in a caching, and that the cache will hit when we touch the same node again for the other channels.

    !Configuration of the processing nodes
    [>img[uml/fig132357.png]]
    Every node is actually decomposed into three parts
    * an interface container of a ProcNode subclass
    * an {{{const}}} WiringDescriptor, which is actually parametrized to a subtype encoding details of how to carry out the intended operation
    * the Invocation state created on the stack for each {{{pull()}}} call. It is comprised of references to an StateAdapter object  and the current overall process state, the WiringDescriptor, and finally a table of suitable buffer handles
    Thus, the outer container can be changed polymorphically to support the different kinds of nodes (large-scale view). The actual wiring of the nodes is contained in the WiringDescriptor, including the {{{process()}}} function pointer. Additionally, this WiringDescriptor knows the actual type of the operation Strategy, and this actual type has been chosen by the builder such as to select details of the desired operation of this node, for example caching / no caching or maybe ~OpenGL rendering or the special case of a node pulling directly from a source reader. Most of this configuration is done by selecting the right template specialisation within the builder; thus in the critical path most of the calls can be inlined

    !!!! composing the actual operation Strategy
    As shown in the class diagram to the right, the actual implementation is assembled by chaining together the various policy classes governing parts of the node operation, like Caching, in-Place calculation capability, etc. (&rarr; see [[here|WiringDescriptor]] for details). The rationale is that the variable part of the Invocation data is allocated at runtime directly on the stack, while a precisely tailored call sequence for "calculating the predecessor nodes" can be defined out of a bunch of simple building blocks. This helps avoiding "spaghetti code", which would be especially dangerous because of the large number of different execution paths to get right. Additionally, a nice side effect of this implementation technique is that a good deal of the implementation is eligible to inlining.
    We //do employ//&nbsp; some virtual calls for the buffer management in order to avoid coupling the policy classes to the actual number of in/out buffers. (As of 6/2008, this is mainly a precaution to be able to control the number of generated template instances. If we ever get in the region of several hundred individual specialisations, we'd need to separate out the allocation of the "buffer table" into a hand-made stack-like buffer allocated from the heap.)

    !Rules for buffer allocation and freeing
    * only output buffers are allocated. It is //never necessary//&nbsp; to allocate input buffers!
    * buffers are to be allocated as late as possible, typically just before invoking {{{process()}}}
    * buffers are allways allocated by calling to the preceeding StateAdapter in the callstack ("parent stae"), because of the possibility of writing the result to cache.
    * {{{pull()}}} returns a handle for the single output requested by this call. Using this ID, the caller may retrieve the actual buffer holding the result from the "current state" StaeProxy.
    * any other buffers filled with results in the course of the same {{{process()}}} call can be released immediately before returning from the {{{pull()}}}
    * similar, and input buffers are to be released immediately after the {{{process()}}} call, but before returing from this {{{pull()}}}
    * buffers are allways released by calling to the "current state" (which is a StateProxy), providing the buffer-ID to be released

    @@clear(right):display(block):@@

    While the render process, with respect to the dependencies, the builder and the processing function is sufficiently characterized by referring to the ''pull principle'' and by defining a [[protocol|NodeOperationProtocol]] each node has to adhere to &mdash; for actually get it coded we have to care for some important details, especially //how to manage the buffers.// It may well be that the length of the code path necessary to invoke the individual processing functions is finally not so important, compared with the time spent at the inner pixel loop within these functions. But my guess is (as of 5/08), that the overall number of data moving and copying operations //will be//&nbsp; of importance.

    !requirements
    * operations should be "in place" as much as possible
    * because caching necessitates a copy, the points where this happens should be controllable.
    * buffers should accommodate automatically to provide the necessary space without clipping the image.
    * the type of the media data can change while passing through the network, and so does the type of the buffers.
    On the other hand, the processing function within the individual node needs to be shielded from these complexities. It can expect to get just //N~~I~~// input buffers and //N~~O~~// output buffers of required type. And, moreover, as the decision how to organize the buffers certainly depends on non-local circumstances, it should be preconfigured while building.

    !data flow
    [>img[uml/fig131973.png]]
    Not everything can be preconfigured though. The pull principle opens the possibility for the node to decide on a per call base what predecessor(s) to pull (if any). This decision may rely on automation parameters, which thus need to be accessible prior to requesting the buffer(s). Additionally, in a later version we plan to have the node network calculate some control values for adjusting the cache and backend timings &mdash; and of course at some point we'll want to utilize the GPU, resulting in the need to feed data from our processing buffers into some texture representation.

    !buffer management
    Besides the StateProxy representing the actual render process and holding a couple of buffer (refs), we employ a lightweight adapter object in between. It is used //for a single {{{pull()}}}-call// &mdash; mapping the actual buffers to the input and output port numbers of the processing node and for dealing with the cache calls. While the StateProxy manages a pool of frame buffers, this interspersed adapter allows us to either use a buffer retrieved from the cache as an input, possibly use a new buffer located within the cache as output, or (in case no caching happens) to just use the same buffer as input and output for "in-place"-processing. The idea is that most of the configuration of this adapter object is prepared in the wiring step while building the node network.

    The usage patern of the buffers can be stack-like when processing nodes require multiple input buffers. In the standard case, which also is the simplest case, a pair of buffers (or a single buffer for "in-place" capable nodes) suffices to calculate a whole chain of nodes. But &mdash; as the recursive descent means depth-first processing &mdash; in case multiple input buffers are needed, we may encounter a situation where some of these input buffers already contain processed data, while we have to descend into yet another predecessor node chain to pull the data for the remaining buffers. Care has to be taken //to allocate the buffers as late as possible,// otherwise we could end up holding onto a buffer almost for each node in the network. Effectively this translates into the rule to allocate output buffers only after all input buffers are ready and filled with data; thus we shouldn't allocate buffers when //entering// the recursive call to the predecessor(s), rather we have to wait until we are about to return from the downcall chain.
    Besides, these considerations also show we need a means of passing on the current buffer usage pattern while calling down. This usage pattern not only includes a record of what buffers are occupied, but also the intended use of these occupied buffers, especially if they can be modified in-place, and at which point they may be released and reused.
    __note__: this process outlined here and below is still an simplification. The actual implementation has some additional [[details to care for|RenderImplDetails]]

    !!Example: calculating a 3 node chain
    # Caller invokes calculation by pulling from exit node, providing the top-level StateProxy
    # node1 (exit node) builds StateAdapter and calls retrieve() on it to get the desired output result
    # this StateAdapter (ad1) knows he could get the result from Cache, so he tries, but it's a miss
    # thus he pulls from the predecessor node2 according to the [[input descriptor|ProcNodeInputDescriptor]] of node1
    # node2 builds its StateAdapter and calls retrieve()
    # but because StateAdapter (ad2) is configured to directly forward the call down (no caching), it pulls from node3
    # node3 builds its StateAdapter and calls retrieve()
    # this StateAdapter (ad3) is configured to look into the Cache...
    # this time producing a Cache hit
    # now StateAdapter ad2 has input data, but needs a output buffer location, which re requests from its //parent state// (ad1)
    # and, because ad1 is configured for Caching and is "in-place" capable, it's clear that this output buffer will be located within the cache
    # thus the allocation request is forwarded to the cache, which provides a new "slot"
    # now node2 has both a valid input and a usable output buffer, thus the process function can be invoked
    # and after the result has been rendered into the output buffer, the input is no longer needed
    # and can be "unlocked" in the Cache
    # now the input data for node1 is available, and as node1 is in-place-capable, no further buffer allocation is necessary prior to calculating
    # the finished result is now in the buffer (which happens to be also the input buffer and is actually located within the Cache)
    # thus it can be marked as ready for the Cache, which may now provide it to other processes (but isn't allowed to overwrite it)
    # finally, when the caller is done with the data, it signalles this to the top-level State object
    # which forwards this information to the cache, which in turn may now do with the released Buffer as he sees fit.
    [img[uml/fig132229.png]]

    @@clear(right):display(block):@@
    __see also__
    &rarr; the [[Entities involved in Rendering|RenderEntities]]
    &rarr; additional [[implementation details|RenderImplDetails]]
    &rarr; [[Memory management for render nodes|ManagementRenderNodes]]
    &rarr; the protocol [[how to operate the nodes|NodeOperationProtocol]]

    For each segment (of the effective timeline), there is a Processor holding the exit node(s) of a processing network, which is a "Directed Acyclic Graph" of small, preconfigured, stateless [[processing nodes|ProcNode]]. This network is operated according to the ''pull principle'', meaning that the rendering is just initiated by "pulling" output from the exit node, causing a cascade of recursive downcalls. Each node knows its predecessor(s) an can pull the necessary input from there. Consequently, there is no centralized "engine object" which may invoke nodes iteratively or table driven &mdash; rather, the rendering can be seen as a passive service provided for the backend, which may pull from the exit nodes at any time, in any order (?), and possibly multithreaded.
    All State necessary for a given calculation process is encapsulated and accessible by a StateProxy object, which can be seen as the representation of "the process". At the same time, this proxy provides the buffers holding data to be processed and acts as a gateway to the backend to handle the communication with the Cache. In addition to this //top-level State,// each calculation step includes a small [[state adapter object|StateAdapter]] (stack allocated), which is pre-configured by the builder and serves the purpose to isolate the processing function from the detals of buffer management.

    __see also__
    &rarr; the [[Entities involved in Rendering|RenderEntities]]
    &rarr; the [[mechanics of rendering and buffer management|RenderMechanics]]
    &rarr; the protocol [[how to operate the nodes|NodeOperationProtocol]]

    The rendering of input sources to the desired output ports happens within the &raquo;''Render Engine''&laquo;, which can be seen as a collaboration of Proc-Layer, Backend together with external/library code for the actual data manipulation. In preparation of the RenderProcess, the [[Builder]] as wired up a network of [[processing nodes|ProcNode]] called the ''low-level model'' (in contrast to the high-level model of objects placed within the EDL/Session). Generally, this network is a "Directed Acyclic Graph" starting at the //exit nodes// (output ports) and pointing down to the //source readers.// In Lumiera, rendering is organized according to the ''pull principle'': when a specific frame of rendered data is requested from an exit node, a recursive calldown happens, as each node asks his predecessor(s) for the necessary input frame(s). This may include pulling frames from various input sources and for several time points, thus pull rendering is more powerful (but also more difficult to understand) than push rendering, where the process would start out with a given source frame.

    Rendering can be seen as a passive service available to the Backend, which remains in charge what to render and when. Render processes may be running in parallel without any limitations. All of the storage and data management falls into the realm of the Backend. The render nodes themselves are ''completely stateless'' &mdash; if some state is necessary for carrying out the calculations, the backend will provide a //state frame// in addition to the data frames.

    A facility allowing the Proc-Layer to work with abstracted [[media stream types|StreamType]], linking (abstract or opaque) [[type tags|StreamTypeDescriptor]] to an [[library|MediaImplLib]], which provides functionality for acutally dealing with data of this media stream type. Thus, the stream type manager is a kind of registry of all the external libraries which can be bridged and accessed by Lumiera (for working with media data, that is). The most basic set of libraries is instelled here automatically at application start, most notably the [[GAVL]] library for working with uncompressed video and audio data. //Later on, when plugins will introduce further external libraries, these need to be registered here too.//

    The Session contains all informations, state and objects to be edited by the User. From a users view, the Session is synonymous to the //current Project//. It can be saved and loaded. The individual Objects within the Session, i.e. Clips, Media, Effects, are contained in one (or several) collections within the Session, which we call [[EDL (Edit Decision List)|EDL]]. &rarr; [[Session design overview|SessionOverview]]

    !Session structure
    The Session object is a singleton &mdash; actually it is a »~PImpl«-Facade object (because the actual implementation object can be swapped for (re)loading Sessions).<br/>The Session is comprised of
    * a collection of ''EDL'' objects
    * a ''current EDL'', which can be switched and accessed {{red{TODO not sure if I keep this design...}}}
    * a collection of ''global Pipes''
    * the ''Fixture'' with a possibility to [[(re)build it|PlanningBuildFixture]]

    <<<
    {{red{WARNING: Naming is currently being discussed (11/08)}}}
    * [[EDL]] probably will be called ''Sequence'' (or maybe ''Arrangement'')
    * [[Session]] maybe renamed to ''Project''
    * there seems to be a new entity called [[Timeline]] which holds the global Pipes
    <<<
    The [[Session]] (sometimes also called //Project// ) contains all informations and objects to be edited by the User. It can be saved and loaded. The individual Objects within the Session, i.e. Clips, Media, Effects, are contained in one (or several) collections within the Session, which we call [[EDL (Edit Decision List)|EDL]]. Moreover, the sesion contains references to all the Media files used, and it contains various default or user defined configuration, all being represented as [[Asset]]. At any given time, there is //only one current session// opened within the application.
    The Session is close to what is visible in the GUI. From a user's perspective, you'll find a [[Timeline]]-like structure, containing an [[EDL (or Sequence)|EDL]], where various Media Objects are arranged and placed. The available building blocks and the rules how they can be combined together form Lumiera's [[high-level data model|HighLevelModel]]. Basically, besides the [[media objects|MObjects]] there are data connections and all processing is organized around processing chains or [[pipes|Pipe]], which can be either global (in the Session) or local (in real or virtual clips)

    !!!larger projects
    For larger editing projects the simple structure of a session containing "the" timeline is not sufficient. Rather
    * we may have several [[EDLs (=Sequences)|EDL]], e.g. one for each scene. These sequences can be even layered or nested (compositional work).
    * within one project, there may be multiple, //independant Timelines// &mdash; each of which may have an associated Viewer or Monitor
    To support these cases without making the default case more complicated, Lumiera introduces a //focus// for selecting the //current EDL,// which will receive all editing operations. [[Timelines|Timeline]] on the other hand are always top-level objects and can't be combined further. You can render a given timeline to output.
    &rarr; see [[Relation of Project, Timelines and Sequences|TimelineSequences]]

    !!!the definitive state
    With all the structural complexities possible within such a session, we need an isolation layer to provide __one__ definitive state where all configuration has been made explicit. Thus the session manages one special object list, the [[Fixture]], which can be seen as all currently active objects placed onto a single timeline.

    !!!organisational devices
    The possibility of having multiple ~EDLs helps organizing larger projects. Each [[EDL]] is just a logical grouping; because all effective properties of any MObject within this EDL are defined by the ~MObject itself and the [[Placement]], by which the object is anchored to some time point, some track, can be connected to some pipe, or linked to another object. In a similar manner, [[Tracks|Track]] are just another organisational aid for grouping objects, disabling them and defining common output pipes.

    !!!global pipes
    [>img[draw/Proc.builder1.png]] Any session should contain a number of global [[(destination) pipes|Pipe]], typically video out and audio out. The goal is, to get any content producing or transforming object in some way connected to one of these outputs, either //by [[placing|Placement]] it directly// to some pipe, or by //placing it to a track// and having the track refer to some pipe. Besides the global destination pipes, we can use internal pipes to form busses or subgroups, either on a global (session) level, or by using the processing pipe within a [[virtual clip|VirtualClip]], which can be placed freely within the EDL(s). Normally, pipes just gather and mix data, but of course any pipe can have an attached effect chain. (&rarr; see [[more on Tracks and Pipes within the EDL|TrackPipeEDL]])

    !!!default configuration
    While all these possibilities may seem daunting, there is a simple default configuration loaded into any pristine new session:
    It will contain a global video and audio out pipe, just one EDL with a single track; this track will have a internal video and audio pipe (bus) configured with one fading device sending to the global output ports. So, by adding some clip with a simple absolute placement to this track and some time position, the clip gets connected and rendered, after [[(re)building|PlanningBuildFixture]] the [[Fixture]] and passing the result to the [[Builder]] &mdash; and using the resulting render nodes network (Render Engine).

    &rarr; [[anatomy of the high-level model|HighLevelModel]]
    &rarr; considerations regarding [[Tracks and Pipes within the EDL|TrackPipeEDL]]
    &rarr; see [[Relation of Project, Timelines and Sequences|TimelineSequences]]

    <<search>><<closeAll>><<permaview>><<newTiddler>><<saveChanges>><<slider chkSliderOptionsPanel OptionsPanel "options »" "Change TiddlyWiki advanced options">>

    Building a Render Nodes Network from Objects in the EDL

    Engine

    {{red{killme}}}

    /***

    ''Inspired by [[TiddlyPom|http://www.warwick.ac.uk/~tuspam/tiddlypom.html]]''

    |Name|SplashScreenPlugin|
    |Created by|SaqImtiaz|
    |Location|http://tw.lewcid.org/#SplashScreenPlugin|
    |Version|0.21 |
    |Requires|~TW2.08+|
    !Description:
    Provides a simple splash screen that is visible while the TW is loading.

    !Installation
    Copy the source text of this tiddler to your TW in a new tiddler, tag it with systemConfig and save and reload. The SplashScreen will now be installed and will be visible the next time you reload your TW.

    !Customizing
    Once the SplashScreen has been installed and you have reloaded your TW, the splash screen html will be present in the MarkupPreHead tiddler. You can edit it and customize to your needs.

    !History
    * 20-07-06 : version 0.21, modified to hide contentWrapper while SplashScreen is displayed.
    * 26-06-06 : version 0.2, first release

    !Code
    ***/
    //{{{
    var old_lewcid_splash_restart=restart;

    restart = function()
    {   if (document.getElementById("SplashScreen"))
            document.getElementById("SplashScreen").style.display = "none";
          if (document.getElementById("contentWrapper"))
            document.getElementById("contentWrapper").style.display = "block";

        old_lewcid_splash_restart();

        if (splashScreenInstall)
           {if(config.options.chkAutoSave)
    			{saveChanges();}
            displayMessage("TW SplashScreen has been installed, please save and refresh your TW.");
            }
    }

    var oldText = store.getTiddlerText("MarkupPreHead");
    if (oldText.indexOf("SplashScreen")==-1)
       {var siteTitle = store.getTiddlerText("SiteTitle");
       var splasher='\n\n<style type="text/css">#contentWrapper {display:none;}</style><div id="SplashScreen" style="border: 3px solid #ccc; display: block; text-align: center; width: 320px; margin: 100px auto; padding: 50px; color:#000; font-size: 28px; font-family:Tahoma; background-color:#eee;"><b>'+siteTitle +'</b> is loading<blink> ...</blink><br><br><span style="font-size: 14px; color:red;">Requires Javascript.</span></div>';
       if (! store.tiddlerExists("MarkupPreHead"))
           {var myTiddler = store.createTiddler("MarkupPreHead");}
       else
          {var myTiddler = store.getTiddler("MarkupPreHead");}
          myTiddler.set(myTiddler.title,oldText+splasher,config.options.txtUserName,null,null);
          store.setDirty(true);
          var splashScreenInstall = true;
    }
    //}}}

    A small (in terms of storage) and specifically configured StateProxy object which is created on the stack for each individual {{{pull()}}} call. It is part of the invocation state of such a call and participates in the buffer management. Thus, in a calldown sequence of {{{pull()}}} calls we get a corresponding sequence of "parent" states. At each level, the &rarr; WiringDescriptor of the respective node defines a Strategy how the call is passed on.

    An Object representing a //Render Process// and containing associated state information.
    * it is created in the Controller subsystem while initiating the BuildProcess
    * it is passed on to the generated Render Engine, which in turn passes it down to the individual Processors
    * moreover, it contains methods to communicate with other state relevant parts of the system, thereby shielding the rendering code from any complexities of Thread communication if necessary. (thus the name Proxy)
    * in a future version, it may also encapsulate the communication in a distributed render farm

    Conversion of a media stream into a stream of another type is done by a processor module (plugin). The problem of finding such a module is closely related to the StreamType and especially [[problems of querying|StreamTypeQuery]] for such. (The builder uses a special Facade, the ConManager, to access this functionality). There can be different kinds of conversions, and the existance or non-existance of such an conversion can influence the stream type classification.
    * different //kinds of media// can be ''transformed'' into each other
    * stream types //subsumed// by a given prototype should be ''lossless convertible'' and thus can be considered //equivalent.//
    * besides, between different stream //implementation types,// there can be a ''rendering'' (lossy conversion) &mdash; or no conversion at all.

    The stream Prototype is part of the specification of a media stream's type. It is a semantic (or problem domain oriented) concept and should be distinguished from the actual implementation type of the media stream. The latter is provided by an [[library implementation|StreamTypeImplFacade]]. While there are some common predefined prototypes, mostly, they are defined within the concrete [[Session]] according to the user's needs.

    Prototypes form an open (extensible) collection, though each prototype belongs to a specific media kind ({{{VIDEO, IMAGE, AUDIO, MIDI,...}}}).
    The ''distinguishing property'' of a stream prototype is that any [[Pipe]] can process //streams of a specific prototype only.// Thus, two streams with different prototype can be considered "something quite different" from the users point of view, while two streams belonging to the same prototype can be considered equivalent (and will be converted automatically when their implementation types differ). Note this definition is //deliberately fuzzy,// because it depends on the actual situation of the project in question.

    Consequently, as we can't get away with an fixed Enum of all stream prototypes, the implementation must rely on a query interface. The intention is to provide a basic set of rules for deciding queries about the most common stream prototypes; besides, a specific session may inject additional rules or utilize a completely different knowledge base. Thus, for a given StreamTypeDescriptor specifying a prototype
    * we can get a [[default|DefaultsManagement]] implementation type
    * we can get a default prototype to a given implementation type by a similar query
    * we can query if a implementation type in question can be //subsumed// by this prototype
    * we can determine if another prototype is //convertible//

    !!Examples
    NTSC and PAL video, video versus digitized film, HD video versus SD video, 3D versus flat video, cinemascope versus 4:3, stereophonic versus monaural, periphonic versus panoramic sound, Ambisonics versus 5.1, dolby versus linear PCM...

    //how to classify and describe media streams//
    Media data is understood to appear structured as stream(s) over time. While there may be an inherent internal structuring, at a given perspective ''any stream is a unit and homogeneous''. In the context of digital media data processing, streams are always ''quantized'', which means they appear as a temporal sequence of data chunks called ''frames''.

    ! Terminology
    * __Media__ is comprised of a set of streams or channels
    * __Stream__ denotes a homogeneous flow of media data of a single kind
    * __Channel__ denotes a elementary stream, which can't be further separated in the given context
    * all of these are delivered and processed in a smallest unit called __Frame__. Each frame corresponds to a //time interval.//
    * a __Buffer__ is a data structure capable of holding a Frame of media data.
    * the __~Stream-Type__ describes the kind of media data contained in the stream

    ! Problem of Stream Type Description
    Media types vary largely and exhibit a large number of different properties, which can't be subsumed under a single classification scheme. But on the other hand, we want to deal with media objects in a uniform and generic manner, because generally all kinds of media behave somewhat similar. But the problem is, these similarities disappear when describing media with logical precision. Thus we are forced into specialized handling and operations for each kind of media, while we want to implement a generic handling concept.

    ! Stream Type handling in the Proc-Layer
    !! Identification
    A stream type is denoted by a StreamTypeID, which is an identifier, acting as an unique key for accessing information related to the stream type. It corresponds to an StreamTypeDescriptor record, containing an &mdash; //not necessarily complete// &mdash; specification of the stream type, according to the classification detailed below.

    !! Classification
    Within the Proc-Layer, media streams are treated largely in a similar manner. But, looking closer, note everything can be connected together, while on the other hand there may be some classes of media streams which can be considered //equivalent// in most respects. Thus, it seems reasonable to separate the distinction between various media streams into several levels
    * Each media belongs to a fundamental ''kind'' of media, examples being __Video__, __Image__, __Audio__, __MIDI__,... Media streams of different kind can be considered somewhat "completely separate" &mdash; just the handling of each of those media kinds follows a common //generic pattern// augmented with specialisations. Basically, it is //impossible to connect// media streams of different kind. Under some circumstances there may be the possibility of a //transformation// though. For example, a still image can be incorporated into video, sound may be visualized, MIDI may control a sound synthesizer.
    * Below the level of distinct kinds of media streams, within every kind we have an open ended collection of ''prototypes'', which, when compared directly, may each be quite distinct and different, but which may be //rendered//&nbsp; into each other. For example, we have stereoscopic (3D) video and we have the common flat video lacking depth information, we have several spatial audio systems (Ambisonics, Wave Field Synthesis), we have panorama simulating sound systems (5.1, 7.1,...), we have common stereophonic and monaural audio. It is considered important to retain some openness and configurability within this level of distinction, which means this classification should better be done by rules then by setting up a fixed property table. For example, it may be desirable for some production to distinguish between digitized film and video NTSC and PAL, while in another production everything is just "video" and can be converted automatically. The most noticeable consequence of such a distinction is that any Bus or [[Pipe]] is always limited to a media stream of a single prototype. (&rarr; [[more|StreamPrototype]])
    * Besides the distinction by prototypes, there are the various media ''implementation types''. This classification is not necessarily hierarchically related to the prototype classification, while in practice commonly there will be some sort of dependency. For example, both stereophonic and monaural audio may be implemented as 96kHz 24bit PCM with just a different number of channel streams, as well we may have a dedicated stereo audio stream with two channels multiplexed into a single stream. For dealing with media streams of various implementation type, we need //library// routines, which also yield a //type classification system.// Most notably, for raw sound and video data we use the [[GAVL]] library, which defines a classification system for buffers and streams.
    * Besides the type classification detailed thus far, we introduce an ''intention tag''. This is a synthetic classification owned by Lumiera and used for internal wiring decisions. Currently (8/08), we recognize the following intention tags: __Source__, __Raw__, __Intermediary__ and __Target__. Only media streams tagged as __Raw__ can be processed.

    !! Media handling requirements involving stream type classification
    * set up a buffer and be able to create/retrieve frames of media data.
    * determine if a given media data source and sink can be connected, and how.
    * determine and enumerate the internal structure of a stream.
    * discover processing facilities
    &rarr; see StreamTypeUse
    &rarr; [[querying types|StreamTypeQuery]]

    A description and classification record usable to find out about the properties of a media stream. The stream type descriptor can be accessed using an unique StreamTypeID. The information contained in this descriptor record can intentionally be //incomplete,// in which case the descriptor captures a class of matching media stream types. The following information is maintained:
    * fundamental ''kind'' of media: {{{VIDEO, IMAGE, AUDIO, MIDI,...}}}
    * stream ''prototype'': this is the abstract high level media type, like NTSC, PAL, Film, 3D, Ambisonics, 5.1, monaural,...
    * stream ''implementation type'' accessible by virtue of an StreamTypeImplFacade
    * the ''intended usage category'' of this stream: {{{SOURCE, RAW, INTERMEDIARY, TARGET}}}.
    &rarr; see &raquo;[[Stream Type|StreamType]]&laquo; detailed specification
    &rarr; notes about [[using stream types|StreamTypeUse]]
    &rarr; more [[about prototypes|StreamPrototype]]

    This ID is an symbolic key linked to a StreamTypeDescriptor. The predicate {{{stream(ID)}}} specifies a media stream with the StreamType as detailed by the corresponding descriptor (which may contain complete or partial data defining the type).

    A special kind of media stream [[implementation type|StreamTypeImplFacade]], which is not fully specified. As such, it is supposed there //actually is// an concrete implementation type, while only caring for some part or detail of this implementation to exhibit a specific property. For example, using an type constraint we can express the requirement of the actual implementation of a video stream to be based on ~RGB-float, or to enforce a fixed frame size in pixels.

    An implementation constraint can //stand-in// for a completely specified implementation type (meaning it's a sub interface of the latter). But actually using it in this way may cause a call to the [[defaults manager|DefaultsImplementation]] to fill in any missing information. An example would be to call {{{createFrame()}}} on the type constraint object, which means being able to allocate memory to hold a data frame, with properties in compliance with the given type constraint. Of cousre, then we need to know all the properties of this stream type, which is where the defaults manager is queried. This allows session customisation to kick in, but may fail under certain cicumstances.

    Common interface for dealing with the implementation of media stream data. From a high level perspective, the various kinds of media ({{{VIDEO, IMAGE, AUDIO, MIDI,...}}}) exhibit similar behaviour, while on the implementation level not even the common classification can be settled down to a complete general and useful scheme. Thus, we need separate library implementations for deailing with the various sorts of media data, all providing at least a set of basic operations:
    * set up a buffer
    * create or accept a frame
    * get an tag describing the precise implementation type
    * ...?

    &rarr; see also &raquo;[[Stream Type|StreamType]]&laquo;
    //Note:// there is a sort-of "degraded" variant just requiring some &rarr; [[implementation constraint|StreamTypeImplConstraint]] to hold

    Querying for media stream type information comes in various flavours
    * you may want to find a structural object (pipe, output, processing patten) associated with / able to deal with a certain stream type
    * you may need a StreamTypeDescriptor for an existing stream given as implementation data
    * you may want to build or complete type information from partial specification.
    Mostly, those queries involve the ConfigRules system in some way or the other. The [[prototype-|StreamPrototype]] and [[implementation type|StreamTypeImplFacade]]-interfaces themselves are mostly a facade for issuing appropriate queries. Some objects (especially [[pipes|Pipe]]) are tied to a certain stream type and thus store a direct link to type information. Others are just associated with a type by virtue of the DefaultsManagement.

    The //problem// with this pivotal role of the config rules is that &mdash; from a design perspective &mdash; not much can be said specifically, besides //"you may be able to find out...", "...depends on the defaults and the session configuration".// This way, a good deal of crucial behaviour is pushed out of the core implementation (and it's quite intentionally being done this way). What can be done regarding the design of the core is mostly to setup a framework for the rules and determine possible ''query situations''.

    !the kind of media
    the information of the fundamental media kind (video, audio, text, MIDI,...) is assiciated with the prototype, for technical reasons. Prototype information is mandatory for each StreamType, and the impl facade provides a query function (because some implementation libraries, e.g. [[GAVL]], support multiple kinds of media).

    !query for a prototype
    __Situation__: given an implementation type, find a prototype to subsume it.
    Required only for building a complete ~StreamType which isn't known at this point.
    The general case of this query is //quite hairy,// because the solution is not necessary clear and unique. And, worse still, it is related to the semantics, requiring semantic information and tagging to be maintained somewhere. For example, while the computer can't "know" what stereopohinc audio is (only a human can by listening to a stereophoic playback and deciding if it actually does convey a spatical sound image), in most cases we can overcome this problem by using the //heuristical rule// of assuming the prototype "stereophonic" when given two identically typed audio channels. This example also shows the necessity of ordering heuristic rules to be able to pick a best fit.

    We can inject two different kinds of fallback solutions for this kind of query:
    * we can always build a "catch-all" prototype just based on the kind of media (e.g. {{{prototype(video).}}}). This should match with lowest priority
    * we can search for existing ~StreamTypes with the same impl type, or an impl type which is //equivalent convertible// (see &rarr; StreamConversion).
    The latter case can yield multiple solutions, which isn't any problem, because the match is limited to classes of equivalent stream implementation, which would be subsumed under the same prototype anyway. Even if the registry holds different prototypes linked to the same implementation type, they would be convertible and thus could //stand-in// for one another. Together this results in the implementation
    # try to get a direct match to an existing impl type which has an associated (complete) ~StreamType, thus bypassing the ConfigRules system altogether
    # run a {{{Query<Prototype>}}} for the given implementation type
    # do the search within equivalence class as described above
    # fall back to the media kind.
    {{red{TODO: how to deal with the problem of hijacking a prototype?}}} &rarr; see [[here|StreamTypeUse]]

    !query for an implementation
    __Situation 1__: given an partially specified ~StreamType (just an [[constraint|StreamTypeImplConstraint]])
    __Situation 2__: find an implementation for a given prototype (without any further impl type guidlines)
    Both cases have to go though the [[defaults manager|DefaultsManagement]] in some way, in order to give any default configuration a chance to kick in. This is //one of the most important use cases// of the defaults system: the ability to configure a default fromat for all streams with certain semantic classification. {{{prototype(video)}}} by default is RGBA 24bit non-interlaced for example.
    But after having queried the defaults system, there remains the problem to build a new solution (which will then automatically become default for this case). To be more precise: invoking the defaults system (as implemented in Lumiera) means first searching through existing objects encountered as default, and then issuing an general query with the capabilities in question. This general query in turn is conducted by the query type handler and usually consists of first searching existing objects and then creating a new object to match the capabilities. But, as said, the details depend on the type (and are defined by the query handler installed for this type). Translated to our problem here in question, this means //we have to define the basic operations from which a type query handler can be built.// Thus, to start with, it's completely sufficient to wire a call to the DefaultsManagement and assume the current session configuration contains some rules to cover it. Plus being prepared for the query to fail (throw, that is).

    Later on this could be augmented by providing some search mechanisms:
    * search through existing stream type implementations (or a suitable pre filtered selection) and narrow down the possible result(s) by using the constraint as a filter. Obviously this requires support by the MediaImplLib facade for the implementation in question. (This covers __Situation 1__)
    * relate a protoype in question to the other existing prototypes and use the convertibility / subsumption as a filter mechanism. Finally pick an existing impl type which is linked to one of the prototypes found thus far.
    Essentially, we need a search mechanism for impltypes and prototypes. This search mechanism is best defined by rules itself, but needs some primitive operations on types, like ennumerating all registered types, filter those selections and match against a constraint.

    !query for an (complete) StreamType
    All situations discussed thus far can also occur wrapped into and triggered by a query for a complete type. Depending on what part is known, the missing bits will be queried.
    Independent from these is __another Situation__ where we query for a type ''by ID''.
    * a simple symbolic ID can be found by searching through all existing stream types (Operation supported by the type registry within STypeManager)
    * a special ''classificating'' ID can be parsed into the components (media kind, prototype, impltype), resulting in sub searches for these.
    {{red{not sure if we want to support queries by symboic ID}}}...problem is the impl type, because probably the library needs to support describing any implementation type by a string. Seemingly GAVL does, but requiring it for every lib?

    Questions regarding the use of StreamType within the Proc-Layer.
    * what is the relation between Buffer and Frame?
    * how to get the required size of a Buffer?
    * who does buffer allocations and how?

    Mostly, stream types are used for querying, either to decide if they can be connected, or to find usable processing modules.
    Even building a stream type from partial information involves some sort of query.
    &rarr; more on [[media stream type queries|StreamTypeQuery]]

    !creating stream types
    seemingly stream types are created based on an already existing media stream (or a Frame of media data?). {{red{really?}}}
    The other use case seems to be that of an //incomplete// stream type based on a [[Prototype|StreamPrototype]]

    !Prototype
    According to my current understanding, a prototype is merely a classification entity. But then &mdash; how to bootstrap a Prototype?
    And how to do the classification of an existing implementation type.

    Besides, there is the problem of //hijacking a prototype:// when a specific implementation type gets tied to a rather generic protoype, like {{{protoype(video)}}}, how to comply to the rule of prototypes subsuming a class of equivalent implementations?

    !Defaults and partial specification
    A StreamType need not be completely defined. It is sufficient to specify the media kind and the Prototype. The implementation type may be just given as a constraint, thus defining some properties and leaving out others. When creating a frame buffer based upon such an //incomplete type,// [[defaults|DefaultsManagement]] are queried to fill in the missing parts.
    Constraints are objects provided by the Lumiera core, but specialized to the internals of the actual implementation library.
    For example there might be a constraint implementation to force a specific {{{gavl_pixelformat_t}}}.

    !the ID problem
    Basically I'd prefer the ~IDs to be real identifiers. So they can be used directly within rules. At least the Prototypes //can// have such a textual identifier. But the implementation type is problematic, and consequently the ID of the StreamType as well. Because the actual implementation should not be nailed down to a fixed set of possibilities. And, generally, we can't expect an implementation library to yield textual identifiers for each implementation type. //Is this really a problem? {{red{what are the use cases?}}}//
    As far as I can see, in most cases this is no problem, as the type can be retrieved or derived from an existing media object. Thus, the only problematic case is when we need to persist the type information without being able to guarantee the persistence of the media object this type was derived from. For example this might be a problem when working with proxy media. But at least we should be able to create a constraint (partial type specification) to cover the important part of the type information, i.e. the part which is needed to re-create the model even when the original media isn't there any longer.
    Thus, //constraints may be viewed as type constructing functors.//

    --------------
    !use cases
    * pulling data from a media file
    * connecting pipes and similar wiring problems
    * describing the properties of an processor plugin

    !! pulling data from a media file
    To open the file, we need //type discovery code,// resulting in a handle to some library module for accessing the contents, which is in compliance with the Lumiera application. Thus, we can determine the possible return values of this type discovery code and provide code which wires up a corresponding StreamTypeImplFacade. Further, the {{{control::STypeManager}}} has the ability to build a complete or partial StreamType from
    * an ~ImplFacade
    * a Prototype
    * maybe even from some generic textual ~IDs?
    Together this allows to associate a StreamType to each media source, and thus to derive the Prototype governing the immediately connected [[Pipe]]
    A pipe can by design handle data of one Prototype solely.

    !! wiring problems
    When deciding if a connection can be made, we can build up the type information starting out from the source. (this requires some work, but it's //possible,// generally speaking.). Thus, we can allways get an ~ImplType for the "lower end" of the connection, and at least a Prototype for the "output side" &mdash; which should be enough to use the query functions provided by the stream type interfaces

    !! describing properties
    {{red{currently difficult to define}}} as of 9/2008, because the property description of plugins is not planned yet.
    My Idea was to use [[type implementation constraints|StreamTypeImplConstraint]] for this, which are a special kind of ~ImplType

    This design lays great emphasis on separating all those components and subsystems, which are considered not to have a //natural link// of their underlying concepts. This often means putting some additional constraints on the implementation, so basically we need to rely on the actual implementation to live up to this goal. In many cases it may seem to be more natural to "just access the necessary information". But on the long run this coupling of not-directly related components makes the whole codebase monolithic and introduces lots of //accidental complexity.//

    Instead, we should try to just connect the various subsystems via Interfaces and &mdash; instead of just using some information, rather use some service to be located on an Interface to query other components for this information. The best approach of course is always to avoid the dependency altogether.

    !Examples
    * There is a separation between the __high level [[EDL]] view__ and the [[Fixture]]: the latter only accesses the MObjects and the Placement Interfaces.
    * same holds true for the Builder: it just uses the same Interfaces. The actual coupling is done rather //by type//, i.e. the Builder relies on several types of MObjects to exist and treats them via overloaded methods. He doesn't rely on a actual object structure layout in the EDL besides the requirement of having a [[Playlist]]
    * the Builder itself is a separation layer. Neither do the Objects in the EDL access directly [[Render Nodes|ProcNode]], nor do the latter call back into the EDL. Both connections seem to be necessary at first sight, but both can be avoided by using the Builder Pattern
    * another separation exists between the Render Engine and the individual Nodes: The Render Engine doesn't need to know the details of the data types processed by the Nodes. It relies on the Builder having done the correct connections and just pulls out the calculated results. If there needs to be additional control information to be passed, then I would prefer to do a direct wiring of separate control connections to specialized components, which in turn could instruct the controller to change the rendering process.
    * to shield the rendering code of all complexities of thread communication and synchronization, we use the StateProxy

    Structural Assets are intended mainly for internal use, but the user should be able to see and query them. They are not "loaded" or "created" direcly, rather they //leap into existance // by creating or extending some other structures in the EDL/Session, hence the name. Some of the structural Asset parametrisation can be modified to control of some aspects of the Proc Layer's (default) behaviour.
    * [[Processing Patterns|ProcPatt]] encode information how to set up some parts of the render network to be created automatically: for example, when building a clip, we use the processing pattern how to decode and preprocess the actual media data.
    * [[Tracks|Track]] are one of the dimensions used for organizing the EDL. They serve as an Anchor to attach parametrisation of output pipe, overlay mode etc. By [[placing|Placement]] to a track, some media object inherits placement properties from this track.
    * [[Pipes|Pipe]] form &mdash; at least as visible to the user &mdash; the basic building block of the render network, because the latter appears to be a collection of interconnected processing pipelines. (this is the //outward view; // in fact the render network consists of [[nodes|ProcNode]] and is [[built|Builder]] from the Pipes, clips, effects...)
    [>img[Asset Classess|uml/fig131205.png]]
    !naming scheme
    The Asset name field of structural Assets utilizes a special naming scheme, which allows to derive the name based on the capabilities of the structural asset. For example, by default all media clips with a given media stream type (e.g. H264) will use the same [[processing Pattern|ProcPatt]] for rendering. {{red{todo: work out the details of this naming scheme??}}}

    !querying
    Structural assets can be queried by specifying the specific type (Pipe, Track, ProcPatt) and a query goal, which means in the current implementation just querying for some predicate defined with the structural asset. For example, you can {{{Query<Pipe> ("stream(mpeg)")}}}, yieliding the first pipe found which declares to have stream type "mpeg"

    /*{{{*/
    /* a contrasting background so I can see where one tiddler ends and the other begins */
    body {
    	background: [[ColorPalette::TertiaryLight]];
    }

    /* sexy colours and font for the header */
    .headerForeground {
    	color: [[ColorPalette::PrimaryPale]];
    }
    .headerShadow, .headerShadow a {
    	color: [[ColorPalette::PrimaryMid]];
    }
    .headerForeground, .headerShadow {
    	padding: 1em 1em 0;
    	font-family: 'Trebuchet MS' sans-serif;
    	font-weight:bold;
    }
    .headerForeground .siteSubtitle {
    	color: [[ColorPalette::PrimaryLight]];
    }
    .headerShadow .siteSubtitle {
    	color: [[ColorPalette::PrimaryMid]];
    }

    /* make shadow go and down right instead of up and left */
    .headerShadow {
    	left: 2px;
    	top: 3px;
    }

    /* prefer monospace for editing */
    .editor textarea {
    	font-family: 'Consolas' monospace;
    }

    /* sexy tiddler titles */
    .title {
    	font-size: 250%;
    	color: [[ColorPalette::PrimaryLight]];
    	font-family: 'Trebuchet MS' sans-serif;
    }

    /* more subtle tiddler subtitle */
    .subtitle {
    	padding:0px;
    	margin:0px;
    	padding-left:0.5em;
    	font-size: 90%;
    	color: [[ColorPalette::TertiaryMid]];
    }
    .subtitle .tiddlyLink {
    	color: [[ColorPalette::TertiaryMid]];
    }

    /* a little bit of extra whitespace */
    .viewer {
    	padding-bottom:3px;
    }

    /* don't want any background color for headings */
    h1,h2,h3,h4,h5,h6 {
    	background: [[ColorPalette::Background]];
    	color: [[ColorPalette::Foreground]];
    }

    /* give tiddlers 3d style border and explicit background */
    .tiddler {
    	background: [[ColorPalette::Background]];
    	border-right: 2px [[ColorPalette::TertiaryMid]] solid;
    	border-bottom: 2px [[ColorPalette::TertiaryMid]] solid;
    	margin-bottom: 1em;
    	padding-bottom: 2em;
    }

    /* make options slider look nicer */
    #sidebarOptions .sliderPanel {
    	border:solid 1px [[ColorPalette::PrimaryLight]];
    }

    /* the borders look wrong with the body background */
    #sidebar .button {
    	border-style: none;
    }

    /* displays the list of a tiddler's tags horizontally. used in ViewTemplate */
    .tagglyTagged li.listTitle {
    	display:none
    }
    .tagglyTagged li {
    	display: inline; font-size:90%;
    }
    .tagglyTagged ul {
    	margin:0px; padding:0px;
    }

    /* this means you can put line breaks in SidebarOptions for readability */
    #sidebarOptions br {
    	display:none;
    }
    /* undo the above in OptionsPanel */
    #sidebarOptions .sliderPanel br {
    	display:inline;
    }

    /* horizontal main menu stuff */
    #displayArea {
    	margin: 1em 15.7em 0em 1em; /* use the freed up space */
    }
    #topMenu br {
    	display: none;
    }
    #topMenu {
    	background: [[ColorPalette::PrimaryMid]];
    	color:[[ColorPalette::PrimaryPale]];
    }
    #topMenu {
    	padding:2px;
    }
    #topMenu .button, #topMenu .tiddlyLink, #topMenu a {
    	margin-left: 0.5em;
    	margin-right: 0.5em;
    	padding-left: 3px;
    	padding-right: 3px;
    	color: [[ColorPalette::PrimaryPale]];
    	font-size: 115%;
    }
    #topMenu .button:hover, #topMenu .tiddlyLink:hover {
    	background: [[ColorPalette::PrimaryDark]];
    }

    /* make it print a little cleaner */
    @media print {
    	#topMenu {
    		display: none ! important;
    	}
    	/* not sure if we need all the importants */
    	.tiddler {
    		border-style: none ! important;
    		margin:0px ! important;
    		padding:0px ! important;
    		padding-bottom:2em ! important;
    	}
    	.tagglyTagging .button, .tagglyTagging .hidebutton {
    		display: none ! important;
    	}
    	.headerShadow {
    		visibility: hidden ! important;
    	}
    	.tagglyTagged .quickopentag, .tagged .quickopentag {
    		border-style: none ! important;
    	}
    	.quickopentag a.button, .miniTag {
    		display: none ! important;
    	}
    }

    /* *** Additions by Ichthyostega *** */
    .red {
    	background: #ffcc99;
    	color: #ff2210;
    	padding: 0px 0.8ex;
    }

    .viewer th {
            background: #91a6af;
    }
    /*}}}*/

    <<timeline better:true maxDays:55 maxEntries:45>>

    /***
    |Name|TaskMacroPlugin|
    |Author|<<extension TaskMacroPlugin author>>|
    |Location|<<extension TaskMacroPlugin source>>|
    |License|<<extension TaskMacroPlugin license>>|
    |Version|<<extension TaskMacroPlugin versionAndDate>>|
    !Description
    A set of macros to help you keep track of time estimates for tasks.

    Macros defined:
    * {{{task}}}: Displays a task description and makes it easy to estimate and track the time spent on the task.
    * {{{taskadder}}}: Displays text entry field to simplify the adding of tasks.
    * {{{tasksum}}}: Displays a summary of tasks sandwiched between two calls to this macro.
    * {{{extension}}}: A simple little macro that displays information about a TiddlyWiki plugin, and that will hopefully someday migrate to the TW core in some form.
    Core overrides:
    * {{{wikify}}}: when wikifying a tiddler's complete text, adds refresh information so the tiddler will be refreshed when it changes
    * {{{config.refreshers}}}: have the built-in refreshers return true; also, add a new refresher ("fullContent") that redisplays a full tiddler whenever it or any nested tiddlers it shows are changed
    * {{{refreshElements}}}: now checks the return value from the refresher and only short-circuits the recursion if the refresher returns true
    !Plugin Information
    ***/
    //{{{
    version.extensions.TaskMacroPlugin = {
    	major: 1, minor: 1, revision: 0,
    	date: new Date(2006,5-1,13),
    	author: "LukeBlanshard",
    	source: "http://labwiki.sourceforge.net/#TaskMacroPlugin",
    	license: "http://labwiki.sourceforge.net/#CopyrightAndLicense"
    }
    //}}}
    /***
    A little macro for pulling out extension info.  Use like {{{<<extension PluginName datum>>}}}, where {{{PluginName}}} is the name you used for {{{version.extensions}}} and {{{datum}}} is either {{{versionAndDate}}} or a property of the extension description object, such as {{{source}}}.
    ***/
    //{{{
    config.macros.extension = {
    	handler: function( place, macroName, params, wikifier, paramString, tiddler ) {
    		var info  = version.extensions[params[0]]
    		var datum = params[1]
    		switch (params[1]) {
    		case 'versionAndDate':
    			createTiddlyElement( place, "span", null, null,
    				info.major+'.'+info.minor+'.'+info.revision+', '+info.date.formatString('DD MMM YYYY') )
    			break;
    		default:
    			wikify( info[datum], place )
    			break;
    		}
    	}
    }
    //}}}
    /***
    !Core Overrides
    ***/
    //{{{
    window.wikify_orig_TaskMacroPlugin = window.wikify
    window.wikify = function(source,output,highlightRegExp,tiddler)
    {
    	if ( tiddler && tiddler.text === source )
    		addDisplayDependency( output, tiddler.title )
    	wikify_orig_TaskMacroPlugin.apply( this, arguments )
    }
    config.refreshers_orig_TaskMacroPlugin = config.refreshers
    config.refreshers = {
    	link: function() {
    		config.refreshers_orig_TaskMacroPlugin.link.apply( this, arguments )
    		return true
    	},
    	content: function() {
    		config.refreshers_orig_TaskMacroPlugin.content.apply( this, arguments )
    		return true
    	},
    	fullContent: function( e, changeList ) {
    		var tiddlers = e.refreshTiddlers
    		if ( changeList == null || tiddlers == null )
    			return false
    		for ( var i=0; i < tiddlers.length; ++i )
    			if ( changeList.find(tiddlers[i]) != null ) {
    				var title = tiddlers[0]
    				story.refreshTiddler( title, null, true )
    				return true
    			}
    		return false
    	}
    }
    function refreshElements(root,changeList)
    {
    	var nodes = root.childNodes;
    	for(var c=0; c<nodes.length; c++)
    		{
    		var e = nodes[c],type;
    		if(e.getAttribute)
    			type = e.getAttribute("refresh");
    		else
    			type = null;
    		var refresher = config.refreshers[type];
    		if ( ! refresher || ! refresher(e, changeList) )
    			{
    			if(e.hasChildNodes())
    				refreshElements(e,changeList);
    			}
    		}
    }
    //}}}
    /***
    !Global Functions
    ***/
    //{{{
    // Add the tiddler whose title is given to the list of tiddlers whose
    // changing will cause a refresh of the tiddler containing the given element.
    function addDisplayDependency( element, title ) {
    	while ( element && element.getAttribute ) {
    		var idAttr = element.getAttribute("id"), tiddlerAttr = element.getAttribute("tiddler")
    		if ( idAttr && tiddlerAttr && idAttr == story.idPrefix+tiddlerAttr ) {
    			var list = element.refreshTiddlers
    			if ( list == null ) {
    				list = [tiddlerAttr]
    				element.refreshTiddlers = list
    				element.setAttribute( "refresh", "fullContent" )
    			}
    			list.pushUnique( title )
    			return
    		}
    		element = element.parentNode
    	}
    }

    // Lifted from Story.prototype.focusTiddler: just return the field instead of focusing it.
    Story.prototype.findEditField = function( title, field )
    {
    	var tiddler = document.getElementById(this.idPrefix + title);
    	if(tiddler != null)
    		{
    		var children = tiddler.getElementsByTagName("*")
    		var e = null;
    		for (var t=0; t<children.length; t++)
    			{
    			var c = children[t];
    			if(c.tagName.toLowerCase() == "input" || c.tagName.toLowerCase() == "textarea")
    				{
    				if(!e)
    					e = c;
    				if(c.getAttribute("edit") == field)
    					e = c;
    				}
    			}
    		return e
    		}
    }

    // Wraps the given event function in another function that handles the
    // event in a standard way.
    function wrapEventHandler( otherHandler ) {
    	return function(e) {
    		if (!e) var e = window.event
    		e.cancelBubble = true
    		if (e.stopPropagation) e.stopPropagation()
    		return otherHandler( e )
    	}
    }
    //}}}
    /***
    !Task Macro
    Usage:
    > {{{<<task orig cur spent>>description}}}
    All of orig, cur, and spent are optional numbers of hours.  The description goes through the end of the line, and is wikified.
    ***/
    //{{{
    config.macros.task = {
    	NASCENT:	0, // Task not yet estimated
    	LIVE:		1, // Estimated but with time remaining
    	DONE:		2, // Completed: no time remaining
    	bullets:	["\u25cb", // nascent (open circle)
    			 "\u25ba", // live (right arrow)
    			 "\u25a0"],// done (black square)
    	styles:		["nascent", "live", "done"],

    	// Translatable text:
    	lingo: {
    		spentTooBig:	"Spent time %0 can't exceed current estimate %1",
    		noNegative:	"Times may not be negative numbers",
    		statusTips:	["Not yet estimated", "To do", "Done"], // Array indexed by state (NASCENT/LIVE/DONE)
    		descClickTip:	" -- Double-click to edit task description",
    		statusClickTip:	" -- Double-click to mark task complete",
    		statusDoneTip:	" -- Double-click to adjust the time spent, to revive the task",
    		origTip:	"Original estimate in hours",
    		curTip:		"Current estimate in hours",
    		curTip2:	"Estimate in hours", // For when orig == cur
    		clickTip:	" -- Click to adjust",
    		spentTip:	"Hours spent on this task",
    		remTip:		"Hours remaining",
    		curPrompt:	"Estimate this task in hours, or adjust the current estimate by starting with + or -.\n\nYou may optionally also set or adjust the time spent by putting a second number after the first.",
    		spentPrompt:	"Enter the number of hours you've spent on this task, or adjust the current number by starting with + or -.\n\nYou may optionally also set or adjust the time remaining by putting a second number after the first.",
    		remPrompt:	"Enter the number of hours it will take to finish this task, or adjust the current estimate by starting with + or -.\n\nYou may optionally also set or adjust the time spent by putting a second number after the first.",
    		numbersOnly:	"Enter numbers only, please",
    		notCurrent:	"The tiddler has been modified since it was displayed, please redisplay it before doing this."
    	},

    	// The macro handler
    	handler: function( place, macroName, params, wikifier, paramString, tiddler )
    	{
    		var start = wikifier.matchStart, end = wikifier.nextMatch

    		var origStr	= params.length > 0? params.shift() : "?"
    		var orig	= +origStr // as a number
    		var cur		= params.length > 1? +params.shift() : orig
    		var spent	= params.length > 0? +params.shift() : 0
    		if ( spent > cur )
    			throw Error( this.lingo.spentTooBig.format([spent, cur]) )
    		if ( orig < 0 || cur < 0 || spent < 0 )
    			throw Error( this.lingo.noNegative )
    		var rem		= cur - spent
    		var state	= isNaN(orig+rem)? this.NASCENT : rem > 0? this.LIVE : this.DONE
    		var table	= createTiddlyElement( place, "table", null, "task "+this.styles[state] )
    		var tbody	= createTiddlyElement( table, "tbody" )
    		var row		= createTiddlyElement( tbody, "tr" )
    		var statusCell	= createTiddlyElement( row,   "td", null, "status", this.bullets[state] )
    		var descCell	= createTiddlyElement( row,   "td", null, "description" )

    		var origCell	= state==this.NASCENT || orig==cur? null
    				: createTiddlyElement( row, "td", null, "numeric original" )
    		var curCell	= createTiddlyElement( row, "td", null, "numeric current" )
    		var spentCell	= createTiddlyElement( row, "td", null, "numeric spent" )
    		var remCell	= createTiddlyElement( row, "td", null, "numeric remaining" )

    		var sums = config.macros.tasksum.tasksums
    		if ( sums && sums.length ) {
    			var summary = [(state == this.NASCENT? NaN : orig), cur, spent]
    			summary.owner = tiddler
    			sums[0].push( summary )
    		}

    		// The description goes to the end of the line
    		wikifier.subWikify( descCell, "$\\n?" )
    		var descEnd = wikifier.nextMatch

    		statusCell.setAttribute( "title", this.lingo.statusTips[state] )
    		descCell.setAttribute(   "title", this.lingo.statusTips[state]+this.lingo.descClickTip )
    		if (origCell) {
    			createTiddlyElement( origCell, "div", null, null, orig )
    			origCell.setAttribute( "title", this.lingo.origTip )
    			curCell.setAttribute( "title", this.lingo.curTip )
    		}
    		else {
    			curCell.setAttribute( "title", this.lingo.curTip2 )
    		}
    		var curDivContents = (state==this.NASCENT)? "?" : cur
    		var curDiv = createTiddlyElement( curCell, "div", null, null, curDivContents )
    		spentCell.setAttribute( "title", this.lingo.spentTip )
    		var spentDiv = createTiddlyElement( spentCell, "div", null, null, spent )
    		remCell.setAttribute( "title", this.lingo.remTip )
    		var remDiv = createTiddlyElement( remCell, "div", null, null, rem )

    		// Handle double-click on the description by going
    		// into edit mode and selecting the description
    		descCell.ondblclick = this.editDescription( tiddler, end, descEnd )

    		function appTitle( el, suffix ) {
    			el.setAttribute( "title", el.getAttribute("title")+suffix )
    		}

    		// For incomplete tasks, handle double-click on the bullet by marking the task complete
    		if ( state != this.DONE ) {
    			appTitle( statusCell, this.lingo.statusClickTip )
    			statusCell.ondblclick = this.markTaskComplete( tiddler, start, end, macroName, orig, cur, state )
    		}
    		// For complete ones, handle double-click on the bullet by letting you adjust the time spent
    		else {
    			appTitle( statusCell, this.lingo.statusDoneTip )
    			statusCell.ondblclick = this.adjustTimeSpent( tiddler, start, end, macroName, orig, cur, spent )
    		}

    		// Add click handlers for the numeric cells.
    		if ( state != this.DONE ) {
    			appTitle( curCell, this.lingo.clickTip )
    			curDiv.className = "adjustable"
    			curDiv.onclick = this.adjustCurrentEstimate( tiddler, start, end, macroName,
    				orig, cur, spent, curDivContents )
    		}
    		appTitle( spentCell, this.lingo.clickTip )
    		spentDiv.className = "adjustable"
    		spentDiv.onclick = this.adjustTimeSpent( tiddler, start, end, macroName, orig, cur, spent )
    		if ( state == this.LIVE ) {
    			appTitle( remCell, this.lingo.clickTip )
    			remDiv.className = "adjustable"
    			remDiv.onclick = this.adjustTimeRemaining( tiddler, start, end, macroName, orig, cur, spent )
    		}
    	},

    	// Puts the tiddler into edit mode, and selects the range of characters
    	// defined by start and end.  Separated for leak prevention in IE.
    	editDescription: function( tiddler, start, end ) {
    		return wrapEventHandler( function(e) {
    			story.displayTiddler( null, tiddler.title, DEFAULT_EDIT_TEMPLATE )
    			var tiddlerElement = document.getElementById( story.idPrefix + tiddler.title )
    			window.scrollTo( 0, ensureVisible(tiddlerElement) )
    			var element = story.findEditField( tiddler.title, "text" )
    			if ( element && element.tagName.toLowerCase() == "textarea" ) {
    				// Back up one char if the last char's a newline
    				if ( tiddler.text[end-1] == '\n' )
    					--end
    				element.focus()
    				if ( element.setSelectionRange != undefined ) { // Mozilla
    					element.setSelectionRange( start, end )
    					// Damn mozilla doesn't scroll to visible.  Approximate.
    					var max = 0.0 + element.scrollHeight
    					var len = element.textLength
    					var top = max*start/len, bot = max*end/len
    					element.scrollTop = Math.min( top, (bot+top-element.clientHeight)/2 )
    				}
    				else if ( element.createTextRange != undefined ) { // IE
    					var range = element.createTextRange()
    					range.collapse()
    					range.moveEnd("character", end)
    					range.moveStart("character", start)
    					range.select()
    				}
    				else // Other? Too bad, just select the whole thing.
    					element.select()
    				return false
    			}
    			else
    				return true
    		} )
    	},

    	// Modifies a task macro call such that the task appears complete.
    	markTaskComplete: function( tiddler, start, end, macroName, orig, cur, state ) {
    		var macro = this, text = tiddler.text
    		return wrapEventHandler( function(e) {
    			if ( text !== tiddler.text ) {
    				alert( macro.lingo.notCurrent )
    				return false
    			}
    			if ( state == macro.NASCENT )
    				orig = cur = 0
    			// The second "cur" in the call below bumps up the time spent
    			// to match the current estimate.
    			macro.replaceMacroCall( tiddler, start, end, macroName, orig, cur, cur )
    			return false
    		} )
    	},

    	// Asks the user for an adjustment to the current estimate, modifies the macro call accordingly.
    	adjustCurrentEstimate: function( tiddler, start, end, macroName, orig, cur, spent, curDivContents ) {
    		var macro = this, text = tiddler.text
    		return wrapEventHandler( function(e) {
    			if ( text !== tiddler.text ) {
    				alert( macro.lingo.notCurrent )
    				return false
    			}
    			var txt = prompt( macro.lingo.curPrompt, curDivContents )
    			if ( txt != null ) {
    				var a = macro.breakInput( txt )
    				cur = macro.offset( cur, a[0] )
    				if ( a.length > 1 )
    					spent = macro.offset( spent, a[1] )
    				macro.replaceMacroCall( tiddler, start, end, macroName, orig, cur, spent )
    			}
    			return false
    		} )
    	},

    	// Asks the user for an adjustment to the time spent, modifies the macro call accordingly.
    	adjustTimeSpent: function( tiddler, start, end, macroName, orig, cur, spent ) {
    		var macro = this, text = tiddler.text
    		return wrapEventHandler( function(e) {
    			if ( text !== tiddler.text ) {
    				alert( macro.lingo.notCurrent )
    				return false
    			}
    			var txt = prompt( macro.lingo.spentPrompt, spent )
    			if ( txt != null ) {
    				var a = macro.breakInput( txt )
    				spent = macro.offset( spent, a[0] )
    				var rem = cur - spent
    				if ( a.length > 1 ) {
    					rem = macro.offset( rem, a[1] )
    					cur = spent + rem
    				}
    				macro.replaceMacroCall( tiddler, start, end, macroName, orig, cur, spent )
    			}
    			return false
    		} )
    	},

    	// Asks the user for an adjustment to the time remaining, modifies the macro call accordingly.
    	adjustTimeRemaining: function( tiddler, start, end, macroName, orig, cur, spent ) {
    		var macro = this
    		var text  = tiddler.text
    		var rem   = cur - spent
    		return wrapEventHandler( function(e) {
    			if ( text !== tiddler.text ) {
    				alert( macro.lingo.notCurrent )
    				return false
    			}
    			var txt = prompt( macro.lingo.remPrompt, rem )
    			if ( txt != null ) {
    				var a = macro.breakInput( txt )
    				var newRem = macro.offset( rem, a[0] )
    				if ( newRem > rem || a.length > 1 )
    					cur += (newRem - rem)
    				else
    					spent += (rem - newRem)
    				if ( a.length > 1 )
    					spent = macro.offset( spent, a[1] )
    				macro.replaceMacroCall( tiddler, start, end, macroName, orig, cur, spent )
    			}
    			return false
    		} )
    	},

    	// Breaks input at spaces & commas, returns array
    	breakInput: function( txt ) {
    		var a = txt.trim().split( /[\s,]+/ )
    		if ( a.length == 0 )
    			a = [NaN]
    		return a
    	},

    	// Adds to, subtracts from, or replaces a numeric value
    	offset: function( num, txt ) {
    		if ( txt == "" || typeof(txt) != "string" )
    			return NaN
    		if ( txt.match(/^[+-]/) )
    			return num + (+txt)
    		return +txt
    	},

    	// Does some error checking, then replaces the indicated macro
    	// call within the text of the given tiddler.
    	replaceMacroCall: function( tiddler, start, end, macroName, orig, cur, spent )
    	{
    		if ( isNaN(cur+spent) ) {
    			alert( this.lingo.numbersOnly )
    			return
    		}
    		if ( spent < 0 || cur < 0 ) {
    			alert( this.lingo.noNegative )
    			return
    		}
    		if ( isNaN(orig) )
    			orig = cur
    		if ( spent > cur )
    			cur = spent
    		var text = tiddler.text.substring(0,start) + "<<" + macroName + " " +
    			orig + " " + cur + " " + spent + ">>" + tiddler.text.substring(end)
    		var title = tiddler.title
    		store.saveTiddler( title, title, text, config.options.txtUserName, new Date(), undefined )
    		//story.refreshTiddler( title, null, true )
    		if ( config.options.chkAutoSave )
    			saveChanges()
    	}
    }
    //}}}
    /***
    !Tasksum Macro
    Usage:
    > {{{<<tasksum "start" ["here" [intro]]>>}}}
    or:
    > {{{<<tasksum "end" [intro]>>}}}
    Put one of the {{{<<tasksum start>>}}} lines before the tasks you want to summarize, and an {{{end}}} line after them.  By default, the summary goes at the end; if you include {{{here}}} in the start line, the summary will go at the top.  The intro argument, if supplied, replaces the default text introducing the summary.
    ***/
    //{{{
    config.macros.tasksum = {

    	// Translatable text:
    	lingo: {
    		unrecVerb:	"<<%0>> requires 'start' or 'end' as its first argument",
    		mustMatch:	"<<%0 end>> must match a preceding <<%0 start>>",
    		defIntro:	"Task summary:",
    		nascentSum:	"''%0 not estimated''",
    		doneSum:	"%0 complete (in %1 hours)",
    		liveSum:	"%0 ongoing (%1 hours so far, ''%2 hours remaining'')",
    		overSum:	"Total overestimate: %0%.",
    		underSum:	"Total underestimate: %0%.",
    		descPattern:	"%0 %1. %2",
                    origTip:	"Total original estimates in hours",
    		curTip:		"Total current estimates in hours",
    		spentTip:	"Total hours spent on tasks",
    		remTip:		"Total hours remaining"
    	},

    	// The macro handler
    	handler: function( place, macroName, params, wikifier, paramString, tiddler )
    	{
    		var sums = this.tasksums
    		if ( params[0] == "start" ) {
    			sums.unshift([])
    			if ( params[1] == "here" ) {
    				sums[0].intro = params[2] || this.lingo.defIntro
    				sums[0].place = place
    				sums[0].placement = place.childNodes.length
    			}
    		}
    		else if ( params[0] == "end" ) {
    			if ( ! sums.length )
    				throw Error( this.lingo.mustMatch.format([macroName]) )
    			var list = sums.shift()
    			var intro = list.intro || params[1] || this.lingo.defIntro
    			var nNascent=0, nLive=0, nDone=0, nMine=0
    			var totLiveSpent=0, totDoneSpent=0
    			var totOrig=0, totCur=0, totSpent=0
    			for ( var i=0; i < list.length; ++i ) {
    				var a = list[i]
    				if ( a.length > 3 ) {
    					nNascent 	+= a[0]
    					nLive 		+= a[1]
    					nDone 		+= a[2]
    					totLiveSpent 	+= a[3]
    					totDoneSpent 	+= a[4]
    					totOrig 	+= a[5]
    					totCur 		+= a[6]
    					totSpent 	+= a[7]
    					if ( a.owner == tiddler )
    						nMine	+= a[8]
    				}
    				else {
    					if ( a.owner == tiddler )
    						++nMine
    					if ( isNaN(a[0]) ) {
    						++nNascent
    					}
    					else {
    						if ( a[1] > a[2] ) {
    							++nLive
    							totLiveSpent += a[2]
    						}
    						else {
    							++nDone
    							totDoneSpent += a[2]
    						}
    						totOrig  += a[0]
    						totCur   += a[1]
    						totSpent += a[2]
    					}
    				}
    			}

    			// If we're nested, push a summary outward
                            if ( sums.length ) {
    				var summary = [nNascent, nLive, nDone, totLiveSpent, totDoneSpent,
    						totOrig, totCur, totSpent, nMine]
    				summary.owner = tiddler
    				sums[0].push( summary )
    			}

    			var descs = [], styles = []
    			if ( nNascent > 0 ) {
    				descs.push( this.lingo.nascentSum.format([nNascent]) )
    				styles.push( "nascent" )
    			}
    			if ( nDone > 0 )
    				descs.push( this.lingo.doneSum.format([nDone, totDoneSpent]) )
    			if ( nLive > 0 ) {
    				descs.push( this.lingo.liveSum.format([nLive, totLiveSpent, totCur-totSpent]) )
    				styles.push( "live" )
    			}
    			else
    				styles.push( "done" )
    			var off = ""
    			if ( totOrig > totCur )
    				off = this.lingo.overSum.format( [Math.round(100.0*(totOrig-totCur)/totCur)] )
    			else if ( totCur > totOrig )
    				off = this.lingo.underSum.format( [Math.round(100.0*(totCur-totOrig)/totOrig)] )

    			var top		= (list.intro != undefined)
    			var table	= createTiddlyElement( null, "table", null, "tasksum "+(top?"top":"bottom") )
    			var tbody	= createTiddlyElement( table, "tbody" )
    			var row		= createTiddlyElement( tbody, "tr", null, styles.join(" ") )
    			var descCell	= createTiddlyElement( row,   "td", null, "description" )

    			var description = this.lingo.descPattern.format( [intro, descs.join(", "), off] )
    			wikify( description, descCell, null, tiddler )

    			var origCell	= totOrig == totCur? null
    					: createTiddlyElement( row, "td", null, "numeric original", totOrig )
    			var curCell	= createTiddlyElement( row, "td", null, "numeric current", totCur )
    			var spentCell	= createTiddlyElement( row, "td", null, "numeric spent", totSpent )
    			var remCell	= createTiddlyElement( row, "td", null, "numeric remaining", totCur-totSpent )

    			if ( origCell )
    				origCell.setAttribute( "title", this.lingo.origTip )
    			curCell  .setAttribute( "title", this.lingo.curTip )
    			spentCell.setAttribute( "title", this.lingo.spentTip )
    			remCell  .setAttribute( "title", this.lingo.remTip )

    			// Discard the table if there are no tasks
    			if ( list.length > 0 ) {
    				var place = top? list.place : place
    				var placement = top? list.placement : place.childNodes.length
    				if ( placement >= place.childNodes.length )
    					place.appendChild( table )
    				else
    					place.insertBefore( table, place.childNodes[placement] )
    			}
    		}
    		else
    			throw Error( this.lingo.unrecVerb.format([macroName]) )

    		// If we're wikifying, and are followed by end-of-line, swallow the newline.
    		if ( wikifier && wikifier.source.charAt(wikifier.nextMatch) == "\n" )
    			++wikifier.nextMatch
    	},

    	// This is the stack of pending summaries
    	tasksums: []
    }
    //}}}
    /***
    !Taskadder Macro
    Usage:
    > {{{<<taskadder ["above"|"below"|"focus"|"nofocus"]...>>}}}
    Creates a line with text entry fields for a description and an estimate.  By default, puts focus in the description field and adds tasks above the entry fields.  Use {{{nofocus}}} to not put focus in the description field.  Use {{{below}}} to add tasks below the entry fields.
    ***/
    //{{{
    config.macros.taskadder = {

    	// Translatable text:
    	lingo: {
    		unrecParam:	"<<%0>> doesn't recognize '%1' as a parameter",
    		descTip:	"Describe a new task",
    		curTip:		"Estimate how long in hours the task will take",
    		buttonText:	"add task",
    		buttonTip:	"Add a new task with the description and estimate as entered",
    		notCurrent:	"The tiddler has been modified since it was displayed, please redisplay it before adding a task this way.",

    		eol:		"eol"
    	},

    	// The macro handler
    	handler: function( place, macroName, params, wikifier, paramString, tiddler )
    	{
    		var above = true
    		var focus = false

    		while ( params.length > 0 ) {
    			var p = params.shift()
    			switch (p) {
    			case "above": 	above = true;  break
    			case "below": 	above = false; break
    			case "focus": 	focus = true;  break
    			case "nofocus":	focus = false; break
    			default:	throw Error( this.lingo.unrecParam.format([macroName, p]) )
    			}
    		}

    		// If we're followed by end-of-line, swallow the newline.
    		if ( wikifier.source.charAt(wikifier.nextMatch) == "\n" )
    			++wikifier.nextMatch

    		var where	= above? wikifier.matchStart : wikifier.nextMatch

    		var table	= createTiddlyElement( place, "table", null, "task" )
    		var tbody	= createTiddlyElement( table, "tbody" )
    		var row		= createTiddlyElement( tbody, "tr" )
    		var statusCell	= createTiddlyElement( row,   "td", null, "status" )
    		var descCell	= createTiddlyElement( row,   "td", null, "description" )
    		var curCell	= createTiddlyElement( row,   "td", null, "numeric" )
    		var addCell	= createTiddlyElement( row,   "td", null, "addtask" )

    		var descId	= this.generateId()
    		var curId	= this.generateId()
    		var descInput	= createTiddlyElement( descCell, "input", descId )
    		var curInput	= createTiddlyElement( curCell,  "input", curId  )

    		descInput.setAttribute( "type", "text" )
    		curInput .setAttribute( "type", "text" )
    		descInput.setAttribute( "size", "40")
    		curInput .setAttribute( "size", "6" )
    		descInput.setAttribute( "autocomplete", "off" );
    		curInput .setAttribute( "autocomplete", "off" );
    		descInput.setAttribute( "title", this.lingo.descTip );
    		curInput .setAttribute( "title", this.lingo.curTip  );

    		var addAction	= this.addTask( tiddler, where, descId, curId, above )
    		var addButton	= createTiddlyButton( addCell, this.lingo.buttonText, this.lingo.buttonTip, addAction )

    		descInput.onkeypress = this.handleEnter(addAction)
    		curInput .onkeypress = descInput.onkeypress
    		addButton.onkeypress = this.handleSpace(addAction)
    		if ( focus || tiddler.taskadderLocation == where ) {
    			descInput.focus()
    			descInput.select()
    		}
    	},

    	// Returns a function that inserts a new task macro into the tiddler.
    	addTask: function( tiddler, where, descId, curId, above ) {
    		var macro = this, oldText = tiddler.text
    		return wrapEventHandler( function(e) {
    			if ( oldText !== tiddler.text ) {
    				alert( macro.lingo.notCurrent )
    				return false
    			}
    			var desc	= document.getElementById(descId).value
    			var cur		= document.getElementById(curId) .value
    			var init	= tiddler.text.substring(0,where) + "<<task " + cur + ">> " + desc + "\n"
    			var text	= init + tiddler.text.substring(where)
    			var title	= tiddler.title
    			tiddler.taskadderLocation = (above? init.length : where)
    			try {
    				store.saveTiddler( title, title, text, config.options.txtUserName, new Date(), undefined )
    				//story.refreshTiddler( title, null, true )
    			}
    			finally {
    				delete tiddler.taskadderLocation
    			}
    			if ( config.options.chkAutoSave )
    				saveChanges()
    		} )
    	},

    	// Returns an event handler that delegates to two other functions: "matches" to decide
    	// whether to consume the event, and "addTask" to actually perform the work.
    	handleGeneric: function( addTask, matches ) {
    		return function(e) {
    			if (!e) var e = window.event
    			var consume = false
    			if ( matches(e) ) {
    				consume = true
    				addTask( e )
    			}
    			e.cancelBubble = consume;
    			if ( consume && e.stopPropagation ) e.stopPropagation();
    			return !consume;
    		}
    	},

    	// Returns an event handler that handles enter keys by calling another event handler
    	handleEnter: function( addTask ) {
    		return this.handleGeneric( addTask, function(e){return e.keyCode == 13 || e.keyCode == 10} ) // Different codes for Enter
    	},

    	// Returns an event handler that handles the space key by calling another event handler
    	handleSpace: function( addTask ) {
    		return this.handleGeneric( addTask, function(e){return (e.charCode||e.keyCode) == 32} )
    	},

    	counter: 0,
    	generateId: function() {
    		return "taskadder:" + String(this.counter++)
    	}
    }
    //}}}
    /***
    !Stylesheet
    ***/
    //{{{
    var stylesheet = '\
    .viewer table.task, table.tasksum {\
    	width: 100%;\
    	padding: 0;\
    	border-collapse: collapse;\
    }\
    .viewer table.task {\
    	border: none;\
    	margin: 0;\
    }\
    table.tasksum, .viewer table.tasksum {\
    	border: solid 2px #999;\
    	margin: 3px 0;\
    }\
    table.tasksum td {\
    	text-align: center;\
    	border: 1px solid #ddd;\
    	background-color: #ffc;\
    	vertical-align: middle;\
    	margin: 0;\
    	padding: 0;\
    }\
    .viewer table.task tr {\
    	border: none;\
    }\
    .viewer table.task td {\
    	text-align: center;\
    	vertical-align: baseline;\
    	border: 1px solid #fff;\
    	background-color: inherit;\
    	margin: 0;\
    	padding: 0;\
    }\
    td.numeric {\
    	width: 3em;\
    }\
    table.task td.numeric div {\
    	border: 1px solid #ddd;\
    	background-color: #ffc;\
    	margin: 1px 0;\
    	padding: 0;\
    }\
    table.task td.original div {\
    	background-color: #fdd;\
    }\
    table.tasksum td.original {\
    	background-color: #fdd;\
    }\
    table.tasksum td.description {\
    	background-color: #e8e8e8;\
    }\
    table.task td.status {\
    	width: 1.5em;\
    	cursor: default;\
    }\
    table.task td.description, table.tasksum td.description {\
    	width: auto;\
    	text-align: left;\
    	padding: 0 3px;\
    }\
    table.task.done td.status,table.task.done td.description {\
    	color: #ccc;\
    }\
    table.task.done td.current, table.task.done td.remaining {\
    	visibility: hidden;\
    }\
    table.task.done td.spent div, table.tasksum tr.done td.current,\
    table.tasksum tr.done td.spent, table.tasksum tr.done td.remaining {\
    	background-color: #eee;\
    	color: #aaa;\
    }\
    table.task.nascent td.description {\
    	color: #844;\
    }\
    table.task.nascent td.current div, table.tasksum tr.nascent td.numeric.current {\
    	font-weight: bold;\
    	color: #c00;\
    	background-color: #def;\
    }\
    table.task.nascent td.spent, table.task.nascent td.remaining {\
    	visibility: hidden;\
    }\
    td.remaining {\
    	font-weight: bold;\
    }\
    .adjustable {\
    	cursor: pointer; \
    }\
    table.task input {\
    	display: block;\
    	width: 100%;\
    	font: inherit;\
    	margin: 2px 0;\
    	padding: 0;\
    	border: 1px inset #999;\
    }\
    table.task td.numeric input {\
    	background-color: #ffc;\
    	text-align: center;\
    }\
    table.task td.addtask {\
    	width: 6em;\
    	border-left: 2px solid white;\
    	vertical-align: middle;\
    }\
    '
    setStylesheet( stylesheet, "TaskMacroPluginStylesheet" )
    //}}}

    /***
    ''TextAreaPlugin for TiddlyWiki version 2.0''
    ^^author: Eric Shulman - ELS Design Studios
    source: http://www.elsdesign.com/tiddlywiki/#TextAreaPlugin
    license: [[Creative Commons Attribution-ShareAlike 2.5 License|http://creativecommons.org/licenses/by-sa/2.5/]]^^

    This plugin 'hijacks' the TW core function, ''Story.prototype.focusTiddler()'', so it can add special 'keyDown' handlers to adjust several behaviors associated with the textarea control used in the tiddler editor.  Specifically, it:
    * Adds text search INSIDE of edit fields.^^
    Use ~CTRL-F for "Find" (prompts for search text), and ~CTRL-G for "Find Next" (uses previous search text)^^
    * Enables TAB characters to be entered into field content^^
    (instead of moving to next field)^^
    * Option to set cursor at top of edit field instead of auto-selecting contents^^
    (see configuration section for checkbox)^^
    !!!!!Configuration
    <<<
    <<option chkDisableAutoSelect>> place cursor at start of textarea instead of pre-selecting content
    <<option chkTextAreaExtensions>> add control-f (find), control-g (find again) and allow TABs as input in textarea
    <<<
    !!!!!Installation
    <<<
    Import (or copy/paste) the following tiddlers into your document:
    ''TextAreaPlugin'' (tagged with <<tag systemConfig>>)
    <<<
    !!!!!Revision History
    <<<
    ''2006.01.22 [1.0.1]''
    only add extra key processing for TEXTAREA elements (not other edit fields).
    added option to enable/disable textarea keydown extensions (default is "standard keys" only)
    ''2006.01.22 [1.0.0]''
    Moved from temporary "System Tweaks" tiddler into 'real' TextAreaPlugin tiddler.
    <<<
    !!!!!Code
    ***/
    //{{{
    version.extensions.textAreaPlugin= {major: 1, minor: 0, revision: 1, date: new Date(2006,1,23)};
    //}}}

    //{{{
    if (!config.options.chkDisableAutoSelect) config.options.chkDisableAutoSelect=false; // default to standard action
    if (!config.options.chkTextAreaExtensions) config.options.chkTextAreaExtensions=false; // default to standard action

    // Focus a specified tiddler. Attempts to focus the specified field, otherwise the first edit field it finds
    Story.prototype.focusTiddler = function(title,field)
    {
    	var tiddler = document.getElementById(this.idPrefix + title);
    	if(tiddler != null)
    		{
    		var children = tiddler.getElementsByTagName("*")
    		var e = null;
    		for (var t=0; t<children.length; t++)
    			{
    			var c = children[t];
    			if(c.tagName.toLowerCase() == "input" || c.tagName.toLowerCase() == "textarea")
    				{
    				if(!e)
    					e = c;
    				if(c.getAttribute("edit") == field)
    					e = c;
    				}
    			}
    		if(e)
    			{
    			e.focus();
    			e.select(); // select entire contents

    			// TWEAK: add TAB and "find" key handlers
    			if (config.options.chkTextAreaExtensions) // add extra key handlers
    				addKeyDownHandlers(e);

    			// TWEAK: option to NOT autoselect contents
    			if (config.options.chkDisableAutoSelect) // set cursor to start of field content
    				if (e.setSelectionRange) e.setSelectionRange(0,0); // for FF
    				else if (e.createTextRange) { var r=e.createTextRange(); r.collapse(true); r.select(); } // for IE

    			}
    		}
    }
    //}}}

    //{{{
    function addKeyDownHandlers(e)
    {
    	// exit if not textarea or element doesn't allow selections
    	if (e.tagName.toLowerCase()!="textarea" || !e.setSelectionRange) return;

    	// utility function: exits keydown handler and prevents browser from processing the keystroke
    	var processed=function(ev) { ev.cancelBubble=true; if (ev.stopPropagation) ev.stopPropagation(); return false; }

    	// capture keypress in edit field
    	e.onkeydown = function(ev) { if (!ev) var ev=window.event;

    		// process TAB
    		if (!ev.shiftKey && ev.keyCode==9) {
    			// replace current selection with a TAB character
    			var start=e.selectionStart; var end=e.selectionEnd;
    			e.value=e.value.substr(0,start)+String.fromCharCode(9)+e.value.substr(end);
    			// update insertion point, scroll it into view
    			e.setSelectionRange(start+1,start+1);
    			var linecount=e.value.split('\n').length;
    			var thisline=e.value.substr(0,e.selectionStart).split('\n').length-1;
    			e.scrollTop=Math.floor((thisline-e.rows/2)*e.scrollHeight/linecount);
    			return processed(ev);
    		}

    		// process CTRL-F (find matching text) or CTRL-G (find next match)
    		if (ev.ctrlKey && (ev.keyCode==70||ev.keyCode==71)) {
    			// if ctrl-f or no previous search, prompt for search text (default to previous text or current selection)... if no search text, exit
    			if (ev.keyCode==70||!e.find||!e.find.length)
    				{ var f=prompt("find:",e.find?e.find:e.value.substring(e.selectionStart,e.selectionEnd)); e.focus(); e.find=f?f:e.find; }
    			if (!e.find||!e.find.length) return processed(ev);
    			// do case-insensitive match with 'wraparound'...  if not found, alert and exit
    			var newstart=e.value.toLowerCase().indexOf(e.find.toLowerCase(),e.selectionStart+1);
    			if (newstart==-1) newstart=e.value.toLowerCase().indexOf(e.find.toLowerCase());
    			if (newstart==-1) { alert("'"+e.find+"' not found"); e.focus(); return processed(ev); }
    			// set new selection, scroll it into view, and report line position in status bar
    			e.setSelectionRange(newstart,newstart+e.find.length);
    			var linecount=e.value.split('\n').length;
    			var thisline=e.value.substr(0,e.selectionStart).split('\n').length;
    			e.scrollTop=Math.floor((thisline-1-e.rows/2)*e.scrollHeight/linecount);
    			window.status="line: "+thisline+"/"+linecount;
    			return processed(ev);
    		}
    	}
    }
    //}}}

    http://tiddlywiki.com/

    Timeline is the top level element within the [[Session (Project)|Session]]. It is visible within a //timeline view// in the GUI and represents the effective (resulting) arrangement of media objects, to be rendered for output or viewed in a Monitor (viewer window). A timeline is comprised of:
    * a time axis in abolute time (WIP: not clear if this is an entity or just a conceptual definition)
    * a PlayControler
    * a list of global Pipes representing the possible outputs (master busses)
    * //exactly one// top-level [[EDL (Sequence)|EDL]], which in turn may contain further nested ~EDLs (Sequences).
    Please note especially that following this design //a timeline doesn't define tracks.// [[Tracks form a Tree|Track]] and are part of the individual ~EDLs (Sequences), together with the media objects placed to these tracks.

    Within the Project, there may be ''multiple timelines'', to be viewed and rendered independently. But, being the top-level entities, multiple timelines may not be combined further. You can always just render (or view) one specific timeline. A given sequence may be referred directly or indirectly from multiple timelines though.

    ''Note'': in early drafts of the design (2007) there was an entity called "Timeline" within the [[Fixture]]. This entity seems superfluous and has been dropped. It never got any relevance in existing code and at most was used in some code comments.

    There is a three-level hierarchy: [[Project|Session]], [[Timeline]], [[Sequence|EDL]]. Each project can contain ''multiple timelines'', to be viewed and rendered independently. But, being the top-level entities, these timelines may not be combined further. You can always just render (or view) one specific timeline. Each of those timelines refers to a Sequence, which is a bunch of [[media objects|MObject]] placed to a tree of [[tracks|Track]]. Of course it is possible to use ~Sub-EDLs (Sub-sequences) within the top-level sequence within a timeline to organize a movie into several scenes or chapters.

    [>img[Relation of Timelines, Sequences and MObjects within the Project|uml/fig132741.png]]
    As stated in the [[definition|Timeline]], a timeline refers to exactly one EDL (Sequence), and the latter defines a tree of [[tracks|Track]] and a bunch of media objects placed to these tracks. A Sequence may optionally also contain nested sequences as [[meta-clips|VirtualClip]]. Moreover, obviously several timelines (top-level entities) may refer to the same Sequence without problems.
    This is because the top-level entities (Timelines) are not permitted to be combined further. You may play or render a given timeline, you may even play several timelines simultaneously in different monitor windows, and these different timelines may incorporate the same sequence in a different way. The Sequence just defines the relations between some objects and may be placed relatively to another object (clip, label,...) or similar reference point, or even anchored at an absolute time if desired. In a similar open fashion, within the track-tree of a sequence, we may define a specific signal routing, or we may just fall back to automatic output wiring.

    !Attaching output
    The Timeline owns a list of global [[pipes (busses)|Pipe]] which are used to collect output. If the track tree of a sequence doesn't contain specific routing advice, then connections will be done directly to these global pipes in order and by matching StreamType (i.e. typically video to video master, audio to stereo audio master). When a monitor (viewer window) is attached to this timeline, similar output connections are made from those global pipes, i.e. the video display will take the contents of the first video (master) bus, and the first stereo audio pipe will be pulled and sent to system audio out. The timeline owns a ''play control'' shared by all attached viewers and coordinating the rendering-for-viewing. Similarly, a render task may be attached to the timeline to pull the pipes needed for a given kind of generated output. The actual implementation of the play controller and the coordination of render tasks is located in the Backend, which uses the service of the Proc-Layer to pull the respective exit nodes of the render engine network.

    !Timeline versus Timeline View
    Actually, what the GUI creates and uses is the //view// of a given timeline. This makes no difference to start with, as the view is modelled to be a sub-concept of "timeline" and thus can stand-in. All different views of the //same// timeline also share one single play control instance, i.e. they all have one single playhead position. Doing it this way should be the default, because it's the least confusing. Anyway, it's also possible to create multiple //independent timelines// &mdash; in an extreme case even so when referring to the same top-level sequence. This configuration gives the ability to play the same arrangement in parallel with multiple independent play controllers (and thus independent playhead positions)

    To complement this possibilities, I'd propose to give the //timeline view// the possibility to be re-linked to a sub-sequence. This way, it would stay connected to the main play control, but at the same time show a sub-sequence //in the way it will be treated as  embedded// within the top-level sequence. This would be the default operation mode when a meta-clip is opened (and showed in a separate tab with such a linked timeline view). The reason for this proposed handling is again to give the user the least surprising behaviour. Because, when &mdash; on the contrary &mdash; the sub-sequence would be opened as //separate timeline,// a different absolute time position and a different signal routing may result; doing such should be reserved for advanced use, e.g. when multiple editors cooperate on a single project and a sequence has to be prepared in isolation prior to being integrated in the global sequence (featuring the whole movie).

    Tracks are just a structure used to organize the Media Objects within the EDL. Tracks are associated allways to a specific EDL and the Tracks of an EDL form a //tree.// They can be considered to be an organizing grid, and besides that, they have no special meaning. They are grouping devices, not first-class entities. A track doesn't "have" a port or pipe or "is" a video track and the like; it can be configured to behave in such manner by using placements.

    The ~Track-IDs are assets on their own, but they can be found within a given EDL. So, several ~EDLs can share a single track or each EDL can have tracks with their own, separate identity. (the latter is the default)
    * Like most ~MObjects, tracks have a asset view: you can find a track asset (a track ID) in the asset manager.
    * and they have an object view: there is an track MObject which can be [[placed|Placement]], thus defining properties of this track within one EDL, e.g. the starting point in time
    Of course, we can place other ~MObjects relative to some track (that's the main reason why we want to have tracks). In this sense, the [[handling of Tracks|TrackHandling]] is somewhat special: the placement of some track can be found directly within the EDL (and not within the general collection of placed objects which form the contents of any EDL). This placement defines properties of the track, which will be inherited if necessary by all ~MObjects placed to this track. For example, if placing (=plugging) a track to some global [[Pipe]], and if placing a clip to this track, without placing the clip directly to another pipe, the associated-to-pipe information of the track will be fetched by the builder when needed to make the output connection of the clip.
    &rarr; [[Handling of Tracks|TrackHandling]]
    &rarr; [[Handling of Pipes|PipeHandling]]

    &rarr; [[Anatomy of the high-level model|HighLevelModel]]

    What //exactly&nbsp;// is denoted by &raquo;Track&laquo; &mdash; //basically&nbsp;// a working area to group media objects placed at this track at various time positions &mdash; varies depending on context:
    * viewed as [[structural asset|StructAsset]], tracks are nothing but global identifiers (possibly with attached tags and description)
    * regarding the structure //within each [[EDL]],// tracks form a tree-like grid, the individual track being attached to this tree by a [[Placement]], thus setting up properties of placement (time reference origin, output connection, layer, pan) which will be inherited down to any objects located on this track and on child tracks, if not overridden more locally.
    * with respect to //object identity,// a given track-ID can have an incarnation or manifestation as real track-object within several [[EDLs|EDL]] (meaning you could select, disable or choose for rendering all objects in any EDL placed onto this track). Moreover, the track-//object// and the //placement&nbsp;// of this track within the tree of tracks of a given EDL are two distinguishable entities (meaning a given track &mdash; with a couple of objects located on it &mdash; could be placed differently several times within the same EDL, for example with different start offset or with different layering, output mode or pan position)

    !Identification
    Tracks thus represent a blend of several concepts, but depending on the context it is allways clear which aspect is meant. Seen as [[assets|Asset]], tracks are known by a unique track-ID, which can be either [[queried|ConfigQuery]], or directly refered to by use of the asset-ID (which is a globally known hash). Usually, all referrals are via track-ID, including when you [[place|Placement]] an object onto a track.
    Under some cincumstances though, especially from within the [[Builder]], we refer to a {{{Placement<Track>}}} rather, denoting a specific instantiation located at a distinct node within the tree of tracks of a given EDL. These latter referrals are always done by direct object reference, e.g. while traversing the track tree (generally there is no way to refer to a placement by name).

    !creating tracks
    Similar to [[pipes|Pipe]] and [[processing patterns|ProcPatt]], track-assets need not be created, but rather leap into existence on first referral. On the contrary, you need to explicitly create the {{{Placement<Track>}}} for attaching it to some node within the tree of tracks of an EDL. The public access point for creating such a placement is {{{asset::Struct::create(trackID}}} (i.e. the {{{asset::StructFactory}}}), which, as a convenience shortcut, can also be accessed from the interface of the track-objects within the EDL for adding new child tracks to a given track.

    !removal
    Deleting a Track is an operation with drastic consequences, as it will cause the removal of all child tracks and the deletion of //all object placements to this track,// which could cause the resepctive objects to go out of scope (being deleted automatically by the placements or other smart pointer classes in charge of them). On the contrary, removing a {{{Placement<Track>}}} from the tree of tracks of an EDL will just cause all objects placed onto this track to disappear (because they are no longer reachable for the build process). On re-adding it, they will show up again. (This is how things behave based on how we defined the relations of the entities to be. Another question is if we want to make this functionality available to the user. Judging from the use of Ardour's &laquo;Playlists&raquo;, such a feature may be quite helpful).

    !using Tracks
    The '''Track Asset''' is a rather static object with limited capabilities. It's main purpose is to be a point of referral. Track assets have a description field and you may assign a list of [[tags|Tag]] to them (which could be used for binding ConfigRules).  Note that track assets are globally known within the session, they can't be limited to just one EDL (but you are allways free not to refer to some track from a given EDL). By virtue of this global nature, you can utilize the track assets to enable/disable a bunch of objects irrespective of what EDL they are located in, and probably it's a good idea to allow the selection of specific tracks for rendering.
    Matters are quite different for the placement of a Track within the tree of tracks of a given EDL, and for placing some media object onto a given track. The track placement defines properties which will be inherited to all objects on this track and on all child tracks and thus plays a key role for wiring the objects up to some output pipe. Typically, the top level track of each EDL has a placement-to "the" video and "the" audio master pipe.

    !!!!details to note
    * Tracks are global, but the placement of a track is local within one EDL
    * when objects are placed onto a track, this is done by referal to the global track asset ID. But because this placement of some media object is allways inherently contained within one EDL, the //meaning&nbsp;// of such a placement is to connect to the properties of any track-placement of this given track //within this EDL.//
    * thus tracks-as-ID appear as something global, but tracks-as-propperty-carrier appear to the user as something local and object-like.
    * in an extreme case, you'll add two different placements of a track at different points within the track tree of an EDL. And because the objects placed onto a track refer to the global track-ID, every object "on" this track //within this EDL&nbsp;// will show up two times independently and possibly with different inherited properties (output pipe, layering mode, pan, temporal position)
    * an interesting configuration results from the fact that you can use an EDL as a [["meta clip" or "virtual clip"|VirtualClip]] within another EDL. In this case, you'll probably configure the tracks of the "inner" EDL such as to send their output not to a global pipe but rather to the [[source ports|ClipSourcePort]] of the virtual clip (which are effectively local pipes). Thus, within the "outer" EDL, you could attach effects to the virutal clip, combine it with transitions and place it onto another track, and any missing properties of this latter placement are to be resolved within the "outer" EDL <br/>(it would be perfectly legal to construct a contrieved example when using the same track-ID within "inner" and the "outer" EDL. Because the Placement of this track will probably be different in the both ~EDLs, the behaviour of this placement could be quite different in the "inner" and the "outer" EDL. All of this may seem weird when discussed here in a textual and logical manner, but when viewed within the context and meaning of the various entities of the application, it's rather the way you'd expect it to be: you work locally and things behave as defined locally)
    * note further, the root of the tree of tracks within each EDL //is itself again a //{{{Placement<Track>}}}. There is no necessitiy for doing it this way, but it seemed more stright forward and logical to Ichthyo, as it allowes for an easy way of configuring some things (like ouput connections) as a default within one EDL. As every track can have a list of child tracks, you'll get the "list of tracks" you'd expect.
    * a nice consequence of the latter is: if you create a new EDL, it automatically gets one top-level track to start with, and this track will get a default configured placement (according to what is defined as [[default|DefaultsManagement]] within the current ConfigRules) &mdash; typically starting at t=0 and being plugged into the master video and master audio pipe
    * nothing prevents us from putting several objects at the same temporal location within one track. If the builder can't derive any additional layering information (which could be provided by some other configuration rules), then //there is no layering precedence// &mdash; simply the object encountered first (or last) wins.
    * obviously, one wants the __edit function__ used to create such an overlapping placement&nbsp; also to create an [[transition|TransitionsHandling]] between the overlapping objects. Meaning this edit function will automatically create an transition processor object and provide it with a placement such as to attach it to the region of overlap.

    ''towards a definition of »Track«''. We don't want to tie ourself to some naive and overly simplistic definition, just because it is convenient. For classical (analogue) media, tracks are physical entities dictated by the nature of the process by which the media works. Especially, Tape machines have read/writing heads, which creates fixed tracks to which to route the signals. This is a practical geometric necessity. For digital media, there is no such necessity. We are bound primarily by the editor's habits of working.

    !!!Assessment of Properties
    Media are used as Clips (contiguous chunks), they are a compound of several elementary streams, and they have a temporal extension (length). Indeed, the temporal dimension is the only fundamental property that can't be changed. Orthogonal to this dimension, we find one or more organisational dimensions forming a grid:
    * a media stream may be sent to one of several possible output destinations (image or sound, subgroup busses, MIDI instruments)
    * for any given output destination there may be variations in the //way of connecting// (overlay mode and layer, pan position, MIDI channel)
    * besides, we often just want to stash away some clip without using it, e.g. as an alternative or for later referral
    This is to say we have //several degrees of freedom// within this organisational grid. It's not because some sound is located on this track that he will be sent to a given output, but the clip is located on this track //and// is connected to that output //and// &mdash; supposed we have full-periphonic surround sound &mdash; it is located 60° to the right and 30°. Combined with the (always contiguous) temporal dimension, this discrete grid is thus extruded to form something like discrete Tracks.

    !!!do we really need Tracks?
    Starting with the assumption "everything is just connected processing nodes", Tracks may seem superfluous. The problem with this approach is: it doesn't scale well. While it is fine to be able to connect clips and effects as we see fit (indeed, we want to build such a system), it is clearly not feasible to wire every clip manually to the output ports or add a panner effect to each and every audio sample. Because while editing, most of the time things are done in a fairly regular and systematic manner. Preferably we use the tracks as //preconfigured group setup// and just //place media onto them;// any such [[Placement]] can do the necessary wiring semi-automatic (rule-based).

    !!!the constant part
    there seems to be some non time-varying part in each EDL, that doesn't fit well with the simple model "objects on a timeline". Tracks seen as an organisational grid fall into this category: they are a global property of the given EDL. They could be associated to the Session as a whole, but effectively this would subvert the concept of having [[several EDLs|SessionOverview]]. On the other hand,
    [[pipes|Pipe]] for Video and Sound output are obviously a global property of the Session. There can be several global pipes forming a matrix of subgroup busses. We could add ports or pipes to tracks by default as well, but we don't do this, because, again, this would run counter to our attempt of treating tracks as merely organisational entities. We have special [[source ports|ClipSourcePort]] on individual clips though, and we will have ports on [[virtual clips|VirtualClip]] too.

    !Design
    [[Tracks|Track]] are just a structure used to organize the Media Objects within the EDL. They form a grid, and besides that, they have no special meaning. It seems convenient to make the tracks not just a list, but allow grouping (tree structure) right from start. __~MObjects__ are ''placed'' rather than wired. The wiring is derived from the __Placement__. Placing can happen in several dimensions:
    * placing in time will define when to activate and show the object.
    * placing onto a track associates the ~MObject with this track; the GUI will show it on this track and the track may be used to resolve other properties of the object.
    * placing to a __Pipe__ brings the object in conjunction with this pipe for the build process. It will be considered when building the render network for this pipe. Source-like objects (clips and exit nodes of effect chains) will be connected to the pipe, while transforming objects (effects) are inserted at the pipe. (you may read "placed to pipe X" as "plug into pipe X")
    * depending on the nature of the pipe and the source, placing to some pipe may create additional degrees of freedom, demanding the object to be placed in this new, additional dimensions: Connecting to video out e.g. creates an overlay mode and a layer position which need to be specified, while connecting to a spatial sound system creates the necessity of a pan position. On the other hand, placing a mono clip onto a mono Pipe creates no additional degrees of freedom.
    Placements are __resolved__ resulting in an ExplicitPlacement. In most cases this is just a gathering of properties, but as Placements can be incomplete and relative, there is room for real solving. The resolving mechanism tries to __derive missing properties__ from the __context__: When a clip isn't placed to some pipe but to a Track, than the Track and its parents will be inspected. If some of them has been placed to a pipe, the object will be connected to this pipe. Similar for layers and pan position. This is done by [[Placement]] and LocatingPin; as the [[Builder]] uses ~ExplicitPlacements, he isn't concerned with this resolving and uses just the data they deliver to drive the [[basic building operations|BasicBuildingOperations]]
    &rarr; [[Definition|Track]] and [[handling of Tracks|TrackHandling]]
    &rarr; [[Definition|Pipe]] and [[handling of Pipes|PipeHandling]]

    Transitions combine the data from at least two processing chains and do this combining in a time varying fashion. So, any transition has
    * N input connections
    * either one or N output connections
    * temporal coordinates (time, length)
    * some control data connection to a ParamProvider, because in the most general case the controling curves are treated similar to  [[automation data|AutomationData]]

    !!!how much output ports?
    The standard case of a transition is sort of mixing together two input streams, like e.g. a simple dissolve. For this to be of any use, this input streams need to be connected to the same ouput destination before and after the transition (with regards to the timeline), i.e. the inputs and the transition share placement to the same output pipe. In this case, when the transition starts, the direct connections can be suspended and the transition will switch in seamlessly.

    Using transitions is a very basic task and thus needs viable support by the GUI. Handling of transitions need to be very convienient, because it is so important. Because of this, it is compelling to subsume a more complicated situation and treat this more complicated case similar. This is the case, when two (or N) elements have to be combined in the way of a transition, but their further processing in the processing chain //after// the transition needs to be different, maybe they belong to differnent subgroups, have to appear on different layers or with different pan positions. Of courese, the workaround is simple, at least "simple" from the programmers point of view. It is //not// simple from the editor's point of view the moment the editor has to do this simple thing (changing the wiring and add manualy synced fade curves to the individual parts) a dozend or even a hundred times in some larger project.

    Because of this experience, ichthyo wants to support a more general case of transitions, which have N output connections, behave similar to their "simple" counterpart, but leave out the mixing step. As a plus, such transitions can be inserted at the source ports of N clips or between any intermediary or final output pipes as well. Any transition processor capable of handling this situation should provide some flag, in order to decide if he can be placed in such a manner. (wichin the builder, encountering a  inconsistently placed transition is just an [[building error|BuildingError]])

    A ''~Meta-Clip'' or ''Virtual Clip'' (both are synonymous) denotes a clip which doesn't just pull media streams out of a source media asset, but rather provides the results of rendering a complete sub-network. In all other respects it behaves exactly like a "real" clip, i.e. it has [[source ports|ClipSourcePort]], can have attached effects (thus forming a local render pipe) and can be placed and combined with other clips. Depending on what is wired to the source ports, we get two flavours:
    * a __placeholder clip__ has no "embedded" content. Rather, by virtue of placements and wiring requests, the output of some other pipe somewhere in the session will be wired to the clip's source ports. Thus, pulling data from this clip will effectively pull from these source pipes wired to it.
    * a __nested EDL __ is like the other ~EDLs in the Session, just that any missing placement properties will be derived from the Virtual Clip, which is thought as to "contain" the objects of the nested EDL. Typically, this also [[configures the tracks|TrackHandling]] of the "inner" EDL such as to connect any output to the source ports of the Virtual Clip.

    Like any "real" clip, Virtual Clips have a start offset and a length, which will simply translate into an offset of the frame number pulled from the Virtual Clip's source connection or embedded EDL, making it possible to cut, splice, trim and roll them as usual. This of course implies we can have several instances of the same virtual clip with different start offset and length placed differently. The only limitation is that we can't handle cyclic dependencies for pulling data (which has to be detected and flagged as an error by the builder)

    The ''Visitor Pattern'' is a special form of //double dispatch// &mdash; selecting the function actually to be executed at runtime based both on the concrete type of some tool object //and // the target this tool is applied to. The rationale is to separate some specific implementation details from the basic infrastructure and the global interfaces, which can be limited to describe the fundamental properties and operations, while all details relevant only for some specific sub-problem can be kept together encapsulated in a tool implementation class. Typically, there is some iteration mechanism, allowing to apply these tools to all objects in a given container, a collection or object graph, without knowing the exact type of the target and tool objects. See the [[Visitor Pattern design discussion|VisitorUse]]

    !Problems with Visitor
    The visitor pattern is not very popular, because any implementation is tricky, difficult to understand and often puts quite some burden on the user code. Even Erich Gamma says that on his list of bottom-ten patterns, Visitor is at the very bottom. This may be due to the fact that this original visitor implementation (often called the ''~GoF visitor'') causes a cyclic dependency between the target objects and the visiting tool objects, and includes some repetitive code (which results in silent failure if forgotten). Robert Martin invented 1996 an interesting variation (commonly labled ''acyclic visitor''). By using a marker base class for each concrete target type to be treated by the visitor, and by applying a dynamic cast, we can get rid of the cyclic dependencies. The top level "Visitor" is reduced to a mere empty marker interface in this design, while the actual visiting capabilities for a concrete target object is discovered at application time by the aforementioned dynamic cast. Besides the runntime cost of such a cast, the catch is that now the user code has still more responsibilities, because the need to maintain consistently two parallel object hierarchies.
    Long time there seemed to be not much room for improvement, at least before the advent of generic programming and the discovery of template metaprogramming. ''Loki'' (Alexandrescu, 2000) showed us how to write a library implementation which hides away the technicallities of the visitor pattern and automatically generates most of the repetitive code.

    !Requirements
    * cyclic dependencies should be avoided or at least restricted to some central, library related place.
    * The responsibilities for user code should be as small as possible. Especially, we should minimize the necessity to do corresponding adjustments to separate code locations, e.g. the necessity to maintain parallel hierarchies.
    * Visitor is about //double dispatch,// thus we can't avoid using some table lookup implementation &mdash; more specifically we can't avoid using the cooperating classes vtables. We can expect at least two table lookups for each call dispatch. Besides that, the implementation should not be too wasteful...
    * individual "visiting tool" implementation classes should be able to opt in or opt out on implementing functions treating some of the visitable subclasses.
    * there should be a safe fallback mechanism backed by the visitable object's hierarchy relations. If some concrete visiting tool class decides not to implement a {{{treat(..)}}}-function for some concrete target type, the call should fall back to the next best match according to the target object's hierarchy, i.e. the next best {{{treat(..)}}}-function should be used.
    The last requirement practically rules out the Loki acyclic visitor, because this implementation calls a general fallback function when an exact match based on the target object's type is not possible. Considering our use of the visitor pattern within the render engine builder, such a solution would not be of much use: Some specific builder tool may implement a {{{treat(CompoundClip&)}}}-function, while most of the other builder tools just implement a {{{treat(Clip&)}}}-function, thus handling any multichannel clip via the general clip interface. This is exactly the reason why we want to use visitor at first place. Isolate specific treatment, implement against interfaces.

    !Implementation Notes
    A good starting point for understanding our library implementation of the visitor pattern is {{{tests/components/common/visitingtoolconcept.cpp}}}, which contains a all-in-one-file proof of concept implementation, on which the real implementation ({{{"common/visitor.hpp"}}}) was based.
    * similar to Loki, we use a {{{Visitable}}} base class and a {{{Tool}}} base class (we prefer the name "Tool" over "Visitor", because it makes the intended use more clear).
    * the objects in the {{{Visitable}}} hierarchy implement an {{{apply(Tool&)}}}-function. This function needs to be implemented in a very specific manner, thus the {{{DEFINE_PROCESSABLE_BY}}} macro should be used when possible to insert the definition into a concrete {{{Visitable}}} class.
    * similar to the acyclic visitor, the concrete visiting tool classes inherit from {{{Applicable<TARGET, ...>}}} marker base classes, where the template parameter {{{TARGET}}} is the concrete Visitable type this tool wants to treat, either directly by defining a {{{treat(ConcreteVisitable&)}}}, or by falling back to some more general {{{treat(...)}}} function.
    * consequently our implementation is //rather not acyclic// &mdash; the concrete tool implementation depends on the full definition (header) of all concrete Visitables, but we avoid cyclic dependencies on the interface level. By using a typelist technique inspired by Loki, we can concentrate these dependencies in one single header file, which keeps things maintainable.
    * we use the {{{Applicable<TARGET, ...>}}} marker base classes to drive the generation of Dispatcher classes, each of which holds a table of trampoline functions for carrying out the actual double dispatch at call time. Each concrete Visitable using the {{{DEFINE_PROCESSABLE_BY}}}-macro generates a separate Dispatcher table containing slots for each concrete tool implementation class. To store and access the index position for these "call slots", we use a tag associated with the concrete visiting tool class, which can be retrieved by going though the tool's vtable
    * __runtime cost__: the concrete tool's ctor stores the trampoline pointers (this could be optimized to be a "once per class" initialisation). Then, for each call, we have 2 virtual function calls and a lookup and call of the trampoline function, typically resulting in another virtual function call for resolving the {{{treat(..)}}} function on the concrete tool class
    * __extension possibilities__: while this system may seem complicated, you should note that it was designed with special focus on extension and implementing against interfaces:
    ** not every Visitable subclass needs a separate Dispatcher. As a rule of thumb, only when some type needs to be treated separately within some concrete visiting tool (i.e. when there is the need of a {{{treat(MySpecialVisitable&)}}}), then this class should use the {{{DEFINE_PROCESSABLE_BY}}}-macro and thus define it's own {{{apply()}}}-function and Dispatcher. In all other cases, it is sufficient just to extend some existing Visitable, which thus will act as an interface with regards to the visiting tools.
    ** because the possibility of utilizing virtual {{{treat(...)}}} functions, not every concrete visiting tool class needs to define a set of {{{Applicable<...>}}} base classes (and thus get a separate dispatcher slot). We need such only for each //unique set// of Applicables. All other concrete tools can extend existing tool implementations, sharing and partially extending the same set of virtual {{{treat()}}}-functions.
    ** when adding a new "first class" Visitable, i.e. a concrete target class that needs to be treated separately in some visiting tool, the user needs to include the {{{DEFINE_PROCESSABLE_BY}}} macro and needs to make sure that all existing "first class" tool implementation classes include the Applicable base class for this new type. In this respect, our implementation is clearly "cyclic". (Generally speaking, the visitor pattern should not be used when the hierarchy of target objects is frequently extended and remoulded). But, when using the typelist facillity to define the Applicable base classes, we'll have one header file defining these collection of Applicables and thus we just need to add our new concrete Visitable to this header and recompile all tool implementation classes.
    ** when creating a new "~Visitable-and-Tool" hierarchy, the user should derive (or typedef) and parametrize the {{{Visitable}}}, {{{Tool}}} and {{{Applicable}}} templates, typically into a new namespace. An example can be seen in {{{proc/mobject/builder/buildertool.hpp}}}

    Using the ''Visitor Design Pattern'' is somewhat controversial, mainly because this pattern is rather complicated, requires certain circumstances to be usefull, and &mdash; especially when used in the original form described by Gamma et al &mdash; puts quite some burden on the implementor. Contrary to some patterns commonly used today (like e.g. singleton or factory), visitor is by no way a popular pattern. Ichthyo thinks that until now it's potential stregths remain still to be discovered, due to obvious and well known weaknesses. This problems can be ameliorated a good deal by using template based library implementation techniques along the lines of the [[Loki library|http://loki-lib.sourceforge.net/]].
    &rarr; [[implementation deatails|VisitingToolImpl]]

    !why bothering with visitor?
    In the Lumiera Proc layer, Ichthyo uses the visitor pattern to overcome another notorious problem when dealing with more complex class hierarchies: either, the //interface// (root class) is so unspecific to be almost useless, or, in spite of having a useful contract, this contract will effectively be broken by some subclasses ("problem of elliptical circles"). Initially, when designing the classes, the problems aren't there (obviously, because they could be taken as design flaws). But then, under the pressure of real features, new types are added later on, which //need to be in this hierarchy// and at the same time //need to have this and that special behaviour// and here we go ...
    Visitor helps us to circumvent this trap: the basic operations can be written against the top level interface, such, that they include visiting some object collection internally. Now, on a case-by-case base, local operations can utilize some sub interface or the given concrete type's public interface. So visitor helps to encapsulate cooperating specific technical details within some concrete visiting tool implementation, while forcing them to be implemented against some interface or sub-interface of the target objects.

    !!well suited for using visitors
    generally speaking, visitors are preferable when the underlying element type hierarchy is rather stable, but new operations are to be added frequently.
    * Implementation of the builder. All sorts of special treatments of some MObject kinds can be added later
    * Operating on the Object collection within the EDL. Effectively, this decouples the invoking interface on the EDL completely from the special operations to be carried out on individual objects.

    To see an simple example of our "visiting tool", have a look at {{{tests/components/common/visitingtooltest.cpp}}}

    The Intention of this text is to help you understanding the design and to show some notable details.

    !!!!Starting Point
    Design is an experiment to find out how things are related. We can't //plan// a Design top down, rather we have to start at some point with some hypothesis and look how it works out. The point of origin for Ichthyo's design is the observation that the Render Engine needs some Separation of Concerns to get the complexity down. And especially, this design ''uses three different Levels'' or Layers within the Render Engine and EDL.
    * the __high level__ within the EDL uses uniformly treated MObjects which are assembled/glued together by a network of [[Placements|Placement]].<br> It is supposed that the GUI will present this and //only this view //to the user, giving him the ability to work with the objects
    * the __builder level__ works on a stripped-down subset of this ~MObject network: it uses the //same Object instances// but only assembled by [[Explicit Placements|ExplicitPlacement]] which locate the objects //on a simple (track, time) grid.// It's the job of the builder to create out of this simplified Network the Configuration of [[Render Nodes|ProcNode]] needed to do the actual rendering
    * the __engine level__ uses solely [[Render Pipeline Nodes (ProcNode)|ProcNode]], i.e. a Graph of interconnected processing nodes. The important constraint here is that //any decisions are ruled out//. The core Render Engine with all its nodes is lacking the ability to do any tests and checks and has no possibility to branch or reconfigure anything. (this is an especially important lesson I draw from studying the current Cinelerra source code)

    !!!!Performance Considerations
    * within the Engine the Render Nodes are containing the ''inner loop'', whose contents are to be executed hundred thousands to million times per frame. Every dispensable concern, which is not strictly necessary to get the job done, is worth the effort of factoring out here.
    * performance pressure at the builder level is far lower, albeit still existent. Compared to the effort of calculating a single processing step, looping even over some hundred nodes and executing quite some logic is negligible. Danger bears on creating memory pressure or causing awkward execution patterns (in the backend) rather. So the main concern should be the ability of reconfiguring different aspects separately without much effort. If for example a given render strategy works out to create lots of deadlocks and waitstates in the backend, the design should account for the possibility to exchange it with another strategy without having to modify the inner workings of the build process.<br>On the other hand, I wouldn't be overly concerned to trigger the build process yet another time to get some specific problem solved. However, the possibility to share one Render configuration for, say, 20 sec of video, instead of triggering the build process 500 times for every frame in this timespan, would sure be worth considering if it's not overly complicated to achieve.
    * contrary to this, the EDL level is harmless with respect to performance. Getting acceptable responsiveness on user interactions is sufficient. We could consider using high level languages here, for it is much more important being able to express and handle complicated object relationships with relative ease. The only (indirect) concern is to avoid generating memory pressure inadvertently. Edit actions generating memory peaks could interfere with an ongoing background render process. If we decide to use lots of relation objects or transient objects, we should use an object pool or still better an garbage collector.

    !!!!Concepts and Interfaces
    This design strives to build each level and subsystem around some central concepts, which are directly expressed as Interfaces. Commonly used Interfaces clamp the different layers.
    * MObject gives an uniform view on all the various entities to be arranged in the EDL.
    * all the arranging and relating of ~MObjects is abstracted as [[Placement]]. The contract of a Placement is that it always has a related Subject, that we can change the //way of placement&nbsp;// by adding and removing [["locating pins"|LocatingPin]], call some test methods on it (still to be defined), and, finally, that we can get an ExplicitPlacement from it.
    * albeit being a special form of a Placement, the ExplicitPlacement is treated as a separate concept. With respect to edit operations within the EDL, it can stand for any sort of Placement. On the other hand the Builder takes a list of ~ExplicitPlacements as input for building up the Render Engine(s). This corresponds to the fact that the render process needs to organize the things to be done on a simple two dimensional grid of (output channel / time). The (extended) contract of an ~ExplicitPlacement provides us with this (output,time).
    * on the lower end of the builder, everything is organized around the Concept of a ProcNode, which enables us to //pull// one (freely addressable) Frame of calculated data. Further, the ProcNode has the ability to be wired with other nodes and [[Parameter Providers|ParamProvider]]
    * the various types of data to be processed are abstracted away under the notion of a [[Frame]]. Basically, a Frame is an Buffer containing an Array of raw data and it can be located by some generic scheme, including (at least) the absolute starting time (and probably some type or channel id).
    * All sorts of (target domain) [[parameters|Parameter]] are treated uniformly. There is a distinction between Parameters (which //could// be variable) and Configuration (which is considered to be fixed). In this context, [[Automation]] just appears as a special kind of ParamProvider.
    * and finally, the calculation //process// together with its current state is represented by a StateProxy. I call this a "proxy", because it should encapsulate and hide all tedious details of communication, be it even asynchronous communication with some Controller or Dispatcher running in another Thread. In order to maintain a view on the current state of the render process, it could eventually be necessary to register as an observer somewhere or to send notifications to other parts of the system.

    !!!!Handling Diversity
    An important goal of this approach is to be able to push down the treatment of variations and special cases. We don't need to know what kind of Placement links one MObject to another, because it is sufficient for us to get an ExplicitPlacement. The Render Engine doesn't need to know if it is pulling audio Frames or video Frames or GOPs or OpenGL textures. It simply relies on the Builder wiring together the correct node types. And the Builder in turn does so by using some overloaded function of an iterator or visitor. At many instances, instead of doing decisions in-code or using hard wired defaults, a system of [[configuration rules|ConfigRules]] is invoked to get a suitable default as a solution (and, as a plus, this provides points of customisation for advanced users). At engine level, there is no need for the video processing node to test for the colormodel on every screen line, because the Builder has already wired up the fitting implementation routine. All of this helps reducing complexity and quite some misconceptions can be detected already by the compiler.

    !!!!Explicit structural differences
    In case it's not already clear: we don't have "the" Render Engine, rather we construct a Render Engine for each structurally differing part of the timeline. (please relate this to the current Cinelerra code base, which constructs and builds up the render pipeline for each frame separately). No need to call back from within the pipeline to find out if a given plugin is enabled or to see if there are any automation keyframes. We don't need to pose any constraints on the structuring of the objects in the EDL, besides the requirement to get an ExplicitPlacement for each. We could even loosen the use of the common metaphor of placing media sequences on fixed tracks, if we want to get at a more advanced GUI at some point in the future.

    !!!!Stateless Subsystems
    The &raquo;current setup&laquo; of the objects in the EDL is sort of a global state. Same holds true for the Controller, as the Engine can be at playback, it can run a background render or scrub single frames. But the whole complicated subsystem of the Builder and one given Render Engine configuration can be made ''stateless''. As a benefit of this we can run this subsystems multi-threaded without the need of any precautions (locking, synchronizing). Because all state information is just passed in as function parameters and lives in local variables on the stack, or is contained in the StateProxy which represents the given render //process// and is passed down as function parameter as well. (note: I use the term "stateless" in the usual, slightly relaxed manner; of course there are some configuration values contained in instance variables of the objects carrying out the calculations, but this values are considered to be constant over the course of the object usage).

    Each [[processing node|ProcNode]] contains a stateless ({{{const}}}) descriptor detailing the inputs, outputs and predecessors. Moreover, this descriptor contains the configuration of the call sequence yielding the &raquo;data pulled from predecessor(s)&laquo;. The actual type of this object is composed out of several building blocks (policy classes) and placed by the builder as a template parameter on the WiringDescriptor of the individual ProcNode. This happens in the WiringFactory in file {{{nodewiring.cpp}}}, which consequently contains all the possible combinations (pre)generated at compile time.

    !building blocks
    * ''Caching'': whether the result frames of this processing step will be communicated to the Cache and thus could be fetched from there instead of actually calculating them.
    * ''Process'': whether this node does any calculations on it's own or just pulls from a source
    * ''Inplace'': whether this node is capable of processing the result "in-place", thereby overwriting the input buffer
    * ''Multiout'': whether this node produces multiple output channels/frames in one processing step

    !!implementation
    !!!!Caching
    When a node participates in ''caching'', a result frame may be pulled immediately from cache instead of calculating it. Moreover, //any output buffer//&nbsp; of this node will be allocated //within the cache.// Consequently, caching interferes with the ability of the next node to calculate "in-Place". In the other case, when ''not using the cache'', the {{{pull()}}} call immediately starts out with calling down to the predecessor nodes, and the allocation of output buffer(s) is always delegated to the parent state (i.e. the StateProxy pulling results from this node).

    Generally, buffer allocation requests from predecessor nodes (while being pulled by this node) will either be satisfied by using the "current state", or treated as if they were our own output buffers when this node is in-Place capable.

    !!!!Multiple Outputs
    Some simplifications are possible in the default case of a node producing just ''one single output'' stream. Otherwise, we'd have to allocate multiple output buffers, and then, after processing, select the one needed as a result and deallocate the superfluous further buffers.

    !!!!in-Place capability
    If a node is capable of calculating the result by ''modifying it's input'' buffer(s), an important performance optimization is possible, because in a chain of in-place capable nodes, we don't need any buffer allocations. But, on the other hand, this optimization may collide with the caching, because a frame retrieved from cache must not be modified.
    Without this optimization, in the base case each processing needs an input and an output. Exceptionally, we could think of special nodes which //require// to process in-place, in which case we'd need to provide it with a copy of the input buffer to work on.

    !!!!Processing
    If ''not processing'' we don't have any input buffers, instead we get our output buffers from an external source.
    Otherwise, in the default case of actually ''processing'' out output, we have to organize input buffers, allocate output buffers, call the {{{process()}}} function of the WiringDescriptor and finally release the input buffers.

    {{red{This is an early draft as of 9/08}}}
    Wiring requests rather belong to the realm of the high-level model, but play an important role in the build process, because the result of "executing" a wiring request will be to establish an actual low-level data connection. Wiring requests will be created automatically in the course of the build process, but they can be created manually and attached to the media objects in the high-level model as a ''wiring plug'', which is a special kind of LocatingPin (&rarr; [[Placement]])

    Wiring requests are small stateful value objects. They will be collected, sorted and processed ordered, such as to finish the building and get a completely wired network of processing nodes. Obviously, there needs to be some reference or smart pointer to the objects to be wired, but this information remains opaque.

    &rarr; ConManager

    The purpose of automation is to vary a parameter of some data processing instance in the course of time while rendering. Thus, automation encompasses all the variability within the render network //which is not a structural change.//

    !Parameters and Automation

    [[Automation]] is treated as a function over time. Everything beyond this definition is considered an implementation detail of the [[parameter provider|ParamProvider]] used to yield the value. Thus automation is closely tied to the concept of a [[Parameter]], which also plays an important role in the communication with the GUI and while [[setting up and wiring the render nodes|BuildRenderNode]] in the course of the build process (&rarr; see [[tag:Builder|Builder]])