merge Wiki changes (including IRC Protocol)
This commit is contained in:
commit
5ea8943a01
2 changed files with 98 additions and 31 deletions
|
|
@ -996,7 +996,14 @@ git push git://git.pipapo.org/cinelerra3/mob
|
|||
|
||||
cinelerra3/mob is an anonymous account at pipapo.org where everyone can commit changes. </pre>
|
||||
</div>
|
||||
<div title="IRC-Transcripts" modifier="Ichthyostega" modified="200708170239" created="200708120209" tags="discuss irclog" changecount="4">
|
||||
<div title="IRC-Transcripts" modifier="Ichthyostega" modified="200802021818" created="200708120209" tags="discuss irclog" changecount="7">
|
||||
<pre>We keep a protocol or short summary of each important discussion. The summaries of the monthly developer meetings are posted to the Mailinglist and can be found on pipapo.org too
|
||||
|
||||
* [[1.official developer meeting 1.Feb.2008|IRC_2008-02-01]]
|
||||
* [[informal developer meeting 10.Aug.2007|IRC_2007-08-10]]
|
||||
</pre>
|
||||
</div>
|
||||
<div title="IRC_2007-08-10" modifier="Ichthyostega" created="200802021815" tags="irclog" changecount="1">
|
||||
<pre>!10/11 aug 2007
|
||||
* maybe let the render graph builder generate a meta language which then is ''jit compiled'' for each configuration
|
||||
* using smart pointers and factories for objects ''and avoid unprotected use of {{{new}}} and {{{delete}}}'', if this wont work because of cycles, we might investigate specialized garbage collectors for specific cases.
|
||||
|
|
@ -1025,6 +1032,66 @@ cinelerra3/mob is an anonymous account at pipapo.org where everyone can commit c
|
|||
* on the question who decides what to do after a cache miss, we tend to put the decision into the render node (because this seems to be the simpler aproach), but still have to work out the details.
|
||||
</pre>
|
||||
</div>
|
||||
<div title="IRC_2008-02-01" modifier="Ichthyostega" modified="200802030342" created="200802021821" tags="irclog" changecount="20">
|
||||
<pre>! 1. feb.2008 on #openvideo
|
||||
21:00 -23:30 GMT. __Participants__:
|
||||
* hermanr
|
||||
* cehteh
|
||||
* ichthyo
|
||||
* Dasypus
|
||||
* gmerlin
|
||||
* ~SimAV
|
||||
* pippin
|
||||
* Plough
|
||||
* Plouj
|
||||
* Velmont
|
||||
|
||||
!! Discuss the open points in http://www.pipapo.org/pipawiki/Cinelerra3/DesignProcess &mdash; do we need this formalism?
|
||||
At start of the project last summer, Cehteh made a [[design process proposal|http://www.pipapo.org/pipawiki/Cinelerra3/DesignProcess]]. We will keeping this up, not for every implementation detail, but for mayor plans, wishes and design decisions. One point in the agenda of future meetings will be to work through proposals in the queue
|
||||
* proposals in the "idea" state are not complete, can be just brainstorming or need further discussion. Comments please to the proposal page.
|
||||
* proposals in the "draft" state are ready for conclusive discussion and will be treated in one of the next meetings
|
||||
* "final" proposals are either "accepted" or "dropped". We don't differentiate the latter, but should write a short note why it was dropped.
|
||||
|
||||
!! The development model
|
||||
We employ a distributed model based on GIT. We want this repository to be as complete as possible, including documentation in embedded ~TiddlyWikis and Bug reports. Each dev has its own GIT repo, devs are pulling from each other, they are free to cherry pick and try to make the combined version work. Point is that everyone can clone the git, negotiate with the others what s(he) wants to do, and hack on. Every dev signs off his branch with an standardized signature. For small changes we provide a "Mob GIT", i.e. anonymously pushable git (which is untrusted of course). Cehteh is currently working on a git web frontend which makes the codebase in the mob-repo web-editable like a wiki.
|
||||
Will we need a stable version or an official branch? not yet &mdash; as long as the team is small it will work more painless without. At some point, when the project is more mature, we will define an official branch. Later on we will have automated builds and regression test runs. As we do test-driven development anyways, it's just a question of someone setting up all the infrastructure, then we'll do it.
|
||||
Ichthyo proposes a new requirement: All devs should ensure the "master" branch of their respective repositories always passes the compiler and the test suite. ~Work-In-Progress should be done on branches. Rationale: it is sufficient to pull from the master branches, and you can be sure the version you pulled worked for the originator.
|
||||
A note on dependencies: it will be hard to target minimal dependencies for such a project, but we shall try not bloat it unnecessarily. Sometimes it can be sensible to rather re-invent a feature &mdash; esp. when it fits into the core focus of the project &mdash; instead of depending on difficult to build and not sufficiently maintained external project. But we should avoid reinventing things like GUI toolkits.
|
||||
And, pretty much obvious, we try to stick to modern programming standards. That means, modules have interfaces, interfaces need some (minimal) documentation, and it is not allowed to bypass the interfaces and tangle everything in a wild fashion.
|
||||
Currently, the project can be separated into three layers, which could evolve into sub-projects: Backend, ~Proc-Layer, GUI. For each part, the dev most deeply involved and most knowledgeable will take on the sometimes necessary leadership and have the last word in case of quarrels. Currently, Cehteh cares for the Backend and Ichthyo headed for the ~Proc-Layer. We have postponed any work on the GUI for now and don't participate in GUI toolkit discussions. If there is a dev willing to care for the GUI, collect proposals, care for usability and the users needs and finally code it up, then he will the one to decide what toolkit to use.
|
||||
We plan to make the discussion about GPL v3 a point on the agenda of the next meeting.
|
||||
|
||||
!! Monthly meetings
|
||||
* make it thursday, not friday
|
||||
* time for now 21:00 GMT &mdash; if some (potential) participants have problems with this time, please speak up (maybe alternating times?)
|
||||
* write a short status report for each mayor part //prior// to the meeting (saves us time). Maybe add an TODO list there
|
||||
* go through the open issues for the design process drafts
|
||||
* publish a protocol of each meeting on the (~Cinelerra-CV, ~LibOpenvideo) Mailinglists, in the TiddlyWiki and on pipapo.org
|
||||
* next meeting on first Thursday in March (6.3.2008)
|
||||
|
||||
!! Who works on what, what are the short term goals, what tasks are open?
|
||||
''Ichthyo'' works on the processing layer. Current goal is to get the core of the builder fleshed out. Next goal is to create a clip object (dummy), add it to the EDL, hand the EDL over to the builder and let the builder make the first (preliminary) render nodes. (note: left many details for later).
|
||||
Ichthyo started coding his design draft and things seem to work out as intended. Some Keywords: Have a high-level model and a low level model. The former are the objects in the Session edited by the user, the latter is a network of completely configured render nodes, employing the same pull model as in Cinelerra-2. In between sits the Builder, translating high-level into low-level. This translation is done on demand (not for every frame).
|
||||
Current state in this part is: basic asset manager is done, asset objects (forming a hierarchy) can be created and will be registered automatically, it is possible to create a clip-"~MediaObject" from a media asset (without caring for any details regarding multichannel clips). Some support lib components are written, Session and data holders start up on demand and shutdown cleanly. The test suite is the only interesting executable, and this will remain so for quite some time. Currently writing the first parts of the Builder.
|
||||
Further plans/ideas: Ichthyo is rather determined to embed a Prolog interpreter (YAP Prolog) to handle all sorts of configuration queries in a rule-driven fashion. Things Ichthyo can't do in the near future: caring for session loading/saving serialisation plus storage backends, caring for a DB based implementation of the asset manager and integration with production support software, target the scheduler which will receive any edit operations initiated from the GUI.
|
||||
|
||||
''Cehteh'' is currently working on webgit, which is somewhat related inasmuch it will make small contributions to the mob repository much simpler. Previously he started with some foundation and support facilities. He plans to come back to the Backend implementation in about two weeks. The Backend is intended to handle all media (and even meta-)data as generalized frames. The render nodes network created by the ~Proc-Layer is completely stateless and all data is served from below. While it will be possible to address and access individual data within a frame (e.g. audio samples), frames are the smallest unit for memory and cache management. No plans to use a tiled memory model or to support frames larger than aprox. 20-40% of the available RAM.
|
||||
Cehteh's design plan includes a scheduler to organize the access to the raw data, monitor the timings and prefetch data as needed. This scheduler will be configurable via quality preferences ("need realtime", "need perfect quality"). Further, there will be an elaborate caching scheme trying hard to avoid re-rendering any frames already calculated previously. Temporary data will be backed by files and thus swapped out &mdash; this swapout and size of temporary data is to be monitored and adjusted on load &mdash; and all temporary data is kept as most-recent-used cache discipline. Incoming and outgoing footage shall mostly be handled by using mmaped buffers. The rationale is to avoid unnecessary copy from kernel to user space and wasting memory for an additional in-kernel buffer. Writing via a mmapped buffer is little more tricky; there will be a in-place writing which is used for indices and other precalculated data which needs updates, and the processing layer can query write buffers which are actually a small cache/ring and then comited to the file. Basically, mmapping is a clean solution if you can design for it, and it's portable (posix)
|
||||
Things to do: object serialisation backend is sometime on Cehteh's schedule, but that's ahead and if someone else helps or takes over it would be OK. Even more true for a DB based backend for the asset manager.
|
||||
|
||||
__about multithreading__: since the render nodes are stateless they can be driven in multiple threads (but inter frame dependencies need to be resolved/serialized). Mostly the backend manages threads and does that quite conservatively (compared to Cinelerra-2 which massively creates separate threads for small tasks). Any edit operations initiated from GUI go to a scheduler in the middle layer, which enqueues and effectively serializes operations done to the "media objects" in the high-level model. The editing operations themselves are //not threadsafe // by design, they rely on being scheduled correctly. The builder is triggered from this ~Proc-Layer scheduler and starts in one separate thread, and when done, we swap whole parts of the render nodes network and then the backend can re(start) rendering as needed.
|
||||
|
||||
!!The naming discussion
|
||||
The discussion looks healthy so far... People can add new proposals on the [[pipawiki|http://www.pipapo.org/pipawiki/Cinelerra3/Names]]. intersting names are still coming in, so we should just let the name-choosing game go on for a while. And, btw, we //can// depart from beeing similar to "Cinelerra" ;-)
|
||||
Let's say, we need a person volonteering to guide/moderate the selection, going over the list and scratching inammprobiate ones. Criteria for good names being:
|
||||
* should not be an existing project
|
||||
* should be "googeable"
|
||||
* should not be offensive
|
||||
* should have one of the free top level domains (.net, .org)
|
||||
* should be compatible with educational institutions (sorry, no pr0nedit :)
|
||||
* should not obviously collide with trade marks
|
||||
</pre>
|
||||
</div>
|
||||
<div title="InlineJavaScript" modifier="Jeremy" created="200603090618" tags="systemConfig" server.type="file" server.host="file:///home/ct/.homepage/home.html" server.page.revision="200603090618">
|
||||
<pre>/***
|
||||
''InlineJavascriptPlugin for ~TiddlyWiki version 1.2.x and 2.0''
|
||||
|
|
|
|||
|
|
@ -42,9 +42,9 @@ DAMAGE.
|
|||
<link rel='alternate' type='application/rss+xml' title='RSS' href='index.xml'/>
|
||||
<!--}}}-->
|
||||
|
||||
<style type="text/css">#contentWrapper {display:none;}</style><div id="SplashScreen" style="border: 3px solid #ccc; display: block; text-align: center; width: 320px; margin: 100px auto; padding: 50px; color:#000; font-size: 28px; font-family:Tahoma; background-color:#eee;">loading <b>Cinelerra Renderengine</b> devel doku<blink> ...</blink><br><br><span style="font-size: 14px; color:red;">Requires Javascript.</span></div>
|
||||
<style type="text/css">#contentWrapper {display:none;}</style><div id="SplashScreen" style="border: 3px solid #ccc; display: block; text-align: center; width: 320px; margin: 100px auto; padding: 50px; color:#000; font-size: 28px; font-family:Tahoma; background-color:#eee;">loading <b>Proc-Layer</b> devel doku<blink> ...</blink><br><br><span style="font-size: 14px; color:red;">Requires Javascript.</span></div>
|
||||
<!--PRE-HEAD-END-->
|
||||
<title> Engine - some aspects of Cinelerra-3 design </title>
|
||||
<title> Engine - Building a Render Nodes Network from Objects in the EDL </title>
|
||||
<style type="text/css">
|
||||
#saveTest {display:none;}
|
||||
#messageArea {display:none;}
|
||||
|
|
@ -1551,12 +1551,12 @@ This Design strives to achieve a StrongSeparation between the low-level Structur
|
|||
{{red{let's see if this approach works...}}}
|
||||
</pre>
|
||||
</div>
|
||||
<div title="MarkupPreHead" modifier="Ichthyostega" modified="200708080243" created="200706172303" changecount="1">
|
||||
<div title="MarkupPreHead" modifier="Ichthyostega" modified="200802030405" created="200706172303" changecount="2">
|
||||
<pre><!--{{{-->
|
||||
<link rel='alternate' type='application/rss+xml' title='RSS' href='index.xml'/>
|
||||
<!--}}}-->
|
||||
|
||||
<style type="text/css">#contentWrapper {display:none;}</style><div id="SplashScreen" style="border: 3px solid #ccc; display: block; text-align: center; width: 320px; margin: 100px auto; padding: 50px; color:#000; font-size: 28px; font-family:Tahoma; background-color:#eee;">loading <b>Cinelerra Renderengine</b> devel doku<blink> ...</blink><br><br><span style="font-size: 14px; color:red;">Requires Javascript.</span></div></pre>
|
||||
<style type="text/css">#contentWrapper {display:none;}</style><div id="SplashScreen" style="border: 3px solid #ccc; display: block; text-align: center; width: 320px; margin: 100px auto; padding: 50px; color:#000; font-size: 28px; font-family:Tahoma; background-color:#eee;">loading <b>Proc-Layer</b> devel doku<blink> ...</blink><br><br><span style="font-size: 14px; color:red;">Requires Javascript.</span></div></pre>
|
||||
</div>
|
||||
<div title="MediaAsset" modifier="Ichthyostega" modified="200801101307" created="200709021530" tags="def classes" changecount="5">
|
||||
<pre>The Interface asset::Media is a //key abstraction// It ties together several concepts and enables to deal with them on the interfaces in a uniform manner. Besides, as every Asset kind it belongs rather to the bookkeeping view: an asset::Media holds the specific properties and parametrisation of the media source it stands for. Regarding the __inward interface__ &mdash; as used from within the [[EDL]] or the [[Render Nodes|ProcNode]], it is irrelevant if any given asset::Media object stands for a complete media source, just a clip taken from this source or if a placeholder version of the real media source is used instead.
|
||||
|
|
@ -2412,6 +2412,30 @@ We need to work out guidelines for dealing with operations going on simultaneous
|
|||
* all EditingOperations are not threadsafe intentionally, because they are [[scheduled|ProcLayerScheduler]]
|
||||
</pre>
|
||||
</div>
|
||||
<div title="Proc-Layer and RenderEngine" modifier="Ichthyostega" modified="200802030402" created="200706190056" tags="overview" changecount="1">
|
||||
<pre>The Render Engine is the part of the application doing the actual video calculations. Its operations are guided by the Objects and Parameters edited by the user in [[the EDL|EDL]] and it retrieves the raw audio and video data from the [[Data backend|backend.html]]. Because the inner workings of the Render Engine are closely related to the structures used in the EDL, this design covers [[the aspect of objects placed into the EDL|MObjects]] as well.
|
||||
<<<
|
||||
''Status'': started out as design draft in summer '07, Ichthyo is now in the middle of a //first, draft, prototypical// implementation in C++
|
||||
<<<
|
||||
|
||||
!Summary
|
||||
We have several kinds of "things" organized as [[assets|Asset]] in the AssetManager, like media, clips, effects, codecs, configuration templates. Within the context of the [[EDL]], we can use these as [["Media Objects"|MObjects]] &mdash; especially, we can [[place|Placement]] them in various kinds within the EDL and relative to one another. Basically, this is the [[editing work|EditingOperations]] done by the user.
|
||||
|
||||
Now, from any given configuration within the EDL, we create sort or a frozen- and tied-down snapshot, here called [["Fixture"|Fixture]], containing all currently active ~MObjects, broken down to elementary parts and made explicit if necessary. This Fixture acts as a isolation layer towards the Render Engine. We will hand it over to the [[Builder]], which in turn will transform it into a network of connected [[render nodes|ProcNode]]. This network //is// the [[Render Engine|OverviewRenderEngine]].
|
||||
|
||||
The system is ''open'' inasmuch every part mirrors the structure of corresponding parts in adjacent subsystems, and the transformation of any given structure from one subsystem (e.g. Asset) to another (e.g. Render Engine) is done with minimal "magic". So the whole system should be able to handle completely new structures mostly by adding new configurations and components, without much need of rewriting basic workings.
|
||||
|
||||
!!see also
|
||||
&rarr; [[Overview]] of Subsystems and Components, and DesignGoals
|
||||
&rarr; [[An Introduction|WalkThrough]] discussing the key features
|
||||
&rarr; [[Overview Render Engine|OverviewRenderEngine]]
|
||||
&rarr; BuildProcess and RenderProcess
|
||||
&rarr; [[Two Examples|Examples]] (Object diagrams)
|
||||
&rarr; how [[Automation]] works {{red{to be defined in more detail}}}
|
||||
&rarr; [[Problems|ProblemsTodo]] to be solved and notable [[design decisions|DesignDecisions]]
|
||||
&rarr; [[Implementation Details|ImplementationDetails]] {{red{WIP}}}
|
||||
</pre>
|
||||
</div>
|
||||
<div title="ProcAsset" modifier="Ichthyostega" created="200709221343" tags="def classes" changecount="1">
|
||||
<pre>All Assets of kind asset::Proc represent //processing algorithms// in the bookkeeping view. They enable loading, browsing and maybe even parametrizing all the Effects, Plugins and Codecs available for use within the Cinelerra Session.
|
||||
|
||||
|
|
@ -2702,30 +2726,6 @@ The link between ~MObject and Asset should be {{{const}}}, so the clip can't cha
|
|||
At first sight the link between asset and clip-MO is a simple logical relation between entities, but it is not strictly 1:1 because typical media are [[multichannel|MultichannelMedia]]. Even if the media is compound, there is //only one asset::Clip//, because in the logical view we have only one "clip-thing". On the other hand, in the Session/EDL, we have a compound clip ~MObject comprised of several elementary clip objects, each of which will refer to its own sub-media (channel) within the compound media (and don't forget, this structure can be tree-like)
|
||||
{{red{open question:}}} do the clip-MO's of the individual channels refer directly to asset::Media? does this mean the relation is different from the top level, where we have a relation to a asset::Clip??</pre>
|
||||
</div>
|
||||
<div title="RenderEngine" modifier="Ichthyostega" modified="200801062200" created="200706190056" tags="overview" changecount="45">
|
||||
<pre>The Render Engine is the part of the application doing the actual video calculations. Its operations are guided by the Objects and Parameters edited by the user in [[the EDL|EDL]] and it retrieves the raw audio and video data from the [[Data backend|backend.html]]. Because the inner workings of the Render Engine are closely related to the structures used in the EDL, this design covers [[the aspect of objects placed into the EDL|MObjects]] as well.
|
||||
<<<
|
||||
''Status'': started out as design draft in summer '07, Ichthyo is now in the middle of a //first, draft, prototypical// implementation in C++
|
||||
<<<
|
||||
|
||||
!Summary
|
||||
We have several kinds of "things" organized as [[assets|Asset]] in the AssetManager, like media, clips, effects, codecs, configuration templates. Within the context of the [[EDL]], we can use these as [["Media Objects"|MObjects]] &mdash; especially, we can [[place|Placement]] them in various kinds within the EDL and relative to one another. Basically, this is the [[editing work|EditingOperations]] done by the user.
|
||||
|
||||
Now, from any given configuration within the EDL, we create sort or a frozen- and tied-down snapshot, here called [["Fixture"|Fixture]], containing all currently active ~MObjects, broken down to elementary parts and made explicit if necessary. This Fixture acts as a isolation layer towards the Render Engine. We will hand it over to the [[Builder]], which in turn will transform it into a network of connected [[render nodes|ProcNode]]. This network //is// the [[Render Engine|OverviewRenderEngine]].
|
||||
|
||||
The system is ''open'' inasmuch every part mirrors the structure of corresponding parts in adjacent subsystems, and the transformation of any given structure from one subsystem (e.g. Asset) to another (e.g. Render Engine) is done with minimal "magic". So the whole system should be able to handle completely new structures mostly by adding new configurations and components, without much need of rewriting basic workings.
|
||||
|
||||
!!see also
|
||||
&rarr; [[Overview]] of Subsystems and Components, and DesignGoals
|
||||
&rarr; [[An Introduction|WalkThrough]] discussing the key features
|
||||
&rarr; [[Overview Render Engine|OverviewRenderEngine]]
|
||||
&rarr; BuildProcess and RenderProcess
|
||||
&rarr; [[Two Examples|Examples]] (Object diagrams)
|
||||
&rarr; how [[Automation]] works {{red{to be defined in more detail}}}
|
||||
&rarr; [[Problems|ProblemsTodo]] to be solved and notable [[design decisions|DesignDecisions]]
|
||||
&rarr; [[Implementation Details|ImplementationDetails]] {{red{WIP}}}
|
||||
</pre>
|
||||
</div>
|
||||
<div title="RenderEntities" modifier="Ichthyostega" modified="200706220406" created="200706190715" changecount="6">
|
||||
<pre>The Render Engine only carries out the low-level and performance critical tasks. All configuration and decision concerns are to be handled by [[Builder]] and [[Controller]]. While the actual connection of the Render Nodes can be highly complex, basically each Segment of the Timeline with uniform characteristics is handled by one Processor, which is a graph of [[Processing Nodes|ProcNode]] discharging into a ExitNode. The Render Engine Components as such are //stateless// themselves; for the actual calculations they are combined with a StateProxy object generated by and connected internally to the [[Controller]], while at the same time holding the Data Buffers (Frames) for the actual calculations.
|
||||
|
||||
|
|
@ -2770,8 +2770,8 @@ It will contain a global video and audio outport, just one EDL with one track; t
|
|||
<div title="SideBarOptions" modifier="CehTeh" created="200706200048" changecount="1">
|
||||
<pre><<search>><<closeAll>><<permaview>><<newTiddler>><<saveChanges>><<slider chkSliderOptionsPanel OptionsPanel "options »" "Change TiddlyWiki advanced options">></pre>
|
||||
</div>
|
||||
<div title="SiteSubtitle" modifier="Ichthyostega" created="200706190044" changecount="1">
|
||||
<pre>some aspects of Cinelerra-3 design</pre>
|
||||
<div title="SiteSubtitle" modifier="Ichthyostega" modified="200802030406" created="200706190044" changecount="2">
|
||||
<pre>Building a Render Nodes Network from Objects in the EDL</pre>
|
||||
</div>
|
||||
<div title="SiteTitle" modifier="Ichthyostega" modified="200708080212" created="200706190042" changecount="2">
|
||||
<pre>Engine</pre>
|
||||
|
|
|
|||
Loading…
Reference in a new issue