benn/lumiera_
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

written a Overview about the data backend

Browse source
This commit is contained in:
Christian Thaeter 2007-06-20 02:42:28 +02:00
parent 5a538b27aa
commit 5d214bbf58
1 changed files with 33 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

36
wiki/backend.html
View file

@ -44,7 +44,7 @@ DAMAGE.
<style type="text/css">#contentWrapper {display:none;}</style><div id="SplashScreen" style="border: 3px solid #ccc; display: block; text-align: center; width: 320px; margin: 100px auto; padding: 50px; color:#000; font-size: 28px; font-family:Tahoma; background-color:#eee;"><b>My TiddlyWiki</b> is loading<blink> ...</blink><br><br><span style="font-size: 14px; color:red;">Requires Javascript.</span></div>
<!--PRE-HEAD-END-->
<title> Cinelerra3 Data Backend - design draft </title>
<title> Data Backend - design draft </title>
<style type="text/css">
#saveTest {display:none;}
#messageArea {display:none;}
@ -764,6 +764,25 @@ TertiaryMid: #999
TertiaryDark: #666
Error: #f88</pre>
</div>
<div title="DataBackend" modifier="CehTeh" modified="200706192335" created="200706192251" changecount="3">
<pre>This just starts as braindump, I will refine it soon:
* handle all files cinelerra uses at runtime (media, edl, temp data)
* manage filehandles, cinelerra might use more more files than available filehandles
* manage temporary data
* do caching
* io will be blocked where the backend tells the core where it can expect the data (not read()/write() like)
* kindof garbage collector
* do prefetching
* no/low latency for the core the prefetcher and other things ensure that data is available in time
* translate any input into a format which the cinelerra core understands (demux, decode)
* same for encoding to output formats
* offer a plugin API for encoders/decoders
* maybe network backend for serving data to distributed rendernodes
* can do some load control or management (trigger adaptive rendering if system is idle etc)
* pull based arch
Look at [[Overview]] for the current design proposal</pre>
</div>
<div title="DefaultTiddlers" modifier="CehTeh" created="200706192002" changecount="1">
<pre>DataBackend
</pre>
@ -1164,6 +1183,17 @@ DataBackend
&lt;style type=&quot;text/css&quot;&gt;#contentWrapper {display:none;}&lt;/style&gt;&lt;div id=&quot;SplashScreen&quot; style=&quot;border: 3px solid #ccc; display: block; text-align: center; width: 320px; margin: 100px auto; padding: 50px; color:#000; font-size: 28px; font-family:Tahoma; background-color:#eee;&quot;&gt;&lt;b&gt;My TiddlyWiki&lt;/b&gt; is loading&lt;blink&gt; ...&lt;/blink&gt;&lt;br&gt;&lt;br&gt;&lt;span style=&quot;font-size: 14px; color:red;&quot;&gt;Requires Javascript.&lt;/span&gt;&lt;/div&gt;</pre>
</div>
<div title="Overview" modifier="CehTeh" created="200706200040" changecount="1">
<pre>Whenever cinelerra needs to access data this is done through the DataBackend described here. The backend uses memory mapping to make data available to the program. This is little different to more common open/read/write/close file access while giving superior performance and much better memory utilization.
The data backend must be capable to handle more data than will fit into the memory or even address space on 32 bit archs. Moreover a project may access more files than the OS can handle at a time, thus the Backend needs a FileHandleCache to manage filehandles dynamically.
Which parts of a File are actually mapped to physical ram is managed by the kernel, but we keep a FileCache to manage the mappings we set up.
To minimize latency and optimize cpu utilization we have a [[Prefetch]] thread which operates a [[Scheduler]] to render and cache frames which are expected to be consumed soon. This prefetcher keeps [[Statistics]] for optimizing performance.
The core accesses data through smart pointers which are described in [[DataLocking]].</pre>
</div>
<div title="PageTemplate" modifier="CehTeh" modified="200706110330" created="200701131624" tags="MPTWTheme" server.type="file" server.host="file:///home/ct/.homepage/home.html" server.page.revision="200706110330">
<pre>&lt;!--{{{--&gt;
&lt;div class='header' macro='gradient vert [[ColorPalette::PrimaryLight]] [[ColorPalette::PrimaryMid]]'&gt;
@ -1969,8 +1999,8 @@ see : &lt;&lt;tag RSSFeed&gt;&gt; for the full list.
<pre>design draft
</pre>
</div>
<div title="SiteTitle" modifier="CehTeh" created="200706192003" changecount="1">
<pre>Cinelerra3 Data Backend</pre>
<div title="SiteTitle" modifier="CehTeh" modified="200706192245" created="200706192003" changecount="2">
<pre>Data Backend</pre>
</div>
<div title="SplashScreenPlugin" modifier="Saq" modified="200607202048" created="200607191631" tags="lewcidExtension systemConfig" server.type="file" server.host="file:///home/ct/.homepage/home.html" server.page.revision="200607202048">
<pre>/***