benn/lumiera_
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

WIP sorting out some details of the node network impl

Browse source
This commit is contained in:
Fischlurch 2008-06-24 05:19:11 +02:00
parent eeb2d04dee
commit da49742826
17 changed files with 448 additions and 309 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

30
src/proc/engine/codecadapter.cpp
View file

@ -1,30 +0,0 @@
/*
CodecAdapter - Processing Node for (de)coding media data
Copyright (C) Lumiera.org
2008, Hermann Vosseler <Ichthyostega@web.de>
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as
published by the Free Software Foundation; either version 2 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
* *****************************************************/
#include "proc/engine/codecadapter.hpp"
namespace engine
{
} // namespace engine

49
src/proc/engine/codecadapter.hpp
View file

@ -1,49 +0,0 @@
/*
CODECADAPTER.hpp - Processing Node for (de)coding media data
Copyright (C) Lumiera.org
2008, Hermann Vosseler <Ichthyostega@web.de>
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as
published by the Free Software Foundation; either version 2 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#ifndef ENGINE_CODECADAPTER_H
#define ENGINE_CODECADAPTER_H
#include "proc/engine/trafo.hpp"
namespace engine
{
/**
* Adapter for wiring an externally loaded
* codec into the render pipeline. The decoding
* process thus is integrated into the framework
* of the Renderengine, while the actual implementation
* can be delegated to an external library accessed
* through Lumiera's data backend.
*/
class CodecAdapter : public Trafo
{};
} // namespace engine
#endif

34
src/proc/engine/glpipe.cpp
View file

@ -1,34 +0,0 @@
/*
GLPipe - specialized connection element for handling OpenGL implementation details
Copyright (C) Lumiera.org
2008, Hermann Vosseler <Ichthyostega@web.de>
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as
published by the Free Software Foundation; either version 2 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
* *****************************************************/
#include "proc/engine/glpipe.hpp"
namespace engine
{
/** */
} // namespace engine

46
src/proc/engine/glpipe.hpp
View file

@ -1,46 +0,0 @@
/*
GLPIPE.hpp - specialized connection element for handling OpenGL implementation details
Copyright (C) Lumiera.org
2008, Hermann Vosseler <Ichthyostega@web.de>
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as
published by the Free Software Foundation; either version 2 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#ifndef ENGINE_GLPIPE_H
#define ENGINE_GLPIPE_H
#include "proc/engine/link.hpp"
namespace engine
{
/**
* specialized connection node used to handle the transfer of OpenGL data
* from a image bitmap into texture form.
*/
class GLPipe : public Link
{};
} // namespace engine
#endif

34
src/proc/engine/hub.cpp
View file

@ -1,34 +0,0 @@
/*
Hub - special ProcNode used to build data distributing connections
Copyright (C) Lumiera.org
2008, Hermann Vosseler <Ichthyostega@web.de>
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as
published by the Free Software Foundation; either version 2 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
* *****************************************************/
#include "proc/engine/hub.hpp"
namespace engine
{
/** */
} // namespace engine

44
src/proc/engine/hub.hpp
View file

@ -1,44 +0,0 @@
/*
HUB.hpp - special ProcNode used to build data distributing connections
Copyright (C) Lumiera.org
2008, Hermann Vosseler <Ichthyostega@web.de>
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as
published by the Free Software Foundation; either version 2 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#ifndef ENGINE_HUB_H
#define ENGINE_HUB_H
#include "proc/engine/procnode.hpp"
namespace engine
{
class Hub : public ProcNode
{};
///TODO: seems rather we don't need this one anymore??
//////// to be decided after the implementing the basic render engine....
} // namespace engine
#endif

4
src/proc/engine/link.hpp
View file

@ -40,6 +40,10 @@ namespace engine
*/
class Link : public ProcNode
{
Link (WiringDescriptor const& wd)
: ProcNode(wd)
{ }
///////TODO: flatten hierarchy if we don't have common functionallity for all Link-Subclasses
///////TODO: can be decdiede when we know more how to handle OpenGL rendering
};

72
src/proc/engine/nodeoperation.hpp Normal file
View file

@ -0,0 +1,72 @@
/*
NODEOPERATION.hpp - Specify how the nodes call each other and how processing is organized
Copyright (C) Lumiera.org
2008, Hermann Vosseler <Ichthyostega@web.de>
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as
published by the Free Software Foundation; either version 2 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#ifndef ENGINE_NODEOPERATION_H
#define ENGINE_NODEOPERATION_H
#include "proc/engine/procnode.hpp"
namespace engine {
/**
* Adapter to shield the ProcNode from the actual buffer management,
* allowing the processing function within ProcNode to use logical
* buffer IDs. StateAdapter is created on the stack for each pull()
* call, using setup/wiring data preconfigured by the builder.
* Its job is to provide the actual implementation of the Cache
* push / fetch and recursive downcall to render the source frames.
*/
class StateAdapter
: public State
{
State& parent_;
State& current_;
protected:
StateAdapter (State& callingProcess, WiringDescriptor const&)
: parent_ (callingProcess),
current_(callingProcess.getCurrentImplementation())
{ }
friend class ProcNode; // both are sharing implementation details...
virtual State& getCurrentImplementation () { return current_; }
/** contains the details of Cache query and recursive calls
* to the predecessor node(s), eventually followed by the
* ProcNode::process() callback
*/
void retrieve();
};
} // namespace engine
#endif

49
src/proc/engine/nodewiring.cpp Normal file
View file

@ -0,0 +1,49 @@
/*
NodeWiring - Implementation of the node network and operation control
Copyright (C) Lumiera.org
2008, Hermann Vosseler <Ichthyostega@web.de>
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as
published by the Free Software Foundation; either version 2 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
* *****************************************************/
#include "proc/engine/procnode.hpp"
#include "proc/engine/nodewiring.hpp"
#include "proc/engine/nodeoperation.hpp"
namespace engine {
/** create and configure a concrete wiring descriptor to tie
* a ProcNode to its predecessor nodes. This includes selecting
* the actual StateProxy type, configuring it out of some operation
* control templates (policy classes). Compiling this operator function
* actually drives the necessary template instantiations for all cases
* encountered while building the node network.
* The created WiringDescriptor objects are bulk allocated similar to the ProcNode
* objects for a given segment of ther Timeline. It should be further configured
* with the actual predecessor nodes pointers and can then be used to create
* the new processing node to be wired up.
*/
WiringDescriptor&
WiringFactory::operator() (uint nrOut, uint nrIn, bool cache)
{
UNIMPLEMENTED ("build the actual wiring descriptor based on given operation options");
}
} // namespace engine

77
src/proc/engine/nodewiring.hpp Normal file
View file

@ -0,0 +1,77 @@
/*
NODEWIRING.hpp - Implementation of the node network and operation control
Copyright (C) Lumiera.org
2008, Hermann Vosseler <Ichthyostega@web.de>
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as
published by the Free Software Foundation; either version 2 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
#ifndef ENGINE_NODEWIRING_H
#define ENGINE_NODEWIRING_H
#include "proc/engine/procnode.hpp"
#include <cstddef>
namespace engine {
class WiringFactory;
/**
* Actual implementation of the link between nodes
* and also the track switch for the execution path
* while operating the node network for rendering.
* @param STATE StateAdapter object controlling the
* behaviour of callDown() while rendering.
* @see NodeFactory
*/
template<class STATE>
class NodeWiring
: public WiringDescriptor
{
friend class WiringFactory;
protected:
virtual BufferID
callDown (State& currentProcess, BufferID requiredOutputNr) const
{
STATE thisStep (currentProcess, *this);
return thisStep.retrieve (requiredOutputNr); // fetch or calculate results
}
};
class WiringFactory
{
public:
WiringDescriptor&
operator() (uint nrOut, uint nrIn, bool cache);
};
} // namespace engine
#endif

100
src/proc/engine/procnode.hpp
View file

@ -20,15 +20,33 @@
*/
/** @file procnode.hpp
** Interface to the processing nodes and the render nodes network.
**
** Actually, there are three different interfaces to consider
** - the ProcNode#pull is the invocation interface. It is call-style (easily callable by C)
** - the builder interface, comprised by the NodeFactory and the WiringFactory. It's C++ (using templates)
** - the actual processing function is supposed to be a C function; it uses a set of C functions
** for accessing the frame buffers with the data to be processed.
**
** By using the builder interface, concrete node and wiring descriptor classes are created,
** based on some templates. These concrete classes form the "glue" to tie the node network
** together and contain much of the operation beahviour in a hard wired fashion.
**
** @see nodefactory.hpp
** @see operationpoint.hpp
*/
#ifndef ENGINE_PROCNODE_H
#define ENGINE_PROCNODE_H
#include <vector>
#include "pre.hpp"
#include "proc/lumiera.hpp"
#include "proc/state.hpp"
#include "proc/mobject/parameter.hpp"
#include <vector>
@ -47,45 +65,21 @@ namespace engine {
* Description of the input and output ports and the
* predecessor nodes for a given ProcNode.
*/
struct WiringDescriptor
class WiringDescriptor
{
};
/**
* Adapter to shield the ProcNode from the actual buffer management,
* allowing the processing function within ProcNode to use logical
* buffer IDs. StateAdapter is created on the stack for each pull()
* call, using setup/wiring data preconfigured by the builder.
* Its job is to provide the actual implementation of the Cache
* push / fetch and recursive downcall to render the source frames.
*/
class StateAdapter
: public State
{
State& parent_;
State& current_;
public:
virtual ~WiringDescriptor() {}
protected:
StateAdapter (State& callingProcess, WiringDescriptor const&)
: parent_ (callingProcess),
current_(callingProcess.getCurrentImplementation())
{ }
virtual BufferID callDown (State& currentProcess, BufferID requiredOutputNr) const =0;
friend class ProcNode; // both are sharing implementation details...
friend class ProcNode;
virtual State& getCurrentImplementation () { return current_; }
/** contains the details of Cache query and recursive calls
* to the predecessor node(s), eventually followed by the
* ProcNode::process() callback
*/
void retrieve();
};
/**
* Key abstraction of the Render Engine: A Data processing Node
*/
@ -93,45 +87,39 @@ namespace engine {
{
typedef mobject::Parameter<double> Param; //////TODO: just a placeholder for automation as of 6/2008
vector<Param> params;
const WiringDescriptor& wiringConfig_;
protected:
ProcNode();
ProcNode (WiringDescriptor const& wd)
: wiringConfig_(wd)
{ }
virtual ~ProcNode() {};
friend class NodeFactory;
/** Callback doing the actual calculations
* after input / output buffers are ready
* @internal dispatch to implementation.
* Client code should use #pull()
*/
virtual void process(State&) = 0;
friend void StateAdapter::retrieve(); // to issue the process() callback if necessary
/** Extension point for subclasses to modify the input sources
* according to automation data and other state dependent properties.
*/
virtual WiringDescriptor const& getWiring (State&);
public:
static NodeFactory create;
/** Engine Core operation: render and pull output from this node.
* On return, currentProcess will hold onto output buffer(s)
* On return, currentProcess will hold onto output buffer
* containing the calculated result frames.
* @param currentProcess the current processing state for
* managing buffers and accessing current parameter values
* @param requiredOutputNr the output channel requested (in case
* this node delivers more than one output channel)
* @return ID of the result buffer (accessible via currentProcess)
*/
void
pull (State& currentProcess)
BufferID
pull (State& currentProcess, BufferID requiredOutputNr=0)
{
StateAdapter thisStep (currentProcess, getWiring (currentProcess));
thisStep.retrieve(); // fetch or calculate results
return this->wiringConfig_.callDown (currentProcess, requiredOutputNr);
}
};
} // namespace engine
#endif

2
src/proc/engine/projector.hpp
View file

@ -37,7 +37,7 @@ namespace engine
*/
class Projector : public Trafo
{
////////////
////////////TODO adapt ctor
};

4
src/proc/engine/source.hpp
View file

@ -40,7 +40,9 @@ namespace engine
*/
class Source : public ProcNode
{
/////////////////TODO
Source (WiringDescriptor const& wd)
: ProcNode(wd)
{ }
};

4
src/proc/engine/trafo.hpp
View file

@ -43,7 +43,9 @@ namespace engine
class Trafo : public ProcNode
{
protected:
Trafo() : ProcNode() { };
Trafo (WiringDescriptor const& wd)
: ProcNode(wd)
{ }
friend class NodeFactory;

22
src/proc/state.hpp
View file

@ -25,13 +25,20 @@
#define PROC_INTERFACE_STATE_H
#include "pre.hpp"
namespace engine { class StateAdapter; }
#include "proc/lumiera.hpp"
namespace proc_interface
{
#include <cstddef>
namespace engine {
class StateAdapter;
typedef std::size_t BufferID;
class State
{
protected:
@ -50,7 +57,14 @@ namespace proc_interface
public:
};
} // namespace engine
namespace proc_interface {
using engine::State;
} // namespace proc_interface

137
wiki/index.html
View file

@ -930,14 +930,16 @@ git push git://git.pipapo.org/lumiera/mob
lumiera/mob is an anonymous account at pipapo.org where everyone can commit changes. </pre>
</div>
<div title="IRC-Transcripts" modifier="Ichthyostega" modified="200804120322" created="200708120209" tags="discuss irclog" changecount="8">
<div title="IRC-Transcripts" modifier="Ichthyostega" modified="200806211625" created="200708120209" tags="discuss irclog" changecount="10">
<pre>We keep a protocol or short summary of each important discussion. The summaries of the monthly developer meetings are posted to the Mailinglist and can be found on pipapo.org too
* [[04-08 developer meeting 3.Apr.2008|IRC_2008-04-03]]
* [[03-08 developer meeting 6.Mar.2008|IRC_2008-03-06]]
* [[1.official developer meeting 1.Feb.2008|IRC_2008-02-01]]
* [[informal developer meeting 10.Aug.2007|IRC_2007-08-10]]
</pre>
!talks about distinct topics
* [[buffer allocation for render nodes (6/08)|IRC_log_BufferAllocation_2008-06]]</pre>
</div>
<div title="IRC_2007-08-10" modifier="Ichthyostega" created="200802021815" tags="irclog" changecount="1">
<pre>!10/11 aug 2007
@ -1448,6 +1450,137 @@ The next meeting will be at thuesday 8th may 21:00GMT
</pre>
</div>
<div title="IRC_log_BufferAllocation_2008-06" modifier="Ichthyostega" created="200806211624" tags="irclog excludeMissing" changecount="1">
<pre>! 5.June 2008 on #lumiera
__cehteh__ and __ichthyo__
{{{
[2008-06-05 19:21:21] &lt;cehteh&gt; do you need me? .. i am away if not .. i have no much to say for the today meeting either
[2008-06-05 19:21:57] &lt;ichthyo&gt; I have one topic I want to discuss with you, cehteh
[2008-06-05 19:22:07] &lt;ichthyo&gt; but it need not be now, or in the meeting
[2008-06-05 19:22:07] &lt;cehteh&gt; ok
[2008-06-05 19:22:24] &lt;ichthyo&gt; it's about allocating processing buffers
[2008-06-05 19:23:20] &lt;cehteh&gt; the backend will provide them as temporary files/cyclic buffers
[2008-06-05 19:23:52] &lt;cehteh&gt; its not there yet but in my mind
[2008-06-05 19:24:11] &lt;ichthyo&gt; haha, same for the builder... it's mostly just in my mind
[2008-06-05 19:24:42] &lt;cehteh&gt; opened almost like a normal file but giving frame properties instead filename
[2008-06-05 19:24:56] &lt;cehteh&gt; (there will be a distinct api for that)
[2008-06-05 19:25:01] &lt;ichthyo&gt; ok
[2008-06-05 19:25:05] &lt;ichthyo&gt; regarding the buffers: my question is more special
[2008-06-05 19:25:19] &lt;ichthyo&gt; if you want to cache a frame (intermediary result)
[2008-06-05 19:25:35] &lt;ichthyo&gt; then I thought we could avoid the copy operation
[2008-06-05 19:25:44] &lt;ichthyo&gt; I could arrange things accordingly
[2008-06-05 19:25:46] &lt;cehteh&gt; then the backend creates a backing mmaped file for that maybe manages its size (or do you want to tell how much frames you want to cache?)
[2008-06-05 19:26:06] &lt;cehteh&gt; yes perfect .. thats what i am planning
[2008-06-05 19:26:26] &lt;ichthyo&gt; you know, many processing nodes will be able to process &quot;in place&quot;
[2008-06-05 19:26:34] &lt;ichthyo&gt; but ther are some that can't do this
[2008-06-05 19:27:44] &lt;ichthyo&gt; so basically each processing function will &quot;see&quot; input frame buffer(s) and output frame buffer(s). but when a node is &quot;in-place-capable&quot;, actually the in and out buffer may point to the same location
[2008-06-05 19:26:55] &lt;cehteh&gt; basiically all temporary frames with the same properties are allcoated from the same backing file (well maybe 2 caching levels but not important here)
[2008-06-05 19:27:06] &lt;cehteh&gt; the index give them meaning
[2008-06-05 19:27:49] &lt;cehteh&gt; do you can query (and lock) a frame by that
[2008-06-05 19:28:04] &lt;ichthyo&gt; that sounds good
[2008-06-05 19:28:24] &lt;cehteh&gt; do some inplace editing and then tell the backend &quot;this is now frame N of node X&quot;
[2008-06-05 19:28:34] &lt;ichthyo&gt; yeah, exactly
[2008-06-05 19:28:44] &lt;cehteh&gt; (plus a uuid or preferably genertion number)
[2008-06-05 19:28:39] &lt;ichthyo&gt; basically my Idea was as follows:
[2008-06-05 19:29:47] &lt;ichthyo&gt; when I know the result will be cached, I'll let the node process into the location of the cache frame, and any node which will /use/ this frame as an input will be wired such that it isn't allowed to modify this frame (which is supposed to be located in the cache)
[2008-06-05 19:30:45] &lt;cehteh&gt; yes thats managed in the index
[2008-06-05 19:30:48] &lt;ichthyo&gt; for this to work, I need to &quot;allocate and lock&quot; a location in the cache, and release it when it contains the final processed result
[2008-06-05 19:31:17] &lt;cehteh&gt; well i try to make no locks there, you query frames which are uniquely identified
[2008-06-05 19:31:44] &lt;ichthyo&gt; it doesnt need to be a &quot;lock
[2008-06-05 19:31:55] &lt;ichthyo&gt; just some way to tell that this frame is &quot;under construction&quot;
[2008-06-05 19:32:01] &lt;cehteh&gt; yes
[2008-06-05 19:32:14] &lt;cehteh&gt; we thinking the same :)
[2008-06-05 19:32:57] &lt;cehteh&gt; well problem is when shortly after that another node queries the source frame which got destructed .. thats something the builder needs to avoid, nothing i can do then
[2008-06-05 19:33:21] &lt;ichthyo&gt; yes, builder will care for that
[2008-06-05 19:33:41] &lt;ichthyo&gt; I can even tell in advance the maximum number of temporary buffers I need
[2008-06-05 19:33:51] &lt;cehteh&gt; and it not only needs a under-construcion
[2008-06-05 19:34:44] &lt;cehteh&gt; there are 2 indices one for source and one for destination
[2008-06-05 19:35:11] &lt;cehteh&gt; indices will have a pointer to the 'job' working on it i tihnk
[2008-06-05 19:35:24] &lt;ichthyo&gt; ok
[2008-06-05 19:35:48] &lt;cehteh&gt; (actually there is a list planed many jobs can read-share an frame)
[2008-06-05 19:36:40] &lt;cehteh&gt; i would like if I do not need to do *any* checks in that direction ... means the builder delivers clean jobs which dont step on their own foot
[2008-06-05 19:36:54] &lt;ichthyo&gt; my understanding too
[2008-06-05 19:37:07] &lt;ichthyo&gt; also, I want the nodes to be freed of any checks
[2008-06-05 19:37:33] &lt;ichthyo&gt; so they can assume they get a valid buffer pointer to the right sort of buffer
[2008-06-05 19:37:54] &lt;cehteh&gt; yes thats guranteed
[2008-06-05 19:37:41] &lt;cehteh&gt; if it fails then we get some hairy bugs ... but checking job dependencies afterwards is costly
[2008-06-05 19:38:53] &lt;ichthyo&gt; I mean -- I want to prepare everything as much as possible while building, so that all that needs to be &quot;filled in&quot; when starting the processing are the actual buffer locations
[2008-06-05 19:38:58] &lt;cehteh&gt; *thinkin* prolly easier than it looks
[2008-06-05 19:39:10] &lt;cehteh&gt; exaxctly
[2008-06-05 19:39:50] &lt;ichthyo&gt; we can't prepare everything, because, some nodes may include a variable ammount of source frames
[2008-06-05 19:40:11] &lt;ichthyo&gt; and this number can depend on automation. Classic example is the &quot;time average&quot; video effect
[2008-06-05 19:39:53] &lt;cehteh&gt; problem is only when the builder generates one node which takes X and in-place generates Y from it
[2008-06-05 19:40:10] &lt;cehteh&gt; and you have a 2nd node which takes X too
[2008-06-05 19:40:23] &lt;cehteh&gt; this needs to be serialized somehow
[2008-06-05 19:41:12] &lt;ichthyo&gt; regarding the problem you mention: I want to exclude/avoid this situation already when building
[2008-06-05 19:41:35] &lt;cehteh&gt; yes thats what i was thinking .. thats easiest addressed there
[2008-06-05 19:41:46] &lt;cehteh&gt; and a very ugly bug when it fails :(
[2008-06-05 19:41:42] &lt;ichthyo&gt; when a node puts its result into a frame located within the cache
[2008-06-05 19:41:59] &lt;ichthyo&gt; this frame is treated as if it is read-only
[2008-06-05 19:42:21] &lt;ichthyo&gt; no node depending on this frame will be wired in a way that allows &quot;in-place&quot;
[2008-06-05 19:42:33] &lt;cehteh&gt; about the time averaging: i plan to hint some dependencies so you can ask for &quot;maybe&quot; or by some priority depending on quality/whatever
[2008-06-05 19:43:19] &lt;cehteh&gt; where quality should be runtime adjusted by the profiler
[2008-06-05 19:43:39] &lt;ichthyo&gt; the problem with time averaging is: we don't know how much frames will be averaged at build time, because that's a automatable effect parameter
[2008-06-05 19:43:47] &lt;cehteh&gt; thats a case where a dependencie might not be fullfilled .. but only on request
[2008-06-05 19:44:08] &lt;ichthyo&gt; but the &quot;quality&quot; thing sounds like a good idea
[2008-06-05 19:44:17] &lt;cehteh&gt; yes then split that rendering into 3 passes?
[2008-06-05 19:44:27] &lt;cehteh&gt; 1st pass : building the graph
[2008-06-05 19:44:53] &lt;cehteh&gt; 2nd pass: determine dependencies (by inspecting automation)
[2008-06-05 19:45:00] &lt;cehteh&gt; 3rd pass: do the render
[2008-06-05 19:45:10] &lt;ichthyo&gt; maybe?
[2008-06-05 19:45:17] &lt;ichthyo&gt; 1st pass of course is clear
[2008-06-05 19:45:25] &lt;cehteh&gt; well do you have a better idea?
[2008-06-05 19:45:49] &lt;cehteh&gt; so far i thought about 2 pass where the dependency analysis was part of the builder
[2008-06-05 19:46:02] &lt;ichthyo&gt; agreed
[2008-06-05 19:46:28] &lt;cehteh&gt; but making it three pass shouldnt be a problem or?
[2008-06-05 19:46:51] &lt;ichthyo&gt; I also thought after having finished the raw graph, I'll do some configuration calldown, e.g. to determine the maximum number of buffers needed
[2008-06-05 19:46:59] &lt;cehteh&gt; maybe even adaptive 2/3 pass depending on effects ... but maybe that would complicate it unnecessary
[2008-06-05 19:47:43] &lt;ichthyo&gt; its not really a problem, just need to be aware and work the details out correctlyy
[2008-06-05 19:47:49] &lt;cehteh&gt; i dont think you need to count the buffers (not yet, maybe i oversee soemthing)
[2008-06-05 19:48:08] &lt;cehteh&gt; otherwise the backend should manage that automatically
[2008-06-05 19:48:18] &lt;ichthyo&gt; yes, I'm fine with that
[2008-06-05 19:48:46] &lt;ichthyo&gt; just to note, if it helps with the allocation, I /can/ tell the maximum number of buffers needed
[2008-06-05 19:49:00] &lt;ichthyo&gt; (for a given segment of the graph, of course)
[2008-06-05 19:49:09] &lt;cehteh&gt; not sure if i overseen something, but i think when needed such a 'configuration calldown' (and optimizing pass) could be added later
[2008-06-05 19:49:27] &lt;ichthyo&gt; sure
[2008-06-05 19:49:50] &lt;cehteh&gt; buffers itself doesnt cost .. but if you really need a lot it will cause IO
[2008-06-05 19:49:57] &lt;cehteh&gt; but that unavoidable anyways
[2008-06-05 19:50:05] &lt;ichthyo&gt; no no
[2008-06-05 19:50:19] &lt;cehteh&gt; yes yes
[2008-06-05 19:50:20] &lt;ichthyo&gt; I'll try to minimize buffer use as much as possible
[2008-06-05 19:50:24] &lt;cehteh&gt; :)
[2008-06-05 19:50:30] &lt;cehteh&gt; of course
[2008-06-05 19:51:00] &lt;cehteh&gt; but there are always some limits on finite machines
[2008-06-05 19:51:26] &lt;ichthyo&gt; the problematic case are usual rather rare corner cases, but it should handle those flawless, of course
[2008-06-05 19:51:36] &lt;ichthyo&gt; similar for the size of the buffers
[2008-06-05 19:51:46] &lt;ichthyo&gt; I dont want Lumiera to clip the image
[2008-06-05 19:51:49] &lt;ichthyo&gt; as cinelerra does
[2008-06-05 19:52:04] &lt;ichthyo&gt; e.g. when using the motion tracker
[2008-06-05 19:52:20] &lt;cehteh&gt; well if you need safe-regions around you need to use biggier frames
[2008-06-05 19:52:31] &lt;cehteh&gt; but memory requirements grow quadratic!
[2008-06-05 19:52:46] &lt;ichthyo&gt; the user should NEVER need to set up the processing buffer size, as is necessary in cinelerra
[2008-06-05 19:52:59] &lt;cehteh&gt; yes agreed
[2008-06-05 19:53:15] &lt;ichthyo&gt; I don't think we need safe-regions allways,
[2008-06-05 19:53:31] &lt;ichthyo&gt; but, as e.g. the motion tracking will create automation data in Lumiera
[2008-06-05 19:53:38] &lt;cehteh&gt; yes only for some effect .. motion tracker doesnt need it
[2008-06-05 19:53:50] &lt;ichthyo&gt; it means the actual buffer size depends on automation data for a given frame
[2008-06-05 19:53:56] &lt;cehteh&gt; that just xyr transformation
[2008-06-05 19:54:35] &lt;cehteh&gt; a blur is something which might require a small safe-region to bleed over the edges
[2008-06-05 19:54:43] &lt;ichthyo&gt; yes, you are right
[2008-06-05 19:54:52] &lt;ichthyo&gt; if the motion tracker is wired intelligently
[2008-06-05 19:55:05] &lt;ichthyo&gt; it doesn't need to move anyting itself
[2008-06-05 19:55:18] &lt;cehteh&gt; mapping the motion tracker transformation to a destionation might need safe regions
[2008-06-05 19:55:30] &lt;cehteh&gt; (in case of rotations)
[2008-06-05 19:55:36] &lt;ichthyo&gt; yes agreed
[2008-06-05 19:56:16] &lt;cehteh&gt; well temp buffers will be allcocated for frames of the same 'class' i aleready saied that
[2008-06-05 19:56:50] &lt;cehteh&gt; and i will group these classes so that some similar sizes fall in the same class
[2008-06-05 19:57:19] &lt;cehteh&gt; that might waste some memory on disk at the end of each frame .. but that doesnt need to be mapped in
[2008-06-05 19:57:32] &lt;cehteh&gt; the kernel will do that automatically for us
[2008-06-05 19:57:37] &lt;ichthyo&gt; :)
[2008-06-05 19:57:59] &lt;cehteh&gt; well mostly .. i need to whip it with the right hints ;)
[2008-06-05 19:58:10] &lt;ichthyo&gt; :-o
}}}</pre>
</div>
<div title="InlineJavaScript" modifier="Jeremy" created="200603090618" tags="systemConfig" server.type="file" server.host="file:///home/ct/.homepage/home.html" server.page.revision="200603090618">
<pre>/***
''InlineJavascriptPlugin for ~TiddlyWiki version 1.2.x and 2.0''

49
wiki/renderengine.html
View file

@ -881,7 +881,7 @@ While building, the application of such a visiting tool (especially the [[NodeCr
</pre>
</div>
<div title="BuilderToolKit" modifier="Ichthyostega" modified="200806030202" created="200805210308" tags="impl Builder" changecount="11">
<div title="BuilderToolKit" modifier="Ichthyostega" modified="200806211539" created="200805210308" tags="impl Builder" changecount="12">
<pre>Besides the primary working tool within the builder (namely the [[Node Creator Tool|PlanningNodeCreatorTool]]), on a lower level, we encounter several [[elementary building situations|BuilderPrimitives]] &amp;mdash; and for each of these elementary situations we can retrieve a suitable &quot;fitting tool&quot; or [[mould|BuilderMould]]. The palette of these moulds is called the ''tool kit'' of the builder. It is subject to configuration by rules.
@ -895,7 +895,7 @@ As each mould kind is different, it has a {{{prepare(...)}}} function with suita
When {{{operate()}}} doesn't throw, the result is a list of //successor moulds// &amp;mdash; you shouldn't use the original mould after triggering its operation, because it may have been retracted as a result and reused for another purpose by the tool factory. It is not necessary to store these resulting moulds either (as they can be retrieved as described above), but they can be used right away for the next building step if applicable. In the state they are returned from a successful building step (mould operation = execution of a contained [[processing pattern|ProcPatt]]), they are usually already holding a reference to the part of the network just created and need to be configured only with the next high-level object (effect, placement, pipe, processing pattern or similar, depending on the concrete situation) in order to carry out the next step.
!!single connection step
at the lowest level within the builder there is the step of building a //connection.// This step is executed by the processing pattern with the help of the mould. Actually, making such a connection is more complicated, because in the standard case it will connect N media streams simultaneously (N=2 for stereo sound or 3D video, N=6 for 5.1 Surround, N=9 for 2nd order Ambisonics). These details are encapsulated within the OperationPoint, which is provides by the mould and exhibits a common interface for the processing pattern to express the connecting operation.
at the lowest level within the builder there is the step of building a //connection.// This step is executed by the processing pattern with the help of the mould. Actually, making such a connection is more complicated, because in the standard case it will connect N media streams simultaneously (N=2 for stereo sound or 3D video, N=6 for 5.1 Surround, N=9 for 2nd order Ambisonics). These details are encapsulated within the OperationPoint, which is provided by the mould and exhibits a common interface for the processing pattern to express the connecting operation.
&amp;rarr;see also: BuilderPrimitives for the elementary working situations corresponding to each of these [[builder moulds|BuilderMould]]
</pre>
@ -1839,7 +1839,7 @@ So, when creating a clip out of such a compound media asset, the clip has to be
<div title="NodeCreaterTool" modifier="Ichthyostega" created="200712100626" tags="def" changecount="1">
<pre>NodeCreaterTool is a [[visiting tool|VisitorUse]] used as second step in the [[Builder]]. Starting out from a [[Fixture]], the builder first [[divides the Timeline into segments|SegmentationTool]] and then processes each segment with the NodeCreaterTool to build a render nodes network (Render Engine) for this part of the timeline. While visiting individual Objects and Placements, the NodeCreaterTool creates and wires the necessary [[nodes|ProcNode]]</pre>
</div>
<div title="NodeOperationProtocol" modifier="Ichthyostega" modified="200806140024" created="200806010251" tags="Rendering dynamic" changecount="10">
<div title="NodeOperationProtocol" modifier="Ichthyostega" modified="200806220134" created="200806010251" tags="Rendering dynamic" changecount="11">
<pre>The [[nodes|ProcNode]] are wired to form a &quot;Directed Acyclic Graph&quot;; each node knows its predecessor(s), but not its successor(s). The RenderProcess is organized according to the ''pull principle'', thus we find an operation {{{pull()}}} at the core of this process. There is no such thing as an &quot;engine object&quot; calling nodes iteratively or table driven, rather, the nodes themselves issue recursive calls to their predecessor(s). For this to work, we need the nodes to adhere to a specific protocol:
# Node is pulled, with a StateProxy object as parameter (encapsulating the access to the frames or buffers)
# Node may now access current parameter values, using the state accessible via the StateProxy
@ -1854,6 +1854,9 @@ some points to note:
* when a node is &quot;inplace-capable&quot;, input and output buffer may actually point to the same location
* but there is no guarantee for this to happen, because the cache may be involved (and we can't overwrite the contents of a cache frame)
* generally, a node may have N inputs and M output frames, which are expected to be processed in a single call
&amp;rarr; the [[&quot;mechanics&quot; of the render process|RenderMechanics]]
&amp;rarr; more fine grained [[implementation details|RenderImplDetails]]
</pre>
</div>
<div title="ObjectCreation" modifier="Ichthyostega" modified="200806030153" created="200709030139" tags="impl design" changecount="18">
@ -2731,7 +2734,7 @@ The system is ''open'' inasmuch every part mirrors the structure of correspondin
&amp;rarr; [[Implementation Details|ImplementationDetails]] {{red{WIP}}}
</pre>
</div>
<div title="ProcNode" modifier="Ichthyostega" modified="200805280214" created="200706220409" tags="def spec" changecount="5">
<div title="ProcNode" modifier="Ichthyostega" modified="200806211606" created="200706220409" tags="def spec" changecount="6">
<pre>A data processing node within the Render Engine. Its key feature is the possibility to pull from it one (freely addressable) [[Frame]] of calculated data. Further, each ~ProcNode has the ability to be wired with other nodes and [[Parameter Providers|ParamProvider]]
!! {{red{open questions}}}
@ -2742,6 +2745,7 @@ The system is ''open'' inasmuch every part mirrors the structure of correspondin
* how to define and query for additional capabilities
&amp;rarr; see also the [[open design process draft|http://www.pipapo.org/pipawiki/Lumiera/DesignProcess/DesignRenderNodesInterface]]
&amp;rarr; see [[mem management|ManagementRenderNodes]]
&amp;rarr; see RenderProcess
</pre>
</div>
@ -3037,10 +3041,39 @@ At first sight the link between asset and clip-MO is a simple logical relation b
[img[Entities comprising the Render Engine|uml/fig128389.png]]
</pre>
</div>
<div title="RenderMechanics" modifier="Ichthyostega" modified="200806130010" created="200806030230" tags="Rendering impl dynamic" changecount="22">
<div title="RenderImplDetails" modifier="Ichthyostega" modified="200806222347" created="200806220211" tags="Rendering impl" changecount="4">
<pre>Below are some notes regarding details of the actual implementation of the render process and processing node operation. In the description of the [[render node operation protocol|NodeOperationProtocol]] and the [[mechanics of the render process|RenderMechanics]], these details were left out deliberately.
!Layered structure of State
State can be seen as structured like an onion. All the [[StateAdapter]]s in one call stack are supposed to be within one layer: they all know of a &quot;current state&quot;, which in turn is a StateProxy (and thus may refer yet to another state, maybe accros the network or in the backend or whatever). The actual {{{process()}}} function &quot;within&quot; the individual nodes just sees a single StateAdapter and thus can be thought to be a layer below.
!Buffer identification
Buffers are identified by a //handle-id,// which actually is an int index into virtual or real array holding pointers to these buffers. This array is supposed to be //in the next state-layer above.// Thus, within the {{{process()}}} function, the buffer ~IDs are organized according to this node's output- and input ports (starting with the output at index 0), and are provided by the StateAdapter specifically created for this node, while in the next layer above, all the [[StateAdapter]]s deal with allocation and freeing of buffers via buffer IDs valid relative to the &quot;current state&quot;, i.e. the shared StateProxy object used in this call stack. (And so on. Probably a render farm will add another layer)
!Problem of multi-channel nodes
Some data processors simply require to work on multiple channels simultanously, while others work just on a single channel and will be replicated by the builder for each channel invoved. Thus, we are struck with the nasty situation that the node graph may go through some nodes spanning the chain of several channels. Now the decision is //not to care for this complexity within a single chain calculating a single channel.// We rely solely on the cache to avoid duplicated calculations. When a given node happens to produce multiple output buffers, we are bound to allocate them for the purpose of this nodes {{{process()}}} call, but we just &quot;let go&quot; the buffers not needed immediately for the channel acutally to be processed. For this to work, it is supposed that the builder has wired in a caching, and that the cache will hit when we touch the same node again for the other channels.
!Configuration of the processing nodes
Every node is actually decomposed into three parts
* an interface container of a ProcNode subclass
* an {{{const}}} WiringDescriptor, which is actually parametrized to a subtype encoding details of the intended operation
* an StateAdapter object created on the stack for each {{{pull()}}} call and configured specifically for the intended operation
Thus, the outer container can be changed polymorphically to support the different kinds of nodes (large-scale view). The actual wiring of the nodes is contained in the WiringDescriptor, including the {{{process()}}} function pointer. Additionally, this WiringDescriptor knows the actual type of the StateAdapter to create, and this actual type has been chosen by the builder such as to select details of the desired operation of this mode, for example caching / no caching or maybe ~OpenGL rendering or the special case of a node pulling directly from a source reader. Most of this configuration is done by selecting the right template specialisation within the builder; thus in the critical path most of the calls can be inlined
!Rules for buffer allocation and freeing
* only output buffers are allocated. It is //never necessary//&amp;nbsp; to allocate input buffers!
* buffers are to be allocated as late as possible, typically just before invoking {{{process()}}}
* buffers are allways allocated by calling to the parent StateAdapter, because of the possibility to write to cache.
* after returing form a {{{pull()}}}, the buffer-ID of the single output requested by this call is returned. By using this ID, the caller can retrieve the actual buffer holding the result from the &quot;current state&quot; StaeProxy.
* any other buffers filled with results in the course of the same {{{process()}}} call can be released immediately before returning from the {{{pull()}}}
* similar, and input buffers are to be released immediately after the {{{process()}}} call, but before returing from this {{{pull()}}}
* buffers are allways released by calling to the &quot;current state&quot; (which is a StateProxy), providing the buffer-ID to be released
</pre>
</div>
<div title="RenderMechanics" modifier="Ichthyostega" modified="200806220133" created="200806030230" tags="Rendering impl dynamic" changecount="24">
<pre>While the render process, with respect to the dependencies, the builder and the processing function is sufficiently characterized by referring to the ''pull principle'' and by defining a [[protocol|NodeOperationProtocol]] each node has to adhere to &amp;mdash; for actually get it coded we have to care for some important details, especially //how to manage the buffers.// It may well be that the length of the code path necessary to invoke the individual processing functions is finally not so important, compared with the time spent at the inner pixel loop within these functions. But my guess is (as of 5/08), that the overall number of data moving and copying operations //will be//&amp;nbsp; of importance.
!reguirements
!requirements
* operations should be &quot;in place&quot; as much as possible
* because caching necessitates a copy, the points where this happens should be controllable.
* buffers should accommodate automatically to provide the necessary space without clipping the image.
@ -3054,8 +3087,9 @@ Not everything can be preconfigured though. The pull principle opens the possibi
!buffer management
Besides the StateProxy representing the actual render process and holding a couple of buffer (refs), we employ a lightweight adapter object in between. It is used //for a single {{{pull()}}}-call// &amp;mdash; mapping the actual buffers to the input and output port numbers of the processing node and for dealing with the cache calls. While the StateProxy manages a pool of frame buffers, this interspersed adapter allows us to either use a buffer retrieved from the cache as an input, possibly use a new buffer located within the cache as output, or (in case no caching happens) to just use the same buffer as input and output for &quot;in-place&quot;-processing. The idea is that most of the configuration of this adapter object is prepared in the wiring step while building the node network.
The usage patern of the buffers can be stack-like when processing nodes requiring multiple input buffers. In the standard case, which also is the simplest case, a pair of buffers (or a single buffer for &quot;in-place&quot; capable nodes) suffices to calculate a whole chain of nodes. But &amp;mdash; as the recursive descent means depth-first processing &amp;mdash; in case multiple input buffers are needed, we may encounter a situation where some of these input buffers already contain processed data, while we have to descend into yet another predecessor node chain to pull the data for the remaining buffers. Care has to be taken //to allocate the buffers as late as possible,// otherwise we could end up holding onto a buffer almost for each node in the network. Effectively this translates into the rule to allocate output buffers only after all input buffers are ready and filled with data; thus we shouldn't allocate buffers when //entering// the recursive call to the predecessor(s), rather we have to wait until we are about to return from the downcall chain.
The usage patern of the buffers can be stack-like when processing nodes require multiple input buffers. In the standard case, which also is the simplest case, a pair of buffers (or a single buffer for &quot;in-place&quot; capable nodes) suffices to calculate a whole chain of nodes. But &amp;mdash; as the recursive descent means depth-first processing &amp;mdash; in case multiple input buffers are needed, we may encounter a situation where some of these input buffers already contain processed data, while we have to descend into yet another predecessor node chain to pull the data for the remaining buffers. Care has to be taken //to allocate the buffers as late as possible,// otherwise we could end up holding onto a buffer almost for each node in the network. Effectively this translates into the rule to allocate output buffers only after all input buffers are ready and filled with data; thus we shouldn't allocate buffers when //entering// the recursive call to the predecessor(s), rather we have to wait until we are about to return from the downcall chain.
Besides, these considerations also show we need a means of passing on the current buffer usage pattern while calling down. This usage pattern not only includes a record of what buffers are occupied, but also the intended use of these occupied buffers, especially if they can be modified in-place, and at which point they may be released and reused.
__note__: this process outlined here and below is still an simplification. The actual implementation has some additional [[details to care for|RenderImplDetails]]
!!Example: calculating a 3 node chain
# Caller invokes calculation by pulling from exit node, providing the top-level StateProxy
@ -3083,6 +3117,7 @@ Besides, these considerations also show we need a means of passing on the curren
@@clear(right):display(block):@@
__see also__
&amp;rarr; the [[Entities involved in Rendering|RenderEntities]]
&amp;rarr; additional [[implementation details|RenderImplDetails]]
&amp;rarr; [[Memory management for render nodes|ManagementRenderNodes]]
&amp;rarr; the protocol [[how to operate the nodes|NodeOperationProtocol]]
</pre>