benn/lumiera_
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

planning the handling of multichannel media/clips

Browse source
This commit is contained in:
Fischlurch 2007-09-20 15:42:26 +02:00
parent 5d54cfc89e
commit 91a4835f6a
7 changed files with 183 additions and 45 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

63
uml/cinelerra3/128133
View file

@ -1,6 +1,6 @@
format 40
"Asset" // ProcessingLayer::Asset
revision 7
revision 8
modified_by 5 "hiv"
// class settings
//class diagram settings
@ -354,6 +354,46 @@ ${inlines}
classrelation_ref 139909 // <realization>
b multiplicity "" parent class_ref 136453 // Asset
end
operation 133253 "createClip"
public return_type class_ref 128901 // Clip
nparams 0
cpp_decl " ${comment}${friend}${static}${inline}${virtual}${type} ${name} ${(}${)}${const}${volatile} ${throw}${abstract};"
cpp_def "${comment}${inline}${type}
${class}::${name} ${(}${)}${const}${volatile} ${throw}${staticnl}
{
${body}
}
"
comment "create a (possibly compound) Clip refering to this media, ready to be added to the EDL."
end
operation 133381 "howtoProc"
public return_type class_ref 138757 // ProcPatt
nparams 0
cpp_decl " ${comment}${friend}${static}${inline}${virtual}${type} ${name} ${(}${)}${const}${volatile} ${throw}${abstract};"
cpp_def "${comment}${inline}${type}
${class}::${name} ${(}${)}${const}${volatile} ${throw}${staticnl}
{
${body}
}
"
comment "@return descriptor how to build a render pipeline corresponding to this media"
end
classrelation 143237 // <dependency>
relation 141317 -_->
a default
cpp default "Generated"
classrelation_ref 143237 // <dependency>
b multiplicity "" parent class_ref 138757 // ProcPatt
end
end
class 138501 "CompoundMedia"
@ -496,6 +536,7 @@ ${inlines}
b multiplicity "*" parent class_ref 136709 // Media
association_type class_ref 136709 // Media
end
end
class 137477 "Unknown"
@ -624,6 +665,26 @@ ${inlines}
end
end
class 138757 "ProcPatt"
visibility package
cpp_decl "${comment}${template}class ${name}${inherit}
{
${members} };
${inlines}
"
java_decl ""
idl_decl ""
explicit_switch_type ""
classrelation 143109 // <generalisation>
relation 141189 ---|>
a public
cpp default "${type}"
classrelation_ref 143109 // <generalisation>
b multiplicity "" parent class_ref 136965 // Struct
end
end
class 138245 "Dataset"
visibility package
cpp_decl "${comment}${template}class ${name}${inherit}

8
uml/cinelerra3/128133.diagram
View file

@ -90,6 +90,10 @@ classcanvas 138629 class_ref 135173 // Segment
draw_all_relations default hide_attributes default hide_operations default show_members_full_definition default show_members_visibility default show_members_stereotype default show_parameter_dir default show_parameter_name default package_name_in_tab default class_drawing_mode default drawing_language default show_context_mode default auto_label_position default show_infonote default shadow default
xyz 417 678 2000
end
classcanvas 139013 class_ref 138629 // CompoundClip
draw_all_relations default hide_attributes default hide_operations default show_members_full_definition default show_members_visibility default show_members_stereotype default show_parameter_dir default show_parameter_name default package_name_in_tab default class_drawing_mode default drawing_language default show_context_mode default auto_label_position default show_infonote default shadow default
xyz 266 466 2000
end
relationcanvas 128389 relation_ref 128005 // <directional aggregation>
from ref 128005 z 1999 stereotype "<<list>>" xyz 139 626 3000 to ref 128133
role_a_pos 201 603 3000 no_role_b
@ -202,4 +206,8 @@ relationcanvas 138757 relation_ref 137093 // <directional aggregation>
line 138885 z 1999 to ref 135813
role_a_pos 524 717 3000 no_role_b
multiplicity_a_pos 505 783 3000 no_multiplicity_b
relationcanvas 139141 relation_ref 140805 // <generalisation>
from ref 139013 z 1999 to ref 131461
no_role_a no_role_b
no_multiplicity_a no_multiplicity_b
end

35
uml/cinelerra3/128261
View file

@ -1,6 +1,6 @@
format 40
"MObject" // ProcessingLayer::MObject
revision 17
revision 18
modified_by 5 "hiv"
// class settings
//class diagram settings
@ -415,18 +415,39 @@ ${inlines}
comment "startpos in source"
end
classrelation 141829 // source (<unidirectional association>)
relation 140037 --->
a role_name "source" multiplicity "1" const_relation protected
classrelation 142469 // source (<unidirectional association>)
relation 140677 --->
a role_name "source" multiplicity "1" protected
comment "the media source this clip referes to"
cpp default " ${comment}${static}${mutable}${volatile}${const}${type}* ${name}${value};
"
classrelation_ref 141829 // source (<unidirectional association>)
b multiplicity "*" parent class_ref 137349 // Clip
association_type class_ref 128901 // Clip
classrelation_ref 142469 // source (<unidirectional association>)
b multiplicity "*" parent class_ref 136709 // Media
end
end
class 138629 "CompoundClip"
visibility package
cpp_decl "${comment}${template}class ${name}${inherit}
{
${members} };
${inlines}
"
java_decl ""
idl_decl ""
explicit_switch_type ""
associated_diagram classdiagram_ref 128133 // Session structure
classrelation 142597 // <generalisation>
relation 140805 ---|>
a public
cpp default "${type}"
classrelation_ref 142597 // <generalisation>
b multiplicity "" parent class_ref 128901 // Clip
end
end
class 129029 "Effect"
visibility package
cpp_decl "${comment}${template}class ${name}${inherit}

33
uml/cinelerra3/130309.diagram
View file

@ -13,7 +13,7 @@ packagecanvas 128517
show_context_mode namespace xyzwh 217 182 1994 608 534
classcanvas 128645 class_ref 136709 // Media
draw_all_relations default hide_attributes default hide_operations default show_members_full_definition default show_members_visibility default show_members_stereotype default show_parameter_dir default show_parameter_name default package_name_in_tab default class_drawing_mode default drawing_language default show_context_mode default auto_label_position default show_infonote default shadow default
xyz 273 347 2005
xyz 259 330 2005
end
classcanvas 128773 class_ref 136837 // Proc
draw_all_relations default hide_attributes default hide_operations default show_members_full_definition default show_members_visibility default show_members_stereotype default show_parameter_dir default show_parameter_name default package_name_in_tab default class_drawing_mode default drawing_language default show_context_mode default auto_label_position default show_infonote default shadow default
@ -71,10 +71,14 @@ classcanvas 133765 class_ref 138501 // CompoundMedia
draw_all_relations default hide_attributes default hide_operations default show_members_full_definition default show_members_visibility default show_members_stereotype default show_parameter_dir default show_parameter_name default package_name_in_tab default class_drawing_mode default drawing_language default show_context_mode default auto_label_position default show_infonote default shadow default
xyz 245 529 2005
end
classcanvas 135813 class_ref 138757 // ProcPatt
draw_all_relations default hide_attributes default hide_operations default show_members_full_definition default show_members_visibility default show_members_stereotype default show_parameter_dir default show_parameter_name default package_name_in_tab default class_drawing_mode default drawing_language default show_context_mode default auto_label_position default show_infonote default shadow default
xyz 638 506 2000
end
relationcanvas 129157 relation_ref 138117 // <realization>
geometry VHV
from ref 128645 z 1999 to point 293 315
line 129797 z 1999 to point 138 315
from ref 128645 z 1999 to point 293 307
line 129797 z 1999 to point 138 307
line 129925 z 1999 to ref 128005
no_role_a no_role_b
no_multiplicity_a no_multiplicity_b
@ -105,15 +109,15 @@ relationcanvas 130949 relation_ref 138629 // <unidirectional association>
multiplicity_a_pos 260 250 3000 multiplicity_b_pos 194 194 3000
relationcanvas 131205 relation_ref 138757 // <generalisation>
geometry VHV
from ref 131077 z 1999 to point 339 415
line 134277 z 1999 to point 293 415
from ref 131077 z 1999 to point 339 423
line 134277 z 1999 to point 293 423
line 134405 z 1999 to ref 128645
no_role_a no_role_b
no_multiplicity_a no_multiplicity_b
relationcanvas 131717 relation_ref 139013 // <generalisation>
geometry VHV
from ref 131461 z 2004 to point 411 415
line 134021 z 2004 to point 293 415
from ref 131461 z 2004 to point 411 423
line 134021 z 2004 to point 293 423
line 134149 z 2004 to ref 128645
no_role_a no_role_b
no_multiplicity_a no_multiplicity_b
@ -155,8 +159,19 @@ relationcanvas 134917 relation_ref 140549 // <directional aggregation>
from ref 133765 z 2004 stereotype "<<vector>>" xyz 229 493 3000 to point 225 545
line 135301 z 2004 to point 225 365
line 135173 z 2004 to ref 128645
role_a_pos 230 378 3000 no_role_b
multiplicity_a_pos 231 348 3000 multiplicity_b_pos 230 556 3000
role_a_pos 217 343 3000 no_role_b
multiplicity_a_pos 232 376 3000 multiplicity_b_pos 230 556 3000
relationcanvas 136069 relation_ref 141189 // <generalisation>
from ref 135813 z 1999 to ref 128901
no_role_a no_role_b
no_multiplicity_a no_multiplicity_b
relationcanvas 136197 relation_ref 141317 // <dependency>
from ref 128645 z 1999 to point 324 392
line 136581 z 1999 to point 447 392
line 136453 z 1999 to point 471 525
line 136325 z 1999 to ref 135813
no_role_a no_role_b
no_multiplicity_a no_multiplicity_b
line 128261 -_-_ geometry HV
from ref 128005 z 1999 to point 331 150
line 128389 z 1999 to ref 128133

63
uml/cinelerra3/130437.diagram
View file

@ -6,7 +6,7 @@ classcanvas 128005 class_ref 137349 // Clip
end
classcanvas 128133 class_ref 136709 // Media
draw_all_relations default hide_attributes default hide_operations default show_members_full_definition default show_members_visibility default show_members_stereotype default show_parameter_dir default show_parameter_name default package_name_in_tab default class_drawing_mode default drawing_language default show_context_mode default auto_label_position default show_infonote default shadow default
xyz 545 50 2005
xyz 533 42 2005
end
classcanvas 128261 class_ref 137477 // Unknown
draw_all_relations default hide_attributes default hide_operations default show_members_full_definition default show_members_visibility default show_members_stereotype default show_parameter_dir default show_parameter_name default package_name_in_tab default class_drawing_mode default drawing_language default show_context_mode default auto_label_position default show_infonote default shadow default
@ -58,16 +58,28 @@ classcanvas 134661 class_ref 133253 // Frame
end
packagecanvas 135685
package_ref 128261 // MObject
xyzwh 24 10 1994 185 313
xyzwh 24 10 1994 187 356
packagecanvas 135813
package_ref 128133 // Asset
xyzwh 454 10 1994 276 313
xyzwh 454 10 1994 367 311
note 136837 "the Builder implements each Clip by a source node and maybe some codec"
xyzwh 53 376 2000 219 61
classcanvas 137221 class_ref 138501 // CompoundMedia
draw_all_relations default hide_attributes default hide_operations default show_members_full_definition default show_members_visibility default show_members_stereotype default show_parameter_dir default show_parameter_name default package_name_in_tab default class_drawing_mode default drawing_language default show_context_mode default auto_label_position default show_infonote default shadow default
xyz 606 148 3005
end
classcanvas 138373 class_ref 138629 // CompoundClip
draw_all_relations default hide_attributes default hide_operations default show_members_full_definition default show_members_visibility default show_members_stereotype default show_parameter_dir default show_parameter_name default package_name_in_tab default class_drawing_mode default drawing_language default show_context_mode default auto_label_position default show_infonote default shadow default
xyz 72 301 2000
end
classcanvas 138885 class_ref 136965 // Struct
draw_all_relations default hide_attributes default hide_operations default show_members_full_definition default show_members_visibility default show_members_stereotype default show_parameter_dir default show_parameter_name default package_name_in_tab default class_drawing_mode default drawing_language default show_context_mode default auto_label_position default show_infonote default shadow default
xyz 758 51 2005
end
classcanvas 139013 class_ref 138757 // ProcPatt
draw_all_relations default hide_attributes default hide_operations default show_members_full_definition default show_members_visibility default show_members_stereotype default show_parameter_dir default show_parameter_name default package_name_in_tab default class_drawing_mode default drawing_language default show_context_mode default auto_label_position default show_infonote default shadow default
xyz 751 115 2000
end
relationcanvas 128517 relation_ref 139141 // <generalisation>
from ref 128261 z 1999 to ref 128389
no_role_a no_role_b
@ -130,31 +142,44 @@ relationcanvas 135301 relation_ref 134021 // <generalisation>
no_multiplicity_a no_multiplicity_b
relationcanvas 135941 relation_ref 139909 // <dependency>
from ref 130821 z 1999 to point 433 383
line 136069 z 1999 to point 433 68
line 136069 z 1999 to point 433 25
line 136197 z 1999 to ref 128133
no_role_a no_role_b
no_multiplicity_a no_multiplicity_b
relationcanvas 136325 relation_ref 140037 // <unidirectional association>
from ref 128901 z 1999 to point 224 231
line 136709 z 1999 to point 370 166
line 136581 z 1999 to ref 128005
role_a_pos 382 147 3000 no_role_b
multiplicity_a_pos 452 177 3000 multiplicity_b_pos 150 242 3000
relationcanvas 136965 relation_ref 140165 // <unidirectional association>
geometry VH
from ref 128005 z 1999 to point 489 69
from ref 128005 z 1999 to point 489 78
line 137093 z 1999 to ref 128133
role_a_pos 491 49 3000 no_role_b
multiplicity_a_pos 524 72 3000 multiplicity_b_pos 477 123 3000
role_a_pos 488 56 3000 no_role_b
multiplicity_a_pos 516 89 3000 multiplicity_b_pos 477 123 3000
relationcanvas 137349 relation_ref 140421 // <generalisation>
from ref 137221 z 1999 to ref 128133
no_role_a no_role_b
no_multiplicity_a no_multiplicity_b
relationcanvas 137477 relation_ref 140549 // <directional aggregation>
from ref 137221 z 1999 to point 654 68
line 137733 z 1999 stereotype "<<vector>>" xyz 662 113 3000 to ref 128133
role_a_pos 602 48 3000 no_role_b
multiplicity_a_pos 605 72 3000 multiplicity_b_pos 642 123 3000
line 136453 -_-_
from ref 136325 z 1998 to ref 128901
from ref 137221 z 1999 to point 654 77
line 137733 z 1999 stereotype "<<vector>>" xyz 670 122 3000 to ref 128133
role_a_pos 615 55 3000 no_role_b
multiplicity_a_pos 590 122 3000 multiplicity_b_pos 642 123 3000
relationcanvas 137861 relation_ref 140677 // <unidirectional association>
from ref 128901 z 1999 to point 218 232
line 138245 z 1999 to point 383 49
line 137989 z 1999 to point 492 49
line 138117 z 1999 to ref 128133
role_a_pos 488 44 3000 no_role_b
multiplicity_a_pos 516 77 3000 multiplicity_b_pos 149 235 3000
relationcanvas 138501 relation_ref 140805 // <generalisation>
from ref 138373 z 1999 to ref 128901
no_role_a no_role_b
no_multiplicity_a no_multiplicity_b
relationcanvas 139141 relation_ref 141189 // <generalisation>
from ref 139013 z 1999 to ref 138885
no_role_a no_role_b
no_multiplicity_a no_multiplicity_b
relationcanvas 139397 relation_ref 141317 // <dependency>
from ref 128133 z 1999 to point 599 104
line 140165 z 1999 to point 697 104
line 139781 z 1999 to ref 139013
no_role_a no_role_b
no_multiplicity_a no_multiplicity_b
end

12
uml/cinelerra3/5.session
View file

@ -3,14 +3,12 @@ diagrams
classdiagram_ref 130309 // Asset Kinds
860 607 100 4 120 0
active classdiagram_ref 130437 // Media-Asset Relations
817 506 100 4 0 0
817 506 100 4 0 38
classdiagram_ref 128133 // Session structure
688 506 100 4 60 0
classdiagram_ref 128389 // Render Entities
688 506 100 4 120 0
688 506 100 4 180 80
end
show_stereotypes
selected classdiagram_ref 130437 // Media-Asset Relations
selected classrelation_ref 143237 // <dependency>
open
package_ref 128005 // design
@ -20,7 +18,9 @@ open
class_ref 136709 // Media
class_ref 138501 // CompoundMedia
class_ref 137349 // Clip
classview_ref 128005 // Session
class_ref 138757 // ProcPatt
class_ref 128901 // Clip
class_ref 138629 // CompoundClip
package_ref 128389 // RenderEngine
class_ref 135685 // Logic

14
wiki/renderengine.html
View file

@ -1391,7 +1391,7 @@ For the case in question this seems to be the ''resource allocation is construct
And, last but not least, doing all actual allocations is the job of the backend. Exceptions being long-lived objects, like the Session or the EDL, which are created once and don't bear the danger of causing memory pressure. Besides that, the ProcLayer code shouldn't issue &quot;new&quot; and &quot;delete&quot;, rather it should use some central [[Factories]] for all allocation and freeing, so we can redirect these calls down to the backend, which may use pooling or special placement allocators or the like. The rationale is, for modern hardware/architectures, care has to be taken with heap allocations, esp. with many small objects and irregular usage patterns.
</pre>
</div>
<div title="MultichannelMedia" modifier="Ichthyostega" created="200709200255" tags="design" changecount="2">
<div title="MultichannelMedia" modifier="Ichthyostega" modified="200709201341" created="200709200255" tags="design" changecount="3">
<pre>Based on practical experiences, Ichthyo tends to consider Multichannel Media as the base case, while counting media files providing just one single media stream as exotic corner cases. This may seem counter intuitive at first sight; you should think of it as an attempt to avoid right from start some of the common shortcomings found in many video editors, especially
* having to deal with keeping a &quot;link&quot; between audio and video clips
* silly limitations on the supported audio setups (e.g. mono, stereo and 5.1)
@ -1399,9 +1399,17 @@ And, last but not least, doing all actual allocations is the job of the backend.
* inability to edit stereoscopic (3D) video in a natural fashion
!Compound Media
Basically, each [[media asset|MediaAsset]] is considered to be a compound of several elementary media (tracks), possibly of various different media kinds. Adding support for placeholders (''proxy clips'') at some point in future will add still more complexity (because then there will be even dependencies between some of these elementary media). We try to keep the structural limitations necessary to be able to handle, edit and render compound media to the bare minimum. This is only possible, if we try to configure as much as possible already at the &quot;asset level&quot; and make the rest of the proc layer behave just according to the configuration given with each asset.
Basically, each [[media asset|MediaAsset]] is considered to be a compound of several elementary media (tracks), possibly of various different media kinds. Adding support for placeholders (''proxy clips'') at some point in future will add still more complexity (because then there will be even dependencies between some of these elementary media). To handle, edit and render compound media, we need to impose some structural limitations. But anyhow, we try to configure as much as possible already at the &quot;asset level&quot; and make the rest of the proc layer behave just according to the configuration given with each asset.
So, when creating a clip out of such a compound media asset, the clip has to be a compound of elementary clips mirroring the given media asset's structure. Besides, it should be possible to //detach// and //attach// elementary clips from a compound clip. On the other hand, the [[Fixture]] created from the current state of the [[EDL]] is explicit to a great extent. So, in the Fixture we deal only with elementary clips placed to absolute positions, and thus the builder will see only simple non-compound clips and translate them into the corresponding source reading nodes.</pre>
So, when creating a clip out of such a compound media asset, the clip has to be a compound of elementary clips mirroring the given media asset's structure. Besides, it should be possible to //detach// and //attach// elementary clips from a compound clip. On the other hand, the [[Fixture]] created from the current state of the [[EDL]] is explicit to a great extent. So, in the Fixture we deal only with elementary clips placed to absolute positions, and thus the builder will see only simple non-compound clips and translate them into the corresponding source reading nodes.
!Handling
* from a Media asset, we can get a [[Processing Pattern|ProcPatt]] describing how to build a render pipeline for this media
* we can create a Clip (MObject) from each Media, which will be linked back to the media asset internally.
* moreover, creating a Clip will create and register a Clip asset as well, and this Clip asset will be tied to the original Clip and will show up in some special Category
* media can be compound and the created Clips will mirror this compound structure
* {{red{to be defined}}} a way to distinguish Elementay Clips (non-compound) and to express that only such can be passed to the Builder
* the Builder gets at the ProcPatt (descriptor) of the underlying media for each clip and uses this description as a template to build the render pipeline. That is, the ProcPatt specifies the codec asset and maybe some additional effect assets (deinterlace, scale) necessary for feeding media data corresponding to this clip/media into the render nodes network.</pre>
</div>
<div title="ObjectCreation" modifier="Ichthyostega" modified="200709040257" created="200709030139" tags="impl design" changecount="10">
<pre>We have to consider carefully how to handle the Creation of new class instances. Because, when done naively, it can defeat all efforts of separating subsystems, or &amp;mdash; the other extreme &amp;mdash; lead to a //switch-on-typeID// programming style. We strive at a solution somewhere in the middle by utilizing __Abstract Factories__ on Interface or key abstraction classes, but providing specialized overloads for the different use cases. So in each use case we have to decide if we want to create a representant of some general concept (Interface), or if we have a direct colaboration and thus need the Factory to provide