From b3b91b732c2312c1dda40f1ba09115ea20dc83e2 Mon Sep 17 00:00:00 2001 From: Ichthyostega Date: Fri, 10 Aug 2007 06:11:31 +0200 Subject: [PATCH 1/7] some planning and problem analysis re. Memory Management... --- wiki/.gitignore | 1 - wiki/renderengine.html | 91 +++++++++++++++++++++++++++++++++++++++--- 2 files changed, 86 insertions(+), 6 deletions(-) delete mode 100644 wiki/.gitignore diff --git a/wiki/.gitignore b/wiki/.gitignore deleted file mode 100644 index 1fb9ef574..000000000 --- a/wiki/.gitignore +++ /dev/null @@ -1 +0,0 @@ -tmp/* diff --git a/wiki/renderengine.html b/wiki/renderengine.html index 2c79b42c5..246def996 100644 --- a/wiki/renderengine.html +++ b/wiki/renderengine.html @@ -514,7 +514,16 @@ ColorPalette SiteUrl -
+
+
Asset management is a subsystem on its own. Assets are "things" that can be loaded into a session, like Media, Clips, Effects, Transitions. It is the "bookkeeping view", while the EDL is the "manipulation and process view".
+
+The Assets are important reference points holding the information needed to access external resources. For example, an Clip asset can reference a Media asset, which in turn holds the external filename from which to get the media stream. Or, an Effect asset can hold a plugin-ID.
+
+!still to be worked out..
+is how to implement the relationship between [[MObject]]s and Assets. Do we use direct pointers, or do we prefer an ID + central registry approach? And how to handle the removal of an Asset (→ see also [[analysis of mem management|ManagementAssetRelation]])
+
+
+
Automation is treated as a function over time. It is always tied to a specific Parameter (which can thus be variable over the course of the timeline). All details //how// this function is defined are completely abstracted away. The Parameter uses a ParamProvider to get the value for a given Time (point). Typically, this will use linear or bezier interpolation over a set of keyframes internally. Parameters can be configured to have different value ranges and distribution types (on-off, stepped, continuous, bounded)
 
 [img[how to implement Automation|uml/fig129669.png]]
@@ -686,8 +695,12 @@ TertiaryMid: #99a
 TertiaryDark: #667
 Error: #f88
-
+
Here, in the context of the Render Engine, the Controller component is responsible for managing the global playback state, for triggering the build process and for activating the backend and the Render Engine configuration created by the Builder to carry out the actual rendering. So you can expect the Controller to encompass a State Machine.
+
+!Facade
+This is an very important external Interface, because it links together all three Layers of our current architecture. It can be used by the backend to initiate [[Render Processes (=StateProxy)|StateProxy]] and it will probably be used by the Dispatcher for GUI actions as well...
+
@@ -746,6 +759,25 @@ To make the intended use of the classes more clear, consider the following two e 
A special kind (subclass) of [[Placement]]. As such it is always linked to a //Subject//, i.e. a MObject. In addition to the properties of a (unspecific) Placement, the ExplicitPlacement specifies a absolute time and track position for locating the Subject
+
+
+
+
We use Factories
+* for centralizing [[memory management|MemoryManagement]]
+* to support polymorphism (of course...)
+
+!Requirements
+* request the actual placement/allocation from the backend
+* allways hand out a smart-pointer
+* encapsulate / make configurable the smart-pointer type
+* install a callback into the smart-pointer for destroying the resource.
+* redirect the destroying request to the backend
+
+!Implementation Questions
+* how much genericity? (Ichthyo is rather inclined not to overdo this one. Often it is preferable to have repeated implementations follow a well known pattern, if this leads to short and simple implementations, while the complete general solution will be much more contrived).
+* how to specify the actual type needed?
+* how to implement the cases where a subtype needs to be selected (abstract factory pattern). Embody this into the Factory, pass it in as a Strategy or treat the Factory just as a simple service taking an explicit type-ID and providing the new object?
+
@@ -846,10 +878,13 @@ For this Cinelerra3 design, we could consider making GOP just another raw media →see in [[Wikipedia|http://en.wikipedia.org/wiki/Group_of_pictures]]
-
-
This wiki page is the entry point to some detail notes covering some technical decisions, details and problems encountered in the course of the implementation of the Cinelerra Renderengine, the Builder and the related parts.
+

+
This wiki page is the entry point to detail notes covering some technical decisions, details and problems encountered in the course of the implementation of the Cinelerra Renderengine, the Builder and the related parts.
 
-* [[Packages, Interfaces and Namespaces|InterfaceNamespaces]]

+* [[Packages, Interfaces and Namespaces|InterfaceNamespaces]]
+* [[Memory Management Issues|MemoryManagement]]
+
+
/***
@@ -1220,6 +1255,12 @@ This Design strives to achieve a StrongSeparation between the low-level Structur
 [[Admin]]
 <<fullscreen>>
+
+
Problem is: when removing an Asset, all corresponding MObjects need to disappear. This means, besides the obvious Ref-Link (MObject refering to an asset) we need backlinks or a sort of registry. And still worse: we need to remove the affetcted MObject from the object network in the EDL and rebuild the Fixture...
+
+{{red{to be considered in more detail later}}}
+
+
<!--{{{-->
 <link rel='alternate' type='application/rss+xml' title='RSS' href='index.xml'/>
@@ -1227,6 +1268,39 @@ This Design strives to achieve a StrongSeparation between the low-level Structur
 
 <style type="text/css">#contentWrapper {display:none;}</style><div id="SplashScreen" style="border: 3px solid #ccc; display: block; text-align: center; width: 320px; margin: 100px auto; padding: 50px; color:#000; font-size: 28px; font-family:Tahoma; background-color:#eee;">loading <b>Cinelerra Renderengine</b> devel doku<blink> ...</blink><br><br><span style="font-size: 14px; color:red;">Requires Javascript.</span></div>
+
+
Of course: Cinelerra currently leaks memory and crashes regularilly. For the newly written code, besides retaining the same performance level, a main goal is to use methods and techniques known to support the writing of quality code. So, besides the MultithreadConsiderations, a solid strategy for managing the ownership of allocated memory blocks is necessary right from start.
+
+!Problems
+# Memory management needs to work correct in a //fault tolerant environment//. That means that we need to be prepared to //handle on a non-local scale// some sorts of error conditions (without aborting the application). To be more precise: some error condition arises locally, which leads to a local abort and just the disabling/failing of some subsystem without affecting the application as a whole. This can happen on a regular base (e.g. rendering fails) and thus is __no excuse for leaking memory__
+# Some (not all) parts of the core application are non-deterministic. That means, we can't tie the memory management to any assumptions on behalf of the execution path
+
+!C++ solution
+First of all -- this doesn't concern //every// allocation. It rather means there are certain //dangerous areas// which need to be identified. And as always, instead of carrying the inherent complexities of the problem into the solution, we should rather look for a common solution pattern which helps factoring out the complexity.
+
+For the case in question this seems to be the ''resource allocation is construction'' pattern. Which boils down to basically never using bare pointers when concerned with ownership. Instead, ownership should be handled by smart-pointers.
+
+!!usage scenarios
+# __existence is being used__: Objects just live for being referred to in a object network. In this case, use refcounting smart-pointers for every ref. (note: problem with cyclic refs)
+# __entity bound ownership__: Objects can be tied to some long living entity in the program, which holds the smart-pointer
+#* if the existence of these ref-holding entity can be //guaranteed// (like a contract), then the other users can build a object network with conventional pointers
+#* otherwise, when the ref-holding entity //can disappear// in a regular program state, we need weak-refs and checking (because by our postulate the controlled resource needs to be destructed immediately, otherwise we would have the first case, existence == being used)
+
+!!!dangerous uses
+* the render nodes &rarr; [[detail analysis|ManagementRenderNodes]] {{red{TODO}}}
+* the MObjects in the EDL &rarr; [[detail analysis|ManagementMObjects]] {{red{TODO}}}
+* Asset - MObject relationship. &rarr; [[detail analysis|ManagementAssetRelation]] {{red{TODO}}}
+
+!!!rather harmless
+* Frames (buffers), because they belong to a given [[RenderProcess (=StateProxy)|StateProxy]] and are just passed in into the individual [[ProcNode]]s. This can be handled consistently with conventional methods.
+* each StateProxy belongs to one top-level call to the [[Controller-Facade|Controller]]
+* same for the builder tools
+* the EDL and the defined [[Asset]]s belong together to one Session. If the Session is closed, this means a internal shutdown of the whole ProcLayer, i.e. closing of all GUI representations and terminating all render processes. If these calles are implemented as blocking operations, we can assert that as long as any GUI representation or any render process is running, there is a valid Session and EDL.
+
+!using Factories
+And, last but not least, doing all actual allocations is the job of the backend. Exceptions being long-lived objects, like the Session or the EDL, which are created once and don't bear the danger of causing memory pressure. Besides that, the ProcLayer code shouldn't issue "new" and "delete", rather it should use some central [[Factories]] for all allocation and freeing, so we can redirect these calls down to the backend, which may use pooling or special placement allocators or the like. The rationale is, for modern hardware/architectures, care has to be taken with heap allocations, esp. with many small objects and irregular usage patterns.
+
+
Cinelerra2 introduced OpenGL support for rendering previews. I must admit, I am very unhappy with this, because
 * it just supports some hardware
@@ -1838,6 +1912,13 @@ Closely related to this is the not-so-obvious problem how to understand the comm
 * is it really necessary to have fixed global tracks?
 * is it really helpful to feed "source tracks" into global processing busses/channels?
 Users accustomed with modern GUI applications typically expect that //everything is a object//  and can be pulled around and  manipulated individually. This seems natural at start, but raises the problem of providing a efficient workflow for handling larger projects and editing tasks. So, if we don't have a hard wired multitrack+bus architecture, we need some sort of templating to get the standard editing use case done efficiently.
+
+
+
+
The middle Layer of our current Architecture plan, i.e. the layer managing all processing and manipulation, while the actual data handling is done in the backend and the user interaction belongs to the GUI Layer.
+
+&rarr; see the [[Overview]]
+
From 8f35701ca818966becdb6a316b09f8a25b964447 Mon Sep 17 00:00:00 2001 From: Ichthyostega Date: Sun, 12 Aug 2007 01:04:55 +0200 Subject: [PATCH 2/7] initial design for a factory encapsulating creation of smart-pointers --- SConstruct | 6 ++ src/common/factory.hpp | 114 +++++++++++++++++++++++++++++++++++++ src/common/factorytest.cpp | 48 ++++++++++++++++ uml/cinelerra3/128517 | 29 ++++++++-- uml/cinelerra3/5.session | 4 +- wiki/renderengine.html | 37 +++++++++++- 6 files changed, 229 insertions(+), 9 deletions(-) create mode 100644 src/common/factory.hpp create mode 100644 src/common/factorytest.cpp diff --git a/SConstruct b/SConstruct index cc2f5c308..0c2271198 100644 --- a/SConstruct +++ b/SConstruct @@ -152,6 +152,10 @@ def configurePlatform(env): if conf.CheckCHeader('valgrind/valgrind.h'): conf.env.Append(CPPFLAGS = ' -DHAS_VALGRIND_VALGIND_H') + if not conf.CheckCXXHeader('tr1/memory'): + print 'We rely on the std::tr1 proposed standard extension for shared_ptr.' + Exit(1) + if not conf.CheckCXXHeader('boost/config.hpp'): print 'We need the C++ boost-lib.' Exit(1) @@ -188,6 +192,8 @@ def defineBuildTargets(env, artifacts): """ cinobj = ( srcSubtree(env,'backend') + srcSubtree(env,'proc') + + srcSubtree(env,'common') +# + srcSubtree(env,'lib') + env.Object('$SRCDIR/main.cpp') ) plugobj = srcSubtree(env,'plugin', isShared=True) diff --git a/src/common/factory.hpp b/src/common/factory.hpp new file mode 100644 index 000000000..b3f9faeb7 --- /dev/null +++ b/src/common/factory.hpp @@ -0,0 +1,114 @@ +/* + FACTORY.hpp - template for object/smart-pointer factories + + Copyright (C) CinelerraCV + 2007, Christian Thaeter + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + +*/ + + +#ifndef CINELERRA_FACTORY_H +#define CINELERRA_FACTORY_H + +#include + + + +namespace cinelerra + { + + namespace factory{ class VanillaAllocator; }//////////////////////////////////TODO + + /** + * Configurable template for creating Factory classes. + * These encapsulate the creating of new objects, indeed + * delegating the memory allocation to the backend layer. + * The clients get just a smart-pointer or similar handle + * to the created object, which will manage the ownership. + * + * The provided default implementation uses just std::auto_ptr, + * but delegates the allocation to cinelerra's backend-layer. + * + */ + template + < + class T, // the product to be created + template class SMP = std::auto_ptr,// smart-pointer actually returned + class ALO = factory::VanillaAllocator // Allocator facility to be used //////////////TODO + > + class Factory : protected ALO + { + public: + /** Object creating facility. + * Intended to be over/written/ with a variant taking + * the appropriate number of parameters and using the + * (privately inherited) functions of the allocator. + * Note: non-virtual. + */ + SMP operator() (){ return SMP (new T ); }; + + private: + void operator= (const Factory&); // copy prohibited + }; + + + + /* -- some example and default instantiiations -- */ + + namespace factory + { + /** + * Example Allocator using just the normal C++ memory management. + * The intended use is for a Factory instance to iherit from this class. + * Specialized Allocators typically overload operator new and delete. + */ + class VanillaAllocator {}; + + /** + * Example Allocator using plain C memory management. + */ + class MallocAllocator + { + void* operator new (size_t siz) { return malloc (siz); }; + void operator delete (void* p) { if (p) free (p); }; + }; + + + using std::tr1::shared_ptr; + + /** a frequently used instantiation of the Factory, + * using the refcounting shared_ptr from Boost + * and for allocation just our default Allocator + */ + template + class RefcountPtr : public Factory + { + /** let the smart-Ptr use the custom operator delete, + * which may be defined in our Allocator baseclass. + */ + static void destroy (T* victim) { delete victim; }; + + public: + shared_ptr operator() () { return shared_ptr (new T, &destroy ); } + }; + + + + } // namespace factory + +} // namespace cinelerra +#endif diff --git a/src/common/factorytest.cpp b/src/common/factorytest.cpp new file mode 100644 index 000000000..6c11cb45b --- /dev/null +++ b/src/common/factorytest.cpp @@ -0,0 +1,48 @@ +/* + Factorytest - check basic workings of object/smart-pointer factory + + Copyright (C) CinelerraCV + 2007, Christian Thaeter + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + +* *****************************************************/ + + +#include "common/factory.hpp" + +namespace cinelerra + { + + class Blubb + { + int uii ; + + public: + typedef factory::RefcountPtr Factory; + static Factory create; + + Blubb() : uii(42) {} ; + }; + + /** a static Factory instance + * for creating refcounting Ptrs to Blubb objects + */ + Blubb::Factory Blubb::create; + + std::tr1::shared_ptr huii = Blubb::create (); + std::tr1::shared_ptr pfuii = huii; + +} // namespace cinelerra diff --git a/uml/cinelerra3/128517 b/uml/cinelerra3/128517 index 8683426d4..7bb6667f7 100644 --- a/uml/cinelerra3/128517 +++ b/uml/cinelerra3/128517 @@ -1,6 +1,6 @@ -format 38 +format 40 "CommonLib" // CommonLib - revision 6 + revision 7 modified_by 5 "hiv" // class settings //class diagram settings @@ -8,7 +8,7 @@ format 38 //use case diagram settings package_name_in_tab default show_context default auto_label_position default draw_all_relations default shadow default //sequence diagram settings - show_full_operations_definition default write_horizontally default drawing_language default draw_all_relations default shadow default + show_full_operations_definition default write_horizontally default class_drawing_mode default drawing_language default draw_all_relations default shadow default //collaboration diagram settings show_full_operations_definition default show_hierarchical_rank default write_horizontally default drawing_language default package_name_in_tab default show_context default draw_all_relations default shadow default //object diagram settings @@ -34,7 +34,7 @@ format 38 //object diagram settings write_horizontally default package_name_in_tab default show_context default auto_label_position default draw_all_relations default shadow default //sequence diagram settings - show_full_operations_definition default write_horizontally default drawing_language default draw_all_relations default shadow default + show_full_operations_definition default write_horizontally default class_drawing_mode default drawing_language default draw_all_relations default shadow default //state diagram settings package_name_in_tab default show_context default auto_label_position default write_trans_label_horizontally default show_trans_definition default draw_all_relations default shadow default show_activities default region_horizontally default drawing_language default @@ -58,6 +58,23 @@ ${inlines} investigate posix.4 realtime timers, wrap these here" end + + class 135301 "Factory" + visibility public + nformals 1 + formal name "T" type "class" explicit_default_value "" + explicit_extends "" + cpp_decl "${comment}${template}class ${name}${inherit} + { +${members} }; +${inlines} +" + java_decl "" + idl_decl "" + explicit_switch_type "" + + comment "a template for generating functor-like Factory objects, used to encapsulate object creation and providing access via smart-pointers only." + end end classview 128138 "Posix Threads Abstraction" @@ -68,7 +85,7 @@ investigate posix.4 realtime timers, wrap these here" //object diagram settings write_horizontally default package_name_in_tab default show_context default auto_label_position default draw_all_relations default shadow default //sequence diagram settings - show_full_operations_definition default write_horizontally default drawing_language default draw_all_relations default shadow default + show_full_operations_definition default write_horizontally default class_drawing_mode default drawing_language default draw_all_relations default shadow default //state diagram settings package_name_in_tab default show_context default auto_label_position default write_trans_label_horizontally default show_trans_definition default draw_all_relations default shadow default show_activities default region_horizontally default drawing_language default @@ -148,7 +165,7 @@ ${inlines} //object diagram settings write_horizontally default package_name_in_tab default show_context default auto_label_position default draw_all_relations default shadow default //sequence diagram settings - show_full_operations_definition default write_horizontally default drawing_language default draw_all_relations default shadow default + show_full_operations_definition default write_horizontally default class_drawing_mode default drawing_language default draw_all_relations default shadow default //state diagram settings package_name_in_tab default show_context default auto_label_position default write_trans_label_horizontally default show_trans_definition default draw_all_relations default shadow default show_activities default region_horizontally default drawing_language default diff --git a/uml/cinelerra3/5.session b/uml/cinelerra3/5.session index 65a46ae7d..e8231ca4c 100644 --- a/uml/cinelerra3/5.session +++ b/uml/cinelerra3/5.session @@ -1,9 +1,9 @@ window_sizes 1104 756 270 824 557 120 show_stereotypes selected -package_ref 129 // cinelerra3 + package_ref 129 // cinelerra3 open -package_ref 129669 // proc + package_ref 129669 // proc end end diff --git a/wiki/renderengine.html b/wiki/renderengine.html index 246def996..7184bad39 100644 --- a/wiki/renderengine.html +++ b/wiki/renderengine.html @@ -761,7 +761,7 @@ To make the intended use of the classes more clear, consider the following two e 
A special kind (subclass) of [[Placement]]. As such it is always linked to a //Subject//, i.e. a MObject. In addition to the properties of a (unspecific) Placement, the ExplicitPlacement specifies a absolute time and track position for locating the Subject
-
+
We use Factories
 * for centralizing [[memory management|MemoryManagement]]
 * to support polymorphism (of course...)
@@ -778,6 +778,41 @@ To make the intended use of the classes more clear, consider the following two e
 * how to specify the actual type needed?
 * how to implement the cases where a subtype needs to be selected (abstract factory pattern). Embody this into the Factory, pass it in as a Strategy or treat the Factory just as a simple service taking an explicit type-ID and providing the new object?
 
+!!chosen design
+My main goal is to have an easy-to-use interface for the implementer of individual classes using this factory mechanism. The intended use should mimic the standard use of operator new, and at the same time there should be a possibility to configure »real« Factory behaviour in.
+
+For this reason I make Factory a Functor, and for handling the concrete memory allocation, it inherits from an Allocator class. The users of this factory template typically either parametrize it with existing smart-pointer types and some Allocator instantiation, or they may chose to create a specialized Factory derived from this Factory template. After typically hiding this configuration behind a typedef, the user adds a static field to the class intended to use the Factory and initializes this field with the concrete Factory (this may pass internal ~IDs to the constructor of the Factory and from there on to the underlying Allocator).
+{{{
+#include "common/factory.hpp"
+
+  class Blubb
+    {
+      int uii ;
+
+    public:
+      typedef cinelerra::factory::RefcountPtr<Blubb> Factory;
+      static Factory create;
+      
+      Blubb() : uii(42) {} ;
+    };
+  
+  /** a static Factory instance 
+   *  for creating refcounting Ptrs to Blubb objects 
+   */
+  Blubb::Factory Blubb::create;  // <----note this is a constructor call
+}}}
+Now, the clients of {{{class Blubb}}} can create ref-counting pointers to Blubb-objects by doing a fully qualified {{{create()}}} functor call:
+{{{
+  std::tr1::shared_ptr<Blubb> huii = Blubb::create ();  // <----will invoke the default Blubb() ctor 
+  std::tr1::shared_ptr<Blubb> pfuii = huii;
+}}}
+Some further details
+* Functor inherits (protected) from Allocator, besides that there are no additional requirements.
+* typically, Allocator provides a overloaded {{{operator new(size_t)}}} etc.
+* thus, when Functor or any derived class issues a new XX(), our custom Allocator gains control
+* the Functor-behaviour relies on a custom {{{operator()}}} which can be overridden in derived classes to take various parameter lists.
+* then, such a Factory class derived from Functor can do specific magic and e.g. create some subclass
+* and, as the created smart-pointer is a template parameter, such a custom Functor can create all sorts of Proxies, wrappers and the like
From 45c21677009dfc733d0ecd6f26d783c99b2818d5 Mon Sep 17 00:00:00 2001 From: Ichthyostega Date: Mon, 13 Aug 2007 09:55:32 +0200 Subject: [PATCH 3/7] wrote a very simple Test-Suite runner and provided a Tests source tree --- .gitignore | 1 + SConstruct | 6 +- admin/scons/Buildhelper.py | 29 +++-- src/common/factory.hpp | 13 +++ src/test/common/factorytest.cpp | 54 +++++++++ src/test/common/helloworldtest.cpp | 55 ++++++++++ src/test/helper/run.hpp | 92 ++++++++++++++++ src/test/helper/suite.cpp | 171 +++++++++++++++++++++++++++++ src/test/helper/suite.hpp | 61 ++++++++++ src/test/mainsuite.cpp | 36 ++++++ tests/SConscript | 16 +++ tests/test.sh | 154 ++++++++++++++++++++++++++ wiki/index.html | 77 ++++++++++--- wiki/renderengine.html | 4 +- 14 files changed, 743 insertions(+), 26 deletions(-) create mode 100644 src/test/common/factorytest.cpp create mode 100644 src/test/common/helloworldtest.cpp create mode 100644 src/test/helper/run.hpp create mode 100644 src/test/helper/suite.cpp create mode 100644 src/test/helper/suite.hpp create mode 100644 src/test/mainsuite.cpp create mode 100644 tests/SConscript create mode 100755 tests/test.sh diff --git a/.gitignore b/.gitignore index ed690f89c..653bd410b 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,5 @@ /wiki/backups/* +/tests/* *~ *.tar.* .sconf_temp diff --git a/SConstruct b/SConstruct index 0c2271198..4409b9a9b 100644 --- a/SConstruct +++ b/SConstruct @@ -32,6 +32,7 @@ OPTIONSCACHEFILE = 'optcache' CUSTOPTIONSFILE = 'custom-options' SRCDIR = 'src' BINDIR = 'src/bin' +TESTDIR = 'tests' VERSION = '3+alpha.01' #-----------------------------------Configuration @@ -57,7 +58,7 @@ def setupBasicEnvironment(): env.Replace( VERSION=VERSION , SRCDIR=SRCDIR , BINDIR=BINDIR - , CPPPATH=SRCDIR # used to find includes + , CPPPATH="#"+SRCDIR # used to find includes, "#" means always absolute to build-root ) appendCppDefine(env,'DEBUG','DEBUG') @@ -196,6 +197,7 @@ def defineBuildTargets(env, artifacts): # + srcSubtree(env,'lib') + env.Object('$SRCDIR/main.cpp') ) + testobj = srcSubtree(env,'test/*', isShared=False) plugobj = srcSubtree(env,'plugin', isShared=True) artifacts['cinelerra'] = env.Program('$BINDIR/cinelerra', cinobj) @@ -203,7 +205,7 @@ def defineBuildTargets(env, artifacts): # call subdir SConscript(s) for independent components SConscript(dirs=[SRCDIR+'/tool'], exports='env artifacts') - + SConscript(dirs=[TESTDIR], exports='env artifacts testobj') def defineInstallTargets(env, artifacts): diff --git a/admin/scons/Buildhelper.py b/admin/scons/Buildhelper.py index 99b84c997..85da4784e 100644 --- a/admin/scons/Buildhelper.py +++ b/admin/scons/Buildhelper.py @@ -23,6 +23,7 @@ import os import sys +import glob import fnmatch import re import tarfile @@ -62,16 +63,28 @@ def srcSubtree(env,tree,isShared=False, **args): SRCPATTERNS = ['*.c','*.cpp','*.cc'] -def scanSrcSubtree(root): - """ scan the given subtree for source filesnames +def scanSrcSubtree(roots): + """ first expand (possible) wildcards and filter out non-dirs. + Then scan the given subtree for source filesnames (python generator function) """ - for (dir,_,files) in os.walk(root): - if dir.startswith('./'): - dir = dir[2:] - for p in SRCPATTERNS: - for f in fnmatch.filter(files, p): - yield os.path.join(dir,f) + for root in globRootdirs(roots): + for (dir,_,files) in os.walk(root): + if dir.startswith('./'): + dir = dir[2:] + for p in SRCPATTERNS: + for f in fnmatch.filter(files, p): + yield os.path.join(dir,f) + + + +def globRootdirs(roots): + """ helper: expand shell wildcards and filter the resulting list, + so that it only contains existing directories + """ + filter = lambda f: os.path.isdir(f) and os.path.exists(f) + roots = glob.glob(roots) + return (dir for dir in roots if filter(dir) ) diff --git a/src/common/factory.hpp b/src/common/factory.hpp index b3f9faeb7..8664675c8 100644 --- a/src/common/factory.hpp +++ b/src/common/factory.hpp @@ -107,6 +107,19 @@ namespace cinelerra }; + /** another convienience instantiiation: auto_ptr-Factory, + * actually creating a subclass of the returned type + */ + template + class SubclassPtr : public Factory + { + typedef std::auto_ptr aP; + + public: + aP operator() (){ return aP (new TImpl ); }; + }; + + } // namespace factory diff --git a/src/test/common/factorytest.cpp b/src/test/common/factorytest.cpp new file mode 100644 index 000000000..61522b66f --- /dev/null +++ b/src/test/common/factorytest.cpp @@ -0,0 +1,54 @@ +/* + Factory(Test) - unittest for the object creating factory + + Copyright (C) CinelerraCV + 2007, Christian Thaeter + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + +* *****************************************************/ + + +#include "test/helper/run.hpp" + + +namespace cinelerra + { + namespace test + { + + /** + * Target object to be created by the Test-Factory + */ + class TargetObj + { + public: + }; + + class Factory_test : public Test + { + virtual void run(Arg arg) + { + } + }; + + /** Register this test class to be invoked in some test groups (suites) */ + Launch run_Factory_test("Factory_test","unit common"); + + + + } // namespace test + +} // namespace cinelerra diff --git a/src/test/common/helloworldtest.cpp b/src/test/common/helloworldtest.cpp new file mode 100644 index 000000000..877830989 --- /dev/null +++ b/src/test/common/helloworldtest.cpp @@ -0,0 +1,55 @@ +/* + HelloWorld(Test) - how to use this test framework... + + Copyright (C) CinelerraCV + 2007, Christian Thaeter + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + +* *****************************************************/ + + +#include +#include "test/helper/run.hpp" + + +namespace cinelerra + { + namespace test + { + + + class HelloWorld_test : public Test + { + virtual void run(Arg arg) + { + greeting(); + } + + void greeting() + { + std::cout << "This is how the world ends...\n\n"; + } + }; + + + + /** Register this test class to be invoked in some test groups (suites) */ + Launch run_HelloWorld_test("HelloWorld_test","unit common"); + + + } // namespace test + +} // namespace cinelerra diff --git a/src/test/helper/run.hpp b/src/test/helper/run.hpp new file mode 100644 index 000000000..4d5cfb0b3 --- /dev/null +++ b/src/test/helper/run.hpp @@ -0,0 +1,92 @@ +/* + RUN.hpp - helper class for grouping, registering and invoking testcases + + Copyright (C) CinelerraCV + 2007, Christian Thaeter + Hermann Vosseler + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + +*/ + + +#ifndef TESTHELPER_RUN_H +#define TESTHELPER_RUN_H + +#include +#include +#include "common/factory.hpp" +#include "test/helper/suite.hpp" + + +namespace test + { + + using std::string; + using std::auto_ptr; + using cinelerra::Factory; + using cinelerra::factory::SubclassPtr; + + typedef std::vector * Arg; + + + + /** + * Abstract Base Class for all testcases. + * Typically, such testcases are created by a Launcher. + */ + class Test + { + public: + virtual void run(Arg arg) = 0; + }; + + + + /** interface: generic testcase creating functor. */ + class Launcher + { + public: + virtual auto_ptr operator() () = 0; + }; + + + /** + * Helper class for running a collection of tests. + * Launch objects are functors, which create on + * invocation an instance of the Test class + * they were created with. Creating such a + * Test Launcher internally registers this + * testcase with the Testsuite-Class, + * optionally under several groups + * (=categories, suite selections). + * @note a subclass of Launcher + */ + template + class Launch : public Launcher + { + public: + Launch (string testID, string groups) { Suite::enroll (this,testID,groups); }; + virtual auto_ptr operator() () { return auto_ptr (new TEST ); }; + }; + +} // namespace test + +// make them global for convienience +using ::test::Arg; +using ::test::Test; +using ::test::Launch; + +#endif diff --git a/src/test/helper/suite.cpp b/src/test/helper/suite.cpp new file mode 100644 index 000000000..84fb7148c --- /dev/null +++ b/src/test/helper/suite.cpp @@ -0,0 +1,171 @@ +/* + Suite - helper class for running collections of tests + + Copyright (C) CinelerraCV + 2007, Christian Thaeter + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + +* *****************************************************/ + + +#include +#include +#include +#include +#include "test/helper/suite.hpp" +#include "test/helper/run.hpp" + +#include /////////////////////////TODO: Debug +#include + +namespace test + { + using std::map; + using std::vector; + using std::auto_ptr; + using std::tr1::shared_ptr; + + typedef map TestMap; + typedef shared_ptr PTestMap; + typedef map GroupMap; + + + /** helper to collect and manage the test cases. + * Every testcase class should create a Launch instance + * which causes a call to Suite::enroll(), so we can add a + * pointer to this Launcher into a map indexed by the + * provided testIDs and groupIDs. + * This enables us to build a Suite instance for any + * requested group and then instantiiate and invoke + * individual testcases accordingly. + */ + class Registry + { + auto_ptr groups; + public: + Registry() : groups(new GroupMap ) {}; + PTestMap& getGroup (string grpID) { return (*groups)[grpID]; }; + void add2group (Launcher* test, string testID, string groupID); + }; + + void Registry::add2group (Launcher* test, string testID, string groupID) + { + // TODO: ASSERT test!=null, testID.length > 0 ... + PTestMap& group = getGroup(groupID); + if (!group) + group.reset( new TestMap ); + (*group)[testID] = test; + } + + Registry testcases; + + + + + /** register the given test-launcher, so it can be later accessed + * either as a member of one of the specified groups, or direcly + * by its testID. Any test is automatically added to the groupID + * #ALLGROUP + * @param groups List of group-IDs selected by whitespace + */ + void Suite::enroll (Launcher* test, string testID, string groups) + { + // TODO learn to use NoBug for logging + std::cerr << "enroll( testID=" << testID << ")\n"; + // TODO: ASSERT test!=null, testID.length() > 0... + + std::istringstream ss(groups); + string group; + while (ss >> group ) + testcases.add2group(test, testID, group); + + // Magic: allways add any testcas to groupID="ALL" + testcases.add2group(test,testID, ALLGROUP); + } + + /** "magic" groupID containing all registered testcases */ + const string Suite::ALLGROUP = "ALL"; + + + + /** create a suite comprised of all the testcases + * previously @link #enroll() registered @endlink with this + * this group. + * @see #run() running tests in a Suite + */ + Suite::Suite(string groupID) + : groupID_(groupID) + { + std::cerr << "Suite( groupID="<< groupID << ")\n"; + if (!testcases.getGroup(groupID)) + throw "empty testsuite"; /////////// TODO Errorhandling! + } + + + /** run all testcases contained in this Suite. + * The first argument in the commandline, if present, will select + * one single testcase with a matching ID. + */ + void Suite::run (int argc, char* argv[]) + { + /////////////////////////////////////////////////////TODO:DEBUG + std::cerr << "Suite::run( (" << argc << "[" ; + for ( int i=0; i= 2) + { + if (Launcher* test = (*tests)[argv[1]]) + { + // first cmdline argument denotes a valid + // testcase registered in this group: + // go ahead and invoke just this test. + if (argc > 2) + { // pass additional cmdline as vector + vector arglist(argc-2); + for ( int i=2; irun(&arglist); + } + else + (*test)()->run(0); // without additional argumens + + return; + } + } + + // no test-ID was specified. + // instantiiate all tests cases and execute them. + for ( TestMap::iterator i=tests->begin(); i!=tests->end(); ++i ) + if (i->second) + { + std::cout << " ----------"<< i->first<< "----------\n"; + Launcher& test = *(i->second); + test()->run(0); // without cmdline arguments + } + + } + + + +} // namespace test diff --git a/src/test/helper/suite.hpp b/src/test/helper/suite.hpp new file mode 100644 index 000000000..6dfefe663 --- /dev/null +++ b/src/test/helper/suite.hpp @@ -0,0 +1,61 @@ +/* + SUITE.hpp - helper class for running collections of tests + + Copyright (C) CinelerraCV + 2007, Christian Thaeter + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. + +*/ + + +#ifndef TESTHELPER_SUITE_H +#define TESTHELPER_SUITE_H + +#include +#include "common/factory.hpp" + + + +namespace test + { + using std::string; + + // Forward decls needed for run.hpp + class Test; + class Launcher; + + + + /** + * Helper class for running a collection of tests. + * + */ + class Suite + { + string groupID_; + + public: + Suite (string groupID); + void run (int argc, char* argv[]); + static void enroll (Launcher *test, string testID, string groups); + + static const string ALLGROUP; + }; + + + +} // namespace test +#endif diff --git a/src/test/mainsuite.cpp b/src/test/mainsuite.cpp new file mode 100644 index 000000000..729f55e6b --- /dev/null +++ b/src/test/mainsuite.cpp @@ -0,0 +1,36 @@ +/* + mainsuite.cpp - "the" cinelerra3 self test suite + + Copyright (C) CinelerraCV + 2007, Christian Thaeter + Hermann Vosseler + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. +*/ + + +#include "test/helper/suite.hpp" + + +/** run all tests or any single test specified in the first + * cmd line argument. + * Note: to ease debugging, we don't catch any exceptions. + */ +int main (int argc, char* argv[]) + { + test::Suite suite (test::Suite::ALLGROUP); + suite.run(argc,argv); + return 0; + } diff --git a/tests/SConscript b/tests/SConscript new file mode 100644 index 000000000..a5fe4a697 --- /dev/null +++ b/tests/SConscript @@ -0,0 +1,16 @@ +# -*- python -*- +## +## SConscript - SCons buildscript for the Testsuite (called by SConstruct) +## + +Import('env','artifacts','testobj') + +# build an application running the testsuite +artifacts['testsuite'] = env.Program('mainsuite',testobj + ['#$SRCDIR/test/mainsuite.cpp']) + +# TODO: we could apply much more "magic" here +# - build /every/ $TESTDIR/*.cpp into an application +# - link the testobj dynamically +# - install additionally scripts into tests-dir if desired + + diff --git a/tests/test.sh b/tests/test.sh new file mode 100755 index 000000000..30516cfde --- /dev/null +++ b/tests/test.sh @@ -0,0 +1,154 @@ +#!/bin/bash +# +# tests/test.sh - run all defined automatic tests +# +# +# Copyright (C) CinelerraCV +# 2007, Christian Thaeter +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation; either version 2 of the +# License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. +# +################################################################### + + +# TESTMODE=FULL yet unimplemented +# run all tests, PLANNED which fail count as error +# +# TESTMODE=FAST +# run only tests which recently failed + +arg0="$0" +srcdir=$(dirname "$arg0") + +ulimit -S -t 1 -v 524288 +valgrind="" +if [ "$VALGRINDFLAGS" = 'DISABLE' ]; then + echo "valgrind explicit disabled" +else + if [ "$(which valgrind)" ]; then + valgrind="$(which valgrind) --suppressions=$srcdir/../valgrind.sup --leak-check=yes --show-reachable=yes -q $VALGRINDFLAGS" + ulimit -S -t 10 + else + echo "no valgrind found, go without it" + fi +fi + +echo +echo ================ $0 ================ + +TESTCNT=0 +SKIPCNT=0 +FAILCNT=0 + +if test -f ,testlog; then + mv ,testlog ,testlog.pre +else + touch ,testlog.pre +fi + +date >,testlog + +function TEST() +{ + name="$1" + shift + cat >,cmp + echo -n "" >,out + echo -n "TEST $name: " + echo -en "\nTEST $name: $* " >>,testlog + + case $TESTMODE in + *FAST*) + if grep "^TEST $name: .* FAILED" ,testlog.pre >&/dev/null; then + MSGOK=" (fixed)" + MSGFAIL=" (still broken)" + elif grep "^TEST $name: .* \\(SKIPPED (ok)\\|OK\\)" ,testlog.pre >&/dev/null; then + echo ".. SKIPPED (ok)" + echo ".. SKIPPED (ok)" >>,testlog + SKIPCNT=$(($SKIPCNT + 1)) + TESTCNT=$(($TESTCNT + 1)) + return + else + MSGOK=" (new)" + MSGFAIL=" (new)" + fi + ;; + *) + MSGOK="" + MSGFAIL="" + ;; + esac + + TESTCNT=$(($TESTCNT + 1)) + if $valgrind $TESTBIN "$@" 2>&1 | tee ,tmp | grep -v ': \(TRACE\|INFO\|NOTICE\|WARNING\|ERR\):' | cmp ,cmp - &>/dev/null; then + echo ".. OK$MSGOK" + echo ".. OK$MSGOK" >>,testlog + else + echo ".. FAILED$MSGFAIL"; + echo ".. FAILED$MSGFAIL" >>,testlog + grep -v ': \(TRACE\|INFO\|NOTICE\|WARNING\|ERR\):' <,tmp >,out + diff -ua ,cmp ,out >>,testlog + # grep 'DEBUG:\|==.*==' <,tmp >>,testlog + cat ,tmp >>,testlog + echo END >>,testlog + FAILCNT=$(($FAILCNT + 1)) + case $TESTMODE in + *FIRSTFAIL*) + break 2 + ;; + esac + fi +} + +function PLANNED() +{ + echo -n "PLANNED $1: " + echo -en "\nPLANNED $* " >>,testlog + echo ".. SKIPPED (planned)" + echo ".. SKIPPED (planned)" >>,testlog + SKIPCNT=$(($SKIPCNT + 1)) + TESTCNT=$(($TESTCNT + 1)) +} + +function RUNTESTS() +{ + for i in $srcdir/*.tests; do + source $i + done + echo + rm ,cmp ,out ,tmp + if [ $FAILCNT = 0 ]; then + echo " ... PASSED $(($TESTCNT - $SKIPCNT)) TESTS, $SKIPCNT SKIPPED" + #rm ,testlog + else + echo " ... SUCCEDED $(($TESTCNT - $FAILCNT - $SKIPCNT)) TESTS" + echo " ... FAILED $FAILCNT TESTS" + echo " ... SKIPPED $SKIPCNT TESTS" + echo " see ',testlog' for details" + exit 1 + fi +} + +function TESTING() +{ + echo + echo "$1" + TESTBIN=$2 +} + +RUNTESTS + +# arch-tag: f4d06a47-6e17-40de-bba8-17240ae3f435 + diff --git a/wiki/index.html b/wiki/index.html index 5f979a5b1..517fd7095 100755 --- a/wiki/index.html +++ b/wiki/index.html @@ -42,7 +42,7 @@ DAMAGE. -
My TiddlyWiki is loading ...

Requires Javascript.
+
Cinelerra TiddlyWiki is loading ...

Requires Javascript.
Cinelerra3 - Distributed Developer Wiki