091785c2d4
This reverts commit 65bae31de4103abb7d7b6fd004a8315973d3144a. and reprocessed the wrapping. Note that the automatic wrapping is not perfect, some manual fixing by removing some hunks was required.
427 lines
19 KiB
Text
427 lines
19 KiB
Text
Design Process : Mistakes to avoid
|
|
==================================
|
|
|
|
[grid="all"]
|
|
`------------`-----------------------
|
|
*State* _Dropped_
|
|
*Date* _2008-04-21_
|
|
*Proposed by* link:rick_777[]
|
|
-------------------------------------
|
|
|
|
|
|
Mistakes to avoid in the Lumiera design
|
|
---------------------------------------
|
|
|
|
|
|
As a multimedia user and experienced programmer, I've found various flaws
|
|
present in open source Non Linear Video editors. Here I will list the problems
|
|
and their proposed (or mandatory) solutions. Please forgive me if some of the
|
|
ideas here have already been approved, I wrote this text before reaching this
|
|
wiki.
|
|
|
|
|
|
Description
|
|
~~~~~~~~~~~
|
|
|
|
|
|
As a multimedia user and experienced programmer, I've found the following flaws
|
|
present in open source Non Linear Video editors (your mileage may vary) :
|
|
|
|
. Frequent crashes (which most of the time make you lose your work)
|
|
. Reinventing the wheel for every new project
|
|
. Lack of a user-friendly (and extensible) UI
|
|
. Lack of support for certain video formats or codecs
|
|
. Lack of documentation
|
|
. Lack of cross-platform support
|
|
. Dependency on scripted languages like Python, which make installation a mess
|
|
|
|
I will expand on the problems and their proposed (or mandatory) solutions.
|
|
|
|
|
|
1. Frequent crashes
|
|
~~~~~~~~~~~~~~~~~~~
|
|
|
|
[grid="all"]
|
|
`------------`------------------------------------------------------
|
|
*Problem* Frequent Crashes and unsaved work.
|
|
*Severity* CRITICAL.
|
|
*Solution* Isolating the UI from the rendering and data handling (also
|
|
improves the extensibility)
|
|
*Required* Yes
|
|
*Workarounds* Auto-save (however it's not a real solution for the problem)
|
|
--------------------------------------------------------------------
|
|
|
|
Working with multimedia (video / audio) editing is a magnet for segfaults
|
|
(crashes) due to the handling of pointers and compression algorithms. A bug in
|
|
a plugin (like in Audacity's low-pass filter) will crash and you suddenly
|
|
realize you lost your work - unless you have an auto-save feature, but that
|
|
doesn't go to the root of the problem.
|
|
|
|
My proposal is to move the low-level handling of video to a separate process,
|
|
which then will do the processing - if it crashes, the UI will only report an
|
|
error with a dialog (i.e. "the process crashed. Try again?"), but you work will
|
|
stay safe. I'm not sure of the implementation difficulties that arise from
|
|
having a shared memory buffer for rendering / processing, but one thing is
|
|
certain: Whenever you move the cursor or rewind a part of a clip in your
|
|
resources, the application isn't supposed to crash. Just moving the cursor
|
|
isn't a time-critical task, so perhaps we can use temporary files for this.
|
|
It's safer if you're not doing the final rendering.
|
|
|
|
|
|
Comments
|
|
^^^^^^^^
|
|
|
|
I am not sure yet about separating things into processes, generally it is clear
|
|
that this would be more robust but there are some performance impacts and
|
|
programming problems (massisve amounts of data in shared memory). But most
|
|
importantly, when a subprocess gets a job and crashes on it, it won't complete
|
|
the job, we don't have a choice except gracefully abort it. From a user
|
|
perspective "It doesn't work!" there is no much difference to a complete crash.
|
|
Well and yes we aim to make it crash proof rather, crashes a bugs and have to
|
|
be fixed, point.
|
|
|
|
Lumiera will never ever loose work, we don't plan to make a project file,
|
|
autosafe way. Lumiera will keep projects in an internal database like format
|
|
which consists of a Dumpfile and a contingous written logfile. After a
|
|
crash/powerdown whatever, this log just gets replayed. The advantages are
|
|
countless, imagine persistent, selective undo and so on. Any other format
|
|
(cinelerra2 XML, MXF, ...) will be realized by importer/exporter plugins.
|
|
-- link:ct[] [[DateTime(2008-04-21T11:27:23Z)]]
|
|
|
|
|
|
|
|
2. Reinventing the wheel for every new project
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
[grid="all"]
|
|
`------------`------------------------------------------------------
|
|
*Problem* Various projects compete and reinvent the wheel
|
|
*Severity* Serious (Slows down development time. A lot)
|
|
*Solution* Multi-tier design, turn the data handling into a backend and use
|
|
whatever UI you prefer
|
|
*Required* Yes. Better now that the project hasn't started
|
|
---------------------------------------------------------------------
|
|
|
|
Imagine the Linux kernel was tied to the window manager. You would have to
|
|
stick with KDE or GNOME and you couldn't improve it! Fortunately it's not like
|
|
that for Linux, but it is for some projects. If you want to change the user
|
|
interface from QT to wxWidgets or GTK you'll need to rewrite every module.
|
|
|
|
If you separate the UI from the project handling engine, you can simply fork
|
|
the project and change the UI to one that supports skinning, without having to
|
|
do the complicated video-processing stuff.
|
|
|
|
Separating the processes has an equivalent for web programming, it's called
|
|
"separation of concerns", or multi-tier design. When you suddenly change the
|
|
database engine, you don't need to change the whole program, just the database
|
|
module. Same goes for changing the UI from HTML to XML or Flash. If they're
|
|
separate modules that only communicate through a clearly defined API.
|
|
|
|
Example case 1: The Code::Blocks IDE. The compiling engine supports various
|
|
compilers, and the engine itself is only a plugin for the main editor. If the
|
|
compiler crashes, you only get an error, but the IDE doesn't crash (unless it's
|
|
the UI part that's doing something wrong).
|
|
|
|
Example case 2: Chessmaster. The user interface and speech synthesis stuff only
|
|
call the chess engine, called "theking.exe". Linux chess games also depend on
|
|
an engine to do the thinking.
|
|
|
|
So I suggest to split the project into four separate tiers (not necessarily
|
|
processes):
|
|
|
|
. User interface - communicates with the "project" tier, handles the user
|
|
events and does the calls.
|
|
. The project tier - the main part of the video editor. This one invokes the
|
|
renderer and decides which effects to apply, saving them as mere parameters
|
|
for later processing. It also tells you where the current pointer for the
|
|
track view is. Also calls the rendering engine for the current frame, or for
|
|
previews of a certain special effect. Note that if this process keeps running
|
|
even if the GUI crashes, later we can restart the GUI and keep working.
|
|
. The rendering engine - This one must be a separate process for the reasons
|
|
stated in problem #1. This also gives us the advantage that it can work on
|
|
the background while we keep working on the project (after all the project is
|
|
just a set of data stating which effects to apply to which tracks, and which
|
|
files are used for the tracks) - instead of just having a window saying
|
|
"Rendering, please wait". Even Adobe Premiere Pro suffered from this problem.
|
|
This means that if we put enough effort, we can surpass commercial software
|
|
in certain areas. Note that the rendering engine uses the same API than the
|
|
project tier, as it works on a copy of the project when doing the final
|
|
rendering.
|
|
. The video processing wrapper, which has interfaces for different video
|
|
processing toolkits (DirectX, GStreamer, etc). This also makes the project
|
|
cross-platform. Tiers 1 and 2 can go in one process, and the 3 and 4 in
|
|
another (this would make tier 2 a library which defines a C++ Class, and tier
|
|
4 would also be a library which is used by the rendering engine).
|
|
|
|
By separating the tiers, these can later become their own projects and overall
|
|
the community would receive great benefits.
|
|
|
|
|
|
Comments
|
|
^^^^^^^^
|
|
|
|
Please look at our design drafts, things will be separated (little different
|
|
than you describe here). We reuse things which are benefitful (gavl, ffmpeg,
|
|
..) but we are also aware that we reinvent the wheel for some things by
|
|
intention. Lumieras goal is not just to glue some existing libraries together
|
|
under a new gui, there are already a lot projects trying this way. We rather
|
|
aim for a ''Professional'' high performance Video editing solution which does
|
|
some things in a different (maybe more complex) way. We do not use existing
|
|
frameworks like MLT or gstreamer because we believe that these do not fit our
|
|
goals (gstreamer will be supported through plugins). We do not produce yet
|
|
another multimedia framework library (this only happen by coincidence) to be
|
|
used by others.
|
|
-- link:ct[] [[DateTime(2008-04-21T11:27:23Z)]]
|
|
|
|
|
|
3. Lack of a user-friendly and extensible UI.
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
[grid="all"]
|
|
`------------`------------------------------------------------------
|
|
*Problem* Often, editors provide a very poor and buggy interface.
|
|
Examples: Jahshaka doesn't even provide tooltips for the various tools, and
|
|
the documentation is poor; In Cinelerra I've noticed some bugs when using the
|
|
open dialog, I'd rather have the KDE one, thanks.
|
|
*Severity* From Annoying to Serious.
|
|
*Solution 1* Use a library that allows you to use different widget
|
|
libraries, like wxWidgets.
|
|
*Required* Recommended, but not obligatory.
|
|
*Solution 2* Write different user interfaces, but they'd be hard to maintain.
|
|
*Required*, No.
|
|
---------------------------------------------------------------------
|
|
|
|
This problem is complicated, we need a good framework for handling the tracks.
|
|
Perhaps this could become a separate project. Ideas are welcome.
|
|
|
|
|
|
Comments
|
|
^^^^^^^^
|
|
|
|
Joel started working on a GUI recently and making good progress. The UI should
|
|
finally be quite flexible as it mostly provides a skeletion where plugins
|
|
render to. We have quite a lot ideas about the UI and user input is welcome.
|
|
The UI is currently the most separate tier in the design, i'd like to make it a
|
|
plugin itself which is loaded when lumiera is started in a gui mode, but it is
|
|
to early to say how exactlly it will be integrated, except that we all agree
|
|
that GUI is optional and Lumiera can also run headless, script driven.
|
|
-- link:ct[] [[DateTime(2008-04-21T11:27:23Z)]]
|
|
|
|
|
|
4. Lack of support for certain video formats or codecs
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
[grid="all"]
|
|
`------------`------------------------------------------------------
|
|
*Problem* Lack of support for certain video formats or codecs.
|
|
*Severity* Critical.
|
|
*Workarounds* 1. Give a help page for the user to do his own conversion, but
|
|
this is very inelegant, annoying, and a waste of time. 2. Provide conversion
|
|
on the fly, and keep a separate "preprocessed" copy of the imported clip in a
|
|
separate directory. This is a nice middle ground, IMHO.
|
|
*Solution* Use a wrapper library as stated in problem # 2, having a
|
|
plugin-based design is recommended.
|
|
*Required* Yes.
|
|
---------------------------------------------------------------------
|
|
|
|
Some editors like Cinelerra are hardwired into using one format, or have a
|
|
phobia to certain formats / codecs (i.e. DivX AVI's). If we separate the
|
|
project editing engine from the video handling libraries, we can use unknown
|
|
formats by simply providing an input/output plugin. This would allows us to use
|
|
files encoded with lossless codecs like
|
|
http://lags.leetcode.net/codec.html[Lagarith]. This also provides forward
|
|
compatibility for future formats.
|
|
|
|
|
|
Comments
|
|
^^^^^^^^
|
|
|
|
Lumiera is a video editor we don't care (*cough*, not really true) about video
|
|
formats. Everything which comes In and goes Out is defined in plugins which
|
|
handle video formats. We currently decided to use 'gavl' because it is a nice
|
|
small library which does exactly what we want. Later on gstreamer and other
|
|
such kinds of decoder/encoder/processing-pipe libs will be realized.
|
|
-- link:ct[] [[DateTime(2008-04-21T11:27:23Z)]]
|
|
|
|
|
|
5. Lack of documentation
|
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
[grid="all"]
|
|
`------------`------------------------------------------------------
|
|
*Problem* Some video editors have very poor documentation (and that's an
|
|
understatement *cough* Jahshaka *cough* )
|
|
*Severity* Critical.
|
|
*Solution* Have a team for the documentation.
|
|
*Required* Yes.
|
|
---------------------------------------------------------------------
|
|
|
|
Nuff said.
|
|
|
|
|
|
Comments
|
|
^^^^^^^^
|
|
|
|
Quote from Ohloh.net: (http://www.ohloh.net/projects/lumiera)[]
|
|
|
|
------------------------------------------------------------
|
|
Extremely well-commented source code
|
|
|
|
Lumiera is written mostly in C++. Across all C++ projects on Ohloh, 23% of all
|
|
source code lines are comments. For Lumiera, this figure is 46%. This very
|
|
impressive number of comments puts Lumiera among the best 10% of all C++
|
|
projects on Ohloh.
|
|
------------------------------------------------------------
|
|
|
|
|
|
Nuff saied... Oh well, about user docs we like to get that impressive ratings
|
|
there too, any helpers?
|
|
-- link:ct[] [[DateTime(2008-04-21T11:27:23Z)]]
|
|
|
|
|
|
6. Lack of cross-platform support
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
[grid="all"]
|
|
`------------`------------------------------------------------------
|
|
*Problem* Where's my Windows version?
|
|
*Severity* Blocker
|
|
*Solution* Use a cross-platform toolkit for the UI.
|
|
*Required* Depends, do you plan to make it Cross-Platform?
|
|
--------------------------------------------------------------------
|
|
|
|
A good example for this is the Code::Blocks IDE, which was thought of being
|
|
cross-platform from the beginning. Curiously, at first the project was
|
|
Windows-only, and its only F/OSS alternative was Dev-C++ from Bloodshed (eew).
|
|
Otherwise you'd have to stick with proprietary applications like Visual C++.
|
|
|
|
In Linux there were various IDE's, but they were Linux-only. Since Code::Blocks
|
|
uses a cross-platform toolkit (wxWidgets), it can be compiled either in Windows
|
|
and Linux. There are RPM's for various distros now that the first public
|
|
version (8.02) got out. I've heard that QT is also cross-platform, but I
|
|
haven't tried it yet.
|
|
|
|
Of course - if you separate the UI from the project engine, someone could make
|
|
his own Windows UI for the project. Now what needs to be taken care of, is that
|
|
the rendering libraries are cross-platform too.
|
|
|
|
|
|
Comments
|
|
^^^^^^^^
|
|
|
|
We refuse to make it cross platform intentionally. Most things are written
|
|
portable, POSIX compatible, some might need platform specific fixes. But our
|
|
target is primary Linux (because thats what we use) secondary any other Free OS
|
|
(hopefully we find some testers/maintainers for that). Lumiera ''might'' run on
|
|
OSX and patches will be accepted, but it is not a free platform so we don't
|
|
care by ourself. Windows due its diffrent system interfaces will be hard to
|
|
port, if someone wants to do that, have fun, we will accept patches to, but we
|
|
do not support it in *any* way by ourself.
|
|
-- link:ct[] [[DateTime(2008-04-21T11:27:23Z)]]
|
|
|
|
|
|
7. Dependency on scripted languages like Python, which make installation a mess
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
[grid="all"]
|
|
`------------`------------------------------------------------------
|
|
*Problem* Installation can be a mess if we depend on scripted languages.
|
|
*Severity* Annoying, the end user might just conform with another project
|
|
that "just works".
|
|
*Solution* Make it in C++ or other easily-compilable language.
|
|
*Required* VERY recommended.
|
|
---------------------------------------------------------------------
|
|
|
|
I've had to install several packages for my distro (whose repository is not as
|
|
large as others like Ubuntu's) from source. Some of them depend on very
|
|
esoteric scripted languages which I also need to install. And then the
|
|
libraries, etc. My suggestion is to free the end user from this burden, and
|
|
work on a common language, like C++.
|
|
|
|
|
|
Comments
|
|
^^^^^^^^
|
|
|
|
At some point a scripting language ''will'' be required, yet to drive the
|
|
testsuite, make headless rendering work and so on. We need to provide
|
|
installation instructions and/or even bundle this language with Lumiera. This
|
|
will likely become a small embedded language like Lua or some kind of forth (or
|
|
maybe some scheme?) it should not depend on strange modules which are not part
|
|
of the core scripting language distribution (or we shall provide them too),
|
|
needs to be worked out.
|
|
-- link:ct[] [[DateTime(2008-04-21T11:27:23Z)]]
|
|
|
|
|
|
|
|
Author's comments
|
|
^^^^^^^^^^^^^^^^^
|
|
|
|
Some of the measures stated in this document are optional, but separating the
|
|
processes for the rendering engine, editor and User Interface are the optimal
|
|
solution and required to avoid common problems.
|
|
|
|
|
|
Discussion
|
|
----------
|
|
|
|
Mostly we agree with the general statements in this Design Entry. But there are
|
|
some points which don't stand the test of a detailed technical discussion. For
|
|
example, you simply can't state it's a 'mistake' not to write code which
|
|
similarly runs on windows and *nix. Well. You could try to write it in Java.
|
|
See my point? While today it's quite feasible to write office stuff or banking
|
|
applications in a cross-platform manner, a video editor still is a different
|
|
kind of a beast.
|
|
|
|
A similar argumentation holds true for the question, wether or not to use
|
|
separate processes and IPC. While it certainly is a good idea to have the X
|
|
server or a database running in a separate process, the situation is really
|
|
quite different for editing video. Hopefully it's clear why.
|
|
|
|
Could you please rework this Design Entry in a way that we can finalize
|
|
(accept) it?
|
|
|
|
* Please remove the section about windows
|
|
* Please separate out things needing technical discussion and are not just
|
|
"mistakes", thus retaining only the big picture statements (on which we all
|
|
agree)
|
|
* How to secure the application against crashes
|
|
* If it is viable/desirable to run the gui in a separate process really needs
|
|
in-depth technical discussion (create a new Design Entry for this)
|
|
* How to deal with the dependencies problem in combination with
|
|
plugins/extensions and script languages
|
|
-- link:Ichthyostega[] [[DateTime(2008-10-05T01:51:50Z)]]
|
|
|
|
|
|
Conclusion
|
|
----------
|
|
The October.2008 dev meeting decided to 'drop' this design proposal as is.
|
|
|
|
Basically, this text just tells us "to make Lumiera good", and especially it
|
|
contains a mixture of topics
|
|
|
|
* We fully agree to 80% of the statements made there, but we think those
|
|
statements are so very basic and self-evident as to be considered off-topic
|
|
here. We are aware of the recurring problems with open source video editing.
|
|
That's why we are here.
|
|
* The proposal draws conclusions on two technically substantial points, at
|
|
which we don't agree. And it fails to provide sufficient (technically sound)
|
|
arguments to prove these statements.
|
|
|
|
While it is certainly 'desirable' to be cross-platform as much as possible and
|
|
especially '''target Microsoft Windows''', we don't see much possibilities with
|
|
today's mainstream technology to build an application which is as
|
|
technologically demanding as a video editor is. We would end up developing two
|
|
or even three sister applications, or we are forced to sacrifice performance
|
|
for portability. When put up to face such options, we have a clear preference
|
|
to concentrate on a really free and open platform.
|
|
|
|
While it is certainly 'desirable' to make the application as robust as
|
|
possible, we don't see how '''using multiple separate processes''' could help
|
|
us with this goal ''without creating major scalability or performance
|
|
problems'' due to the use of shared memory. And, yet more important: we don't
|
|
share the basic assumption made in the proposal, namely that video processing
|
|
is inherently dangerous. We think the basic algorithms involved are
|
|
sufficiently well-known and understandable to implement them in a sound manner.
|
|
|
|
''''
|
|
Back to link:Lumiera/DesignProcess[]
|