[grid="all"]
`------------`-----------------------
*State*         _Pending_
*Date*          _2008-07-26_
*Proposed by*   link:ct[]
-------------------------------------


Scripting Language
------------------
Add support for the *Lua* scripting language in Lumiera.


Description
~~~~~~~~~~~
We talked since the beginning about that we want to have scripting support
within Lumiera. Some weeks ago we did a non formal decision on IRC to bless Lua
as »official« scripting language.


Tasks
^^^^^
 * Investigate Lua's C bindings and integrate it
 * It will attach to the link:Plugin/Interface[] System cehteh is working on


Pros
^^^^
 * Small, Lightweight
 * Simple Syntax
 * reasonable fast
 * incremental GC (predictable performance)


Cons
^^^^
 * Lua has only a moderate library compared to Python for example, though I
   don't think this is a problem for our purpose



Alternatives
^^^^^^^^^^^^
There are quite a lot other scripting language which would be suitable. When it
makes sense these could be bound later too.


Rationale
~~~~~~~~~
Wee need scripting yet alone for driving the testsuite soon. Later on scripting
might be used to customize workflows and other application internals. Further
one might implement a high level / batch interface to lumiera to give it a
scripting interface similar to link:AviSynth[].





Comments
--------

To make it more clear: the intention is to have the scripts call into well
defined Interfaces / API functions, which are accessed via the plugin system. It
is _not_ intended to have just arbitrary scripts anywhere and everywhere, but
-- on the other hand -- all important functionality which can be activated via
the GUI should be similarly accessible via the scripting APIs without
restrictions. So, as Python and Ruby and Perl are popular scripting language,
we'll have the necessary bindings sooner or later.

Beyond that, it is possible to have some _extension points_ where
configuration is added to the application in a well defined manner by scripts.
These scripts provide for some basic customisation and can even add some of the
important higher-level features. With respect to these, the idea of this
proposal is to have one _required scripting language_, so scripts in this
language are guaranteed to work and may be used to add essential features. I
consider Lua to be an almost perfect fit for this purpose.

Ichthyostega:: '2008-07-27T22:36:52Z'

Well my intention is to make Lua a real first class binding where any internal
interface gets exported and would be useable from scripting, that contradicts
your limitation to make is only an extension language; but hold on:

 * this internal bindings should be announced as volatile and do-not-use for
   anyone external and being 'unsupported'
 * we can build a 'exported' scripting api on top of that which then will be
   the official way to control Lumiera from a high level script.
 * the reason why I want to make it this way is that it will become possible to
   implement tests, mock-ups and prototypes of some functionality in Lua. I
   predict that this will help us greatly on development when we progress
   further. Things which usefulness is doubtful can be prototyped and tried out
   in a afternoon rather than a week. That makes it possible to 'see' things
   which otherwise would be rejected because they are not worth a try.
 * some very custom specializations (studio workflows) would be easier
   integrateable
 * of course if this is used wrong it can really damage the health of the
   system, but I think this is oblivious and very explicit, there are easier
   ways to damage it, just whack your computer with a sledgehammer for example.
 * there might some laziness to keep prototypes in Lua instead reimplement them
   properly in C/C++, well IMHO that's OK, at some point need will arise to
   make it proper, if the Lua implementation is insufficient, but that's
   arguable.
   
ct:: '2008-07-30T16:22:32Z'


I have no problems using Lua. It is proven in the industry, well supported,
fast, efficient, high level and designed for this purpose. My only "complaint"
is that Lua isn't my pet language (Scheme). And that really isn't a complaint
at all.

PercivalTiglao:: '2008-07-28T19:56:25Z'


I think Python should be reconsidered: it's given that all languages in this
class are powerful at what they do, however python has particularly well
developed libraries and is used as the scripting language in the main raster
(GIMP), vector (Inkscape) and 3D (Blender, link:PythonOgre[], PyCrystal) Apps.
Combinations of these Apps are all going to be working in a stack in
professional production, so the fact that all the others use python makes a
more persuasive case for adoption than any micro-benefit in performance or
features that can be found between Python/Ruby/Perl/Lua etc. Python is also
used extensively in RedHat and Ubuntu admin scripting where most professional
deployments will be. If the goal is to truly get a professional setup, i.e. to
get this into professional production houses, then I think having a single
language from OS admin the whole way through the stack is a massive gain for
the types of users who will be using it. I personally prefer Ruby. Naturally
it's your decision to make, all the best, we are looking forward to alphas and
betas in the future +
 -- *mytwocents*


This proposal is about the _required_ scripting language, i.e. when
accepted, Lua will be a necessary prerequisite for running Lumiera. This
doesn't rule out the _use_ of other scripting languages. We strive at
having clean interfaces, thus it shouldn't be much of a problem to create
Python bindings. And given the popularity of Python, I guess it won't be long
until we have some Python bindings. But _requiring_ Python would mean
having a Python runtime in memory most of the time -- for such Lua obviously
is a better choice, because it's much more lightweight and minimalistic.

Ichthyostega:: '2008-09-30T02:17:08Z'


Many Years Later
~~~~~~~~~~~~~~~~
(See link:https://issues.lumiera.org/ticket/134[Ticket #134])
Hereby I *overrule* and *reject* the decision to support Lua or any
other scripting language directly; this topic shall be moved back into discussion later.

After careful consideration, and weighting in my experience as professional developer,
I came to the conclusion that we want _scriptability of the application,_ yet turning
the application itself into a multi-language codebase, even more so using any kind of
``easy going'' dynamically typed language, is detrimental to longevity.

I can understand -- even sympathise with the original proposal. +
Developers, as non-developers alike, tend to foster the dream of a fluid and limitless
technology of expression, a technology that just makes our intention flow into reality,
be it through the promise of new fancy languages, the ability for ad hoc extensions,
the reliance on almighty frameworks or even some kind of artificial entity able to guess
what we want -- anything to relieve us from the pain of writing down and spelling out
clearly what we aspire, with all the consequences and limitations of reality.

Building a coherent architecture with clean and understandable interfaces is hard work.
There is no shortcut around that, and the only path towards a scriptable application is:

- build a coherent architecture first, with _well defined functionality..._
- build a script-runner component with the ability to actuate and control the application
- cast the abilities of this script-runner in terms of a clear self-explanatory interface
- define a binding into the object model of one or several scripting languages.
- build test coverage both for the interface and the language binding.

Thus, as far as _scriptability_ is concerned, we have yet a long way to go indeed.

However, this »scripting language« proposal seems to take quite a different, if not antithetical
approach (as confirmed by the ensuing discussion and the tickets), namely to open up internals
of the application for easy prototyping, while just promising strict design work for later.
To quote ``Things which usefulness is doubtful can be prototyped and tried out in a afternoon
rather than a week''.

As compelling as it may sound -- based on my experience, this is a common anti-pattern:
If something is of doubtful usefulness, and requires a week to be built properly, you should
rather spend an afternoon to write a specification in plain natural language, instead of sneaking in
a half-baked prototype; if you can not write down your intentions in clear terms, using simple language
and coherent terminology, chances are that the code you write will be confused and tricky. And worse,
this kind of sketchy code has the tendency to stick; the more it needs to be amended and bashed
into submission, the higher the emotional investment. And soon further new and exciting additions
will be based on it, progressively corroding the application.

Adding _more_ technology, like adding _yet another_ language or library to the mix, can never
be a means of simplification -- it is an investment rather, and has to pay off. When you
``move fast and break things'', you end up with lots of broken stuff -- unless you have the
actual capacity to clean up the mess and build everything from scratch, sound and solid.

Ichthyostega:: '2023-02-04' ~<prg@ichthyostega.de>~


Conclusion
----------
Lua was _accepted_ as the required scripting language by October.2008 dev
meeting. However, Ichthyo _questions and overrules_ this decision in Feb.2023
and moves this proposal back into the inception stage.

Back to link:/documentation/devel/rfc.html[Lumiera Design Process overview]

