LUMIERA.clone/doc/devel/rfc/ConfigLoader.txt

Config Loader
=============
:toc:

// please don't remove the //word: comments

[grid="all"]
`------------`-----------------------
*State*         _Parked_
*Date*          _2008-08-07_
*Proposed by*   **ct**
-------------------------------------

********************************************************************************
.Abstract
Load application configuration from plaintext files with INI syntax.
Build an elaborate system on top to combine several configuration files.
********************************************************************************

Description
-----------
//description: add a detailed description:
After some initial discussion, it soon became clear that Lumiera needs to load
configuration in some way. A lot of things should be configurable, but the
syntax and implementation should be kept simple to make it easy to use


Tasks
~~~~~
// List what needs to be done to implement this Proposal:
 * initial implementation draft ([green]#✔ done#)
// * second step [yellow-background]#WIP#
 * extended requirement analysis [red yellow-background]#TBD#


Discussion
~~~~~~~~~~
NOTE: (by _Ichthyostega,_ 2023) +
      Indeed this never became an actual RfC, and was rather treated as
      implementation prototype. With ongoing development, this draft
      became sidelined, and for the time being, it rests as unused code
      in the code base. Some accompanying documentation pages
      are hereby attached for later referral.



Parked
~~~~~~
While the general idea seams adequate, the topic of Configuration
requires a much more in-depth Requirement Analysis. Moreover, considering
the state of tooling today (2023), I consider it questionable still to rely
on plain-C to implement configuration handling, given that the C++ Language
offers a real `string` data type and is complemented by an extensive, safe
and performant string handling library, including even regular expressions.

In a recent
link:{ldoc}/devel/meeting_summary/2023-09-13.html[Developer Meeting (IRC)]
we decided to include this topic into a list of
https://issues.lumiera.org/report/17[»Focus Topics«],
to be addressed _after achieving the first integration_ of the Engine.

-> see https://issues.lumiera.org/ticket/1224[#1224 Layered Configuration]

Ichthyostega:: '2023-10-24' ~<prg@ichthyostega.de>~



//Conclusion
//----------
//conclusion: When approbate (this proposal becomes a Final)
//            write some conclusions about its process:



''''''''''''''''

[big]#🙞 🙜#



The incomplete Guide to Lumiera Configuration
---------------------------------------------

Lumiera uses plaintext files with a INI file like syntax for
configuration. This Syntax is strictly line based. There are only a
few syntactic elements.

TODO:describe config syntax here

Config Subsystem
~~~~~~~~~~~~~~~~

The path where Lumiera searches its configuration. Single components are
separated by colons as in PATH and other such environment variables.
Here it might be handy that any Lumiera configuration can be
overridden by a environment variable:
'LUMIERA_CONFIG_PATH=somewhere:else lumiera ...'
A default are initialized at installation time, this is important to
bootstrap the whole configuration system.

 config.path


The config system check for a preferred format when writing config
entries. For each key 'foo.bar', these can be overridden with a key
'config.format.foo.bar' linking to the desired format.

 config.formatkey ='config.format.%s'


The following are links to the default formatting when no explicit
format is set for a key. Changing these to a wrong type will break the
system!

 config.formatdef.link < config.formatstr.link
 config.formatdef.number < config.formatstr.number.dec
 config.formatdef.real < config.formatstr.real
 config.formatdef.string < config.formatstr.string
 config.formatdef.word < config.formatstr.word
 config.formatdef.bool < config.formatstr.bool


This are the low level formating specifications for the buildin
types, DONT TOUCH THESE!

 config.formatstr.link = '< %s'
 config.formatstr.number.dec = '= %lld'
 config.formatstr.number.hex = '= 0x%llX'
 config.formatstr.number.oct = '= 0%llo'
 config.formatstr.real = '= %Lg'
 config.formatstr.real.dec = '= %Lf'
 config.formatstr.real.sci = '= %Le'
 config.formatstr.string = '= %s'
 config.formatstr.string.dquoted = '= \"%s\"'
 config.formatstr.string.quoted = '= ''%s'''
 config.formatstr.word = '= %s'
 config.formatstr.bool = '= %d'


API
~~~

* Namespace is lumiera_config_*
* `struct lumiera_config_struct` for the config system, only one object needed
* `lumiera_config_init (...)`
  - does only initialize the variables, so that they get valid values, but does not allocate them as they will be allocated before as they are singleton.
  - lumiera_config_init (const char* searchpath) searchpath is a buildin-default, can be changed via configure and can be appended and overridden by using a flag, e.g. `--config-path-append=""` or `--config-path=""`

* `lumiera_config_destroy(...)`
  - frees all space allocated by the link:ConfigLoader[].

* `lumiera_config_load(...)`
  - reads *one* single configuration file that will include all settings from other files.
  - does not read itself but give delegates reading. The actual reading and parsing will be done in configfile object. s.later.

* `lumiera_config_save(...)`
  - saves all the changed settings to user's configuration files, but recognizes where settings came from and will write them to an appropriate named file. Example: *changed* values from  `'/usr/local/share/lumiera/plugins/blur.conf'` will be saved into `'~/.lumiera/plugins/blur.conf'`
  - finds out which files are dirty and which settings have to be written to user's config files.
  - does initiate the actual saving procedure by delegating the save to the actual configfile objects, see below.
  - empty user configuration files in RAM will be deleted from disk on write.
  - checks whether the file has changed since last read, and will print out an error if necessary instead of overriding it without notification.
+
`lumiera_config_save () { LLIST_FOREACH(config_singleton.files, f) { LumieraFile file = (LumieraFile) f; if(lumiera_configfile_isdirty (file)) lumiera_configfile_save(file); } }`

* `lumiera_config_purge(const char* filename)` removes all configs loaded from filename

* `lumiera_config_get(...)`
  - get a value by key
  - handles internally everything as string:string key:value pair.
  - lowlevel function
  - lumiera_config_integer_get (const char* key, int *value) will return integers instead of strings and return 0 if succeeded and -1 if it failed.

  - 'int lumiera_config_TYPE_get(const char* key, TYPE* value, const char* default)'
    High level config interface for different types.
    if default is given (!NULL) then value is set to default in case key was not found or any other error occured.
    error code is still set and -1 (fail) is returned in case of an error, but it might be cleared with no ill effects.
    NOTE: errors are persistent in our error handler, they must still be cleared, even when ignored.
          if default is given then `KEY_NOT_FOUND` is not a error here, if default is NULL then it is
    NOTE2: default values are given as strings, the config loader remembers a given default value and checks if it got changed
           when it is _set(). Thus a default value can be suppressed when set/written

* `lumiera_config_set(...)`
  - set a value by key
  - handles internally everything as string:string key:value pair.
  - lowlevel function
  - tag file as dirty
  - set will create a new user configuration file if it does not exist yet or will append a line to the existing one in RAM. These  files, tagged as \'dirty', will be only written if save() is called.
  - `lumiera_config_TYPE_set (const char* key, TYPE*value, const char* fmt)`
    Highlevel interface for different types, fmt is a printf format specifier for the desired format, when NULL, defaults apply.

* `lumiera_config_reset(...)`
  - reset a value by key to the system default values, thus removes a user's configuration line.

* `lumiera_config_info (const char* key, const char** filename, unsigned* line)`
  - Find exact place of a setting.

[NOTE]
=======================================
* multiple config items with the same key stack (to allow undo/removing custom configs)
* same keys in the same file stack also, the _get api returns the last value the _set api sets the last value
* we might need to implement some cursor api to retrieve all values with the same key, this should allow to filter keys from single files!
* if one queries a setting with a default, this default should be stored in the configuration system without an backing file on disk. Rationale is, that when one later _set's a value and this value is found to be equivalent to the default, this _set option is a no-op
=======================================

.foo.bar
=======================================
`integer_set("foo.bar", 123)`
looks up "foo.bar"
this returns as well file and line
it will be decided whether it's a system file or a user's config file.
if systemfile: loop for user's config file, if necessary create it in our RAM but not yet on disk
set the value in the line.
=======================================


anchor:anchor_config[]lumiera_configfile
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* `lumiera_configfile_struct`
  - contains a `llist` with all lines stored as `lumiera_configline_struct`
  - contains an integer that indicates how \'dirty' the file is, i.e. how many settings were changed. (one increment per _set() call)
  - contains a stat struct for comparison while saving if the file was altered since last read

* `lumiera_configline_struct`
  - stores the line exactly as read from file, but holds as well pointers to the key and the value.

* lumiera_configfile_load
* lumiera_configfile_save
* lumiera_configfile_new
* lumiera_configfile_delete
* lumiera_configfile_init
* lumiera_configfile_destroy



anchor:anchor_types[]provided TYPES
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[grid="all"]
`--------------.--------------------------------------'------------------------------------------------------------
config type     C type					semantic (example)
-------------------------------------------------------------------------------------------------------------------
number		signed long long			also hex and octal (foo.bar = 12345 #comment)
real		long double				differerent formats as well (foo.bar=1.2345 #comment)
string		const char*				optionaly quoted string (foo.bar="this string" #this not)
word		const char*				first optionaly quoted word of the config (foo.bar= first this # not)
bool		int					common representations for bools (0,1,true,false,yes,no,on,off,set,clear)
-------------------------------------------------------------------------------------------------------------------

... add more types on demand (ichthyo mentioned a tuple type for color triples etc.)

config files/strings are utf-8!


anchor:anchor_syntax[]Syntax/Semantics
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

...to be defined...

in short:

* configfiles allow comments,
* some kind of include directive
* includes happen only \'once', its impossible to create include recursions
* config settings are key:value pairs where the key is a hierachical string as in "foo.bar.baz"


anchor:anchor_proposal[]Syntax Proposal by cehteh
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

simple example:

------------------------------------------------------------
@directive options
key = value
[prefix]
[prefix suffix]
------------------------------------------------------------

ebnf:

------------------------------------------------------------
configline ::= [ whitespace ] , { comment | directive | configentry | section } , ? end of line ? ;

whitespace ::= ? one to many tab or blank ? ;

comment ::= "#" , ? arbitary text ? ;

directive ::= "@", ? commandword ? , whitespace, ? arguments ? ;

configentry ::= key, { whitespace } , assignop, value ;

key ::= ? lowercase characters and _ ? , [ "." , key ] ;

assignop ::=  " " | ":" | "=" ;

value ::= ? arbitary text ? ;

section ::= "link:, [ whitespace ] , prefix, [ whitespace, suffix ] , [ whitespace ] []" | emptysection ;

emptysection ::= "[]" ;
prefix ::= key ;
suffix ::= key ;
------------------------------------------------------------

semantics notes:

* The format is strictly line-based
* directives inside non empty sections are not yet defined
* sections can't nest so far (idea: thinking about `[ .extend.with.leading.dot ]`)
* section semantics: a key inside a section `[foo bar]` as in `key=value` is treated as `foo.key.bar=value`


anchor:anchor_links[]links
~~~~~~~~~~~~~~~~~~~~~~~~~~

http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html[]


anchor:anchor_internals[]internals/implementation sketch
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* `config` *singleton*
  - has a cuckoo hash for lookups
  - holds many `configfile` in a linked list

* `configfile` is a `configitem`
  - holds `lines` as `section` or `directive` starting with the empty section
  - there is one special unnamed (non-disk) file for _defaults_

* `configitem`
  - just an abstraction, we prolly dont need an implementation

* `section` is a `line`
  - parses prefix/suffix
  - holds list of childs of `configline` or `other` lines

* `line` is a `configitem`
  - holds the allocated string (char*)
  - backpointer to parent item (section or file)
  - 2 pointers+length to parsed data, key/value for `configline`, prefix/suffix for `section`

* `configline` is a `line`
  - parses key/value as raw strings
  - maintains lookup hash in ctor/dtor

* `other` is a `line`
  - comment or empty line, just preserved nothing special

* `directive` is a `line`
  - must appear in a empty section, else this is an error


Brainstorming
-------------

 * support for successive dereferencing foo.bar > baz.barf > blah.boo = final value
 * support for alternatives, query a set of keys, return the first existing
 * provide a @readonly directive which marks a file to be handled readonly (even if writing would be permitted)


Discussion on irc
~~~~~~~~~~~~~~~~~

------------------------------------------------------------
[01:21] <cehteh> joelholdsworth_: you seen the upcoming config interface i planned with simav?
[01:21] <mno> what about using ardours icons to start with?
[01:21] <joelholdsworth_> *least
[01:22] <joelholdsworth_> cehteh: not yeat, can I see
[01:22] <joelholdsworth_> mno: ardour icons arn't that useful
[01:22] <joelholdsworth_> there aren't that many
[01:22] <cehteh> http://www.pipapo.org/pipawiki/Lumiera/ConfigLoader[]
[01:22] <cehteh> question for you: do you need any more 'types'
[01:22] <joelholdsworth_> and those that are there are probably among the gtk stock set anywhere
[01:22] <cehteh> ichthyo suggested color triples
[01:23] <cehteh> anythinfg you have in mind, just suggest it
[01:23] <joelholdsworth_> so is the this structured like an ini file
[01:23] <joelholdsworth_> sections and keys?
[01:23] <joelholdsworth_> or is it treelike?
[01:24] <cehteh> little unordered, you basically just need to care for lumiera_config_TYPE_get ... and *_set
[01:24] <cehteh> like ini, not nested
[01:24] <joelholdsworth_> flat is good
[01:24] <joelholdsworth_> simple
[01:24] <cehteh> the namespaces are nested
[01:24] <cehteh> but the sectioning is not
[01:24] <cehteh> you can have
[01:24] <cehteh> [gui]
[01:24] <__nasa__> Have you all see Agave (for colorscheme generation)?
[01:24] <cehteh>  foo = bar
[01:25] <cehteh> [gui.baz]
[01:25] <cehteh>  barf= 123
[01:25] <cehteh> its not exactly ini style but close
[01:25] <joelholdsworth_> cehteh: ooh - does mean we can levarge an ini parser?
[01:25] <joelholdsworth_> that would save some work and maintenance effore
[01:25] <cehteh> hey simav want to write the parser himself :P
[01:25] <joelholdsworth_> __nasa__: agave: interesting
[01:26] <joelholdsworth_> cehteh: would he though
[01:26] <cehteh> well i think thats fun to do by ourself :)
[01:26] <joelholdsworth_> it is just like ini
[01:26] <joelholdsworth_> yeah
[01:26] <cehteh> and i plan some special things
[01:26] <joelholdsworth_> but reduction of volume is good
[01:26] <cehteh> like for example setting values from the gui preserves the rest of the data
[01:26] <cehteh> when you have
[01:26] <__nasa__> cehteh: suffixes?
[01:27] <cehteh> [gui.foo bar]
[01:27] <cehteh>  dada = 1234 #comment
[01:27] <__nasa__> I just saw the bottom...
[01:27] <cehteh> that is    gui.foo.dada.bar = 1234 # comment
[01:27] <cehteh> see suffix
[01:28] <cehteh> more in a moment
[01:28] <joelholdsworth_> still an ini parser would be happy with that
[01:28] <cehteh> what i want to tell to joel
[01:28] <joelholdsworth_> of course you have to add some extra code to the end to make the tree
[01:28] <joelholdsworth_> but that would make our lives simpler
[01:29] <cehteh> when you now have a gui which does gui_config_number_set ("gui.foo.dada.bar", 6666)
[01:29] <cehteh> then
[01:29] <cehteh> [gui.foo bar]
[01:29] <cehteh>  dada = 6666 #comment
[01:29] <cehteh> appears in the configfile
[01:29] <joelholdsworth_> did you mean...
[01:29] <cehteh> meaning preserving everything manually set up
[01:29] <joelholdsworth_> [gui.foo.dada]
[01:29] <__nasa__> cehteh: How does it know not to do
[01:29] <joelholdsworth_> bar = 6666 #comment
[01:29] <joelholdsworth_> ?
[01:30] <__nasa__> [gui dada.bar]
[01:30] <__nasa__> foo = 6666 #comment
[01:30] <__nasa__> joelholdsworth_: that works as well :)
[01:31] <cehteh> it just selects best fit sections which already exist
[01:31] <joelholdsworth_> but isn't it supposed to be this way...
[01:31] <cehteh> best fit .. first prefix, then suffix
[01:31] <joelholdsworth_> so [foo.bar] is different from [bar.foo]
[01:31] <__nasa__> Ok.
[01:31] <cehteh> if nothing fits then it places it at a non sectioned part in verbose way
[01:31] <cehteh> joelholdsworth_: yes
[01:31] <__nasa__> (I just wanted to make sure we had a decision on that, as I agree that is the best way)
[01:31] <cehteh> so rationale about sections:
[01:32] <cehteh> rather rarely used but you can do things like
[01:32] <cehteh> [plugin radius]
[01:32] <cehteh>  blur = 1
[01:32] <cehteh>  sharpen = 2
[01:33] <cehteh> or same for key shortcuts what simav initiall intended
[01:33] <joelholdsworth_> wouldn't it be simpler to have simpler systaxx
[01:34] <joelholdsworth_> and have it always with dots or with spaces
[01:34] <cehteh> well the prefix stuff is completely optinal
[01:34] <joelholdsworth_> that way you can't have 1 section in twice
[01:34] <cehteh> eh?
[01:34] <joelholdsworth_> well is [plugin radius] the same as [plugin.radius] ?
[01:34] <cehteh> no
[01:35] <joelholdsworth_> good
[01:35] <joelholdsworth_> ok
[01:35] <cehteh> [plugin radius]  would be suffix prefix
[01:35] <joelholdsworth_> seems overkill to me
[01:35] <cehteh> err opposite
[01:35] <joelholdsworth_> I'd just stick with one or the other
[01:35] <cehteh> <tired
[01:36] <cehteh> [plugin radius]  would be prefix suffix
[01:36] <cehteh> [plugin.radius]  is just prefix
[01:36] <mno> i'm trying to understand all this hehe
[01:37] <cehteh> the idea is that with just prefix you have to settle with a quite rigid hierachy where you know that things will always be ordered by the leaves
[01:37] <cehteh> that doesnt fit for everyone
[01:38] <joelholdsworth_> "ordered by the leaves" ?
[01:38] <cehteh> one might want to configure his plugins one affter each other
[01:38] <__nasa__> Here is what I think is cool about the setup: it means that we can treat builtins and plugins the same.
[01:38] <cehteh> imagine we have    blur and sharben
[01:38] <cehteh> sharpen
[01:38] <__nasa__> [plugin radius]
[01:39] <__nasa__> cinelerra.sharpen = 3
[01:39] <cehteh> both have a radius and a algorihtm. lets say horizontal or vertical
[01:39] <__nasa__> rotate = 2
[01:39] <cehteh> so then you can either first configure blur
[01:39] <cehteh> [plugin.blur]
[01:39] <cehteh>   radius = 1
[01:39] <cehteh>   algo = horiz
[01:40] <joelholdsworth_> oooh! - have you thought of simply using libconfig?
[01:40] <cehteh> # and then sharpen
[01:40] <cehteh> [plugin.sharpen]
[01:40] <cehteh>   radius = 2
[01:40] <cehteh>   algo = horiz
[01:40] <cehteh> ..
[01:40] <__nasa__> joelholdsworth_: that format looks a little painful, but it could work.
[01:41] <joelholdsworth_> it's like JSON I think
[01:41] <cehteh> hey so far thats just like ini
[01:41] <cehteh> well now about suffix
[01:41] <joelholdsworth_> ok
[01:41] <cehteh> imagine you want to configure things by 'topics' ..
[01:42] <cehteh> # first define radius for our plugins
[01:42] <cehteh> [plugin radius]
[01:42] <cehteh>   blur = 1
[01:42] <cehteh>   sharpen = 2
[01:42] <cehteh> # then algo
[01:43] <cehteh> [plugin algo]
[01:43] <joelholdsworth_> yes that's quite clever
[01:43] <cehteh>   sharpen = horiz
[01:43] <cehteh>   blur = horiz
[01:43] <joelholdsworth_> but still it seems overkill
[01:43] <__nasa__> joelholdsworth_: It does look similar to JSON, but more C-like. For example, there are semi-colons instead of commas at the ends of single values.
[01:43] <cehteh> well its completely optional and we can leave it out
[01:43] <joelholdsworth_> these files are not primarily for humans
[01:44] <cehteh> yeah
[01:44] <joelholdsworth_> and also that prefix thing while clever is quite non obvious when you first see it in the file
[01:44] <__nasa__> Yes, but I am a big proponent of trying to make things human readable
[01:44] <cehteh> git works the same way
[01:44] <joelholdsworth_> sure it should be human readable
[01:44] <cehteh> .git/config
[01:44] <__nasa__> So if anything does go wrong, the user can fix it.
[01:45] <joelholdsworth_> well I'm not sure the prefixes thing is obvious enough
[01:45] <cehteh> well and we will have plenty of values which dont need a gui
[01:45] <cehteh> see about:config for firefox
[01:45] <__nasa__> I think that INI is much easier to read than libconfig, even though I have been programming in C for years and I have never edited a windows INI file.
[01:46] <joelholdsworth_> I think I agree
[01:46] <cehteh> they have all defaults and only when altered they are recorded to a config file but they dont need any special gui
[01:46] <__nasa__> about:config is so nasty.
[01:46] <cehteh> well its mighty
[01:46] <joelholdsworth_> but yes all of about:config could be stored in ini format
[01:46] <joelholdsworth_> [accessibility.typeaheadfind]
[01:46] <cehteh> and anything which is commonly changed should have a preferences gui
[01:46] <joelholdsworth_> linksonly=false
[01:47] <cehteh> btw our configs are not enumerateable
[01:47] <cehteh> i dont want the rest of the system to register there, that will be pita
[01:47] <cehteh> you can freely introduce configs (within your namespace)
[01:48] <-- benG has quit (Remote closed the connection)
[01:48] <cehteh> i'd rather make a grep script which goes through the sources and searched for   lumiera_config_.*_get (".*", .*)
[01:49] <joelholdsworth_> yeah that's ok
[01:49] <cehteh> to have some idea what configs are actually used
[01:49] <joelholdsworth_> well maybe we can use libini
[01:49] <cehteh> so a about:config like ui wont be possible
[01:49] <__nasa__> which libini? There are quite a few.
[01:49] <joelholdsworth_> ahh
[01:49] <joelholdsworth_> that's a shame
[01:49] <cehteh> he forget about some libs .. thats so trivial to write by ourself :P
[01:49] <__nasa__> I assume the SF one is the best, as it is GPL.
[01:50] <cehteh> and simav wants to do something :P
[01:50] <joelholdsworth_> yeah
[01:50] <joelholdsworth_> anyway... time for bed for me
[01:50] <cehteh> well and what i rate high is that one can retrieve typed values  and that one can add/set values
[01:51] <cehteh> and this setting should not destroy the existing file structure
[01:51] <joelholdsworth_> yeah
[01:51] <cehteh> just read and write values is no magic .. but preserving human structured files is
[01:52] <joelholdsworth_> that is hard, yes
[01:52] <joelholdsworth_> anyway... got to go
[01:52] <__nasa__> Bye!
[01:52] <cehteh> n8 :)
[01:52] <-- joelholdsworth_ has quit ("Ex-Chat")
[01:52] <__nasa__> It looks like libini uses [plugin/blur] instead, but that is an easy fix.
[01:52] <cehteh> url?
[01:52] <__nasa__> Or we just let simav do the whole thing
[01:53] <cehteh> well i dont know how much time he has
[01:53] <__nasa__> http://sourceforge.net/project/showfiles.php?group_id=25464[]
[01:54] <cehteh> last change 3 year ago :P
[01:56] * cehteh looks at the source
[01:57] <cehteh> fails standard C++ conformance :P
[01:58] <cehteh> and no @include like directive
[01:58] * __nasa__ thinks we should look elsewhere...
[01:58] * cehteh things we should just do ourself :P
[01:58] * __nasa__ agrees.
[01:59] <cehteh> the syntax i typed there is not very special and can be replaced
[01:59] <cehteh> but some other features are important
[01:59] * __nasa__ thinks it should PRE. That way we don't have to totally agree.
[01:59] <cehteh> pre?
[01:59] <__nasa__> (On syntax)
[01:59] <cehteh> no suffix?
[02:00] <cehteh> well as saied its just an idea and completely optional, we can leave it out
[02:00] <__nasa__> sorry -- PCRE.
[02:00] <cehteh> ah
[02:00] <cehteh> i dont see how PCRE helps for a syntax
[02:00] <__nasa__> I always think of it as "Perl Regular Expressions" and forget the "Compatible"
[02:01] <cehteh> anyways: features matter
[02:01] <__nasa__> I was thinking in terms of plugins/blend vs. plugins.blend.
[02:01] <cehteh> that is @include which follows some LUMIERA_CONFIG_PATH
[02:02] <__nasa__> We should have some way to only include certain sections.
[02:02] <__nasa__> Like have the default config be
[02:02] <cehteh> yes
[02:02] <__nasa__> [lumiera.core]
[02:02] <__nasa__> ...
[02:02] <__nasa__> then
[02:02] <cehteh> but thats setup by the administrator/user
[02:02] <__nasa__> @include<plugin> ~/.lumi/plugins/
[02:02] <cehteh> if the user changes lumiera.core.somevalue
[02:03] <cehteh> it will find the file where it is already defined
[02:03] <__nasa__> and it adds all preferences for plugins from <dir>
[02:03] <cehteh> then check if thats a file in the users config space (~/.lumiera/*)
[02:03] <cehteh> and edit it there
[02:03] <__nasa__> Ok.
[02:04] <cehteh> if it is a system wide file it will place a correspondending file in the users config space
[02:04] <cehteh> if the value is not configured already it will search for [lumiera.core] and then [lumiera] and then []
[02:04] <cehteh> best fit the section where it would apply
[02:05] <cehteh> and then do the same write or create into correspondending user config
[02:05] <cehteh> thats why i saied 'features matter'
[02:06] <__nasa__> There should be a callback for a plugin to add a config file for that plugin.
[02:06] <__nasa__> Ooh
[02:06] <__nasa__> cool idea
[02:06] <cehteh> this is not really complicated to do but would give a big usage experience link:JustWorks[][TM]
[02:06] <__nasa__> ~/.lumi/config is the main configuration directory
[02:06] <__nasa__> inside, there is a ~/.lumi/config/config.ini which is main configuration
[02:07] <cehteh> no callbacks .. the idea is that the config system is independent and nothing else need to register there, that makes it much easier
[02:07] <__nasa__> Then, for example accessing plugin.shade.
[02:07] <cehteh> if you install a plugin this plugin should come with a system config file
[02:07] <cehteh> that can be a empty one
[02:07] <__nasa__> either find it in ~/.lumi/config/config.ini or ~/.lumi/config/plugin/config.ini.
[02:08] <cehteh>  /usr/local/share/lumiera/plugin/shade
[02:08] <cehteh> containing just
[02:08] <cehteh> [plugin.shade]
[02:08] <cehteh> when the user edits some value
[02:09] <cehteh> plugin.shade.transparency = 209
[02:09] <cehteh> then this will automatically create a correspondending ~/.lumiera/plugin/shade  in the user config space
[02:09] <__nasa__> yeah, so what I am saying is that instead of [plugin.shade] only [] is needed
[02:10] <cehteh> using the system one as template
[02:10] <cehteh> well if its []
[02:10] <__nasa__> since the file is already in config/PLUGIN/SHADE
[02:10] <cehteh> or that
[02:10] <cehteh> this is not yet decided
[02:10] <cehteh> we can do that too
[02:10] <__nasa__> so automatically add [plugin.shade] to the prefix of files in config/plugin/shade, etc
[02:11] <cehteh> but i would be careful to tie filenames and internal namespaces together
[02:11] <cehteh> yes doable
[02:11] <cehteh> well should still have [plugin.shade]
[02:12] <cehteh> else we have a nested section mess and files loose their identity
[02:12] <__nasa__> Yeah, I just realized it would be a pain to have default configs
[02:12] <__nasa__> Since where does the file go? you have to look inside to see if it had [plugin.shade], tec
[02:12] <__nasa__> etc*
[02:12] <cehteh> anyways nothing is etched in stone yet, we are just brainstorming
[02:13] <cehteh> most important thing is the interface the applciation can use
[02:13] <cehteh> then we can make a mockup for that which only returns defaults
[02:13] <cehteh> then we can use it already even when the config loader isnt finished
[02:14] <__nasa__> We could have a warning if a file in e.g. config/plugin/blur had a section that was [plugin.shade] or something
[02:14] <cehteh> actually the inital plan was to bootstrap the config system by one single site config file
[02:14] <__nasa__> to prevent hackery
[02:14] <cehteh> but plugins are a good point .. they might have their own configs
[02:15] <cehteh> which is not @included by the site config
[02:15] <__nasa__> lumiera_config_t *plugin_shade.request_default_config() or something
[02:15] <cehteh> but the plugin itself loads it when the plugin is used
[02:15] <__nasa__> Yeah
[02:16] <cehteh> nah
[02:16] <cehteh> lumiera_config is just singleton
[02:16] <cehteh> after you loaded plugin.shade it becomes visible there
[02:16] <cehteh> thats way easier
[02:16] <__nasa__> that is lumiera_config_struct
[02:16] <__nasa__> ok
[02:16] <cehteh> not even the struct
[02:17] <cehteh> thats an implementation detail
[02:17] <__nasa__> yeah -- to deal with later
[02:17] <cehteh> the programmers use only   lumiera_coonfig_TYPE_get   or _set
[02:17] <cehteh> where TYPE is to be defined .. see my notes
[02:18] <cehteh> number real string word ... and whatever else we need
[02:19] <cehteh> so you get typed data back .. when you provide a default then you can be even sure that it is always valid
[02:19] <__nasa__> Yeah, I do like that structure.
[02:19] <__nasa__> It also makes mass configuration setup very easy.
[02:21] <cehteh> btw file must have identity ...
[02:21] <__nasa__> meaning?
[02:21] <__nasa__> like they are not all config.ini?
[02:22] <cehteh> i mean a config file has to define for what kinds of values it applies
[02:22] <cehteh> [plugin.shade]    for the plugin/shade file
[02:22] <__nasa__> Ok, I see.
[02:22] <cehteh> then changing a values there will create the correspondending user file
[02:23] <cehteh> *thinking*
[02:23] <cehteh> the user could write  [plugin.shade]
[02:23] <cehteh> into some other config file which then becomes target for that
[02:23] <cehteh> (best fit)
[02:24] <cehteh> just wondering if the user just writes [pluin]
[02:24] <cehteh> [plugin]
[02:24] <cehteh> well i am sure we solve that
[02:25] <__nasa__> [plugin]
[02:26] <__nasa__> shade.radius = 3
[02:26] <__nasa__> in like the plugin/blur folder?
[02:28] <cehteh> no in the users config somewher
[02:28] <__nasa__> oh
[02:28] <cehteh> ~/.lumiera/myconfig
[02:29] <cehteh> shall it pick and mirror /usr/share/lumiera/plugin/blur to the user config space
[02:29] <cehteh> or write it to the [plugin] section in myconfig
[02:30] <cehteh> well maybe we add a   [plugin*]    syntax which hints the best fit
[02:30] <__nasa__> I would think it makes more sense to add a plugin/blur folder
[02:30] <__nasa__> or else lumiera/myconfig would be huge after configuring one project ;-)
[02:30] <cehteh> yes .. thats what you think .. i just wondering how to give the user the control to fit it to his needs
[02:31] <cehteh> ~/.lumiera/all_my_pluginmyconfig
[02:31] <cehteh> [plugin*]
[02:31] <cehteh> hints the best fit algo
[02:32] <cehteh> well just thinking about optional things
[02:32] <__nasa__> I think I am confused -- how is [plugin*] different from [plugin]
[02:33] <cehteh> when you change plugin.blur.radius   then the best fit algo will search the section which covers it best
[02:33] <cehteh> that would be [plugin.blur] in  /usr/share/lumiera/plugin/blur
[02:33] <__nasa__> Oh, unless you have [plugin*]
[02:34] <cehteh> so it will create a correspondending ~/.lumiera/plugin/blur  file for you
[02:34] <cehteh> yes exactly
[02:34] <__nasa__> Ok, that makes sense. Good idea!
[02:34] <cehteh> [plugin*] will take precedence
[02:34] <cehteh> when you have that then you can hint it not to create many tiny files
[02:35] <__nasa__> so that way the config is in one file
[02:35] <__nasa__> yeah
[02:35] <cehteh> hehe the suffux thing comes in mind here
[02:35] <cehteh> [plugin* radius]     in ~/.lumiera/plugin_radius_conf
[02:35] <__nasa__> I was thinking suffixed are only allowed at the e.g. plugin/ level if they effect multiple plugins.
[02:35] <cehteh> well or as trivial as
[02:36] <__nasa__> Ok.
[02:36] <cehteh> [plugin* enable]     in ~/.lumiera/plugins_enabled
[02:36] <__nasa__> Fun!
[02:36] <cehteh>  blur = on
[02:36] <cehteh>  sharpen = off
[02:36] <cehteh> ah .. we need a bool TYPE
[02:36] <cehteh> *blink*
[02:37] <__nasa__> on/off or true/false?
[02:37] <__nasa__> Well, either way.
[02:38] <cehteh> bool
[02:38] <cehteh>
[02:38] <cehteh> int
[02:38] <cehteh>
[02:38] <cehteh> common representations for bools (0,1,yes,no,on,off,set,clear)
[02:38] <cehteh> we can add more
[02:38] <__nasa__> It should be reasonably good at guessing
[02:40] <cehteh> yeah .. i would like to make this little intelligent
[02:40] <cehteh> for example when you want to set something it might look at the existing thing before doing so and then flavor the set with that
[02:41] <cehteh> when a string was quoted, the new one will be quoted to
[02:41] <__nasa__> "Weaken the blur effect"
[02:41] <cehteh> if there was a bool with 'yes' then it will choose 'no' ..
[02:41] <cehteh> and so on
[02:42] <cehteh> this are generally just a few extra lines of code, nothing dramatic .. but that makes it much nicer in detail
[02:43] <__nasa__> Yeah, I think that it will improve the user experience
[02:43] <cehteh> ok i add this irclog to the wiki
[02:43] <__nasa__> for most people, editing a config file is scary
[02:43] <__nasa__> so making it as easy and friendly as possible is good
[02:44] <__nasa__> All right
------------------------------------------------------------