Design Process : Data Backend
=============================

[options="autowidth"]
|====================================
|*State*       | _Parked_
|*Date*        | _2007-06-04_
|*Proposed by* | ct
|====================================

DataBackend
-----------
Describe the DataBackend functionality.


Description
~~~~~~~~~~~
This just starts as braindump, I will refine it soon:

- handle all files lumiera uses at runtime (media, edl, temp data)
- manage filehandles, lumiera might use more more files than available filehandles
- manage temporary data
- do caching
- IO will be blocked where the backend tells the core where it can expect the data (not `read()`/`write()` like)
- kindof garbage collector
- do prefetching
- no/low latency for the core the prefetcher and other things ensure that data is available in time
- translate any input into a format which the lumiera core understands (demux, decode)
- same for encoding to output formats
- offer a plugin API for encoders/decoders
- maybe network backend for serving data to distributed rendernodes
- can do some load control or management (trigger adaptive rendering if system is idle etc)
- pull based architecture

.Notes:
* ichthyo wrote also some ideas regarding
  link:http://localhost:8888/project/background/history/PossibilitiesAtHand.html[things possible in the middle layer]...


Parked
~~~~~~
The underlying principles remain valid, yet development took another
direction during the last years. Special frameworks for high-performance
asynchronous IO will be used for dedicated use cases. A garbage collector
is not necessary, since Lumiera relies on deterministic memory management.
Furthermore, decoding, transcoding and encoding is handled as part of the
pipeline and not through a separate service.

Ichthyostega:: '2023-10-24' ~<prg@ichthyostega.de>~



Comments
--------

Sounds fairly complete to me

Ichthyostega:: '2007-06-16T23:19:44Z'

Developement takes place in the repo now

ct:: '2007-06-27T16:14:56Z'

Development took another direction over the last years;
the former »Backend« layer is called »Vault« now and structured differently.

Ichthyostega:: '2023-10-24T22:45:55Z'


''''
Back to link:/x/DesignProcess.html[Lumiera Design Process overview]
