ichthyo/wegen
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Projektantrag eingereicht.

Browse source 
Hab den Text noch etwas ausgefeilt.
Auch noch einige Formulierungs-Verbesserungen gemäß ChatGPT eingearbeitet.
This commit is contained in:
Fischlurch 2026-05-23 19:24:30 +02:00
parent df04350e33
commit 3f7baa8e24
2 changed files with 39 additions and 32 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

61
FrOSCon26/Antrag.txt
View file

@ -12,47 +12,50 @@ OpenSource-Projekt lösen mussten: eine große Menge Video-Daten effizient einle
Vor diesem Hintergrund möchte ich das neue IO_URING Kernel-Interface erläutern
und mit vereinfachten Code-Beispielen in C++ zeigen, wie es in eine Applikation
eingebunden werden kann. Rudimentäre C++ Kenntnisse sollten genügen, um dem
Vortrag zu folgen.
Vortrag zu folgen, Kennnisse in Systemprogrammierung werden nicht vorrausgesetzt.
Für die eigentliche Darstellung rechne ich mit 40 Minuten.
Kurzfassung
~~~~~~~~~~~
IO_URING is a fairly new IO interface in the Linux kernel.
It marks a paradigm shift in how the kernel is interacted with.
Instead of passing control thorough an interface function to enter into kernel mode,
messages are passed asynchronously.
<b>io_uring</b> is a fairly new I/O interface in the Linux kernel.
It represents a shift in how applications interact with the kernel.
Instead of transferring control through a system-call interface into kernel mode,
operations are submitted asynchronously through shared queues.
In this talk, patterns for dealing with asynchronous IO are discussed and a setup to handle
high volume file reading using this framework is demonstrated with code examples in C++.
This talk discusses patterns for dealing with asynchronous I/O and demonstrates a setup
for high-volume file reading using this framework, illustrated with code examples in C++.
<i>Lumiera</i> is a video editing application under development in C++ -- and is used here to
illustrate that asynchronous I/O can play a key role in achieving high throughput for video processing.
<i>Lumiera</i> is a video editing application under development in C++ and serves here
as a real-world example showing how asynchronous I/O can play a key role in achieving
adequate throughput for video processing.
Langfassung
~~~~~~~~~~~
The usual simple way of reading data from a file is to just invoke a `read()` function, directly
getting the data in response. This function is accessible as part of the programming language's
standard library, yet in the end it maps down onto a function of the OS kernel. And while this
form of reading data from a file is easy to handle, in terms of program logic, it has the downside
that the current thread is blocked, for an unknown period of time, until the data is ready.
The usual straightforward way to read data from a file is to invoke a <tt>read()</tt>
function and receive the requested data directly in response. While this function
is typically exposed through the programming language's standard library, it ultimately
maps to a system call provided by the OS-kernel. Although this approach is simple in
terms of program logic, it has the disadvantage that the calling thread is blocked,
for an unknown amount of time, until the data becomes available.
Blocking is particularly problematic in high-performance computing, as it means losing control
over the timings and leaving resources unused. Various interfaces and frameworks have been
developed over time to improve this, both for network communication and for file operations.
The Linux kernel system call interface IO_URING, adopted in kernel version 5.1 (2019) and
gradually expanded since then, takes these efforts one step further and provides an uniform
framework for various kinds of IO operations: The client code instructs the kernel to perform
some IO related task by sending it a message, and the kernel returns the completed work items
via a different queue.
Blocking behaviour is particularly problematic in high-performance applications, because it
reduces control over execution timing and may leave computational resources idle. Various
interfaces and frameworks have been developed over time to address this issue, both for
network communication and for file operations. The Linux kernel system call interface
<b>io_uring</b>, adopted in kernel version 5.1 (2019) and gradually expanded since then,
takes these efforts one step further by providing a uniform framework for many kinds of
I/O operations: Client code submits requests to the kernel via shared queue, while
completed work items are returned asynchronously through a second queue.
In theory, it is possible to achieve an extreme throughput or very low latency; yet in practice,
it proves quite challenging to structure the client code accordingly to achieve these goals.
In our use case, related to rendering video frames based on the arrangement made by the film
editor, expensive computations need to be chained-up behind the IO operations, all involving
a massive amount of data. The talk will discuss some common solution patterns, and the approach
taken by the Lumiera project to cope with this situation. A simplified demo level setup will
then be explained with code examples in C++
In theory, this model allows extremely high throughput and very low latency.
In practice, however, structuring client code to achieve these goals remains challenging.
In our use case -- rendering video frames based on the timeline arrangement made by the
film editor -- expensive computations must be chained behind I/O operations, all together
involving a massive amount of data.
The talk discusses common solution patterns and presents the approach taken by the
<i>Lumiera</i> project to address these challenges. A simplified demonstration setup
will then be explained using code examples in C++.

10
FrOSCon26/FrOSCon.26.mm
View file

@ -474,7 +474,7 @@
<node CREATED="1715854555555" ID="ID_954941333" MODIFIED="1715854555555" TEXT="Termin">
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1519001747051" ID="ID_1181924241" MODIFIED="1779386001214" TEXT="Call-of-Papers">
<icon BUILTIN="flag-yellow"/>
<node BACKGROUND_COLOR="#fdfdcf" COLOR="#ff0000" CREATED="1519001776927" ID="ID_562121416" MODIFIED="1779385981305" TEXT="Deadline">
<node BACKGROUND_COLOR="#c8c0b6" COLOR="#435e98" CREATED="1519001776927" ID="ID_562121416" MODIFIED="1779556988899" TEXT="Deadline">
<icon BUILTIN="clanbomber"/>
<node COLOR="#338800" CREATED="1715854555555" ID="ID_897784401" MODIFIED="1779385985746" TEXT="23.5.">
<font NAME="SansSerif" SIZE="12"/>
@ -537,10 +537,14 @@
</node>
</node>
</node>
<node BACKGROUND_COLOR="#eef0c5" COLOR="#990000" CREATED="1519329836328" ID="ID_214079495" MODIFIED="1779504752558" TEXT="formulieren">
<icon BUILTIN="pencil"/>
<node COLOR="#338800" CREATED="1519329836328" ID="ID_214079495" MODIFIED="1779556986278" TEXT="formulieren">
<icon BUILTIN="button_ok"/>
<node CREATED="1779504756681" ID="ID_466048202" MODIFIED="1779504767437" TEXT="halte mich an das Schema aus den letzten Jahren"/>
<node CREATED="1779504768195" ID="ID_531662550" MODIFIED="1779504785771" TEXT="Kurzform und Langform ergeben auch zusammen einen Text"/>
<node BACKGROUND_COLOR="#accdc3" COLOR="#3f4015" CREATED="1779554095269" ID="ID_1290433500" MODIFIED="1779554118621" STYLE="fork" TEXT="habe mir noch einige Vorschl&#xe4;ge von ChatGPT geben lassen">
<edge COLOR="#808080" STYLE="bezier" WIDTH="thin"/>
<icon BUILTIN="wizard"/>
</node>
<node BACKGROUND_COLOR="#f0d5c5" COLOR="#990033" CREATED="1779504789243" ID="ID_649621958" MODIFIED="1779504956035" TEXT="Kl&#xe4;ren: kann man mit IO_URING gezielt bestimmte Bereiche eines Files lesen?">
<arrowlink COLOR="#d60714" DESTINATION="ID_628756969" ENDARROW="Default" ENDINCLINATION="497;-40;" ID="Arrow_ID_347382746" STARTARROW="None" STARTINCLINATION="347;-14;"/>
<icon BUILTIN="help"/>