Ichthyostega
df04350e33
Mit dem etablierten Schema war das nicht schwer... Fehlen nur noch ein paar technische Details
58 lines
3.1 KiB
Text
58 lines
3.1 KiB
Text
Antrag für einen Vortrag auf der FrOSCon 26
|
|
===========================================
|
|
:author: Ichthyostega
|
|
:date: 23.5.2026
|
|
|
|
|
|
Reading Video data with IO_URING
|
|
--------------------------------
|
|
.putting a new kernel framework into practical use
|
|
In diesem Vortrag geht es um eine technische Herausforderung, die wir für unser
|
|
OpenSource-Projekt lösen mussten: eine große Menge Video-Daten effizient einlesen.
|
|
Vor diesem Hintergrund möchte ich das neue IO_URING Kernel-Interface erläutern
|
|
und mit vereinfachten Code-Beispielen in C++ zeigen, wie es in eine Applikation
|
|
eingebunden werden kann. Rudimentäre C++ Kenntnisse sollten genügen, um dem
|
|
Vortrag zu folgen.
|
|
|
|
Für die eigentliche Darstellung rechne ich mit 40 Minuten.
|
|
|
|
|
|
Kurzfassung
|
|
~~~~~~~~~~~
|
|
IO_URING is a fairly new IO interface in the Linux kernel.
|
|
It marks a paradigm shift in how the kernel is interacted with.
|
|
Instead of passing control thorough an interface function to enter into kernel mode,
|
|
messages are passed asynchronously.
|
|
|
|
In this talk, patterns for dealing with asynchronous IO are discussed and a setup to handle
|
|
high volume file reading using this framework is demonstrated with code examples in C++.
|
|
|
|
<i>Lumiera</i> is a video editing application under development in C++ -- and is used here to
|
|
illustrate that asynchronous I/O can play a key role in achieving high throughput for video processing.
|
|
|
|
|
|
Langfassung
|
|
~~~~~~~~~~~
|
|
The usual simple way of reading data from a file is to just invoke a `read()` function, directly
|
|
getting the data in response. This function is accessible as part of the programming language's
|
|
standard library, yet in the end it maps down onto a function of the OS kernel. And while this
|
|
form of reading data from a file is easy to handle, in terms of program logic, it has the downside
|
|
that the current thread is blocked, for an unknown period of time, until the data is ready.
|
|
|
|
Blocking is particularly problematic in high-performance computing, as it means losing control
|
|
over the timings and leaving resources unused. Various interfaces and frameworks have been
|
|
developed over time to improve this, both for network communication and for file operations.
|
|
The Linux kernel system call interface IO_URING, adopted in kernel version 5.1 (2019) and
|
|
gradually expanded since then, takes these efforts one step further and provides an uniform
|
|
framework for various kinds of IO operations: The client code instructs the kernel to perform
|
|
some IO related task by sending it a message, and the kernel returns the completed work items
|
|
via a different queue.
|
|
|
|
In theory, it is possible to achieve an extreme throughput or very low latency; yet in practice,
|
|
it proves quite challenging to structure the client code accordingly to achieve these goals.
|
|
In our use case, related to rendering video frames based on the arrangement made by the film
|
|
editor, expensive computations need to be chained-up behind the IO operations, all involving
|
|
a massive amount of data. The talk will discuss some common solution patterns, and the approach
|
|
taken by the Lumiera project to cope with this situation. A simplified demo level setup will
|
|
then be explained with code examples in C++
|
|
|