Projektantrag eingereicht.
Hab den Text noch etwas ausgefeilt. Auch noch einige Formulierungs-Verbesserungen gemäß ChatGPT eingearbeitet.
This commit is contained in:
parent
df04350e33
commit
3f7baa8e24
2 changed files with 39 additions and 32 deletions
|
|
@ -12,47 +12,50 @@ OpenSource-Projekt lösen mussten: eine große Menge Video-Daten effizient einle
|
||||||
Vor diesem Hintergrund möchte ich das neue IO_URING Kernel-Interface erläutern
|
Vor diesem Hintergrund möchte ich das neue IO_URING Kernel-Interface erläutern
|
||||||
und mit vereinfachten Code-Beispielen in C++ zeigen, wie es in eine Applikation
|
und mit vereinfachten Code-Beispielen in C++ zeigen, wie es in eine Applikation
|
||||||
eingebunden werden kann. Rudimentäre C++ Kenntnisse sollten genügen, um dem
|
eingebunden werden kann. Rudimentäre C++ Kenntnisse sollten genügen, um dem
|
||||||
Vortrag zu folgen.
|
Vortrag zu folgen, Kennnisse in Systemprogrammierung werden nicht vorrausgesetzt.
|
||||||
|
|
||||||
Für die eigentliche Darstellung rechne ich mit 40 Minuten.
|
Für die eigentliche Darstellung rechne ich mit 40 Minuten.
|
||||||
|
|
||||||
|
|
||||||
Kurzfassung
|
Kurzfassung
|
||||||
~~~~~~~~~~~
|
~~~~~~~~~~~
|
||||||
IO_URING is a fairly new IO interface in the Linux kernel.
|
<b>io_uring</b> is a fairly new I/O interface in the Linux kernel.
|
||||||
It marks a paradigm shift in how the kernel is interacted with.
|
It represents a shift in how applications interact with the kernel.
|
||||||
Instead of passing control thorough an interface function to enter into kernel mode,
|
Instead of transferring control through a system-call interface into kernel mode,
|
||||||
messages are passed asynchronously.
|
operations are submitted asynchronously through shared queues.
|
||||||
|
|
||||||
In this talk, patterns for dealing with asynchronous IO are discussed and a setup to handle
|
This talk discusses patterns for dealing with asynchronous I/O and demonstrates a setup
|
||||||
high volume file reading using this framework is demonstrated with code examples in C++.
|
for high-volume file reading using this framework, illustrated with code examples in C++.
|
||||||
|
|
||||||
<i>Lumiera</i> is a video editing application under development in C++ -- and is used here to
|
<i>Lumiera</i> is a video editing application under development in C++ and serves here
|
||||||
illustrate that asynchronous I/O can play a key role in achieving high throughput for video processing.
|
as a real-world example showing how asynchronous I/O can play a key role in achieving
|
||||||
|
adequate throughput for video processing.
|
||||||
|
|
||||||
|
|
||||||
Langfassung
|
Langfassung
|
||||||
~~~~~~~~~~~
|
~~~~~~~~~~~
|
||||||
The usual simple way of reading data from a file is to just invoke a `read()` function, directly
|
The usual straightforward way to read data from a file is to invoke a <tt>read()</tt>
|
||||||
getting the data in response. This function is accessible as part of the programming language's
|
function and receive the requested data directly in response. While this function
|
||||||
standard library, yet in the end it maps down onto a function of the OS kernel. And while this
|
is typically exposed through the programming language's standard library, it ultimately
|
||||||
form of reading data from a file is easy to handle, in terms of program logic, it has the downside
|
maps to a system call provided by the OS-kernel. Although this approach is simple in
|
||||||
that the current thread is blocked, for an unknown period of time, until the data is ready.
|
terms of program logic, it has the disadvantage that the calling thread is blocked,
|
||||||
|
for an unknown amount of time, until the data becomes available.
|
||||||
|
|
||||||
Blocking is particularly problematic in high-performance computing, as it means losing control
|
Blocking behaviour is particularly problematic in high-performance applications, because it
|
||||||
over the timings and leaving resources unused. Various interfaces and frameworks have been
|
reduces control over execution timing and may leave computational resources idle. Various
|
||||||
developed over time to improve this, both for network communication and for file operations.
|
interfaces and frameworks have been developed over time to address this issue, both for
|
||||||
The Linux kernel system call interface IO_URING, adopted in kernel version 5.1 (2019) and
|
network communication and for file operations. The Linux kernel system call interface
|
||||||
gradually expanded since then, takes these efforts one step further and provides an uniform
|
<b>io_uring</b>, adopted in kernel version 5.1 (2019) and gradually expanded since then,
|
||||||
framework for various kinds of IO operations: The client code instructs the kernel to perform
|
takes these efforts one step further by providing a uniform framework for many kinds of
|
||||||
some IO related task by sending it a message, and the kernel returns the completed work items
|
I/O operations: Client code submits requests to the kernel via shared queue, while
|
||||||
via a different queue.
|
completed work items are returned asynchronously through a second queue.
|
||||||
|
|
||||||
In theory, it is possible to achieve an extreme throughput or very low latency; yet in practice,
|
In theory, this model allows extremely high throughput and very low latency.
|
||||||
it proves quite challenging to structure the client code accordingly to achieve these goals.
|
In practice, however, structuring client code to achieve these goals remains challenging.
|
||||||
In our use case, related to rendering video frames based on the arrangement made by the film
|
In our use case -- rendering video frames based on the timeline arrangement made by the
|
||||||
editor, expensive computations need to be chained-up behind the IO operations, all involving
|
film editor -- expensive computations must be chained behind I/O operations, all together
|
||||||
a massive amount of data. The talk will discuss some common solution patterns, and the approach
|
involving a massive amount of data.
|
||||||
taken by the Lumiera project to cope with this situation. A simplified demo level setup will
|
|
||||||
then be explained with code examples in C++
|
|
||||||
|
|
||||||
|
The talk discusses common solution patterns and presents the approach taken by the
|
||||||
|
<i>Lumiera</i> project to address these challenges. A simplified demonstration setup
|
||||||
|
will then be explained using code examples in C++.
|
||||||
|
|
|
||||||
|
|
@ -474,7 +474,7 @@
|
||||||
<node CREATED="1715854555555" ID="ID_954941333" MODIFIED="1715854555555" TEXT="Termin">
|
<node CREATED="1715854555555" ID="ID_954941333" MODIFIED="1715854555555" TEXT="Termin">
|
||||||
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1519001747051" ID="ID_1181924241" MODIFIED="1779386001214" TEXT="Call-of-Papers">
|
<node BACKGROUND_COLOR="#eee5c3" COLOR="#990000" CREATED="1519001747051" ID="ID_1181924241" MODIFIED="1779386001214" TEXT="Call-of-Papers">
|
||||||
<icon BUILTIN="flag-yellow"/>
|
<icon BUILTIN="flag-yellow"/>
|
||||||
<node BACKGROUND_COLOR="#fdfdcf" COLOR="#ff0000" CREATED="1519001776927" ID="ID_562121416" MODIFIED="1779385981305" TEXT="Deadline">
|
<node BACKGROUND_COLOR="#c8c0b6" COLOR="#435e98" CREATED="1519001776927" ID="ID_562121416" MODIFIED="1779556988899" TEXT="Deadline">
|
||||||
<icon BUILTIN="clanbomber"/>
|
<icon BUILTIN="clanbomber"/>
|
||||||
<node COLOR="#338800" CREATED="1715854555555" ID="ID_897784401" MODIFIED="1779385985746" TEXT="23.5.">
|
<node COLOR="#338800" CREATED="1715854555555" ID="ID_897784401" MODIFIED="1779385985746" TEXT="23.5.">
|
||||||
<font NAME="SansSerif" SIZE="12"/>
|
<font NAME="SansSerif" SIZE="12"/>
|
||||||
|
|
@ -537,10 +537,14 @@
|
||||||
</node>
|
</node>
|
||||||
</node>
|
</node>
|
||||||
</node>
|
</node>
|
||||||
<node BACKGROUND_COLOR="#eef0c5" COLOR="#990000" CREATED="1519329836328" ID="ID_214079495" MODIFIED="1779504752558" TEXT="formulieren">
|
<node COLOR="#338800" CREATED="1519329836328" ID="ID_214079495" MODIFIED="1779556986278" TEXT="formulieren">
|
||||||
<icon BUILTIN="pencil"/>
|
<icon BUILTIN="button_ok"/>
|
||||||
<node CREATED="1779504756681" ID="ID_466048202" MODIFIED="1779504767437" TEXT="halte mich an das Schema aus den letzten Jahren"/>
|
<node CREATED="1779504756681" ID="ID_466048202" MODIFIED="1779504767437" TEXT="halte mich an das Schema aus den letzten Jahren"/>
|
||||||
<node CREATED="1779504768195" ID="ID_531662550" MODIFIED="1779504785771" TEXT="Kurzform und Langform ergeben auch zusammen einen Text"/>
|
<node CREATED="1779504768195" ID="ID_531662550" MODIFIED="1779504785771" TEXT="Kurzform und Langform ergeben auch zusammen einen Text"/>
|
||||||
|
<node BACKGROUND_COLOR="#accdc3" COLOR="#3f4015" CREATED="1779554095269" ID="ID_1290433500" MODIFIED="1779554118621" STYLE="fork" TEXT="habe mir noch einige Vorschläge von ChatGPT geben lassen">
|
||||||
|
<edge COLOR="#808080" STYLE="bezier" WIDTH="thin"/>
|
||||||
|
<icon BUILTIN="wizard"/>
|
||||||
|
</node>
|
||||||
<node BACKGROUND_COLOR="#f0d5c5" COLOR="#990033" CREATED="1779504789243" ID="ID_649621958" MODIFIED="1779504956035" TEXT="Klären: kann man mit IO_URING gezielt bestimmte Bereiche eines Files lesen?">
|
<node BACKGROUND_COLOR="#f0d5c5" COLOR="#990033" CREATED="1779504789243" ID="ID_649621958" MODIFIED="1779504956035" TEXT="Klären: kann man mit IO_URING gezielt bestimmte Bereiche eines Files lesen?">
|
||||||
<arrowlink COLOR="#d60714" DESTINATION="ID_628756969" ENDARROW="Default" ENDINCLINATION="497;-40;" ID="Arrow_ID_347382746" STARTARROW="None" STARTINCLINATION="347;-14;"/>
|
<arrowlink COLOR="#d60714" DESTINATION="ID_628756969" ENDARROW="Default" ENDINCLINATION="497;-40;" ID="Arrow_ID_347382746" STARTARROW="None" STARTINCLINATION="347;-14;"/>
|
||||||
<icon BUILTIN="help"/>
|
<icon BUILTIN="help"/>
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue