diff --git a/FrOSCon26/FrOSCon.26.mm b/FrOSCon26/FrOSCon.26.mm index 65671fb..368fc52 100644 --- a/FrOSCon26/FrOSCon.26.mm +++ b/FrOSCon26/FrOSCon.26.mm @@ -8,8 +8,10 @@ - + + + @@ -34,6 +36,9 @@ + + + @@ -99,6 +104,7 @@ + @@ -114,14 +120,52 @@ + + + + +

+ Das sollte der gedankliche Leitfaden sein: nicht I/O selber ist "langsam", nicht Mutexes sind "langsam", sondern die Koordination über Thread-Grenzen hinweg, außerhalb eines determinstischen Schemas ist die eigentliche Schwierigkeit +

+ + + - + + + - + + + + +

+ Dieser Fokus hat sich ergeben aus dem Kontext, in dem der Vortrag angeboten wird — und er paßt gut zu meiner Hinterabsicht, Lumiera ins Gespräch zu bringen.... +

+

+ +

+

+ Im Rückblick erweist sich diese Wahl als angemessen: das Publikum hätte mit einem allgemeiner / mehr geschichtlich / mehr theoretisch ausgerichtetem Vortrag wenig anfangen können +

+ + + - + + + + + +

+ Die Themenwahl, der Fokus auf »hard-core« und die Darstellung waren ein voller Erfolg und haben sehr gut zur Veranstaltung gepaßt +

+ + + + @@ -131,10 +175,22 @@ + + + + + + - + + + + + + + @@ -152,7 +208,8 @@ - + + @@ -269,8 +326,8 @@ - - + + @@ -333,9 +390,7 @@ - - - +

allein der Umstand daß Cores direkt an einem Cache hängen, genügt, um die Notwendigkeit der Synchronisation aufzuzeigen ... daher ist das Schlüsselwort auch "cache-coherence" @@ -454,9 +509,6 @@ - - - @@ -695,7 +747,16 @@ - + + + + +

+ Wie ich das typischerweise mache, wird das mehr eine begleitende Animation, die parallel zu meinem gesprochenen Text abläuft. Oftmals kann mich diese Darstellung entlasten, da ich auf die Folien die gedanklich-systematische Formulierung schreiben kann, und diese nicht mehr eigens im gesprochenen Gedankengang ausgeführt werden muß (die Leute lesen es ja gleichzeitig) +

+ + + @@ -707,15 +768,13 @@ - + - - - +

Producer: @@ -769,18 +828,26 @@ - - + + - - + + - - + + + - - + + + + + + + + + @@ -807,9 +874,7 @@ - - - +

Treat it as a tool, @@ -837,9 +902,7 @@ - - - +

in software, an abstraction provides access @@ -946,9 +1009,7 @@ - - - +

Ringbuffer:
@@ -1062,9 +1123,7 @@ - - - +

Thread polls »readiness« ⟹ Reactor @@ -1074,9 +1133,7 @@ - - - +

Thread polls »completion« ⟹ Proactor @@ -1206,7 +1263,8 @@ - + + @@ -1351,9 +1409,7 @@ - - - +

devel/froscon/research/drawings/DemoApp.svg @@ -1363,9 +1419,7 @@ - - - +

...alles ausgeblendet, außder dem einen Callback, den ich brauche; Seitengröße an neue Präsentation angepaßt, als PNG exportiert, in Libreoffice gelinkt @@ -1378,7 +1432,8 @@ - + + @@ -1403,6 +1458,9 @@ + + + @@ -1421,7 +1479,7 @@ - + @@ -1455,7 +1513,11 @@ - + + + + + @@ -1465,9 +1527,7 @@ - - - +

skip all animations and start with next / previous slide @@ -1483,9 +1543,7 @@ - - - +

...weil diese auch jeweils ein ganzes Stylesheet definieren. @@ -1506,9 +1564,7 @@ - - - +

drag-n-drop, oder Slide > Import from File @@ -1539,9 +1595,7 @@ - - - +

es ergeben sich daraus die Pixel @@ -1581,9 +1635,7 @@ - - - +

Transition auf "None", dann Zeit eingeben, dann Transition auf "Wipe", dann Richtung wählen.... @@ -1599,9 +1651,7 @@ - - - +

...das würde zwar letztlich funktionieren, wenn man dann ab dem 2.Absatz alle auf "with previous" umschaltet .... ist aber irre unübersichtlich.  Verstehe nicht, warum er das manchmal macht (vielleicht, wenn man einzelne Absätze schon formatiert hat?) @@ -1611,9 +1661,7 @@ - - - +

Zwei Objekte markieren, Animation hinzufügen, dann Animation des 2. Objekts wieder löschen ⟹ Ergebnis: man bekommt die Animation auf den ganzen Frame, nicht einzelne Absätze @@ -1625,16 +1673,10 @@ - - - - - - - - + + @@ -1648,8 +1690,11 @@ - - + + + + + @@ -1658,9 +1703,7 @@ - - - +

das war ohnehin nur ein Extra, und letzlich off-topic @@ -1673,40 +1716,127 @@ - - + + + + + + + + + + + +

+ aber ich hab ihn aufmerksam gehalten +

+ + + + - + + + + +

+ hat auch tatsächlich 52 Minuten gedauert; +

+

+ ganz bewußt habe ich für diese Probe auf »Expansion« gesetzt, um noch eine bessere Gesamtdarstellung zu erzielen. +

+ + + - + + + + - - + + - - + + + + +

- ich hatte mir ja im Vorfeld überlegt, was für ein Publikum vermutlich auf der FrOSCon herumläuft; dem entsprechend war der Vortrag auf »Allgemeinbildungs-Niveau« angelegt.... (was allerdings auch bereits für Benny eine erhebliche Herausforderung war, also vermutlich auch für andere Leute) + Dieses Mal habe ich viele Leute gesehen, die ich als native/system-Programmers einschätzen würde. Das war also ein deutlicher Unterschied zu den vorausgegangenen Jahren, und war wohl einfach dem Thema geschuldet, da hatte ich etwas zu bieten. Es waren auch ein paar Studenten und Studentinnen da (die haben sich vorsichtig ganz hinten reingesetzt). Direkt in der Mitte saßen ein paar 20-30järige, die waren richtig begeistert von dem Vortrag. Anschließend habe ich zwei Anfragen auf LInked-In bekommen. +

+

+ Und der Mann, der letztes Jahr mit dem Schlaf gekämpft hatte, war auch wieder da (und hat wieder mit dem Schlaf gekämpft) ... zwei weitere bekannte Gesichter waren auch noch da, an einen davon konnte ich mich jetzt in allen drei Vorträgen erinnern. Das alles zeigt mir, daß ich nicht völlig falsch liege, und etwas bei den Leuten ansprechen kann. +

+ + + + + + + + +

+ Dieses Mal hatte ich einen weit gespannten Bogen zur Darstellung, der kaum in 45 Minuten unterzubringen war. Und ich wollte unbedingt den ganzen »hard-core«-Code im einzelnen Erläutern, weil das nun schon in das Schema meiner Vorträge paßt.... +

+

+ Also habe ich alles sehr gründlich als Präsentation aufbereitet, mit umfangreichen Animationen und präzise ausgewählten Darstellungen, was ich ihm Code zeigen möchte. Wegen der gleichzeitig laufenden Vorbereitung auf das Lumiera-Workflow-Treffen mit Wouter Verweijlen bin ich massiv unter Zeitdruck gekommen. Das Anlegen all dieser Animationen in Libreoffice-Calc war ein Albtraum (und ich hab noch die letzten zwei Nächte mehr oder weniger durchgemacht, die letzte sogar in Bernbach). Allerdings waren diese präzise vorbereiteten Animationen dann auch das tragende Element, das die Darstellung des Codes überhaupt erst möglich gemacht hat. Benny hat das unabhängig bemerkt und eigens hervorgehoben: durch meine Bullet-Listen auf der rechten Seite hatte ich genau dosierte Trittsteine und habe mich nicht in der Erklärung von Details verzettelt. Nach Benny's Einschätzung war meine Code-Darstellung grade noch an der Grenze des Zumutbaren ... aber beim Publikum ist das sehr gut angekommen

- - - + + + + +

+ Etwa 20 waren frühzeitig da; in diesem Hörsaal war vorher ein Vortrag zu DTrace gewesen. Direkt zum Start und kurz danach kam noch ein ganzer Schwall an Leuten +

+ + + + + + + +

+ Es hat sich also sehr gut bewährt, daß ich so ausführlich in die Hintergründe gegraben hatte; ich konnte alle Fragen sehr locker und kompetent beantworten. Gefragt wurde nach dem echten Overhead, nach zero-Copy, und nach der architektonischer Problematik, die sich Linux durch io_uring nun eingehandelt hat. +

+ + + + + + + +

+ Zwei davon waren mit Systemprogrammierung vertraut, kannten aber tatsächlich io_uring noch nicht. Es ging um epoll() und memory-mapping, und wie man das wohl auf io_uring umschreiben könne. +

+ + + + + + + +

+ ...die haben in den Seminarraum reingeschaut; wir haben dann gesagt, wir machen hier User-Interface-Design für »Lumiera« — „ah, davon habe ich doch kürzlich gehört“ (wie sich dann herausstellte, hatten die Leute in beiden Fällen das Programm gelesen, und hatten sich so den Namen »Lumiera« gemerkt). Das ist jetzt keine dramatische »Reichweite«, aber es zeigt mir, daß mein Ansatz funktioniert, nämlich Themen von allgemeinem Interesse als Vehikel zu verwenden, anstatt Lumiera direkt zu thematisieren. +

+ + @@ -1714,11 +1844,40 @@ - - + + - - + + + + +

+ Halte mich an die Methode aus dem letzten Jahr: +

+
    +
  • + extrahiere das Recherche- und das Code-Subdirectory per git-subtree split +
    • +
  • + binde diese dann per git-subtree merge an der korrekten Stelle in den Documentation-Tree ein +
    • +
  • + verwende eine einfache Asciidoc-Container-Seite für jeden Code, und includiere den in einen Code-Block +
    • +
+ + + + + + + + +

+ ...aber bei dem Umfang noch gut gerechtfertigt. +

+ + @@ -1741,20 +1900,33 @@ + - - + +

- neues Git-Repo nur für den Demo-Code anlegen + diesmal ist es nicht so viel Code, und auch nur ein einziges Directory ... und zudem glaube ich nicht, daß sich irgendjemand im Ernst für solchen Demo-Code interessiert; es geht mehr darum, daß die Recherche auf Lumiera.org steht und längerfristig gefunden wird.

- + + + + + +

+ Im Anschluß an das Code-Review hatte ich Claude eine Gliederung auf dieser Seite vorgegeben, und die Abschnitte befüllen lassen; das hat sehr gut funktioniert, und Claude hat zudem auch noch zwei SVG-Diagramme erzeugt, die ich auch für den Vortrag verwenden konnte. +

+ + + + + @@ -1776,9 +1948,7 @@ - - - +
  • @@ -1803,9 +1973,11 @@ - - + + + + @@ -1821,20 +1993,20 @@ - - - - - - + + + + + + - - - - + + + + - - + + @@ -1897,12 +2069,12 @@ - +

    - der Talk kommt fachlich daher — ist aber eigentlich ein Narrativ  und allgemeines Lehrstück + der Talk kommt fachlich daher — ist aber eigentlich ein Narrativ und allgemeines Lehrstück

    @@ -1910,11 +2082,11 @@     - - + + - + @@ -1928,7 +2100,8 @@ - + + @@ -1942,8 +2115,39 @@ + + + + +
    +
    + Glauber Costa +
    +
    +

    + Glauber Costa is the founder and CEO of Turso, the database company building the next evolution of SQLite. Before that he was a staff software engineer at Datadog, a Distinguished Engineer at ScyllaDB, and a core contributor to the Linux kernel at Red Hat. +

    +
    +
    + +     + + + + + + + + +

    + Das ist Gabriel's eigenes Gegenargument gegen die catchy-phrase »worse is better« — veröffentlicht unter einem Pseudonym +

    + +     +     +         @@ -2014,7 +2218,7 @@     - + @@ -2033,8 +2237,9 @@ - - + + + @@ -2090,7 +2295,7 @@ - + @@ -2147,7 +2352,8 @@ - + + @@ -2443,25 +2649,8 @@ - - - - - - - - - -

    - Ist notwendig für andere Sprach-Bindings oder ältere Compiler; dadurch hat dann natürlich jeder Call einen Overhead -

    - -     -     -     -     -     - + + @@ -2537,7 +2726,7 @@ - + @@ -2547,10 +2736,24 @@ + + + + + + + +

    + Ist notwendig für andere Sprach-Bindings oder ältere Compiler; dadurch hat dann natürlich jeder Call einen Overhead +

    + +     +     +     - + @@ -2954,7 +3157,7 @@ - + @@ -3005,7 +3208,7 @@ - + @@ -3039,10 +3242,10 @@ - + - + @@ -3094,7 +3297,7 @@ - + @@ -3164,8 +3367,8 @@ - - + + @@ -3256,7 +3459,7 @@ - + @@ -3274,13 +3477,14 @@ - + - + + @@ -3302,13 +3506,25 @@ + + + + +

    + Beachte: am Ende vertritt er beide Positionen — und beruft sich auf Christopher Alexander +

    + +     + +     - - + + + @@ -3368,8 +3584,8 @@ - - + + @@ -4888,9 +5104,9 @@ - - - + + + @@ -5177,7 +5393,9 @@ - + + + @@ -5354,9 +5572,9 @@ - - - + + + @@ -5463,13 +5681,13 @@ - + - + @@ -5485,15 +5703,17 @@ - - + + - + + + @@ -5567,12 +5787,42 @@ - + - - + + + + + +

    + Nach dem Code-Review hatte ich Claude eine vorbereitete Gliederung für den DemoSpec-Text gegeben, und ihn instruiert, diese gemäß den nun gewonnenen Einsichten auszufüllen (incl. zitiertes Code-Beispiel); dazu habe ich auch einige Diagramme generieren lassen, die ich dann in der Präsentation verwenden konnte +

    + +     + + +     + + + + + + +

    + Genauer gesagt, die Formulierung war extrem verknappt, und im Resultat irreführend: es erweckt den Anschein, als würden wir die Frame-Nummer auch in den Timer-Ticks verwenden — was nicht korrekt ist, da wir genau dort einen Sentinel-Wert verwenden, um die Timer-Events zu erkennen +

    + +     + +     + + + + + + @@ -5592,7 +5842,7 @@ - + @@ -5610,18 +5860,21 @@ - - + + - - + + - - + + - + + + + @@ -6737,8 +6990,8 @@ - - + + @@ -6746,8 +6999,8 @@ - - + + @@ -6838,16 +7091,16 @@ - - + + - + - - + + @@ -6855,12 +7108,13 @@ - - - + + + + diff --git a/FrOSCon26/research/DemoSpec.txt b/FrOSCon26/research/DemoSpec.txt index 9fc2103..67ea290 100644 --- a/FrOSCon26/research/DemoSpec.txt +++ b/FrOSCon26/research/DemoSpec.txt @@ -6,21 +6,33 @@ Specification for the Demo :toclevels: 3 This specification is the result of our research and provides a short summary of the relevant -aspects of io_uring. Furthermore, the specific design of the demo code is explained and motivated. +aspects of io_uring. A simple video player was developed as demo code and explained in a +https://programm.froscon.org/froscon2026/talk/d1635d41-1fac-4a43-909b-b8f07672c541/[Talk at FrOSCon 2026]. +The following text gives a summary and motivates the specific design of the demo code. IO_URING kernel interface ------------------------- -`io_uring` is a Linux kernel interface for asynchronous I/O (available since kernel 5.1). It is -built around two ring buffers, shared and memory-mapped between user space and kernel: a -_Submission Queue_ (SQ) and a _Completion Queue_ (CQ). The application fills a _Submission Queue -Entry_ (SQE) describing an operation — read, write, timeout, accept, … — and notifies the kernel; -once the operation has run to completion, the kernel deposits a _Completion Queue Entry_ (CQE) +`io_uring` is a Linux kernel interface for asynchronous I/O (available since kernel 5.1). +It is built around two ring buffers, shared and memory-mapped between user space and kernel: + + * a _Submission Queue_ (SQ) + * a _Completion Queue_ (CQ) + +Instead of invoking the Kernel directly to instruct I/O operation, the application rather +fills one or several _Submission Queue Entries_ (SQE) that describe I/O operations -- +read, write, timeout, accept,... -- and then notifies the kernel; once the operation has run +to completion, the kernel deposits a _Completion Queue Entry_ (CQE) into the other queue, carrying the result. Submission and completion are thus decoupled: many operations can be in -flight concurrently, and — unlike the classic `read(2)`/`write(2)` model — no thread has to block -while an operation is pending, and no syscall is required per individual operation (a single -`io_uring_submit` hands over a whole batch, and with a polled queue even that can be elided). +flight concurrently, and -- unlike the classic `read(2)`/`write(2)` model -- no thread or process +has to _block_ while an operation is pending, and no syscall is required per individual operation. +One single `io_uring_submit` hands over a whole batch.footnote:[ + As an extension, the kernel can even _poll_ on the submission queue, so that any syscall can elided. + This setup lends itself well for high-performance computing with lots of tiny I/O operation, but + has the obvious downside that now the _kernel_ has to run a dedicated polling thread -- which + in the typical use case means to dedicate one core entirely for the dispatch of operations. +] For a media application this is attractive: playback and editing require moving large volumes of data with predictable latency, ideally without dedicating a blocked thread to each stream. The @@ -45,23 +57,26 @@ and binary symbol for each function; obviously using these explicit calls adds a Reading of raw video as example of using IO_URING ------------------------------------------------- -Playing a raw video file is a minimal yet realistic streaming workload — a steady, latency-bound -sequence of large block reads that has to keep pace with a fixed frame beat — and therefore serves -well to demonstrate the decoupling of submission and completion. +Playing a raw video file is a minimal yet realistic streaming workload -- a steady, latency-bound +sequence of large block reads that has to keep pace with a fixed frame beat -- and therefore serves +well to demonstrate the decoupling of submission and completion. Another aspect demonstrated in here +is the need for _flow control,_ so that the consumer (in this case, the GUI display) is not flooded +with the data being read. The »Proactor« pattern ~~~~~~~~~~~~~~~~~~~~~~ It is instructive to contrast two patterns for event-driven I/O. In the *Reactor* pattern the program waits for a resource to become _ready_ and then performs the operation itself, synchronously, from within the event handler. In the *Proactor* pattern the operation is _initiated_ up front, the -operating system carries it out, and a handler is invoked on _completion_ — with the result already -available. io_uring is inherently proactor-shaped: we submit a read, the kernel performs it, and we -later receive a completion that carries the outcome. +operating system carries it out, and a handler is invoked on _completion_ -- with the result already +available. While io_uring in itself is an abstraction and hides the actual I/O handling in the kernel, +the form of the interface fits well into the Proactor pattern: a read operation is submitted, the kernel +performs it, and a completion can later be received to follow-up with processing of the retrieved data. -The demo realises this with a single dedicated *»Proactor« thread* (`UringVideoPlayer::playVideo()`). +The demo realises this pattern with a single dedicated *»Proactor« thread* (`UringVideoPlayer::playVideo()`). Its whole life is one loop around `io_uring_wait_cqe()`: block until the next completion arrives, -dispatch it to a handler based on a tag carried in the CQE `user_data`, then re-arm further work and -submit. Two kinds of completion are multiplexed onto the same ring and told apart by that tag: +dispatch it to a handler based on a tag carried in the CQE `user_data`, then re-supply further work items +and submit. Two kinds of completion are multiplexed onto the same ring and told apart by that tag: * a periodic *frame-beat timer* (a multishot timeout, tagged with the sentinel `TICK_TAG`) * *read completions* for individual video frames (tagged with the running frame number) @@ -74,23 +89,26 @@ The code is deliberately split so that the io_uring logic stands apart from the The subject of this write-up. Owns the ring, the buffers and their state, and the Proactor thread. `EglDisplay` (egl-display.hpp):: - A self-contained OpenGL/EGL display component, best read as a _black box_. It is reached only - through the small `PixelFeed` interface — `retrieveFrame()` to obtain the current frame's pixel - data and `completedFrame()` to signal that the data is no longer needed. + A self-contained OpenGL/EGL display component, best read as a _black box_.footnote:[ + In fact we re-used the demo code from last year's talk at FrOSCon(2025) + ] + , where various technologies were explored to display raw video frames, using the GPU for acceleration. + ] + It is reached only through the small `PixelFeed` interface -- `retrieveFrame()` to obtain the current + frame's pixel data and `completedFrame()` to signal that the data was handed over to the GPU (using OpenGL) + and the buffer can thus be repurposed for the next frame. `GtkUringApp` (gtk-uring-app.hpp):: A minimal GTK-3 host application providing the window, the start button and the shutdown hook. Two threads are involved: the *GTK/GUI main thread* and the *Proactor thread*. The player is -constructed on the GUI thread and — following a RAII spirit — immediately launches the Proactor +constructed on the GUI thread and -- following a RAII pattern -- immediately launches the Proactor thread from its constructor. Note that `proactorThread_` is declared _last_ among the members, so -by C++'s declaration-order construction rule all buffers, state and the display connection are fully -initialised before the thread can observe them. Data flows in one direction: the Proactor reads file -blocks into buffers and publishes a ready frame; the GUI thread uploads that frame into a GPU texture -and draws it. +by C++'s declaration-order construction rule all the buffers, state and the display connection machinery +are fully initialised before the thread can observe them. Data flows in one direction: the Proactor reads +blocks of raw video data from the file into buffers and publishes a ready frame; the GUI thread uploads +that frame into a GPU texture and draws it. -.Overall data flow across the two threads, with the atomic baton `toDisplay_` between them -image::diagrams/thread-handoff.svg[Cross-thread data flow and visibility handoff,align="center"] The problem of data visibility synchronisation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -101,8 +119,8 @@ The two threads share the buffer array, which raises two questions of _cross-thr . the Proactor must know when the GUI is _done_ with a buffer, so the slot may be reused safely. Rather than making the entire state array atomic, or sharing cursor indices across the boundary, the -design funnels the whole cross-thread contract through a *single* atomic — `std::atomic -toDisplay_` — which _is_ the payload rather than a key to look it up: +design funnels the whole cross-thread contract through a *single* atomic -- `std::atomic +toDisplay_` -- which _is_ the payload rather than a key to look it up: [cols="1,3"] |=== @@ -115,27 +133,29 @@ and thereby is guaranteed to see the buffer contents written before the store. W GUI writes `nullptr` with a *release*-store, which the Proactor observes with an *acquire*-load before reclaiming the slot. This is the classic single-slot, single-producer/single-consumer _mailbox_ handoff: the two release/acquire pairs establish the _happens-before_ relations that carry -buffer-content visibility in both directions. Everything else — the `BuffState` array and the frame -cursors — is confined to the Proactor thread alone and therefore needs no synchronisation at all. +buffer-content visibility in both directions. Everything else -- the `BuffState` array and the frame +cursors -- is confined to the Proactor thread alone and therefore needs no synchronisation at all. Buffer data structure ~~~~~~~~~~~~~~~~~~~~~ A fixed set of `RING_SIZ` frame-sized buffers is held in a `unique_ptr` and used as a -_ring_: the slot for a frame is simply `frameNr % RING_SIZ`. Frame numbers themselves are monotonic, -unbounded counters — `nextRead_` for the production side and `nextShow_` for the consumption side — +_ring_: the slot for a frame can be accessed by taking `frameNr % RING_SIZ`. Frame numbers themselves are +monotonic, unbounded counters -- `nextRead_` for the production side and `nextShow_` for the consumption side -- so the modulo maps an ever-growing frame index onto the small cyclic set of buffers. The same running -frame number does triple duty: as the SQE `user_data` tag that later identifies the completed read, as -the key of the per-slot IO timer, and (via modulo) as the slot index. The io_uring queues are sized -`RING_SIZ+1`, the one extra entry accommodating the always-present timer event. +frame number does double duty: as the SQE `user_data` tag that later identifies the completed read, +and (via modulo) as the slot index. A similar addressing scheme is also employed to access a _state flag_, +that is associated with each buffer slot. Notably the io_uring kernel queues are sized as `RING_SIZ+1`, +the one extra entry accommodating the always-present timer event. .Snapshot of the ring: the two cursors chase each other, the gap holds the in-flight frames -image::diagrams/ring-buffer.svg[Ring-buffer snapshot with production and consumption cursors,align="center"] +image::demo/ring-buffer.svg[Ring-buffer snapshot with production and co]nsumption cursors,align="center"] Tracking state of a buffer slot ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Each slot carries a `BuffState` describing where in its life-cycle it currently is: - FREE -> READ -> FEED -> SHOW -> FREE (with STOP as terminal state) +.Buffer-slot state engine (transitions driven by the Proactor thread) +image::demo/state-engine.svg[State-engine for buffer slots,align="center"] [cols="1,4"] |=== @@ -148,10 +168,9 @@ Each slot carries a `BuffState` describing where in its life-cycle it currently Because this array is written and read _only_ from the Proactor thread, it is a plain, non-atomic array. The `SHOW` marker records _which_ slot is presently on loan to the GUI, complementing the -`toDisplay_` baton which records _whether_ it has come back yet. +`toDisplay_` baton which marks _whether_ the GUI has finished transferring the pixel data into +the GPU for display. -.The life-cycle of a single buffer slot (all transitions driven by the Proactor thread) -image::diagrams/state-engine.svg[Buffer-slot state engine,align="center"] Control flow and backpressure ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -162,7 +181,7 @@ a completed frame is _reclaimed_, then the next frame is _published_ if its data The essential back-pressure is that the single show-cursor does not advance while a frame is still out with the GUI (`toDisplay_` non-null). No new frame is published until the previous one has been -displayed and released, and — since reads only ever refill `FREE` slots — a lagging consumer naturally +displayed and released, and -- since reads only ever refill `FREE` slots -- a lagging consumer naturally throttles production. A frame whose read has not completed in time, or whose predecessor the GUI has not yet released, simply _slips_ to the next tick. @@ -173,7 +192,7 @@ UringVideoPlayer::maybeDisplayFrame() { // reclaim previously displayed buffer if (SHOW == buffState_[nextShow_ % RING_SIZ] and - nullptr == toDisplay_.load (memory_order_acquire)) // <1> + nullptr == toDisplay_.load (memory_order_acquire)) // <1> { buffState_[nextShow_ % RING_SIZ] = FREE; ++nextShow_; // <2> @@ -196,7 +215,7 @@ UringVideoPlayer::maybeDisplayFrame() } -------------------------------------------------------------------------- <1> reclaim only the slot lent to the GUI, and only once it cleared the baton -<2> advance the single show-cursor — this is where the back-pressure lives +<2> advance the single show-cursor -- this is where the back-pressure lives <3> the frame data has arrived (`FEED`): hand it over <4> the release-store publishes the buffer and makes its contents visible to the GUI -<5> `READ` (IO still pending) or `SHOW` (GUI not yet caught up) ⇒ the frame slips to the next tick +<5> `READ` (IO still pending) or `SHOW` (GUI not yet caught up) -> the frame slips to the next tick diff --git a/FrOSCon26/research/Resources.txt b/FrOSCon26/research/Resources.txt index 2d6e501..e00a31f 100644 --- a/FrOSCon26/research/Resources.txt +++ b/FrOSCon26/research/Resources.txt @@ -6,11 +6,27 @@ Topic: Kernel and IO -------------------- -Asynchronous IO -~~~~~~~~~~~~~~~ - - TODO +Background: Asynchronous IO +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + - https://www.kegel.com/c10k.html[The C10k problem]: a very frequently quoted (and actually + quite old page (2003-2011) that analyses the problem of serving more than 10k requests from a server, + the bottlenecks and possible solution strategies. + - https://dreamsongs.com/WorseIsBetter.html[The crux with »worse is better«]. This is a summary + by Richard P. Gabriel himself, looking back, and he still has not made up his mind if _worse is better_. + The tension between the »*Berkely*« (or »New Jersy«) style and the »*MIT* style« helps to understand + design choices taken in the original design of Unix -- and with the advent of fast SSDs and finally + an interface like `io_uring` in the Linux kernel, all these unresolved questions are coming back + to haunt us. + + * https://dreamsongs.com/Files/WorseIsBetterPositionPaper.pdf[Back to the Future: Is Worse (Still) Better?] + * https://dreamsongs.com/Files/ProWorseIsBetterPosition.pdf[Back to the Future: Worse (Still) is Better!] +History of io_uring +~~~~~~~~~~~~~~~~~~~ + - https://web.archive.org/web/20251010072644/https://kernel.dk/io_uring.pdf[Whitepaper(2019): Efficient IO with io_uring] + - https://lwn.net/Articles/810414/[LWN.net: The rapid growth of io_uring] + - https://web.archive.org/web/20200421163135/https://thenewstack.io/how-io_uring-and-ebpf-will-revolutionize-programming-in-linux/[2020: io_uring and eBPF will revolutionize programming in linux] IO_URING @@ -18,10 +34,13 @@ IO_URING Introduction ~~~~~~~~~~~~ * https://unixism.net/loti/[»Lord of the uring«] + * https://towardsdev.com/io-uring-vs-epoll-syscall-overhead-linux-performance-a3d7fcc6c9b9?gi=e429808ce081[The Linux I/O Interface That Finally Makes Sense] + * https://lwn.net/Articles/776703/[LWN.2019: Ringing in a new asynchronous I/O API] (explains the low-level API) Reference ~~~~~~~~~ + * https://man7.org/linux/man-pages/man7/io_uring.7.html[Manpage (low-level API)] * https://unixism.net/loti/ref-liburing/index.html[Overview (liburing)] ** https://unixism.net/loti/ref-liburing/submission.html[Submission and Submission helpers] @@ -52,6 +71,3 @@ Reference - https://unixism.net/loti/ref-liburing/submission.html#c.io_uring_prep_epoll_ctl[io_uring_prep_epoll_ctl] - https://unixism.net/loti/ref-liburing/submission.html#c.io_uring_prep_poll_add[io_uring_prep_poll_add] - https://unixism.net/loti/ref-liburing/submission.html#c.io_uring_prep_poll_remove[io_uring_prep_poll_remove] - - -[purple]#maybe more resources regarding the low-level API...?# diff --git a/FrOSCon26/research/TimeScales.txt b/FrOSCon26/research/TimeScales.txt index 366bc47..0c2e78d 100644 --- a/FrOSCon26/research/TimeScales.txt +++ b/FrOSCon26/research/TimeScales.txt @@ -10,18 +10,23 @@ cycle up to the point where a human gives up waiting. Numbers on that scale resist intuition, and so we tend to reason about ``fast'' and ``slow'' without noticing that the two words can be separated by a factor of a million. -The following table collects the figures that matter in this context. It is -meant as a rough orientation, not as a benchmark result. +CAUTION: Knowledge-Technology (»AI«) was used to structure and investigate this topic, + starting from my own practical observations. Additional corroborating sources + were identified and findings integrated into a coherent time axis view. NOTE: All values are order-of-magnitude figures for contemporary desktop and laptop x86 hardware. Expect a deviation of 2--3{nbsp}x depending on microarchitecture, memory configuration, kernel version and load. The _ratios_ are the point, not the digits. -The last column rescales everything by a single constant factor, chosen so -that one CPU clock cycle (≙ 0.3{nbsp}ns at 3.3{nbsp}GHz) becomes one second -of human time. This is the most effective way we know to convey what a cache -miss or a disk seek actually costs from the machine's point of view. +The following table collects the figures that matter in this context. It is +meant as a rough orientation, not as a benchmark result. + +The last column in the table rescales everything by a single constant factor, +chosen so that one CPU clock cycle (≙ 0.3{nbsp}ns at 3.3{nbsp}GHz) becomes +one second of human time. Presenting this kind of _human scale proportions_ +is an effective way to convey what a cache miss or a disk seek actually costs, +and what this means, assuming a _machine's point of view_. Overview Table @@ -33,54 +38,53 @@ Overview Table 4+^s| Nanoseconds -- inside the CPU -| 0.3{nbsp}ns | one clock cycle | at 3.3{nbsp}GHz; the machine's own time quantum | *1 second* -| ~1{nbsp}ns | L1 cache hit | about 4 cycles, fully pipelined | ≈{nbsp}3 seconds -| ~1.5{nbsp}ns | `sfence` | store fence, ≈{nbsp}5 cycles; rarely needed on x86 | ≈{nbsp}5 seconds +| 0.3{nbsp}ns | one clock cycle | at 3.3{nbsp}GHz; the machine's own time quantum | *1 second* +| ~1{nbsp}ns | L1 cache hit | about 4 cycles, fully pipelined | ≈{nbsp}3 seconds +| ~1.5{nbsp}ns | `sfence` | store fence, ≈{nbsp}5 cycles; rarely needed on x86 | ≈{nbsp}5 seconds | ~2.7{nbsp}ns | `lfence` | ≈{nbsp}9 cycles; no ordering semantics on x86 beyond blocking speculation | ≈{nbsp}9 seconds -| ~4{nbsp}ns | L2 cache hit | ≈{nbsp}12--15 cycles | ≈{nbsp}13 seconds -| ~5{nbsp}ns | `lock add $0,(%rsp)` | the actual `smp_mb()` on Linux; ≈{nbsp}half the cost of `mfence` footnoteref:[lkml,LKML thread "[PATCH 3/4] x86,asm: Re-work smp_store_mb()", Nov 2015. Davidlohr Bueso measures XCHG against MFENCE on Ivy Bridge; Linus Torvalds reports that on Haswell "lock addq" costs pretty much exactly half of MFENCE, and explains that locked operations were the ones CPU designers had reason to optimise -- https://lkml.iu.edu/hypermail/linux/kernel/1511.0/01052.html] | ≈{nbsp}17 seconds -| ~5{nbsp}ns | branch misprediction | pipeline refill | ≈{nbsp}17 seconds -| ~10{nbsp}ns | `mfence` / full barrier | ≈{nbsp}23--33 cycles; comparable to a `lock`-prefixed RMW footnoteref:[downs,Travis Downs, "A Concurrency Cost Hierarchy", 6 July 2020, see the discussion of "vanilla" versus "non-vanilla" instructions: on x86 plain accesses and non-atomic read-modify-writes are cheap while LOCK-prefixed RMWs are an order of magnitude slower, with MFENCE comparable to a locked instruction -- https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html] | ≈{nbsp}33 seconds -| ~15{nbsp}ns | L3 cache hit | shared, hence variable | ≈{nbsp}50 seconds -| ~13{nbsp}ns | DRAM, open row | a plain load that happens to hit an already-activated row footnoteref:[cxl,Measured values in Table III of: Performance Characterizations and Usage Guidelines of Samsung CXL Memory Module Hybrid Prototype, arXiv:2503.22017. Local DDR5 median latency 122.9 ns under Intel MLC pointer chasing versus 13.1 ns for a plain load; methodology (rdtsc, 10 000 repetitions, median) described in section III-C -- https://arxiv.org/pdf/2503.22017] | ≈{nbsp}43 seconds -| ~14{nbsp}ns | DRAM, first access | device-level figure; essentially unchanged from DDR4 to DDR5 footnote:[Measurement-oriented analysis of DDR5 sub-channel architecture; the relevant passage is in section "Latency Implications", noting that both DDR4 and DDR5 sit at approximately 14 ns first-access latency at their respective standard speeds. arXiv:2605.08725 -- https://arxiv.org/pdf/2605.08725] - | ≈{nbsp}47 seconds -| 40--100{nbsp}ns | contended cache line | core-to-core transfer; strongly topology dependent (same CCX / cross CCX / cross socket) | 2--5 minutes -| ~100{nbsp}ns | DRAM random access | full miss through the whole hierarchy; 122.9{nbsp}ns measured by pointer chasing on DDR5 footnoteref:[cxl] | *≈{nbsp}5 minutes* +| ~4{nbsp}ns | L2 cache hit | ≈{nbsp}12--15 cycles | ≈{nbsp}13 seconds +| ~5{nbsp}ns | `lock add $0,(%rsp)` | the actual `smp_mb()` on Linux; ≈{nbsp}half the cost of `mfence` <> | ≈{nbsp}17 seconds +| ~5{nbsp}ns | branch misprediction | pipeline refill | ≈{nbsp}17 seconds +| ~10{nbsp}ns | `mfence` / full barrier | ≈{nbsp}23--33 cycles; comparable to a `lock`-prefixed RMW <> | ≈{nbsp}33 seconds +| ~15{nbsp}ns | L3 cache hit | shared, hence variable | ≈{nbsp}50 seconds +| ~13{nbsp}ns | DRAM, open row | a plain load that happens to hit an already-activated row <> | ≈{nbsp}43 seconds +| ~14{nbsp}ns | DRAM, first access | device-level figure; essentially unchanged from DDR4 to DDR5 <> | ≈{nbsp}47 seconds +| 40...100{nbsp}ns | contended cache line | core-to-core transfer; strongly topology dependent (same CCX / cross CCX / cross socket) | 2--5 minutes +| ~100{nbsp}ns | DRAM random access | full miss through the whole hierarchy; 122.9{nbsp}ns measured by pointer chasing on DDR5 <> | *≈{nbsp}5 minutes* 4+^s| Nanoseconds to microseconds -- entering the kernel -| ~25{nbsp}ns | syscall via vDSO | not a real trap; almost as cheap as a function call footnoteref:[arkanis,"Measurements of system call performance and overhead", Arkanis Development, 5 Jan 2017 -- compares a plain function call against getpid() issued via the syscall instruction and via the vDSO -- https://arkanis.de/weblog/2017-01-05-measurements-of-system-call-performance-and-overhead/] | ≈{nbsp}1.5 minutes -| 70--100{nbsp}ns | syscall trap, bare | `getpid()` on a mitigation-free host footnoteref:[gms,Georg Sauthoff, "On the Costs of Syscalls", 30 Aug 2021. Median of 100 repetitions across a heterogeneous set of hosts; the conclusion is that user-kernel mode switches cost on the order of a few hundred nanoseconds on all hosts, with the fastest host switching modes in under 100 ns, and that the higher figures are explained by CPU-bug mitigations being enabled by default -- https://gms.tf/on-the-costs-of-syscalls.html] | 4--6 minutes -| 200--600{nbsp}ns | syscall trap, mitigated | ``a few hundred nanoseconds'' on typical hosts with CPU-bug mitigations enabled footnoteref:[gms] | 11--33 minutes -| ~1{nbsp}µs | syscall doing real work | the round figure we use for the diagram | *≈{nbsp}1 hour* -| ~2{nbsp}µs | context switch | scheduler plus cold cache on the new task | ≈{nbsp}2 hours +| ~25{nbsp}ns | syscall via vDSO | not a real trap; almost as cheap as a function call <> | ≈{nbsp}1.5 minutes +| 70...100{nbsp}ns | syscall trap, bare | `getpid()` on a mitigation-free host <> | 4--6 minutes +| 200...600{nbsp}ns | syscall trap, mitigated | ``a few hundred nanoseconds'' on typical hosts with CPU-bug mitigations enabled <> | 11--33 minutes +| ~1{nbsp}µs | syscall doing real work | the round figure we use for the diagram | *≈{nbsp}1 hour* +| ~2{nbsp}µs | context switch | scheduler plus cold cache on the new task | ≈{nbsp}2 hours 4+^s| Microseconds -- the operating system -| 10--30{nbsp}µs | thread wake-up | IRQ handling, `try_to_wake_up()`, scheduling onto a CPU footnoteref:[lkmlwake,Michal Schmidt, reply in the thread "nanosleep() accuracy", linux-kernel mailing list, 17 Aug 2007: the point being that a process always carries non-zero wake-up latency covering interrupt processing, waking and scheduling, and that 10-30 us is not unreasonable -- https://lkml.iu.edu/hypermail/linux/kernel/0708.2/0423.html] | 9{nbsp}hours -- 1{nbsp}day -| ~31--57{nbsp}µs | scheduling latency, avg | `cyclictest` at `SCHED_FIFO` 95 with memory locked footnoteref:[rhel,Sample cyclictest output and interpretation in: Finding Realtime Linux Kernel Latencies, Clark Williams (Red Hat), section "Using cyclictest" -- https://people.redhat.com/williams/latency-howto/rt-latency-howto.txt] | 1--2 days -| ~59--90{nbsp}µs | scheduling latency, max | same run; the tail is what hurts footnoteref:[rhel] | 2--3.5 days -| *50{nbsp}µs* | *OS timer slack, default* | the kernel may delay a timed wake-up by this much, deliberately, to coalesce wakeups footnoteref:[slack,Primary documentation: `man 7 time`, section "High-resolution timers", and `man 2 prctl`, description of PR_SET_TIMERSLACK. The 50 us default for non-realtime threads is stated explicitly in: Metronome -- adaptive and precise intermittent packet retrieval in DPDK, section II (discussion of nanosleep limits), arXiv:2103.13263 -- https://arxiv.org/pdf/2103.13263 ; a practical encounter with the same default, including the /proc interface, is documented in OpenJDK issue JDK-8307766 -- https://bugs.openjdk.org/browse/JDK-8307766] | *≈{nbsp}2 days* -| 1{nbsp}ns | OS timer slack, minimum | via `prctl(PR_SET_TIMERSLACK)` or `/proc/PID/timerslack_ns` footnoteref:[slack] | ≈{nbsp}3 seconds -| ~9{nbsp}µs | SSD read, best case | Optane / Z-NAND class at QD1 footnoteref:[anand,AnandTech, review of the Samsung 983 ZET (Z-NAND), page 3, "QD1 Random Read Performance" -- explains why queue depth 1 exposes best-case access latency, and reports an average of just under 9 us for an Optane drive against 16 us for the Z-NAND based 983 ZET -- https://at-web1.www.anandtech.com/show/13951/the-samsung-983-zet-znand-ssd-review/3] | ≈{nbsp}8 hours -| 20--70{nbsp}µs | NVMe SSD 4K random read | typical contemporary drive footnoteref:[simply,Summary of the storage latency hierarchy: a modern NVMe SSD serves a random 4K read in roughly 20-70 us, a SATA SSD in 100-200 us, and a spinning disk in 5-10 ms. simplyblock glossary entry "NVMe Latency" -- https://simplyblock.io/glossary/nvme-latency/] | 1--3 days -| 50--100{nbsp}µs | NVMe SSD, interrupt path | what you see without polling | *2--4 days* -| 100--200{nbsp}µs | SATA SSD 4K random read | the interface costs, not just the medium footnoteref:[simply] | 4--8 days -| 100{nbsp}µs -- 1{nbsp}ms | deep C-state exit | wake from C6 and below; worse when many cores wake at once | 4{nbsp}days -- 5{nbsp}weeks +| 10...30{nbsp}µs | thread wake-up | IRQ handling, `try_to_wake_up()`, scheduling onto a CPU <> | 9{nbsp}hours -- 1{nbsp}day +| ~31...57{nbsp}µs | scheduling latency, avg | `cyclictest` at `SCHED_FIFO` 95 with memory locked <> | 1--2 days +| ~59...90{nbsp}µs | scheduling latency, max | same run; the tail is what hurts <> | 2--3.5 days +| *50{nbsp}µs* | *OS timer slack, default* | the kernel may delay a timed wake-up by this much, deliberately, to coalesce wakeups <> | *≈{nbsp}2 days* +| 1{nbsp}ns | OS timer slack, minimum | via `prctl(PR_SET_TIMERSLACK)` or `/proc/PID/timerslack_ns` <> | ≈{nbsp}3 seconds +| ~10{nbsp}µs | SSD read, best case | Optane-class (3D XPoint) at QD1 <> | ≈{nbsp}8 hours +| 20...70{nbsp}µs | NVMe SSD 4K random read | typical contemporary drive <> | 1--3 days +| 50...100{nbsp}µs | NVMe SSD, interrupt path | what you see without polling | *2--4 days* +| 100...200{nbsp}µs| SATA SSD 4K random read | the interface costs, not just the medium <> | 4--8 days +| 100{nbsp}µs ... 1{nbsp}ms | deep C-state exit | wake from C6 and below; worse when many cores wake at once | 4{nbsp}days -- 5{nbsp}weeks 4+^s| Milliseconds -- where machine and human meet -| 5--10{nbsp}ms | rotating disk, seek + rotation | 7200{nbsp}rpm; the number that has not improved in decades footnoteref:[simply] | *6{nbsp}months -- 1{nbsp}year* -| ~10{nbsp}ms | barely noticeable latency | the threshold below which a human stops perceiving delay | ≈{nbsp}1 year -| 20{nbsp}ms | one frame @{nbsp}50{nbsp}fps | | ≈{nbsp}2 years -| 40{nbsp}ms | one frame @{nbsp}25{nbsp}fps | our actual deadline | *≈{nbsp}4 years* -| ~100{nbsp}ms | UI starts to feel sluggish | | ≈{nbsp}11 years +| 5...10{nbsp}ms | rotating disk, seek + rotation | 7200{nbsp}rpm; the number that has not improved in decades <> | *6{nbsp}months -- 1{nbsp}year* +| ~10{nbsp}ms | barely noticeable latency | the threshold below which a human stops perceiving delay | ≈{nbsp}1 year +| 20{nbsp}ms | one frame @{nbsp}50{nbsp}fps | | ≈{nbsp}2 years +| 40{nbsp}ms | one frame @{nbsp}25{nbsp}fps | our actual deadline | *≈{nbsp}4 years* +| ~100{nbsp}ms | UI starts to feel sluggish | | ≈{nbsp}11 years 4+^s| Seconds and beyond -- the human | ~1{nbsp}s | ``the software takes time to respond'' | acceptable, but noticed | *≈{nbsp}1 century* -| ~3{nbsp}min | ``time for another cup of coffee'' | | ≈{nbsp}19 000 years +| ~3{nbsp}min | ``time for another cup of coffee'' | | ≈{nbsp}19 000 years | ~15{nbsp}min | ``can't wait, I'll do something else'' | ≙{nbsp}1000{nbsp}s, which conveniently closes the fourth decade block | ≈{nbsp}95 000 years |========================================================================= @@ -112,24 +116,24 @@ become visible to another core at the moment the instruction retires: it first lands in that core's store buffer. Cache coherence guarantees that the cores eventually agree on the content of a cache line, but it says nothing about _when_, nor about the order in which several independent writes become visible. -This is what memory fences are for -- they constrain the order, and thereby -allow one core to reason about what an adjacent core has already done. The cost -shows up in the table twice: an uncontended fence is roughly the price of an -L3 hit, whereas a cache line actually bouncing between two cores costs about as -much as going to DRAM. It is worth internalising that the expensive part is -usually not the synchronisation primitive itself, but the coherence traffic it -provokes. +This is what memory fences (memory barriers) are for -- they constrain the order, +and thereby allow one core to reason about what an adjacent core has already done. +The cost shows up in the table twice: an uncontended fence is roughly the price of +an L3 hit, whereas a cache line actually bouncing between two cores costs about as +much as going to DRAM. It should be noted that the expensive part is usually _not_ +the synchronisation primitive itself, but the cache coherence traffic it provokes. -x86 implements a comparatively strong model, _Total Store Order_ (TSO): of the -four possible reorderings, only StoreLoad is permitted. Acquire and release -semantics are therefore free on x86 -- they compile to plain `MOV` -- and only -sequential consistency requires an actual instruction. ARM and POWER are -weakly ordered and consequently offer a more differentiated set of barriers: -ARM distinguishes `DMB` from the heavier `DSB`, and provides graded load -acquire flavours such as `LDAR` and `LDAPR`; POWER separates the heavyweight -`sync` from the cheaper `lwsync`. The trap for the unwary is that C++ -`memory_order` distinctions which cost nothing on x86 have a real price on -ARM -- and, worse, that under-synchronised code can appear perfectly correct +And as far as memory synchronisation is concerned, there is another twist that +has already led quite some programmers astray. Intel's x86 implements a comparatively +strong coherency model, _Total Store Order_ (TSO): of the four possible reorderings, +only StoreLoad is permitted. Acquire and release semantics are therefore free on x86 -- +they compile to plain `MOV` -- and only sequential consistency requires an actual +instruction. Quite to the contrary, ARM and POWER are weakly ordered and consequently +offer a more differentiated set of barriers: ARM distinguishes `DMB` from the heavier +`DSB`, and provides graded load acquire flavours such as `LDAR` and `LDAPR`; POWER +separates the heavyweight `sync` from the cheaper `lwsync`. The trap for the unwary +is that C++ `memory_order` distinctions which cost nothing on x86 have a real price +on ARM -- and, worse, that under-synchronised code can appear perfectly correct on x86 and fail on ARM. @@ -145,17 +149,19 @@ attendant TLB consequences) and one or more branch-prediction barriers -- retpolines, IBPB or IBRS depending on hardware and configuration. On return, the whole dance runs in reverse. -The direct cost is measurable: a minimal syscall such as `getpid()` amounts to -little more than two mode switches, and lands in the region of a hundred +The direct cost is measurable: a minimal syscall such as `getpid()` amounts +to little more than two mode switches, and lands in the region of a hundred nanoseconds on a fast host without mitigations, rising to several hundred once -mitigations are enabled.footnoteref:[gms] The indirect cost is harder to see +mitigations are enabled <>. The indirect cost is harder to see and frequently larger. Kernel entry code and the syscall handler evict user-space data from L1 and L2, and disturb the branch predictors; the application resumes with a cold working set and pays for it in misses that no -syscall benchmark will attribute to the syscall. This is precisely why the -interesting question is rarely ``how expensive is one syscall'' but rather -``how many syscalls does this design require per unit of work'' -- which is -the question `io_uring` was built to answer. +syscall benchmark will attribute to the syscall. A simple `read()` that happens +to hit data already in the kernel cache can incur costs of around 1{nbsp}µs. + +The interesting question thus is is rarely ``how expensive is one syscall'' but +rather ``how many syscalls does this design require per unit of work'' -- +which is the question `io_uring` was built to answer. Operating System Slack @@ -165,100 +171,244 @@ A thread that asks to sleep for a specific duration will not, in general, wake at that instant. Part of this is unavoidable: an interrupt must be handled, the sleeping task must be moved to the run queue, and the scheduler must actually place it on a CPU, which together account for something in the range of -10--30{nbsp}µs.footnoteref:[lkmlwake] +10--30{nbsp}µs <>. The larger part, however, is deliberate. Linux maintains a per-thread _timer slack_ value, and the kernel is free to delay a timed wake-up by up to that amount in order to coalesce several wake-ups into one and thereby let the CPU stay in a low-power state longer. For every thread not running under a -real-time scheduling policy, the default is 50{nbsp}µs.footnoteref:[slack] +real-time scheduling policy, the default is 50{nbsp}µs <>. This is not imprecision; it is a power-versus-precision trade that the kernel makes on the application's behalf, and it can be adjusted -- down to 1{nbsp}ns -- via `prctl(PR_SET_TIMERSLACK)` or the per-process `/proc/PID/timerslack_ns` interface. -The distinction worth carrying away is between _resolution_ and _accuracy_. +The key distinction is therefore that between _resolution_ and _accuracy_. Since Linux 2.6.21 the high-resolution timer subsystem reports nanosecond resolution, and `clock_getres()` will happily confirm it. The accuracy with -which a `nanosleep()` actually returns is a different quantity altogether, two +which a `nanosleep()` actually returns is a different story altogether, two to three orders of magnitude coarser, and dominated by slack, wake-up latency and -- if the core had gone idle -- C-state exit time. Any design that assumes it can steer a worker thread with microsecond-scale sleeps should be checked against these figures first. -// ---------------------------------------------------------------------- -// Footnote definitions are attached at first use above via footnoteref. -// Repeated references reuse the id; see AsciiDoc footnoteref macro. -// ---------------------------------------------------------------------- - Sources ------- -The footnotes above carry the precise locations. Collected here as a reading -list, roughly in order of usefulness for further work. +The in-text markers such as <> link down to the full citation here. +Collected as a reading list, roughly in order of usefulness for further work. -- Travis Downs, *A Concurrency Cost Hierarchy* -- - https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html + - The best single entry point on why synchronisation costs what it costs. +[[downs]] +-- +downs:: + Travis Downs (6 July 2020) -- + https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html[ + A Concurrency Cost Hierarchy + ] +//Archive https://web.archive.org/web/20260801071948/https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html ++ +The best single entry point on why synchronisation costs what it costs. ++ +See the discussion of "vanilla" versus "non-vanilla" instructions: on x86 +plain accesses and non-atomic read-modify-writes are cheap while +LOCK-prefixed RMWs are an order of magnitude slower, with `MFENCE` +comparable to a locked instruction. +-- -- LKML thread, *[PATCH 3/4] x86,asm: Re-work smp_store_mb()* (Nov 2015) -- - https://lkml.iu.edu/hypermail/linux/kernel/1511.0/01052.html + - Kernel developers measuring `MFENCE` against `XCHG` and `lock addq`. +[[lkml]] +-- +lkml:: + LKML thread (Nov 2015) -- + https://lkml.iu.edu/hypermail/linux/kernel/1511.0/01052.html[ + [PATCH 3/4] x86,asm: Re-work smp_store_mb() + ] +//Archive https://web.archive.org/web/20240818210644/https://lkml.iu.edu/hypermail/linux/kernel/1511.0/01052.html ++ +Kernel developers measuring `MFENCE` against `XCHG` and `lock addq`: +Davidlohr Bueso measures XCHG against MFENCE on Ivy Bridge; Linus Torvalds +reports that on Haswell `lock addq` costs pretty much exactly half of +MFENCE, and explains that locked operations were the ones CPU designers had +reason to optimise. +-- -- Georg Sauthoff, *On the Costs of Syscalls* (Aug 2021) -- - https://gms.tf/on-the-costs-of-syscalls.html + - Careful cross-host microbenchmark of the mode-switch cost. +[[gms]] +-- +gms:: + Georg Sauthoff (30 Aug 2021) -- + https://gms.tf/on-the-costs-of-syscalls.html[ + On the Costs of Syscalls + ] +//Archive https://web.archive.org/web/20260725162647/https://gms.tf/on-the-costs-of-syscalls.html ++ +Careful cross-host microbenchmark of the mode-switch cost. ++ +Median of 100 repetitions across a heterogeneous set of hosts; the +conclusion is that user-kernel mode switches cost on the order of a few +hundred nanoseconds on all hosts, with the fastest host switching modes in +under 100 ns, and that the higher figures are explained by CPU-bug +mitigations being enabled by default. +-- -- *Measurements of system call performance and overhead* (Jan 2017) -- - https://arkanis.de/weblog/2017-01-05-measurements-of-system-call-performance-and-overhead/ + - Function call versus `syscall` instruction versus vDSO. +[[arkanis]] +-- +arkanis:: + Arkanis Development (5 Jan 2017) -- + https://arkanis.de/weblog/2017-01-05-measurements-of-system-call-performance-and-overhead/[ + Measurements of system call performance and overhead + ] +//Archive https://web.archive.org/web/20260311003705/https://arkanis.de/weblog/2017-01-05-measurements-of-system-call-performance-and-overhead/ ++ +Compares a plain function call against `getpid()` issued via the syscall +instruction and via the vDSO. +-- -- Brendan Gregg, *KPTI/KAISER Meltdown Initial Performance Regressions* -- - https://www.brendangregg.com/blog/2018-02-09/kpti-kaiser-meltdown-performance.html + - Relates mitigation overhead to syscall rate, context switch rate and working - set size -- the systemic view rather than the per-call view. +[[gregg]] +-- +gregg:: + Brendan Gregg (9 Feb 2018) -- + https://www.brendangregg.com/blog/2018-02-09/kpti-kaiser-meltdown-performance.html[ + KPTI/KAISER Meltdown Initial Performance Regressions + ] +//Archive https://web.archive.org/web/20260603211334/https://www.brendangregg.com/blog/2018-02-09/kpti-kaiser-meltdown-performance.html ++ +Relates mitigation overhead to syscall rate, context switch rate and working +set size -- the systemic view rather than the per-call view. +-- -- Clark Williams (Red Hat), *Finding Realtime Linux Kernel Latencies* -- - https://people.redhat.com/williams/latency-howto/rt-latency-howto.txt + - Walks through a `cyclictest` run and how to read its output. +[[lkmlwake]] +-- +lkmlwake:: + Michal Schmidt (17 Aug 2007) -- + https://lkml.iu.edu/hypermail/linux/kernel/0708.2/0423.html[ + Reply in the LKML thread "nanosleep() accuracy" + ] +//Archive https://web.archive.org/web/20260824155617/https://lkml.iu.edu/hypermail/linux/kernel/0708.2/0423.html ++ +The point being that a process always carries non-zero wake-up latency +covering interrupt processing, waking and scheduling, and that +10--30{nbsp}µs is not unreasonable. +-- -- `man 7 time` and `man 2 prctl` (PR_SET_TIMERSLACK) + - Primary documentation for timer slack. The 50{nbsp}µs default is stated in - https://arxiv.org/pdf/2103.13263 (Metronome, DPDK packet retrieval) and - encountered in practice in https://bugs.openjdk.org/browse/JDK-8307766 +[[rhel]] +-- +rhel:: + Clark Williams (Red Hat) -- + https://people.redhat.com/williams/latency-howto/rt-latency-howto.txt[ + Finding Realtime Linux Kernel Latencies + ] +//Archive https://web.archive.org/web/20250620155700/https://people.redhat.com/williams/latency-howto/rt-latency-howto.txt ++ +Walks through a `cyclictest` run and how to read its output: sample +`cyclictest` output and interpretation, section "Using cyclictest". +-- -- *Performance Characterizations ... Samsung CXL Memory Module Hybrid - Prototype*, arXiv:2503.22017 -- https://arxiv.org/pdf/2503.22017 + - Table III contains a well-documented measurement of plain local DDR5. +[[slack]] +-- +slack:: + Primary documentation -- + `man 7 time`, section "High-resolution timers", and `man 2 prctl`, + description of `PR_SET_TIMERSLACK` ++ +The 50{nbsp}µs default for non-realtime threads is stated explicitly in +https://arxiv.org/pdf/2103.13263[ + Metronome -- adaptive and precise intermittent packet retrieval in DPDK +] +//Archive https://web.archive.org/web/20260725162634/https://arxiv.org/pdf/2103.13263 +(section II, discussion of nanosleep limits), and a practical encounter +with the same default, including the `/proc` interface, is documented in +https://bugs.openjdk.org/browse/JDK-8307766[ + OpenJDK issue JDK-8307766 +]. +//Archive https://web.archive.org/web/20251130011408/https://bugs.openjdk.org/browse/JDK-8307766 +-- -- *Single 32-bit Sub-Channel DDR5 DIMMs*, arXiv:2605.08725 -- - https://arxiv.org/pdf/2605.08725 + - Useful for the point that DDR5 did not improve absolute latency over DDR4. +[[cxl]] +-- +cxl:: + arXiv:2503.22017 -- + https://arxiv.org/pdf/2503.22017[ + Performance Characterizations and Usage Guidelines of Samsung CXL Memory + Module Hybrid Prototype + ] +//Archive https://web.archive.org/web/20260811083139/https://arxiv.org/pdf/2503.22017 ++ +Table III contains a well-documented measurement of plain local DDR5: +median latency 122.9{nbsp}ns under Intel MLC pointer chasing versus +13.1{nbsp}ns for a plain load; methodology (rdtsc, 10 000 repetitions, +median) described in section III-C. +-- -- simplyblock glossary, *NVMe Latency* -- - https://simplyblock.io/glossary/nvme-latency/ + - Compact summary of the NVMe / SATA SSD / rotating disk hierarchy. +[[ddr5b]] +-- +ddr5b:: + arXiv:2605.08725 -- + https://arxiv.org/pdf/2605.08725[ + Single 32-bit Sub-Channel DDR5 DIMMs + ] +//Archive https://web.archive.org/web/20260607161507/https://arxiv.org/pdf/2605.08725 ++ +Useful for the point that DDR5 did not improve absolute latency over DDR4. ++ +The relevant passage is in section "Latency Implications", noting that both +DDR4 and DDR5 sit at approximately 14{nbsp}ns first-access latency at their +respective standard speeds. +-- -- AnandTech, *Samsung 983 ZET (Z-NAND) review*, page 3 -- - https://at-web1.www.anandtech.com/show/13951/the-samsung-983-zet-znand-ssd-review/3 + - Explains why queue depth 1 exposes best-case access latency. +[[simply]] +-- +simply:: + simplyblock glossary -- + https://simplyblock.io/glossary/nvme-latency/[ + NVMe Latency + ] +//Archive https://web.archive.org/web/20260418070043/https://simplyblock.io/glossary/nvme-latency/ ++ +Compact summary of the NVMe / SATA SSD / rotating disk hierarchy: a modern +NVMe SSD serves a random 4K read in roughly 20--70{nbsp}µs, a SATA SSD in +100--200{nbsp}µs, and a spinning disk in 5--10{nbsp}ms. +-- + +[[intel]] +-- +intel:: + Intel -- + https://cdrdv2-public.intel.com/840898/Windows_Intel_Optane_SSD_DC_P4800X_Eval-Guide_336586.pdf[ + Optane SSD DC P4800X Evaluation Guide + ] +//Archive https://web.archive.org/web/20260824160928/https://cdrdv2-public.intel.com/840898/Windows_Intel_Optane_SSD_DC_P4800X_Eval-Guide_336586.pdf ++ +In Chapter 1.1, Page 5: ``at queue depth of one, 99 out of 100 read operations ... +complete in single digit microseconds.'' ++ +Corroborated independently by +https://www.tomshardware.com/reviews/intel-optane-3d-xpoint-p4800x,5030.html[ + Tom's Hardware's 2017 preview, +] +//Archive https://web.archive.org/web/20260313082242/https://www.tomshardware.com/reviews/intel-optane-3d-xpoint-p4800x,5030.html + which reports 10 µs latency for the same drive. ++ +Note: Optane was discontinued by Intel in 2022; the figure is retained as the +best-documented physical floor for this latency class. +-- A Note on Microbenchmarks ------------------------- -One caveat deserves to be stated separately, because it explains why published -figures for synchronisation primitives vary so wildly. A hot microbenchmark -keeps every thread runnable and every core out of idle by construction. Under -those conditions an uncontended atomic operation or a mutex acquisition costs -single-digit to low-three-digit nanoseconds, and a benchmark will faithfully -report that. What such a benchmark structurally cannot observe is the cost of -_blocking_ -- of a thread actually parking and later being woken -- because -pressing the benchmark is precisely what prevents that from happening. +One caveat deserves a separate mention, as it explains why the published +figures on synchronisation primitives differ so drastically from one another. +A hot microbenchmark keeps every thread runnable and every core out of idle by +its very construction. Under those conditions an uncontended atomic operation or +a Mutex acquisition costs single-digit to low-three-digit nanoseconds, and a +benchmark will faithfully report that. What such a benchmark _structurally_ +cannot observe is the cost of _blocking_ -- of a thread actually parking and +later being woken -- because pressing the benchmark is precisely what +prevents that from happening. The instruction-level cost of a synchronisation primitive is what a hot microbenchmark measures. Its systemic cost -- coherence traffic under -contention, and blocking and wake-up latency when it is not contended -- is -what production tail latency measures. The two can differ by two to four orders -of magnitude, and the table above spans both regimes for exactly that reason. +contention, and blocking and wake-up latency when it is not contended -- +is what production tail latency measures. The two values can differ by +two to four orders of magnitude, and it is for this very reason that +the table above covers both regimes.