diff --git a/FrOSCon26/FrOSCon.26.mm b/FrOSCon26/FrOSCon.26.mm index 307b75e..a98f14a 100644 --- a/FrOSCon26/FrOSCon.26.mm +++ b/FrOSCon26/FrOSCon.26.mm @@ -49,9 +49,7 @@ - - - +

hab dafür zwar keinen Beleg (jenseits von ein paar Videos), aber ich nehme hier den Namen »Alexandrescu« in Gebrauch, um etwas — für sich selbst plausibles — auch mit Autorität sagen zu können @@ -107,16 +105,13 @@ - - - +

die allermeisten Entwickler heutzutage gehen niemals über diese Schwelle, und verstecken sich deshalb hinter Frameworks

- - + @@ -160,9 +155,7 @@ - - - +

raw video with very high quality parameters; elaborate processing stack @@ -205,9 +198,7 @@ - - - +

I/O is no longer slow — and that's the problem @@ -370,7 +361,101 @@ - + + + + + + + + + + + + + + +

+ habe jetzt einen ziemlich langen Thread mit Recherche / Hintergrundwissen für mich; in dem Rahmen sind die diversen Zeitangaben bereits alle aufgetaucht +

+ + + + + + + + + + + + + + + + + + + + + + + + +

+ Interessante Idee ... Claude greift also mein Framing vom ersten Prompt wieder auf ... erscheint mir sehr plausibel +

+ + + + + + + + + +

+ Das hat Claude interessanterweise erst gesehen, nachdem ich an den längsten Zeitspannen etwas getweakt hatte. Mir war aufgefallen 1000sec ≈ 15min ⟹ damit kommen wir mit einer Sekunden-Skala als 4.Zeile gut aus +

+ + + + + + + +

+ ...das ergab sich aus der Konversation: im Grunde reichen die ganzen Maschinen-relevanten Zeiten hoch bis 10ms / etwa an dem Punkt beginnen Timings, die vom Menschen wahrnehmbar sind. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -578,9 +663,7 @@ - - - +

Es war von Anfang an klar, daß wir speziell dieses Thema selber unter Kontrolle bringen müssen. @@ -621,16 +704,13 @@ - - - +

Damit meine ich, ob der Ansatz mit symbolisch repräsentierten externen Fähigkeiten überhaupt aufgeht, und ob es tatsächlich gelingt, ein Werkzeug-Paradigma auf der Basis von Placements zu schaffen. Und, last but not least, ob ich Leute finden, die mich unterstützen und sich diese Vision aneignen

- - + @@ -638,16 +718,13 @@ - - - +

der Talk kommt fachlich daher — ist aber eigentlich ein Narrativ  und allgemeines Lehrstück

- - + @@ -658,9 +735,7 @@ - - - +

ich habe zur Vorbereitung meine anfänglichen Ideen in einer Konversation mit Claude validiert.... @@ -1255,11 +1330,35 @@ + + + + +

+ Recherche / Background - Konversation mit Claude +

+ + + + + + + + + + + + + + + + + + + - - - +

Ich verfüge über vertieftes Wissen und einiges an Erfahrung in diesen Bereichen, sollte aber meine Kenntnisse überprüfen und festigen, um auch auf Fragen eingehen zu können. Im Besonderen zu Kernel-nahen Themen weiß ich ehr wenig. @@ -1299,9 +1398,7 @@ - - - +

Doug Lea's "JSR-133 Cookbook for Compiler Writers" @@ -1312,9 +1409,7 @@ - - - +

Gilt als Quelle für Doug Lea @@ -1332,9 +1427,7 @@ - - - +

loads before must complete before loads after @@ -1344,9 +1437,7 @@ - - - +

stores before must become visible before stores after @@ -1356,9 +1447,7 @@ - - - +

loads before must complete before stores after @@ -1368,9 +1457,7 @@ - - - +

stores before must become visible before loads after @@ -1393,9 +1480,7 @@ - - - + "barrier/fence is hardware vocabulary for a specific machine instruction; memory_order is C++'s hardware-independent vocabulary for a guarantee; the compiler's job is to translate one into the other — and 'fence' in the standard specifically means the standalone @@ -1411,9 +1496,7 @@ - - - +

Douglas C. Schmidt
Pattern-Oriented Software Architecture Volume 2:
Patterns for Concurrent and Networked Objects @@ -1453,9 +1536,7 @@ - - - +

ABER: gemeinsamer Speicherzugriff kostet immer @@ -1470,9 +1551,7 @@ - - - +

ABER: der Compiler kann auch umordnen @@ -1484,9 +1563,7 @@ - - - +

Bezug zu liburing @@ -1512,9 +1589,7 @@ - - - +

weil die Benchmark-Thread(s) warm @@ -1544,9 +1619,7 @@ - - - + "A synchronization primitive's instruction-level cost is what a hot microbenchmark measures; its systemic cost — coherence traffic under contention, and blocking/wake latency when it isn't contended — is what your production tail latency measures, and the two can differ by two to four orders of magnitude." @@ -1556,9 +1629,7 @@ - - - +

Verbindung zum Proactor @@ -1582,9 +1653,7 @@ - - - +

server-vacation model @@ -1596,9 +1665,7 @@ - - - +

man kann einen Kernel-Thread konfiguieren, der die Submission-Queue pollt. Dort konfiguriert man eine Einschlafe-Periode. Wenn der Poller schläft, setzt er ein Bit und muß von der Applikation explizit wieder reaktiviert werden. @@ -1622,9 +1689,7 @@ - - - +

Windows has WaitOnAddress/WakeByAddressSingle/WakeByAddressAll @@ -1634,9 +1699,7 @@ - - - +

FreeBSD has _umtx_op @@ -1654,9 +1717,7 @@ - - - +

weckt n threads auf, @@ -1671,9 +1732,7 @@ - - - +

beachte: der uncontended-case bleibt komplett im userspace @@ -1729,9 +1788,7 @@ - - - +

man kann nur aufrufen, was der Kernel an Einsprungspunkten bietet; eigentlicher Kernel-Code ist nicht vom Userpace zugänglich @@ -1744,9 +1801,7 @@ - - - +

...um Ausspionieren des Kernel via speculative execution zu verhindern @@ -1809,9 +1864,7 @@ - - - +

man hat die Herausforderung durch die SSDs zeitnah erkannt und mit tiefgreifenden internen Umbauten begonnen @@ -1824,9 +1877,7 @@ - - - +

...was nicht realisierbar ist, und deshalb ist der Ansatz von libaio stecken geblieben und bleibt auf einige Randfälle begrenzt (und noch schlimmer, kann jederzeit auf blocking waits zurückfallen) @@ -1849,9 +1900,7 @@ - - - +

FreeBSD: linuxulator wird erzwingen, daß man io_uring einfach nachbaut @@ -1874,9 +1923,7 @@ - - - +

Stichwort: seccomp @@ -1891,9 +1938,7 @@ - - - +

u.a. auf Android @@ -1908,9 +1953,7 @@ - - - +

Grund ist: eBPF erfordert höhere Privilegien @@ -1920,9 +1963,7 @@ - - - +

Where it stands now — and a lovely structural irony @@ -1964,16 +2005,13 @@ - - - +

Stichwort: Channel Programs

- - + @@ -1981,14 +2019,11 @@ - - - + a completion routine the system delivers by interrupting your thread. Results land in an I/O Status Block; you synchronize with event flags. Submit, continue, get called back. - - + @@ -2027,16 +2062,13 @@ - - - +

d.h. das Problem mit den coloured functions wird nicht gelöst, aber der Schmerz wird reduziert

- - + @@ -2067,9 +2099,7 @@ - - - +

nachdem der Artikel so stark eingeschlagen hat, @@ -2078,8 +2108,7 @@ hat er selber, unter einem Pseudonym, eine Replik geschrieben

- - + @@ -4005,9 +4034,7 @@ - - - +

ffplay -f rawvideo -pixel_format rgb24 -video_size 1200x675 -framerate 50 demovideo-1200.raw @@ -4018,9 +4045,7 @@ - - - +

vlc --demux rawvideo --rawvid-width=1200 --rawvid-height=675 --rawvid-fps=50 --rawvid-chroma=RV24 --vout=opengl demovideo-1200.raw @@ -4350,6 +4375,61 @@ + + + + + + + + + + + + + + + +

+ alle meine eigenen Erfahrungen und Richtwerte konnten vollumfänglich bestätigt werden; manche Meßwerte kann ich nun besser einordnen, z.B. durch die Unterscheidung in hott-pressed und systemic-cost. Erheblich dazugelernt habe ich bezüglich Futexes +

+ + + + + + + + + + + + + + +

+ wenn ein Clock-cycle eine Sekunde dauern würde.... +

+ + + + + + + + + + + + + + + + + + + + diff --git a/FrOSCon26/drawings/gen-time-scales-diagram.py b/FrOSCon26/drawings/gen-time-scales-diagram.py new file mode 100644 index 0000000..00cf4cc --- /dev/null +++ b/FrOSCon26/drawings/gen-time-scales-diagram.py @@ -0,0 +1,245 @@ +#!/usr/bin/env python3 +"""Generate the FrOSCon io_uring talk 'time scales' wrapped-log-scale diagram. + +Page: 254 x 142.9 mm (== LibreOffice Impress 16:9 slide, 25.4 x 14.29 cm) +User unit == 1 mm. + +Structure: four bands, each spanning exactly three decades (1 -> 1000 of its +unit), all at identical horizontal scale. Each band is a separate Inkscape +layer so the whole thing can be exported step by step (Export > Page) and +stacked as transparent overlays in Impress. + +@note Code generated 8/2026 by Claude -- prompted by Ichthyostega +""" +import math + +W, H = 254.0, 142.9 + +X0, X1 = 34.0, 240.0 # axis: value 1 .. value 1000 +DEC = (X1 - X0) / 3.0 # mm per decade +AXIS_END = 243.5 # slight overshoot + chevron + +BANDS_Y = [40.0, 75.0, 102.0, 120.0] + +# ---- colours ------------------------------------------------------------- +C_AXIS = "#555555" +C_MINOR = "#b3b3b3" +C_MACH = "#14507d" +C_HUMAN = "#b4530a" +C_VAL_M = "#4a7ba6" +C_VAL_H = "#c98342" +C_MUTED = "#8a8a8a" +C_TITLE = "#222222" + +# ---- type ---------------------------------------------------------------- +F_NAME, F_VAL, F_BAND, F_TITLE, F_NOTE, F_DISC = 3.8, 3.2, 5.5, 8.0, 3.6, 3.0 +FONT = "DejaVu Sans, Verdana, sans-serif" + +# tier offsets relative to band axis Y +UP1_NAME, UP1_VAL, UP1_STUB = -10.3, -6.5, -5.0 +UP2_NAME, UP2_VAL, UP2_STUB = -18.3, -14.5, -13.0 +LO1_NAME, LO1_VAL, LO1_STUB = 11.2, 7.5, 5.0 +BRACKET_Y = 4.0 + + +def X(v): + return X0 + DEC * math.log10(v) + + +def esc(s): + return (s.replace("&", "&").replace("<", "<").replace(">", ">")) + + +def text(x, y, s, size, fill, anchor="middle", weight="normal"): + return (f'{esc(s)}') + + +def line(x1, y1, x2, y2, stroke, w, dash=None): + d = f' stroke-dasharray="{dash}"' if dash else "" + return (f'') + + +# ========================================================================== +# entry data +# ========================================================================== +# tier: 'up1' | 'up2' | 'lo1' +# kind: 'machine' | 'human' +# span: (v_lo, v_hi) -> draw a bracket instead of a dot +BANDS = [ + dict(unit="1 ns", ay=BANDS_Y[0], layer="ns", entries=[ + dict(v=1, name="L1 cache", val="1 ns", tier="up1", kind="machine"), + dict(v=4, name="L2 cache", val="4 ns", tier="up1", kind="machine"), + dict(v=10, name="full memory barrier", val="10 ns", tier="up2", kind="machine"), + dict(v=15, name="L3 cache", val="15 ns", tier="up1", kind="machine"), + dict(v=100, name="DRAM access", val="100 ns", tier="up1", kind="machine"), + dict(span=(50, 100), name="contended cache line", val="50-100 ns", + tier="lo1", kind="machine"), + dict(offscale=True, name="1 clock cycle", val="\u2248 0.3 ns", + tier="lo1", kind="machine"), + ]), + dict(unit="1 \u00b5s", ay=BANDS_Y[1], layer="us", entries=[ + dict(v=1, name="system call", val="1 \u00b5s", tier="up1", kind="machine"), + dict(v=2, name="context switch", val="2 \u00b5s", tier="up2", kind="machine"), + dict(v=50, name="OS timer slack", val="50 \u00b5s", tier="up1", kind="machine"), + dict(span=(10, 30), name="thread wake-up", val="10-30 \u00b5s", + tier="lo1", kind="machine"), + dict(span=(50, 100), name="SSD read", val="50-100 \u00b5s", + tier="lo1", kind="machine"), + ]), + dict(unit="1 ms", ay=BANDS_Y[2], layer="ms", entries=[ + dict(v=10, name="disk seek", val="10 ms", tier="up1", kind="machine"), + dict(v=40, name="one frame @ 25 fps", val="40 ms", tier="up1", kind="machine"), + dict(v=10, name="barely noticeable", val="10 ms", tier="lo1", kind="human"), + dict(v=100, name="UI feels sluggish", val="100 ms", tier="lo1", kind="human"), + ]), + dict(unit="1 s", ay=BANDS_Y[3], layer="s", entries=[ + dict(v=1, name="\u201cit\u2019s thinking\u2026\u201d", val="1 s", + tier="lo1", kind="human"), + dict(v=180, name="another coffee", val="3 min", tier="lo1", kind="human"), + dict(v=900, name="go do something else", val="\u2248 15 min", + tier="lo1", kind="human", anchor="end", ax=250.0), + ]), +] + + +# ========================================================================== +# scaffold +# ========================================================================== +def band_axis(ay, unit): + o = [] + o.append(line(X0, ay, AXIS_END, ay, C_AXIS, 0.6)) + # continuation chevron + o.append(f'') + # ticks + for d in range(3): + base = 10 ** d + for m in range(1, 10): + v = base * m + x = X(v) + if m == 1: + o.append(line(x, ay - 3.0, x, ay + 3.0, C_AXIS, 0.6)) + else: + o.append(line(x, ay - 1.6, x, ay + 1.6, C_MINOR, 0.25)) + o.append(line(X(1000), ay - 3.0, X(1000), ay + 3.0, C_AXIS, 0.6)) + o.append(text(32.0, ay + 2.0, unit, F_BAND, C_MUTED, "end", "600")) + return o + + +def scaffold(): + o = [] + o.append(text(10.0, 15.0, "Time scales", F_TITLE, C_TITLE, "start", "600")) + o.append(text(244.0, 12.5, "each row spans three decades", F_NOTE, C_MUTED, "end")) + o.append(text(244.0, 17.0, "one row down = \u00d7 1000", F_NOTE, C_MUTED, "end")) + for b in BANDS: + o += band_axis(b["ay"], b["unit"]) + o.append(text(10.0, 137.0, + "Order of magnitude only \u2014 contemporary desktop / laptop x86. " + "Expect \u00b1 2\u20133\u00d7; the ratios are the point.", + F_DISC, C_MUTED, "start")) + return o + + +# ========================================================================== +# entries +# ========================================================================== +def render_entry(e, ay): + kind = e.get("kind", "machine") + cn = C_MACH if kind == "machine" else C_HUMAN + cv = C_VAL_M if kind == "machine" else C_VAL_H + tier = e["tier"] + anchor = e.get("anchor", "middle") + o = [] + + if tier == "up1": + ny, vy, sy = ay + UP1_NAME, ay + UP1_VAL, ay + UP1_STUB + elif tier == "up2": + ny, vy, sy = ay + UP2_NAME, ay + UP2_VAL, ay + UP2_STUB + else: + ny, vy, sy = ay + LO1_NAME, ay + LO1_VAL, ay + LO1_STUB + + if e.get("offscale"): + # dashed stub running off the left-hand end of the scale + y = ay + BRACKET_Y + 0.5 + o.append(line(X0, y, 22.5, y, cn, 0.5, dash="1.6,1.2")) + o.append(f'') + ax = 22.0 + o.append(text(ax, vy, e["val"], F_VAL, cv, "start")) + o.append(text(ax, ny, e["name"], F_NAME, cn, "start", "600")) + return o + + if "span" in e: + x1, x2 = X(e["span"][0]), X(e["span"][1]) + by = ay + BRACKET_Y + o.append(line(x1, by, x2, by, cn, 0.7)) + o.append(line(x1, by, x1, ay, cn, 0.7)) + o.append(line(x2, by, x2, ay, cn, 0.7)) + ax = e.get("ax", (x1 + x2) / 2.0) + else: + x = X(e["v"]) + o.append(f'') + stub_from = ay - 2.0 if tier.startswith("up") else ay + 2.0 + o.append(line(x, stub_from, x, sy, cn, 0.4)) + ax = e.get("ax", x) + + o.append(text(ax, vy, e["val"], F_VAL, cv, anchor)) + o.append(text(ax, ny, e["name"], F_NAME, cn, anchor, "600")) + return o + + +# ========================================================================== +# assemble +# ========================================================================== +def layer(lid, label, body, display="inline"): + return (f'\n ' + + "\n ".join(body) + "\n") + + +def build(): + layers = [] + layers.append(layer("layer-1-scaffold", "1 - scaffold", scaffold())) + + for i, b in enumerate(BANDS): + if b["layer"] == "ms": + mach = [x for e in b["entries"] if e["kind"] == "machine" + for x in render_entry(e, b["ay"])] + hum = [x for e in b["entries"] if e["kind"] == "human" + for x in render_entry(e, b["ay"])] + layers.append(layer("layer-4-ms-machine", "4 - ms band (machine)", mach)) + layers.append(layer("layer-5-ms-human", "5 - ms band (human)", hum)) + elif b["layer"] == "ns": + body = [x for e in b["entries"] for x in render_entry(e, b["ay"])] + layers.append(layer("layer-2-ns", "2 - ns band", body)) + elif b["layer"] == "us": + body = [x for e in b["entries"] for x in render_entry(e, b["ay"])] + layers.append(layer("layer-3-us", "3 - us band", body)) + else: + body = [x for e in b["entries"] for x in render_entry(e, b["ay"])] + layers.append(layer("layer-6-s", "6 - s band (human)", body)) + + dim = [f''] + layers.append(layer("layer-7-dim", "7 - dim ns+us (optional)", dim, + display="none")) + + return f''' + + + Time scales - ns to minutes, wrapped logarithmic +{chr(10).join(layers)} + +''' + + +if __name__ == "__main__": + open("/mnt/user-data/outputs/time-scales.svg", "w").write(build()) + print("written") diff --git a/FrOSCon26/drawings/time-scales.svg b/FrOSCon26/drawings/time-scales.svg new file mode 100644 index 0000000..6a68cfe --- /dev/null +++ b/FrOSCon26/drawings/time-scales.svg @@ -0,0 +1,230 @@ + + + + Time scales - ns to minutes, wrapped logarithmic + + Time scales + each row spans three decades + one row down = × 1000 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 1 ns + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 1 µs + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 1 ms + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 1 s + Order of magnitude only — contemporary desktop / laptop x86. Expect ± 2–3×; the ratios are the point. + + + + + 1 ns + L1 cache + + + 4 ns + L2 cache + + + 10 ns + full memory barrier + + + 15 ns + L3 cache + + + 100 ns + DRAM access + + + + 50-100 ns + contended cache line + + + ≈ 0.3 ns + 1 clock cycle + + + + + 1 µs + system call + + + 2 µs + context switch + + + 50 µs + OS timer slack + + + + 10-30 µs + thread wake-up + + + + 50-100 µs + SSD read + + + + + 10 ms + disk seek + + + 40 ms + one frame @ 25 fps + + + + + 10 ms + barely noticeable + + + 100 ms + UI feels sluggish + + + + + 1 s + “it’s thinking…” + + + 3 min + another coffee + + + ≈ 15 min + go do something else + + + diff --git a/FrOSCon26/research/TimeScales.txt b/FrOSCon26/research/TimeScales.txt new file mode 100644 index 0000000..366bc47 --- /dev/null +++ b/FrOSCon26/research/TimeScales.txt @@ -0,0 +1,264 @@ +Time Scales +=========== +:Author: Lumiera Project +:toc: + +Anyone reasoning about I/O, threading or real-time media processing eventually +needs a feeling for _how long things take_. The trouble is that the relevant +durations span more than twelve orders of magnitude -- from a single CPU clock +cycle up to the point where a human gives up waiting. Numbers on that scale +resist intuition, and so we tend to reason about ``fast'' and ``slow'' without +noticing that the two words can be separated by a factor of a million. + +The following table collects the figures that matter in this context. It is +meant as a rough orientation, not as a benchmark result. + +NOTE: All values are order-of-magnitude figures for contemporary desktop and +laptop x86 hardware. Expect a deviation of 2--3{nbsp}x depending on +microarchitecture, memory configuration, kernel version and load. The +_ratios_ are the point, not the digits. + +The last column rescales everything by a single constant factor, chosen so +that one CPU clock cycle (≙ 0.3{nbsp}ns at 3.3{nbsp}GHz) becomes one second +of human time. This is the most effective way we know to convey what a cache +miss or a disk seek actually costs from the machine's point of view. + + +Overview Table +-------------- + +[width="100%",cols="12%,26%,40%,22%",options="header"] +|========================================================================= +| Figure | Item | Remarks | Felt as (1{nbsp}cycle{nbsp}≙{nbsp}1{nbsp}s) + +4+^s| Nanoseconds -- inside the CPU + +| 0.3{nbsp}ns | one clock cycle | at 3.3{nbsp}GHz; the machine's own time quantum | *1 second* +| ~1{nbsp}ns | L1 cache hit | about 4 cycles, fully pipelined | ≈{nbsp}3 seconds +| ~1.5{nbsp}ns | `sfence` | store fence, ≈{nbsp}5 cycles; rarely needed on x86 | ≈{nbsp}5 seconds +| ~2.7{nbsp}ns | `lfence` | ≈{nbsp}9 cycles; no ordering semantics on x86 beyond blocking speculation | ≈{nbsp}9 seconds +| ~4{nbsp}ns | L2 cache hit | ≈{nbsp}12--15 cycles | ≈{nbsp}13 seconds +| ~5{nbsp}ns | `lock add $0,(%rsp)` | the actual `smp_mb()` on Linux; ≈{nbsp}half the cost of `mfence` footnoteref:[lkml,LKML thread "[PATCH 3/4] x86,asm: Re-work smp_store_mb()", Nov 2015. Davidlohr Bueso measures XCHG against MFENCE on Ivy Bridge; Linus Torvalds reports that on Haswell "lock addq" costs pretty much exactly half of MFENCE, and explains that locked operations were the ones CPU designers had reason to optimise -- https://lkml.iu.edu/hypermail/linux/kernel/1511.0/01052.html] | ≈{nbsp}17 seconds +| ~5{nbsp}ns | branch misprediction | pipeline refill | ≈{nbsp}17 seconds +| ~10{nbsp}ns | `mfence` / full barrier | ≈{nbsp}23--33 cycles; comparable to a `lock`-prefixed RMW footnoteref:[downs,Travis Downs, "A Concurrency Cost Hierarchy", 6 July 2020, see the discussion of "vanilla" versus "non-vanilla" instructions: on x86 plain accesses and non-atomic read-modify-writes are cheap while LOCK-prefixed RMWs are an order of magnitude slower, with MFENCE comparable to a locked instruction -- https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html] | ≈{nbsp}33 seconds +| ~15{nbsp}ns | L3 cache hit | shared, hence variable | ≈{nbsp}50 seconds +| ~13{nbsp}ns | DRAM, open row | a plain load that happens to hit an already-activated row footnoteref:[cxl,Measured values in Table III of: Performance Characterizations and Usage Guidelines of Samsung CXL Memory Module Hybrid Prototype, arXiv:2503.22017. Local DDR5 median latency 122.9 ns under Intel MLC pointer chasing versus 13.1 ns for a plain load; methodology (rdtsc, 10 000 repetitions, median) described in section III-C -- https://arxiv.org/pdf/2503.22017] | ≈{nbsp}43 seconds +| ~14{nbsp}ns | DRAM, first access | device-level figure; essentially unchanged from DDR4 to DDR5 footnote:[Measurement-oriented analysis of DDR5 sub-channel architecture; the relevant passage is in section "Latency Implications", noting that both DDR4 and DDR5 sit at approximately 14 ns first-access latency at their respective standard speeds. arXiv:2605.08725 -- https://arxiv.org/pdf/2605.08725] + | ≈{nbsp}47 seconds +| 40--100{nbsp}ns | contended cache line | core-to-core transfer; strongly topology dependent (same CCX / cross CCX / cross socket) | 2--5 minutes +| ~100{nbsp}ns | DRAM random access | full miss through the whole hierarchy; 122.9{nbsp}ns measured by pointer chasing on DDR5 footnoteref:[cxl] | *≈{nbsp}5 minutes* + +4+^s| Nanoseconds to microseconds -- entering the kernel + +| ~25{nbsp}ns | syscall via vDSO | not a real trap; almost as cheap as a function call footnoteref:[arkanis,"Measurements of system call performance and overhead", Arkanis Development, 5 Jan 2017 -- compares a plain function call against getpid() issued via the syscall instruction and via the vDSO -- https://arkanis.de/weblog/2017-01-05-measurements-of-system-call-performance-and-overhead/] | ≈{nbsp}1.5 minutes +| 70--100{nbsp}ns | syscall trap, bare | `getpid()` on a mitigation-free host footnoteref:[gms,Georg Sauthoff, "On the Costs of Syscalls", 30 Aug 2021. Median of 100 repetitions across a heterogeneous set of hosts; the conclusion is that user-kernel mode switches cost on the order of a few hundred nanoseconds on all hosts, with the fastest host switching modes in under 100 ns, and that the higher figures are explained by CPU-bug mitigations being enabled by default -- https://gms.tf/on-the-costs-of-syscalls.html] | 4--6 minutes +| 200--600{nbsp}ns | syscall trap, mitigated | ``a few hundred nanoseconds'' on typical hosts with CPU-bug mitigations enabled footnoteref:[gms] | 11--33 minutes +| ~1{nbsp}µs | syscall doing real work | the round figure we use for the diagram | *≈{nbsp}1 hour* +| ~2{nbsp}µs | context switch | scheduler plus cold cache on the new task | ≈{nbsp}2 hours + +4+^s| Microseconds -- the operating system + +| 10--30{nbsp}µs | thread wake-up | IRQ handling, `try_to_wake_up()`, scheduling onto a CPU footnoteref:[lkmlwake,Michal Schmidt, reply in the thread "nanosleep() accuracy", linux-kernel mailing list, 17 Aug 2007: the point being that a process always carries non-zero wake-up latency covering interrupt processing, waking and scheduling, and that 10-30 us is not unreasonable -- https://lkml.iu.edu/hypermail/linux/kernel/0708.2/0423.html] | 9{nbsp}hours -- 1{nbsp}day +| ~31--57{nbsp}µs | scheduling latency, avg | `cyclictest` at `SCHED_FIFO` 95 with memory locked footnoteref:[rhel,Sample cyclictest output and interpretation in: Finding Realtime Linux Kernel Latencies, Clark Williams (Red Hat), section "Using cyclictest" -- https://people.redhat.com/williams/latency-howto/rt-latency-howto.txt] | 1--2 days +| ~59--90{nbsp}µs | scheduling latency, max | same run; the tail is what hurts footnoteref:[rhel] | 2--3.5 days +| *50{nbsp}µs* | *OS timer slack, default* | the kernel may delay a timed wake-up by this much, deliberately, to coalesce wakeups footnoteref:[slack,Primary documentation: `man 7 time`, section "High-resolution timers", and `man 2 prctl`, description of PR_SET_TIMERSLACK. The 50 us default for non-realtime threads is stated explicitly in: Metronome -- adaptive and precise intermittent packet retrieval in DPDK, section II (discussion of nanosleep limits), arXiv:2103.13263 -- https://arxiv.org/pdf/2103.13263 ; a practical encounter with the same default, including the /proc interface, is documented in OpenJDK issue JDK-8307766 -- https://bugs.openjdk.org/browse/JDK-8307766] | *≈{nbsp}2 days* +| 1{nbsp}ns | OS timer slack, minimum | via `prctl(PR_SET_TIMERSLACK)` or `/proc/PID/timerslack_ns` footnoteref:[slack] | ≈{nbsp}3 seconds +| ~9{nbsp}µs | SSD read, best case | Optane / Z-NAND class at QD1 footnoteref:[anand,AnandTech, review of the Samsung 983 ZET (Z-NAND), page 3, "QD1 Random Read Performance" -- explains why queue depth 1 exposes best-case access latency, and reports an average of just under 9 us for an Optane drive against 16 us for the Z-NAND based 983 ZET -- https://at-web1.www.anandtech.com/show/13951/the-samsung-983-zet-znand-ssd-review/3] | ≈{nbsp}8 hours +| 20--70{nbsp}µs | NVMe SSD 4K random read | typical contemporary drive footnoteref:[simply,Summary of the storage latency hierarchy: a modern NVMe SSD serves a random 4K read in roughly 20-70 us, a SATA SSD in 100-200 us, and a spinning disk in 5-10 ms. simplyblock glossary entry "NVMe Latency" -- https://simplyblock.io/glossary/nvme-latency/] | 1--3 days +| 50--100{nbsp}µs | NVMe SSD, interrupt path | what you see without polling | *2--4 days* +| 100--200{nbsp}µs | SATA SSD 4K random read | the interface costs, not just the medium footnoteref:[simply] | 4--8 days +| 100{nbsp}µs -- 1{nbsp}ms | deep C-state exit | wake from C6 and below; worse when many cores wake at once | 4{nbsp}days -- 5{nbsp}weeks + +4+^s| Milliseconds -- where machine and human meet + +| 5--10{nbsp}ms | rotating disk, seek + rotation | 7200{nbsp}rpm; the number that has not improved in decades footnoteref:[simply] | *6{nbsp}months -- 1{nbsp}year* +| ~10{nbsp}ms | barely noticeable latency | the threshold below which a human stops perceiving delay | ≈{nbsp}1 year +| 20{nbsp}ms | one frame @{nbsp}50{nbsp}fps | | ≈{nbsp}2 years +| 40{nbsp}ms | one frame @{nbsp}25{nbsp}fps | our actual deadline | *≈{nbsp}4 years* +| ~100{nbsp}ms | UI starts to feel sluggish | | ≈{nbsp}11 years + +4+^s| Seconds and beyond -- the human + +| ~1{nbsp}s | ``the software takes time to respond'' | acceptable, but noticed | *≈{nbsp}1 century* +| ~3{nbsp}min | ``time for another cup of coffee'' | | ≈{nbsp}19 000 years +| ~15{nbsp}min | ``can't wait, I'll do something else'' | ≙{nbsp}1000{nbsp}s, which conveniently closes the fourth decade block | ≈{nbsp}95 000 years +|========================================================================= + + +The Memory Wall +--------------- + +For roughly two decades, CPU clock rates have hovered around 3{nbsp}GHz while +DRAM access latency has stayed near 60--100{nbsp}ns. The consequence is +arithmetic: a cache miss that cost a handful of cycles on early-1990s hardware +now costs several hundred. The processor did not merely get faster than +memory -- it got faster than memory _by a growing margin_, and the entire +cache hierarchy exists to paper over that gap. Note in the table above that +the raw device-level latency of DDR5 is barely distinguishable from DDR4; +the generational improvements went into bandwidth, not into latency. + +Once several cores share one memory subsystem, a single uniform path to memory +becomes the bottleneck. The answer was to move the memory controller onto the +CPU die and attach memory per socket -- and, with chiplet designs, effectively +per die group. This is what NUMA (non-uniform memory access) describes: memory +that ``belongs'' to the core you are running on is reached quickly, while +memory belonging to a neighbour costs noticeably more. The practical upshot is +that thread placement and memory placement are no longer independent concerns, +and that a scheduler which migrates a worker between cores may silently convert +local accesses into remote ones. + +Cores also keep private caches, and a store performed on one core does not +become visible to another core at the moment the instruction retires: it first +lands in that core's store buffer. Cache coherence guarantees that the cores +eventually agree on the content of a cache line, but it says nothing about +_when_, nor about the order in which several independent writes become visible. +This is what memory fences are for -- they constrain the order, and thereby +allow one core to reason about what an adjacent core has already done. The cost +shows up in the table twice: an uncontended fence is roughly the price of an +L3 hit, whereas a cache line actually bouncing between two cores costs about as +much as going to DRAM. It is worth internalising that the expensive part is +usually not the synchronisation primitive itself, but the coherence traffic it +provokes. + +x86 implements a comparatively strong model, _Total Store Order_ (TSO): of the +four possible reorderings, only StoreLoad is permitted. Acquire and release +semantics are therefore free on x86 -- they compile to plain `MOV` -- and only +sequential consistency requires an actual instruction. ARM and POWER are +weakly ordered and consequently offer a more differentiated set of barriers: +ARM distinguishes `DMB` from the heavier `DSB`, and provides graded load +acquire flavours such as `LDAR` and `LDAPR`; POWER separates the heavyweight +`sync` from the cheaper `lwsync`. The trap for the unwary is that C++ +`memory_order` distinctions which cost nothing on x86 have a real price on +ARM -- and, worse, that under-synchronised code can appear perfectly correct +on x86 and fail on ARM. + + +Crossing into the Kernel +------------------------ + +A system call is not a function call: it is a controlled trap. The `syscall` +instruction switches the processor to privilege level 0, and the kernel entry +path then swaps to the kernel `GS` base, switches stacks, and saves the user +register state. Since the Meltdown/Spectre family of vulnerabilities, that +sequence also includes a page-table switch (KPTI writes `CR3`, with the +attendant TLB consequences) and one or more branch-prediction barriers -- +retpolines, IBPB or IBRS depending on hardware and configuration. On return, +the whole dance runs in reverse. + +The direct cost is measurable: a minimal syscall such as `getpid()` amounts to +little more than two mode switches, and lands in the region of a hundred +nanoseconds on a fast host without mitigations, rising to several hundred once +mitigations are enabled.footnoteref:[gms] The indirect cost is harder to see +and frequently larger. Kernel entry code and the syscall handler evict +user-space data from L1 and L2, and disturb the branch predictors; the +application resumes with a cold working set and pays for it in misses that no +syscall benchmark will attribute to the syscall. This is precisely why the +interesting question is rarely ``how expensive is one syscall'' but rather +``how many syscalls does this design require per unit of work'' -- which is +the question `io_uring` was built to answer. + + +Operating System Slack +---------------------- + +A thread that asks to sleep for a specific duration will not, in general, wake +at that instant. Part of this is unavoidable: an interrupt must be handled, the +sleeping task must be moved to the run queue, and the scheduler must actually +place it on a CPU, which together account for something in the range of +10--30{nbsp}µs.footnoteref:[lkmlwake] + +The larger part, however, is deliberate. Linux maintains a per-thread _timer +slack_ value, and the kernel is free to delay a timed wake-up by up to that +amount in order to coalesce several wake-ups into one and thereby let the CPU +stay in a low-power state longer. For every thread not running under a +real-time scheduling policy, the default is 50{nbsp}µs.footnoteref:[slack] +This is not imprecision; it is a power-versus-precision trade that the kernel +makes on the application's behalf, and it can be adjusted -- down to +1{nbsp}ns -- via `prctl(PR_SET_TIMERSLACK)` or the per-process +`/proc/PID/timerslack_ns` interface. + +The distinction worth carrying away is between _resolution_ and _accuracy_. +Since Linux 2.6.21 the high-resolution timer subsystem reports nanosecond +resolution, and `clock_getres()` will happily confirm it. The accuracy with +which a `nanosleep()` actually returns is a different quantity altogether, two +to three orders of magnitude coarser, and dominated by slack, wake-up latency +and -- if the core had gone idle -- C-state exit time. Any design that assumes +it can steer a worker thread with microsecond-scale sleeps should be checked +against these figures first. + + +// ---------------------------------------------------------------------- +// Footnote definitions are attached at first use above via footnoteref. +// Repeated references reuse the id; see AsciiDoc footnoteref macro. +// ---------------------------------------------------------------------- + +Sources +------- + +The footnotes above carry the precise locations. Collected here as a reading +list, roughly in order of usefulness for further work. + +- Travis Downs, *A Concurrency Cost Hierarchy* -- + https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html + + The best single entry point on why synchronisation costs what it costs. + +- LKML thread, *[PATCH 3/4] x86,asm: Re-work smp_store_mb()* (Nov 2015) -- + https://lkml.iu.edu/hypermail/linux/kernel/1511.0/01052.html + + Kernel developers measuring `MFENCE` against `XCHG` and `lock addq`. + +- Georg Sauthoff, *On the Costs of Syscalls* (Aug 2021) -- + https://gms.tf/on-the-costs-of-syscalls.html + + Careful cross-host microbenchmark of the mode-switch cost. + +- *Measurements of system call performance and overhead* (Jan 2017) -- + https://arkanis.de/weblog/2017-01-05-measurements-of-system-call-performance-and-overhead/ + + Function call versus `syscall` instruction versus vDSO. + +- Brendan Gregg, *KPTI/KAISER Meltdown Initial Performance Regressions* -- + https://www.brendangregg.com/blog/2018-02-09/kpti-kaiser-meltdown-performance.html + + Relates mitigation overhead to syscall rate, context switch rate and working + set size -- the systemic view rather than the per-call view. + +- Clark Williams (Red Hat), *Finding Realtime Linux Kernel Latencies* -- + https://people.redhat.com/williams/latency-howto/rt-latency-howto.txt + + Walks through a `cyclictest` run and how to read its output. + +- `man 7 time` and `man 2 prctl` (PR_SET_TIMERSLACK) + + Primary documentation for timer slack. The 50{nbsp}µs default is stated in + https://arxiv.org/pdf/2103.13263 (Metronome, DPDK packet retrieval) and + encountered in practice in https://bugs.openjdk.org/browse/JDK-8307766 + +- *Performance Characterizations ... Samsung CXL Memory Module Hybrid + Prototype*, arXiv:2503.22017 -- https://arxiv.org/pdf/2503.22017 + + Table III contains a well-documented measurement of plain local DDR5. + +- *Single 32-bit Sub-Channel DDR5 DIMMs*, arXiv:2605.08725 -- + https://arxiv.org/pdf/2605.08725 + + Useful for the point that DDR5 did not improve absolute latency over DDR4. + +- simplyblock glossary, *NVMe Latency* -- + https://simplyblock.io/glossary/nvme-latency/ + + Compact summary of the NVMe / SATA SSD / rotating disk hierarchy. + +- AnandTech, *Samsung 983 ZET (Z-NAND) review*, page 3 -- + https://at-web1.www.anandtech.com/show/13951/the-samsung-983-zet-znand-ssd-review/3 + + Explains why queue depth 1 exposes best-case access latency. + + +A Note on Microbenchmarks +------------------------- + +One caveat deserves to be stated separately, because it explains why published +figures for synchronisation primitives vary so wildly. A hot microbenchmark +keeps every thread runnable and every core out of idle by construction. Under +those conditions an uncontended atomic operation or a mutex acquisition costs +single-digit to low-three-digit nanoseconds, and a benchmark will faithfully +report that. What such a benchmark structurally cannot observe is the cost of +_blocking_ -- of a thread actually parking and later being woken -- because +pressing the benchmark is precisely what prevents that from happening. + +The instruction-level cost of a synchronisation primitive is what a hot +microbenchmark measures. Its systemic cost -- coherence traffic under +contention, and blocking and wake-up latency when it is not contended -- is +what production tail latency measures. The two can differ by two to four orders +of magnitude, and the table above spans both regimes for exactly that reason. diff --git a/FrOSCon26/research/core2core.cpp b/FrOSCon26/research/core2core.cpp new file mode 100644 index 0000000..6d20d55 --- /dev/null +++ b/FrOSCon26/research/core2core.cpp @@ -0,0 +1,244 @@ +/* + * core2core.cpp — measure core-to-core latency by cache-line ping-pong. + * + * Code generated 8/2026 by Claude -- prompted by Ichthyostega + * Context: FrOSCon-26 talk on IO_URING + * + * Two threads are pinned to two specific logical CPUs and share a single + * cache line holding two atomic counters. Thread A stores i, then spins + * until thread B has answered with i; thread B does the mirror image. + * Each round trip therefore forces the line to migrate twice: A -> B and + * back. The one-way latency reported is half the measured round trip. + * + * What this actually measures is the cost of a *contended* cache line — + * the coherence traffic, not the atomic instruction. An uncontended + * atomic on a line already held in M state costs a couple of nanoseconds; + * the numbers here are one to two orders of magnitude larger, and that + * difference is the whole point. + * + * build: g++ -O2 -std=c++17 -pthread -o core2core core2core.cpp + * run: ./core2core # full matrix over all allowed CPUs + * ./core2core 3 9 # just the pair (3,9) + * + * Linux only (uses pthread_setaffinity_np / sched_getaffinity). + * + * Reading the results + * ------------------- + * * SMT siblings of one physical core: lowest of all, often 5–20 ns. + * They share L1/L2, so almost nothing migrates. Cross-check which + * CPUs are siblings with lscpu -e or lscpu -p=CPU,CORE,SOCKET. + * * Cores on the same die / CCX / cluster: typically 25–50 ns. + * * Across an AMD CCX or CCD boundary, or across sockets: 70–150 ns + * and upwards. On a multi-socket machine the cross-socket figure + * can exceed local DRAM latency. + * + * Getting numbers you can trust + * ----------------------------- + * * Set the CPU governor to performance; on an idle laptop the cores + * will otherwise be clocked down and the figures inflate. + * sudo cpupower frequency-set -g performance + * * Run on an otherwise idle machine. The tool reports the minimum + * over several repetitions, which filters most interference, but it + * cannot filter a busy system. + * * Do not read more than two significant digits into the result. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +namespace { + +constexpr std::size_t CACHELINE = 64; // deliberately hard-coded; see note below + +/** the two counters live in one cache line, so a store by either side + * invalidates the other side's copy. `alignas` keeps neighbouring data + * out of that line, so we measure our own ping-pong and nothing else. */ +struct alignas(CACHELINE) PingPong + { + std::atomic a{0}; + std::atomic b{0}; + char pad[CACHELINE - 2 * sizeof(std::atomic)]{}; + }; + +static_assert (sizeof(PingPong) == CACHELINE, "PingPong must occupy exactly one line"); + + +bool pin_to (int cpu) + { + cpu_set_t set; + CPU_ZERO (&set); + CPU_SET (cpu, &set); + return 0 == pthread_setaffinity_np (pthread_self(), sizeof(set), &set); + } + + +std::vector available_cpus() + { + std::vector cpus; + cpu_set_t set; + CPU_ZERO (&set); + if (0 == sched_getaffinity (0, sizeof(set), &set)) + for (int i = 0; i < CPU_SETSIZE; ++i) + if (CPU_ISSET (i, &set)) + cpus.push_back (i); + return cpus; + } + + +/** one ping-pong run between two pinned threads. + * @return round-trip time per iteration in nanoseconds, or -1 on failure */ +double one_run (PingPong& pp, int cpuA, int cpuB, + std::uint64_t warmup, std::uint64_t iters) + { + using Clock = std::chrono::steady_clock; + + pp.a.store (0, std::memory_order_relaxed); + pp.b.store (0, std::memory_order_relaxed); + + std::atomic ready {0}; + std::atomic go {false}; + std::atomic ok {true}; + + const std::uint64_t total = warmup + iters; + + std::thread responder{[&] + { + if (not pin_to (cpuB)) { ok.store (false); } + ready.fetch_add (1, std::memory_order_release); + while (not go.load (std::memory_order_acquire)) + ; + for (std::uint64_t i = 1; i <= total; ++i) + { + while (pp.a.load (std::memory_order_acquire) != i) + ; + pp.b.store (i, std::memory_order_release); + } + }}; + + Clock::time_point t0, t1; + + std::thread initiator{[&] + { + if (not pin_to (cpuA)) { ok.store (false); } + ready.fetch_add (1, std::memory_order_release); + while (not go.load (std::memory_order_acquire)) + ; + for (std::uint64_t i = 1; i <= total; ++i) + { + if (i == warmup + 1) // caches and predictors are warm now + t0 = Clock::now(); + pp.a.store (i, std::memory_order_release); + while (pp.b.load (std::memory_order_acquire) != i) + ; + } + t1 = Clock::now(); + }}; + + while (ready.load (std::memory_order_acquire) < 2) + ; + go.store (true, std::memory_order_release); + + initiator.join(); + responder.join(); + + if (not ok.load()) + return -1.0; + + auto elapsed = std::chrono::duration_cast (t1 - t0); + return double(elapsed.count()) / double(iters); + } + + +/** @return best (minimum) one-way latency in ns over @p reps repetitions */ +double measure_pair (int cpuA, int cpuB, int reps, + std::uint64_t warmup, std::uint64_t iters) + { + PingPong pp; + double best = std::numeric_limits::infinity(); + for (int r = 0; r < reps; ++r) + { + double rtt = one_run (pp, cpuA, cpuB, warmup, iters); + if (rtt < 0) return -1.0; + if (rtt < best) best = rtt; + } + return best / 2.0; // round trip covers two line migrations + } + +} // namespace (anon) + + + +int main (int argc, char** argv) +{ + constexpr int REPS = 5; + constexpr std::uint64_t WARMUP = 2000; + constexpr std::uint64_t ITERS = 20000; + + if (argc == 3) + { // single pair + int a = std::atoi (argv[1]); + int b = std::atoi (argv[2]); + if (a == b) + { + std::fprintf (stderr, "the two CPUs must differ " + "(both threads spin, they cannot share one core)\n"); + return 1; + } + double ns = measure_pair (a, b, REPS, WARMUP, ITERS); + if (ns < 0) + { + std::fprintf (stderr, "could not pin to CPU %d / %d\n", a, b); + return 1; + } + std::printf ("CPU %d <-> CPU %d : %.1f ns one-way (%.1f ns round trip)\n", + a, b, ns, 2 * ns); + return 0; + } + + auto cpus = available_cpus(); + if (cpus.size() < 2) + { + std::fprintf (stderr, + "need at least two available CPUs, found %zu.\n" + "(inside a restricted container, check the cpuset)\n", + cpus.size()); + return 1; + } + + std::printf ("core-to-core latency, one-way, nanoseconds\n"); + std::printf ("minimum of %d runs x %llu iterations; %zu CPUs available\n\n", + REPS, (unsigned long long)ITERS, cpus.size()); + + // header + std::printf (" "); + for (std::size_t j = 0; j + 1 < cpus.size(); ++j) + std::printf ("%6d", cpus[j]); + std::printf ("\n"); + + for (std::size_t i = 1; i < cpus.size(); ++i) + { + std::printf ("%5d ", cpus[i]); + for (std::size_t j = 0; j < i; ++j) + { + double ns = measure_pair (cpus[i], cpus[j], REPS, WARMUP, ITERS); + if (ns < 0) std::printf (" -"); + else std::printf ("%6.0f", ns); + std::fflush (stdout); + } + std::printf ("\n"); + } + + std::printf ("\nlowest values are usually SMT siblings; check with lscpu -e\n"); + return 0; +}