ichthyo/wegen
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Prototype: implement complete Proactor buffer state-engine

Browse source 
Turns out we need to add further states and transitions,
to establish full sync-link (backpressure) in both directions...
 - need to introduce a third cursor since the callback from GUI-Thread can be late
 - consequently also need a further state transition SHOW -> WAIT to create the barrier
 - furthermore, need marker state STOP to halt playback after a short read (end-of-file)
This commit is contained in:
Fischlurch 2026-07-16 03:28:48 +02:00
parent ae74bd323e
commit 0cb388d4a7
2 changed files with 239 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

156
FrOSCon26/FrOSCon.26.mm
View file

@ -334,6 +334,7 @@
</html></richcontent>
<icon BUILTIN="idea"/>
</node>
<node CREATED="1784162872108" ID="ID_558276865" LINK="https://unixism.net/loti/ref-liburing/sqe.html" MODIFIED="1784162885766" TEXT="submission-queue-entry(SQE)"/>
<node CREATED="1783707924633" ID="ID_1323434414" MODIFIED="1783707927633" TEXT="Operationen">
<node CREATED="1783707941501" ID="ID_360987965" LINK="https://unixism.net/loti/ref-liburing/submission.html#c.io_uring_prep_nop" MODIFIED="1783707941501" TEXT="io_uring_prep_nop"/>
<node CREATED="1783707947926" ID="ID_364910722" LINK="https://unixism.net/loti/ref-liburing/submission.html#c.io_uring_prep_read" MODIFIED="1783708042268" TEXT="io_uring_prep_read"/>
@ -361,6 +362,49 @@
<node CREATED="1783708346006" ID="ID_892211122" LINK="https://unixism.net/loti/ref-liburing/submission.html#c.io_uring_prep_poll_add" MODIFIED="1783708346006" TEXT="io_uring_prep_poll_add"/>
<node CREATED="1783708353854" ID="ID_1735840226" LINK="https://unixism.net/loti/ref-liburing/submission.html#c.io_uring_prep_poll_remove" MODIFIED="1783708353854" TEXT="io_uring_prep_poll_remove"/>
</node>
<node CREATED="1784162891066" ID="ID_1881921721" LINK="https://unixism.net/loti/ref-liburing/completion.html" MODIFIED="1784162910708" TEXT="Completion and waiting/polling"/>
<node CREATED="1784162918158" ID="ID_1019141045" LINK="https://unixism.net/loti/ref-liburing/cqe.html" MODIFIED="1784162936375" TEXT="completion-queue-entry (CQE)">
<node CREATED="1784162938404" ID="ID_181617694" MODIFIED="1784163095092" TEXT="res-Feld enth&#xe4;lt entweder operations-spezifisches Resultat &gt; 0">
<linktarget COLOR="#a9b4c1" DESTINATION="ID_181617694" ENDARROW="Default" ENDINCLINATION="255;0;" ID="Arrow_ID_472331517" SOURCE="ID_526495709" STARTARROW="None" STARTINCLINATION="251;9;"/>
</node>
<node CREATED="1784162963010" ID="ID_1477358179" MODIFIED="1784163021269" TEXT="oder direkt die negierte ERRNO">
<richcontent TYPE="NOTE"><html>
<head>
</head>
<body>
<p>
IO_URING kann errno nicht sinnvoll direkt verwenden (errno ist eine globale variable)
</p>
</body>
</html>
</richcontent>
</node>
</node>
</node>
<node CREATED="1784163031989" ID="ID_1146473140" MODIFIED="1784163035387" TEXT="Lese-Hinweise">
<node CREATED="1784163038135" ID="ID_998160291" MODIFIED="1784163050297" TEXT="die betr. prep_*-Operation anschauen"/>
<node CREATED="1784163051325" ID="ID_1399742238" MODIFIED="1784163064375" TEXT="diese verlinkt auf die unterliegende klassische Kernel-Operation"/>
<node CREATED="1784163065192" ID="ID_526495709" MODIFIED="1784163095092" TEXT="deren Parameter / R&#xfc;ckgabewerte werden systematisch in SQE / CQE gemapped">
<arrowlink DESTINATION="ID_181617694" ENDARROW="Default" ENDINCLINATION="255;0;" ID="Arrow_ID_472331517" STARTARROW="None" STARTINCLINATION="251;9;"/>
</node>
</node>
<node CREATED="1784162289299" ID="ID_653583150" MODIFIED="1784162292871" TEXT="Manpages">
<node CREATED="1784162294307" ID="ID_1880608167" LINK="https://man7.org/linux/man-pages/man7/io_uring.7.html" MODIFIED="1784162660448" TEXT="io_uring (low-level)"/>
<node CREATED="1784162298130" ID="ID_812019689" MODIFIED="1784162300427" TEXT="Liburing">
<node CREATED="1784162301417" ID="ID_1425990055" LINK="https://man7.org/linux/man-pages/man3/io_uring_queue_init.3.html" MODIFIED="1784162314395" TEXT="io_uring_queue_init"/>
<node CREATED="1784162324350" ID="ID_754199569" LINK="https://man7.org/linux/man-pages/man3/io_uring_wait_cqe.3.html" MODIFIED="1784162332857" TEXT="io_uring_wait_cqe"/>
<node CREATED="1784162374598" ID="ID_1701741193" LINK="https://man7.org/linux/man-pages/man3/io_uring_submit.3.html" MODIFIED="1784162382762" TEXT="io_uring_submit">
<node CREATED="1784162384092" ID="ID_1562969484" MODIFIED="1784162631504" TEXT="verweist auf sehr viele weitere (siehe ganz unten)...">
<icon BUILTIN="idea"/>
</node>
<node CREATED="1784162438390" ID="ID_967395162" LINK="https://man7.org/linux/man-pages/man3/io_uring_prep_read.3.html" MODIFIED="1784162578416" TEXT="io_uring_prep_read"/>
<node CREATED="1784162450985" ID="ID_703973678" LINK="https://man7.org/linux/man-pages/man3/io_uring_prep_write.3.html" MODIFIED="1784162588103" TEXT="io_uring_prep_write"/>
<node CREATED="1784162522735" ID="ID_1545552991" LINK="https://man7.org/linux/man-pages/man3/io_uring_prep_readv.3.html" MODIFIED="1784162595286" TEXT="io_uring_prep_readv"/>
<node CREATED="1784162531872" ID="ID_399339523" LINK="https://man7.org/linux/man-pages/man3/io_uring_prep_readv_fixed.3.html" MODIFIED="1784162602735" TEXT="io_uring_prep_readv_fixed"/>
<node CREATED="1784162619555" ID="ID_1359309346" LINK="https://man7.org/linux/man-pages/man3/io_uring_prep_timeout.3.html" MODIFIED="1784162627225" TEXT="io_uring_prep_timeout"/>
</node>
</node>
</node>
</node>
</node>
@ -1963,7 +2007,13 @@
<icon BUILTIN="info"/>
<node CREATED="1784078717834" ID="ID_17978925" MODIFIED="1784078954065" TEXT="IO beauftragen: FREE &#x27f6; READ"/>
<node CREATED="1784078958419" ID="ID_1531715805" MODIFIED="1784078974343" TEXT="&#xdc;bergabe an Display: READ &#x27f6; SHOW"/>
<node CREATED="1784078981634" ID="ID_1578413583" MODIFIED="1784079026236" TEXT="Frame-Flip: SHOW &#x27f6; FREE"/>
<node CREATED="1784165852364" ID="ID_721661485" MODIFIED="1784167391018" TEXT="Warten auf Display: SHOW &#x27f6; WAIT">
<linktarget COLOR="#6574c5" DESTINATION="ID_721661485" ENDARROW="Default" ENDINCLINATION="925;77;" ID="Arrow_ID_86014624" SOURCE="ID_312946304" STARTARROW="None" STARTINCLINATION="397;-23;"/>
</node>
<node CREATED="1784078981634" ID="ID_1578413583" MODIFIED="1784167275599" TEXT="Frame-Flip: WAOT &#x27f6; FREE"/>
<node CREATED="1784165852364" ID="ID_849967229" MODIFIED="1784167346009" TEXT="Abbrunch/End-of-File: markiert durch STOP">
<linktarget COLOR="#6574c5" DESTINATION="ID_849967229" ENDARROW="Default" ENDINCLINATION="755;51;" ID="Arrow_ID_1693130420" SOURCE="ID_948621808" STARTARROW="None" STARTINCLINATION="411;-22;"/>
</node>
</node>
<node CREATED="1784078466234" ID="ID_424139171" MODIFIED="1784078496653">
<richcontent TYPE="NODE"><html>
@ -2222,7 +2272,22 @@
</node>
<node BACKGROUND_COLOR="#eef0c5" COLOR="#990000" CREATED="1784143738263" ID="ID_1385880788" MODIFIED="1784152297376" TEXT="Handler-Loop">
<icon BUILTIN="pencil"/>
<node CREATED="1784143781367" ID="ID_1463866164" MODIFIED="1784143789996" TEXT="gelesene Daten entgegennehmen"/>
<node COLOR="#338800" CREATED="1784143781367" ID="ID_1463866164" MODIFIED="1784165740325" TEXT="gelesene Daten entgegennehmen">
<icon BUILTIN="button_ok"/>
<node CREATED="1784165743594" ID="ID_465943099" MODIFIED="1784165755001" TEXT="ERRNO steckt im cqe.res">
<icon BUILTIN="idea"/>
<node BACKGROUND_COLOR="#e0ceaa" COLOR="#690f14" CREATED="1784165759121" HGAP="34" ID="ID_469358369" MODIFIED="1784165775260" TEXT="mache hier aber keine Fehlerbehandlung...." VSHIFT="4">
<font NAME="SansSerif" SIZE="10"/>
</node>
</node>
<node CREATED="1784165783214" ID="ID_907378266" MODIFIED="1784165801510" TEXT="short-reads erkennen &#x27f9; Playback stoppen">
<node CREATED="1784165802955" ID="ID_165426868" MODIFIED="1784165813591" TEXT="das geht bei dieser Konstrollstuktur nur indirekt"/>
<node COLOR="#435e98" CREATED="1784165814417" ID="ID_948621808" MODIFIED="1784167329763" TEXT="f&#xfc;hre dazu eine weitere Status-Auspr&#xe4;gung ein: STOP">
<arrowlink COLOR="#6574c5" DESTINATION="ID_849967229" ENDARROW="Default" ENDINCLINATION="755;51;" ID="Arrow_ID_1693130420" STARTARROW="None" STARTINCLINATION="411;-22;"/>
<icon BUILTIN="yes"/>
</node>
</node>
</node>
<node COLOR="#338800" CREATED="1784143790590" ID="ID_1916682510" MODIFIED="1784152292235" TEXT="neue Leseauftr&#xe4;ge absetzen">
<icon BUILTIN="button_ok"/>
<node CREATED="1784152255150" ID="ID_1732543149" MODIFIED="1784152264977" TEXT="lagere das in eine Hilfsfunktion aus"/>
@ -2231,7 +2296,92 @@
<icon BUILTIN="idea"/>
</node>
</node>
<node CREATED="1784143798563" ID="ID_532286724" MODIFIED="1784143809876" TEXT="Time-Tick : Frame weiterschalten"/>
<node COLOR="#338800" CREATED="1784143798563" ID="ID_532286724" MODIFIED="1784168674393" TEXT="Time-Tick : Frame weiterschalten">
<icon BUILTIN="button_ok"/>
<node CREATED="1784166180437" ID="ID_411616254" MODIFIED="1784166202948">
<richcontent TYPE="NODE"><html>
<head>
</head>
<body>
<p>
verwende hier den zweiten Cursor: <font face="Monospaced" color="#271bd4">nextShow_</font>
</p>
</body>
</html>
</richcontent>
</node>
<node CREATED="1784166214698" ID="ID_255228689" MODIFIED="1784166237247" TEXT="dar&#xfc;ber auf den State dieses Buffer-Slot zugreifen"/>
<node CREATED="1784166238079" ID="ID_503678940" MODIFIED="1784166404098" TEXT="wenn Daten da sind (state == SHOW) &#x27f9; trigger Displayer">
<richcontent TYPE="NOTE"><html>
<head>
</head>
<body>
<p>
...und zwar via Glib::Dispatcher &#10230; GUI-Thread.
</p>
<p>
Der zugeh&#246;rige Handler-Code l&#228;uft dann bei n&#228;chster Gelegenheit in der GUI-Event-Loop, und holt sich per Callback (Interface PixelFeed) die Daten; in diesem Callback mu&#223; die Read-Barrier realisiert sein, damit der GUI-Thread alle vor diesem Punkt erfolgten Daten&#228;nderungen auch tats&#228;chlich sieht
</p>
</body>
</html>
</richcontent>
</node>
<node BACKGROUND_COLOR="#e0ceaa" COLOR="#690f14" CREATED="1784166414946" ID="ID_1720545051" MODIFIED="1784166498112" TEXT="wenn hier Short-Read markiert &#x27f9; Player anhalten">
<richcontent TYPE="NOTE"><html>
<head>
</head>
<body>
<p>
Beachte: das darf erst hier passieren, weil wir hier den n&#228;chst niedersten erwarteten Frame anschauen; da wir immer etwas im Voraus lesen, werden wir auch ein end-of-File schon einige Ticks fr&#252;her markieren, m&#252;ssen aber die Schleife noch solange weiterlaufen lassen, bis die letzten erfolgreich gelesenen Frames in die Anzeige &#252;bertragen wurden
</p>
</body>
</html>
</richcontent>
<icon BUILTIN="messagebox_warning"/>
</node>
<node BACKGROUND_COLOR="#fafe99" COLOR="#fa002a" CREATED="1784166503838" ID="ID_645230473" MODIFIED="1784166532804" TEXT="Bei genauerer Betrachtung : t&#xfc;ckisches Korrektheits-Problem hier">
<icon BUILTIN="broken-line"/>
<node CREATED="1784166547264" ID="ID_225348495" MODIFIED="1784166564736" TEXT="und zwar k&#xf6;nnte der GUI-Thread &#xfc;berlastet sein, und die Daten nicht rechzeitig abholen"/>
<node CREATED="1784166577960" ID="ID_1754845143" MODIFIED="1784166592939" TEXT="damit k&#xf6;nnte der n&#xe4;chste Tick passieren, w&#xe4;hrend der GUI-Thread grade arbeitet"/>
<node CREATED="1784166593774" ID="ID_383603118" MODIFIED="1784166605762" TEXT="oder vorher, bevor die Daten &#xfc;berahupt &#xfc;bernommen wurden"/>
<node CREATED="1784166606710" ID="ID_499843773" MODIFIED="1784166618527" TEXT="in beiden F&#xe4;llen ist der Cursor inkonsitent"/>
<node CREATED="1784166652416" ID="ID_261339357" MODIFIED="1784166866154">
<richcontent TYPE="NODE"><html>
<head>
</head>
<body>
<p>
brauche doch einen 3.Cursor: <font face="Monospaced" color="#5f15de">currShow_</font>
</p>
</body>
</html>
</richcontent>
</node>
<node BACKGROUND_COLOR="#d9b995" COLOR="#690f14" CREATED="1784165814417" ID="ID_312946304" MODIFIED="1784167391017" TEXT="mu&#xdf; zwingend weitere State-Transition einf&#xfc;rhen: WAIT">
<arrowlink COLOR="#6574c5" DESTINATION="ID_721661485" ENDARROW="Default" ENDINCLINATION="925;77;" ID="Arrow_ID_86014624" STARTARROW="None" STARTINCLINATION="397;-23;"/>
<icon BUILTIN="yes"/>
<node CREATED="1784167398651" ID="ID_977847492" MODIFIED="1784167422438" TEXT="denn der 3.Cursor kann erst hier bewegt werden"/>
<node CREATED="1784167422935" ID="ID_953106086" MODIFIED="1784167433471" TEXT="und es ist nun der 3.Cursor, der vom GUI-Thread verwendet wird"/>
<node CREATED="1784167433959" ID="ID_1914570598" MODIFIED="1784167447134">
<richcontent TYPE="NODE"><html>
<head>
</head>
<body>
<p>
also brauche ich die cross-Thread-Barriere <b>erst hier</b>
</p>
</body>
</html>
</richcontent>
</node>
</node>
</node>
</node>
</node>
</node>
<node CREATED="1783644430659" ID="ID_269686877" MODIFIED="1783644452555" TEXT="Aufrabe: Vorlauf steuern">

90
FrOSCon26/examples/demo-uring.cpp
View file

@ -46,7 +46,7 @@ class UringVideoPlayer
using VideoDisplay = EglDisplay<uint8_t, VIDEO_WIDTH,VIDEO_HEIGHT>;
using Buffer = VideoDisplay::Buffer;
enum BuffState { FREE, READ, SHOW };
enum BuffState { FREE=0, READ, SHOW, WAIT, STOP };
using AtomicBuffState = std::atomic<BuffState>;
using BufferAlloc = unique_ptr<Buffer[]>;
@ -73,6 +73,7 @@ class UringVideoPlayer
size_t nextRead_{0};
size_t nextShow_{0};
size_t currShow_{0};
BufferAlloc buffer_{};
BuffStateAlloc buffState_{};
std::atomic_bool playing_{false};
@ -92,6 +93,8 @@ class UringVideoPlayer
private:
void playVideo();
void addNextReadSubmissions (io_uring&, int inputFD);
void handleCompletedRead (io_uring_cqe&);
void maybeDisplayFrame();
};
@ -120,7 +123,7 @@ UringVideoPlayer::playVideo()
// Setup periodic frame beat, firing infinitely
const auto TICK_TAG = UINT64_MAX; // special marker for timer events (different from possible frame numbers)
auto timespec = asKernelTimespec (40ms); // 40ms ≈ 25fps
auto timespec = asKernelTimespec (40ms); // 25fps ≙ 40ms per frame
io_uring_sqe* timerSubmission = io_uring_get_sqe (&uRing);
io_uring_prep_timeout (timerSubmission, &timespec, /*no count limit*/ 0, IORING_TIMEOUT_MULTISHOT);
io_uring_sqe_set_data64(timerSubmission, TICK_TAG);
@ -129,7 +132,27 @@ UringVideoPlayer::playVideo()
io_uring_submit (&uRing);
/* ===== proactor-loop ===== */
playing_ = true;
while (playing_.load (std::memory_order_acquire))
{
io_uring_cqe* completion;
// blocking wait for the next completion entry
int res = io_uring_wait_cqe (&uRing, &completion);
if (0 > res)
__FAIL ("retrieval of io_uring completion failed");
uint64_t tag = completion->user_data;
if (tag == TICK_TAG)
{
maybeDisplayFrame();
addNextReadSubmissions (uRing, inputFD);
}
else
handleCompletedRead (*completion);
// mark completion as treated (Barrier)
io_uring_cqe_seen (&uRing, completion);
}
close (inputFD);
}
@ -160,17 +183,76 @@ UringVideoPlayer::addNextReadSubmissions (io_uring& uRing, int inputFD)
}
/** transition state of the buffer slot due to reported result */
void
UringVideoPlayer::handleCompletedRead (io_uring_cqe& completion)
{
if (0 > completion.res)
__FAIL ("async read of frame from video file failed");
size_t frameNr = completion.user_data; // Note: completion was marked with the frameNr as »user data«
auto& state = buffState_[frameNr % RING_SIZ]; // Can thus pick buffer slot based on frame number, modulus ring size
if (completion.res != sizeof(Buffer))
state.store(STOP, std::memory_order_relaxed); // Insufficient read, possibly end-of-file => signal stop of playback (to ourselves)
else
state.store(SHOW, std::memory_order_relaxed); // Success; frame data is in the buffer => mark ready for use at the next tick event
}
/**
* At each timer tick: see if the next frame can be displayed.
* @remark determine the buffer slot where the frame data was placed,
* based on the cursor for the next frame to display, wrapping
* through the ring of buffers. Depending on the state marked
* for this buffer, the frame can be displayed if the data
* was already loaded, or otherwise the frame will slip
* to the next regular time tick.
*/
void
UringVideoPlayer::maybeDisplayFrame()
{
auto prevState = buffState_[currShow_ % RING_SIZ].load (std::memory_order_relaxed);
auto nextState = buffState_[nextShow_ % RING_SIZ].load (std::memory_order_relaxed);
if (nextState == SHOW and prevState == FREE)
{ // can hand over this next frame to the VideoDisplay in the GUI-thread
currShow_ = nextShow_;
++nextShow_;
// mark state to wait for display to complete; this also creates a write barrier
buffState_[currShow_ % RING_SIZ].store (WAIT, std::memory_order_release);
// Notify GUI thread via Glib::Dispatcher
dispatchFrame_();
}
else
if (nextState == STOP)
{
playing_ = false;
std::cout << "Playback stopped due to short read" << std::endl;
}
else
std::cout << "WARNING: Frame slipped" << std::endl;
}
UringVideoPlayer::Buffer*
UringVideoPlayer::BufferDispatcher::retrieveFrame()
{
///////OOO threadsafe hand-over of current frame data
// retrieve buffer state from proactor-thread; this also creates a read barrier
auto frameState = player_.buffState_[player_.currShow_ % RING_SIZ].load (std::memory_order_acquire);
if (frameState == WAIT)
return & player_.buffer_[player_.currShow_ % RING_SIZ];
else
return nullptr;
// (can happen only in boundary cases)
}
void
UringVideoPlayer::BufferDispatcher::completedFrame()
{
///////OOO mark end of buffer access
// mark buffer as display complete for the proactor-thread; this also creates a write barrier
player_.buffState_[player_.currShow_ % RING_SIZ].store (FREE, std::memory_order_release);
}