benn/lumiera_
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
lumiera_/doc/devel/dump/2024-04-08.Scheduler-LoadTest/Graph-15

181 lines
7.7 KiB
Text
Raw Normal View History

Scheduler-test: adjust contention mitigation as result of testing Investigate the behaviour over a wider range of job loads, job count and worker pool sizes. Seemingly the processing can not fully utilise the available worker pool capacity. By inspection of trace-dumps, one impeding mechanism could be identified: the »stickiness« of the contention mitigation. Whenever a worker encounters repeated contention, it steps up and adds more and more wait cycles to remove pressure from the schedule coordination. As such this is fine and prevents further degradation of performance by repeated atomic synchronisation. However, this throttling was kept up needlessly after further successful work-pulls. Since job times of several milliseconds can be expected on average in media processing, such a long retention would spread a performance degradation over a duration of several frames. Thus, the scheme for step-down was changed to decrease the throttling by a power series rather than just documenting the level.
2024-04-10 03:29:44 +02:00
#
#
# GNUPLOT - data plot from Lumiera
#
set term wxt size 600,600
set datafile separator ",;"
####---------Data--------------------------------
$RunData << _End_of_Data_
"load size","result time","concurrency","avg jobtime","avg impeded"
179,35.63124,1.79985243286509,358.273597765363,105.151083798883
177,35.333096,1.84486881647733,368.276423728813,68.0845141242937
59,11.826895,2.1317825177276,427.328271186441,125.632440677966
139,27.656609,1.69648524155655,337.546971223022,93.1134460431655
69,7.592238,3.84578236351389,423.160797101449,15.62
76,15.253411,1.56144366660021,313.386078947368,93.5487631578947
139,27.697876,1.64504964207364,327.801302158273,73.7196402877698
162,32.371138,1.72661112500895,345.014611111111,60.4682901234568
119,23.779756,1.75157289250571,350.016605042017,148.353
148,29.496382,1.78577796422626,355.905331081081,111.492858108108
156,31.151338,1.6940306063258,338.277692307692,109.569416666667
39,7.804463,2.27069703578581,454.399256410256,22.2919743589744
158,31.64639,1.64517557926828,329.518151898734,91.2047848101266
125,24.638671,1.81110016039421,356.984808,57.763192
80,8.89486,3.31325776909361,368.38705,23.6623875
41,3.246095,3.96197523485912,313.681658536585,5.52668292682927
86,12.878812,2.63628617297931,394.793418604651,55.0483953488372
69,15.889,1.00159663918434,230.643028985507,222.759507246377
61,12.498178,2.15249630786183,441.021016393443,114.346213114754
199,39.870461,1.82091468668998,364.82767839196,50.0526582914573
142,28.522807,1.74354817181913,350.217521126761,60.3965422535211
156,31.253045,1.79720718413198,360.052544871795,56.4501730769231
119,23.819071,1.85057036859246,370.410647058824,46.6504537815126
162,32.279941,1.67515166771835,333.78887037037,71.3886975308642
157,31.518053,1.97087783309457,395.65752866242,95.0335286624204
25,8.924809,0.964483161488386,344.31312,356.99236
181,36.250303,1.781842347635,356.863674033149,56.5681712707182
184,36.75795,1.69997083624087,339.605668478261,108.827619565217
16,1.685641,3.52622355531219,371.4966875,46.988875
79,15.651189,1.63294660872091,323.513367088607,70.1948987341772
20,2.156301,2.13973512974302,230.69565,36.92085
191,38.367116,1.94760476654018,391.225015706806,36.3046230366492
55,7.196328,2.82408125366159,369.509363636364,36.3265636363636
75,14.963745,1.31130422230531,261.62696,137.272213333333
130,25.941701,1.61283537266889,321.843792307692,80.9860692307692
120,21.701709,1.8366870093042,332.160391666667,83.3058583333333
147,29.460911,1.72671754787216,346.058993197279,72.4423809523809
70,7.844502,2.89830176600121,324.7962,40.4968
82,15.553826,1.55505089230135,294.963304878049,164.758268292683
65,7.127141,4.72273300051171,517.839753846154,4.49523076923077
18,3.341583,1.8750475448313,348.090388888889,23.1966111111111
38,3.918406,3.34534731725094,344.958657894737,24.6373684210526
100,20.205017,1.74584866718994,352.74902,63.26982
172,34.398834,1.67980103627931,335.948819767442,90.5519593023256
159,31.833826,1.90874169507617,382.154408805031,91.7036666666667
176,35.29628,1.61548984765533,323.981715909091,121.156585227273
13,3.022598,1.8098172499287,420.796153846154,97.3303076923077
45,7.57213,2.18830553622296,368.2252,38.1020444444444
45,9.667158,1.49805103009592,321.819911111111,171.646822222222
70,13.978447,1.41961027573378,283.484957142857,122.369428571429
44,6.995391,1.7957738173606,285.503181818182,31.7503409090909
183,36.697165,1.62957740741008,326.780715846994,87.2222677595628
26,4.485739,2.88331220340729,497.453307692308,46.1854230769231
159,31.623796,1.71386158069069,340.873012578616,66.8181823899371
145,29.027672,1.77185642031507,354.709427586207,96.5250965517241
179,35.790229,1.74933599893982,349.771709497207,60.8866592178771
180,36.0188,1.74721054005131,349.624594444444,69.2989166666667
126,25.17708,1.89118313164195,377.892611111111,137.614285714286
163,32.608486,1.53913907563816,307.907944785276,118.752975460123
192,38.643666,1.7102021583563,344.210838541667,62.0867760416667
4,0.907267,2.19658490830153,498.22225,52.75375
179,35.906926,1.68108564904721,337.221329608939,100.840290502793
183,36.508054,1.57717162903287,314.641896174863,121.484327868853
56,12.837901,1.76890599171936,405.518571428571,51.546875
53,11.64554,1.53189530069022,336.599018867925,171.561660377358
113,22.543634,1.6258490534401,324.358814159292,82.5855398230088
172,34.462216,1.9703688816761,394.7865,57.0161802325581
196,39.243665,1.69912741330352,340.204015306122,69.6907602040817
179,35.903897,1.72076719137201,345.152223463687,81.7715195530726
54,7.078423,2.57057694913118,336.95612962963,87.4055740740741
61,12.715775,0.967899793760113,201.763868852459,208.455327868853
193,38.714267,1.80170227683763,361.407165803109,59.2353730569948
88,10.61802,3.25951815875276,393.291238636364,25.2093863636364
157,31.433111,1.6798099939901,336.316267515924,107.236203821656
165,32.836242,1.62567391238011,323.521345454545,79.4364363636363
47,4.921991,3.77766497338171,395.609212765957,51.0604680851064
134,26.475937,1.58844844660266,313.84821641791,137.973470149254
174,34.912692,1.83480391600854,368.149103448276,59.1995862068966
200,40.095864,1.63798313461957,328.381745,117.2451
174,34.836438,1.9257432978653,385.551936781609,47.0866149425287
_End_of_Data_
#
#---common-styles-for-plots-from-Lumiera---------
set style line 1 linetype 1 linewidth 2 linecolor rgb '#240CC3'
set style line 2 linetype 3 linewidth 2 linecolor rgb '#1149D6'
set style line 3 linetype 2 linewidth 2 linecolor rgb '#0B7FCC'
set style line 4 linetype 6 linewidth 2 linecolor rgb '#37999D'
set style line 5 linetype 7 linewidth 2 linecolor rgb '#248269'
set style line 6 linetype 13 linewidth 2 linecolor rgb '#00C443'
set style line 7 linetype 12 linewidth 2 linecolor rgb '#EBE83E'
set style line 8 linetype 5 linewidth 2 linecolor rgb '#762C14'
set style line 9 linetype 4 linewidth 2 linecolor rgb '#AA0519'
set style line 10 linetype 1 linewidth 1 linecolor rgb '#303030'
set style line 11 linetype 0 linewidth 1 linecolor rgb '#A0A0A0' dashtype 3
#---(End)styles----------------------------------
#
#---axis-and-grid-setup--------------------------
unset border
set tics nomirror
set grid back linestyle 11
set arrow 10 from graph 0,0 to graph 1.04,0 size screen 0.025,15,60 filled ls 10
set arrow 11 from graph 0,0 to graph 0,1.08 size screen 0.025,15,60 filled ls 10
set xlabel 'load size ⟶ number of jobs'
Scheduler-test: complete and document the Load-peak tests - use parameters known to produce a clean linear model - assert on properties of this linear model Add extended documentation into the !TiddlyWiki, with a textual account of the various findings, also including some of the images and diagrams, rendered as SVG
2024-04-11 23:08:20 +02:00
set ylabel 'active time ⟶ ms' offset 2
Scheduler-test: adjust contention mitigation as result of testing Investigate the behaviour over a wider range of job loads, job count and worker pool sizes. Seemingly the processing can not fully utilise the available worker pool capacity. By inspection of trace-dumps, one impeding mechanism could be identified: the »stickiness« of the contention mitigation. Whenever a worker encounters repeated contention, it steps up and adds more and more wait cycles to remove pressure from the schedule coordination. As such this is fine and prevents further degradation of performance by repeated atomic synchronisation. However, this throttling was kept up needlessly after further successful work-pulls. Since job times of several milliseconds can be expected on average in media processing, such a long retention would spread a performance degradation over a duration of several frames. Thus, the scheme for step-down was changed to decrease the throttling by a power series rather than just documenting the level.
2024-04-10 03:29:44 +02:00
set key autotitle columnheader tmargin
#
stats $RunData using 1:2 nooutput
# regression line function derived from data
regLine(x) = STATS_slope * x + STATS_intercept
#
regLabel = sprintf("Model: %3.2f·p + %3.2f", STATS_slope, STATS_intercept)
set key horizontal width -4
set xtics 10
set yrange [0:*]
plots = STATS_columns - 1
# Adjust layout based on number of data sequences;
# additional sequences placed into secondary diagram
#
if (plots > 1) {
set multiplot layout 2,1 # 2 rows 1 column
set lmargin at screen 0.12 # fixed margins to align diagrams
set rmargin at screen 0.88
}
#
#
####---------Scatter-Regression-Plot-------------
plot $RunData using 1:2 with points linestyle 1, \
regLine(x) with line linestyle 9 title regLabel
if (plots > 1) {
# switch off decorations for secondary diagram
unset arrow 10
unset arrow 11
set border 2+8
set key bmargin vertical
unset xlabel
set format x ""
set ylabel 'concurrent threads ⟶'
if (plots <= 2) {
####---------------------------------
plot $RunData using 1:3 with impulses linestyle 3
} else {
# more than one additional data sequence
#
set y2tics
Scheduler-test: complete and document the Load-peak tests - use parameters known to produce a clean linear model - assert on properties of this linear model Add extended documentation into the !TiddlyWiki, with a textual account of the various findings, also including some of the images and diagrams, rendered as SVG
2024-04-11 23:08:20 +02:00
set y2label 'avg job time ⟶ µs' offset -2.6
Scheduler-test: adjust contention mitigation as result of testing Investigate the behaviour over a wider range of job loads, job count and worker pool sizes. Seemingly the processing can not fully utilise the available worker pool capacity. By inspection of trace-dumps, one impeding mechanism could be identified: the »stickiness« of the contention mitigation. Whenever a worker encounters repeated contention, it steps up and adds more and more wait cycles to remove pressure from the schedule coordination. As such this is fine and prevents further degradation of performance by repeated atomic synchronisation. However, this throttling was kept up needlessly after further successful work-pulls. Since job times of several milliseconds can be expected on average in media processing, such a long retention would spread a performance degradation over a duration of several frames. Thus, the scheme for step-down was changed to decrease the throttling by a power series rather than just documenting the level.
2024-04-10 03:29:44 +02:00
####---------------------------------------------
plot $RunData using 1:3 with impulses linestyle 3, \
for [i=4:*] $RunData using 1:i with points linestyle 5+(i-4) axes x1y2
}
}
Reference in a new issue Copy permalink