182 lines
4.9 KiB
Text
182 lines
4.9 KiB
Text
Resource Management: Profiling
|
|
==============================
|
|
|
|
// please don't remove the //word: comments
|
|
|
|
[grid="all"]
|
|
`------------`-----------------------
|
|
*State* _Idea_
|
|
*Date* _Fri Jul 23 19:34:29 2010_
|
|
*Proposed by* Christian Thaeter <ct@pipapo.org>
|
|
-------------------------------------
|
|
|
|
[abstract]
|
|
******************************************************************************
|
|
From the beginning on we planned some kind of 'profiling' to adapt dynamically
|
|
to workload and machine capabilities. I describe here how statistic data can be
|
|
gathered in a generic way. This will later work together with other components
|
|
tuning the system automatically.
|
|
******************************************************************************
|
|
|
|
|
|
Description
|
|
-----------
|
|
//description: add a detailed description:
|
|
|
|
I just introduce some ideas about the planned profiling framework here, nothing
|
|
is defined/matured yet this is certainly subject for futher discussion and
|
|
refinement.
|
|
|
|
.Requirements/Evaluation generic::
|
|
Profiling should be sufficiently abstracted to have a single set of
|
|
datastructures and algorithms to work on a broad range of subjects
|
|
being profiled. Moreover the profiling core just offers unitless
|
|
counters, semantic will be added on top of that on a higher level.
|
|
|
|
least possible overhead::
|
|
Profiling itself must not cost much, it must not block and should avoid
|
|
expensive operations. Simple integer arithmetic without divisions is
|
|
suggested.
|
|
|
|
accurate::
|
|
We may sample data on in stochastic way to reduce the overhead,
|
|
nevertheless data which gets sampled must be accurately stored and
|
|
processed without rounding losses and drifts.
|
|
|
|
transient values::
|
|
It's quite common that some values can be far off either in maximum or
|
|
in minimum direction, the system should adapt to this and recover from
|
|
such false alarms. Workload also changes over time we need to find some
|
|
way to measure the current/recent workload an grand total over the
|
|
whole application runtime is rather uninteresting. While it is also
|
|
important that we adapt slow enough not to get into some osccilating
|
|
cycle.
|
|
|
|
active or passive system::
|
|
Profiling can be only passive collecting data and let it be analyzed by
|
|
some other component or active triggering some action when some limits
|
|
are reached. I am yet a bit undecided and keep it open for both.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.Brainstorming in Code
|
|
[source,C]
|
|
------------------------------------------------------------------------------
|
|
|
|
typedef int64_t profile_value;
|
|
|
|
struct profile
|
|
{
|
|
ProfileVTable vtable;
|
|
|
|
/*
|
|
Using trylock for sampling makes it never contend on the lock but some
|
|
samples are lost. Should be ok.
|
|
*/
|
|
mutex_t lock; /* with trylock? */
|
|
|
|
|
|
/* statistics / running averages */
|
|
|
|
/* n being a small number 2-5 or so */
|
|
profile_value max[n]; /* n maximum values seen so far,
|
|
decreased by recovery */
|
|
profile_value min[n]; /* n minimum values seen so far,
|
|
increased by recovery */
|
|
|
|
/* store sum & count, but average calculation implies a division and will be
|
|
only done on demand */
|
|
profile_value count; /* count profile calls */
|
|
profile_value sum; /* sum up all calls, average =
|
|
sum/count */
|
|
|
|
/* current is the sampled value to be integrated */
|
|
|
|
/* trend is caclulated before theb new run_average */
|
|
profile_value trend; /* trend = (trend +
|
|
(run_average-current))>>1 */
|
|
|
|
/* we may need some slower diverging formula for running average */
|
|
profile_value run_average; /* run_average = (run_average +
|
|
current)>>1) */
|
|
|
|
|
|
/* active limits, define whats good and whats bad, calls back to vtable when
|
|
limit is hit */
|
|
profile_value max_limit;
|
|
profile_value min_limit;
|
|
/* do we want limits for trends too? */
|
|
|
|
/* we count how often we hit limits, a hit/miss ratio will give a good value
|
|
for optimization */
|
|
profile_value hit_cnt;
|
|
profile_value high_miss_cnt;
|
|
profile_value low_miss_cnt;
|
|
|
|
/* recovery state */
|
|
int rec_init;
|
|
int rec_current;
|
|
int rec_percent;
|
|
|
|
|
|
void* extra;
|
|
};
|
|
------------------------------------------------------------------------------
|
|
|
|
|
|
Tasks
|
|
~~~~~
|
|
// List what would need to be done to implement this Proposal in a few words:
|
|
// * item ...
|
|
|
|
|
|
|
|
|
|
Discussion
|
|
~~~~~~~~~~
|
|
|
|
Pros
|
|
^^^^
|
|
// add just a fact list/enumeration which make this suitable:
|
|
// * foo
|
|
// * bar ...
|
|
|
|
|
|
|
|
Cons
|
|
^^^^
|
|
// fact list of the known/considered bad implications:
|
|
|
|
|
|
|
|
Alternatives
|
|
^^^^^^^^^^^^
|
|
//alternatives: if possible explain/link alternatives and tell why they are not
|
|
viable:
|
|
|
|
|
|
|
|
Rationale
|
|
---------
|
|
//rationale: Describe why it should be done *this* way:
|
|
|
|
|
|
|
|
//Conclusion
|
|
//----------
|
|
//conclusion: When approbate (this proposal becomes a Final) write some
|
|
conclusions about its process:
|
|
|
|
|
|
|
|
|
|
Comments
|
|
--------
|
|
//comments: append below
|
|
|
|
|
|
//endof_comments:
|