ichthyo/LUMIERA.clone
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Library: a little bit of modernising and overhaul

Browse source 
- fix some warnings due to uninitialised members
  (no real problem, since these members get assigned anyway)
- use a lambda as example function right in the test
- use move initialisation and the new util::join
This commit is contained in:
Fischlurch 2017-10-29 13:22:25 +01:00
parent f0a32c986a
commit 7e241d9a11
2 changed files with 123 additions and 102 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

32
src/lib/itertools.hpp
View file

@ -42,12 +42,24 @@
** The FilterIter template can be used to build a filter into a pipeline,
** as it forwards only those elements from its source iterator, which pass
** the predicate evaluation. Anything acceptable as ctor parameter for a
** tr1::function object can be passed in as predicate, but of course the
** std::function object can be passed in as predicate, but of course the
** signature must be sensible. Please note, that -- depending on the
** predicate -- already the ctor or even a simple \c bool test might
** predicate -- already the ctor or even a simple `bool` test might
** pull and exhaust the source iterator completely, in an attempt
** to find the first element passing the predicate test.
**
** \par extensible Filter
** Based on the FilterIter, this facility allows to elaborate the filter
** function while in the middle of iteration. The new augmented filter
** will be in effect starting with the current element, which might even
** be filtered away now due to a more restrictive condition. However,
** since this is still an iterator, any "past" elements are already
** extracted and gone and can thus not be subject to changed filtering.
** The ExtensibleFilterIter template provides several _builder functions_
** to elaborate the initial filter condition, like adding conjunctive or
** disjunctive clauses, flip the filter's meaning or even replace it
** altogether by a completely different filter function.
**
** \par processing Iterator
** the TransformIter template can be used as processing (or transforming)
** step within the pipeline. It is created with a functor, which, when
@ -55,11 +67,9 @@
** source iterator. The signature of the functor must match the
** desired value (output) type.
**
** @todo some more building blocks are planned, see Ticket #347
**
** @see iter-adapter.hpp
** @see itertools-test.cpp
** @see contents-query.hpp
** @see event-log.hpp
*/
@ -311,7 +321,7 @@ namespace lib {
: Raw{forward<IT>(source)}
, predicate_(prediDef) // induces a signature check
, cached_(false) // not yet cached
, isOK_() // some value
, isOK_(false) // not yet relevant
{ }
template<typename PRED>
@ -319,7 +329,7 @@ namespace lib {
: Raw{source}
, predicate_(prediDef)
, cached_(false)
, isOK_()
, isOK_(false)
{ }
};
@ -393,7 +403,7 @@ namespace lib {
* for the added clause.
* @warning the addition of disjunctive and negated clauses might
* actually weaken the filter condition. Yet still there is
* \em no reset of the source iterator, i.e. we don't
* _no reset of the source iterator,_ i.e. we don't
* re-evaluate from start, but just from current head.
* Which means we might miss elements in the already consumed
* part of the source sequence, which theoretically would
@ -643,8 +653,8 @@ namespace lib {
/**
* Iterator tool treating pulled data by a custom transformation (function)
* @param IT source iterator
* @param VAL result (output) type
* @tparam IT source iterator
* @tparam VAL result (output) type
*/
template<class IT, class VAL>
class TransformIter
@ -676,7 +686,7 @@ namespace lib {
/** Build a TransformIter: convenience free function shortcut,
* picking up the involved types automatically.
* @param processingFunc to be invoked for each source element
* @tparam processingFunc to be invoked for each source element
* @return Iterator processing the source feed
*/
template<class IT, typename FUN>

193
tests/library/iter-explorer-test.cpp
View file

@ -21,7 +21,32 @@
* *****************************************************/
/** @file iter-explorer-test.cpp
** unit test \ref IterExplorer_test
** The \ref IterExplorer_test covers and demonstrates several usage scenarios and
** extensions built on top of the \ref lib::IterExplorer template. These introduce some
** elements from Functional Programming, especially the _Monad Pattern_ to encapsulate
** and isolate intricacies of evolving embedded state. At usage site, only a _state
** transition function_ need to be provided, thereby focusing at the problem domain
** and thus reducing complexity.
**
** The setup for this test relies on a demonstration example of encapsulated state:
** a counter with start and end value, embedded into an iterator. Basically, this
** running counter, when iterated, generates a sequence of numbers start ... end.
** So -- conceptually -- this counting iterator can be thought to represent this
** sequence of numbers. Note that this is a kind of abstract or conceptual
** representation, not a factual representation of the sequence in memory.
** The whole point is _not to represent_ this sequence in runtime state at once,
** rather to pull and expand it on demand. Thus, all the examples demonstrate in
** this case "build" on this sequence, they expand it into various tree-like
** structures, without actually performing these structural operations in memory.
**
** All these tests work by first defining these _functional structures_, which just
** yields an iterator entity. We get the whole structure it conceptually defines
** only if we "pull" this iterator until exhaustion -- which is precisely what
** the test does to verify proper operation. Real world code of course would
** just not proceed in this way, like pulling everything from such an iterator.
** Often, the very reason we're using such a setup is the ability to represent
** infinite structures. Like e.g. the evaluation graph of video passed through
** a complex processing pipeline.
*/
@ -30,17 +55,16 @@
#include "lib/test/test-helper.hpp"
#include "lib/iter-adapter-stl.hpp"
#include "lib/format-cout.hpp"
#include "lib/format-util.hpp"
#include "lib/util.hpp"
#include "lib/iter-explorer.hpp"
#include "lib/meta/trait.hpp"
#include <boost/lexical_cast.hpp>
#include <sstream>
#include <utility>
#include <vector>
#include <string>
#include "lib/meta/trait.hpp"
namespace lib {
namespace test{
@ -48,16 +72,16 @@ namespace test{
using ::Test;
using util::isnil;
using util::isSameObject;
using std::string;
using lib::iter_stl::eachElm;
using lib::iter_explorer::ChainedIters;
using lumiera::error::LUMIERA_ERROR_ITER_EXHAUST;
using std::string;
namespace { // test substrate: simple number sequence iterator
/**
* This iteration "state core" type describes
* This iteration _"state core" type_ describes
* a sequence of numbers yet to be delivered.
*/
class State
@ -92,7 +116,7 @@ namespace test{
/**
/**
* A straight ascending number sequence as basic test iterator.
* The tests will dress up this source sequence in various ways.
*/
@ -111,25 +135,25 @@ namespace test{
/** allow using NumberSequence in LinkedElements
* (intrusive single linked list) */
NumberSequence* next;
NumberSequence* next =nullptr;
};
inline NumberSequence
seq (uint end)
{
return NumberSequence(end);
return NumberSequence(end);
}
inline NumberSequence
seq (uint start, uint end)
{
return NumberSequence(start, end);
return NumberSequence(start, end);
}
NumberSequence NIL_Sequence;
/**
/**
* an arbitrary series of numbers
* @note deliberately this is another type
* and not equivalent to a NumberSequence,
@ -138,7 +162,8 @@ namespace test{
typedef IterQueue<int> NumberSeries;
/** "exploration function" to generate a functional datastructure.
/**
* _"exploration function"_ to generate a functional datastructure.
* Divide the given number by 5, 3 and 2, if possible. Repeatedly
* applying this function yields a tree of decimation sequences,
* each leading down to 1
@ -159,15 +184,9 @@ namespace test{
*/
template<class II>
inline string
materialise (II ii)
materialise (II&& ii)
{
std::ostringstream buff;
while (ii)
{
buff << *ii;
if (++ii) buff << "-";
}
return buff.str();
return util::join (std::forward<II> (ii), "-");
}
template<class II>
@ -190,27 +209,28 @@ namespace test{
/*****************************************************************//**
/*******************************************************************//**
* @test use a simple source iterator yielding numbers
* to build various functional evaluation structures,
* based on the IterExplorer template.
* - the [state adapter](\ref verifyStateAdapter) iterator
* construction pattern
* - helper to [chain iterators](\ref verifyChainedIterators)
* - building [tree exploring structures](\ref verifyDepthFirstExploration)
* - the [monadic nature](\ref verifyMonadOperator) of IterExplorer
* - a [recursively self-integrating](\ref verifyRecrusiveSelfIntegration)
* based on the \ref IterExplorer template.
* - the [state adapter](\ref verifyStateAdapter() )
* iterator construction pattern
* - helper to [chain iterators](\ref verifyChainedIterators() )
* - building [tree exploring structures](\ref verifyDepthFirstExploration())
* - the [monadic nature](\ref verifyMonadOperator()) of IterExplorer
* - a [recursively self-integrating](\ref verifyRecrusiveSelfIntegration())
* evaluation pattern
*
* \par Explanation
* ## Explanation
*
* Both this test and the IterExplorer template might be bewildering
* and cryptic, unless you know the Monad design pattern. Monads are
* and cryptic, unless you know the *Monad design pattern*. Monads are
* heavily used in functional programming, actually they originate
* from Category Theory. Basically, Monad is a pattern where we
* combine several computation steps in a specific way; but instead
* of intermingling the individual computation steps and their
* combination, the goal is to separate and isolate the mechanics
* of combination, so we can focus on the actual computation steps:
* combination, the goal is to isolate and separate the _mechanics
* of combination_, so we can focus on the actual _computation steps:_
* The mechanics of combination are embedded into the Monad type,
* which acts as a kind of container, holding some entities
* to be processed. The actual processing steps are then
@ -250,16 +270,15 @@ namespace test{
/** @test demonstrate the underlying solution approach of IterExplorer.
* All of the following IterExplorer flavours are built on top of
* a special iterator adapter, centred at the notion of an iterable
* state element type. The actual iterator just embodies one element
* of this state representation, and typically this element alone holds
* all the relevant state and information. Essentially this means the
* iterator is self contained. Contrast this to the more conventional
* approach of iterator implementation, where the iterator entity actually
* maintains a hidden back-link to some kind of container, which in turn
* is the one in charge of the elements yielded by the iterator.
*
* All of the following IterExplorer flavours are built on top of a
* special iterator adapter, centred at the notion of an iterable state
* element type. The actual iterator just embodies one element of this
* state representation, and typically this element alone holds all the
* relevant state and information. Essentially this means the iterator is
* _self contained_. Contrast this to the more conventional approach of
* iterator implementation, where the iterator entity actually maintains
* a hidden back-link to some kind of container, which in turn is the one
* in charge of the elements yielded by the iterator.
*/
void
verifyStateAdapter ()
@ -288,14 +307,12 @@ namespace test{
/** @test verify a helper to chain a series of iterators into a
* "flat" result sequence. This convenience helper is built using
* IterExplorer building blocks.
* The resulting iterator \em combines and \em flattens a sequence
* of source iterators, resulting in a simple sequence accessible
* as iterator again. Here we verify the convenience / default
* implementation; it uses a STL container (actually std:deque)
* behind the scenes to keep track of all added source iterators.
/** @test verify a helper to chain a series of iterators into a "flat" result sequence.
* This convenience helper is built using IterExplorer building blocks. The resulting
* iterator _combines_ and _flattens_ a sequence of source iterators, resulting in a
* simple sequence accessible as iterator again. Here we verify the convenience
* implementation; this uses a STL container (actually std:deque) behind the scenes
* to keep track of all added source iterators.
*/
void
verifyChainedIterators ()
@ -314,7 +331,7 @@ namespace test{
CHECK (!iterChain (NIL_Sequence));
// Iterator chaining "flattens" one level of packaging
NumberSequence s9 = seq(9);
NumberSequence s9 = seq(9);
ci = iterChain (s9);
for ( ; s9 && ci; ++s9, ++ci )
@ -398,12 +415,12 @@ namespace test{
/** @test a depth-first visiting and exploration scheme
* of a tree like system, built on top of the IterExplorer monad.
/** @test a depth-first visiting and exploration scheme of a tree like system,
* built on top of the IterExplorer monad.
*
* \par Test data structure
* ## Test data structure
* We build a functional datastructure here, on the fly, while exploring it.
* The \c exploreChildren(m) function generates this tree like datastructure:
* The `exploreChildren(m)` function generates this tree like datastructure:
* For a given number, it tries to divide by 5, 3 and 2 respectively, possibly
* generating multiple decimation sequences.
*
@ -416,23 +433,23 @@ namespace test{
* \endcode
* This tree has no meaning in itself, beyond being an easy testbed for tree exploration schemes.
*
* \par How the exploration works
* We use a pre defined Template \link DepthFirstExplorer \endlink, which is built on top of IterExplorer.
* It contains the depth-first exploration strategy in a hardwired fashion. Actually this effect is achieved
* by defining a specific way how to \em combine the results of an \em exploration -- the latter being the
* function which generates the data structure. To yield a depth-first exploration, all we have to do
* is to delve down immediately into the children, right after visiting the node itself.
* ## How the exploration works
* We use a pre defined Template \ref DepthFirstExplorer, which is built on top of IterExplorer.
* It contains the depth-first exploration strategy in a hardwired fashion. Actually this effect is
* achieved by defining a specific way how to _combine the results of an exploration_ -- the latter
* being the function which generates the data structure. To yield a depth-first exploration, all we
* have to do is to delve down immediately into the children, right after visiting the node itself.
*
* Now, when creating such a DepthFirstExplorer by wrapping a given source iterator, the result is again
* an iterator, but a specific iterator which at the same time is a Monad: It supports the \c >>= operation
* (also known as \em bind operator or \em flatMap operator). This operator takes as second argument a
* function, which in our case is the function to generate or explore the data structure.
* an iterator, but a specific iterator which at the same time is a Monad: It supports the `>>=` operation
* (also known as _bind_ operator or _flatMap_ operator). This operator takes as second argument a function,
* which in our case is the function to generate or explore the data structure.
*
* The result of applying this \c >>= operation is a \em transformed version of the source iterator,
* The result of applying this monadic `>>=` operation is a _transformed_ version of the source iterator,
* i.e. it is again an iterator, which yields the results of the exploration function, combined together
* in the order as defined by the built-in exploration strategy (here: depth first)
*
* @note technical detail: the result type of the exploration function (here \c exploreChildren() ) determines
* @note technical detail: the result type of the exploration function (here `exploreChildren()`) determines
* the iterator type used within IterExplorer and to drive the evaluation. The source sequence used to
* seed the evaluation process can actually be any iterator yielding assignment compatible values: The
* second example uses a NumberSequence with unsigned int values 0..6, while the actual expansion and
@ -453,11 +470,11 @@ namespace test{
/** @test a breadth-first visiting and exploration scheme
* of a tree like system, built on top of the IterExplorer monad;
* here, an internal queue is used to explore the hierarchy in layers.
/** @test a breadth-first visiting and exploration scheme of a tree like system,
* built on top of the IterExplorer monad.
* Here, an internal queue is used to explore the hierarchy in layers.
* The (functional) datastructure is the same, just we're visiting it
* differently here (in rows or layers).
* in a different way here, namely in rows or layers.
*/
void
verifyBreadthFirstExploration ()
@ -471,20 +488,19 @@ namespace test{
/** @test verify a variation of recursive exploration, this time to rely
* directly on the result set iterator type to provide the re-integration
* of intermediary results. Since our \c exploreChildren() function returns
* of intermediary results. Since our `exploreChildren()` function returns
* a NumberSeries, which basically is a IterQueue, the re-integration of expanded
* elements will happen at the end, resulting in breadth-first visitation order --
* but contrary to the dedicated \c breadthFirst(..) explorer, this expansion is done
* but contrary to the dedicated `breadthFirst(..)` explorer, this expansion is done
* separately for each element in the initial seed sequence. Note for example how the
* expansion series for number 30, which is also generated in
* \link #verifyBreadthFirstExploration \endlink, appears here at the end of the
* explorationResult sequence
* expansion series for number 30, which is also generated in verifyBreadthFirstExploration(),
* appears here at the end of the explorationResult sequence
* @remarks this "combinator strategy" is really intended for use with custom sequences,
* where the "Explorer" function works together with a specific implementation
* and exploits knowledge about specifically tailored additional properties of
* the input sequence elements to yield the desired overall effect.
* Actually this is what we use in the proc::engine::Dispatcher to generate a
* series of frame render jobs, including all prerequisite jobs
* the input sequence elements, in order to yield the desired overall effect.
* Actually this is what we use in the proc::engine::Dispatcher to generate
* a series of frame render jobs, including all prerequisite jobs
*/
void
verifyRecursiveSelfIntegration ()
@ -501,14 +517,16 @@ namespace test{
/** @test cover the basic monad bind operator,
* which is used to build all the specialised Iterator flavours.
* The default implementation ("Combinator strategy") just joins and flattens
* the result sequences created by the functor bound into the monad. For this test,
* we use a functor \c explode(top), which returns the sequence 0...top.
* which is used to build all the specialised Iterator flavours.
* The default implementation ("Combinator strategy") just joins and flattens the result sequences
* created by the functor bound into the monad. For this test, we use a functor `explode(top)`,
* which returns the sequence 0...top.
*/
void
verifyMonadOperator ()
{
auto explode = [](uint top) { return seq(0,top); };
// IterExplorer as such is an iterator wrapping the source sequence
string result = materialise (exploreIter(seq(5)));
CHECK (result == "0-1-2-3-4");
@ -517,7 +535,7 @@ namespace test{
result = materialise (exploreIter(seq(5,6)));
CHECK (result == "5");
// then binding the explode()-Function yields just the result of invoking explode(5)
// then binding the explode()-Function yields just the result of invoking explode(5)
result = materialise (exploreIter(seq(5,6)) >>= explode);
CHECK (result == "0-1-2-3-4");
@ -550,13 +568,6 @@ namespace test{
// Note: when cascading multiple >>= the parentheses are necessary, since in C++ unfortunately
// the ">>=" associates to the right, while the proper monad bind operator should associate to the left
}
/** @internal exploration function used in ::verifyMonadOperator */
static NumberSequence
explode (uint top)
{
return seq(0,top);
}
};