From 3fc5a94b87005fe02bd79a7004d6de18a4643c4d Mon Sep 17 00:00:00 2001 From: Ichthyostega Date: Sun, 2 Sep 2018 16:01:07 +0200 Subject: [PATCH] TreeExplorer: investigate the backtracking abilities There is a bug or shortcoming in the existing ErrorLog matcher implementation. It is not really difficult to fix, however doing so would require us to intersperse yet another helper facility into the log matcher. And it occurred to me, that this helper would effectively re-implement the stack based backtracking ability, which is already present in TreeExplorer (and was created precisely to support this kind of recursive evaluation strategies). Thus I intend to switch the implementation of the EventLog matcher from the old IterTool framework to the newer TreeExplorer framework. And this intention made me re-read the code, fixing several comments and re-thinking the design --- src/lib/iter-tree-explorer.hpp | 6 +- tests/library/iter-tree-explorer-test.cpp | 82 +-- wiki/thinkPad.ichthyo.mm | 705 ++++++++++++++++------ 3 files changed, 557 insertions(+), 236 deletions(-) diff --git a/src/lib/iter-tree-explorer.hpp b/src/lib/iter-tree-explorer.hpp index 6aa6ea8a4..870747646 100644 --- a/src/lib/iter-tree-explorer.hpp +++ b/src/lib/iter-tree-explorer.hpp @@ -574,7 +574,7 @@ namespace lib { * yielding a compatible value type). Now, this _sequence of children_ effectively * replaces the expanded source element in the overall resulting sequence; which * means, the nested sequence was _flattened_ into the results. Since this expand() - * operation can again invoked on the results, the implementation of such an evaluation + * operation can again be invoked on the results, the implementation of such an evaluation * requires a stack datastructure, so the nested iterator from each expand() invocation * can be pushed to become the new active source for iteration. Thus the primary purpose * of this Expander (decorator) is to integrate those "nested child iterators" seamlessly @@ -583,7 +583,7 @@ namespace lib { * the source iterator wrapped by this decorator. * @remark since we allow a lot of leeway regarding the actual form and definition of the * _expansion functor_, there is a lot of minute technical details, mostly confined - * within the _BoundFunctorTraits. + * within the _BoundFunctor traits. * @tparam SRC the wrapped source iterator, typically a TreeExplorer or nested decorator. * @tparam FUN the concrete type of the functor passed. Will be dissected to find the signature */ @@ -1108,7 +1108,7 @@ namespace lib { /* ==== Builder functions ==== */ /** preconfigure this TreeExplorer to allow for _"expansion of children"_. - * The resulting iterator exposes an `expand()` function, which consumes + * The resulting iterator exposes an `expandChildren()` function, which consumes * the current head element of this iterator and feeds it through the * _expansion functor_, which was provided to this builder function here. * The _expansion functor_ is expected to yield a sequence of "child" elements, diff --git a/tests/library/iter-tree-explorer-test.cpp b/tests/library/iter-tree-explorer-test.cpp index 3425789c6..02c5d4021 100644 --- a/tests/library/iter-tree-explorer-test.cpp +++ b/tests/library/iter-tree-explorer-test.cpp @@ -218,42 +218,46 @@ namespace test{ /*******************************************************************//** * @test use a simple source iterator yielding numbers - * to build various functional evaluation structures, - * based on the \ref IterExplorer template. - * - the [state adapter](\ref verifyStateAdapter() ) - * iterator construction pattern - * - helper to [chain iterators](\ref verifyChainedIterators() ) - * - building [tree exploring structures](\ref verifyDepthFirstExploration()) - * - the [monadic nature](\ref verifyMonadOperator()) of IterExplorer - * - a [recursively self-integrating](\ref verifyRecrusiveSelfIntegration()) - * evaluation pattern + * to build various functional evaluation pipelines, + * based on the \ref TreeExplorer template. + * - the adapter to wrap the source, which can either + * [be a "state core"](\ref verify_wrappedState() ), or can + * [be a "Lumiera Forward Iterator"](\ref verify_wrappedIterator() ) + * - the defining use case for TreeExplorer is to build a + * [pipeline for depth-first exploration](\ref verify_expandOperation() ) + * of a (functional) tree structure. This "tree" is created by invoking + * a "expand functor", which can be defined in various ways. + * - the usual building blocks for functional evaluation pipelines, that is + * [filtering](\ref verify_FilterIterator() ) and + * [transforming](\ref verify_transformOperation() ) of + * the elements yielded by the wrapped source iterator. + * - building complex pipelines by combining the aforementioned building blocks + * - using an opaque source, hidden behind the IterSource interface, and + * an extension (sub interface) to allow for "tree exploration" without + * any knowledge regarding the concrete implementation of the data source. * * ## Explanation * - * Both this test and the IterExplorer template might be bewildering - * and cryptic, unless you know the *Monad design pattern*. »Monads« - * are heavily used in functional programming, actually they originate - * from Category Theory. Basically, Monad is a pattern where we combine - * several computation steps in a specific way; but instead of intermingling - * the individual computation steps and their combination, the goal is to - * isolate and separate the _mechanics of combination_, so we can focus on - * the actual _computation steps:_ The mechanics of combination are embedded - * into the Monad type, which acts as a kind of container, holding some entities - * to be processed. The actual processing steps are then attached to the monad as - * "function object" parameters. It is up to the monad to decide if, and when - * those processing steps are applied to the embedded values and how to combine - * the results into a new monad. + * These tests build a evaluation pipeline by _wrapping_ some kind of data source + * and then layering some evaluation stages on top. There are two motivations why + * one might want to build such a _filter pipeline:_ + * - on demand processing ("pull principle") + * - separation of source computation and "evaluation mechanics" + * when building complex search and backtracking algorithms. * - * Using the monad pattern is well suited when both the mechanics of - * combination and the individual computation steps tend to be complex. - * In such a situation, it is beneficial to develop and test both - * in isolation. The IterExplorer template applies this pattern - * to the task of processing a source sequence. Typically we use - * this in situations where we can't afford building elaborate - * data structures in (global) memory, but rather strive at - * doing everything on-the-fly. A typical example is the - * processing of a variably sized data set without - * using heap memory for intermediary results. + * This usage style is inspired from the *Monad design pattern*. In our case here, + * the Iterator pipeline would be the monad, and can be augmented and reshaped by + * attaching further processing steps. How those processing steps are to be applied + * remains an internal detail, defined by the processing pipeline. »Monads« are heavily + * used in functional programming, actually they originate from Category Theory. Basically, + * Monad is a pattern where we combine several computation steps in a specific way; but + * instead of intermingling the individual computation steps and their combination, + * the goal is to isolate and separate the _mechanics of combination_, so we can focus + * on the actual _computation steps:_ The mechanics of combination are embedded into the + * Monad type, which acts as a kind of container, holding some entities to be processed. + * The actual processing steps are then attached to the monad as "function object" parameters. + * It is up to the monad to decide if, and when those processing steps are applied to the + * embedded values and how to combine the results into a new monad. * * @see TreeExplorer * @see IterAdapter @@ -440,7 +444,7 @@ namespace test{ template void - verify_treeExpandingIterator(EXP ii) + verify_treeExpandingIterator (EXP ii) { CHECK (!isnil (ii)); CHECK (5 == *ii); @@ -620,7 +624,7 @@ namespace test{ * matching algorithms. Even more so, when other operations like filtering are intermingled; * in that case it might even happen that the downstream consumer does not even see the * items resulting from child expansion, because they are evaluated and then filtered - * away by a transformer and filter placed in between. + * away by transformers and filters placed in between. * @note as a consequence of the flexible automatic adapting of bound functors, it is * possible for bound functors within different "layers" to collaborate, based on * additional knowledge regarding the embedded data source internals. This test @@ -779,6 +783,16 @@ namespace test{ + // contrived example to verify interplay of filtering and child expansion; + // especially note that the filter is re-evaluated after expansion happened. + cout << "VERIFY->" + < - + @@ -5509,7 +5509,7 @@ - + @@ -5921,7 +5921,7 @@ - + @@ -5943,7 +5943,7 @@ - + @@ -5979,15 +5979,12 @@ - - + + - - - - + @@ -6110,7 +6107,7 @@ - + @@ -6139,7 +6136,7 @@ - + @@ -6171,7 +6168,7 @@ - + @@ -6337,7 +6334,7 @@ - + @@ -6351,16 +6348,8 @@

+ - - - - - - - - - @@ -6582,10 +6571,10 @@ - + @@ -9056,9 +9045,10 @@ - + + @@ -9067,7 +9057,7 @@ - + @@ -9076,7 +9066,7 @@ - + @@ -9092,7 +9082,7 @@ - + @@ -9111,8 +9101,8 @@ - - + + @@ -9219,7 +9209,7 @@ - + @@ -9331,7 +9321,7 @@ - + @@ -9373,7 +9363,7 @@ - + @@ -9472,7 +9462,7 @@ - + @@ -9681,7 +9671,7 @@ - + @@ -10109,7 +10099,7 @@ - + @@ -15311,7 +15301,7 @@ - + @@ -17293,7 +17283,7 @@ - + @@ -25101,6 +25091,45 @@ + + + + + + +

+ Beispiel: +

+

+ a-b-c-a +

+

+ match("a").after("b") scheitert, weil sich die Suche am ersten "a" festbeißt. +

+ + + + + + + + + + + + + + + + + + + + + + + + @@ -30432,6 +30461,436 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

+ Thema: Monaden +

+ + + + + + + + + + + +

+ sind Monaden +

+

+ wirklich hilfreich? +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

+ ...bindet die Betrachtung auf einen technischen Level, +

+

+ und führt dazu, daß die Abstraktion undicht wird +

+ + + + + + + + + + + + + + + +

+ genau der Umstand, +

+

+ daß funktionale Sprachen von einer Zustands-Phobie getrieben sind, +

+

+ macht Monaden nützlich, um inhärenten Zustand wegzuabstrahieren. +

+

+ Das kann genutzt werden, um den Zustand einer Wechselwirkung +

+

+ nach einer Seite der Glieder auszukoppeln. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

+ gehört zu dem Themenkomplex "itertools" +

+

+ Python hat das auch, Java neuerdings auch +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

+ ...will sagen, es ist klar, wie man sowas machen kann. +

+

+ Seinerzeit war mir das auch klar, aber ich wollte es nicht gleich ausprogrammieren. +

+

+ Inzwischen kam dann das Thema UI-Coordinaten, und dort habe ich es ausprogrammiert, +

+

+ und zwar direkt in die Low-Level-Schicht integriert, was nicht schlecht ist, +

+

+ da eine Abstraktion hier sehr technisch werden würde +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

+ ...von der entsprechenden Methode im Transformer +

+ + + + + + + + + + + +

+ ...hatte die Notwendigkeit hierfür seinerzeit während der Tests entdeckt, +

+

+ und im Unit-Test nur für die Kombination Transformer + Explorer abgedeckt.... +

+ + + + + + + + + + + + + + + + + + +
    +
  • + Suche wird geleitet durch die ViewSpecDSL +
    • +
  • + hinter dem (opaquen) TreeExplorer steckt die konkrete UI-Topologie +
    • +
+ + + + + + + + + + +

+ EventLog ist ein Test-Hilfsmittel, +

+

+ um Unit-Tests über UI-Bus-Interaktionen schreiben zu können. +

+

+ Es gibt hierzu Test-Varianten, die jeden Aufruf in ein internes Log notieren. +

+

+ Im Test verwendet man dann eine Pattern-Such-DSL, +

+

+ hinter der sich eine verkettete Suche mit Backtracking verbirgt +

+ + + + + + + + + + + + + + + + + + + + + + +

+ sofern längerfristig Itertools durch TreeExplorer abgelöst werden könnte +

+ + + + + + + + + + + + + + + + + +

+ Invariante: pullMatch() +

+ + + + + + + + + + + + + + + + + + + + @@ -33154,7 +33613,7 @@ - + @@ -33529,7 +33988,7 @@ - + @@ -34090,166 +34549,14 @@ - - - - - - - - -

- Thema: Monaden -

- - - - - - - - - - - - - - - - - - -

- gehört zu dem Themenkomplex "itertools" -

-

- Python hat das auch, Java neuerdings auch -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- ...will sagen, es ist klar, wie man sowas machen kann. -

-

- Seinerzeit war mir das auch klar, aber ich wollte es nicht gleich ausprogrammieren. -

-

- Inzwischen kam dann das Thema UI-Coordinaten, und dort habe ich es ausprogrammiert, -

-

- und zwar direkt in die Low-Level-Schicht integriert, was nicht schlecht ist, -

-

- da eine Abstraktion hier sehr technisch werden würde -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- ...bindet die Betrachtung auf einen technischen Level, -

-

- und führt dazu, daß die Abstraktion undicht wird -

- - - - - - - - - - - - - - - -

- genau der Umstand, -

-

- daß funktionale Sprachen von einer Zustands-Phobie getrieben sind, -

-

- macht Monaden nützlich, um inhärenten Zustand wegzuabstrahieren. -

-

- Das kann genutzt werden, um den Zustand einer Wechselwirkung -

-

- nach einer Seite der Glieder auszukoppeln. -

- - - - - - - - - - - - - - + + + + + + +