ichthyo/LUMIERA.clone
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
LUMIERA.clone/tests/lib/iter-zip-test.cpp

562 lines
26 KiB
C++
Raw Normal View History

Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
/*
IterZip(Test) - verify the iterator-combining iterator
Copyright (C)
2024, Hermann Vosseler <Ichthyostega@web.de>
  **Lumiera** is free software; you can redistribute it and/or modify it
  under the terms of the GNU General Public License as published by the
  Free Software Foundation; either version 2 of the License, or (at your
  option) any later version. See the file COPYING for further details.
* *****************************************************************/
/** @file iter-stack-test.cpp
** unit test \ref IterZip_test
*/
Tests: build an link a test-support library This is something I wanted to clean-up since a very long time indeed, but always postponed it, since it requires a lot of nasty clean-up and disentangling.... Several parts had been stashed away into `src/lib/test`, which are not really related: * the testrunner and testsuite builder * generic helper utilities * specialised test-support facilities, e.g. the EventLog This changeset introduces a new top-level test folder and a namespace `::test` Code from this folder is linked into a shared library `libtestsupport.so`, which resides in the linkage hierarchy between »Vessel« and »Vault«, yet is no longer part of the Lumiera application and only linked into the test-suite.
2026-03-06 21:34:20 +01:00
#include "test/run.hpp"
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
#include "lib/iter-zip.hpp"
Library: extend implementation to support references With this minor change, the internal result-tuple may now also hold references, in case a source iterator exposes a reference (which is in fact the standard case). Under the right circumstances, source-manipulation through the iterator becomes possible. Moreover, the optimiser should now be able to elide the result-value tuple in many cases. and access the iterator internals directly instead. Obviously this is an advanced and possibly dangerous feature, and only possible when no additional transformer functions are interspersed; moreover this prompted a review of some long standing type definitions to more precisely reflect the intention. Note: most deliberately, the Transformer element in IterExplorer must expose a reference type, and capture the results into an internal ItemWrapper. This is the only way we can support arbitrary functions.
2024-11-22 23:54:57 +01:00
#include "lib/iter-explorer.hpp"
Library: introduce a concept for the »Lumiera Forward Iterator« Well, finally. Lumiera Iterators were conceived as a concept from the very beginning, and this idea dates back even before C++11. Furthermore, over time some ''trait templates'' have been developed, to detect if some template argument might match the known pattern of functions associated with a Lumiera Iterator. Meanwhile this concept is used pervasively in the whole code base, and also the traits templates proved to be reliable. It therefore seems that the time is ripe to gradually introduce a genuine **concept** `lum_iter` I'll take a defensive approach here, since any mistake would lead to a massive breakage of compilation. The new concept is defined simply by delegating to to the tried-and-tested trait template, which is also renamed `can_IterForEach` ⟼ `can_LumieraIter` One immediate benefit is that the compiler is now able to select overloads based on that concept, which allows to drop the tricky selection of variants based on the SFINAE technique and `enable_if`. For validation, one very heavily used case of that pattern was switched in `lib/format-util.hpp', which notably lies in the code-path used by `util::join`
2026-05-13 23:44:48 +02:00
#include "lib/meta/tuple-helper.hpp"
Tests: build an link a test-support library This is something I wanted to clean-up since a very long time indeed, but always postponed it, since it requires a lot of nasty clean-up and disentangling.... Several parts had been stashed away into `src/lib/test`, which are not really related: * the testrunner and testsuite builder * generic helper utilities * specialised test-support facilities, e.g. the EventLog This changeset introduces a new top-level test folder and a namespace `::test` Code from this folder is linked into a shared library `libtestsupport.so`, which resides in the linkage hierarchy between »Vessel« and »Vault«, yet is no longer part of the Lumiera application and only linked into the test-suite.
2026-03-06 21:34:20 +01:00
#include "test/test-helper.hpp"
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
#include "lib/format-util.hpp"
#include "lib/util.hpp"
#include <array>
Library: extract the basic setup for a tuple-zipping iterator Indeed the solution worked out yesterday could be extracted and turned generic. Some in-depth testing is necessary though, and possibly some qualifications to allow pass-through of references... Moreover, last days I started collecting notes regarding problem solving patterns, which I tend to use frequently, but which might not be obvious and thus can easily be forgotten. In fact, I had encountered several cases, where I did invent some roughly similar solution repeatedly, having forgotten about already settled matters. Hopefully the habit of collecting notes and hints at a central location serves to remedy
2024-11-22 19:11:53 +01:00
#include <vector>
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
namespace lib {
namespace test{
Library: extend implementation to support references With this minor change, the internal result-tuple may now also hold references, in case a source iterator exposes a reference (which is in fact the standard case). Under the right circumstances, source-manipulation through the iterator becomes possible. Moreover, the optimiser should now be able to elide the result-value tuple in many cases. and access the iterator internals directly instead. Obviously this is an advanced and possibly dangerous feature, and only possible when no additional transformer functions are interspersed; moreover this prompted a review of some long standing type definitions to more precisely reflect the intention. Note: most deliberately, the Transformer element in IterExplorer must expose a reference type, and capture the results into an internal ItemWrapper. This is the only way we can support arbitrary functions.
2024-11-22 23:54:57 +01:00
using util::join;
Library: more aggressive testing with feature combinations and yes ... this revealed a **long standing bug** The `Filter::pullFilter()` invocation in the ctor may produce dangling refs, whenever an underlying source-iterator generates a reference that points into the iterator itself. The reason is: due to the »onion shell« design of the iterator pipeline, we are bound to move a source iterator into the next layer constructor.
2024-11-23 03:30:07 +01:00
using util::isnil;
Library: solve forwarding of child-expansion For sake of completeness, since the `IterExplorer` supports building extended search- and evaluation patterns, a tuple-zipping adapter can be expected to handle these extended usages transparently. While the idea is simple, making this actually happen had several ramifications and required to introduce additional flexibility within the adaptor-framework to cope better with those cases were some iterator must return a value, not a ref. In the end, this could be solved with a bit of metaprogramming based on `std::common_type` ...and indeed, this is all quite nasty stuff — in hindsight, my initial intuition to shy away from this topic was spot-on....
2024-11-26 02:56:28 +01:00
using util::noneg;
Library: extend implementation to support references With this minor change, the internal result-tuple may now also hold references, in case a source iterator exposes a reference (which is in fact the standard case). Under the right circumstances, source-manipulation through the iterator becomes possible. Moreover, the optimiser should now be able to elide the result-value tuple in many cases. and access the iterator internals directly instead. Obviously this is an advanced and possibly dangerous feature, and only possible when no additional transformer functions are interspersed; moreover this prompted a review of some long standing type definitions to more precisely reflect the intention. Note: most deliberately, the Transformer element in IterExplorer must expose a reference type, and capture the results into an internal ItemWrapper. This is the only way we can support arbitrary functions.
2024-11-22 23:54:57 +01:00
using LERR_(ITER_EXHAUST);
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
using lib::meta::forEach;
using lib::meta::mapEach;
using std::make_tuple;
using std::tuple;
using std::get;
namespace {// Test Fixture ...
auto num5() { return NumIter{0,5}; }
template<uint N, uint S=0>
auto numS() { return explore(num5()).transform([](int i){ return i*N + S; }); }
auto num31(){ return numS<3,1>(); }
auto num32(){ return numS<3,2>(); }
auto num33(){ return numS<3,3>(); }
auto hexed = [](int i){ return util::showHash(i,1); };
/** Diagnostic helper: join all the elements from the iterator */
template<class II>
inline string
materialise (II&& ii)
{
return util::join (std::forward<II> (ii), "-");
}
}
Tests: build an link a test-support library This is something I wanted to clean-up since a very long time indeed, but always postponed it, since it requires a lot of nasty clean-up and disentangling.... Several parts had been stashed away into `src/lib/test`, which are not really related: * the testrunner and testsuite builder * generic helper utilities * specialised test-support facilities, e.g. the EventLog This changeset introduces a new top-level test folder and a namespace `::test` Code from this folder is linked into a shared library `libtestsupport.so`, which resides in the linkage hierarchy between »Vessel« and »Vault«, yet is no longer part of the Lumiera application and only linked into the test-suite.
2026-03-06 21:34:20 +01:00
#define TYPE(_EXPR_) ::test::showType<decltype(_EXPR_)>()
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
/*********************************************************************************//**
* @test demonstrate construction and verify behaviour of a combined-iterator builder.
* - construction from arbitrary arguments by tuple-mapping a builder function
Library: more aggressive testing with feature combinations and yes ... this revealed a **long standing bug** The `Filter::pullFilter()` invocation in the ctor may produce dangling refs, whenever an underlying source-iterator generates a reference that points into the iterator itself. The reason is: due to the »onion shell« design of the iterator pipeline, we are bound to move a source iterator into the next layer constructor.
2024-11-23 03:30:07 +01:00
* - defining the operation on the product type by lifting individual operations
* - use the library building blocks to construct a zip-iter-builder
* - iterate a mix of source iterators and containers
* - apply additional processing logic by pipelining
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
* @see IterExplorer
* @see IterExplorer_test
*/
class IterZip_test : public Test
{
virtual void
run (Arg)
{
Library: extract the basic setup for a tuple-zipping iterator Indeed the solution worked out yesterday could be extracted and turned generic. Some in-depth testing is necessary though, and possibly some qualifications to allow pass-through of references... Moreover, last days I started collecting notes regarding problem solving patterns, which I tend to use frequently, but which might not be obvious and thus can easily be forgotten. In fact, I had encountered several cases, where I did invent some roughly similar solution repeatedly, having forgotten about already settled matters. Hopefully the habit of collecting notes and hints at a central location serves to remedy
2024-11-22 19:11:53 +01:00
simpleUsage();
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
test_Fixture();
demo_mapToTuple();
demo_construction();
Library: extend implementation to support references With this minor change, the internal result-tuple may now also hold references, in case a source iterator exposes a reference (which is in fact the standard case). Under the right circumstances, source-manipulation through the iterator becomes possible. Moreover, the optimiser should now be able to elide the result-value tuple in many cases. and access the iterator internals directly instead. Obviously this is an advanced and possibly dangerous feature, and only possible when no additional transformer functions are interspersed; moreover this prompted a review of some long standing type definitions to more precisely reflect the intention. Note: most deliberately, the Transformer element in IterExplorer must expose a reference type, and capture the results into an internal ItemWrapper. This is the only way we can support arbitrary functions.
2024-11-22 23:54:57 +01:00
verify_iteration();
verify_references();
Library: more aggressive testing with feature combinations and yes ... this revealed a **long standing bug** The `Filter::pullFilter()` invocation in the ctor may produce dangling refs, whenever an underlying source-iterator generates a reference that points into the iterator itself. The reason is: due to the »onion shell« design of the iterator pipeline, we are bound to move a source iterator into the next layer constructor.
2024-11-23 03:30:07 +01:00
verify_pipelining();
Library: solve forwarding of child-expansion For sake of completeness, since the `IterExplorer` supports building extended search- and evaluation patterns, a tuple-zipping adapter can be expected to handle these extended usages transparently. While the idea is simple, making this actually happen had several ramifications and required to introduce additional flexibility within the adaptor-framework to cope better with those cases were some iterator must return a value, not a ref. In the end, this could be solved with a bit of metaprogramming based on `std::common_type` ...and indeed, this is all quite nasty stuff — in hindsight, my initial intuition to shy away from this topic was spot-on....
2024-11-26 02:56:28 +01:00
verify_exploration();
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
}
Library: extract the basic setup for a tuple-zipping iterator Indeed the solution worked out yesterday could be extracted and turned generic. Some in-depth testing is necessary though, and possibly some qualifications to allow pass-through of references... Moreover, last days I started collecting notes regarding problem solving patterns, which I tend to use frequently, but which might not be obvious and thus can easily be forgotten. In fact, I had encountered several cases, where I did invent some roughly similar solution repeatedly, having forgotten about already settled matters. Hopefully the habit of collecting notes and hints at a central location serves to remedy
2024-11-22 19:11:53 +01:00
/** @test demonstrate combined iteration */
void
simpleUsage()
{
auto a = std::array{1u,2u,3u};
auto v = std::vector{{2l,3l}};
Library: extend implementation to support references With this minor change, the internal result-tuple may now also hold references, in case a source iterator exposes a reference (which is in fact the standard case). Under the right circumstances, source-manipulation through the iterator becomes possible. Moreover, the optimiser should now be able to elide the result-value tuple in many cases. and access the iterator internals directly instead. Obviously this is an advanced and possibly dangerous feature, and only possible when no additional transformer functions are interspersed; moreover this prompted a review of some long standing type definitions to more precisely reflect the intention. Note: most deliberately, the Transformer element in IterExplorer must expose a reference type, and capture the results into an internal ItemWrapper. This is the only way we can support arbitrary functions.
2024-11-22 23:54:57 +01:00
// loop over both in lockstep
for (auto [u,l] : zip(a,v))
CHECK (u + 1 == l);
Library: extract the basic setup for a tuple-zipping iterator Indeed the solution worked out yesterday could be extracted and turned generic. Some in-depth testing is necessary though, and possibly some qualifications to allow pass-through of references... Moreover, last days I started collecting notes regarding problem solving patterns, which I tend to use frequently, but which might not be obvious and thus can easily be forgotten. In fact, I had encountered several cases, where I did invent some roughly similar solution repeatedly, having forgotten about already settled matters. Hopefully the habit of collecting notes and hints at a central location serves to remedy
2024-11-22 19:11:53 +01:00
Library: extend implementation to support references With this minor change, the internal result-tuple may now also hold references, in case a source iterator exposes a reference (which is in fact the standard case). Under the right circumstances, source-manipulation through the iterator becomes possible. Moreover, the optimiser should now be able to elide the result-value tuple in many cases. and access the iterator internals directly instead. Obviously this is an advanced and possibly dangerous feature, and only possible when no additional transformer functions are interspersed; moreover this prompted a review of some long standing type definitions to more precisely reflect the intention. Note: most deliberately, the Transformer element in IterExplorer must expose a reference type, and capture the results into an internal ItemWrapper. This is the only way we can support arbitrary functions.
2024-11-22 23:54:57 +01:00
// iterate-with index
Library: extract the basic setup for a tuple-zipping iterator Indeed the solution worked out yesterday could be extracted and turned generic. Some in-depth testing is necessary though, and possibly some qualifications to allow pass-through of references... Moreover, last days I started collecting notes regarding problem solving patterns, which I tend to use frequently, but which might not be obvious and thus can easily be forgotten. In fact, I had encountered several cases, where I did invent some roughly similar solution repeatedly, having forgotten about already settled matters. Hopefully the habit of collecting notes and hints at a central location serves to remedy
2024-11-22 19:11:53 +01:00
auto it = izip(v);
CHECK (it);
Library: extend implementation to support references With this minor change, the internal result-tuple may now also hold references, in case a source iterator exposes a reference (which is in fact the standard case). Under the right circumstances, source-manipulation through the iterator becomes possible. Moreover, the optimiser should now be able to elide the result-value tuple in many cases. and access the iterator internals directly instead. Obviously this is an advanced and possibly dangerous feature, and only possible when no additional transformer functions are interspersed; moreover this prompted a review of some long standing type definitions to more precisely reflect the intention. Note: most deliberately, the Transformer element in IterExplorer must expose a reference type, and capture the results into an internal ItemWrapper. This is the only way we can support arbitrary functions.
2024-11-22 23:54:57 +01:00
CHECK (*it == "«tuple<ulong, long&>»──(0,2)"_expect );
Library: extract the basic setup for a tuple-zipping iterator Indeed the solution worked out yesterday could be extracted and turned generic. Some in-depth testing is necessary though, and possibly some qualifications to allow pass-through of references... Moreover, last days I started collecting notes regarding problem solving patterns, which I tend to use frequently, but which might not be obvious and thus can easily be forgotten. In fact, I had encountered several cases, where I did invent some roughly similar solution repeatedly, having forgotten about already settled matters. Hopefully the habit of collecting notes and hints at a central location serves to remedy
2024-11-22 19:11:53 +01:00
++it;
Library: extend implementation to support references With this minor change, the internal result-tuple may now also hold references, in case a source iterator exposes a reference (which is in fact the standard case). Under the right circumstances, source-manipulation through the iterator becomes possible. Moreover, the optimiser should now be able to elide the result-value tuple in many cases. and access the iterator internals directly instead. Obviously this is an advanced and possibly dangerous feature, and only possible when no additional transformer functions are interspersed; moreover this prompted a review of some long standing type definitions to more precisely reflect the intention. Note: most deliberately, the Transformer element in IterExplorer must expose a reference type, and capture the results into an internal ItemWrapper. This is the only way we can support arbitrary functions.
2024-11-22 23:54:57 +01:00
CHECK (*it == "«tuple<ulong, long&>»──(1,3)"_expect );
Library: extract the basic setup for a tuple-zipping iterator Indeed the solution worked out yesterday could be extracted and turned generic. Some in-depth testing is necessary though, and possibly some qualifications to allow pass-through of references... Moreover, last days I started collecting notes regarding problem solving patterns, which I tend to use frequently, but which might not be obvious and thus can easily be forgotten. In fact, I had encountered several cases, where I did invent some roughly similar solution repeatedly, having forgotten about already settled matters. Hopefully the habit of collecting notes and hints at a central location serves to remedy
2024-11-22 19:11:53 +01:00
CHECK (it);
++it;
CHECK (not it);
Library: extend implementation to support references With this minor change, the internal result-tuple may now also hold references, in case a source iterator exposes a reference (which is in fact the standard case). Under the right circumstances, source-manipulation through the iterator becomes possible. Moreover, the optimiser should now be able to elide the result-value tuple in many cases. and access the iterator internals directly instead. Obviously this is an advanced and possibly dangerous feature, and only possible when no additional transformer functions are interspersed; moreover this prompted a review of some long standing type definitions to more precisely reflect the intention. Note: most deliberately, the Transformer element in IterExplorer must expose a reference type, and capture the results into an internal ItemWrapper. This is the only way we can support arbitrary functions.
2024-11-22 23:54:57 +01:00
VERIFY_ERROR (ITER_EXHAUST, *it );
VERIFY_ERROR (ITER_EXHAUST, ++it );
Library: extract the basic setup for a tuple-zipping iterator Indeed the solution worked out yesterday could be extracted and turned generic. Some in-depth testing is necessary though, and possibly some qualifications to allow pass-through of references... Moreover, last days I started collecting notes regarding problem solving patterns, which I tend to use frequently, but which might not be obvious and thus can easily be forgotten. In fact, I had encountered several cases, where I did invent some roughly similar solution repeatedly, having forgotten about already settled matters. Hopefully the habit of collecting notes and hints at a central location serves to remedy
2024-11-22 19:11:53 +01:00
}
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
/** @test demonstrate how the test Fixture is used */
void
test_Fixture()
{
CHECK (materialise (num5() ) == "0-1-2-3-4"_expect);
CHECK (materialise (num31() ) == "1-4-7-10-13"_expect);
CHECK (materialise (num33() ) == "3-6-9-12-15"_expect);
CHECK (materialise (num32()
.transform(hexed)
) == "02-05-08-0B-0E"_expect);
}
/** @test demonstrate to apply a function to tuple contents */
void
demo_mapToTuple()
{
auto t1 = make_tuple (41u, 0.61803, '6');
CHECK (t1 == "«tuple<uint, double, char>»──(41,0.61803,6)"_expect );
auto t1f = mapEach (t1, [](auto v){ return v+1; });
CHECK (t1f == "«tuple<uint, double, int>»──(42,1.61803,55)"_expect ); // ASCII('6') ≙ 54 promoted to int
auto t1ff = mapEach (t1, [](auto& v){ v += 1; return v; });
CHECK (t1ff == "«tuple<uint, double, char>»──(42,1.61803,7)"_expect );
CHECK (t1f == "«tuple<uint, double, int>»──(42,1.61803,55)"_expect );
CHECK (t1 == "«tuple<uint, double, char>»──(42,1.61803,7)"_expect ); // src-tuple t1 affected by side-effect
// tuple may hold a reference....
Library: extract the basic setup for a tuple-zipping iterator Indeed the solution worked out yesterday could be extracted and turned generic. Some in-depth testing is necessary though, and possibly some qualifications to allow pass-through of references... Moreover, last days I started collecting notes regarding problem solving patterns, which I tend to use frequently, but which might not be obvious and thus can easily be forgotten. In fact, I had encountered several cases, where I did invent some roughly similar solution repeatedly, having forgotten about already settled matters. Hopefully the habit of collecting notes and hints at a central location serves to remedy
2024-11-22 19:11:53 +01:00
tuple<char, char&> t2{get<2>(t1), get<2>(t1ff)};
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
CHECK (t2 == "«tuple<char, char&>»──(7,7)"_expect );
auto t2f = mapEach (t2, [](auto& v){ v -= 1; return v; });
CHECK (t2f == "«tuple<char, char>»──(6,6)"_expect ); // function-result is value, thus res-tuple holds values
CHECK (t2 == "«tuple<char, char&>»──(6,6)"_expect); // ...but src-tuple t2 was affected by side-effect
CHECK (t1ff == "«tuple<uint, double, char>»──(42,1.61803,6)"_expect ); // ...which in turn holds a ref, so value in t1ff changed
CHECK (t1 == "«tuple<uint, double, char>»──(42,1.61803,7)"_expect ); // ...while the other one was picked by value => t1 unchanged
// function may return references....
Upgrade: allow for build on »Trixie« with GCC-14 * need to upgrade our custom packages to current standards * switch those packages from CDBS to dh * re-build on Trixie and upgrade the Lumiera DEB-Depot After these (in detail quite expensive) preparations, build with Scons and GCC-14 can be started. Fix some further (basically trivial) compile problems, uncovered by the improved type checking of modern compilers. Note: a tremendous amount of warnings (and depreciations) is also indicated, which will be addressed later....
2025-04-06 18:18:52 +02:00
auto refr = [](auto& v) -> decltype(auto) { return v; };
int five{5};
int& fiveR{five};
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
CHECK (TYPE (refr(five)) == "int&"_expect);
Upgrade: allow for build on »Trixie« with GCC-14 * need to upgrade our custom packages to current standards * switch those packages from CDBS to dh * re-build on Trixie and upgrade the Lumiera DEB-Depot After these (in detail quite expensive) preparations, build with Scons and GCC-14 can be started. Fix some further (basically trivial) compile problems, uncovered by the improved type checking of modern compilers. Note: a tremendous amount of warnings (and depreciations) is also indicated, which will be addressed later....
2025-04-06 18:18:52 +02:00
CHECK (TYPE (refr(fiveR)) == "int&"_expect);
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
auto t2r = mapEach (t2, refr);
CHECK (t2r == "«tuple<char&, char&>»──(6,6)"_expect ); // function yields references, which are placed into res-tuple
forEach (t2r, [](auto& v){ v +=23; });
Library: extract the basic setup for a tuple-zipping iterator Indeed the solution worked out yesterday could be extracted and turned generic. Some in-depth testing is necessary though, and possibly some qualifications to allow pass-through of references... Moreover, last days I started collecting notes regarding problem solving patterns, which I tend to use frequently, but which might not be obvious and thus can easily be forgotten. In fact, I had encountered several cases, where I did invent some roughly similar solution repeatedly, having forgotten about already settled matters. Hopefully the habit of collecting notes and hints at a central location serves to remedy
2024-11-22 19:11:53 +01:00
CHECK (t2r == "«tuple<char&, char&>»──(M,M)"_expect ); // apply operation with side-effect to the last res-tuple t2r
CHECK (t2 == "«tuple<char, char&>»──(M,M)"_expect ); // the referred src-tuple t2 is also affected
CHECK (t2f == "«tuple<char, char>»──(6,6)"_expect ); // (while previously constructed t2f holds values unaffected)
CHECK (t1 == "«tuple<uint, double, char>»──(42,1.61803,7)"_expect ); // the first elm in t2 was bound by value, so no side-effect
CHECK (t1ff == "«tuple<uint, double, char>»──(42,1.61803,M)"_expect ); // but the second elm in t2 was bound by ref to t1ff
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
}
template<typename...ITS>
auto
buildIterTuple (ITS&& ...iters)
{
return make_tuple (lib::explore (std::forward<ITS> (iters)) ...);
}
/** @test demonstrate how a tuple-zipping iterator can be constructed */
void
demo_construction()
{
// let's start with the basics...
// We can use lib::explore() to construct a suitable iterator,
// and thus we can apply it to each var-arg and place the results into a tuple
auto arry = std::array{3u,2u,1u};
auto iTup = buildIterTuple (num5(), arry);
CHECK (TYPE(iTup) == "tuple<IterExplorer<iter_explorer::BaseAdapter<NumIter<int> > >, "
"IterExplorer<iter_explorer::BaseAdapter<iter_explorer::StlRange<array<uint, 3ul>&> > > >"_expect);
// and we can use them as iterators...
auto iterate_it = [](auto& it){ ++it; };
auto access_val = [](auto& it){ return *it; };
forEach (iTup, iterate_it);
auto vTup = mapEach (iTup, access_val);
CHECK (vTup == "«tuple<int, uint>»──(1,2)"_expect);
using ITup = decltype(iTup);
// Next step: define a »product iterator«
// by mapping down each of the base operations onto the tuple elements
struct ProductCore
{
ITup iters_;
ProductCore(ITup&& iterTup)
: iters_{move (iterTup)}
{ }
/* === »state core« protocol API === */
bool
checkPoint() const
{
bool active{true}; // note: optimiser can unroll this
forEach (iters_, [&](auto& it){ active = active and bool(it); });
return active;
}
ITup&
yield() const
{
Library: extract the basic setup for a tuple-zipping iterator Indeed the solution worked out yesterday could be extracted and turned generic. Some in-depth testing is necessary though, and possibly some qualifications to allow pass-through of references... Moreover, last days I started collecting notes regarding problem solving patterns, which I tend to use frequently, but which might not be obvious and thus can easily be forgotten. In fact, I had encountered several cases, where I did invent some roughly similar solution repeatedly, having forgotten about already settled matters. Hopefully the habit of collecting notes and hints at a central location serves to remedy
2024-11-22 19:11:53 +01:00
return unConst(iters_); // ◁─────────────── note: we expose the iterator-tuple itself as »product«
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
}
void
iterNext()
{
forEach (iters_, [](auto& it){ ++it; });
}
};
// ....and now we're essentially set!
// use library building blocks to construct a tuple-iter-explorer...
auto ii = explore (ProductCore{buildIterTuple (num5(), arry)})
.transform ([&](ITup& iTup){ return mapEach (iTup, access_val); })
;
Library: simplify state-core wrapper parameters As follow-up from the preceding refactorings, it is now possible to drastically simplify several type signatures. Generally speaking, iterator pipelines can now pass-through the result type, and thus it is no longer necessary to handle this result type explicitly In the case of `IterStateWrapper`, the result type parameter was retained, but moved to the second position and defaulted; sometimes it can be relevant to force a specific type; this is especially useful when defining an `iterator` and a `const_iterator` based on the same »state-core«
2024-11-26 22:15:33 +01:00
// demonstrate the composed pipeline type...
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
CHECK (TYPE(ii) == "IterExplorer<"
"IterableDecorator<"
"CheckedCore<"
"iter_explorer::Transformer<" // ◁──────────────────────────────── the top-layer is a Transformer (to access the value from each src-iter)
"iter_explorer::BaseAdapter<"
"IterableDecorator<" // ◁──────────────────────────── the product-iterator we constructed
"CheckedCore<" // ◁──────────────────────────── ....and using the given ProductCore as »state core«
"IterZip_test::demo_construction()::ProductCore> > >, "
"tuple<int, uint> " // ◁──────────────────────────────── back to top-layer: result-type of the Transformer
"> "
"> "
"> "
">"_expect);
// ....
// This is indeed a valid iterator,
// that can be iterated for three steps
// (limited by the shorter sequence from the array)
// (first value from num5(), second from the array)
CHECK (materialise (ii) == "«tuple<int, uint>»──(0,3)-"
"«tuple<int, uint>»──(1,2)-"
"«tuple<int, uint>»──(2,1)"_expect);
}
Library: extend implementation to support references With this minor change, the internal result-tuple may now also hold references, in case a source iterator exposes a reference (which is in fact the standard case). Under the right circumstances, source-manipulation through the iterator becomes possible. Moreover, the optimiser should now be able to elide the result-value tuple in many cases. and access the iterator internals directly instead. Obviously this is an advanced and possibly dangerous feature, and only possible when no additional transformer functions are interspersed; moreover this prompted a review of some long standing type definitions to more precisely reflect the intention. Note: most deliberately, the Transformer element in IterExplorer must expose a reference type, and capture the results into an internal ItemWrapper. This is the only way we can support arbitrary functions.
2024-11-22 23:54:57 +01:00
/** @test create various product (tuple) iterators
* from mixed source iterators and verify basic iteration.
*/
void
verify_iteration()
{
CHECK (materialise (
zip (num31(), num32(), num33())
)
== "«tuple<uint&, uint&, uint&>»──(1,2,3)-"
"«tuple<uint&, uint&, uint&>»──(4,5,6)-"
"«tuple<uint&, uint&, uint&>»──(7,8,9)-"
"«tuple<uint&, uint&, uint&>»──(10,11,12)-"
"«tuple<uint&, uint&, uint&>»──(13,14,15)"_expect);
CHECK (materialise(
izip (num31(), num32(), num33())
)
== "«tuple<ulong, uint&, uint&, uint&>»──(0,1,2,3)-"
"«tuple<ulong, uint&, uint&, uint&>»──(1,4,5,6)-"
"«tuple<ulong, uint&, uint&, uint&>»──(2,7,8,9)-"
"«tuple<ulong, uint&, uint&, uint&>»──(3,10,11,12)-"
"«tuple<ulong, uint&, uint&, uint&>»──(4,13,14,15)"_expect);
Library: more aggressive testing with feature combinations and yes ... this revealed a **long standing bug** The `Filter::pullFilter()` invocation in the ctor may produce dangling refs, whenever an underlying source-iterator generates a reference that points into the iterator itself. The reason is: due to the »onion shell« design of the iterator pipeline, we are bound to move a source iterator into the next layer constructor.
2024-11-23 03:30:07 +01:00
auto s6 = std::array{1,1,2,3,5,8};
auto s3 = {3,2,1};
auto s0 = eachNum(5u,5u);
CHECK (TYPE(s6) == "array<int, 6ul>"_expect );
CHECK (TYPE(s3) == "initializer_list<int>"_expect );
CHECK (TYPE(s0) == "NumIter<uint>"_expect );
CHECK (materialise (
zip (s6,s6,s6,eachNum('a'))
)
== "«tuple<int&, int&, int&, char>»──(1,1,1,a)-"
"«tuple<int&, int&, int&, char>»──(1,1,1,b)-"
"«tuple<int&, int&, int&, char>»──(2,2,2,c)-"
"«tuple<int&, int&, int&, char>»──(3,3,3,d)-"
"«tuple<int&, int&, int&, char>»──(5,5,5,e)-"
"«tuple<int&, int&, int&, char>»──(8,8,8,f)"_expect);
CHECK (materialise (
zip (s6,s3,s6,eachNum('a'))
)
== "«tuple<int&, int const&, int&, char>»──(1,3,1,a)-"
"«tuple<int&, int const&, int&, char>»──(1,2,1,b)-"
"«tuple<int&, int const&, int&, char>»──(2,1,2,c)"_expect);
CHECK (isnil (s0));
CHECK (materialise (
zip (s0,s3,s6,eachNum('a'))
)
== ""_expect);
CHECK (materialise (
zip (eachNum('a'),eachNum(-1),s0,s0)
)
== ""_expect);
CHECK (materialise (
zip (eachNum('a'),eachNum(-1),s3,s0)
)
== ""_expect);
CHECK (materialise (
zip (eachNum('a'),eachNum(-1),s3,s3)
)
== "«tuple<char, int, int const&, int const&>»──(a,-1,3,3)-"
"«tuple<char, int, int const&, int const&>»──(b,0,2,2)-"
"«tuple<char, int, int const&, int const&>»──(c,1,1,1)"_expect);
// a wild mix of data sources,
// including infinite and virtual ones....
CHECK (materialise (
izip (s6 // a STL container given by reference
,explore(s6).filter([](int i){ return i%2; }) // IterExplorer pipeline with filtering
,numS<17,170>().transform(hexed) // IterExplorer pipeline with transformer and object value result
,eachNum((1+sqrt(5))/2) // a Lumiera iterator which happens to be almost inexhaustible
,explore(s3).asIterSource() // an IterSource, which is a virtual (OO) iterator interface
)
)
== "«tuple<ulong, int&, int&, string&, double, int const&>»──(0,1,1,AA,1.618034,3)-"
"«tuple<ulong, int&, int&, string&, double, int const&>»──(1,1,1,BB,2.618034,2)-"
"«tuple<ulong, int&, int&, string&, double, int const&>»──(2,2,3,CC,3.618034,1)"_expect);
Library: extend implementation to support references With this minor change, the internal result-tuple may now also hold references, in case a source iterator exposes a reference (which is in fact the standard case). Under the right circumstances, source-manipulation through the iterator becomes possible. Moreover, the optimiser should now be able to elide the result-value tuple in many cases. and access the iterator internals directly instead. Obviously this is an advanced and possibly dangerous feature, and only possible when no additional transformer functions are interspersed; moreover this prompted a review of some long standing type definitions to more precisely reflect the intention. Note: most deliberately, the Transformer element in IterExplorer must expose a reference type, and capture the results into an internal ItemWrapper. This is the only way we can support arbitrary functions.
2024-11-22 23:54:57 +01:00
}
Library: more aggressive testing with feature combinations and yes ... this revealed a **long standing bug** The `Filter::pullFilter()` invocation in the ctor may produce dangling refs, whenever an underlying source-iterator generates a reference that points into the iterator itself. The reason is: due to the »onion shell« design of the iterator pipeline, we are bound to move a source iterator into the next layer constructor.
2024-11-23 03:30:07 +01:00
Library: extend implementation to support references With this minor change, the internal result-tuple may now also hold references, in case a source iterator exposes a reference (which is in fact the standard case). Under the right circumstances, source-manipulation through the iterator becomes possible. Moreover, the optimiser should now be able to elide the result-value tuple in many cases. and access the iterator internals directly instead. Obviously this is an advanced and possibly dangerous feature, and only possible when no additional transformer functions are interspersed; moreover this prompted a review of some long standing type definitions to more precisely reflect the intention. Note: most deliberately, the Transformer element in IterExplorer must expose a reference type, and capture the results into an internal ItemWrapper. This is the only way we can support arbitrary functions.
2024-11-22 23:54:57 +01:00
/** @test verify pass-through of references */
void
verify_references()
{
auto vec = std::vector{1,5};
auto arr = std::array{2,3};
Library: more aggressive testing with feature combinations and yes ... this revealed a **long standing bug** The `Filter::pullFilter()` invocation in the ctor may produce dangling refs, whenever an underlying source-iterator generates a reference that points into the iterator itself. The reason is: due to the »onion shell« design of the iterator pipeline, we are bound to move a source iterator into the next layer constructor.
2024-11-23 03:30:07 +01:00
// Case-1 ------
Library: extend implementation to support references With this minor change, the internal result-tuple may now also hold references, in case a source iterator exposes a reference (which is in fact the standard case). Under the right circumstances, source-manipulation through the iterator becomes possible. Moreover, the optimiser should now be able to elide the result-value tuple in many cases. and access the iterator internals directly instead. Obviously this is an advanced and possibly dangerous feature, and only possible when no additional transformer functions are interspersed; moreover this prompted a review of some long standing type definitions to more precisely reflect the intention. Note: most deliberately, the Transformer element in IterExplorer must expose a reference type, and capture the results into an internal ItemWrapper. This is the only way we can support arbitrary functions.
2024-11-22 23:54:57 +01:00
auto i1 = izip (vec,arr);
CHECK (*i1 == "«tuple<ulong, int&, int&>»──(0,1,2)"_expect ); // initial state points to the first elements, prefixed with index≡0
get<1>(*i1) = 5; // manipulate through the exposed reference
CHECK (*i1 == "«tuple<ulong, int&, int&>»──(0,5,2)"_expect ); // effect of manipulation is visible
CHECK (join(vec) == "5, 5"_expect ); // manipulation indeed flipped the first element in the vector
CHECK (join(arr) == "2, 3"_expect ); // (while the array remains unaffected)
Library: more aggressive testing with feature combinations and yes ... this revealed a **long standing bug** The `Filter::pullFilter()` invocation in the ctor may produce dangling refs, whenever an underlying source-iterator generates a reference that points into the iterator itself. The reason is: due to the »onion shell« design of the iterator pipeline, we are bound to move a source iterator into the next layer constructor.
2024-11-23 03:30:07 +01:00
// Case-2 ------
Library: extend implementation to support references With this minor change, the internal result-tuple may now also hold references, in case a source iterator exposes a reference (which is in fact the standard case). Under the right circumstances, source-manipulation through the iterator becomes possible. Moreover, the optimiser should now be able to elide the result-value tuple in many cases. and access the iterator internals directly instead. Obviously this is an advanced and possibly dangerous feature, and only possible when no additional transformer functions are interspersed; moreover this prompted a review of some long standing type definitions to more precisely reflect the intention. Note: most deliberately, the Transformer element in IterExplorer must expose a reference type, and capture the results into an internal ItemWrapper. This is the only way we can support arbitrary functions.
2024-11-22 23:54:57 +01:00
auto i2 = izip (explore(vec).transform([](uint v){ return v-1; }) // this time the first iterator is a pipeline with a transformer
,arr); // while the second one is again a direct iteration of the array
CHECK (*i2 == "«tuple<ulong, uint&, int&>»──(0,4,2)"_expect ); // again can see the first elements, and the effect of the transformer
get<0>(*i2) = 9; // manipulate complete result tuple
get<1>(*i2) = 9;
get<2>(*i2) = 9;
CHECK (*i2 == "«tuple<ulong, uint&, int&>»──(9,9,9)"_expect ); // effect of the manipulation is visible
++i2; // ...but iteration re-uses the internal result-tuple storage
CHECK (*i2 == "«tuple<ulong, uint&, int&>»──(1,4,3)"_expect ); // and so the effect of the manipulation seems gone
CHECK (join(vec) == "5, 5"_expect ); // ...which is in fact true for the vector, due to the transformer
CHECK (join(arr) == "9, 3"_expect ); // ...while the array could be reached through the reference
}
Library: more aggressive testing with feature combinations and yes ... this revealed a **long standing bug** The `Filter::pullFilter()` invocation in the ctor may produce dangling refs, whenever an underlying source-iterator generates a reference that points into the iterator itself. The reason is: due to the »onion shell« design of the iterator pipeline, we are bound to move a source iterator into the next layer constructor.
2024-11-23 03:30:07 +01:00
/** @test the result is actually an IterExplorer pipeline builder,
Library: further test and documentation of tuple-zipping
2024-11-26 17:35:05 +01:00
* which can be used to attach further processing downstream.
Library: handle chaining of iterator-pipelines This involves some quite tricky changes in the way types are composed to form an iterator-pipeline. Some wrappers are added as adaptors or for additional safety-checks, and to provide a builder-API. Unfortunately, when building a new `IterExplorer` iterator pipeline from an existing pipeline naively, composing all those types will add several unecessary intermediary wrapper-layers. Worse even, the handling of `BaseAdapter` prevents the new tuple-zipping iterator actually to pass-through any `expandChildren()` call. These issues are a consequence of using templated types, instead of fixed types with an interface; we can not just determine if some wrapper is present — unless the wrapper itself ''helps by exposing a tag.'' Even while I must admit that the whole packaging and adaptation machinery of `IterExplorer` looks dangerously complex already, using dedicated type tags for this single purpose seems like a tenable soulution.
2024-11-24 19:53:07 +01:00
* @note the design of IterExplorer inherently requires that
* generic lambdas accept the _iterator type_ by reference;
* structural bindings can only be used in a second step.
Library: more aggressive testing with feature combinations and yes ... this revealed a **long standing bug** The `Filter::pullFilter()` invocation in the ctor may produce dangling refs, whenever an underlying source-iterator generates a reference that points into the iterator itself. The reason is: due to the »onion shell« design of the iterator pipeline, we are bound to move a source iterator into the next layer constructor.
2024-11-23 03:30:07 +01:00
*/
void
verify_pipelining()
{
Library: further test and documentation of tuple-zipping
2024-11-26 17:35:05 +01:00
// for reference: this is the base data.......
CHECK (materialise (
zip (num31(), num32(), num33())
)
== "«tuple<uint&, uint&, uint&>»──(1,2,3)-"
"«tuple<uint&, uint&, uint&>»──(4,5,6)-"
"«tuple<uint&, uint&, uint&>»──(7,8,9)-"
"«tuple<uint&, uint&, uint&>»──(10,11,12)-"
"«tuple<uint&, uint&, uint&>»──(13,14,15)"_expect);
Library: handle chaining of iterator-pipelines This involves some quite tricky changes in the way types are composed to form an iterator-pipeline. Some wrappers are added as adaptors or for additional safety-checks, and to provide a builder-API. Unfortunately, when building a new `IterExplorer` iterator pipeline from an existing pipeline naively, composing all those types will add several unecessary intermediary wrapper-layers. Worse even, the handling of `BaseAdapter` prevents the new tuple-zipping iterator actually to pass-through any `expandChildren()` call. These issues are a consequence of using templated types, instead of fixed types with an interface; we can not just determine if some wrapper is present — unless the wrapper itself ''helps by exposing a tag.'' Even while I must admit that the whole packaging and adaptation machinery of `IterExplorer` looks dangerously complex already, using dedicated type tags for this single purpose seems like a tenable soulution.
2024-11-24 19:53:07 +01:00
// transform the tuple into another data value
CHECK (materialise (
Library: more aggressive testing with feature combinations and yes ... this revealed a **long standing bug** The `Filter::pullFilter()` invocation in the ctor may produce dangling refs, whenever an underlying source-iterator generates a reference that points into the iterator itself. The reason is: due to the »onion shell« design of the iterator pipeline, we are bound to move a source iterator into the next layer constructor.
2024-11-23 03:30:07 +01:00
zip (num31(), num32(), num33())
. transform([](auto& it){ auto [a,b,c] = *it;
return a+b+c;
})
)
Library: handle chaining of iterator-pipelines This involves some quite tricky changes in the way types are composed to form an iterator-pipeline. Some wrappers are added as adaptors or for additional safety-checks, and to provide a builder-API. Unfortunately, when building a new `IterExplorer` iterator pipeline from an existing pipeline naively, composing all those types will add several unecessary intermediary wrapper-layers. Worse even, the handling of `BaseAdapter` prevents the new tuple-zipping iterator actually to pass-through any `expandChildren()` call. These issues are a consequence of using templated types, instead of fixed types with an interface; we can not just determine if some wrapper is present — unless the wrapper itself ''helps by exposing a tag.'' Even while I must admit that the whole packaging and adaptation machinery of `IterExplorer` looks dangerously complex already, using dedicated type tags for this single purpose seems like a tenable soulution.
2024-11-24 19:53:07 +01:00
== "6-15-24-33-42"_expect);
// filter tuples based on inspecting contents
CHECK (materialise (
Library: more aggressive testing with feature combinations and yes ... this revealed a **long standing bug** The `Filter::pullFilter()` invocation in the ctor may produce dangling refs, whenever an underlying source-iterator generates a reference that points into the iterator itself. The reason is: due to the »onion shell« design of the iterator pipeline, we are bound to move a source iterator into the next layer constructor.
2024-11-23 03:30:07 +01:00
zip (num31(), num32(), num33())
. filter ([](auto& it){ auto [a,b,c] = *it;
return not ((a+b+c) % 2);
})
)
Library: handle chaining of iterator-pipelines This involves some quite tricky changes in the way types are composed to form an iterator-pipeline. Some wrappers are added as adaptors or for additional safety-checks, and to provide a builder-API. Unfortunately, when building a new `IterExplorer` iterator pipeline from an existing pipeline naively, composing all those types will add several unecessary intermediary wrapper-layers. Worse even, the handling of `BaseAdapter` prevents the new tuple-zipping iterator actually to pass-through any `expandChildren()` call. These issues are a consequence of using templated types, instead of fixed types with an interface; we can not just determine if some wrapper is present — unless the wrapper itself ''helps by exposing a tag.'' Even while I must admit that the whole packaging and adaptation machinery of `IterExplorer` looks dangerously complex already, using dedicated type tags for this single purpose seems like a tenable soulution.
2024-11-24 19:53:07 +01:00
== "«tuple<uint&, uint&, uint&>»──(1,2,3)-"
"«tuple<uint&, uint&, uint&>»──(7,8,9)-"
"«tuple<uint&, uint&, uint&>»──(13,14,15)"_expect);
// reduce with accessor and std::plus
CHECK (zip (num31(), num32(), num33())
. reduce ([](auto& it){ auto [a,b,c] = *it;
return a+b+c;
})
== 6+15+24+33+42);
Library: solve forwarding of child-expansion For sake of completeness, since the `IterExplorer` supports building extended search- and evaluation patterns, a tuple-zipping adapter can be expected to handle these extended usages transparently. While the idea is simple, making this actually happen had several ramifications and required to introduce additional flexibility within the adaptor-framework to cope better with those cases were some iterator must return a value, not a ref. In the end, this could be solved with a bit of metaprogramming based on `std::common_type` ...and indeed, this is all quite nasty stuff — in hindsight, my initial intuition to shy away from this topic was spot-on....
2024-11-26 02:56:28 +01:00
}
Library: further test and documentation of tuple-zipping
2024-11-26 17:35:05 +01:00
Library: solve forwarding of child-expansion For sake of completeness, since the `IterExplorer` supports building extended search- and evaluation patterns, a tuple-zipping adapter can be expected to handle these extended usages transparently. While the idea is simple, making this actually happen had several ramifications and required to introduce additional flexibility within the adaptor-framework to cope better with those cases were some iterator must return a value, not a ref. In the end, this could be solved with a bit of metaprogramming based on `std::common_type` ...and indeed, this is all quite nasty stuff — in hindsight, my initial intuition to shy away from this topic was spot-on....
2024-11-26 02:56:28 +01:00
/** @test verify the interplay of _child expansion_ and tuple-zipping.
* @remark the expansion mechanism implies that a _child sequence_ is generated
* by an _expand functor,_ based on the current iterator value at that point.
* The tricky part here is that this expand functor can sit somewhere in the
* source iterators, while the actual signal to expand is sent from »downstream«
* and has to be propagated to all children.
* Thus two expander-setups are demonstrated first, and then triggered from
* a combined iterator, dispatching the trigger over the tuple-zipping step.
* - the expansion-sequences unfold the same in each case
* - the shortest sequence terminates the overall zip()-evaluation
* - when generating the `expandChildrem()` call _after_ the `zip()`,
* it is also passed to other iterators that have no expand-functor defined;
* for those, it is absorbed without effect. Now, since the expandAll()
* actually works by replacing the iterate() by expandChildern(), this means
* that the _other sequences_ just do not make any progress.
*/
void
verify_exploration()
{
CHECK (materialise (
num31()
)
== "1-4-7-10-13"_expect);
CHECK (materialise (
explore(num31())
.expand ([](int i){ return NumIter{noneg(i-1),i}; })
.expandAll()
)
== "1-0-4-3-2-1-0-7-6-5-4-3-2-1-0-10-9-8-7-6-5-4-3-2-1-0-13-12-11-10-9-8-7-6-5-4-3-2-1-0"_expect);
CHECK (materialise (
explore(num31())
.expand ([](int i){ return NumIter{noneg(i-2),i-1}; })
.expandAll()
)
== "1-4-2-0-7-5-3-1-10-8-6-4-2-0-13-11-9-7-5-3-1"_expect);
CHECK (materialise (
zip
( eachNum(10)
, explore(num31())
.expand ([](int i){ return NumIter{noneg(i-1),i}; })
Library: further test and documentation of tuple-zipping
2024-11-26 17:35:05 +01:00
.expandAll() // ◁────────────────────────────────────────────── expand triggered in source pipeline, before the zip()
Library: solve forwarding of child-expansion For sake of completeness, since the `IterExplorer` supports building extended search- and evaluation patterns, a tuple-zipping adapter can be expected to handle these extended usages transparently. While the idea is simple, making this actually happen had several ramifications and required to introduce additional flexibility within the adaptor-framework to cope better with those cases were some iterator must return a value, not a ref. In the end, this could be solved with a bit of metaprogramming based on `std::common_type` ...and indeed, this is all quite nasty stuff — in hindsight, my initial intuition to shy away from this topic was spot-on....
2024-11-26 02:56:28 +01:00
, explore(num31())
.expand ([](int i){ return NumIter{noneg(i-2),i-1}; })
.expandAll()
)
)
== "«tuple<int, uint, uint>»──(10,1,1)-"
"«tuple<int, uint, uint>»──(11,0,4)-"
"«tuple<int, uint, uint>»──(12,4,2)-"
"«tuple<int, uint, uint>»──(13,3,0)-"
"«tuple<int, uint, uint>»──(14,2,7)-"
"«tuple<int, uint, uint>»──(15,1,5)-"
"«tuple<int, uint, uint>»──(16,0,3)-"
"«tuple<int, uint, uint>»──(17,7,1)-"
"«tuple<int, uint, uint>»──(18,6,10)-"
"«tuple<int, uint, uint>»──(19,5,8)-"
"«tuple<int, uint, uint>»──(20,4,6)-"
"«tuple<int, uint, uint>»──(21,3,4)-"
"«tuple<int, uint, uint>»──(22,2,2)-"
"«tuple<int, uint, uint>»──(23,1,0)-"
"«tuple<int, uint, uint>»──(24,0,13)-"
"«tuple<int, uint, uint>»──(25,10,11)-"
"«tuple<int, uint, uint>»──(26,9,9)-"
"«tuple<int, uint, uint>»──(27,8,7)-"
"«tuple<int, uint, uint>»──(28,7,5)-"
"«tuple<int, uint, uint>»──(29,6,3)-"
"«tuple<int, uint, uint>»──(30,5,1)"_expect);
CHECK (materialise (
zip
( eachNum(10)
, explore(num31())
.expand ([](int i){ return NumIter{noneg(i-1),i}; })
, explore(num31())
.expand ([](int i){ return NumIter{noneg(i-2),i-1}; })
)
.expandAll() // ◁──────────┲━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ note the difference: expand triggered after the zip()
) // ▽
== "«tuple<int, uint, uint>»──(10,1,1)-"
"«tuple<int, uint, uint>»──(10,0,4)-"
"«tuple<int, uint, uint>»──(10,4,2)-"
"«tuple<int, uint, uint>»──(10,3,0)-"
"«tuple<int, uint, uint>»──(10,2,7)-"
"«tuple<int, uint, uint>»──(10,1,5)-"
"«tuple<int, uint, uint>»──(10,0,3)-"
"«tuple<int, uint, uint>»──(10,7,1)-"
"«tuple<int, uint, uint>»──(10,6,10)-"
"«tuple<int, uint, uint>»──(10,5,8)-"
"«tuple<int, uint, uint>»──(10,4,6)-"
"«tuple<int, uint, uint>»──(10,3,4)-"
"«tuple<int, uint, uint>»──(10,2,2)-"
"«tuple<int, uint, uint>»──(10,1,0)-"
"«tuple<int, uint, uint>»──(10,0,13)-"
"«tuple<int, uint, uint>»──(10,10,11)-"
"«tuple<int, uint, uint>»──(10,9,9)-"
"«tuple<int, uint, uint>»──(10,8,7)-"
"«tuple<int, uint, uint>»──(10,7,5)-"
"«tuple<int, uint, uint>»──(10,6,3)-"
"«tuple<int, uint, uint>»──(10,5,1)"_expect);
Library: more aggressive testing with feature combinations and yes ... this revealed a **long standing bug** The `Filter::pullFilter()` invocation in the ctor may produce dangling refs, whenever an underlying source-iterator generates a reference that points into the iterator itself. The reason is: due to the »onion shell« design of the iterator pipeline, we are bound to move a source iterator into the next layer constructor.
2024-11-23 03:30:07 +01:00
}
Library: investigate how a »zip iterator« can be built Basically I am sick of writing for-loops in those cases where the actual iteration is based on one or several data sources, and I just need some damn index counter. Nothing against for-loops in general — they have their valid uses — sometimes a for-loop is KISS But in these typical cases, an iterator-based solution would be a one-liner, when also exploiting the structured bindings of C++17 ''I must admit that I want this for a loooooong time —'' ...but always got intimidated again when thinking through the fine points. Basically it „should be dead simple“ — as they say Well — — it ''is'' simple, after getting the nasty aspects of tuple binding and reference data types out of the way. Yesterday, while writing those `TestFrame` test cases (which are again an example where you want to iterate over two word sequences simultaneously and just compare them), I noticed that last year I learned about the `std::apply`-to-fold-expression trick, and that this solution pattern could be adapted to construct a tuple directly, thereby circumventing most of the problems related to ''perfect forwarding'' So now we have a new util function `mapEach` (defined in `tuple-helper.hpp`) and I have learned how to make this application completely generic. As a second step, I implemented a proof-of-concept in `IterZip_test`, which indeed was not really challenging, because the `IterExplorer` is so very sophisticated by now and handles most cases with transparent type-driven adaptors. A lot of work went into `IterExplorer` over the years, and this pays off now. The solution works as follows: * apply the `lib::explore()` constructor function to the varargs * package the resulting `IterExplorer` instantiations into a tuple * build a »state core« implementation which just lifts out the three iterator primitives onto this ''product type'' (i.e. the tuple) * wrap it in yet another `IterExplorer` * add a transformer function on top to extract a value-tuple for each ''yield' As expected, works out-of-the-box, with all conceivable variants and wild mixes of iterators, const, pointers, references, you name it.... PS: I changed the rendering of unsigned types in diagnostic output to use the short notation, e.g. `uint` instead of `unsigned int`. This dramatically improves the legibility of verification strings.
2024-11-21 23:30:07 +01:00
};
LAUNCHER (IterZip_test, "unit common");
}} // namespace lib::test
Reference in a new issue Copy permalink