diff --git a/doc/technical/overview.txt b/doc/technical/overview.txt index 20f6becc2..c401a36ba 100644 --- a/doc/technical/overview.txt +++ b/doc/technical/overview.txt @@ -6,7 +6,7 @@ Lumiera: The Inner Core [abstract] ****************************************************************************** -The Lumiera Developers have a distinct vision about the core of a modern NLE. +The Lumiera Developers have a distinct vision about the core of a modern NLE. This document outlines some of the basic technical concepts and highlights the fundamental design decisions. New Developers may find this Document useful to get an idea about how the different components work together. @@ -25,7 +25,7 @@ This three Layers are: The Stage Layer:: Interaction and presentation. User interfaces are implemented as plug-ins, and while most commonly one will see the GTK UI, it would be possible to - construct a entirely different user interface, a CLI intereface, or even + construct a entirely different user interface, a CLI interface, or even operate the application ``headless'', script driven. The Steam Layer:: @@ -116,7 +116,7 @@ for the foreseeable future. Lumiera Application ------------------- -Generally speaking, the Application is comprised of several self contained +Generally speaking, the Application is comprised of several self contained _subsystems_, which may depend on each other. Dependencies between components are to be abstracted through interfaces. Based on the current configuration, the application framework brings up the necessary subsystems and finally @@ -210,13 +210,13 @@ Yet it addresses some central concerns: uniformity:: All playback and render processes are on equal footing, handled in a similar way. - + integration:: The player cares for the necessary integration with the other subsystems -+ ++ it consults the _Output Management,_ retrieves the necessary informations from the _Session_ and coordinates the forwarding of Vault-Layer calls. - + time quantisation:: The player translates continuous time values into discrete frame counts. + @@ -228,17 +228,17 @@ The player service ^^^^^^^^^^^^^^^^^^ Client code accesses the player (subsystem) through the play-facade (`lumiera::Play`). The exposed service allows to _set up an output connection for playback or rendering,_ -resulting in a play-controller object. +resulting in a play-controller object. .Play::Controller -This controller frontend represents the presence of such an active output connection +This controller frontend represents the presence of such an active output connection and incorporates a state machine supporting the usual things you'd expect to do with a player (Play, pause, FFwd, Rew, scrubbing, jumping, looping). This controller object is a copyable smart-handle -- all instances act as if wired in parallel. .time control The play-controller frontend makes heavy use of `time::Control`. This is a mediator -to accept and forward _mutations_ on time values and time ranges, possibly involving +to accept and forward _mutations_ on time values and time ranges, possibly involving frame quantisation. After attaching it to a target time value, it accepts changes, offsets and nudging, translates these into the appropriate target modifications and notifies any attached _change listeners_. @@ -248,7 +248,7 @@ Ongoing effort to calculate a stream of frames for playback or rendering. + The play process is an conceptual entity linking together several activities in the vault and the render engine. It maintains a registration entry for the process to keep track of associated entities, resources allocated and calls dispatched as a consequence. Besides -each play process is wired to at leas one play-controller acting as frontend interface +each play process is wired to at least one play-controller acting as frontend interface and information hub for the client code. NOTE: the player is in no way engaged in any of the actual calculation and management tasks @@ -266,38 +266,6 @@ performance critical. Vault Layer ----------- -I/O Subsystem -~~~~~~~~~~~~~ - -.OS Filehandles -as mru cache, round robin reused - -.Files -Lumiera has its own abstract file handles which store the state and name of a -file. The associated filehandle doesn't need to be kept open and will be -reopened on demand. Hardlinked files are recognized and opened only once. - -.Memory Mapping -All file access is done by memory mapping to reduce data copies between -userland and kernel. Moreover the kernel becomes responsible to schedule -paging (which will be augmented by lumiera) to make the best use of available -resources. Memory is mapped in larger windows of reasonable sized chunks, possibly -overlapping each other. Clients may request a specific continuous set of data from -the file to be accessible as memory block. - - -.Indexing - -.Frameprovider - - -Threadpools -~~~~~~~~~~~ - -Manages serveral classes of threads in pools. The threadpool is reasonable -dumb. Higher level management will be done by the Schedulers and Jobs. - - Engine Interface ~~~~~~~~~~~~~~~~ While on itself just a thin interface and adaptation layer forwarding calls to @@ -326,30 +294,49 @@ the job ticket acts as blueprint for the actual jobs to be enqueued with the _Scheduler._ -Schedulers -~~~~~~~~~~ +Scheduler +~~~~~~~~~ +The Scheduler serves as the central hub in the implementation of the RenderEngine +and coordinates the processing resources of the application. Its purpose is to +invoke and control the dependency and time based execution of the _Render Jobs._ -Scheduling Queues for different purposes: +Regarding architecture, the Scheduler is located in the Vault-Layer and running the Scheduler +is equivalent to activating the »Vault Subsystem«. An EngineFaçade acts as entrance point, +providing high-level render services to other parts of the application: render jobs can be +activated under various timing and dependency constraints. +Internally, the implementation is organised into two layers: + +Layer-2: Coordination:: + Maintain a network of interconnected activities, track dependencies and observe + timing constraints; coordinate a pool of _active Workers_ to dispatch the next activities. + +Layer-1: Invocation:: + Operates a low-level priority scheduling mechanism for time-bound execution of activities. + +Generally speaking, a render calculation can be organised by different modes of control: .Deadline -Higher priority jobs ordered by a deadline time plus some (negative) hysteresis. Jobs are -started when they approach their deadline. Jobs who miss their deadline are -never scheduled here. +Higher priority jobs ordered by a deadline time plus some (negative) hysteresis. +Jobs are started when they approach their deadline. When it is foreseeable that a +jobs will miss the deadline, it will be skipped, preferring to use resources on +calculations expected to be successful and in time. .Background -Background jobs scheduled by priority and timeout. +Background calculations will use excess resources of the system; +a typical example would be the calculation of preview images and sound waveform +outlines for the GUI. - -.Realtime -Timer driven queue which starts jobs at defined absolute times. Timer might be -also an external synchronization entity. +.Best Effort +Calculation for final rendering will use all available resources and never skip any +processing, irrespecive of the time it takes to complete. Job ^^^ -a job can be part of multiple queues, the queue which picks them first runs -them. When other queues hit a running job they either just drop it or promote -its priority (to be decided). +A _Render Job_ is prepared as a self-contained functor, with a well defined time window +of execution. A job, once triggered, will run to completion without and can not be aborted. +The scheduler is able to observe and maintain dependencies between render jobs, allowing +to break down an extended calculation into a chain of small steps. Resource Management @@ -372,31 +359,11 @@ Collects statistic about resource load, helps to decide if job constraints can be fulfilled. Things to watch: - * cpu utilization + * CPU utilisation * memory usage (swapping, paging) * I/O load, latency -Budget Manager -^^^^^^^^^^^^^^ - -resources need to be distributed among a lot subsystems and jobs. Each of this -component can become part of a budgeting system which accounts resource usage -and helps to distribute it. Resource usage is only voluntary managed. - - -Resource collector -^^^^^^^^^^^^^^^^^^ - -Handles system errors related to resource shortage. There are several classes -of resources defined. Other subsystems can hook in functions to free -resources. Has multiple policies about how aggressive resources should be freed. - -If no one cares it does a final abort(). So all systems should hook better -recovery here in! - - - Common Services @@ -514,13 +481,15 @@ and disjunction. Locking ~~~~~~~ General purpose Locking is based on object monitors. Performance critical code -in the Vault uses mutexes, condition vars and rwlocks direcly. -Intentionally no semaphores. - -- C++ locks are managed by scoped automatic variables -- C code uses macros to wrap critical sections - +in the Render Engine relies on ''Atomics'', or may use futures and further +synchronisation primitives through the wrappers of the C++ standard library. +NOTE: At the time when Lumiera project started, the standard library provided + no portable support for concurrency and locking. As a remedy, various + solutions were built on top of the POSIX primitives. Some of these + ad-hoc solutions turned out to be viable and were reworked and adapted, + as proper concurrency support became part of the language standard. + For new code, the direct use of POSIX primitives is discouraged. Time ~~~~ @@ -557,7 +526,7 @@ A time alignment grid is exactly that: a set of functions to perform this lossy conversion. Implicitly this involves the definition of an _time origin_ (reference point where the external time is zero), and typically this also includes the definition of a _frame rate_ (but -in the most general case, this frame rate might be variable and +in the most general case, this frame rate might be variable and change at various places of the time axis). Consequently, all time grids are Assets and defined as part of the concrete session. @@ -601,7 +570,7 @@ output connection or something similar to establish a frame grid. Errors ~~~~~~ - + * As a Rule, Exceptions + RAII are to be preferred over error codes and manual cleanup. At external interfaces we rely on error states though. @@ -615,8 +584,6 @@ Errors case of raising an exception / error state (transactional behaviour). - *EX_SANE* functions might leave a partial change, but care to leave any involved objects in a sane state. - * Raising an Exception creates an _error state_ -- error states can also - be set directly per thread. * Error states are identified by pointers to static strings. * Error states are thread local and sticky (a new state can't be set unless a pending state got cleared). @@ -647,7 +614,7 @@ Error states Errors states get declared in headers with the `LUMIERA_ERROR_DECLARE(err)` macro. A matching definition needs to reside in some translation unit, using the `LUMIERA_ERROR_DEFINE(err, msg)` macro. There is no central registry, any component -can introduce its own errorcodes but must ensure that the error identifier is +can introduce its own error codes but must ensure that the error identifier is unique. .Error handling in C @@ -717,15 +684,12 @@ unaware of such special setup. .mock testing support Moreover, the Dependency-Factory also offers a mechanism to ``push aside'' the existing singleton -or serviec instance and shadow it temporarily with a mock implementation. Again, it turned out that +or service instance and shadow it temporarily with a mock implementation. Again, it turned out that such mocking mechanisms are used only occasionally and rarely required. Likely this is a result of Lumiera being developed _test-driven_ -- things are being written mostly in a unit test friendly way. -> more about link:{ldoc}/technical/library/Dependencies.html[dependency handling] -Extensible Factory -~~~~~~~~~~~~~~~~~~ -_tbw_ Visiting Tool ~~~~~~~~~~~~~ @@ -749,7 +713,7 @@ _once_. The iterator can be disposed when _exhausted_ -- there is no way of rese moving backwards or doing any kind of arithmetic with such an object. The _exhausted state can be detected by a +bool+ conversion (contrast this with STL iterators, where you need to compare to an +end+ iterator). Beyond that, the usage is quite similar, -even compatible to +std::for_each+. +even compatible to +std::for_each+. Iterator Adapters ^^^^^^^^^^^^^^^^^ @@ -802,7 +766,7 @@ Front-end for boost::format Formatting values with `printf` is a notorious source for intricate errors. Additionally, using (s|n)`printf` can be clunky in practice, and it doesn't support custom defined string conversions, which are an important diagnostic -aid when working with objects. We might +aid when working with objects. We might link:{ldoc}/technical/howto/UsingBoost.html[use Boost], which provides the `boost::format` library to address those problems. Unfortunately including this header-only library solution incurs a significant overhead, both in terms of @@ -841,7 +805,7 @@ smart-pointer type of C\++ (`boost::shared_ptr` and C++11). Typically this also includes a pointer to some kind of implementation service. + Yet still, handles based on `lib::Handle` should not be confused with smart pointers. Rather, we use the ref-counting mechanism to invoke a custom -clean-up callback when the last handle goes out of scope. Typically, the +clean-up callback when the last handle goes out of scope. Typically, the implementation service is kept entirely opaque, while the copyable handle objects also implement a front-end interface for client access. @@ -863,7 +827,7 @@ are defined non-copyable in Lumiera). .opaque holder -There is a family of holder objects, all based on placement-new of the +There is a family of holder objects, all based on placement-new of the contained object into an embedded buffer. The purpose is to ``piggyback'' an object inline, without the need for heap allocated storage. Frequently the motivation for this usage pattern is *type erasure*: the detailed knowledge @@ -882,7 +846,7 @@ at the usage site. To avoid this dilemma, we utilise the technique of the opaque holder to provide objects with value semantics, while actually placing the instance of a subclass into the inline buffer. Clients access this embedded object by automatic type -conversion to the interface type, which gives us polymorphism. While the +conversion to the interface type, which gives us polymorphism. While the definition of such a beast is quite involved, the runtime overhead is surprisingly low. When compared with standard polymorphism, creating objects and invoking operations has zero overhead, while copying @@ -936,7 +900,7 @@ WARNING: currently (as of 2011) the low-level pooled allocator within the See Ticket #231 .Allocation Cluster -This allocation scheme is used within the context of the Builder and the +This allocation scheme is used within the context of the Builder and the Low-Level-Model. The predominant usage trend in this realm is to create and wire a family of small objects right after each other, within a build process. These objects are intended to work together and will be discarded @@ -944,27 +908,21 @@ all at once, after hot-swapping a new version of that model segment. .Typed Allocation Manager This allocation framework is used at various places when a large number of -similar objects is expected to be coming and going. New objects are +similar objects is expected to be coming and going. New objects are placement-constructed into the allocated space and immediately wrapped with a ref-counting smart-ptr to manage ownership. -.Simple Allocator -Based on the allocator interface of the STL, allowing just for plain -allocations and de-allocations without any further instance and lifecycle -management. Currently (as of 2011) this allocator isn't used much -- it is -conceivable that later we'll detect some specific STL based containers to be -performance critical with respect to allocation. +.BlockFlow (Epoch Allocator) +A custom memory management scheme for the Scheduler in the Render Engine. +The scheduler produces a _ongoing flow_ of _Render-Activity_ records, which +are added and wired, and then activated once based on temporal ordering. +For each frame calculation job and thus also for each supporting calculation +a _temporal deadline_ can be defined. This observation is exploited to order +the allocations into _extents based on deadline._ Once the deadline for such +an »epoch« has passed, all data records can be discarded without any further +checks or clean-up work. -Memory Pools -~~~~~~~~~~~~ - -Fast memory pools for moderately small static sized allocations in highly -dynamic situations. - - * optimized for cache locality - * supporting a destructor callback to free all objects - Temporary Buffers ~~~~~~~~~~~~~~~~~ @@ -1004,36 +962,13 @@ Preprocessor Metaprogramming ppmpl.h -CLib wrappers -~~~~~~~~~~~~~ - -Some wrapers for the C memory management functions malloc, calloc, realloc and -free which never fail. In case of an error the resourcecollector in the -vault is invoked to free resources or doing an emergency shutdown. - -Safe wrapers for some string functions from the C-library which also never -fail. NULL strings are propagated to "" empty strings. - - -Polymorphic Programming in C -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Just a macro for simplifying vtable function calls - VCALL(Handle, function, arguments...) -translates to - Handle->vtable->function (Handle, arguments...) - -The user is responsible for setting up a `vtable` member in his datastructures -this macro does some NoBug checks that self and function are initialized. - - Algorithms & Datastructures ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Probabilistic Splay Tree ^^^^^^^^^^^^^^^^^^^^^^^^ -Self optimizing splay tree +Self optimising splay tree Advantage: * can be iterated @@ -1044,21 +979,6 @@ Disadvantages * (almost) every access is a mutation and needs an exclusive lock, bad for concurrency -BTree -^^^^^ - -Generic B+/B* Tree implementation. Details are provided by a vtable at an -actual implementation. - - * Fine grained (block level) locking with different modes - * supports cursors to iterate over the data, in both directions - * exact and inexact searched (what's close before/after something) - - -Cuckoo Hashing -^^^^^^^^^^^^^^ -_Currently defunct, to be revived someday_ - Hash functions ^^^^^^^^^^^^^^ @@ -1072,18 +992,6 @@ Linked Lists Cyclic double linked intrusive list. -.slist - -Single linked variant. - - -Most Recent used Cachelists -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -A list where old entries are recycled from the tail, used things are removed -from the list (ownership acquired) and released back to the head. - -Items not used propagate towards the tail where they will be reused. //Undocumented diff --git a/doc/user/intro/intro.txt b/doc/user/intro/intro.txt index 569a966a0..4a8f7e6d3 100755 --- a/doc/user/intro/intro.txt +++ b/doc/user/intro/intro.txt @@ -6,7 +6,7 @@ Lumiera (as seen) from Outer Space ****************************************************************************** .abstract -The Lumiera Community is in the process of making a non-linear video editing +The Lumiera Community is in the process of making a non-linear video editing and compositing FOSS application for Linux/Unix/Posix operating systems. The application is geared towards professional, high-quality work; but it is equally suitable for low-end users, due to its in-design scalability. @@ -70,7 +70,7 @@ aspects film production tools will have to exhibit to aspire to be labelled requires crisp-quality results with no concessions made throughout the entire workflow that might compromise quality of the final work. A common requirement is the ability to reproduce all rendering - calculations down to the last bit. + calculations down to the last bit. Performance and Productivity:: Professionals want to get things done on time, but @@ -84,14 +84,14 @@ aspects film production tools will have to exhibit to aspire to be labelled Lumiera must be adept to reap the maximum from the hardware at hand. Durability:: - The rapid pace at which software and hardware rampage forward is surely + The rapid pace at which software and hardware rampage forward is surely a warning to new software projects and the dangers of locking into any current technological ``fashion'' to achieve a ``cheap'' goal, feature or performance boost. Once the fad fades, the software woes begin. The software must be able to engage new technological developments without any compromise to functionality or backward compatibility. - + Fundamental Forces ------------------ @@ -129,7 +129,7 @@ details such as header format conversion or access to individual channels. To complement this approach, Lumiera does _not_ rely on hard-wired, global conventions -- rather we allow the building up of project specific conventions and -rules xref:rules[<-] to fit a given requirement and preferred working style. +rules xref:rules[<-] to fit a given requirement and preferred working style. Lumiera will be supplied with a conventional template and a default configuration to ease the learning curve for new users. @@ -172,7 +172,7 @@ At a first glance, it looks fairly natural to set up the graphs xref:graphs[<-] as described above. Data can then be pushed into the system through the input nodes and the final result can be seen at the output node. Several multimedia frameworks use this approach. However this scheme exhibits a number of -shortcomings which make it inappropriate for non-linear video editing. +shortcomings which make it inappropriate for non-linear video editing. Lumiera takes a different approach. Data is pulled through a pipe: any rendering request starts at the output node and makes its way back up through the graph to @@ -183,8 +183,8 @@ the inputs. This scheme offers a number of advantages over the naive scheme. Don't waste work ~~~~~~~~~~~~~~~~ -Rendering A/V data can be computationally intensive. To avoid duplicated or wasted work and -ensure it is possible to render on time, +Rendering A/V data can be computationally intensive. To avoid duplicated or wasted work and +ensure it is possible to render on time, Lumiera employs a sophisticated means of using cache xref:caching[<-] and profiling xref:profiling[<-]. @@ -234,12 +234,12 @@ The screenshot of the GUI presented above is more or less the standard GUI when it is started for the first time without any user configuration etc. A second viewer is planned for later to be added to the default. We support a much more sophisticated screen concept xref:screenconcept[<-] to adapt to different -workplaces and workflows. +workplaces and workflows. // TODO // The above paragrph isn't all it should be, but I can't tell what it should // be without seeing it in the context of the other changes I'm making. // I suspect it's better to discuss how rewritable the GUI is rather than -// the specifics ot the GUI presented above. -- hendrik +// the specifics of the GUI presented above. -- hendrik At a first glance, the GUI might seem somewhat overwhelming, something similar to the cockpit in a jumbo jet. However, nearly all the bells and whistles can @@ -318,7 +318,7 @@ individual clips, or the pipes collecting output from transitions, from nested sequences and from groups of forks (``tracks''). At some point, at the timeline level, all processed data is collected within the aforementioned global pipes to form the small number of output streams produced by rendering and playback. Each timeline -uses a separate set of global pipes. +uses a separate set of global pipes. Asset View @@ -334,7 +334,7 @@ There are various kinds of assets available in any project: * source media/footage/soundfiles * all available effects and transitions * user defined ``effect stacks'' and effect collections ("effect palette") - * internal artefacts like sequences and automation data sets + * internal artifacts like sequences and automation data sets * markers, labels, tags and similar kinds of meta data Actually, the same underlying _fork_ data structure is used to implement the @@ -368,10 +368,10 @@ Vault. Session Storage ~~~~~~~~~~~~~~~ -The current representation of a project resident in memory is internally known +The current representation of a project resident in memory is internally known as a session, whereas from a GUI user perspective, this is known as a project. In this section, we will use the term ``session'' as we are discussing the internals -of Lumiera here. +of Lumiera here. Everything is stored in a session. If a user saves a session, then there should be no difference to the user whether they continue working or throw all their @@ -384,7 +384,7 @@ operations on these objects. For example, unlimited ability to undo previous steps, selective undo of previous steps or the possibility of merging various steps might even be a possibility. On a practical note, work on a project at the office and work on the same project at home can be merged each morning and -evening. +evening. Session storage is envisaged to operate similar to a database or journalling file system. Any operation will be logged prior to execution. This protects @@ -499,7 +499,7 @@ will not reinvent the wheel each time he sits down to programme an application. A programmer will typically borrow and use features and functionality from other programmers---or even borrow from himself, stuff written long ago in the past. Such features are collected together in -libraries. +libraries. A library is used in an application by _linking_ the library into the application. (There are other things to be done, but we'll call these 'details', @@ -531,7 +531,7 @@ available to an application is known as run-time linking, aka plug-ins. Plug-ins offer other benefits: the application can continue to use both the old features and the new features together, side-by-side, by using the version number associated with the plug-in. This saves the application from the havoc -of library incompatibility that is +of library incompatibility that is associated with other linking methods. Most modern applications use plug-ins, some are heavily dependent on plug-ins @@ -552,7 +552,7 @@ backend, right up to the GUI -- the interface will have to be compatible to both and C++. In effect, this means that LPI will be implemented in C. Using LPI a plugin interface can be defined for each external library. The -Lumiera code base can then use the external library by including the +Lumiera code base can then use the external library by including the plugin interface for that particular library. LPI defines a uniform framework of how to manage plugins within Lumiera. It @@ -560,7 +560,7 @@ declares a number of macros that allow details of a plugin to be defined: - name of the plugin - version of the plugin (include major and minor version numbers) - name each call-back function defined by the plugin and exported by the - external library + external library - methods to initialise and destroy the plugin Defining some plugin interface then entails using LPI to specify the details of @@ -602,6 +602,6 @@ The Inner Core ~~~~~~~~~~~~~~ Developers and Readers curious about the inner workings of the application might want to have a look into the overview document of our technical -documentation section. Learn more about +documentation section. Learn more about link:{ldoc}/technical/overview.html[The Inner Core]... diff --git a/src/lib/condition.c b/src/lib/condition.c deleted file mode 100644 index 2218f40ce..000000000 --- a/src/lib/condition.c +++ /dev/null @@ -1,75 +0,0 @@ -/* - condition.c - condition variable - - Copyright (C) Lumiera.org - 2008, 2009, 2010, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. -*/ - -#include "lib/condition.h" - -/** - * @file - * Condition variables - */ - - -LumieraCondition -lumiera_condition_init (LumieraCondition self, - const char* purpose, - struct nobug_flag* flag, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_HANDLE_INIT (self->rh); - NOBUG_RESOURCE_ANNOUNCE_RAW_CTX (flag, "cond_var", purpose, self, self->rh, ctx) - { - pthread_mutex_init (&self->cndmutex, NULL); - pthread_cond_init (&self->cond, NULL); - } - } - return self; -} - - -LumieraCondition -lumiera_condition_destroy (LumieraCondition self, - struct nobug_flag* flag, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_FORGET_RAW_CTX (flag, self->rh, ctx) - { - if (pthread_mutex_destroy (&self->cndmutex)) - LUMIERA_DIE (LOCK_DESTROY); - - if (pthread_cond_destroy (&self->cond)) - LUMIERA_DIE (LOCK_DESTROY); - } - } - return self; -} - - -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/lib/condition.h b/src/lib/condition.h deleted file mode 100644 index 8d017d1ec..000000000 --- a/src/lib/condition.h +++ /dev/null @@ -1,362 +0,0 @@ -/* - condition.h - condition variables - - Copyright (C) Lumiera.org - 2008, 2010, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. -*/ - -#ifndef LUMIERA_CONDITION_H -#define LUMIERA_CONDITION_H - -#include "lib/error.h" -#include "lib/sectionlock.h" -#include "lib/lockerror.h" - -#include -#include -#include -#include - -/** - * @file - * Condition variables, header - */ - - -/** - * Condition section. - * Locks the condition mutex, one can use LUMIERA_CONDITION_WAIT to wait for signals or - * LUMIERA_CONDITION_SIGNAL or LUMIERA_CONDITION_BROADCAST to wake waiting threads - * Condition variables must be at the end of locking chains, they can not be used at - * intermediate position. - * @param nobugflag NoBug flag used to log actions on the condition - * @param cnd Condition variable to be locked - */ -#define LUMIERA_CONDITION_SECTION(nobugflag, cnd) \ - for (lumiera_sectionlock NOBUG_CLEANUP(lumiera_sectionlock_ensureunlocked) \ - lumiera_lock_section_ = { \ - cnd, (lumiera_sectionlock_unlock_fn) lumiera_condition_unlock \ - NOBUG_ALPHA_COMMA(&NOBUG_FLAG(nobugflag)) NOBUG_ALPHA_COMMA_NULL}; \ - ({ \ - if (lumiera_lock_section_.lock) \ - lumiera_lock_section_.lock = \ - lumiera_condition_lock (cnd, &NOBUG_FLAG(nobugflag), \ - &lumiera_lock_section_.rh, NOBUG_CONTEXT); \ - lumiera_lock_section_.lock; \ - }); \ - ({ \ - LUMIERA_CONDITION_SECTION_UNLOCK; \ - })) - - -#define LUMIERA_CONDITION_SECTION_CHAIN(nobugflag, cnd) \ - for (lumiera_sectionlock *lumiera_lock_section_old_ = &lumiera_lock_section_, \ - NOBUG_CLEANUP(lumiera_sectionlock_ensureunlocked) lumiera_lock_section_ = { \ - cnd, (lumiera_sectionlock_unlock_fn) lumiera_condition_unlock \ - NOBUG_ALPHA_COMMA(&NOBUG_FLAG(nobugflag)) NOBUG_ALPHA_COMMA_NULL}; \ - ({ \ - if (lumiera_lock_section_.lock) \ - { \ - REQUIRE (lumiera_lock_section_old_->lock, "section prematurely unlocked"); \ - lumiera_lock_section_.lock = \ - lumiera_condition_lock (cnd, &NOBUG_FLAG(nobugflag), \ - &lumiera_lock_section_.rh, NOBUG_CONTEXT); \ - LUMIERA_SECTION_UNLOCK_(lumiera_lock_section_old_); \ - } \ - lumiera_lock_section_.lock; \ - }); \ - ({ \ - LUMIERA_CONDITION_SECTION_UNLOCK; \ - })) - - -#define LUMIERA_CONDITION_SECTION_UNLOCK \ - LUMIERA_SECTION_UNLOCK_(&lumiera_lock_section_) - - -/** - * Wait for a condition. - * Must be used inside a CONDITION_SECTION. - * @param expr Conditon which must become true, else the condition variable goes back into sleep - */ -#define LUMIERA_CONDITION_WAIT(expr) \ - while (!(expr)) { \ - REQUIRE (lumiera_lock_section_.lock, "Condition mutex not locked"); \ - lumiera_condition_wait (lumiera_lock_section_.lock, \ - lumiera_lock_section_.flag, \ - &lumiera_lock_section_.rh, \ - NOBUG_CONTEXT); \ - } while (!(expr)) - - -/** - * Timed wait for a condition. - * Must be used inside a CONDITION_SECTION. - * @param expr Conditon which must become true, else the condition variable goes back into sleep - * @param timeout time when the wait expired - * sets LUMIERA_ERROR_LOCK_TIMEOUT when the timeout passed - */ -#define LUMIERA_CONDITION_TIMEDWAIT(expr, timeout) \ - while (!(expr)) { \ - REQUIRE (lumiera_lock_section_.lock, "Condition mutex not locked"); \ - if (!lumiera_condition_timedwait (lumiera_lock_section_.lock, \ - timeout, \ - lumiera_lock_section_.flag, \ - &lumiera_lock_section_.rh, \ - NOBUG_CONTEXT)) \ - break; \ - } - -/** - * Signal a condition variable - * Must be used inside a CONDITION_SECTION. - * Wakes one thread waiting on the condition variable - */ -#define LUMIERA_CONDITION_SIGNAL \ - do { \ - REQUIRE (lumiera_lock_section_.lock, "Condition mutex not locked"); \ - lumiera_condition_signal (lumiera_lock_section_.lock, \ - lumiera_lock_section_.flag, \ - NOBUG_CONTEXT); \ - } while (0) - - -/** - * Broadcast a condition variable - * Must be used inside a CONDITION_SECTION. - * Wakes all threads waiting on the condition variable - */ -#define LUMIERA_CONDITION_BROADCAST \ - do { \ - REQUIRE (lumiera_lock_section_.lock, "Condition mutex not locked"); \ - lumiera_condition_broadcast (lumiera_lock_section_.lock, \ - lumiera_lock_section_.flag, \ - NOBUG_CONTEXT); \ - } while (0) - - -/** - * Condition variables. - * - */ -struct lumiera_condition_struct -{ - pthread_cond_t cond; - pthread_mutex_t cndmutex; - RESOURCE_HANDLE (rh); -}; -typedef struct lumiera_condition_struct lumiera_condition; -typedef lumiera_condition* LumieraCondition; - - -/** - * Initialise a condition variable - * @param self is a pointer to the condition variable to be initialised - * @return self as given - */ -LumieraCondition -lumiera_condition_init (LumieraCondition self, - const char* purpose, - struct nobug_flag* flag, - const struct nobug_context ctx); - - -/** - * Destroy a condition variable - * @param self is a pointer to the condition variable to be destroyed - * @return self as given - */ -LumieraCondition -lumiera_condition_destroy (LumieraCondition self, - struct nobug_flag* flag, - const struct nobug_context ctx); - - -static inline LumieraCondition -lumiera_condition_lock (LumieraCondition self, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_WAIT_CTX (NOBUG_FLAG_RAW(flag), self->rh, "acquire condvar", *handle, ctx) - { - if (pthread_mutex_lock (&self->cndmutex)) - LUMIERA_DIE (LOCK_ACQUIRE); /* never reached (in a correct program) */ - - NOBUG_RESOURCE_STATE_CTX (NOBUG_FLAG_RAW(flag), NOBUG_RESOURCE_EXCLUSIVE, *handle, ctx) /*{}*/; - } - } - - return self; -} - - -static inline LumieraCondition -lumiera_condition_trylock (LumieraCondition self, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_TRY_CTX (NOBUG_FLAG_RAW(flag), self->rh, "try acquire condvar", *handle, ctx) - { - int err = pthread_mutex_trylock (&self->cndmutex); - - if (!err) - { - NOBUG_RESOURCE_STATE_CTX (NOBUG_FLAG_RAW(flag), NOBUG_RESOURCE_EXCLUSIVE, *handle, ctx) /*{}*/; - } - else - { - NOBUG_RESOURCE_LEAVE_RAW_CTX (flag, *handle, ctx) /*{}*/; - lumiera_lockerror_set (err, flag, ctx); - self = NULL; - } - } - } - - return self; -} - - -static inline LumieraCondition -lumiera_condition_timedlock (LumieraCondition self, - const struct timespec* timeout, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_TRY_CTX (NOBUG_FLAG_RAW(flag), self->rh, "timed acquire condvar", *handle, ctx) - { - int err = pthread_mutex_timedlock (&self->cndmutex, timeout); - - if (!err) - { - NOBUG_RESOURCE_STATE_CTX (NOBUG_FLAG_RAW(flag), NOBUG_RESOURCE_EXCLUSIVE, *handle, ctx) /*{}*/; - } - else - { - NOBUG_RESOURCE_LEAVE_RAW_CTX (flag, *handle, ctx) /*{}*/; - lumiera_lockerror_set (err, flag, ctx); - self = NULL; - } - } - } - - return self; -} - - -static inline LumieraCondition -lumiera_condition_wait (LumieraCondition self, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_STATE_RAW_CTX (flag, NOBUG_RESOURCE_WAITING, *handle, ctx) - { - pthread_cond_wait (&self->cond, &self->cndmutex); - - NOBUG_RESOURCE_STATE_RAW_CTX (flag, NOBUG_RESOURCE_EXCLUSIVE, *handle, ctx) /*{}*/; - } - } - return self; -} - - -static inline LumieraCondition -lumiera_condition_timedwait (LumieraCondition self, - const struct timespec* timeout, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_STATE_RAW_CTX (flag, NOBUG_RESOURCE_WAITING, *handle, ctx) - { - int err; - do { - err = pthread_cond_timedwait (&self->cond, &self->cndmutex, timeout); - } while(err == EINTR); - - NOBUG_RESOURCE_STATE_RAW_CTX (flag, NOBUG_RESOURCE_EXCLUSIVE, *handle, ctx) /*{}*/; - - if (err) - { - lumiera_lockerror_set (err, flag, ctx); - self = NULL; - } - } - } - - return self; -} - - - -static inline void -lumiera_condition_signal (LumieraCondition self, - struct nobug_flag* flag, - const struct nobug_context ctx) -{ - NOBUG_TRACE_CTX (NOBUG_FLAG_RAW(flag), ctx, "Signal %p", self); - pthread_cond_signal (&self->cond); -} - - -static inline void -lumiera_condition_broadcast (LumieraCondition self, - struct nobug_flag* flag, - const struct nobug_context ctx) -{ - NOBUG_TRACE_CTX(NOBUG_FLAG_RAW(flag), ctx, "Broadcast %p", self); - pthread_cond_broadcast (&self->cond); -} - - -static inline void -lumiera_condition_unlock (LumieraCondition self, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - NOBUG_REQUIRE_CTX (self, ctx); - - NOBUG_RESOURCE_LEAVE_RAW_CTX (flag, *handle, ctx) - { - if (pthread_mutex_unlock (&self->cndmutex)) - LUMIERA_DIE (LOCK_RELEASE); /* never reached (in a correct program) */ - } -} - - -#endif -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/lib/mpool.c b/src/lib/mpool.c deleted file mode 100644 index 9aaa93a01..000000000 --- a/src/lib/mpool.c +++ /dev/null @@ -1,552 +0,0 @@ -/* - mpool.c - memory pool for constant sized objects - - Copyright (C) Lumiera.org - 2009, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. -*/ - - -/** @file mpool.c - ** Implementation of pooling memory allocation functions for constant sized objects. - */ - -#include -#include -#include -#include -#include - -#include "mpool.h" - - - -#if UINTPTR_MAX > 4294967295U /* 64 bit */ -#define MPOOL_DIV_SHIFT 6 -#define MPOOL_C(c) c ## ULL -#else /* 32 bit */ -#define MPOOL_DIV_SHIFT 5 -#define MPOOL_C(c) c ## UL -#endif - -/* - defaults for the hooks, used when creating mpools - */ -void *(*mpool_malloc_hook)(size_t size) = malloc; -void (*mpool_free_hook)(void *ptr) = free; - -/** called after a mpool got initialised */ -void (*mpool_init_hook) (MPool self) = NULL; -/** called before a mpool gets destroyed */ -void (*mpool_destroy_hook) (MPool self) = NULL; - -/* - Cluster and node structures are private -*/ -typedef struct mpoolcluster_struct mpoolcluster; -typedef mpoolcluster* MPoolcluster; -typedef const mpoolcluster* const_MPoolcluster; - -struct mpoolcluster_struct -{ - llist node; /* all clusters */ - char data[]; /* bitmap and elements */ -}; - - -typedef struct mpoolnode_struct mpoolnode; -typedef mpoolnode* MPoolnode; -typedef const mpoolnode* const_MPoolnode; - -struct mpoolnode_struct -{ - llist node; -}; - - -MPool -mpool_cluster_alloc_ (MPool self); - - -#define MPOOL_BITMAP_SIZE(elements_per_cluster) \ - (((elements_per_cluster) + sizeof(uintptr_t)*CHAR_BIT - 1) \ - / (sizeof(uintptr_t) * CHAR_BIT) * sizeof (uintptr_t)) - -MPool -mpool_init (MPool self, size_t elem_size, unsigned elements_per_cluster, mpool_destroy_fn dtor) -{ - TRACE (mpool_dbg, "%p: elem_size %zd: elem_per_cluster %u", self, elem_size, elements_per_cluster); - - if (self) - { - llist_init (&self->freelist); - llist_init (&self->clusters); - self->elem_size = (elem_size+sizeof(void*)-1) / sizeof(void*) * sizeof(void*); /* void* aligned */ - - /* minimum size is the size of a llist node */ - if (self->elem_size < sizeof(llist)) - self->elem_size = sizeof(llist); - - self->elements_per_cluster = elements_per_cluster; - - self->cluster_size = sizeof (mpoolcluster) + /* header */ - MPOOL_BITMAP_SIZE (self->elements_per_cluster) + /* bitmap */ - self->elem_size * self->elements_per_cluster; /* elements */ - - self->elements_free = 0; - self->destroy = dtor; - self->locality = NULL; - - self->malloc_hook = mpool_malloc_hook; - self->free_hook = mpool_free_hook; - - if (mpool_init_hook) - mpool_init_hook (self); - } - - return self; -} - - -static inline void* -cluster_element_get (MPoolcluster cluster, MPool self, unsigned n) -{ - return (void*)cluster + /* start address */ - sizeof (*cluster) + /* header */ - MPOOL_BITMAP_SIZE (self->elements_per_cluster) + /* bitmap */ - self->elem_size * n; /* offset*/ -} - - -static inline bool -bitmap_bit_get_nth (MPoolcluster cluster, unsigned index) -{ - TRACE (mpool_dbg, "cluster %p: index %u", cluster, index); - - uintptr_t quot = index>>MPOOL_DIV_SHIFT; - uintptr_t rem = index & ~((~MPOOL_C(0))<data; - - return bitmap[quot] & ((uintptr_t)1<clusters, cluster) - { - if (self->destroy) - for (unsigned i = 0; i < self->elements_per_cluster; ++i) - { - if (bitmap_bit_get_nth ((MPoolcluster)cluster, i)) - { - void* obj = cluster_element_get ((MPoolcluster)cluster, self, i); - TRACE (mpool_dbg, "dtor: cluster %p: obj %p: freelist %p", cluster, obj, &self->freelist); - self->destroy (obj); - } - } - - llist_unlink_fast_ (cluster); - TRACE (mpool_dbg, "freeing cluster %p" , cluster); - self->free_hook (cluster); - } - - llist_init (&self->freelist); - self->elements_free = 0; - self->locality = NULL; - } - - return self; -} - - -MPool -mpool_purge (MPool self) -{ - // not UNIMPLEMENTED because valid no-op - PLANNED("To be implemented"); - return self; -} - - -MPool -mpool_cluster_alloc_ (MPool self) -{ - MPoolcluster cluster = self->malloc_hook (self->cluster_size); - TRACE (mpool_dbg, "%p", cluster); - - if (!cluster) - return NULL; - - /* clear the bitmap */ - memset (&cluster->data, 0, MPOOL_BITMAP_SIZE (self->elements_per_cluster)); - - /* initialise freelist */ - for (unsigned i = 0; i < self->elements_per_cluster; ++i) - { - MPoolnode node = cluster_element_get (cluster, self, i); - TRACE (mpool_dbg, "node %p", node); - llist_insert_tail (&self->freelist, llist_init (&node->node)); - } - - /* we insert the cluster at head because its likely be used next */ - llist_insert_head (&self->clusters, llist_init (&cluster->node)); - self->elements_free += self->elements_per_cluster; - - return self; -} - - -static int -cmp_cluster_contains_element (const_LList cluster, const_LList element, void* cluster_size) -{ - if (element < cluster) - return -1; - - if ((void*)element > (void*)cluster + (uintptr_t)cluster_size) - return 1; - - return 0; -} - - -static inline MPoolcluster -element_cluster_get (MPool self, void* element) -{ - return (MPoolcluster) llist_ufind (&self->clusters, (const_LList) element, cmp_cluster_contains_element, (void*)self->cluster_size); -} - - -static inline unsigned -uintptr_nearestbit (uintptr_t v, unsigned n) -{ - unsigned r = 0; - uintptr_t mask = MPOOL_C(1)<>1)); - } -} - - -static inline void* -alloc_near (MPoolcluster cluster, MPool self, void* locality) -{ - TRACE (mpool_dbg, "locality %p", locality); - void* begin_of_elements = - (void*)cluster + - sizeof (*cluster) + /* header */ - MPOOL_BITMAP_SIZE (((MPool)self)->elements_per_cluster); /* bitmap */ - - uintptr_t index = (locality - begin_of_elements) / self->elem_size; - uintptr_t quot = index>>MPOOL_DIV_SHIFT; - uintptr_t rem = index & ~((~MPOOL_C(0))<data; - unsigned r = ~0U; - - /* the bitmap word at locality */ - if (bitmap[quot] < UINTPTR_MAX) - { - r = uintptr_nearestbit (~bitmap[quot], rem); - } - /* the bitmap word before locality, this gives a slight bias towards the begin, keeping the pool compact */ - else if (quot && bitmap[quot-1] < UINTPTR_MAX) - { - --quot; - r = uintptr_nearestbit (~bitmap[quot], sizeof(uintptr_t)*CHAR_BIT-1); - } - - if (r != ~0U && (quot*sizeof(uintptr_t)*CHAR_BIT+r) < self->elements_per_cluster) - { - void* ret = begin_of_elements + ((uintptr_t)(quot*sizeof(uintptr_t)*CHAR_BIT+r)*self->elem_size); - return ret; - } - return NULL; -} - - -static inline void -bitmap_set_element (MPoolcluster cluster, MPool self, void* element) -{ - void* begin_of_elements = - (void*)cluster + - sizeof (*cluster) + /* header */ - MPOOL_BITMAP_SIZE (((MPool)self)->elements_per_cluster); /* bitmap */ - - uintptr_t index = (element - begin_of_elements) / self->elem_size; - uintptr_t quot = index>>MPOOL_DIV_SHIFT; - uintptr_t rem = index & ~((~MPOOL_C(0))<data; - bitmap[quot] |= ((uintptr_t)1<elements_per_cluster); /* bitmap */ - - uintptr_t index = (element - begin_of_elements) / self->elem_size; - uintptr_t quot = index>>MPOOL_DIV_SHIFT; - uintptr_t rem = index & ~((~MPOOL_C(0))<data; - bitmap[quot] &= ~((uintptr_t)1<elements_free) - { - if (mpool_cluster_alloc_ (self)) - { - self->locality = NULL; /* supress alloc_near() */ - } - else - { - ERROR (mpool_dbg, "allocation failure"); - return NULL; - } - } - - void* ret = NULL; - - if (self->locality) - { - ret = alloc_near (element_cluster_get (self, self->locality), self, self->locality); - TRACE_IF (ret, mpool_dbg, "near allocation %p", ret); - } - - if (!ret) - { - ret = llist_head (&self->freelist); - TRACE_IF (ret, mpool_dbg, "far allocation %p", ret); - } - - if (ret) - { - bitmap_set_element (element_cluster_get (self, ret), self, ret); - llist_unlink_fast_ ((LList)ret); - } - - self->locality = ret; - --self->elements_free; - - return ret; -} - - -void* -mpool_alloc_near (MPool self, void* near) -{ - TRACE (mpool_dbg); - - if (!self->elements_free) - { - if (mpool_cluster_alloc_ (self)) - { - near = NULL; /* supress alloc_near() */ - } - else - { - ERROR (mpool_dbg, "allocation failure"); - return NULL; - } - } - - void* ret = NULL; - - if (near) - { - ret = alloc_near (element_cluster_get (self, near), self, near); - TRACE_IF (ret, mpool_dbg, "near allocation %p", ret); - } - - if (!ret) - { - ret = llist_head (&self->freelist); - TRACE_IF (ret, mpool_dbg, "far allocation %p", ret); - } - - if (ret) - { - bitmap_set_element (element_cluster_get (self, ret), self, ret); - llist_unlink_fast_ ((LList)ret); - } - - --self->elements_free; - - return ret; -} - - -static inline MPoolnode -find_near (MPoolcluster cluster, MPool self, void* element) -{ - void* begin_of_elements = - (void*)cluster + - sizeof (*cluster) + /* header */ - MPOOL_BITMAP_SIZE (((MPool)self)->elements_per_cluster); /* bitmap */ - - uintptr_t index = (element - begin_of_elements) / self->elem_size; - uintptr_t quot = index>>MPOOL_DIV_SHIFT; - uintptr_t rem = index & ~((~MPOOL_C(0))<data; - unsigned r = ~0U; - - /* the bitmap word at locality */ - if (bitmap[quot] < UINTPTR_MAX) - { - r = uintptr_nearestbit (~bitmap[quot], rem); - } - /* the bitmap word after element, we assume that elements after the searched element are more likely be free */ - else if (index < self->elements_per_cluster && bitmap[quot+1] < UINTPTR_MAX) - { - ++quot; - r = uintptr_nearestbit (~bitmap[quot], 0); - } - /* finally the bitmap word before element */ - else if (index > 0 && bitmap[quot-1] < UINTPTR_MAX) - { - --quot; - r = uintptr_nearestbit (~bitmap[quot], sizeof(uintptr_t)*CHAR_BIT-1); - } - - if (r != ~0U && (quot*sizeof(uintptr_t)*CHAR_BIT+r) < self->elements_per_cluster) - return begin_of_elements + ((uintptr_t)(quot*sizeof(uintptr_t)*CHAR_BIT+r)*self->elem_size); - - return NULL; -} - - -void -mpool_free (MPool self, void* element) -{ - if (self && element) - { - TRACE (mpool_dbg, "mpool %p: element %p", self, element); - - MPoolcluster cluster = element_cluster_get (self,element); - MPoolnode near = find_near (cluster, self, element); - - bitmap_clear_element (cluster, self, element); - llist_init (&((MPoolnode)element)->node); - - if (near) - { - TRACE (mpool_dbg, "found near %p", near); - if (near < (MPoolnode)element) - llist_insert_next (&near->node, &((MPoolnode)element)->node); - else - llist_insert_prev (&near->node, &((MPoolnode)element)->node); - } - else - llist_insert_tail (&self->freelist, &((MPoolnode)element)->node); - - ++self->elements_free; - } -} - - - -MPool -mpool_reserve (MPool self, unsigned nelements) -{ - if (self) - while (self->elements_free < nelements) - if (!mpool_cluster_alloc_ (self)) - return NULL; - - return self; -} - - -void -nobug_mpool_dump (const_MPool self, - const int depth, - const struct nobug_context dump_context, - void* extra) -{ - (void) extra; - - if (self && depth) - { - DUMP_LOG ("mpool %p: ", self); - - if (depth > 1) - { - DUMP_LOG (" elements_per_cluster %u: ", self->elements_per_cluster); - DUMP_LOG (" elements_free %u: ", self->elements_free); - } - - if (depth > 2) - { - DUMP_LOG (" clusters %p: ", &self->clusters); - int i = 0; - LLIST_FOREACH (&self->clusters, cluster) - DUMP_LOG (" %p: %u", cluster, ++i); - } - - if (depth > 3) - { - DUMP_LOG (" freelist %p: ", &self->freelist); - int i = 0; - LLIST_FOREACH (&self->freelist, node) - DUMP_LOG (" %p: %u", node, ++i); - } - } -} - - -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/lib/mpool.h b/src/lib/mpool.h deleted file mode 100644 index f4ab30369..000000000 --- a/src/lib/mpool.h +++ /dev/null @@ -1,182 +0,0 @@ -/* - mpool.h - memory pool for constant sized objects - - Copyright (C) Lumiera.org - 2009, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. -*/ - - -/** @file mpool.h - ** Pooled memory allocator for constant sized objects - ** *Memory Pools* are implemented as clusters of fixed sized elements. New clusters - ** are allocated on demand or manually preallocated with a mpool_reserve() operation. - ** Some efforts are taken to ensure (cache) locality of the provided memory. - ** All functions are reentrant but not threadsafe, if this is desired it is advised to - ** care for proper locking elsewhere. - ** @warning as of 2016, this is a stalled development effort towards a pooling allocator. - ** An initial working draft was created in 2009, but never challenged by any - ** widespread use beyond some test code. We acknowledge that there _will be_ - ** some kind of optimised allocator -- yet for the time being we rely on - ** several preliminary front-ends, which _could be attached_ to such an - ** allocator eventually, but use plain flat heap allocations right now. - */ - -#include -#include -#include "lib/llist.h" -#include "include/logging.h" - -/** - * Function prototype for destroying elements. - * When a memory pool gets destroyed it can call a destructor for any element which is still in the pool. - * Use of such a destructor is optional. - * @param self element to be destroyed - */ -typedef void (*mpool_destroy_fn)(void* self); - - -/** - * memory pool management structure. - * This structure should be considered opaque. - */ -typedef struct mpool_struct mpool; -typedef mpool* MPool; -typedef const mpool* const_MPool; - -struct mpool_struct -{ - llist freelist; - llist clusters; - size_t elem_size; - unsigned elements_per_cluster; - uintptr_t cluster_size; - unsigned elements_free; /* a counter of free elements is the price we pay to support a reserve() operation */ - void* locality; - mpool_destroy_fn destroy; - void *(*malloc_hook)(size_t); - void (*free_hook)(void *); - void* udata; /* free to use by the user, resourcecollector stuff in lumiera*/ -}; - - -extern void *(*mpool_malloc_hook)(size_t size); -extern void (*mpool_free_hook)(void *ptr); - -/** called after a mpool got initialised */ -extern void (*mpool_init_hook) (MPool self); -/** called before a mpool gets destroyed */ -extern void (*mpool_destroy_hook) (MPool self); - -/** - * Initialise a new memory pool. - * Memory pools must be initialised before being used. One can supply - * an optional destructor function for elements, this will be used to destroy elements which are still - * in the pool when it gets destroyed itself. The destructor is _NOT_ called when elements are freed. - * @param self pointer to the memory pool structure to be initialised - * @param elem_size size for a single element - * @param elements_per_cluster how many elements to put into a cluster - * @param dtor pointer to an optional destructor function, may be `NULL` - * @return `self` pointer to the initialised object - */ -MPool -mpool_init (MPool self, size_t elem_size, unsigned elements_per_cluster, mpool_destroy_fn dtor); - - -/** - * destroy a memory pool. - * A memory pool not used anymore should be destroyed. This frees all memory allocated with it. - * When a destructor was provided at construction time, then this destructor is used on all non free elements - * before before the clusters are freed. If no destructor was given then the clusters are just freed. - * The destroyed memory pool behaves as if it was freshly initialised and can be used again, this is - * some kind of exceptional behaviour. - * @param self pointer to an initialised memory pool to be destroyed. - * @return `self` - */ -MPool -mpool_destroy (MPool self); - -/** - * free unused clusters. - */ -MPool -mpool_purge (MPool self); - - -/** - * query number of free elements. - * @param self pointer to the memory pool to be queried - * @return number of elements available in the pool _without allocating a new cluster_ - */ -static inline unsigned -mpool_available (MPool self) -{ - return self->elements_free; -} - - -/** - * preallocate elements. - * Resize the pool that at least the given number of elements are available without cluster reallocations. - * @param self pointer to the memory pool - * @param nelements minimum number of elements to preallocate - * @return self on success or `NULL` on error - */ -MPool -mpool_reserve (MPool self, unsigned nelements); - - -/** - * allocate one element from a MPool. To improve cache locality allocations - * are grouped close together to recent allocations. - * @param self pointer to the memory pool - * @return pointer to the allocated memory on success or `NULL` on error - * @note will never fail when enough space was preallocated - */ -void* -mpool_alloc (MPool self); - - -/** - * allocate one element close to the given reference element. - * To improve cache locality the allocation tries to get an element close by. - * @param self pointer to the memory pool - * @param near reference to another element which should be close to the returned element _(hint only)_ - * @return pointer to the allocated memory on success or `NULL` on error - * @note will never fail when enough space was preallocated - */ -void* -mpool_alloc_near (MPool self, void* near); - - -/** - * free one element. - * Frees the given element and puts it back into the pool for further allocations. - * @param self pointer to the memory pool - * @param element element to be freed - */ -void -mpool_free (MPool self, void* element); - - - -/** diagnostic dump of MPool allocation */ -void -nobug_mpool_dump (const_MPool self, - const int depth, - const struct nobug_context dump_context, - void* extra); - diff --git a/src/lib/mrucache.c b/src/lib/mrucache.c deleted file mode 100644 index cfcc2d25e..000000000 --- a/src/lib/mrucache.c +++ /dev/null @@ -1,64 +0,0 @@ -/* - mrucache.h - most recent used cache - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. -*/ - - -/** @file mrucache.c - ** Implementation of a caching by most recent use. - */ - -#include "lib/safeclib.h" -#include "lib/mrucache.h" - - -LumieraMruCache -lumiera_mrucache_init (LumieraMruCache self, lumiera_cache_destructor_fn destructor_cb) -{ - REQUIRE (self); - llist_init (&self->cache_list); - self->cached = 0; - self->destructor_cb = destructor_cb; - return self; -} - -LumieraMruCache -lumiera_mrucache_destroy (LumieraMruCache self) -{ - LLIST_WHILE_TAIL (&self->cache_list, node) - { - llist_unlink (node); - if (self->destructor_cb) - lumiera_free (self->destructor_cb (node)); - else - lumiera_free (node); - } - self->cached = 0; - return self; -} - -int -lumiera_mrucache_age (LumieraMruCache self, int nelem) -{ - REQUIRE (self); - while (self->cached && nelem--) - lumiera_free (lumiera_mrucache_pop (self)); - - return nelem; -} diff --git a/src/lib/mrucache.h b/src/lib/mrucache.h deleted file mode 100644 index e2ae8c03d..000000000 --- a/src/lib/mrucache.h +++ /dev/null @@ -1,181 +0,0 @@ -/* - mrucache.h - most recent used cache - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. -*/ - - -/** @file mrucache.h - ** Most recent used cache. - ** Elements (addressed by a LList node) are either checked in the cache and thereby subject of aging - ** or checked out under control of the user. Most operations require that the cache is locked. - ** @warning not threadsafe. Locking must be done from usage site. - */ - - -#ifndef LIB_MRUCACHE_H -#define LIB_MRUCACHE_H - -#include "lib/llist.h" - -#include - - -/** - * Callback function used to destroy/cleanup aged elements. - * shall clean the element sufficiently up to be ready for being freed or reused, - * this callback function must be reentrant and prepared to be called twice. - * @param node the llist node used to link cache elements (will be empty at call) - * @return pointer to the begin of the element. - */ -typedef void* (*lumiera_cache_destructor_fn)(LList node); - -struct lumiera_mrucache_struct -{ - llist cache_list; - size_t cached; - lumiera_cache_destructor_fn destructor_cb; -}; -typedef struct lumiera_mrucache_struct lumiera_mrucache; -typedef lumiera_mrucache* LumieraMruCache; - -/** - * Initialise a cache. - * @param self cache to be initialised - * @param destructor_cb function for destroying a cache node, preparing for reuse. - * @return self - */ -LumieraMruCache -lumiera_mrucache_init (LumieraMruCache self, lumiera_cache_destructor_fn destructor_cb); - -/** - * Destroy a cache. - * calls the registered destructor and frees all checked in items. - * @param self cache to be destroyed - * @return self - */ -LumieraMruCache -lumiera_mrucache_destroy (LumieraMruCache self); - - -/** - * Add an element to a mrucache. - * When added the element is subject of aging and must not be accessed anymore. - * To acces elements they have to be checked out again. - * Checkin and checkout operations must be protected by a lock over the cache. - * @param self cache where to checkin - * @param node a llist member in the cached data which is used as queue in the cache - */ -static inline void -lumiera_mrucache_checkin (LumieraMruCache self, LList node) -{ - REQUIRE (self); - REQUIRE (node && llist_is_empty (node)); - llist_insert_head (&self->cache_list, node); - ++self->cached; -} - - -/** - * Schedule an element for fast aging. - * When an element is not longer needed it can be placed at the end of the chache - * aging queue and thus become the first one to be reused when a new element is queried. - * This can be done on a checked out element as well as on an element which is in the cache. - * The cache must be locked for this operation. - * @param self cache - * @param node a llist member in the cached data which is used as queue in the cache - */ -static inline void -lumiera_mrucache_drop (LumieraMruCache self, LList node) -{ - REQUIRE (self); - REQUIRE (node); - - if (llist_is_empty (node)) - { - /* was not in list, we need to count it */ - ++self->cached; - } - else - { - /* speedup loop warning :P, this check is costly */ - REQUIRE (llist_is_member (&self->cache_list, node), "node must be empty or member of cache"); - } - llist_insert_tail (&self->cache_list, node); - - if (self->destructor_cb) - self->destructor_cb (node); -} - - -/** - * Checkout an element from a cache. - * A checked out element is not under cache control anymore until is gets checked in again. - * The code which checked the element out takes ownership of the element. - * The cache must be locked for this operation. - * @param self cache - * @param node a llist member in the cached data which is used as queue in the cache - */ -static inline void -lumiera_mrucache_checkout (LumieraMruCache self, LList node) -{ - REQUIRE (self); - /* speedup loop warning :P, this check is costly */ - REQUIRE (node && llist_is_member (&self->cache_list, node), "Node not in cache"); - llist_unlink (node); - --self->cached; -} - - -/** - * Destroy the oldest element from the cache and return it. - * This function is used to get a new element by deleting the oldest least used one from the cache. - * The cache must be locked for this operation. - * @param self cache - * @return pointer to the uninitialised memory ready for being reused or NULL when no element was available - */ -static inline void* -lumiera_mrucache_pop (LumieraMruCache self) -{ - REQUIRE (self); - if (llist_is_empty (&self->cache_list)) - return NULL; - - LList node = llist_tail (&self->cache_list); - llist_unlink (node); - --self->cached; - - if (self->destructor_cb) - return self->destructor_cb (node); - else - return node; -} - - -/** - * Destroy and free the nelem oldest elements. - * Used to free up resources and memory. - * @param self cache where to free elements. - * @param nelem number of elements wished to be freed - * @return nelem-(numer of elements which got freed), that is 0 if all requested elements got freed - */ -int -lumiera_mrucache_age (LumieraMruCache self, int nelem); - - -#endif /*LIB_MRUCACHE_H*/ diff --git a/src/lib/priqueue.c b/src/lib/priqueue.c deleted file mode 100644 index e32da1d84..000000000 --- a/src/lib/priqueue.c +++ /dev/null @@ -1,260 +0,0 @@ -/* - PriQueue - simple heap based priority queue - - Copyright (C) Lumiera.org - 2011, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - - -/** @file priqueue.c - ** a simple "text book" implementation of a priority queue, based on a binary heap - */ - - -#include "lib/priqueue.h" -#include "include/logging.h" - -#include -#include - - - - - -LumieraPriQueue -lumiera_priqueue_init (LumieraPriQueue self, - size_t element_size, - lumiera_priqueue_cmp_fn cmpfn, - lumiera_priqueue_copy_fn copyfn, - lumiera_priqueue_resize_fn resizefn) -{ - TRACE (priqueue, "%p", self); - - REQUIRE (element_size); - REQUIRE (cmpfn); - - if (self) - { - self->queue = NULL; - self->element_size = element_size; - self->used = self->high_water = self->low_water = 0; - self->cmpfn = cmpfn; - - if (!copyfn) - copyfn = memcpy; - self->copyfn = copyfn; - - if (!resizefn) - resizefn = lumiera_priqueue_clib_resize; - self->resizefn = resizefn; - - self = self->resizefn (self); - } - return self; -} - - - -LumieraPriQueue -lumiera_priqueue_destroy (LumieraPriQueue self) -{ - TRACE (priqueue, "%p", self); - if (self) - { - WARN_IF (self->used, priqueue, "queue was not empty"); - self->used = 0; - self = self->resizefn (self); - } - return self; -} - - - -LumieraPriQueue -lumiera_priqueue_reserve (LumieraPriQueue self, unsigned elements) -{ - TRACE (priqueue, "%p %d", self, elements); - if (self) - { - if (self->used+elements >= self->high_water) - { - self->used += elements; - self = self->resizefn (self); - if (!self) - { - ERROR (priqueue, "resize failed"); - return NULL; - } - self->used -= elements; - } - self->low_water = 0; - } - - return self; -} - - - -LumieraPriQueue -lumiera_priqueue_clib_resize (LumieraPriQueue self) -{ - if (self) - { - if (!self->queue) - { - INFO (priqueue, "%p: initial alloc", self); - self->queue = malloc (64*self->element_size); - ERROR_IF (!self->queue, priqueue, "%p: allocation failed", self); - if (!self->queue) - return NULL; - self->high_water = 64; - } - else - { - if (self->used) - { - if (self->used >= self->high_water) - { - unsigned newwater = self->high_water; - while (self->used >= newwater) - newwater *= 2; - - INFO (priqueue, "%p: resize %d -> %d", self, self->high_water, newwater); - - void* newqueue = realloc (self->queue, self->element_size * newwater); - ERROR_IF (!newqueue, priqueue, "%p: allocation failed", self); - if (!newqueue) - return NULL; - self->queue = newqueue; - self->high_water = newwater; - self->low_water = self->high_water/8-8; - TRACE (priqueue, "%p: low_water: %d", self, self->low_water); - } - else - { - INFO (priqueue, "%p: shrink %d -> %d", self, self->high_water, (self->low_water+8)*4); - void* newqueue = realloc (self->queue, self->element_size * (self->low_water+8)*4); - - ERROR_IF (!newqueue, priqueue, "allocation failed"); - if (!newqueue) - return NULL; - self->queue = newqueue; - self->high_water = (self->low_water+8)*4; - self->low_water = self->high_water/8-8; - TRACE (priqueue, "%p: low_water: %d", self, self->low_water); - } - } - else - { - INFO (priqueue, "%p: freeing", self); - free (self->queue); - self->queue = NULL; - } - } - } - return self; -} - - - -static inline void* -pq_index (LumieraPriQueue self, unsigned nth) -{ - return (char*)self->queue+self->element_size*nth; -} - - -static inline void -pq_up (LumieraPriQueue self, void* tmp) -{ - unsigned i = self->used; - unsigned p = i/2; - - while (p && self->cmpfn (tmp, pq_index(self, p-1)) < 0) - { - self->copyfn (pq_index (self, i-1), pq_index (self, p-1), self->element_size); - i=p; p=i/2; - } - - self->copyfn (pq_index (self, i-1), tmp, self->element_size); -} - - - -LumieraPriQueue -lumiera_priqueue_insert (LumieraPriQueue self, void* element) -{ - TRACE (priqueue, "%p: insert %p", self, element); - - if (self && self->used >= self->high_water) - self = self->resizefn (self); - - if (self) - { - ++self->used; - pq_up (self, element); - } - return self; -} - - - - -static inline void -pq_down (LumieraPriQueue self, void* tmp) -{ - if (!self->used) - return; - - unsigned i = 1; - - while (i <= self->used/2) - { - unsigned n=i+i; - if (nused && self->cmpfn (pq_index(self, n-1), pq_index(self, n)) >= 0) - ++n; - - if (self->cmpfn (tmp, pq_index(self, n-1)) < 0) - break; - - self->copyfn (pq_index (self, i-1), pq_index (self, n-1), self->element_size); - i = n; - } - self->copyfn (pq_index (self, i-1), tmp, self->element_size); -} - - -LumieraPriQueue -lumiera_priqueue_remove (LumieraPriQueue self) -{ - TRACE (priqueue, "%p: remove", self); - - if (self) - { - if (!self->used) - return NULL; - - --self->used; - pq_down (self, pq_index (self, self->used)); - - if (self->used < self->low_water) - self = self->resizefn (self); - } - - return self; -} diff --git a/src/lib/priqueue.h b/src/lib/priqueue.h deleted file mode 100644 index d5f1f5d63..000000000 --- a/src/lib/priqueue.h +++ /dev/null @@ -1,160 +0,0 @@ -/* - PRIQUEUE.h - simple heap based priority queue - - Copyright (C) Lumiera.org - 2011, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -*/ - -/** @file priqueue.h - ** Simple priority queue implementation based on a binary heap. - ** Only 'insert' 'remove' and 'peek' operations are supported. - ** Memory is dynamically managed through an optionally user supplied 'resize' function. - ** Elements in the queue have a user-define size but should kept as small as possible. - ** This is only intended to associate lightweight data such as a key and a pointer, - ** storing the key in the element can save dereferencing cost and thus improve - ** cache locality. - ** - ** @warning elements in the queue get moved in memory, so referencing them is not allowed. - ** - ** @todo we might add operations to change the priority of an arbitrary element or remove - ** any but the topmost element. The idea is to let expired elements sink to the top - ** and just detect and remove them on next access. - ** - ** @see vault::engine::SchedulerFrontend - ** - */ - - -#ifndef LIB_PRIQUEUE_H -#define LIB_PRIQUEUE_H - - -#include "lib/error.h" - - -typedef struct lumiera_priqueue_struct lumiera_priqueue; -typedef lumiera_priqueue* LumieraPriQueue; - -/** function to compare 2 keys, mandatory */ -typedef int (*lumiera_priqueue_cmp_fn)(void*, void*); - -/** function to copy elements, optional. - * Has the same prototype as memcpy which is used by default */ -typedef void *(*lumiera_priqueue_copy_fn)(void *dest, const void *src, size_t n); - -/** called when used hits the high or low water marks and initially by priqueue_init() (with queue==NULL) - * or at priqueue_destroy (with queue != NULL, used elements == 0), optional. - * - * @note must be aware of resizes by more than just incrementing the queue by one - */ -typedef LumieraPriQueue (*lumiera_priqueue_resize_fn) (LumieraPriQueue); - - -/** - * @remarks this structure is not opaque to make it possible - * to implement a low level resize operation - * which has to reallocate the queue and update - * the high and low water marks. - */ -struct lumiera_priqueue_struct -{ - void* queue; - size_t element_size; - unsigned used; - unsigned high_water; ///< elements in the queue - unsigned low_water; ///< size for shrinking the queue - - lumiera_priqueue_cmp_fn cmpfn; - lumiera_priqueue_copy_fn copyfn; - - lumiera_priqueue_resize_fn resizefn; -}; - - - - - - -LumieraPriQueue -lumiera_priqueue_init (LumieraPriQueue self, - size_t element_size, - lumiera_priqueue_cmp_fn cmpfn, - lumiera_priqueue_copy_fn copyfn, - lumiera_priqueue_resize_fn resizefn); - - - -LumieraPriQueue -lumiera_priqueue_destroy (LumieraPriQueue self); - - -/** - * calls resize to match for at least 'elements' in the queue - * and then sets low_water to 0, disabling shrinking - * @note on overflow resize will re-enable low_water if it is not aware of this - */ -LumieraPriQueue -lumiera_priqueue_reserve (LumieraPriQueue self, unsigned elements); - - -/** - * supplied/default resize function based on realloc - * initially allocates an array for 64 elements, - * doubles this when the high water mark is hit, - * shrinks at high_water/8-8 (that is, 64 is the minimum size) - */ -LumieraPriQueue -lumiera_priqueue_clib_resize (LumieraPriQueue self); - - - - -/** - * insert a new element into the priority queue - * the element will be copied - * @return \c NULL on error - */ -LumieraPriQueue -lumiera_priqueue_insert (LumieraPriQueue self, void* element); - - -/** - * @return pointer to the topmost element, `NULL` on empty queue - * @note returned pointer is only valid as long - * as no insert or remove is called - */ -static inline void* -lumiera_priqueue_peek (LumieraPriQueue self) -{ - if (self && self->queue) - return self->queue; - - return NULL; -} - - -/** - * removes the topmost element - * @return \c NULL on error (empty queue, resize failure) - */ -LumieraPriQueue -lumiera_priqueue_remove (LumieraPriQueue self); - - - -#endif/*LIB_PRIQUEUE_H*/ diff --git a/src/lib/reccondition.c b/src/lib/reccondition.c deleted file mode 100644 index b7d0730d5..000000000 --- a/src/lib/reccondition.c +++ /dev/null @@ -1,88 +0,0 @@ -/* - reccondition.c - condition variable, w/ recursive mutex - - Copyright (C) Lumiera.org - 2008, 2009, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. -*/ - -#include "lib/reccondition.h" - -/** - * @file - * Condition variables - */ - - - -static pthread_once_t recursive_mutexattr_once = PTHREAD_ONCE_INIT; -static pthread_mutexattr_t recursive_mutexattr; - -static void recursive_mutexattr_init() -{ - pthread_mutexattr_init (&recursive_mutexattr); - pthread_mutexattr_settype (&recursive_mutexattr, PTHREAD_MUTEX_RECURSIVE); -} - - -LumieraReccondition -lumiera_reccondition_init (LumieraReccondition self, - const char* purpose, - struct nobug_flag* flag, - const struct nobug_context ctx) -{ - if (self) - { - pthread_once (&recursive_mutexattr_once, recursive_mutexattr_init); - - NOBUG_RESOURCE_HANDLE_INIT (self->rh); - NOBUG_RESOURCE_ANNOUNCE_RAW_CTX (flag, "reccond_var", purpose, self, self->rh, ctx) - { - pthread_mutex_init (&self->reccndmutex, &recursive_mutexattr); - pthread_cond_init (&self->cond, NULL); - } - } - return self; -} - - -LumieraReccondition -lumiera_reccondition_destroy (LumieraReccondition self, - struct nobug_flag* flag, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_FORGET_RAW_CTX (flag, self->rh, ctx) - { - if (pthread_mutex_destroy (&self->reccndmutex)) - LUMIERA_DIE (LOCK_DESTROY); - - if (pthread_cond_destroy (&self->cond)) - LUMIERA_DIE (LOCK_DESTROY); - } - } - return self; -} - - -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/lib/reccondition.h b/src/lib/reccondition.h deleted file mode 100644 index 1a4b02828..000000000 --- a/src/lib/reccondition.h +++ /dev/null @@ -1,362 +0,0 @@ -/* - reccondition.h - recursive locked condition variables - - Copyright (C) Lumiera.org - 2008, 2009, 2010, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. -*/ - -#ifndef LUMIERA_RECCONDITION_H -#define LUMIERA_RECCONDITION_H - -#include "lib/error.h" -#include "lib/sectionlock.h" -#include "lib/lockerror.h" - -#include -#include -#include -#include - -/** - * @file - * Condition variables, header - */ - - -/** - * Recursive Condition section. - * Locks the condition mutex, one can use LUMIERA_RECCONDITION_WAIT to wait for signals or - * LUMIERA_RECCONDITION_SIGNAL or LUMIERA_RECCONDITION_BROADCAST to wake waiting threads - * @param nobugflag NoBug flag used to log actions on the condition - * @param cnd Condition variable to be locked - */ -#define LUMIERA_RECCONDITION_SECTION(nobugflag, cnd) \ - for (lumiera_sectionlock NOBUG_CLEANUP(lumiera_sectionlock_ensureunlocked) \ - lumiera_lock_section_ = { \ - cnd, (lumiera_sectionlock_unlock_fn) lumiera_reccondition_unlock \ - NOBUG_ALPHA_COMMA(&NOBUG_FLAG(nobugflag)) NOBUG_ALPHA_COMMA_NULL}; \ - ({ \ - if (lumiera_lock_section_.lock) \ - lumiera_lock_section_.lock = \ - lumiera_reccondition_lock (cnd, &NOBUG_FLAG(nobugflag), \ - &lumiera_lock_section_.rh, NOBUG_CONTEXT); \ - lumiera_lock_section_.lock; \ - }); \ - ({ \ - LUMIERA_RECCONDITION_SECTION_UNLOCK; \ - })) - - - -#define LUMIERA_RECCONDITION_SECTION_CHAIN(nobugflag, cnd) \ - for (lumiera_sectionlock *lumiera_lock_section_old_ = &lumiera_lock_section_, \ - NOBUG_CLEANUP(lumiera_sectionlock_ensureunlocked) lumiera_lock_section_ = { \ - cnd, (lumiera_sectionlock_unlock_fn) lumiera_reccondition_unlock \ - NOBUG_ALPHA_COMMA(&NOBUG_FLAG(nobugflag)) NOBUG_ALPHA_COMMA_NULL}; \ - ({ \ - if (lumiera_lock_section_.lock) \ - { \ - REQUIRE (lumiera_lock_section_old_->lock, "section prematurely unlocked"); \ - lumiera_lock_section_.lock = \ - lumiera_reccondition_lock (cnd, &NOBUG_FLAG(nobugflag), \ - &lumiera_lock_section_.rh, NOBUG_CONTEXT); \ - LUMIERA_SECTION_UNLOCK_(lumiera_lock_section_old_); \ - } \ - lumiera_lock_section_.lock; \ - }); \ - ({ \ - LUMIERA_RECCONDITION_SECTION_UNLOCK; \ - })) - - -#define LUMIERA_RECCONDITION_SECTION_UNLOCK \ - LUMIERA_SECTION_UNLOCK_(&lumiera_lock_section_) - - -/** - * Wait for a condition. - * Must be used inside a RECCONDITION_SECTION. - * @param expr Condition which must become true, else the condition variable goes back into sleep - */ -#define LUMIERA_RECCONDITION_WAIT(expr) \ - while (!(expr)) { \ - REQUIRE (lumiera_lock_section_.lock, "Reccondition mutex not locked"); \ - lumiera_reccondition_wait (lumiera_lock_section_.lock, \ - lumiera_lock_section_.flag, \ - &lumiera_lock_section_.rh, \ - NOBUG_CONTEXT); \ - } while (!(expr)) - - -/** - * Timed wait for a condition. - * Must be used inside a RECCONDITION_SECTION. - * @param expr Recconditon which must become true, else the condition variable goes back into sleep - * @param timeout time when the wait expired - * sets LUMIERA_ERROR_LOCK_TIMEOUT when the timeout passed - */ -#define LUMIERA_RECCONDITION_TIMEDWAIT(expr, timeout) \ - while (!(expr)) { \ - REQUIRE (lumiera_lock_section_.lock, "Reccondition mutex not locked"); \ - if (!lumiera_reccondition_timedwait (lumiera_lock_section_.lock, \ - timeout, \ - lumiera_lock_section_.flag, \ - &lumiera_lock_section_.rh, \ - NOBUG_CONTEXT)) \ - break; \ - } - - - -/** - * Signal a condition variable - * Must be used inside a RECCONDITION_SECTION. - * Wakes one thread waiting on the condition variable - */ -#define LUMIERA_RECCONDITION_SIGNAL \ - do { \ - REQUIRE (lumiera_lock_section_.lock, "Reccondition mutex not locked"); \ - lumiera_reccondition_signal (lumiera_lock_section_.lock, \ - lumiera_lock_section_.flag, \ - NOBUG_CONTEXT); \ - } while (0) - - -/** - * Broadcast a condition variable - * Must be used inside a RECCONDITION_SECTION. - * Wakes all threads waiting on the condition variable - */ -#define LUMIERA_RECCONDITION_BROADCAST \ - do { \ - REQUIRE (lumiera_lock_section_.lock, "Reccondition mutex not locked"); \ - lumiera_reccondition_broadcast (lumiera_lock_section_.lock, \ - lumiera_lock_section_.flag, \ - NOBUG_CONTEXT); \ - } while (0) - - -/** - * Condition variables. Recursive mutex variant. - * - */ -struct lumiera_reccondition_struct -{ - pthread_cond_t cond; - pthread_mutex_t reccndmutex; - RESOURCE_HANDLE (rh); -}; -typedef struct lumiera_reccondition_struct lumiera_reccondition; -typedef lumiera_reccondition* LumieraReccondition; - - -/** - * Initialise a condition variable - * @param self is a pointer to the condition variable to be initialised - * @return self as given - */ -LumieraReccondition -lumiera_reccondition_init (LumieraReccondition self, - const char* purpose, - struct nobug_flag* flag, - const struct nobug_context ctx); - - -/** - * Destroy a condition variable - * @param self is a pointer to the condition variable to be destroyed - * @return self as given - */ -LumieraReccondition -lumiera_reccondition_destroy (LumieraReccondition self, - struct nobug_flag* flag, - const struct nobug_context ctx); - - - -static inline LumieraReccondition -lumiera_reccondition_lock (LumieraReccondition self, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_WAIT_CTX (NOBUG_FLAG_RAW(flag), self->rh, "acquire reccondvar", *handle, ctx) - { - if (pthread_mutex_lock (&self->reccndmutex)) - LUMIERA_DIE (LOCK_ACQUIRE); /* never reached (in a correct program) */ - NOBUG_RESOURCE_STATE_CTX (NOBUG_FLAG_RAW(flag), NOBUG_RESOURCE_RECURSIVE, *handle, ctx) /*{}*/; - } - } - - return self; -} - - -static inline LumieraReccondition -lumiera_reccondition_trylock (LumieraReccondition self, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_TRY_CTX (NOBUG_FLAG_RAW(flag), self->rh, "try acquire reccondvar", *handle, ctx) - { - int err = pthread_mutex_trylock (&self->reccndmutex); - - if (!err) - { - NOBUG_RESOURCE_STATE_CTX (NOBUG_FLAG_RAW(flag), NOBUG_RESOURCE_RECURSIVE, *handle, ctx) /*{}*/; - } - else - { - NOBUG_RESOURCE_LEAVE_RAW_CTX (flag, *handle, ctx) /*{}*/; - lumiera_lockerror_set (err, flag, ctx); - self = NULL; - } - } - } - - return self; -} - - -static inline LumieraReccondition -lumiera_reccondition_timedlock (LumieraReccondition self, - const struct timespec* timeout, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_TRY_CTX (NOBUG_FLAG_RAW(flag), self->rh, "timed acquire reccondvar", *handle, ctx) - { - int err = pthread_mutex_timedlock (&self->reccndmutex, timeout); - - if (!err) - { - NOBUG_RESOURCE_STATE_CTX (NOBUG_FLAG_RAW(flag), NOBUG_RESOURCE_RECURSIVE, *handle, ctx) /*{}*/; - } - else - { - NOBUG_RESOURCE_LEAVE_RAW_CTX (flag, *handle, ctx) /*{}*/; - lumiera_lockerror_set (err, flag, ctx); - self = NULL; - } - } - } - - return self; -} - - -static inline LumieraReccondition -lumiera_reccondition_wait (LumieraReccondition self, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_STATE_RAW_CTX (flag, NOBUG_RESOURCE_WAITING, *handle, ctx) - { - pthread_cond_wait (&self->cond, &self->reccndmutex); - NOBUG_RESOURCE_STATE_RAW_CTX (flag, NOBUG_RESOURCE_RECURSIVE, *handle, ctx) /*{}*/; - } - } - return self; -} - - -static inline LumieraReccondition -lumiera_reccondition_timedwait (LumieraReccondition self, - const struct timespec* timeout, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_STATE_RAW_CTX (flag, NOBUG_RESOURCE_WAITING, *handle, ctx) - { - int err; - do { - err = pthread_cond_timedwait (&self->cond, &self->reccndmutex, timeout); - } while(err == EINTR); - - if (!err) - NOBUG_RESOURCE_STATE_RAW_CTX (flag, NOBUG_RESOURCE_RECURSIVE, *handle, ctx) /*{}*/; - else - { - lumiera_lockerror_set (err, flag, ctx); - self = NULL; - } - } - } - - return self; -} - - - -static inline void -lumiera_reccondition_signal (LumieraReccondition self, - struct nobug_flag* flag, - const struct nobug_context ctx) -{ - NOBUG_TRACE_CTX (NOBUG_FLAG_RAW (flag), ctx, "Signal %p", self); - pthread_cond_signal (&self->cond); -} - - -static inline void -lumiera_reccondition_broadcast (LumieraReccondition self, - struct nobug_flag* flag, - const struct nobug_context ctx) -{ - NOBUG_TRACE_CTX (NOBUG_FLAG_RAW(flag), ctx, "Broadcast %p", self); - pthread_cond_broadcast (&self->cond); -} - - -static inline void -lumiera_reccondition_unlock (LumieraReccondition self, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - NOBUG_REQUIRE_CTX (self, ctx); - - NOBUG_RESOURCE_LEAVE_RAW_CTX (flag, *handle, ctx) - { - if (pthread_mutex_unlock (&self->reccndmutex)) - LUMIERA_DIE (LOCK_RELEASE); /* never reached (in a correct program) */ - } -} - - -#endif -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/lib/rwlock.c b/src/lib/rwlock.c deleted file mode 100644 index 8e293543f..000000000 --- a/src/lib/rwlock.c +++ /dev/null @@ -1,72 +0,0 @@ -/* - rwlock.c - read/write locks - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. -*/ - -#include "lib/error.h" -#include "lib/rwlock.h" - - -/** - * @file - * Read/write locks. - */ - -LumieraRWLock -lumiera_rwlock_init (LumieraRWLock self, - const char* purpose, - struct nobug_flag* flag, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_HANDLE_INIT (self->rh); - NOBUG_RESOURCE_ANNOUNCE_RAW_CTX (flag, "rwlock", purpose, self, self->rh, ctx) - { - pthread_rwlock_init (&self->rwlock, NULL); - } - } - return self; -} - - -LumieraRWLock -lumiera_rwlock_destroy (LumieraRWLock self, - struct nobug_flag* flag, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_FORGET_RAW_CTX (flag, self->rh, ctx) - { - if (pthread_rwlock_destroy (&self->rwlock)) - LUMIERA_DIE (LOCK_DESTROY); - } - } - return self; -} - - -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/lib/rwlock.h b/src/lib/rwlock.h deleted file mode 100644 index 41b2316af..000000000 --- a/src/lib/rwlock.h +++ /dev/null @@ -1,358 +0,0 @@ -/* - rwlock.h - read/write locks - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. -*/ - -#ifndef LUMIERA_RWLOCK_H -#define LUMIERA_RWLOCK_H - -#ifndef _GNU_SOURCE -#error "This header must be included with _GNU_SOURCE or _POSIX_C_SOURCE >= 200112L defined" -#endif - -#include "lib/sectionlock.h" -#include "lib/lockerror.h" - -#include -#include -#include - - -/** - * @file - * Read/write locks, header. - */ - - -/** - * Read locked section. - * readlocks may fail when there are too much readers, one has to check the error afterwards! - */ -#define LUMIERA_RDLOCK_SECTION(nobugflag, rwlck) \ - for (lumiera_sectionlock NOBUG_CLEANUP(lumiera_sectionlock_ensureunlocked) \ - lumiera_lock_section_ = { \ - rwlck, (lumiera_sectionlock_unlock_fn) lumiera_rwlock_unlock \ - NOBUG_ALPHA_COMMA(&NOBUG_FLAG(nobugflag)) NOBUG_ALPHA_COMMA_NULL}; \ - ({ \ - if (lumiera_lock_section_.lock) \ - lumiera_lock_section_.lock = \ - lumiera_rwlock_rdlock (rwlck, &NOBUG_FLAG(nobugflag), \ - &lumiera_lock_section_.rh, NOBUG_CONTEXT); \ - lumiera_lock_section_.lock; \ - }); \ - ({ \ - LUMIERA_RWLOCK_SECTION_UNLOCK; \ - })) - - -#define LUMIERA_RDLOCK_SECTION_CHAIN(nobugflag, rwlck) \ - for (lumiera_sectionlock *lumiera_lock_section_old_ = &lumiera_lock_section_, \ - NOBUG_CLEANUP(lumiera_sectionlock_ensureunlocked) lumiera_lock_section_ = { \ - rwlck, (lumiera_sectionlock_unlock_fn) lumiera_rwlock_unlock \ - NOBUG_ALPHA_COMMA(&NOBUG_FLAG(nobugflag)) NOBUG_ALPHA_COMMA_NULL}; \ - ({ \ - if (lumiera_lock_section_.lock) \ - { \ - REQUIRE (lumiera_lock_section_old_->lock, "section prematurely unlocked"); \ - lumiera_lock_section_.lock = \ - lumiera_rwlock_rdlock (rwlck, &NOBUG_FLAG(nobugflag), \ - lumiera_lock_section_.rh, NOBUG_CONTEXT); \ - LUMIERA_SECTION_UNLOCK_(lumiera_lock_section_old_); \ - } \ - lumiera_lock_section_.lock; \ - }); \ - ({ \ - LUMIERA_RWLOCK_SECTION_UNLOCK; \ - })) - - - -/** - * Write locked section. - */ -#define LUMIERA_WRLOCK_SECTION(nobugflag, rwlck) \ - for (lumiera_sectionlock NOBUG_CLEANUP(lumiera_sectionlock_ensureunlocked) \ - lumiera_lock_section_ = { \ - rwlck, (lumiera_sectionlock_unlock_fn) lumiera_rwlock_unlock \ - NOBUG_ALPHA_COMMA(&NOBUG_FLAG(nobugflag)) NOBUG_ALPHA_COMMA_NULL}; \ - ({ \ - if (lumiera_lock_section_.lock) \ - lumiera_lock_section_.lock = \ - lumiera_rwlock_wrlock (rwlck, &NOBUG_FLAG(nobugflag), \ - &lumiera_lock_section_.rh, NOBUG_CONTEXT); \ - lumiera_lock_section_.lock; \ - }); \ - ({ \ - LUMIERA_RWLOCK_SECTION_UNLOCK; \ - })) - - -#define LUMIERA_WRLOCK_SECTION_CHAIN(nobugflag, rwlck) \ - for (lumiera_sectionlock *lumiera_lock_section_old_ = &lumiera_lock_section_, \ - NOBUG_CLEANUP(lumiera_sectionlock_ensureunlocked) lumiera_lock_section_ = { \ - rwlck, (lumiera_sectionlock_unlock_fn) lumiera_rwlock_unlock \ - NOBUG_ALPHA_COMMA(&NOBUG_FLAG(nobugflag)) NOBUG_ALPHA_COMMA_NULL}; \ - ({ \ - if (lumiera_lock_section_.lock) \ - { \ - REQUIRE (lumiera_lock_section_old_->lock, "section prematurely unlocked"); \ - lumiera_lock_section_.lock = \ - lumiera_rwlock_wrlock (rwlck, &NOBUG_FLAG(nobugflag), \ - lumiera_lock_section_.rh, NOBUG_CONTEXT); \ - LUMIERA_SECTION_UNLOCK_(lumiera_lock_section_old_); \ - } \ - lumiera_lock_section_.lock; \ - }); \ - ({ \ - LUMIERA_RWLOCK_SECTION_UNLOCK; \ - })) - - -#define LUMIERA_RWLOCK_SECTION_UNLOCK \ - LUMIERA_SECTION_UNLOCK_(&lumiera_lock_section_) - - -/** - * RWLock. - * - */ -struct lumiera_rwlock_struct -{ - pthread_rwlock_t rwlock; - RESOURCE_HANDLE (rh); -}; -typedef struct lumiera_rwlock_struct lumiera_rwlock; -typedef lumiera_rwlock* LumieraRWLock; - -/** - * Initialize a rwlock - * @param self is a pointer to the rwlock to be initialised - * @return self as given - */ -LumieraRWLock -lumiera_rwlock_init (LumieraRWLock self, - const char* purpose, - struct nobug_flag* flag, - const struct nobug_context ctx); - -/** - * destroy a rwlock - * @param self is a pointer to the rwlock to be initialised - * @return self on success or NULL at error - */ -LumieraRWLock -lumiera_rwlock_destroy (LumieraRWLock self, - struct nobug_flag* flag, - const struct nobug_context ctx); - - -static inline LumieraRWLock -lumiera_rwlock_rdlock (LumieraRWLock self, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_WAIT_CTX (NOBUG_FLAG_RAW(flag), self->rh, "acquire rwlock for reading", *handle, ctx) - { - int err = pthread_rwlock_rdlock (&self->rwlock); - - if (!err) - { - NOBUG_RESOURCE_STATE_CTX (NOBUG_FLAG_RAW(flag), NOBUG_RESOURCE_SHARED, *handle, ctx) /*{}*/; - } - else - { - NOBUG_RESOURCE_LEAVE_RAW_CTX (flag, *handle, ctx) /*{}*/; - lumiera_lockerror_set (err, flag, ctx); - self = NULL; - } - } - } - - return self; -} - - -static inline LumieraRWLock -lumiera_rwlock_tryrdlock (LumieraRWLock self, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_TRY_CTX (NOBUG_FLAG_RAW(flag), self->rh, "try acquire rwlock for reading", *handle, ctx) - { - int err = pthread_rwlock_tryrdlock (&self->rwlock); - - if (!err) - { - NOBUG_RESOURCE_STATE_CTX (NOBUG_FLAG_RAW(flag), NOBUG_RESOURCE_SHARED, *handle, ctx) /*{}*/; - } - else - { - NOBUG_RESOURCE_LEAVE_RAW_CTX (flag, *handle, ctx) /*{}*/; - lumiera_lockerror_set (err, flag, ctx); - self = NULL; - } - } - } - - return self; -} - - -static inline LumieraRWLock -lumiera_rwlock_timedrdlock (LumieraRWLock self, - const struct timespec* timeout, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_TRY_CTX (NOBUG_FLAG_RAW(flag), self->rh, "timed acquire rwlock for reading", *handle, ctx) - { - int err = pthread_rwlock_timedrdlock (&self->rwlock, timeout); - - if (!err) - { - NOBUG_RESOURCE_STATE_CTX (NOBUG_FLAG_RAW(flag), NOBUG_RESOURCE_SHARED, *handle, ctx) /*{}*/; - } - else - { - NOBUG_RESOURCE_LEAVE_RAW_CTX (flag, *handle, ctx) /*{}*/; - lumiera_lockerror_set (err, flag, ctx); - self = NULL; - } - } - } - - return self; -} - - -static inline LumieraRWLock -lumiera_rwlock_wrlock (LumieraRWLock self, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_WAIT_CTX (NOBUG_FLAG_RAW(flag), self->rh, "acquire rwlock for writing", *handle, ctx) - { - if (pthread_rwlock_wrlock (&self->rwlock)) - LUMIERA_DIE (LOCK_ACQUIRE); /* never reached (in a correct program) */ - - NOBUG_RESOURCE_STATE_CTX (NOBUG_FLAG_RAW(flag), NOBUG_RESOURCE_EXCLUSIVE, *handle, ctx) /*{}*/; - } - } - - return self; -} - - -static inline LumieraRWLock -lumiera_rwlock_trywrlock (LumieraRWLock self, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_TRY_CTX (NOBUG_FLAG_RAW(flag), self->rh, "try acquire rwlock for writing", *handle, ctx) - { - int err = pthread_rwlock_trywrlock (&self->rwlock); - - if (!err) - { - NOBUG_RESOURCE_STATE_CTX (NOBUG_FLAG_RAW(flag), NOBUG_RESOURCE_EXCLUSIVE, *handle, ctx) /*{}*/; - } - else - { - NOBUG_RESOURCE_LEAVE_RAW_CTX (flag, *handle, ctx) /*{}*/; - lumiera_lockerror_set (err, flag, ctx); - self = NULL; - } - } - } - - return self; -} - - -static inline LumieraRWLock -lumiera_rwlock_timedwrlock (LumieraRWLock self, - const struct timespec* timeout, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - if (self) - { - NOBUG_RESOURCE_TRY_CTX (NOBUG_FLAG_RAW(flag), self->rh, "timed acquire rwlock for writing", *handle, ctx) - { - int err = pthread_rwlock_timedwrlock (&self->rwlock, timeout); - - if (!err) - { - NOBUG_RESOURCE_STATE_CTX (NOBUG_FLAG_RAW(flag), NOBUG_RESOURCE_EXCLUSIVE, *handle, ctx) /*{}*/; - } - else - { - NOBUG_RESOURCE_LEAVE_RAW_CTX (flag, *handle, ctx) /*{}*/; - lumiera_lockerror_set (err, flag, ctx); - self = NULL; - } - } - } - - return self; -} - - -static inline void -lumiera_rwlock_unlock (LumieraRWLock self, - struct nobug_flag* flag, - struct nobug_resource_user** handle, - const struct nobug_context ctx) -{ - NOBUG_REQUIRE_CTX (self, ctx); - - NOBUG_RESOURCE_LEAVE_RAW_CTX (flag, *handle, ctx) - { - if (pthread_rwlock_unlock (&self->rwlock)) - LUMIERA_DIE (LOCK_RELEASE); /* never reached (in a correct program) */ - } -} - - - - -#endif -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/lib/safeclib.c b/src/lib/safeclib.c index 2444e7cfa..b6c53dddd 100644 --- a/src/lib/safeclib.c +++ b/src/lib/safeclib.c @@ -29,8 +29,6 @@ #include "lib/error.h" #include "lib/safeclib.h" -#include "vault/resourcecollector.h" - #include #include #include @@ -41,8 +39,76 @@ LUMIERA_ERROR_DEFINE (NO_MEMORY, "Out of Memory!"); +/** + * Resources known to the resource collector + * @todo 10/2023 this is a left-over of the »resource-collector« plan + */ +enum lumiera_resource + { + /** memory blocks, context is a pointer to the size_t required **/ + LUMIERA_RESOURCE_MEMORY, +// /** OS filehandles **/ +// LUMIERA_RESOURCE_FILEHANDLE, +// /** CPU time, as in threads and such **/ +// LUMIERA_RESOURCE_CPU, +// /** mmaped regions **/ +// LUMIERA_RESOURCE_MMAP, +// /** disk space for the storage area, context is a pointer to the filename indication the device **/ +// LUMIERA_RESOURCE_DISKSTORAGE, +// /** disk bandwidth for the storage area, context is a pointer to the filename indication the device **/ +// LUMIERA_RESOURCE_STORAGEBANDWIDTH, +// /** disk space for the caching area, context is a pointer to the filename indication the device **/ +// LUMIERA_RESOURCE_DISKCACHE, +// /** disk bandwidth for the caching area, context is a pointer to the filename indication the device **/ +// LUMIERA_RESOURCE_CACHEBANDWIDTH, -/* placeholder function until the resourcecollector gets hooked in */ + LUMIERA_RESOURCE_END /* last entry */ + }; + + +/** + * Iteration indicator + * @todo 10/2023 this is a left-over of the »resource-collector« plan + * Resource collection works iteratively freeing more and more resources. + * Handlers do not need to obey the request and shall return LUMIERA_RESOURCE_NONE + * which will then continue with the next handler. + * This goes through all available handlers until one returns a higher or same value + * than the current iteration to indicate that it freed enough resources to continue the task. + * Then control is passed back to the calling loop which retries the resource allocation. + * LUMIERA_RESOURCE_PANIC is somewhat special since it will always call all registered handlers + * for all resources, not only the queried one and finally _exit() the application. + * The exact amounts of resources to be freed for ONE, SOME and MANY in intentionally + * kept vague the handlers are free to interpret this in some sensible way. + */ +enum lumiera_resource_try + { + /** No op, returned by a handler when it did nothing **/ + LUMIERA_RESOURCE_NONE, + /** try to free one or really few of this resources **/ + LUMIERA_RESOURCE_ONE, + /** try to free a small reasonable implementation defined amount of resources **/ + LUMIERA_RESOURCE_SOME, + /** try to free a bigger implementation defined amount of resources **/ + LUMIERA_RESOURCE_MANY, + /** free as much as possible **/ + LUMIERA_RESOURCE_ALL, + /** die! **/ + LUMIERA_RESOURCE_PANIC, + /** When a handler gets unregistered it will be called with this value to give it a chance to clean up the user 'data' **/ + LUMIERA_RESOURCE_UNREGISTER + }; + + +/** + * @internal placeholder function in case the resource-collector was not installed + * @todo 10/2023 in the early stages of the Lumiera project a centralised resource pooling + * was considered. On resource shortage, the (planned) resource-collector would attempt + * to reclaim excess resources. This would allow to use resources (e.g. memory) wastefully + * as trade-off for performance gains. + * This concept was never developed beyond an interface draft; most notably it was never + * clarified how excess resources could have been identified on-demand while in-flight. + * Today I view this concept as an immature plan and remove the remaining draft code. + */ static int die_no_mem (enum lumiera_resource which, enum lumiera_resource_try* iteration, void* context) { @@ -51,17 +117,6 @@ die_no_mem (enum lumiera_resource which, enum lumiera_resource_try* iteration, v return 0; /* not reached */ } -static lumiera_resourcecollector_run_fn lumiera_safeclib_resourcecollector_run_hook = die_no_mem; - -void -lumiera_safeclib_set_resourcecollector (void* hook) -{ - if (hook) - lumiera_safeclib_resourcecollector_run_hook = (lumiera_resourcecollector_run_fn)hook; - else - lumiera_safeclib_resourcecollector_run_hook = die_no_mem; -} - void* lumiera_malloc (size_t size) @@ -70,9 +125,11 @@ lumiera_malloc (size_t size) void* o = NULL; if (size) - do + { o = malloc (size); - while (!o && lumiera_safeclib_resourcecollector_run_hook (LUMIERA_RESOURCE_MEMORY, &iteration, &size)); + if (!o) + die_no_mem (LUMIERA_RESOURCE_MEMORY, &iteration, &size); + } return o; } @@ -87,9 +144,11 @@ lumiera_calloc (size_t n, size_t size) size_t gross = n*size; if (n&&size) - do + { o = calloc (n, size); - while (!o && lumiera_safeclib_resourcecollector_run_hook (LUMIERA_RESOURCE_MEMORY, &iteration, &gross)); + if (!o) + die_no_mem (LUMIERA_RESOURCE_MEMORY, &iteration, &gross); + } return o; } @@ -102,9 +161,11 @@ lumiera_realloc (void* ptr, size_t size) void* o = NULL; if (size) - do + { o = realloc (ptr, size); - while (!o && lumiera_safeclib_resourcecollector_run_hook (LUMIERA_RESOURCE_MEMORY, &iteration, &size)); + if (!o) + die_no_mem (LUMIERA_RESOURCE_MEMORY, &iteration, &size); + } return o; } @@ -116,12 +177,13 @@ lumiera_strndup (const char* str, size_t len) enum lumiera_resource_try iteration = LUMIERA_RESOURCE_ONE; void* o = NULL; - do if (str && len) o = strndup (str, len); else o = strdup (""); - while (!o && lumiera_safeclib_resourcecollector_run_hook (LUMIERA_RESOURCE_MEMORY, &iteration, &len)); + + if (!o) + die_no_mem (LUMIERA_RESOURCE_MEMORY, &iteration, &len); return o; } diff --git a/src/lib/slist.h b/src/lib/slist.h deleted file mode 100644 index 8dd89d5f3..000000000 --- a/src/lib/slist.h +++ /dev/null @@ -1,666 +0,0 @@ -/* - * slist.h - simple intrusive cyclic single linked list - * - * Copyright (C) Lumiera.org - * 2009 Anton Yakovlev - * - * This program is free software; you can redistribute it and/or - * modify it under the terms of the GNU General Public License as - * published by the Free Software Foundation; either version 2 of the - * License, or (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program; if not, write to the Free Software - * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - */ - -#ifndef SLIST_H -#define SLIST_H - -#include - -/** - * @file - * Intrusive cyclic single linked list. - * - * List node is a structure, which consists only of a forward pointer. This is - * much easier and makes code much cleaner, than to have forward pointer as is. - * In a empty initialised node, this pointer points to the node itself. Note - * that this pointer can never ever become NULL. - * - * This lists are used by using one node as 'root' node where it's pointer is - * the head pointer to the actual list. Care needs to be taken to ensure not to - * apply any operations meant to be applied to data nodes to the root node. - * This way is the preferred way to use this lists. - * - * Alternatively one can store only a chain of data nodes and use a SList - * pointer to point to the first item (which might be NULL in case no data is - * stored). When using such approach care must be taken since most functions - * below expect lists to have a root node. - * - * Due to nature of single linked list, there's no easy way to implement - * functions, which need reverse passing through a list. But some of L1-list - * interface functions need such ability (for example, when we need to find - * previous element for current element). Because search of previous element - * requires visiting of exactly N-1 nodes (where N is length of L1-list), we - * use root node as start point. This gives to us probability of visiting - * 1 <= C <= N-1 nodes, and, thus, speed up search. - * - * This header can be used in 2 different ways: - * 1) (preferred) just including it provides all functions as static inlined - * functions. This is the default - * 2) #define LLIST_INTERFACE before including this header gives only the declarations - * #define LLIST_IMPLEMENTATION before including this header yields in definitions - * this can be used to generate a library. This is currently untested and not - * recommended. - * The rationale for using inlined functions is that most functions are very - * small and likely to be used in performance critical parts. Inlining can give - * a huge performance and optimisation improvement here. The few functions - * which are slightly larger are expected to be the less common used ones, so - * inlining them too shouldn't be a problem either. - */ - -/* TODO __STDC_VERSION__ 199901L -150) This macro was not specified in ISO/IEC 9899:1990 and was specified as 199409L in -ISO/IEC 9899/AMD1:1995. The intention is that this will remain an integer constant of type long -int that is increased with each revision of this International Standard. -*/ - -#ifdef HAVE_INLINE -# define SLIST_MACRO static inline -#else -# ifdef __GNUC__ -# define SLIST_MACRO static __inline__ -# else -# define SLIST_MACRO static -# endif -#endif - -#if defined(SLIST_INTERFACE) -/* only the interface is generated */ -#define SLIST_FUNC(proto, ...) proto -#elif defined(SLIST_IMPLEMENTATION) -/* generate a non inlined implementation */ -#define SLIST_FUNC(proto, ...) proto { __VA_ARGS__ } -#else -/* all functions are macro-like inlined */ -#define SLIST_FUNC(proto, ...) SLIST_MACRO proto { __VA_ARGS__ } -#endif - -/* - * Type of a slist node. - */ - -#ifndef SLIST_DEFINED -#define SLIST_DEFINED - -struct slist_struct { - struct slist_struct* next; -}; - -#endif - -typedef struct slist_struct slist; -typedef slist* SList; -typedef const slist* const_SList; -typedef slist** SList_ref; - -/** - * Macro to instantiate a local llist. - * - * @param name of the slist node - */ - -#define SLIST_AUTO( name ) slist name = { &name } - -/* - * some macros for convenience - */ - -#define slist_head slist_next -#define slist_insert_head( list, element ) slist_insert( list, element ) - -/** - * Сast back from a member of a structure to a pointer of the structure. - * - * Example: - * - * struct point { - * int x; - * int y; - * slist list; - * }; - * - * SList points = ...; // some initialization; must be the root of our list - * - * SLIST_FOREACH( points, current_node ) { - * struct point* current_point = SLIST_TO_STRUCTP( current_node, struct point, list ); - * printf( "point = ( %d, %d )\n", current_point -> x, current_point -> y ); - * } - * - * @param list is a pointer to the SList member of the linked structures - * @param type is type name of the linked structures - * @param member is a name of the SList member of the linked structures - */ - -#define SLIST_TO_STRUCTP( list, type, member ) \ - ( ( type* ) ( ( ( char* )( list ) ) - offsetof( type, member ) ) ) - -/** - * Iterate forward over a list. - * - * @param list the root node of the list to be iterated - * @param node pointer to the iterated node - */ - -#define SLIST_FOREACH( list, node ) \ - for ( SList node = slist_head( list ); ! slist_is_end( node, list ); slist_forward( &node ) ) - -/** - * Iterate forward over a range. - * - * @param start first node to be interated - * @param end node after the last node be iterated - * @param node pointer to the iterated node - */ - -#define SLIST_FORRANGE( start, end, node ) \ - for ( SList node = start; node != end; slist_forward( &node ) ) - -/** - * Consume a list from head. - * The body of this statement should remove the head from the list, else it would be a infinite loop - * - * @param list the root node of the list to be consumed - * @param head pointer to the head node - */ - -#define SLIST_WHILE_HEAD( list, head ) \ - for ( SList head = slist_head( list ); ! slist_is_empty( list ); head = slist_head( list ) ) - -/** - * Initialise a new llist. - * Must not be applied to a list node which is not empty! Lists need to be initialised - * before any other operation on them is called. - * - * @param list node to be initialised - */ - -SLIST_FUNC ( - void slist_init( SList list ), - list -> next = list; -); - -/** - * Check if a node is not linked with some other node. - */ - -SLIST_FUNC ( - int slist_is_empty( const_SList list ), - return list -> next == list; -); - -/** - * Check if self is the only node in a list or self is not in a list. - * - * Warning: - * There's no check for empty list, so if you have a list with no items, - * you'll get seg fault here. - * - * @param list is root node of the list to be checked - */ - -SLIST_FUNC ( - int slist_is_single( const_SList list ), - return list -> next -> next == list; -); - -/** - * Check for the head of a list. - * - * @param list is root node of the list - * @param head is expected head of the list - */ - -SLIST_FUNC ( - int slist_is_head( const_SList list, const_SList head ), - return list -> next == head; -); - -/** - * Check for the end of a list. - * The end is by definition one past the tail of a list, which is the root node itself. - * - * @param list is root node of the list - * @param end is expected end of the list - */ - -SLIST_FUNC ( - int slist_is_end( const_SList list, const_SList end ), - return list == end; -); - -/** - * Check if a node is a member of a list. - * - * @param list is root node of the list - * @param member is node to be searched - */ - -SLIST_FUNC ( - int slist_is_member( const_SList list, const_SList member ), - for ( const_SList i = member -> next; i != member; i = i -> next ) { - if ( i == list ) { - return 1; - } - } - return 0; -); - -/** - * Check the order of elements in a list. - * - * @param list is root node of the list - * @param before is expected to be before after - * @param after is expected to be after before - */ - -SLIST_FUNC ( - int slist_is_before_after( const_SList list, const_SList before, const_SList after ), - for ( const_SList i = before -> next; i != list; i = i -> next ) { - if ( i == after ) { - return 1; - } - } - return 0; -); - -/** - * Count the nodes of a list. - * - * @param list is root node of the list - * @return number of nodes in `list` - */ - -SLIST_FUNC ( - unsigned slist_count( const_SList list ), - unsigned cnt = 0; - for ( const_SList i = list; i -> next != list; ++cnt, i = i -> next ) { - ; - } - return cnt; -); - -/** - * Get next node. - * Will not stop at tail. - * - * @param node is current node - * @return node after current node - */ - -SLIST_FUNC ( - SList slist_next( const_SList node ), - return node -> next; -); - -/** - * Get previous node. - * - * @param list is root node of the list - * @param node is current node - * @return node before current node - */ - -SLIST_FUNC ( - SList slist_prev( SList list, SList node ), - while ( list -> next != node ) { - list = list -> next; - } - return list; -); - -/** - * Remove a node from a list. - * - * @param list is root node of the list - * @param node to be removed - * @return node - */ - -SLIST_FUNC ( - SList slist_unlink( SList list, SList node ), - SList prev_node = slist_prev( list, node ); - prev_node -> next = node -> next; - return node -> next = node; -); - -/** - * Insert a node after another. - * - * @param head is node after which we want to insert - * @param node is node which shall be inserted after `head`. Could already linked to a list from where it will be removed. - * @return head - */ - -SLIST_FUNC ( - SList slist_insert( SList head, SList node ), - if ( ! slist_is_empty( node ) ) { - slist_unlink( node, node ); - } - node -> next = head -> next; - head -> next = node; - return head; -); - -/** - * Move the content of a list after a node in another list. - * - * @param xnode is node after which we want to insert a list - * @param ylist is root node of the list which shall be inserted after self. This list will be empty after call. - * @return xnode - */ - -SLIST_FUNC ( - SList slist_insert_list( SList xnode, SList ylist ), - if ( ! slist_is_empty( ylist ) ) { - SList tail = slist_prev( ylist, ylist ); // search for the Y list tail - tail -> next = xnode -> next; - xnode -> next = ylist -> next; - - ylist -> next = ylist; // clear the Y list - } - return xnode; -); - -/** - * Move a range of nodes after a given node. - * - * @param node is node after which the range shall be inserted - * @param start first node in range to be moved - * @param end node after the last node of the range - * @return node - */ - -SLIST_FUNC ( - SList slist_insert_range( SList node, SList start, SList end ), - // insert range - SList tail = slist_prev( start, end ); // search for the end of range - tail -> next = node -> next; - node -> next = start -> next; - // fix list - start -> next = end; - return node; -); - -/** - * Swap a node with its next node. - * Advancing will not stop at tail, one has to check that if this is intended. - * - * @param list is root node of the list - * @param node is node to be advaced - * @return node - */ - -SLIST_FUNC ( - SList slist_advance( SList list, SList node ), - SList prev = slist_prev( list, node ); - prev -> next = node -> next; - node -> next = node -> next -> next; - prev -> next -> next = node; - return node; -); - -/** - * Advance a pointer to a node to its next node. - * - * @param node pointer-to-pointer to the current node. `node` will point to the next node after this call. - */ - -SLIST_FUNC ( - void slist_forward( SList_ref node ), - *node = ( *node ) -> next; -); - -/** - * Get the nth element of a list (this function does not stop at head/tail). - * - * @param list is root node of the list to be queried - * @param n is number of element to find - * @return |n|-th element of list - */ - -SLIST_FUNC ( - SList slist_get_nth( SList list, unsigned int n ), - while ( n-- > 0 ) { - list = slist_next( list ); - } - return list; -); - -/** - * Get the nth element of a list with a stop node. - * - * @param list is root node of the list to be queried - * @param n is number of element to find - * @param stop is node which will abort the iteration - * @return |n|-th element of list or NULL if `stop` node has been reached - */ - -SLIST_FUNC ( - SList slist_get_nth_stop( SList list, unsigned int n, const_SList stop ), - while ( n-- > 0 ) { - list = slist_next( list ); - if ( list == stop ) { - return NULL; - } - } - return list; -); - -/** - * Sort a list. - * - * This is iterative version of bottom-up merge sort for (L1/L2) linked-list: - * + there's no recursion - * + there's no extra stackspace allocation (only a few bytes for locals) - * Such implementation should be optimal and fast enough. - * - * Maybe this function is too big for inlining (though I don't think so), so - * maybe somebody can make it smaller without losing perfomance? ;) - * - * @param list is root node of a list to be sorted - * @param cmp is compare function of 2 SList items - * @return list - */ - -typedef int ( *slist_cmpfn )( const_SList a, const_SList b ); - -SLIST_FUNC ( - SList slist_sort( SList list, slist_cmpfn cmp ), - if ( ! slist_is_single( list ) ) { - - unsigned int length = slist_count( list ); - - // `max_size` is a half of minimum power of 2, greater of equal to `length` - // ( 2 * max_size = 2^k ) >= length - // We need `max_size` value for proper binary division of a list for sorting. - - unsigned long long max_size = 1; - while ( ( max_size << 1 ) < length ) { - max_size <<= 1; - } - - // The main idea of bottom-up merge sort is sequential merging of each pair - // of sequences of { 1, .. 2^k, .. max_size } length. That's all. :) - - for ( unsigned int size = 1; size <= max_size; size <<= 1 ) { - - // On each iteration: - // * `result` points to the current node of global (merged/sorted) list. - // thus, we can holds all nodes are linked. - // * `left` and `right` points to begin of (sub)lists for merging. - - SList result = list; - SList left = list -> next; - SList right; - - // Process each pairs of sequences of size=2^k length. - - for ( unsigned int position = 0; position < length; position += size + size ) { - - right = slist_get_nth_stop( left, size, list ); - - unsigned int size_left = size; - unsigned int size_right = right == NULL ? 0 : size; - - // Here we have 2 sublists of `size_left` and `size_right` sizes. - // Implementation of `merge` function is next three loops. - - while ( ( size_left > 0 ) && ( size_right > 0 ) ) { - - if ( cmp( left, right ) <= 0 ) { - - result -> next = left; - - left = left -> next; - if ( left == list ) { - size_left = 0; - } else { - size_left--; - } - - } else { - - result -> next = right; - - right = right -> next; - if ( right == list ) { - size_right = 0; - } else { - size_right--; - } - - } - - result = result -> next; - - } - - while ( size_left > 0 ) { - result -> next = left; - result = left; - - left = left -> next; - if ( left == list ) { - break; - } - size_left--; - } - - while ( size_right > 0 ) { - result -> next = right; - result = right; - - right = right -> next; - if ( right == list ) { - break; - } - size_right--; - } - - // go to begin of next pair of sequences - - left = right; - - } - - // here `result` points to the last node of a list. - // we wanna keep cyclic list. - - result -> next = list; - - } - - } - return list; -) - -/** - * Find the first occurence of an element in a list. - * Does not change the order of a list. - * - * @param list is root node of a list to be searched - * @param pattern is template for the element being searched - * @param cmp is compare function of 2 SList items - * @return pointer to the found SList element or NULL if nothing found - */ - -SLIST_FUNC ( - SList slist_find( const_SList list, const_SList pattern, slist_cmpfn cmp ), - SLIST_FOREACH( list, node ) { - if ( cmp( node, pattern ) == 0 ) { - return node; - } - } - return NULL; -) - -/** - * Find the first occurence of an element in an unsorted list. - * - * Searches the list until it finds the searched element and moves it then to - * the head. Useful if the order of the list is not required and few elements - * are frequently searched. - * - * @param list is root node of a list to be searched - * @param pattern is template for the element being searched - * @param cmp is compare function of 2 SList items - * @return pointer to the found SList element (head) or NULL if nothing found - */ - -SLIST_FUNC ( - SList slist_ufind( SList list, const_SList pattern, slist_cmpfn cmp ), - SLIST_FOREACH( list, node ) { - if ( cmp( node, pattern ) == 0 ) { - slist_insert_head( list, node ); - return node; - } - } - return NULL; -) - -/** - * Find the first occurence of an element in a sorted list. - * - * Searches the list until it finds the searched element, exits searching when - * found an element biggier than the searched one. - * - * @param list is root node of a list to be searched - * @param pattern is template for the element being searched - * @param cmp is compare function of 2 SList items - * @return pointer to the found SList element (head) or NULL if nothing found - */ - -SLIST_FUNC ( - SList slist_sfind( const_SList list, const_SList pattern, slist_cmpfn cmp ), - SLIST_FOREACH( list, node ) { - - int result = cmp( node, pattern ); - - if ( result == 0 ) { - return node; - } else if ( result > 0 ) { - break; - } - - } - return NULL; -) - -#endif /* SLIST_H */ diff --git a/src/lib/sync.hpp b/src/lib/sync.hpp index 1e3fbae8d..6e9c885b6 100644 --- a/src/lib/sync.hpp +++ b/src/lib/sync.hpp @@ -26,7 +26,7 @@ ** The actual locking, signalling and waiting is implemented by delegating to the ** a _mutex_ and for waiting also to a _condition variable_ as provided by the C++ ** standard library. The purpose of the Sync baseclass is to provide a clear and simple - ** „everyday“ concurrency coordination based on the object monitor pattern. This + ** „everyday“ concurrency coordination based on the *Object Monitor Pattern*. This ** pattern describes a way of dealing with synchronisation known to play well with ** scoping, encapsulation and responsibility for a single purpose. For performance ** critical code, other solutions (e.g. Atomics) might be preferable. @@ -45,7 +45,7 @@ ** It must be awakened from another thread by invoking `notify_one|all` and will ** then re-check the condition predicate. The `wait_for` variant allows to set ** a timeout to limit the sleep state, which implies however that the call may - ** possibly return `false` in case the condition predicate is not (yet) fulfilled. + ** possibly return `false` in case the condition predicate is not (yet) fulfilled. ** @note ** - It is important to select a suitable parametrisation of the monitor. ** This is done by specifying one of the defined policy classes. @@ -58,13 +58,12 @@ ** when adding the monitor to a given class may cause size problems. ** - There is a special variant of the Lock guard called ClassLock, which ** can be used to lock based on a type, not an instance. - ** - ** @todo WIP-WIP 10/2023 switch from POSIX to C++14 ///////////////////////////////////////////////////////TICKET #1279 : also clean-up the Object-Monitor implementation + ** ** [wait-on-condition]: https://en.cppreference.com/w/cpp/thread/condition_variable_any/wait ** @see mutex.h ** @see sync-locking-test.cpp ** @see sync-waiting-test.cpp - ** @see \ref asset::AssetManager::reg() "usage example: asset registration" + ** @see \ref steam::control::DispatcherLoop "usage example: receiving session coomands" ** @see \ref subsystem-runner.hpp "usage example: subsystem start/shutdown" */ @@ -75,125 +74,18 @@ #include "lib/error.hpp" #include "lib/nocopy.hpp" #include "lib/util.hpp" - + #include #include #include - namespace lib { /** Helpers and building blocks for Monitor based synchronisation */ namespace sync { - - - - /* ========== adaptation layer for accessing backend/system level code ============== */ - - struct Wrapped_ExclusiveMutex - : util::NonCopyable - { - std::mutex mutex_; - - protected: - Wrapped_ExclusiveMutex() = default; - - void - lock() - { - mutex_.lock(); - } - - void - unlock() - { - mutex_.unlock(); - } - }; - - - - struct Wrapped_RecursiveMutex - : util::NonCopyable - { - std::recursive_mutex mutex_; - - protected: - Wrapped_RecursiveMutex() = default; - - void - lock() - { - mutex_.lock(); - } - - void - unlock() - { - mutex_.unlock(); - } - }; - - - template - struct Wrapped_Condition - : MTX - { - std::condition_variable_any cond_; - - protected: - Wrapped_Condition() = default; - - void - wait() - { - cond_.wait (this->mutex_); - } - - template - bool - timedwait (std::chrono::duration const& timeout) - { - auto ret = cond_.wait_for (this->mutex_, timeout); - return (std::cv_status::no_timeout == ret); - } - - void signal() { cond_.notify_one(); } - void broadcast() { cond_.notify_all(); } - }; - - - - - - - - /* ========== abstractions defining the usable synchronisation primitives ============== */ - - template - class Mutex - : protected MTX - { - protected: - Mutex () = default; - - public: - void - acquire() - { - MTX::lock (); - } - - void - release() - { - MTX::unlock (); - } - }; - - + /* ========== policies to define required degree of locking functionality ============== */ template struct Condition @@ -238,33 +130,7 @@ namespace lib { - - /* ==== functor types for defining the waiting condition ==== */ - - typedef volatile bool& Flag; - - struct BoolFlagPredicate - { - Flag flag_; - BoolFlagPredicate (Flag f) : flag_(f) {} - - bool operator() () { return flag_; } - }; - - - template - struct BoolMethodPredicate - { - typedef bool (X::*Method)(void); - - X& instance_; - Method method_; - BoolMethodPredicate (X& x, Method m) : instance_(x), method_(m) {} - - bool operator() () { return (instance_.*method_)(); } - }; - - + /* ========== Object Monitor ============== */ /** * Object Monitor for synchronisation and waiting. @@ -311,8 +177,6 @@ namespace lib { - - /* Interface to be used by client code: * Inherit from class Sync with a suitable Policy. * Then use the embedded Lock class. @@ -426,7 +290,5 @@ namespace lib { }; - - } // namespace lumiera #endif diff --git a/src/vault/backend.c b/src/vault/backend.c deleted file mode 100644 index 17f68cdde..000000000 --- a/src/vault/backend.c +++ /dev/null @@ -1,182 +0,0 @@ -/* - Backend - common lumiera vault layer facilities - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - - -/** @file backend.c - ** Lumiera Vault-Layer: implementation of global initialisation and services. - */ - - -#include "include/logging.h" -#include "lib/safeclib.h" -#include "lib/tmpbuf.h" -#include "lib/mpool.h" - -#include "vault/backend.h" -#include "common/config.h" -#include "vault/filehandlecache.h" -#include "vault/filedescriptor.h" -#include "vault/filedescriptorregistry.h" -#include "vault/mmapcache.h" -#include "vault/threadpool.h" -#include "vault/resourcecollector.h" - -#include -#include - - - - -static enum lumiera_resource_try -lumiera_backend_mpool_purge (enum lumiera_resource_try itr, void* data, void* context); - -static void -lumiera_backend_resourcecollector_register_mpool (MPool self); - -static void -lumiera_backend_resourcecollector_unregister_mpool (MPool self); - - -size_t lumiera_backend_pagesize; - - -int -lumiera_backend_init (void) -{ - - TRACE (backend_dbg); - lumiera_mutex_init (&lumiera_filecreate_mutex, "fileaccess", &NOBUG_FLAG (mutex_dbg), NOBUG_CONTEXT); - - lumiera_resourcecollector_init (); - - /* hook the resourcecollector into the mpool*/ - mpool_malloc_hook = lumiera_malloc; - mpool_free_hook = lumiera_free; - mpool_init_hook = lumiera_backend_resourcecollector_register_mpool; - mpool_destroy_hook = lumiera_backend_resourcecollector_unregister_mpool; - - /* hook the resourcecollector into the safeclib allocation functions */ - lumiera_safeclib_set_resourcecollector (lumiera_resourcecollector_run); - - PLANNED("The resourcecollector aborts by default when there is no final strategy for recovery, TODO: initiate sane shutdown"); - - lumiera_threadpool_init (); - PLANNED ("hook threadpool into the resourcecollector (maybe in threadpool_init() instead here"); - - lumiera_filedescriptorregistry_init (); - - lumiera_backend_pagesize = sysconf(_SC_PAGESIZE); - - /////////////////////////////////////////////////////////////////////TICKET #838 add config options to override following defaults" - - - const char* filehandles = lumiera_tmpbuf_snprintf (SIZE_MAX, - "vault.file.max_handles = %d", - /* roughly 2/3 of all available filehandles are managed by the Lumiera Vault */ - (sysconf (_SC_OPEN_MAX)-10)*2/3); - - lumiera_config_setdefault (filehandles); - - long long max_entries; - lumiera_config_number_get ("vault.file.max_handles", &max_entries); - lumiera_filehandlecache_new (max_entries); - -#if SIZE_MAX <= 4294967295UL - lumiera_config_setdefault ("vault.mmap.as_limit = 3221225469"); -#else - lumiera_config_setdefault ("vault.mmap.as_limit = 211106232532992"); -#endif - - struct rlimit as_rlimit; - getrlimit (RLIMIT_AS, &as_rlimit); - - long long as_limit = (long long)as_rlimit.rlim_cur; - if (as_rlimit.rlim_cur == RLIM_INFINITY) - { - lumiera_config_number_get ("vault.mmap.as_limit", &as_limit); - } - else - { - INFO (vault, "address space limited to %luMiB", as_rlimit.rlim_cur/1024/1024); - } - - lumiera_mmapcache_new (as_limit); - - return 0; -} - - -void -lumiera_backend_destroy (void) -{ - TRACE (backend_dbg); - - lumiera_mmapcache_delete (); - lumiera_filehandlecache_delete (); - lumiera_filedescriptorregistry_destroy (); - lumiera_threadpool_destroy (); - - lumiera_safeclib_set_resourcecollector (NULL); - - mpool_init_hook = NULL; - mpool_destroy_hook = NULL; - mpool_malloc_hook = malloc; - mpool_free_hook = free; - - lumiera_resourcecollector_destroy (); - - lumiera_mutex_destroy (&lumiera_filecreate_mutex, &NOBUG_FLAG (mutex_dbg), NOBUG_CONTEXT); -} - - -static enum lumiera_resource_try -lumiera_backend_mpool_purge (enum lumiera_resource_try itr, void* data, void* context) -{ - (void) context; - (void) data; - (void) itr; - ///////////////////////////////////////////////////////////TICKET #837 actually implement mpool purging - return LUMIERA_RESOURCE_NONE; -} - -static void -lumiera_backend_resourcecollector_register_mpool (MPool self) -{ - self->udata = - lumiera_resourcecollector_register_handler (LUMIERA_RESOURCE_MEMORY, lumiera_backend_mpool_purge, self); -} - -static void -lumiera_backend_resourcecollector_unregister_mpool (MPool self) -{ - lumiera_resourcehandler_unregister ((LumieraResourcehandler) self->udata); -} - - - -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/backend.h b/src/vault/backend.h deleted file mode 100644 index b95014020..000000000 --- a/src/vault/backend.h +++ /dev/null @@ -1,54 +0,0 @@ -/* - BACKEND.h - common lumiera vault definitions - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -*/ - - -/** @file backend.h - ** Lumiera Vault-Layer: global initialisation and definitions. - */ - - -#ifndef VAULT_BACKEND_H -#define VAULT_BACKEND_H - -#include "lib/mutex.h" - -#include - - - -extern size_t lumiera_backend_pagesize; - -/** - * Protect lookup and creation of files. - * Trying to access a nonexistent file with O_CREAT would be racy. - * Defined in filedescriptor.c - */ -extern lumiera_mutex lumiera_filecreate_mutex; - - -int -lumiera_backend_init (void); - -void -lumiera_backend_destroy (void); - -#endif /*VAULT_BACKEND_H*/ diff --git a/src/vault/file-handling.c b/src/vault/file-handling.c deleted file mode 100644 index 92cba38b4..000000000 --- a/src/vault/file-handling.c +++ /dev/null @@ -1,241 +0,0 @@ -/* - FILE-HANDLING - file access and handling - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - - -/** @file file-handling.c - ** Implementation of file management functions - */ - - -#include "include/logging.h" -#include "lib/mutex.h" -#include "lib/safeclib.h" - -#include "vault/file.h" -#include "vault/filehandlecache.h" - -#include -#include - - -LUMIERA_ERROR_DEFINE (FILE_CHANGED, "File changed unexpected"); -LUMIERA_ERROR_DEFINE (FILE_NOMMAPINGS, "MMapings (chunksize/bias) not initialised"); - - - -LumieraFile -lumiera_file_init (LumieraFile self, const char* name, int flags) -{ - TRACE (file_dbg); - - if (self) - { - llist_init (&self->node); - - if (!(self->descriptor = lumiera_filedescriptor_acquire (name, flags, &self->node))) - return NULL; - - self->name = lumiera_strndup (name, PATH_MAX); - } - - return self; -} - -LumieraFile -lumiera_file_destroy (LumieraFile self, int do_unlink) -{ - TRACE (file_dbg); - - lumiera_filedescriptor_release (self->descriptor, self->name, &self->node); - if (do_unlink) - unlink (self->name); - - lumiera_free (self->name); - return self; -} - - -LumieraFile -lumiera_file_new (const char* name, int flags) -{ - TRACE (file_dbg); - TRACE (file, "opening file '%s' with flags '%x'", name, flags); - - LumieraFile self = lumiera_malloc (sizeof (lumiera_file)); - if (!lumiera_file_init (self, name, flags)) - { - lumiera_free (self); - self = NULL; - } - - return self; -} - -void -lumiera_file_delete (LumieraFile self) -{ - TRACE (file_dbg); - TRACE (file, "close file '%s'", self->name); - lumiera_free (lumiera_file_destroy (self, 0)); -} - - -void -lumiera_file_delete_unlink (LumieraFile self) -{ - TRACE (file_dbg); - TRACE (file, "close and unlink file '%s'", self->name); - lumiera_free (lumiera_file_destroy (self, 1)); -} - - -int -lumiera_file_handle_acquire (LumieraFile self) -{ - TRACE (file_dbg); - REQUIRE (self); - REQUIRE (self->descriptor); - - return lumiera_filedescriptor_handle_acquire (self->descriptor); -} - - -void -lumiera_file_handle_release (LumieraFile self) -{ - TRACE (file_dbg); - REQUIRE (self); - REQUIRE (self->descriptor); - - return lumiera_filedescriptor_handle_release (self->descriptor); -} - - -size_t -lumiera_file_set_chunksize_bias (LumieraFile self, size_t chunksize, size_t bias) -{ - if (chunksize && !self->descriptor->mmapings) - self->descriptor->mmapings = lumiera_mmapings_new (self, chunksize, bias); - - return self->descriptor->mmapings->chunksize; -} - - -size_t -lumiera_file_chunksize_get (LumieraFile self) -{ - if (!self->descriptor->mmapings) - { - LUMIERA_ERROR_SET (file, FILE_NOMMAPINGS, lumiera_filedescriptor_name (self->descriptor)); - return 0; - } - - return self->descriptor->mmapings->chunksize; -} - - -size_t -lumiera_file_bias_get (LumieraFile self) -{ - if (!self->descriptor->mmapings) - { - LUMIERA_ERROR_SET (file, FILE_NOMMAPINGS, lumiera_filedescriptor_name (self->descriptor)); - return 0; - } - - return self->descriptor->mmapings->bias; -} - - -LumieraMMapings -lumiera_file_mmapings (LumieraFile self) -{ - if (!self->descriptor->mmapings) - LUMIERA_ERROR_SET (file, FILE_NOMMAPINGS, lumiera_filedescriptor_name (self->descriptor)); - - return self->descriptor->mmapings; -} - - -int -lumiera_file_checkflags (LumieraFile self, int flags) -{ - return self->descriptor->flags & flags; -} - - -LumieraMMap -lumiera_file_mmap_acquire (LumieraFile self, off_t start, size_t size) -{ - TRACE (file_dbg); - return lumiera_mmapings_mmap_acquire (lumiera_file_mmapings (self), self, start, size); -} - - -void -lumiera_file_release_mmap (LumieraFile self, LumieraMMap map) -{ - TRACE (file_dbg); - lumiera_mmapings_release_mmap (lumiera_file_mmapings (self), map); -} - - - -LumieraFile -lumiera_file_rdlock (LumieraFile self) -{ - if (self && !lumiera_filedescriptor_rdlock (self->descriptor)) - return NULL; - - return self; -} - - -LumieraFile -lumiera_file_wrlock (LumieraFile self) -{ - if (self && !lumiera_filedescriptor_wrlock (self->descriptor)) - return NULL; - - return self; -} - - -LumieraFile -lumiera_file_unlock (LumieraFile self) -{ - if (self && !lumiera_filedescriptor_unlock (self->descriptor)) - return NULL; - - return self; -} - - - - -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/file.h b/src/vault/file.h deleted file mode 100644 index 3cd4aefec..000000000 --- a/src/vault/file.h +++ /dev/null @@ -1,281 +0,0 @@ -/* - FILE.h - interface to files by filename - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -*/ - - -/** @file file.h - ** File management. - ** Handling Files is split into different classes: - ** 1. The 'lumiera_file' class which acts as interface to the outside for managing files. - ** 'lumiera_file' is addressed by the name of the file. Since files can have more than one name (hardlinks) - ** many 'lumiera_file' can point to a single 'lumiera_filedescriptor' - ** 2. The 'lumiera_filedescriptor' class which does the real work managing the file in the back. - ** 3. Since OS-filehandles are a limited resource we access the lazily as 'lumiera_filehandle' which are - ** managed in a 'lumiera_filehandlecache' - */ - -#ifndef VAULT_FILE_H -#define VAULT_FILE_H - - -#include "lib/mutex.h" -#include "lib/llist.h" -#include "lib/error.h" - - -LUMIERA_ERROR_DECLARE(FILE_CHANGED); -LUMIERA_ERROR_DECLARE(FILE_NOMMAPINGS); - - -typedef struct lumiera_file_struct lumiera_file; -typedef lumiera_file* LumieraFile; - - -#include "vault/filedescriptor.h" -#include "vault/filehandle.h" -#include "vault/mmapings.h" -#include "vault/mmap.h" - -/** - * File modes: - * - \c LUMIERA_FILE_READONLY existing file for reading only - * - \c LUMIERA_FILE_READWRITE existing file for reading and writing - * - \c LUMIERA_FILE_CREATE non-existing file for reading and writing - * - \c LUMIERA_FILE_RECREATE remove and recreated existing, file for reading and writing - */ -#define LUMIERA_FILE_READONLY (O_RDONLY | O_LARGEFILE | O_NOATIME) -#define LUMIERA_FILE_READWRITE (O_RDWR | O_LARGEFILE | O_NOATIME) -#define LUMIERA_FILE_CREATE (O_RDWR | O_LARGEFILE | O_NOATIME | O_CREAT | O_EXCL) -#define LUMIERA_FILE_RECREATE (O_RDWR | O_LARGEFILE | O_NOATIME | O_CREAT | O_TRUNC) - -/* \c creat and \c excl flags will be masked out for descriptor lookup */ -#define LUMIERA_FILE_MASK ~(O_CREAT | O_EXCL | O_TRUNC) - -struct lumiera_file_struct -{ - /* all files of one descriptor */ - llist node; - char* name; - LumieraFiledescriptor descriptor; -}; - - -/** - * Initialise a file structure. - * @param self pointer to the file structure - * @param name filename - * @param flags open flags - * @return self or NULL in case of an error - */ -LumieraFile -lumiera_file_init (LumieraFile self, const char* name, int flags); - - -/** - * Destroy a file structure. - * frees all associated resources, releases the file descriptor etc. - * @param self file structure to be destroyed - * @param do_unlink if 1 then delete the file physically from disk (only the associated name) - * @return self - */ -LumieraFile -lumiera_file_destroy (LumieraFile self, int do_unlink); - - -/** - * Allocate a new file structure. - * @param name filename - * @param flags open flags - * @return new file structure or NULL in case of an error - */ -LumieraFile -lumiera_file_new (const char* name, int flags); - - -/** - * Frees a file structure. - * @param self file structure to be freed - */ -void -lumiera_file_delete (LumieraFile self); - - -/** - * Frees a file structure and deletes the associated file name from disk. - * @param self file structure to be freed - */ -void -lumiera_file_delete_unlink (LumieraFile self); - - -/** - * Get a POSIX filehandle for a file. - * Filehandles are opened on demand and must be acquired for use. - * The use of filehandles is refcounted and might be nested. - * After using them they must be released which puts them back into filehandle cache aging. - * @param self file structure - * @return POSIX filehandle or -1 on error, check lumiera_error() to retrieve the errorcode - * Currently only LUMIERA_ERROR_ERRNO will be raised but this might change in future. - * Opening files can fail for many reasons and at any time! - */ -int -lumiera_file_handle_acquire (LumieraFile self); - - -/** - * Put filehandle back into cache aging. - * @param self file which handle to be released - */ -void -lumiera_file_handle_release (LumieraFile self); - - -/** - * acquire a mmap which covers the given range - * @param self file from where the mmap shall be acquired - * @param start begin of the required range - * @param size requested size - * @return MMap object covering the requested range or NULL on error - * note: the chunksize for the file must be set prior accessing mmaps - */ -LumieraMMap -lumiera_file_mmap_acquire (LumieraFile self, off_t start, size_t size); - - -/** - * release a previously acquired MMap object - * @param self file to which the map belongs - * @param map object to be released - */ -void -lumiera_file_release_mmap (LumieraFile self, LumieraMMap map); - - -/** - * helper macro for acquiring and releasing maped regions - * @param nobugflag unused for now - * @param file the file from from where to acquire the mapped region - * @param start the start offset for the mmaped region - * @param size the length of the requested block - * @param addr name of a void* variable pointing to the requested memory - */ -#define LUMIERA_FILE_MMAP_SECTION(nobugflag, file, start, size, addr) \ - for (LumieraMMap map_##__LINE__ = \ - lumiera_file_mmap_acquire (file, start, size); \ - map_##__LINE__; \ - ({ \ - lumiera_file_release_mmap (file, map_##__LINE__); \ - map_##__LINE__ = NULL; \ - })) \ - for (void* addr = lumiera_mmap_address (map_##__LINE__, start); \ - addr; \ - addr = NULL) - - -/** - * Query the underlying mmapings object from a file - * The MMapings only exists after a chunksize got set with lumiera_file_chunksize_set() - * @param self the file to query - * @return Handle to the MMapings object or NULL on error (setting the error state) - */ -LumieraMMapings -lumiera_file_mmapings (LumieraFile self); - - -/** - * Query the flags effective for a file - * @param self the file to query - * @return flags - */ -int -lumiera_file_checkflags (LumieraFile self, int flags); - - -/** - * Set the chunksize for mapping operations - * can only set once for a file descriptor, subsequent calls are no-ops - * @param chunksize allocation/mmaping granularity, must be 2's exponent of page size - * @param bias offset to shift chunks, used for stepping over a header for example. - * @return the effective chunksize used for the file - */ -size_t -lumiera_file_set_chunksize_bias (LumieraFile self, size_t chunksize, size_t bias); - - -/** - * Get the chunksize for mapping operations - * @return the effective chunksize used for the file - */ -size_t -lumiera_file_chunksize_get (LumieraFile self); - -/** - * Get the bias for mapping operations - * @return the effective bias used for the file - */ -size_t -lumiera_file_bias_get (LumieraFile self); - - - - -/** - * Place and remove locks on a file. - * This locks are per thread and lock the file across multiple lumiera processes - * (or any other program which respect to advisory file locking). - * Only exclusive locks over the whole file are supported for initially accessing - * a file, other locking is done somewhere else. - */ - -LumieraFile -lumiera_file_rdlock (LumieraFile self); - -LumieraFile -lumiera_file_wrlock (LumieraFile self); - - -LumieraFile -lumiera_file_unlock (LumieraFile self); - - -#define LUMIERA_FILE_RDLOCK_SECTION(nobugflag, file) \ - for (LumieraFile filelock_##__LINE__ = \ - lumiera_file_rdlock (file); \ - filelock_##__LINE__; \ - ({ \ - lumiera_file_unlock (filelock_##__LINE__); \ - filelock_##__LINE__ = NULL; \ - })) - -#define LUMIERA_FILE_WRLOCK_SECTION(nobugflag, file) \ - for (LumieraFile filelock_##__LINE__ = \ - lumiera_file_wrlock (file); \ - filelock_##__LINE__; \ - ({ \ - lumiera_file_unlock (filelock_##__LINE__); \ - filelock_##__LINE__ = NULL; \ - })) - - - - - -#endif /*VAULT_FILE_H*/ diff --git a/src/vault/filedescriptor.c b/src/vault/filedescriptor.c deleted file mode 100644 index 8210ba8e7..000000000 --- a/src/vault/filedescriptor.c +++ /dev/null @@ -1,396 +0,0 @@ -/* - FileDescriptor - file handling - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - - -/** @file filedescriptor.c - ** Implementation of a file descriptor management framework - ** @todo development in this area is stalled since 2010 - */ - - -#include "include/logging.h" -#include "lib/mutex.h" -#include "lib/safeclib.h" -#include "lib/tmpbuf.h" - -#include "vault/file.h" -#include "vault/filedescriptor.h" -#include "vault/filedescriptorregistry.h" -#include "vault/filehandle.h" -#include "vault/filehandlecache.h" - -#include -#include -#include -#include -#include - - - -/* lookup and creation of files, initialised in backend.c */ - -lumiera_mutex lumiera_filecreate_mutex; - - -LumieraFiledescriptor -lumiera_filedescriptor_acquire (const char* name, int flags, LList filenode) -{ - TRACE (filedescriptor_dbg, "%s", name); - REQUIRE (llist_is_empty (filenode)); - - LumieraFiledescriptor dest = NULL; - - LUMIERA_MUTEX_SECTION (mutex_sync, &lumiera_filecreate_mutex) - { - lumiera_filedescriptor fdesc; - fdesc.flags = flags; - - if (stat (name, &fdesc.stat) != 0) - { - if (errno == ENOENT && flags&O_CREAT) - { - errno = 0; - char* dir = lumiera_tmpbuf_strndup (name, PATH_MAX); - char* slash = dir; - while ((slash = strchr (slash+1, '/'))) - { - *slash = '\0'; - INFO (filedescriptor_dbg, "try creating dir: %s", dir); - if (mkdir (dir, 0777) == -1 && errno != EEXIST) - { - LUMIERA_ERROR_SET_CRITICAL (file, ERRNO, name); - goto error; - } - *slash = '/'; - } - int fd; - INFO (filedescriptor_dbg, "try creating file: %s", name); - TODO ("create mode from config"); - fd = creat (name, 0666); - if (fd == -1) - { - LUMIERA_ERROR_SET_CRITICAL (file, ERRNO, name); - goto error; - } - close (fd); - if (stat (name, &fdesc.stat) != 0) - { - /* finally, no luck */ - LUMIERA_ERROR_SET_CRITICAL (file, ERRNO, name); - goto error; - } - } - } - - /* lookup/create descriptor */ - dest = lumiera_filedescriptorregistry_ensure (&fdesc); - - if (dest) - llist_insert_head (&dest->files, filenode); - - error: - ; - } - - return dest; -} - - - - -void -lumiera_filedescriptor_release (LumieraFiledescriptor self, const char* name, LList filenode) -{ - TRACE (filedescriptor_dbg); - - if (filenode) - LUMIERA_MUTEX_SECTION (mutex_sync, &self->lock) - { - REQUIRE (llist_is_member (&self->files, filenode)); - llist_unlink (filenode); - } - - if (llist_is_empty (&self->files)) - lumiera_filedescriptor_delete (self, name); -} - - -int -lumiera_filedescriptor_handle_acquire (LumieraFiledescriptor self) -{ - TRACE (filedescriptor_dbg); - - int fd = -1; - - LUMIERA_MUTEX_SECTION (mutex_sync, &self->lock) - { - if (!self->handle) - /* no handle yet, get a new one */ - lumiera_filehandlecache_handle_acquire (self); - else - lumiera_filehandlecache_checkout (self->handle); - - fd = lumiera_filehandle_handle (self->handle); - } - - return fd; -} - - -void -lumiera_filedescriptor_handle_release (LumieraFiledescriptor self) -{ - TRACE (filedescriptor_dbg); - REQUIRE (self->handle); - - LUMIERA_MUTEX_SECTION (mutex_sync, &self->lock) - lumiera_filehandlecache_checkin (self->handle); -} - - -const char* -lumiera_filedescriptor_name (LumieraFiledescriptor self) -{ - REQUIRE (!llist_is_empty (&self->files)); - - return ((LumieraFile)(llist_head (&self->files)))->name; -} - - -int -lumiera_filedescriptor_flags (LumieraFiledescriptor self) -{ - return self->flags; -} - - -int -lumiera_filedescriptor_samestat (LumieraFiledescriptor self, struct stat* stat) -{ - if (self->stat.st_dev == stat->st_dev && self->stat.st_ino == stat->st_ino) - return 1; - - return 0; -} - - -LumieraFiledescriptor -lumiera_filedescriptor_new (LumieraFiledescriptor template) -{ - LumieraFiledescriptor self = lumiera_malloc (sizeof (lumiera_filedescriptor)); - TRACE (filedescriptor_dbg, "at %p", self); - - psplaynode_init (&self->node); - self->stat = template->stat; - - self->realsize = template->stat.st_size; - self->flags = template->flags; - self->handle = NULL; - self->mmapings = NULL; - llist_init (&self->files); - - RESOURCE_USER_INIT(self->filelock_rh); - - lumiera_mutex_init (&self->lock, "filedescriptor", &NOBUG_FLAG (mutex_dbg), NOBUG_CONTEXT); - - lumiera_rwlock_init (&self->filelock, "filelock", &NOBUG_FLAG (filedescriptor_dbg), NOBUG_CONTEXT); - self->lock_cnt = 0; - - return self; -} - - -void -lumiera_filedescriptor_delete (LumieraFiledescriptor self, const char* name) -{ - TRACE (filedescriptor_dbg, "%p %s", self, name); - - REQUIRE (!self->lock_cnt, "File still locked"); - REQUIRE (llist_is_empty (&self->files)); - - lumiera_filedescriptorregistry_remove (self); - - TODO ("destroy other members (WIP)"); - - lumiera_mmapings_delete (self->mmapings); - - if (self->handle && name && ((self->flags & O_RDWR) == O_RDWR)) - { - TRACE (filedescriptor_dbg, "truncate %s to %lld", name, (long long)self->realsize); - lumiera_filehandlecache_checkout (self->handle); - int dummy = ftruncate (lumiera_filehandle_handle (self->handle), self->realsize); - (void) dummy; /* this is present to silence a warning */ - TODO ("handle error case better"); - lumiera_filehandlecache_checkin (self->handle); - TODO ("really release filehandle"); - } - - lumiera_rwlock_destroy (&self->filelock, &NOBUG_FLAG (mutex_dbg), NOBUG_CONTEXT); - lumiera_mutex_destroy (&self->lock, &NOBUG_FLAG (mutex_dbg), NOBUG_CONTEXT); - - lumiera_free (self); -} - - -LumieraFiledescriptor -lumiera_filedescriptor_rdlock (LumieraFiledescriptor self) -{ - TRACE (filedescriptor_dbg); - - if (self) - { - lumiera_rwlock_rdlock (&self->filelock, &NOBUG_FLAG (filedescriptor_dbg), &self->filelock_rh, NOBUG_CONTEXT); - - int fd = lumiera_filedescriptor_handle_acquire (self); - int err = 0; - - LUMIERA_MUTEX_SECTION (mutex_dbg, &self->lock) - { - if (!self->lock_cnt) - { - - struct flock lock; - lock.l_type = F_RDLCK; - lock.l_whence = SEEK_SET; - lock.l_start = 0; - lock.l_len = 0; - - while ((err = fcntl (fd, F_SETLKW, &lock)) == -1 && errno == EINTR) - ; - - } - if (!err) - ++self->lock_cnt; - } - - if (err) - { - lumiera_filedescriptor_handle_release (self); - lumiera_rwlock_unlock (&self->filelock, &NOBUG_FLAG (filedescriptor_dbg), &self->filelock_rh, NOBUG_CONTEXT); - - LUMIERA_ERROR_SET_WARNING (filedescriptor_dbg, ERRNO, lumiera_filedescriptor_name (self)); - self = NULL; - } - } - - return self; -} - - - -LumieraFiledescriptor -lumiera_filedescriptor_wrlock (LumieraFiledescriptor self) -{ - TRACE (filedescriptor_dbg); - - if (self) - { - lumiera_rwlock_wrlock (&self->filelock, &NOBUG_FLAG (filedescriptor_dbg), &self->filelock_rh, NOBUG_CONTEXT); - - int fd = lumiera_filedescriptor_handle_acquire (self); - int err = 0; - - LUMIERA_MUTEX_SECTION(mutex_dbg, &self->lock) - { - - struct flock lock; - lock.l_type = F_WRLCK; - lock.l_whence = SEEK_SET; - lock.l_start = 0; - lock.l_len = 0; - - while ((err = fcntl (fd, F_SETLKW, &lock)) == -1 && errno == EINTR) - ; - - if (!err) - self->lock_cnt = -1; - - } - - if (err) - { - lumiera_filedescriptor_handle_release (self); - lumiera_rwlock_unlock (&self->filelock, &NOBUG_FLAG (filedescriptor_dbg), &self->filelock_rh, NOBUG_CONTEXT); - LUMIERA_ERROR_SET_WARNING (filedescriptor_dbg, ERRNO, lumiera_filedescriptor_name (self)); - self = NULL; - } - } - - return self; -} - - -LumieraFiledescriptor -lumiera_filedescriptor_unlock (LumieraFiledescriptor self) -{ - TRACE (filedescriptor_dbg); - - if (self) - { - int fd = lumiera_filehandle_get (self->handle); - REQUIRE (fd >= 0, "was not locked?"); - int err = 0; - - LUMIERA_MUTEX_SECTION(mutex_dbg, &self->lock) - { - if (self->lock_cnt == -1) - self->lock_cnt = 0; - else - --self->lock_cnt; - - if (!self->lock_cnt) - { - - struct flock lock; - lock.l_type = F_UNLCK; - lock.l_whence = SEEK_SET; - lock.l_start = 0; - lock.l_len = 0; - - while ((err = fcntl (fd, F_SETLK, &lock)) == -1 && errno == EINTR) - ; - } - } - - if (err) - { - LUMIERA_ERROR_SET_WARNING (filedescriptor_dbg, ERRNO, lumiera_filedescriptor_name (self)); - self = NULL; - } - else - { - lumiera_filedescriptor_handle_release (self); - lumiera_rwlock_unlock (&self->filelock, &NOBUG_FLAG (filedescriptor_dbg), &self->filelock_rh, NOBUG_CONTEXT); - } - } - - return self; -} - - -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/filedescriptor.h b/src/vault/filedescriptor.h deleted file mode 100644 index 7b6e3fd5d..000000000 --- a/src/vault/filedescriptor.h +++ /dev/null @@ -1,153 +0,0 @@ -/* - FILEDESCRIPTOR.h - file handling - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. -*/ - - -/** @file filedescriptor.h - ** File descriptors are the underlying working horse in accessing files. - ** All information associated with managing a file is kept here. - */ - -#ifndef VAULT_FILEDESCRIPTOR_H -#define VAULT_FILEDESCRIPTOR_H - -#include "lib/mutex.h" -#include "lib/rwlock.h" -#include "lib/psplay.h" -#include "lib/llist.h" - - -#include -#include -#include - -typedef struct lumiera_filedescriptor_struct lumiera_filedescriptor; -typedef lumiera_filedescriptor* LumieraFiledescriptor; - - -#include "vault/filehandle.h" -#include "vault/file.h" -#include "vault/mmapings.h" - - -struct lumiera_filedescriptor_struct -{ - /** node for the lookup tree */ - psplaynode node; - - /** create after first open, maintained metadata */ - struct stat stat; - - /** - * files which are written are rounded up to the next chunk boundary - * by the mmaping backend and will be ftruncated to the realsize on close. - */ - off_t realsize; - - /** open flags, must be masked for reopen */ - int flags; - - /** locks operations on this file descriptor */ - lumiera_mutex lock; - - /** Associated posix filehandle */ - LumieraFilehandle handle; - - /** established memory mappings */ - LumieraMMapings mmapings; - - /** list of all attached 'file' structures, that are the names of the files */ - llist files; - - /** file locking, a rwlock for thread locking */ - lumiera_rwlock filelock; - - /** readlock counter for releasing the file lock, -1 for write lock, 0 = unlocked */ - int lock_cnt; - RESOURCE_USER (filelock_rh); -}; - - -/** - * Find existing filedescriptor or create one - * @param name name of the file - * @param flags opening flags for the filedescriptor - * @return descriptor on success or NULL on error - */ -LumieraFiledescriptor -lumiera_filedescriptor_acquire (const char* name, int flags, LList filenode); - - -/** - * Release a filedescriptor. - * @param self filedescriptor to be released - * @param file the file struct which initiated the release - */ -void -lumiera_filedescriptor_release (LumieraFiledescriptor self, const char* name, LList filenode); - -int -lumiera_filedescriptor_handle_acquire (LumieraFiledescriptor self); - -void -lumiera_filedescriptor_handle_release (LumieraFiledescriptor self); - -const char* -lumiera_filedescriptor_name (LumieraFiledescriptor self); - -int -lumiera_filedescriptor_flags (LumieraFiledescriptor self); - -int -lumiera_filedescriptor_samestat (LumieraFiledescriptor self, struct stat* stat); - -/** - * Allocate a new filedescriptor cloned from a template - * @param template the source filedescriptor - * @return the constructed filedescriptor - */ -LumieraFiledescriptor -lumiera_filedescriptor_new (LumieraFiledescriptor template); - -/** - * Delete a filedescriptor - * Called whenever its reference count drops to zero. - * @param self the filedescriptor to be deleted - */ -void -lumiera_filedescriptor_delete (LumieraFiledescriptor self, const char* name); - - - -LumieraFiledescriptor -lumiera_filedescriptor_rdlock (LumieraFiledescriptor self); - - -LumieraFiledescriptor -lumiera_filedescriptor_wrlock (LumieraFiledescriptor self); - - -LumieraFiledescriptor -lumiera_filedescriptor_unlock (LumieraFiledescriptor self); - - - - -#endif /*VAULT_FILEDESCRIPTOR_H*/ diff --git a/src/vault/filedescriptorregistry.c b/src/vault/filedescriptorregistry.c deleted file mode 100644 index 469107d04..000000000 --- a/src/vault/filedescriptorregistry.c +++ /dev/null @@ -1,152 +0,0 @@ -/* - FiledescriptorRegistry - registry for tracking all files in use - - Copyright (C) Lumiera.org - 2008, 2010, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - - -/** @file filedescriptorregistry.c - ** Implementation of a registry to manage used filedescriptors - ** @todo development in this area is stalled since 2010 - */ - - -#include "include/logging.h" -#include "lib/safeclib.h" -#include "lib/mutex.h" -#include "lib/psplay.h" - -#include "vault/file.h" -#include "vault/filedescriptor.h" -#include "vault/filedescriptorregistry.h" - - -/** - * the global registry for file descriptors. - */ -static PSplay filedescriptorregistry = NULL; -static lumiera_mutex filedescriptorregistry_mutex; - - -static int -cmp_fn (const void* keya, const void* keyb) -{ - const LumieraFiledescriptor a = (const LumieraFiledescriptor)keya; - const LumieraFiledescriptor b = (const LumieraFiledescriptor)keyb; - - if (a->stat.st_dev < b->stat.st_dev) - return -1; - else if (a->stat.st_dev > b->stat.st_dev) - return 1; - - if (a->stat.st_ino < b->stat.st_ino) - return -1; - else if (a->stat.st_ino > b->stat.st_ino) - return 1; - - if ((a->flags&LUMIERA_FILE_MASK) < (b->flags&LUMIERA_FILE_MASK)) - return -1; - else if ((a->flags&LUMIERA_FILE_MASK) > (b->flags&LUMIERA_FILE_MASK)) - return 1; - - return 0; -} - - -static void -delete_fn (PSplaynode node) -{ - TODO ("figure name out? or is the handle here always closed"); - lumiera_filedescriptor_delete ((LumieraFiledescriptor) node, NULL); -} - - -static const void* -key_fn (const PSplaynode node) -{ - return node; -} - - - -void -lumiera_filedescriptorregistry_init (void) -{ - TRACE (filedescriptor_dbg); - REQUIRE (!filedescriptorregistry); - - filedescriptorregistry = psplay_new (cmp_fn, key_fn, delete_fn); - if (!filedescriptorregistry) - LUMIERA_DIE (NO_MEMORY); - - lumiera_mutex_init (&filedescriptorregistry_mutex, "filedescriptorregistry", &NOBUG_FLAG (mutex_dbg), NOBUG_CONTEXT); -} - -void -lumiera_filedescriptorregistry_destroy (void) -{ - TRACE (filedescriptor_dbg); - REQUIRE (!psplay_nelements (filedescriptorregistry)); - - lumiera_mutex_destroy (&filedescriptorregistry_mutex, &NOBUG_FLAG (mutex_dbg), NOBUG_CONTEXT); - - if (filedescriptorregistry) - psplay_delete (filedescriptorregistry); - - filedescriptorregistry = NULL; -} - - -LumieraFiledescriptor -lumiera_filedescriptorregistry_ensure (LumieraFiledescriptor template) -{ - REQUIRE (filedescriptorregistry, "not initialised"); - - LumieraFiledescriptor ret = NULL; - - LUMIERA_MUTEX_SECTION (mutex_sync, &filedescriptorregistry_mutex) - { - /* lookup/create descriptor */ - ret = (LumieraFiledescriptor) psplay_find (filedescriptorregistry, template, 100); - - if (!ret) - { - ret = lumiera_filedescriptor_new (template); - if (!ret) - goto error; - - psplay_insert (filedescriptorregistry, &ret->node, 100); - } - - error: - ; - } - - return ret; -} - - -void -lumiera_filedescriptorregistry_remove (LumieraFiledescriptor self) -{ - LUMIERA_MUTEX_SECTION (mutex_sync, &filedescriptorregistry_mutex) - psplay_remove (filedescriptorregistry, &self->node); -} - - diff --git a/src/vault/filedescriptorregistry.h b/src/vault/filedescriptorregistry.h deleted file mode 100644 index c21bbeb46..000000000 --- a/src/vault/filedescriptorregistry.h +++ /dev/null @@ -1,67 +0,0 @@ -/* - FILEDESCRIPTORREGISTRY.h - registry for tracking all files in use - - Copyright (C) Lumiera.org - 2008, 2010, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -*/ - - -/** @file filedescriptorregistry.h - ** Registry for used file descriptors. - ** This registry stores all acquired file descriptors for lookup, - ** they will be freed when not referenced anymore. - */ - -#ifndef VAULT_FILEDESCRIPTORREGISTRY_H -#define VAULT_FILEDESCRIPTORREGISTRY_H - -#include "vault/filedescriptor.h" - - -/** - * Initialise the global file descriptor registry. - * Opening hard linked files will be targeted to the same file descriptor. - * This function never fails but dies on error. - * @todo proper vault/subsystem failure - */ -void -lumiera_filedescriptorregistry_init (void); - -/** - * Destroy and free the global file descriptor registry. - * Never fails. - */ -void -lumiera_filedescriptorregistry_destroy (void); - -/** - * Ensures that a filedescriptor is in the registry. - * Looks template up and if not found, create a new one from template. - * @return filedescriptor from registry - */ -LumieraFiledescriptor -lumiera_filedescriptorregistry_ensure (LumieraFiledescriptor template); - - -/** Removes a file descriptor from the registry. */ -void -lumiera_filedescriptorregistry_remove (LumieraFiledescriptor self); - - - -#endif /*VAULT_FILEDESCRIPTORREGISTRY_H*/ diff --git a/src/vault/filehandle.c b/src/vault/filehandle.c deleted file mode 100644 index 871dc7800..000000000 --- a/src/vault/filehandle.c +++ /dev/null @@ -1,115 +0,0 @@ -/* - FileHandle - filehandle management and caching - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - - -/** @file filehandle.c - ** Implementation of filehandle representation. - ** @todo development in this area is stalled since 2010 - */ - - -#include "include/logging.h" -#include "lib/llist.h" -#include "lib/safeclib.h" - -#include "vault/file.h" -#include "vault/filehandle.h" -#include "vault/filedescriptor.h" - -#include - - - -LumieraFilehandle -lumiera_filehandle_init (LumieraFilehandle self, LumieraFiledescriptor desc) -{ - TRACE (filehandle_dbg, "%p", self); - if (self) - { - llist_init (&self->cachenode); - self->fd = -1; - self->use_cnt = 1; - self->descriptor = desc; - } - return self; -} - - -LumieraFilehandle -lumiera_filehandle_new (LumieraFiledescriptor desc) -{ - LumieraFilehandle self = lumiera_malloc (sizeof (*self)); - return lumiera_filehandle_init (self, desc); -} - - -void* -lumiera_filehandle_destroy_node (LList node) -{ - TRACE (filehandle_dbg); - REQUIRE (llist_is_empty (node)); - LumieraFilehandle self = (LumieraFilehandle)node; - REQUIRE (self->use_cnt == 0); - - if (self->fd >= 0) - close (self->fd); - return self; -} - - - -int -lumiera_filehandle_handle (LumieraFilehandle self) -{ - TRACE (filehandle_dbg); - - int fd = -1; - if (self->fd == -1) - { - fd = open (lumiera_filedescriptor_name (self->descriptor), lumiera_filedescriptor_flags (self->descriptor) & LUMIERA_FILE_MASK); - if (fd == -1) - { - //////////////////////TODO Handle EMFILE etc with the resourcecollector - LUMIERA_ERROR_SET_CRITICAL (file, ERRNO, lumiera_filedescriptor_name (self->descriptor)); - } - else - { - struct stat st; - if (fstat (fd, &st) == -1) - { - close (fd); - fd = -1; - LUMIERA_ERROR_SET_CRITICAL (file, ERRNO, lumiera_filedescriptor_name (self->descriptor)); - } - else if (!lumiera_filedescriptor_samestat (self->descriptor, &st)) - { - close (fd); - fd = -1; - /* Woops this is not the file we expected to use */ - LUMIERA_ERROR_SET_CRITICAL (file, FILE_CHANGED, lumiera_filedescriptor_name (self->descriptor)); - } - } - self->fd = fd; - } - - return self->fd; -} diff --git a/src/vault/filehandle.h b/src/vault/filehandle.h deleted file mode 100644 index ad69ca5a4..000000000 --- a/src/vault/filehandle.h +++ /dev/null @@ -1,101 +0,0 @@ -/* - FILEHANDLE - filehandle management and caching - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -*/ - - -/** @file filehandle.h - ** Filehandles manage the underlying POSIX filehandle for a file descriptor. - ** Since we want to support handling of more files than POSIX filehandles are available - ** on a common system the filehandles are opened, cached and closed on demand, see 'filehandlecache'. - ** Access to filehandles is locked from elsewhere (filedescriptor, filehandlecache) - */ - -#ifndef VAULT_FILEHANDLE_H -#define VAULT_FILEHANDLE_H - -#include "lib/error.h" -#include "lib/llist.h" - -typedef struct lumiera_filehandle_struct lumiera_filehandle; -typedef lumiera_filehandle* LumieraFilehandle; - -#include "vault/filedescriptor.h" - -//NOBUG_DECLARE_FLAG (filehandle); - - - -/** - * File handles - */ -struct lumiera_filehandle_struct -{ - llist cachenode; - int fd; - unsigned use_cnt; - LumieraFiledescriptor descriptor; -}; - - -/** - * Initialise filehandle structure. - * @param self filehandle structure to be initialised - * @param descriptor on which this filehandle will be attached - * @return new filehandle structure - */ -LumieraFilehandle -lumiera_filehandle_init (LumieraFilehandle self, LumieraFiledescriptor descriptor); - - -/** - * Allocate a new filehandle structure. - * @param descriptor on which this filehandle will be attached - * @return new filehandle structure - */ -LumieraFilehandle -lumiera_filehandle_new (LumieraFiledescriptor descriptor); - - -/** - * destroy the resources associated either a filehandle structure. - * This function is used by the filehandle cache to recycle filehandle structs. - * @param node pointer to the cache node member of a struct filehandle - * @return pointer to the start of the memory of the destroyed filehandle - */ -void* -lumiera_filehandle_destroy_node (LList node); - - -int -lumiera_filehandle_handle (LumieraFilehandle self); - - -/** - * just accessor, no safety net - */ -static inline int -lumiera_filehandle_get (LumieraFilehandle self) -{ - REQUIRE (self->descriptor); - return self->fd; -} - -#endif /*VAULT_FILEHANDLE_H*/ diff --git a/src/vault/filehandlecache.c b/src/vault/filehandlecache.c deleted file mode 100644 index f44ad0c8b..000000000 --- a/src/vault/filehandlecache.c +++ /dev/null @@ -1,149 +0,0 @@ -/* - FileHandleCache - filehandle management and caching - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - - -/** @file filehandlecache.c - ** Implementation of caching for filehandle representation data - ** @todo development in this area is stalled since 2010 - */ - - -#include "include/logging.h" -#include "lib/safeclib.h" - -#include "vault/file.h" -#include "vault/filehandlecache.h" - - - -LUMIERA_ERROR_DEFINE (FILEHANDLECACHE_NOHANDLE, "No filehandle available"); - - -/** - * the global cache for file handles. - */ -LumieraFilehandlecache lumiera_fhcache = NULL; - - - -void -lumiera_filehandlecache_new (int max_entries) -{ - REQUIRE (!lumiera_fhcache, "Filehandlecache already initialised"); - - lumiera_fhcache = lumiera_malloc (sizeof (lumiera_filehandlecache)); - lumiera_mrucache_init (&lumiera_fhcache->cache, lumiera_filehandle_destroy_node); - lumiera_fhcache->available = max_entries; - lumiera_fhcache->checked_out = 0; - lumiera_mutex_init (&lumiera_fhcache->lock, "filehandlecache", &NOBUG_FLAG (mutex_dbg), NOBUG_CONTEXT); -} - - -void -lumiera_filehandlecache_delete (void) -{ - if (lumiera_fhcache) - { - REQUIRE (!lumiera_fhcache->checked_out, "Filehandles in use at shutdown"); - lumiera_mrucache_destroy (&lumiera_fhcache->cache); - lumiera_mutex_destroy (&lumiera_fhcache->lock, &NOBUG_FLAG (mutex_dbg), NOBUG_CONTEXT); - lumiera_free (lumiera_fhcache); - lumiera_fhcache = NULL; - } -} - - -LumieraFilehandle -lumiera_filehandlecache_handle_acquire (LumieraFiledescriptor desc) -{ - TRACE (filehandlecache_dbg); - LumieraFilehandle ret = NULL; - - LUMIERA_MUTEX_SECTION (mutex_sync, &lumiera_fhcache->lock) - { - if (lumiera_fhcache->available <= 0 && lumiera_fhcache->cache.cached) - { - /* pop a filehandle from cache */ - ret = lumiera_mrucache_pop (&lumiera_fhcache->cache); - ret = lumiera_filehandle_init (lumiera_mrucache_pop (&lumiera_fhcache->cache), desc); - if (lumiera_fhcache->available < 0) - /* try to free overallocated filehandles */ - lumiera_fhcache->available -= lumiera_fhcache->available + lumiera_mrucache_age (&lumiera_fhcache->cache, -lumiera_fhcache->available); - } - else - { - /* allocate new filehandle if we are below the limit or no cached handles are available (overallocating) */ - NOTICE (file, "overallocating filehandle"); - ret = lumiera_filehandle_new (desc); - TODO ("use resourcecollector here"); - if (!ret) - LUMIERA_ERROR_SET_ALERT (file, FILEHANDLECACHE_NOHANDLE, lumiera_filedescriptor_name (desc)); - else - --lumiera_fhcache->available; - } - desc->handle = ret; - ++lumiera_fhcache->checked_out; - } - return ret; -} - - -LumieraFilehandle -lumiera_filehandlecache_checkout (LumieraFilehandle handle) -{ - TRACE (filehandlecache_dbg); - REQUIRE (handle); - /* This function is called with the associated descriptor locked, nothing can modify 'handle' */ - if (!handle->use_cnt) - { - /* lock cache and checkout */ - LUMIERA_MUTEX_SECTION (mutex_sync, &lumiera_fhcache->lock) - { - lumiera_mrucache_checkout (&lumiera_fhcache->cache, &handle->cachenode); - ++lumiera_fhcache->checked_out; - } - } - ++handle->use_cnt; - - return handle; -} - - -void -lumiera_filehandlecache_checkin (LumieraFilehandle handle) -{ - TRACE (filehandlecache_dbg); - REQUIRE (handle); - REQUIRE (handle->use_cnt); - - /* This function is called with the associated descriptor locked, nothing can modify the fhcache */ - if (!--handle->use_cnt) - { - /* lock cache and checin */ - LUMIERA_MUTEX_SECTION (mutex_sync, &lumiera_fhcache->lock) - { - --lumiera_fhcache->checked_out; - lumiera_mrucache_checkin (&lumiera_fhcache->cache, &handle->cachenode); - } - } -} - diff --git a/src/vault/filehandlecache.h b/src/vault/filehandlecache.h deleted file mode 100644 index 58908fb94..000000000 --- a/src/vault/filehandlecache.h +++ /dev/null @@ -1,99 +0,0 @@ -/* - FILEHANDLECACHE - filehandle management and caching - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -*/ - - -/** @file filehandlecache.h - ** Caching and management of filehandles. - ** The number of filehandles a program can held open is usually limited, since we want to support - ** using a less limited number of files and closing/opening for each operation is expensive, we - ** provide a cache to keep the most frequent used files open and gracefully close/recycle unused filehandles. - ** The filehandle cache defined here protects all operations on the cache with a mutex. - */ - -#ifndef VAULT_FILEHANDLECACHE_H -#define VAULT_FILEHANDLECACHE_H - -#include "lib/error.h" -#include "lib/mrucache.h" -#include "lib/mutex.h" - -typedef struct lumiera_filehandlecache_struct lumiera_filehandlecache; -typedef lumiera_filehandlecache* LumieraFilehandlecache; - -#include "vault/filehandle.h" - - - -/** File handle cache manages file handles */ -struct lumiera_filehandlecache_struct -{ - lumiera_mrucache cache; - int available; - int checked_out; - lumiera_mutex lock; -}; - -/** - * Initialises the filehandle cache. - * @param max_entries number how much filehandles shall be managed - * The number of elements the cache can hold is static and should be - * determined by sysconf (_SC_OPEN_MAX) minus some (big) safety margin. - */ -void -lumiera_filehandlecache_new (int max_entries); - -/** - * Delete the filehandle cache. - * No filehandles in the cache must be locked, this would be a fatal error. - * The handles are closed automatically. - */ -void -lumiera_filehandlecache_delete (void); - -/** - * Get a fresh filehandle. - * @param self pointer to the cache - * @return the new filehandle - */ -LumieraFilehandle -lumiera_filehandlecache_handle_acquire (LumieraFiledescriptor desc); - -/** - * Remove a filehandle from cache aging - * Filehandles which are subject of cache aging must be checked out before they can be used. - * @param self the filehandlecache - * @param handle the filehandle to be checked out - */ -LumieraFilehandle -lumiera_filehandlecache_checkout (LumieraFilehandle handle); - -/** - * Put a filehandle into the cache - * Filehandles which are checked in are subject of cache aging and might get destroyed and reused. - * @param handle the filehandle to be checked in - */ -void -lumiera_filehandlecache_checkin (LumieraFilehandle handle); - - - -#endif /*VAULT_FILEHANDLECACHE_H*/ diff --git a/src/vault/fileheader.c b/src/vault/fileheader.c deleted file mode 100644 index 539c414c1..000000000 --- a/src/vault/fileheader.c +++ /dev/null @@ -1,234 +0,0 @@ -/* - Fileheader - Definitions of generic lumiera file headers and identification - - Copyright (C) Lumiera.org - 2010, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - - -/** @file fileheader.c - ** Implementation of a common header format for working data files created by the Lumiera Vault. - ** @todo development in this area is stalled since 2010 - */ - - -#include "lib/tmpbuf.h" - -#include "vault/fileheader.h" -#include "vault/file.h" -#include "include/logging.h" - -#include - - - - -LUMIERA_ERROR_DEFINE (FILEHEADER_NOWRITE, "File is not writable"); -LUMIERA_ERROR_DEFINE (FILEHEADER_HEADER, "Error in header"); -LUMIERA_ERROR_DEFINE (FILEHEADER_FLAGS, "Inconsistent Flags"); -LUMIERA_ERROR_DEFINE (FILEHEADER_FLAGSPACE, "No more space for flags left"); -LUMIERA_ERROR_DEFINE (FILEHEADER_ENDIANESS, "Unsupported Endianess"); - - - -lumiera_fileheader -lumiera_fileheader_create (LumieraFile file, char* fourcc, int version, size_t size, const char* flags) -{ - lumiera_fileheader self = {NULL, NULL}; - - REQUIRE (fourcc && strlen (fourcc) == 4, "fourcc must be of size 4 and nothing else: %s", fourcc); - REQUIRE_IF (flags, strlen (flags) <= sizeof(self.header->flags), "to much flags given"); - - if (file) - { - if (lumiera_file_checkflags (file, O_RDWR)) - { - WARN_IF (version == 0, fileheader, "Experimental version 0 given for file %s, %s", file->name, fourcc); - - if ((self.map = lumiera_mmap_new_exact (file, 0, size))) - { - self.header = lumiera_mmap_address (self.map, 0); - - memcpy (self.header->fourcc, fourcc, 4); - snprintf (self.header->version, 4, "%3u", version); - self.header->newline1 = '\n'; - for (size_t i = 0; i < sizeof(self.header->meta); ++i) - self.header->meta[i] = ' '; - self.header->newline2 = '\n'; - for (size_t i = 0; i < sizeof(self.header->flags); ++i) - self.header->flags[i] = ' '; - self.header->newline3 = '\n'; - self.header->null = '\0'; - self.header->endianess_mark = LUMIERA_FILEHEADER_ENDIANMAGIC; - - lumiera_fileheader_flags_set (&self, flags); - } - } - else - LUMIERA_ERROR_SET_CRITICAL(fileheader, FILEHEADER_NOWRITE, file->name); - } - - return self; -} - - -lumiera_fileheader -lumiera_fileheader_open (LumieraFile file, char* fourcc, size_t size, const char* flags_expected, const char* flags_remove) -{ - lumiera_fileheader self = {NULL, NULL}; - - REQUIRE (fourcc && strlen (fourcc) == 4, "fourcc must be of size 4 and nothing else: %s", fourcc); - - if (file) - { - if ((self.map = lumiera_mmap_new_exact (file, 0, size))) - { - self.header = lumiera_mmap_address (self.map, 0); - - if (memcmp (self.header->fourcc, fourcc, 4)) - { - LUMIERA_ERROR_SET_WARNING (fileheader, FILEHEADER_HEADER, file->name); - goto err; - } - - if (!lumiera_fileheader_flags_validate (&self, flags_expected, NULL)) - { - LUMIERA_ERROR_SET_WARNING (fileheader, FILEHEADER_FLAGS, self.header->flags); - goto err; - } - - ////////////////////TODO only clear flags when file is writable! - lumiera_fileheader_flags_clear (&self, flags_remove); - - static uint64_t endianess_mark = LUMIERA_FILEHEADER_ENDIANMAGIC; - - if (lumiera_fileheader_flags_validate (&self, LUMIERA_FILEHEADER_FLAG_ENDIANESS, NULL) && - memcmp (&self.header->endianess_mark, (char*)&endianess_mark, sizeof (endianess_mark))) - { - LUMIERA_ERROR_SET_CRITICAL (fileheader, FILEHEADER_ENDIANESS, NULL); - goto err; - } - } - } - - return self; - - err: - lumiera_mmap_delete (self.map); - self.map = NULL; - self.header = NULL; - return self; -} - - -void -lumiera_fileheader_close (LumieraFileheader self, const char* flags_add) -{ - if (self && self->header) - { - lumiera_fileheader_flags_set (self, flags_add); - self->header = NULL; - lumiera_mmap_delete (self->map); - } -} - - -int -lumiera_fileheader_version (LumieraFileheader self) -{ - int ret = -1; - if (self && self->header) - { - errno = 0; - ret = (int) strtol(self->header->version, NULL, 10); - if (errno) - { - ret = -1; - LUMIERA_ERROR_SET_WARNING (fileheader, FILEHEADER_HEADER, lumiera_tmpbuf_snprintf (32, "%s", self->header->fourcc)); - } - } - return ret; -} - - - - -/** - * check if all flags given from some sets are either set or not. - */ -int -lumiera_fileheader_flags_validate (LumieraFileheader self, const char* expected, const char* unexpected) -{ - for (; expected && *expected; ++expected) - if (!strchr (self->header->flags, *expected)) - return 0; - - for (; unexpected && *unexpected; ++unexpected) - if (strchr (self->header->flags, *unexpected)) - return 0; - - return 1; -} - - -/** - * Sets flags if not already set - */ -LumieraFileheader -lumiera_fileheader_flags_set (LumieraFileheader self, const char* flags) -{ - for (; flags && *flags; ++flags) - if (!strchr(self->header->flags, *flags)) - { - char* pos = strchr (self->header->flags, ' '); - if (!pos) - { - LUMIERA_ERROR_SET_CRITICAL (fileheader, FILEHEADER_FLAGSPACE, flags); - return NULL; - } - *pos = *flags; - } - - return self; -} - - -/** - * Clear flags if present - */ -LumieraFileheader -lumiera_fileheader_flags_clear (LumieraFileheader self, const char* flags) -{ - char* pos; - - for (; flags && *flags; ++flags) - if ((pos = strchr (self->header->flags, *flags))) - *pos = ' '; - - return self; -} - - - -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/fileheader.h b/src/vault/fileheader.h deleted file mode 100644 index 8998a98fa..000000000 --- a/src/vault/fileheader.h +++ /dev/null @@ -1,202 +0,0 @@ -/* - FILEHEADER.h - Definitions of generic lumiera file headers and identification - - Copyright (C) Lumiera.org - 2010, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -*/ - - -/** @file fileheader.h - ** Common header format to identify various kinds of files. - ** Lumiera creates some files on itself, caches, indexes and so on. - ** Here we define a unified header format for identifying and handling these files. - ** - ** Most of this files store binary data in host order for performance reasons and are not yet - ** intended to be transfered between computers. While the transferability depends on the - ** concrete implementation later and is not constrained here. - ** - */ - -#ifndef VAULT_FILEHEADER_H -#define VAULT_FILEHEADER_H - -#include "lib/error.h" - - - -LUMIERA_ERROR_DECLARE (FILEHEADER_NOWRITE); -LUMIERA_ERROR_DECLARE (FILEHEADER_HEADER); -LUMIERA_ERROR_DECLARE (FILEHEADER_FLAGS); -LUMIERA_ERROR_DECLARE (FILEHEADER_FLAGSPACE); -LUMIERA_ERROR_DECLARE (FILEHEADER_ENDIANESS); - -#define LUMIERA_FILEHEADER_ENDIANMAGIC 0x0123456789ABCDEFULL - -typedef struct lumiera_fileheader_struct lumiera_fileheader; -typedef lumiera_fileheader* LumieraFileheader; - -typedef struct lumiera_fileheader_raw_struct lumiera_fileheader_raw; -typedef lumiera_fileheader_raw* LumieraFileheaderRaw; - - -#include "vault/file.h" - - -#include -#include - - - - -#ifndef LUMIERA_PACKED -#define LUMIERA_PACKED __attribute__((__packed__)) -#endif - -/** - * A basic file header - * On-Disk representation starts with 32 bytes identifying the file. - * The first 32 bytes are human readable text. - */ -struct LUMIERA_PACKED lumiera_fileheader_raw_struct -{ - /** four character codes identifying this file type */ - char fourcc[4]; - /** decimal digits, right bound space filled, denoting the file version, 0 is reserved for experimental things */ - char version[3]; - /** always '\n' */ - char newline1; - /** free form string, comment or so on, initialised to spaces */ - char meta[15]; - /** always '\n' */ - char newline2; - - /** Initialised to spaces, flags are single chars, unsorted */ - char flags[6]; - /** always '\n' */ - char newline3; - /** always '\0' */ - char null; - - /* natively written 0x0123456789ABCDEFULL by the host created this */ - uint64_t endianess_mark; -}; - -/** - * File header flags are chars to support debugging and inspection, - * these add another human readable line to the header. - */ - -/** file is clean */ -#define LUMIERA_FILEHEADER_FLAG_CLEAN "c" - -/** check for host order endianess */ -#define LUMIERA_FILEHEADER_FLAG_ENDIANESS "e" - - - - - -/** - * A file header object encapsulates the underlying mmap object which keeps the - * raw header data in memory and and the dereferenced thereof. - */ -struct lumiera_fileheader_struct -{ - LumieraFileheaderRaw header; - LumieraMMap map; -}; - - -/** - * Create a file header on a file open for writing. - * This overwrites any existing date, take care. The created file header is mmaped into memory - * and must be closed after use. The File should be locked for operations on the file header. - * @param file The file on which to create the header. - * @param fourcc pointer to a string of length 4 - * @param version version number for the header (should be incremented after changes) - * the value '0' is reserved for experimental versions. - * @param size The actual size of all header data, including following format specific data. - * @param flags initial flags which should be set (don't include CLEAN here, should be set on close) - * @return A lumiera_fileheader object by value, .header and .map are set to NULL on error. - */ -lumiera_fileheader -lumiera_fileheader_create (LumieraFile file, char* fourcc, int version, size_t size, const char* flags); - - -/** - * Open an existing file header. - * The underlying file might be readonly. The opened file header is mmaped into memory - * and must be closed after use. The File should be locked for operations on the file header. - * @param file The file on which to open the header. - * @param fourcc pointer to a string of length 4 with the expected identifier for the file - * @param size The actual size of all header data, including following format specific data - * @param flags_expected expect this flags being set - * @param flags_remove remove this flags when opening - * @return A lumiera_fileheader object by value, .header and .map are set to NULL on error. - */ -lumiera_fileheader -lumiera_fileheader_open (LumieraFile file, char* fourcc, size_t size, const char* flags_expected, const char* flags_remove); - - -/** - * Closes a previously created or opened file header. - * @param self the file header to close. - * @param flags_add set this flags if not already set - * no errors, no nothing returned (yet) - */ -void -lumiera_fileheader_close (LumieraFileheader self, const char* flags_add); - - -/** - * Queries the version of a file header. - * @param self the file header to query - * @return the version stored in the file header or -1 on error. - */ -int -lumiera_fileheader_version (LumieraFileheader self); - - -/** - * check if all flags given from some sets are either set or not. - */ -int -lumiera_fileheader_flags_validate (LumieraFileheader self, const char* expected, const char* unexpected); - - -/** - * Sets flags if not already set - */ -LumieraFileheader -lumiera_fileheader_flags_set (LumieraFileheader self, const char* flags); - -/** - * Clear flags if present - */ -LumieraFileheader -lumiera_fileheader_flags_clear (LumieraFileheader self, const char* flags); - - -#endif /*VAULT_FILEHEADER_H*/ -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/mmap.c b/src/vault/mmap.c deleted file mode 100644 index 1f15b3054..000000000 --- a/src/vault/mmap.c +++ /dev/null @@ -1,356 +0,0 @@ -/* - mmap.c - memory mapped access to files - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - - -/** @file mmap.c - ** Implementation of management for memory mapped file regions - ** @todo development in this area is stalled since 2010 - */ - - -#include "include/logging.h" -#include "lib/safeclib.h" - -#include "vault/mmap.h" -#include "vault/mmapcache.h" -#include "common/config.h" - -#include -#include -#include - - - -LUMIERA_ERROR_DEFINE (MMAP_NWRITE, "Backing file not writable"); -LUMIERA_ERROR_DEFINE (MMAP_SPACE, "Address space exhausted"); - - - - - -LumieraMMap -lumiera_mmap_init (LumieraMMap self, LumieraFile file, off_t start, size_t size) -{ - TRACE (mmap_dbg); - - TODO ("enforce size instead using chunksize (rounded to page size) for parsing headers"); - - REQUIRE (self); - REQUIRE (file); - REQUIRE (start >= 0); - REQUIRE (size); - - /** - * default size for the mmapping window. - * - 128MB on 32 bit arch - * - 2GB on 64 bit arch - * - * \par - * Maintaining the right[tm] mmapping size is a bit tricky: - * - We have the default mmap_window_size which will be backed off when address space gets exhausted - * - When a bigger size is requested we have to fulfil it - * - The last mmapped chunk of a file can be as small as possible when the file is readonly - * - When the file is writable, the last chunk should be rounded up to chunksize - * - All boundaries will be aligned up/down to chunk boundaries - * - Requests beyond the file end must ftruncate and map additional pages - * - Create the 'refmap' which contains a refcounter per chunk - */ - TODO("move the setdefaults somewhere else, backend_defaults.c or so"); -#if SIZE_MAX <= 4294967295U - lumiera_config_setdefault ("vault.mmap.window_size = 134217728"); -#else - lumiera_config_setdefault ("vault.mmap.window_size = 2147483648"); -#endif - - long long mmap_window_size = 0; - lumiera_config_number_get ("vault.mmap.window_size", &mmap_window_size); - - LumieraFiledescriptor descriptor = file->descriptor; - - int fd = lumiera_file_handle_acquire (file); - TRACE (mmap_dbg, "got fd %d", fd); - if (fd == -1) - goto efile; - - void* addr = (void*)-1; - off_t begin = 0; - size_t length = 0; - size_t chunksize = lumiera_file_chunksize_get (file); - size_t bias = lumiera_file_bias_get (file); - - REQUIRE(start >= (off_t)bias, "begin before first chunk"); - - TODO ("error here? or just map as asked for?"); - ENSURE(chunksize); - - - /** - * Recovering address space strategies: - * mmap() will fail when too much memory got mmapped after some time which is then - * recovered in the following way - * 1. create a new mmap while the cachelimit is not reached. - * 2. All unused mmaps are kept in a mrucache, drop the oldest one. - * mmap() still fails.. - * 3.a When the intended mmapping size is the same as mmap_window_size then reduce (/2) the window size and retry. - * 3.b When the intended mmapping size was bigger than mmap_window_size then free more mmaps from the cache. - * 4 When the cache is empty (that means all mmaps in use), scan the mmaps in use if they can be reduced - * mmap_window_size is already reduced now (half of refmap free from either end) - **/ - enum { - FIRST_TRY, - DROP_FROM_CACHE, - REDUCE_WINDOW, - REDUCE_IN_USE, - GIVE_UP - } strategy = FIRST_TRY; - - while (addr == (void*)-1) - { - TODO ("check if current mmapped size exceeds configured as_size (as_size be smaller than retrieved from getrlimit())"); - - TODO ("use resourcecllector here"); - switch (strategy++) - { - case FIRST_TRY: - TRACE (mmap_dbg, "FIRST_TRY"); - /* align begin and end to chunk boundaries */ - begin = ((start-bias) & ~(chunksize-1)) + bias; - length = ((start+size+chunksize-1) & ~(chunksize-1)) - begin; - - if (begin+(off_t)length > descriptor->stat.st_size) - { - /* request past the end */ - if ((descriptor->flags & O_ACCMODE) == O_RDWR) - { - /* extend file (writable) */ - if (ftruncate (fd, begin+length) == -1) - { - LUMIERA_ERROR_SET (mmap, ERRNO, lumiera_filedescriptor_name (file->descriptor)); - goto etruncate; - }; - descriptor->stat.st_size = begin+length; - descriptor->realsize = start+size; - } - } - else if (length < (size_t)mmap_window_size) - length = mmap_window_size; - - if ((descriptor->flags & O_ACCMODE) == O_RDONLY) - { - /* The last mmapped chunk of a file can be as small as possible when the file is readonly */ - length = start+size - begin; - } - break; - - case DROP_FROM_CACHE: - TRACE (mmap_dbg, "drop a mapping from cache"); - UNIMPLEMENTED ("mmap cache drop"); - break; - - case REDUCE_WINDOW: - NOTICE (mmap_dbg, "mmaping window reduced to NN MB"); - UNIMPLEMENTED ("mmap window reduce"); - break; - - case REDUCE_IN_USE: - NOTICE (mmap_dbg, "reduce mmapings in use"); - UNIMPLEMENTED ("mmapings in use reduce"); - break; - - case GIVE_UP: - LUMIERA_ERROR_SET_ALERT (mmap, MMAP_SPACE, lumiera_filedescriptor_name (file->descriptor)); - goto espace; - } - - addr = mmap (NULL, - length, - (descriptor->flags & O_ACCMODE) == O_RDONLY ? PROT_READ : PROT_READ|PROT_WRITE, - MAP_SHARED, - fd, - begin); - - INFO_IF (addr==(void*)-1, mmap_dbg, "mmap failed %s", strerror (errno)); - ENSURE (errno == 0 || errno == ENOMEM, "unexpected mmap error %s", strerror (errno)); - } - - llist_init (&self->cachenode); - llist_init (&self->searchnode); - - self->start = begin; - self->size = length; - self->address = addr; - self->refmap = lumiera_calloc ((length-1)/chunksize+1, sizeof (unsigned short)); - self->refcnt = 1; - lumiera_mmapcache_announce (self); - - lumiera_file_handle_release (file); - return self; - - espace: - etruncate: - efile: - lumiera_file_handle_release (file); - return NULL; -} - - -LumieraMMap -lumiera_mmap_init_exact (LumieraMMap self, LumieraFile file, off_t start, size_t size) -{ - TRACE (mmap_dbg); - - REQUIRE (self); - REQUIRE (file); - REQUIRE (start >= 0); - REQUIRE (size); - - LumieraFiledescriptor descriptor = file->descriptor; - - int fd = lumiera_file_handle_acquire (file); - TRACE (mmap_dbg, "got fd %d", fd); - if (fd == -1) - goto efile; - - if ((off_t)(start + size) > descriptor->stat.st_size) - { - /* request past the end */ - if ((descriptor->flags & O_ACCMODE) == O_RDWR) - { - /* extend file (writable) */ - if (ftruncate (fd, start+size) == -1) - { - LUMIERA_ERROR_SET (mmap, ERRNO, lumiera_filedescriptor_name (file->descriptor)); - goto etruncate; - }; - descriptor->stat.st_size = descriptor->realsize = start+size; - } - } - - /////////////////////////TODO use resourcecllector here - void* addr = mmap (NULL, - size, - (descriptor->flags & O_ACCMODE) == O_RDONLY ? PROT_READ : PROT_READ|PROT_WRITE, - MAP_SHARED, - fd, - start); - - INFO_IF (addr==(void*)-1, mmap_dbg, "mmap failed %s", strerror (errno)); - ENSURE (errno == 0 || errno == ENOMEM, "unexpected mmap error %s", strerror (errno)); - - llist_init (&self->cachenode); - llist_init (&self->searchnode); - - self->start = start; - self->size = size; - self->address = addr; - self->refmap = NULL; - self->refcnt = 1; - lumiera_mmapcache_announce (self); - - lumiera_file_handle_release (file); - return self; - - etruncate: - efile: - lumiera_file_handle_release (file); - return NULL; -} - - -LumieraMMap -lumiera_mmap_new (LumieraFile file, off_t start, size_t size) -{ - TRACE (mmap_dbg); - - LumieraMMap self = lumiera_mmapcache_mmap_acquire (); - - if (lumiera_mmap_init (self, file, start, size)) - return self; - else - { - lumiera_free (self); - return NULL; - } -} - - -LumieraMMap -lumiera_mmap_new_exact (LumieraFile file, off_t start, size_t size) -{ - TRACE (mmap_dbg); - - LumieraMMap self = lumiera_mmapcache_mmap_acquire (); - - if (lumiera_mmap_init_exact (self, file, start, size)) - return self; - else - { - lumiera_free (self); - return NULL; - } -} - - -void -lumiera_mmap_delete (LumieraMMap self) -{ - TRACE (mmap_dbg); - if (self) - { - REQUIRE (self->refcnt <= 1); - lumiera_mmapcache_forget (self); - - /* The matching mappings->lock must be hold or being irrelevant (mappings destructor) here, we can't asset this from here, good luck */ - llist_unlink (&self->searchnode); - - TRACE (mmap_dbg, "unmap at %p with size %zd", self->address, self->size); - munmap (self->address, self->size); - lumiera_free (self->refmap); - lumiera_free (self); - } -} - - -void* -lumiera_mmap_destroy_node (LList node) -{ - TRACE (mmap_dbg); - REQUIRE (llist_is_empty (node)); - LumieraMMap self = (LumieraMMap)node; - - lumiera_mmapcache_forget (self); - - llist_unlink (&self->searchnode); FIXME ("must lock mmappings -> deadlock"); - - munmap (self->address, self->size); - lumiera_free (self->refmap); - - return self; -} - -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/mmap.h b/src/vault/mmap.h deleted file mode 100644 index 2efb56891..000000000 --- a/src/vault/mmap.h +++ /dev/null @@ -1,135 +0,0 @@ -/* - MMAP - memory mapped access to files - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -*/ - - -/** @file mmap.h - ** MMap objects cover a memory mapped range in a file. - ** They are managed through a global mmap registry/cache. - */ - -#ifndef VAULT_MMAP_H -#define VAULT_MMAP_H - -#include "lib/llist.h" - - -typedef struct lumiera_mmap_struct lumiera_mmap; -typedef lumiera_mmap* LumieraMMap; - - -#include "vault/file.h" -#include "vault/filedescriptor.h" - - -#include -#include - - - - - - -/** - * Descriptor of a memory mapped area - */ -struct lumiera_mmap_struct -{ - /** used for the mrucache when checked in the cache **/ - llist cachenode; - - /** all mmaps regarding a file are chained in this list, used to find ranges **/ - llist searchnode; - - off_t start; - size_t size; - void* address; - - /** accumulated references, this is 0 when checked into the cache **/ - long refcnt; - - /** array with a refcounter per chunk **/ - short* refmap; ///////////////////// TODO make this a flexible array? -}; - - -/** - * Initialise a MMap object. - * The mmap objects are aligned and shifted by the chunksize and bias defined for the file - * @param self the mmap object to be initialised - * @param file file from which to map - * @param start offset in file which must be part of the mmaped region - * @param size minimum size after start to map - * @return self on success or NULL on error - */ -LumieraMMap -lumiera_mmap_init (LumieraMMap self, LumieraFile file, off_t start, size_t size); - -/** - * Initialise a MMap object. - * Maps exactly the given range - * @param self the mmap object to be initialised - * @param file file from which to map - * @param start offset in file which must be part of the mmaped region - * @param size minimum size after start to map - * @return self on success or NULL on error - */ -LumieraMMap -lumiera_mmap_init_exact (LumieraMMap self, LumieraFile file, off_t start, size_t size); - - -LumieraMMap -lumiera_mmap_new (LumieraFile file, off_t start, size_t size); - -LumieraMMap -lumiera_mmap_new_exact (LumieraFile file, off_t start, size_t size); - - -/** - * Translate a 'external' offset to a address in memory - * @param self MMap object to query - * @param offset position on the mmaped file to get - * @return address in memory which relates to offset - */ -static inline void* -lumiera_mmap_address (LumieraMMap self, off_t offset) -{ - REQUIRE_IF (self, offset >= self->start, "offset before mmaped region"); - REQUIRE_IF (self, offset < self->start + (off_t)self->size, "offset after mmaped region"); - return self?(self->address + (offset - self->start)):NULL; -} - -void -lumiera_mmap_delete (LumieraMMap self); - - -void* -lumiera_mmap_destroy_node (LList node); - - -#endif /*VAULT_MMAP_H*/ -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/mmapcache.c b/src/vault/mmapcache.c deleted file mode 100644 index a7668b4a9..000000000 --- a/src/vault/mmapcache.c +++ /dev/null @@ -1,175 +0,0 @@ -/* - MMAP-Cache - handle aging of mmap objects - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - - -/** @file mmapcache.c - ** Implementation of caching for currently unused memory mapped file regions - ** @todo development in this area is stalled since 2010 - */ - - -#include "include/logging.h" -#include "lib/safeclib.h" - -#include "vault/mmapcache.h" - - - -LumieraMMapcache lumiera_mcache = NULL; - - -void -lumiera_mmapcache_new (size_t limit) -{ - TRACE (mmapcache_dbg); - lumiera_mcache = lumiera_malloc (sizeof (*lumiera_mcache)); - - lumiera_mrucache_init (&lumiera_mcache->cache, lumiera_mmap_destroy_node); - - lumiera_mcache->limit = limit; - lumiera_mcache->total = 0; - lumiera_mcache->cached = 0; - - lumiera_mutex_init (&lumiera_mcache->lock, "mmapcache", &NOBUG_FLAG (mutex_dbg), NOBUG_CONTEXT); -} - - -void -lumiera_mmapcache_delete (void) -{ - TRACE (mmapcache_dbg); - if (lumiera_mcache) - { - REQUIRE (lumiera_mcache->total == lumiera_mcache->cached, "MMaps still checked out at shutdown"); - lumiera_mrucache_destroy (&lumiera_mcache->cache); - lumiera_mutex_destroy (&lumiera_mcache->lock, &NOBUG_FLAG (mutex_dbg), NOBUG_CONTEXT); - free (lumiera_mcache); - lumiera_mcache = NULL; - } -} - - -void* -lumiera_mmapcache_mmap_acquire (void) -{ - TRACE (mmapcache_dbg); - void* map = NULL; - - LUMIERA_MUTEX_SECTION (mutex_sync, &lumiera_mcache->lock) - { - map = lumiera_mrucache_pop (&lumiera_mcache->cache); - } - - if (!map) - { - map = lumiera_malloc (sizeof (lumiera_mmap)); - TRACE (mmapcache_dbg, "allocated new mmap"); - } - else - { - TRACE (mmapcache_dbg, "Popped mmap from cache"); - } - - return map; -} - - -void -lumiera_mmapcache_announce (LumieraMMap map) -{ - TRACE (mmapcache_dbg); - LUMIERA_MUTEX_SECTION (mutex_sync, &lumiera_mcache->lock) - { - lumiera_mcache->total += map->size; - } -} - - -void -lumiera_mmapcache_forget (LumieraMMap map) -{ - TRACE (mmapcache_dbg); - LUMIERA_MUTEX_SECTION (mutex_sync, &lumiera_mcache->lock) - { - if (!llist_is_empty (&map->cachenode)) - { - TODO ("cached stats"); - REQUIRE (llist_is_member (&lumiera_mcache->cache.cache_list, &map->cachenode), "Map object not in cache"); - llist_unlink (&map->cachenode); - } - lumiera_mcache->total -= map->size; - } -} - -#if 0 -int -lumiera_mmapcache_age (void) -{ - TRACE (mmapcache_dbg); - int ret = 0; - - LUMIERA_MUTEX_SECTION (mmapcache, &lumiera_mcache->lock) - { - ret = lumiera_mrucache_age (&lumiera_mcache->cache, 10); TODO ("age nelem == 20%(configurable) of the cache"); - } - - return ret; -} -#endif - -LumieraMMap -lumiera_mmapcache_checkout (LumieraMMap handle) -{ - TRACE (mmapcache_dbg); - REQUIRE (handle->refcnt == 0); - - LUMIERA_MUTEX_SECTION (mutex_sync, &lumiera_mcache->lock) - { - ////////////////////TODO cached stats - lumiera_mrucache_checkout (&lumiera_mcache->cache, &handle->cachenode); - } - - return handle; -} - - -void -lumiera_mmapcache_checkin (LumieraMMap handle) -{ - TRACE (mmapcache_dbg); - REQUIRE (handle->refcnt == 0); - - LUMIERA_MUTEX_SECTION (mutex_sync, &lumiera_mcache->lock) - { - ////////////////////TODO cached stats - lumiera_mrucache_checkin (&lumiera_mcache->cache, &handle->cachenode); - } -} - - -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/mmapcache.h b/src/vault/mmapcache.h deleted file mode 100644 index 11082994e..000000000 --- a/src/vault/mmapcache.h +++ /dev/null @@ -1,131 +0,0 @@ -/* - MMAPCACHE.h - handle aging of mmap objects - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -*/ - - -/** @file mmapcache.h - ** Lumiera's Mmapcache stores a MRU cache of all established mmaped memory regions which are currently not in use. - ** The mmapcache also manages the upper limit about how much memory can be mmaped. - */ - -#ifndef VAULT_MMAPCACHE_H -#define VAULT_MMAPCACHE_H - -#include "lib/error.h" -#include "lib/mrucache.h" -#include "lib/mutex.h" - - -typedef struct lumiera_mmapcache_struct lumiera_mmapcache; -typedef lumiera_mmapcache* LumieraMMapcache; - -#include "vault/mmap.h" - -#include - - - - -struct lumiera_mmapcache_struct -{ - lumiera_mrucache cache; - size_t limit; - size_t total; - size_t cached; - lumiera_mutex lock; -}; - - -/** - * Initialises the mmapcache. - * @param limit the mmapcache will drop elements when the sum of all mmappings gives over limit - */ -void -lumiera_mmapcache_new (size_t limit); - -/** - * Delete the mmap cache. - * No mmaps in the cache must be locked, this would be a fatal error. - * The handles are closed automatically. - */ -void -lumiera_mmapcache_delete (void); - -/** - * Get a fresh mmap object. - * when mmaped_limit is reached, the oldest mmap object gets dropped else - * a new allocated object is returned - * @return the new uninitialised mmap (void* because this is uninitialised) - */ -void* -lumiera_mmapcache_mmap_acquire (void); - - -/** - * Announce a new mmap object to the cache quotas. - * Update the statistics kept in the cache, - * the map object is still considered to be checked out - * @param map object to be announced - */ -void -lumiera_mmapcache_announce (LumieraMMap map); - -/** - * Remove a mmap object from the cache quotas. - * Update the statistics kept in the cache, remove it from the cache. - * @param map object to be removed - */ -void -lumiera_mmapcache_forget (LumieraMMap map); - -/** - * Destroy and free the nelem oldest elements. - * Used to free up resources and memory. - * @return nelem-(number of elements which got freed), that is 0 if all requested elements got freed - */ -int -lumiera_mmapcache_age (void); - -/** - * Remove a mmap from cache aging - * Mmaps which are subject of cache aging must be checked out before they can be used. - * @param handle the mmap to be checked out - */ -LumieraMMap -lumiera_mmapcache_checkout (LumieraMMap handle); - -/** - * Put a mmap into the cache - * Mmaps which are checked in are subject of cache aging and might get destroyed and reused. - * @param handle the mmap to be checked in - */ -void -lumiera_mmapcache_checkin (LumieraMMap handle); - - -#endif /*VAULT_MMAPCACHE_H*/ -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/mmapings.c b/src/vault/mmapings.c deleted file mode 100644 index 1f901bf7b..000000000 --- a/src/vault/mmapings.c +++ /dev/null @@ -1,168 +0,0 @@ -/* - MMapings - manage ranges of mmaped areas on a file descriptor - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - - -/** @file mmapings.c - ** Implementation of organisational data of memory mapped file regions - ** @todo development in this area is stalled since 2010 - */ - - -#include "include/logging.h" -#include "lib/mutex.h" -#include "lib/safeclib.h" - - -#include "vault/mmapings.h" -#include "vault/mmapcache.h" - - - -LumieraMMapings -lumiera_mmapings_init (LumieraMMapings self, LumieraFile file, size_t chunksize, size_t bias) -{ - TRACE (mmapings_dbg); - REQUIRE (!file->descriptor->mmapings); - - llist_init (&self->mmaps); - self->descriptor = file->descriptor; - TODO("align chunksize on 2's exponent or error out?"); - self->chunksize = chunksize; - self->bias = bias; - - lumiera_mutex_init (&self->lock, "mmapings", &NOBUG_FLAG(mutex_dbg), NOBUG_CONTEXT); - - return self; -} - - -LumieraMMapings -lumiera_mmapings_destroy (LumieraMMapings self) -{ - TRACE (mmapings_dbg); - if (!self) - return NULL; - - LLIST_WHILE_TAIL (&self->mmaps, node) - { - LumieraMMap map = LLIST_TO_STRUCTP (node, lumiera_mmap, searchnode); - REQUIRE (map->refcnt == 0, "map still in use: %p", map); - lumiera_mmap_delete (map); - } - - lumiera_mutex_destroy (&self->lock, &NOBUG_FLAG(mutex_dbg), NOBUG_CONTEXT); - - return self; -} - - -LumieraMMapings -lumiera_mmapings_new (LumieraFile file, size_t chunksize, size_t bias) -{ - TRACE (mmapings_dbg); - LumieraMMapings self = lumiera_malloc (sizeof (*self)); - return lumiera_mmapings_init (self, file, chunksize, bias); -} - - -void -lumiera_mmapings_delete (LumieraMMapings self) -{ - TRACE (mmapings_dbg); - free (lumiera_mmapings_destroy (self)); -} - - -LumieraMMap -lumiera_mmapings_mmap_acquire (LumieraMMapings self, LumieraFile file, off_t start, size_t size) -{ - TRACE (mmapings_dbg); - - LumieraMMap ret = NULL; - - if (self) - LUMIERA_MUTEX_SECTION (mutex_sync, &self->lock) - { - /* find first matching mmap, crude way */ - LLIST_FOREACH (&self->mmaps, node) - { - ////////////TODO improve the algorithm used here: choose mmaps by size, move mfu to head etc... - - LumieraMMap mmap = LLIST_TO_STRUCTP (node, lumiera_mmap, searchnode); - - if (mmap->size >= size && mmap->start <= start && mmap->start+mmap->size >= start+size) - { - ret = mmap; - break; - } - } - - /* found? */ - if (ret) - { - if (!ret->refcnt) - /* in cache, needs to be checked out */ - lumiera_mmapcache_checkout (ret); - ++ret->refcnt; - } - else - { - /* create new mmap */ - TRACE (mmapings_dbg, "mmap not found, creating"); - ret = lumiera_mmap_new (file, start, size); - - llist_insert_head (&self->mmaps, &ret->searchnode); - - TODO ("sort search list?"); - } - - PLANNED ("use refmap for finer grained refcounting"); - - ENSURE (llist_is_empty(&ret->cachenode)); - } - - return ret; -} - -void -lumiera_mmapings_release_mmap (LumieraMMapings self, LumieraMMap map) -{ - TRACE (mmapings_dbg); - - if (self) - LUMIERA_MUTEX_SECTION (mutex_sync, &self->lock) - { - if (!--map->refcnt) - { - TRACE (mmapcache_dbg, "checkin"); - lumiera_mmapcache_checkin (map); - } - } -} - -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/mmapings.h b/src/vault/mmapings.h deleted file mode 100644 index 4d6969ace..000000000 --- a/src/vault/mmapings.h +++ /dev/null @@ -1,118 +0,0 @@ -/* - MMAPINGS.h - manage ranges of mmaped areas on a file descriptor - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -*/ - - -/** @file mmapings.h - ** Manage the mmap objects of a file. - */ - -#ifndef VAULT_MMAPINGS_H -#define VAULT_MMAPINGS_H - -#include "lib/mutex.h" -#include "lib/llist.h" - -typedef struct lumiera_mmapings_struct lumiera_mmapings; -typedef lumiera_mmapings* LumieraMMapings; - -#include "vault/filedescriptor.h" -#include "vault/mmap.h" -#include "vault/file.h" - -#include - - - - -struct lumiera_mmapings_struct -{ - /** mmaped ranges are kept in an list sorted by the size of the mmaping, might be improved to a tree someday **/ - llist mmaps; - - /** - * chunksize is the smallest granularity which is used for mmapping files, it - * should reflect the intended file usage, that is 'pagesize' for small or non growing - * files and some MB for media files. Must be a 2's exponent of pagesize. - **/ - size_t chunksize; - - /** - * bias shifts the chunk begin to suppress headers for example - **/ - size_t bias; - - LumieraFiledescriptor descriptor; - lumiera_mutex lock; -}; - - - -/** initialise mmapings container */ -LumieraMMapings -lumiera_mmapings_init (LumieraMMapings self, LumieraFile file, size_t chunksize, size_t bias); - - -/** destroy mmapings container and free all resources. */ -LumieraMMapings -lumiera_mmapings_destroy (LumieraMMapings self); - - -/** allocate and initialise new mmapings container */ -LumieraMMapings -lumiera_mmapings_new (LumieraFile file, size_t chunksize, size_t bias); - - -/** destroy and free mmapings container and all its resources */ -void -lumiera_mmapings_delete (LumieraMMapings self); - - -/** - * acquire a mmap which covers the given range - * @param self mmapings where to search - * @param start begin of the required range - * @param size requested size - * @return MMap object covering the requested range or NULL on error - */ -LumieraMMap -lumiera_mmapings_mmap_acquire (LumieraMMapings self, LumieraFile file, off_t start, size_t size); - - -/** - * release a previously acquired MMap object - * @param self mmapings to which the map belongs - * @param map object to be released - */ -void -lumiera_mmapings_release_mmap (LumieraMMapings self, LumieraMMap map); - - - - -#endif /*VAULT_MMAPINGS_H*/ -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/resourcecollector.c b/src/vault/resourcecollector.c deleted file mode 100644 index 16d25ddea..000000000 --- a/src/vault/resourcecollector.c +++ /dev/null @@ -1,192 +0,0 @@ -/* - ResourceCollector - manage/collect resources when they get short - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - - -/** @file resourcecollector.c - ** Implementation of a global service to manage scarce system resources - ** @todo this was a plan from 2008 and never got beyond an initial concept stage - */ - - -#include "include/logging.h" -#include "lib/llist.h" -#include "lib/mutex.h" -#include "lib/safeclib.h" - -#include "vault/resourcecollector.h" - -#include - - - - -llist lumiera_resourcecollector_registry[LUMIERA_RESOURCE_END]; -lumiera_mutex lumiera_resourcecollector_lock; - -struct lumiera_resourcehandler_struct -{ - llist node; - lumiera_resource_handler_fn handler; - void* data; -}; - - -void -lumiera_resourcecollector_init (void) -{ - //NOBUG_INIT_FLAG (resourcecollector); - TRACE (resourcecollector_dbg); - - for (int i = 0; i < LUMIERA_RESOURCE_END; ++i) - llist_init (&lumiera_resourcecollector_registry[i]); - - lumiera_mutex_init (&lumiera_resourcecollector_lock, "resourcecollector", &NOBUG_FLAG(mutex_dbg), NOBUG_CONTEXT); -} - - - -void -lumiera_resourcecollector_destroy (void) -{ - TRACE (resourcecollector_dbg); - - for (int i = 0; i < LUMIERA_RESOURCE_END; ++i) - LLIST_WHILE_HEAD (&lumiera_resourcecollector_registry[i], head) - lumiera_resourcehandler_unregister ((LumieraResourcehandler)head); - - lumiera_mutex_destroy (&lumiera_resourcecollector_lock, &NOBUG_FLAG(mutex_dbg), NOBUG_CONTEXT); -} - - -int -lumiera_resourcecollector_run (enum lumiera_resource which, enum lumiera_resource_try* iteration, void* context) -{ - TRACE (resourcecollector_dbg); - - LUMIERA_MUTEX_SECTION (mutex_sync, &lumiera_resourcecollector_lock) - { - for (enum lumiera_resource_try progress = LUMIERA_RESOURCE_NONE; progress < *iteration; ++*iteration) - { - if (*iteration < LUMIERA_RESOURCE_PANIC) - { - LLIST_FOREACH (&lumiera_resourcecollector_registry[which], node) - { - LumieraResourcehandler self = (LumieraResourcehandler) node; - progress = self->handler (*iteration, self->data, context); - - if (*iteration < LUMIERA_RESOURCE_ALL) - { - if (progress >= *iteration) - { - llist_insert_next (node, &lumiera_resourcecollector_registry[which]); - break; - } - } - } - } - else - { - ALERT (resourcecollector, "PANIC, Not enough resources %d", which); - for (int i = 0; i < LUMIERA_RESOURCE_END; ++i) - LLIST_FOREACH (&lumiera_resourcecollector_registry[i], node) - { - LumieraResourcehandler self = (LumieraResourcehandler) node; - progress = self->handler (LUMIERA_RESOURCE_PANIC, self->data, NULL); - } - _exit (EXIT_FAILURE); - } - } - } - - return 1; -} - - - -LumieraResourcehandler -lumiera_resourcecollector_register_handler (enum lumiera_resource resource, lumiera_resource_handler_fn handler, void* data) -{ - TRACE (resourcecollector_dbg); - - LumieraResourcehandler self = lumiera_malloc (sizeof (*self)); - - llist_init (&self->node); - self->handler = handler; - self->data = data; - - LUMIERA_MUTEX_SECTION (mutex_sync, &lumiera_resourcecollector_lock) - { - llist_insert_tail (&lumiera_resourcecollector_registry[resource], &self->node); - } - - return self; -} - - -void -lumiera_resourcehandler_unregister (LumieraResourcehandler self) -{ - TRACE (resourcecollector_dbg); - - if (self) - { - LUMIERA_MUTEX_SECTION (mutex_sync, &lumiera_resourcecollector_lock) - { - llist_unlink (&self->node); - self->handler (LUMIERA_RESOURCE_UNREGISTER, self->data, NULL); - } - - lumiera_free (self); - } -} - - -LumieraResourcehandler -lumiera_resourcecollector_handler_find (enum lumiera_resource resource, lumiera_resource_handler_fn handler, void* data) -{ - TRACE (resourcecollector_dbg); - LumieraResourcehandler self = NULL; - - LUMIERA_MUTEX_SECTION (mutex_sync, &lumiera_resourcecollector_lock) - { - LLIST_FOREACH (&lumiera_resourcecollector_registry[resource], node) - { - self = (LumieraResourcehandler) node; - - if (((LumieraResourcehandler)node)->handler == handler && ((LumieraResourcehandler)node)->data == data) - { - self = (LumieraResourcehandler)node; - break; - } - } - } - - return self; -} - -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/resourcecollector.h b/src/vault/resourcecollector.h deleted file mode 100644 index dc7d35d48..000000000 --- a/src/vault/resourcecollector.h +++ /dev/null @@ -1,191 +0,0 @@ -/* - RESOURCECOLLECTOR.h - manage/collect resources when they get short - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -*/ - - -/** @file resourcecollector.h - ** A global manager for scarce system resources - ** @todo this was a plan from 2008 and never got beyond an initial concept stage - */ - - -#ifndef VAULT_RESOURCECOLLECTOR_H -#define VAULT_RESOURCECOLLECTOR_H - -#include - - - -/** - * Resources known to the resource collector - */ -enum lumiera_resource - { - /** memory blocks, context is a pointer to the size_t required **/ - LUMIERA_RESOURCE_MEMORY, - /** OS filehandles **/ - LUMIERA_RESOURCE_FILEHANDLE, - /** CPU time, as in threads and such **/ - LUMIERA_RESOURCE_CPU, - /** mmaped regions **/ - LUMIERA_RESOURCE_MMAP, - /** disk space for the storage area, context is a pointer to the filename indication the device **/ - LUMIERA_RESOURCE_DISKSTORAGE, - /** disk bandwidth for the storage area, context is a pointer to the filename indication the device **/ - LUMIERA_RESOURCE_STORAGEBANDWIDTH, - /** disk space for the caching area, context is a pointer to the filename indication the device **/ - LUMIERA_RESOURCE_DISKCACHE, - /** disk bandwidth for the caching area, context is a pointer to the filename indication the device **/ - LUMIERA_RESOURCE_CACHEBANDWIDTH, - - LUMIERA_RESOURCE_END /* last entry */ - }; - - -/** - * Iteration indicator - * Resource collection works iteratively freeing more and more resources. - * Handlers do not need to obey the request and shall return LUMIERA_RESOURCE_NONE - * which will then continue with the next handler. - * This goes through all available handlers until one returns a higher or same value - * than the current iteration to indicate that it freed enough resources to continue the task. - * Then control is passed back to the calling loop which retries the resource allocation. - * LUMIERA_RESOURCE_PANIC is somewhat special since it will always call all registered handlers - * for all resources, not only the queried one and finally _exit() the application. - * The exact amounts of resources to be freed for ONE, SOME and MANY in intentionally - * kept vague the handlers are free to interpret this in some sensible way. - */ -enum lumiera_resource_try - { - /** No op, returned by a handler when it did nothing **/ - LUMIERA_RESOURCE_NONE, - /** try to free one or really few of this resources **/ - LUMIERA_RESOURCE_ONE, - /** try to free a small reasonable implementation defined amount of resources **/ - LUMIERA_RESOURCE_SOME, - /** try to free a bigger implementation defined amount of resources **/ - LUMIERA_RESOURCE_MANY, - /** free as much as possible **/ - LUMIERA_RESOURCE_ALL, - /** die! **/ - LUMIERA_RESOURCE_PANIC, - /** When a handler gets unregistered it will be called with this value to give it a chance to clean up the user 'data' **/ - LUMIERA_RESOURCE_UNREGISTER - }; - - -/** - * The type for the resource collector handler functions. - * Handlers are always run with a global resourcecollector mutex locked, the user does not need to - * care about synchronisation. - * @param itr the current iteration try in freeing resources - * @param data user supplied data at registration time for the handler - * @param context context pointer for this collection run, might be NULL (at least for UNREGISTER and PANIC) - * @return indication what the the handler really did (LUMIERA_RESOURCE_NONE when it didn't obey the request) - */ -typedef enum lumiera_resource_try (*lumiera_resource_handler_fn)(enum lumiera_resource_try itr, void* data, void* context); -typedef int (*lumiera_resourcecollector_run_fn) (enum lumiera_resource which, enum lumiera_resource_try* iteration, void* context); - -typedef struct lumiera_resourcehandler_struct lumiera_resourcehandler; -typedef lumiera_resourcehandler* LumieraResourcehandler; - - -/** - * Initialise the Resourcecollector. - * The Resourcecollector is singleton and can be used after initialised once. - */ -void -lumiera_resourcecollector_init (void); - - -/** - * Destroy the resource collector registry. - * Unregisters and deletes all handlers. - */ -void -lumiera_resourcecollector_destroy (void); - - -/** - * Try to free resources. - * - * @param which The kind of resource to be acquired - * @param iteration a pointer to a local iterator, initialised with the start - * value for the loop - * @param context NULL or some context dependent data for the needed resource - * this is a pointer to a size_t for MEMORY and a pointer to a filename - * (to find out about the device) for STORAGE and CACHE resources - * @return either returns 1 or calls _exit() - * - * @code - * void* data; - * size_t size = 1000; - * enum lumiera_resource_try iteration = LUMIERA_RESOURCE_ONE; - * do { - * data = malloc (size); - * } while (!data && lumiera_resourcecollector_run (LUMIERA_RESOURCE_MEMORY, &iteration, &size)); - * @endcode - */ -int -lumiera_resourcecollector_run (enum lumiera_resource which, enum lumiera_resource_try* iteration, void* context); - - -/** - * Registers a new collector handler - * @param resource resource for which this handler shall be registered - * @param handler pointer to the handler function - * @param data opaque user-data pointer which will be passed to the handler - * @return pointer to the internal handler structure. This can be used to unregister the handler. - */ -LumieraResourcehandler -lumiera_resourcecollector_register_handler (enum lumiera_resource resource, lumiera_resource_handler_fn handler, void* data); - - -/** - * Unregisters a collector handle - * Removes the handler from the registry and calls it once with LUMIERA_RESOURCE_UNREGISTER - * to give it a chance to free the user supplied data. Must not be called after lumiera_resourcecollector_destroy() - * @param self pointer to internal handler structure, obtained from register_handler() or handler_find(), might be NULL - */ -void -lumiera_resourcehandler_unregister (LumieraResourcehandler self); - -/** - * Looks up a handler. - * Used to find a registered handler when the return value of register_handler() was not practical to store. - * @param resource resource for which this handler was registered - * @param handler pointer to the handler function, same as used for registering - * @param data opaque user-data pointer, same as used for registering - * @return pointer to the internal handler structure or NULL if no handler was found - */ -LumieraResourcehandler -lumiera_resourcecollector_handler_find (enum lumiera_resource resource, lumiera_resource_handler_fn handler, void* data); - - - -#endif /*VAULT_RESOURCECOLLECTOR_H*/ -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/thread-wrapper.hpp b/src/vault/thread-wrapper.hpp deleted file mode 100644 index fc95284de..000000000 --- a/src/vault/thread-wrapper.hpp +++ /dev/null @@ -1,300 +0,0 @@ -/* - THREADWRAPPER.hpp - thin convenience wrapper for starting lumiera threads - - Copyright (C) Lumiera.org - 2008, 2010 Hermann Vosseler - Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -*/ - - -/** @file thread-wrapper.hpp - ** Convenience front-end for basic thread handling needs. - ** The Lumiera vault contains a dedicated low-level thread handling framework, - ** which is relevant for scheduling render activities to make best use of parallelisation - ** abilities of the given system. Typically, the upper layers should not have to deal much - ** with thread handling, yet at some point there is the need to implement a self contained - ** action running within a dedicated thread. The vault::Thread class is a wrapper to - ** represent such an parallel action conveniently and safely; together with the object - ** monitor, this allows to abstract away intricacies into self contained objects. - ** - ** @deprecated will be replaced by a thin wrapper on top of C++17 threads //////////////////////////////TICKET #1279 : consolidate to C++17 features - */ - - -#ifndef LIB_THREADWRAPPER_H -#define LIB_THREADWRAPPER_H - - -#include "lib/error.hpp" -#include "lib/nocopy.hpp" -#include "include/logging.h" -#include "lib/meta/function.hpp" -#include "lib/result.hpp" - -extern "C" { -#include "vault/threads.h" -} -#include "vault/threadpool-init.hpp" - -#include -#include - - -namespace vault { - - using lib::Literal; - namespace error = lumiera::error; - using error::LERR_(STATE); - using error::LERR_(EXTERNAL); - - typedef struct nobug_flag* NoBugFlag; - - - - /************************************************************************//** - * A thin convenience wrapper for dealing with threads, - * as implemented by the threadpool in the vault (based on pthread). - * Using this wrapper... - * - helps with passing data to the function executed in the new thread - * - allows to bind to various kinds of functions including member functions - * The new thread starts immediately within the ctor; after returning, the new - * thread has already copied the arguments and indeed actively started to run. - * - * # Joining, cancellation and memory management - * In the basic version (class Thread), the created thread is completely detached - * and not further controllable. There is no way to find out its execution state, - * wait on termination or even cancel it. Client code needs to implement such - * facilities explicitly, if needed. Care has to be taken with memory management, - * as there are no guarantees beyond the existence of the arguments bound into - * the operation functor. If the operation in the started thread needs additional - * storage, it has to manage it actively. - * - * There is an extended version (class ThreadJoinable) to allow at least to wait - * on the started thread's termination (joining). Building on this it is possible - * to create a self-contained "thread in an object"; the dtor of such an class - * must join to prevent pulling away member variables the thread function will - * continue to use. - * - * # failures in the thread function - * The operation started in the new thread is protected by a top-level catch block. - * Error states or caught exceptions can be propagated through the lumiera_error - * state flag, when using ThreadJoinable::join(). By invoking `join().maybeThrow()` - * on a join-able thread, exceptions can be propagated. - * @note any errorstate or caught exception detected on termination of a standard - * async Thread is considered a violation of policy and will result in emergency - * shutdown of the whole application. - * - * # synchronisation barriers - * Lumiera threads provide a low-level synchronisation mechanism, which is used - * to secure the hand-over of additional arguments to the thread function. It - * can be used by client code, but care has to be taken to avoid getting out - * of sync. When invoking the #sync and #syncPoint functions, the caller will - * block until the counterpart has also invoked the corresponding function. - * If this doesn't happen, you'll block forever. - * @deprecated will be replaced by a thin wrapper on top of C++17 threads /////////////////////////////TICKET #1279 : consolidate to C++17 features - */ - class Thread - : util::MoveOnly - { - /** @internal perfect forwarding through a C-style `void*` */ - template - static FUN&& - forwardInitialiser (void* rawPtr) noexcept - { - REQUIRE (rawPtr); - FUN& initialiser = *reinterpret_cast (rawPtr); - return static_cast (initialiser); - } - - - template - static void - threadMain (void* arg) - { - using Fun= typename lib::meta::_Fun::Functor; - Fun _doIt_{forwardInitialiser (arg)}; - - lumiera_thread_sync (); // sync point: arguments handed over - - try { - _doIt_(); // execute the actual operation in the new thread - } - - catch (std::exception& failure) - { - if (!lumiera_error_peek()) - LUMIERA_ERROR_SET (sync, STATE - ,failure.what()); - } - catch (...) - { - LUMIERA_ERROR_SET_ALERT (sync, EXTERNAL - , "Thread terminated abnormally"); - } - } - - - protected: - LumieraThread threadHandle_; - - /** @internal derived classes may create an inactive thread */ - Thread() : threadHandle_(0) { } - - - /** @internal use the Lumiera thread manager to start a new thread and hand over the operation */ - template - void - launchThread (Literal purpose, FUN&& operation, NoBugFlag logging_flag, uint additionalFlags =0) - { - REQUIRE (!lumiera_error(), "Error pending at thread start"); - using Functor = typename std::remove_reference::type; - threadHandle_ = - lumiera_thread_run ( LUMIERA_THREADCLASS_INTERACTIVE | additionalFlags - , &threadMain - , reinterpret_cast (&operation) - , purpose.c() - , logging_flag - ); - if (!threadHandle_) - throw error::State ("Failed to start a new Thread for \"+purpose+\"" - , lumiera_error()); - - // make sure the new thread had the opportunity to take the Operation - // prior to leaving and thereby possibly destroying this local context - lumiera_thread_sync_other (threadHandle_); - } - - - - public: - /** Create a new thread to execute the given operation. - * The new thread starts up synchronously, can't be cancelled and it can't be joined. - * @param purpose fixed char string used to denote the thread for diagnostics - * @param logging_flag NoBug flag to receive diagnostics regarding the new thread - * @param operation a functor holding the code to execute within the new thread. - * Any function-like entity with signature `void(void)` is acceptable. - * @warning The operation functor will be forwarded to create a copy residing - * on the stack of the new thread; thus it can be transient, however - * anything referred through a lambda closure here must stay alive - * until the new thread terminates. - */ - template - Thread (Literal purpose, FUN&& operation, NoBugFlag logging_flag = &NOBUG_FLAG(thread)) - : threadHandle_{nullptr} - { - launchThread (purpose, std::forward (operation), logging_flag); - } - - - /** @note by design there is no possibility to find out - * just based on the thread handle if some thread is alive. - * We define our own accounting here based on the internals - * of the thread wrapper. This will break down, if you mix - * uses of the C++ wrapper with the raw C functions. */ - bool - isValid() const - { - return threadHandle_; - } - - - /** Synchronisation barrier. In the function executing in this thread - * needs to be a corresponding Thread::syncPoint() call. Blocking until - * both the caller and the thread have reached the barrier. - */ - void - sync() - { - REQUIRE (isValid(), "Thread not running"); - if (!lumiera_thread_sync_other (threadHandle_)) - lumiera::throwOnError(); - } - - /** counterpart of the synchronisation barrier, to be called from - * within the thread to be synchronised. Will block until both - * this thread and the outward partner reached the barrier. - * @warning blocks on the _current_ thread's condition var - */ - static void - syncPoint () - { - lumiera_thread_sync (); - } - - protected: - /** determine if the currently executing code runs within this thread */ - bool - invokedWithinThread() const - { - REQUIRE (isValid(), "Thread not running"); - LumieraThread current = lumiera_thread_self (); - return current - and current == this->threadHandle_; - } - }; - - - - - - - /** - * Variant of the standard case, allowing additionally - * to join on the termination of this thread. - * @deprecated will be replaced by a thin wrapper on top of C++17 threads /////////////////////////////TICKET #1279 : consolidate to C++17 features - */ - class ThreadJoinable - : public Thread - { - public: - template - ThreadJoinable (Literal purpose, FUN&& operation, - NoBugFlag logging_flag = &NOBUG_FLAG(thread)) - : Thread{} - { - launchThread (purpose, std::forward (operation), logging_flag, - LUMIERA_THREAD_JOINABLE); - } - - - /** put the caller into a blocking wait until this thread has terminated. - * @return token signalling either success or failure. - * The caller can find out by invoking `isValid()` - * or `maybeThrow()` on this result token - */ - lib::Result - join () - { - if (!isValid()) - throw error::Logic ("joining on an already terminated thread"); - - lumiera_err errorInOtherThread = - lumiera_thread_join (threadHandle_); - threadHandle_ = 0; - - if (errorInOtherThread) - return error::State ("Thread terminated with error", errorInOtherThread); - else - return true; - } - }; - - - -} // namespace vault -#endif /*LIB_THREADWRAPPER_H*/ diff --git a/src/vault/threadpool-init.cpp b/src/vault/threadpool-init.cpp deleted file mode 100644 index b50e5d1d3..000000000 --- a/src/vault/threadpool-init.cpp +++ /dev/null @@ -1,54 +0,0 @@ -/* - ThreadpoolInit - pull up the Thread management automagically at application initialisation - - Copyright (C) Lumiera.org - 2010, Hermann Vosseler - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - - -/** @file threadpool-init.cpp - ** Implementation of automatic initialisation of the low-level thread handling framework. - */ - - -#include "vault/threadpool-init.hpp" - - - -namespace lumiera { - - void - initialise_Threadpool () - { - lumiera_threadpool_init(); - -////////////////////////////////////////////////////////////////////////TODO: a better way to detect Alpha/beta builds -#ifdef DEBUG - static uint callCount = 0; - ASSERT ( 0 == callCount++ ); -#endif - } - - - void - shutdown_Threadpool () - { - lumiera_threadpool_destroy(); - } -} - diff --git a/src/vault/threadpool-init.hpp b/src/vault/threadpool-init.hpp deleted file mode 100644 index 809c1ad95..000000000 --- a/src/vault/threadpool-init.hpp +++ /dev/null @@ -1,59 +0,0 @@ -/* - THREADPOOL-INIT.hpp - pull up the Thread management automagically at application initialisation - - - Copyright (C) Lumiera.org - 2010, Hermann Vosseler - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -*/ - -/** @file threadpool-init.hpp - ** Automatically bring up the threading management and threadpool in the vault layer. - ** This works by registering a lifecycle callback, which is activated at the start - ** of main or when running the testsuite. Similarly, a shutdown hook is registered. - ** - ** @todo as of 1/10 it is not clear if this will be the final solution. - ** Alternatively, we may rely on a subsystem "vault" or "threadpool" - ** - ** @see threads.h - ** @see thread-wrapper.hpp - ** - */ - - -#ifndef VAULT_THREADPOOL_INIT_H -#define VAULT_THREADPOOL_INIT_H - -#include "include/lifecycle.h" - -extern "C" { -#include "vault/threads.h" -} - - - -namespace lumiera { - void initialise_Threadpool (); - void shutdown_Threadpool (); - - namespace { - LifecycleHook trigger_1_ (ON_GLOBAL_INIT, &initialise_Threadpool); - LifecycleHook trigger_2_ (ON_GLOBAL_SHUTDOWN, &shutdown_Threadpool); -} } - - -#endif /* VAULT_THREADPOOL_INIT_H */ diff --git a/src/vault/threadpool.c b/src/vault/threadpool.c deleted file mode 100644 index 194ac5dad..000000000 --- a/src/vault/threadpool.c +++ /dev/null @@ -1,189 +0,0 @@ -/* - Threadpool - Manage pools of threads - - Copyright (C) Lumiera.org - 2009, Michael Ploujnikov - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - - -/** @file threadpool.c - ** Implementation of a threadpool. - ** The plan is to manage the massively parallel activities by a scheduler. - ** @todo development in this area is stalled since 2010 - */ - - -#include "include/logging.h" -#include "lib/safeclib.h" -#include "vault/threadpool.h" - -#include - - -LUMIERA_ERROR_DEFINE(THREADPOOL_OFFLINE, "tried to acquire thread while threadpool is not available"); - - - -static lumiera_threadpool threadpool; - - - -void -lumiera_threadpool_init(void) -{ - TRACE (threadpool); - - for (int i = 0; i < LUMIERA_THREADCLASS_COUNT; ++i) - { - llist_init (&threadpool.pool[i].working_list); - llist_init (&threadpool.pool[i].idle_list); - threadpool.pool[i].status = LUMIERA_THREADPOOL_ONLINE; - - //TODO: configure each pools' pthread_attrs appropriately - pthread_attr_init (&threadpool.pool[i].pthread_attrs); - //cancel... - - lumiera_condition_init (&threadpool.pool[i].sync,"pool of threads", &NOBUG_FLAG (threadpool), NOBUG_CONTEXT); - } -} - -void -lumiera_threadpool_destroy(void) -{ - TRACE (threadpool); - - /* set all threadpools offline must be done first, since running threads may attempt to start new ones */ - for (int i = 0; i < LUMIERA_THREADCLASS_COUNT; ++i) - LUMIERA_CONDITION_SECTION (cond_sync, &threadpool.pool[i].sync) - threadpool.pool[i].status = LUMIERA_THREADPOOL_OFFLINE; - - /* wait that all threads have finished */ - for (int i = 0; i < LUMIERA_THREADCLASS_COUNT; ++i) - { - LUMIERA_CONDITION_SECTION (cond_sync, &threadpool.pool[i].sync) - { - //////////////////////////////////////////TICKET #843 check threads deadlines, kill them when they are stalled" - //////////////////////////////////////////TICKET #843 for threads without deadline use a timeout from config system, 500ms or so by default - - LUMIERA_CONDITION_WAIT(llist_is_empty (&threadpool.pool[i].working_list)); - } - } - - /* now we can delete all threads */ - for (int i = 0; i < LUMIERA_THREADCLASS_COUNT; ++i) - { - TRACE (threadpool, "destroying individual pool #%d", i); - - LUMIERA_CONDITION_SECTION (cond_sync, &threadpool.pool[i].sync) - { - ENSURE (llist_is_empty (&threadpool.pool[i].working_list), - "threads are still running"); - - LLIST_WHILE_HEAD (&threadpool.pool[i].idle_list, t) - { - lumiera_thread_delete ((LumieraThread)t); - } - } - lumiera_condition_destroy (&threadpool.pool[i].sync, &NOBUG_FLAG (threadpool), NOBUG_CONTEXT); - pthread_attr_destroy (&threadpool.pool[i].pthread_attrs); - } -} - -/** - * @return thread handle or NULL on error (lumiera error will be set) - */ -LumieraThread -lumiera_threadpool_acquire_thread (enum lumiera_thread_class kind, - const char* purpose, - struct nobug_flag* flag) -{ - TRACE (threadpool); - LumieraThread ret = NULL; - - REQUIRE (kind < LUMIERA_THREADCLASS_COUNT, "unknown pool kind specified: %d", kind); - - LUMIERA_CONDITION_SECTION (cond_sync, &threadpool.pool[kind].sync) - { - if (threadpool.pool[kind].status != LUMIERA_THREADPOOL_ONLINE) - LUMIERA_ERROR_SET_WARNING (threadpool, THREADPOOL_OFFLINE, purpose); - else - { - if (llist_is_empty (&threadpool.pool[kind].idle_list)) - { - ret = lumiera_thread_new (kind, purpose, flag, - &threadpool.pool[kind].pthread_attrs); - TRACE (threadpool, "created thread %p", ret); - - /* - a newly created thread flows somewhere in the air; it is not yet released into the idle list, - nor in the working list, While we are holding this CONDITION_SECION we can safely put it on the working list, - this removes a small race. - */ - llist_insert_head (&threadpool.pool[kind].working_list, &ret->node); - - ENSURE (ret, "did not create a valid thread"); - //////////////////////////////////////////////////////////////////////TICKET #844 no error must be pending here - //////////////////////////////////////////////////////////////////////TICKET #844 let the resourcecollector do it, no need when returning the thread - - LUMIERA_CONDITION_WAIT (!llist_is_empty (&threadpool.pool[kind].idle_list)); - } - // use an existing thread, pick the first one - // remove it from the pool's list - ret = (LumieraThread) (llist_head (&threadpool.pool[kind].idle_list)); - TRACE (threadpool, "got thread %p", ret); - - REQUIRE (ret->state == LUMIERA_THREADSTATE_IDLE, "trying to return a non-idle thread (state=%s)", lumiera_threadstate_names[ret->state]); - - // move thread to the working_list - llist_insert_head (&threadpool.pool[kind].working_list, &ret->node); - } - } - return ret; -} - -void -lumiera_threadpool_release_thread(LumieraThread thread) -{ - TRACE (threadpool); - REQUIRE (thread, "invalid thread given"); - thread->kind = thread->kind&0xff; - REQUIRE (thread->kind < LUMIERA_THREADCLASS_COUNT, "thread belongs to an unknown pool kind: %d", thread->kind); - - REQUIRE (thread->state != LUMIERA_THREADSTATE_IDLE, "trying to park an already idle thread"); - LUMIERA_CONDITION_SECTION (cond_sync, &threadpool.pool[thread->kind].sync) - { - REQUIRE (!llist_is_member (&threadpool.pool[thread->kind].idle_list, &thread->node), "thread is already in the idle list"); - REQUIRE (llist_is_member (&threadpool.pool[thread->kind].working_list, &thread->node) - || thread->state == LUMIERA_THREADSTATE_STARTUP, - "thread is not in the working list (state=%s)", - lumiera_threadstate_names[thread->state]); - thread->state = LUMIERA_THREADSTATE_IDLE; - // move thread to the idle_list - llist_insert_head (&threadpool.pool[thread->kind].idle_list, &thread->node); - - LUMIERA_CONDITION_BROADCAST; - } -} - -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/threadpool.h b/src/vault/threadpool.h deleted file mode 100644 index 670e35e07..000000000 --- a/src/vault/threadpool.h +++ /dev/null @@ -1,95 +0,0 @@ -/* - THREADPOOL.h - Manage pools of threads - - Copyright (C) Lumiera.org - 2009, Michael Ploujnikov - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -*/ - - -/** @file threadpool.h - ** @todo development in this area is stalled since 2010 - */ - - -#ifndef VAULT_THREADPOOL_H -#define VAULT_THREADPOOL_H - -#include "lib/condition.h" -#include "lib/llist.h" -#include "threads.h" - -#include - - - - -/** - * Acquire a thread from a threadpool. - * This may either pick a thread from an appropriate pool or create a new one when the pool is empty. - * This function doesn't need to be accessible outside of the threadpool implementation. - */ -LumieraThread -lumiera_threadpool_acquire_thread(enum lumiera_thread_class kind, - const char* purpose, - struct nobug_flag* flag); - -/** - * Park a thread - * This ends up putting a finished thread back on the list of an appropriate threadpool. - * This function doesn't need to be accessible outside of the threadpool implementation. - */ -void -lumiera_threadpool_release_thread(LumieraThread thread); - -typedef struct lumiera_threadpool_struct lumiera_threadpool; -typedef lumiera_threadpool* LumieraThreadpool; - -enum lumiera_threadpool_state { - LUMIERA_THREADPOOL_OFFLINE, - LUMIERA_THREADPOOL_ONLINE -}; - -struct lumiera_threadpool_struct -{ - struct - { - llist working_list; - llist idle_list; - pthread_attr_t pthread_attrs; - lumiera_condition sync; - enum lumiera_threadpool_state status; - } pool[LUMIERA_THREADCLASS_COUNT]; -}; - - -/** Initialise the thread pool. */ -void -lumiera_threadpool_init(void); - -void -lumiera_threadpool_destroy(void); - - -#endif /*VAULT_THREADPOOL_H*/ -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/threads.c b/src/vault/threads.c deleted file mode 100644 index 0f80c47c5..000000000 --- a/src/vault/threads.c +++ /dev/null @@ -1,354 +0,0 @@ -/* - Threads - Helper for managing threads - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - - -/** @file threads.c - ** Implementation of Lumiera's low-level thread handling framework - ** @todo development in this area is stalled since 2010 - */ - - -#include "include/logging.h" -#include "lib/safeclib.h" -#include "vault/threads.h" - -#include -#include -#include - - -LUMIERA_ERROR_DEFINE(THREAD, "fatal threads initialisation error"); - - - -/** Macro for enum string trick: expands as an array of thread class name strings */ -#define LUMIERA_THREAD_CLASS(name) #name, - - -const char* lumiera_threadclass_names[] = { - LUMIERA_THREAD_CLASSES -}; - -#undef LUMIERA_THREAD_CLASS - -#define LUMIERA_THREAD_STATE(name) #name, -const char* lumiera_threadstate_names[] = { - LUMIERA_THREAD_STATES -}; -#undef LUMIERA_THREAD_STATE - - -/** thread local storage pointing back to the thread structure of each thread */ -static pthread_key_t lumiera_thread_tls; -static pthread_once_t lumiera_thread_initialised = PTHREAD_ONCE_INIT; - - -static void -lumiera_thread_tls_init (void) -{ - if (!!pthread_key_create (&lumiera_thread_tls, NULL)) - LUMIERA_DIE (THREAD); /* should never happen */ -} - - -static void* -thread_loop (void* thread) -{ - TRACE (threads); - NOBUG_THREAD_ID_SET ("worker"); - LumieraThread t = (LumieraThread)thread; - - pthread_setspecific (lumiera_thread_tls, t); - - pthread_setcancelstate (PTHREAD_CANCEL_DISABLE, NULL); - - REQUIRE (t, "thread does not exist"); - - LUMIERA_CONDITION_SECTION (cond_sync, &t->signal) - { - t->rh = &lumiera_lock_section_.rh; - - do { - lumiera_threadpool_release_thread(t); - LUMIERA_CONDITION_WAIT (t->state != LUMIERA_THREADSTATE_IDLE); - TRACE (threads, "Thread awaken with state %s", lumiera_threadstate_names[t->state]); - - // NULL function means: no work to do - TRACE (threads, "function %p", t->function); - if (t->function) - t->function (t->arguments); - TRACE (threads, "function done"); - - if (t->kind & LUMIERA_THREAD_JOINABLE) - { - TRACE (threads, "Thread zombified"); - /* move error state to data the other thread will it pick up from there */ - t->arguments = (void*)lumiera_error (); - t->state = LUMIERA_THREADSTATE_ZOMBIE; - ERROR_IF (t->arguments, threads, "joinable thread ended with error %s", (char*)t->arguments); - - LUMIERA_CONDITION_SIGNAL; - LUMIERA_CONDITION_WAIT (t->state == LUMIERA_THREADSTATE_JOINED); - TRACE (threads, "Thread joined"); - } - - } while (t->state != LUMIERA_THREADSTATE_SHUTDOWN); - // SHUTDOWN state - - - TRACE (threads, "Thread done."); - } - //////////////////////////////////////////////////////////////////////TICKET #844 no error must be pending here, else do app shutdown - return 0; -} - - -/** - * @remarks when this is called it should have already been decided - * that the function shall run in parallel, as a thread. - */ -LumieraThread -lumiera_thread_run (int kind, - void (*function)(void *), - void * arg, - const char* purpose, - struct nobug_flag* flag) -{ - TRACE (threads); - // REQUIRE (function, "invalid function"); - - // ask the threadpool for a thread (it might create a new one) - LumieraThread self = lumiera_threadpool_acquire_thread (kind&0xff, purpose, flag); - - // set the function and data to be run - self->function = function; - self->arguments = arg; - self->kind = kind; - self->deadline.tv_sec = 0; - - // and let it really run (signal the condition var, the thread waits on it) - self->state = LUMIERA_THREADSTATE_WAKEUP; - - LUMIERA_CONDITION_SECTION (cond_sync, &self->signal) - LUMIERA_CONDITION_SIGNAL; - - // NOTE: example only, add solid error handling! - - return self; -} - -/** - * Create a new thread structure with a matching pthread - */ -LumieraThread -lumiera_thread_new (enum lumiera_thread_class kind, - const char* purpose, - struct nobug_flag* flag, - pthread_attr_t* attrs) -{ - pthread_once (&lumiera_thread_initialised, lumiera_thread_tls_init); - - // TODO: do something with this string: - (void) purpose; - REQUIRE (attrs, "invalid pthread attributes structure passed"); - - LumieraThread self = lumiera_malloc (sizeof (*self)); - llist_init (&self->node); - lumiera_condition_init (&self->signal, "thread-control", flag, NOBUG_CONTEXT); - self->kind = kind; - self->state = LUMIERA_THREADSTATE_STARTUP; - self->function = NULL; - self->arguments = NULL; - self->deadline.tv_sec = 0; - self->deadline.tv_nsec = 0; - - int error = pthread_create (&self->id, attrs, &thread_loop, self); - if (error) - { - LUMIERA_DIE (ERRNO); - } - return self; -} - - -LumieraThread -lumiera_thread_destroy (LumieraThread self) -{ - TRACE (threads); - REQUIRE (self, "trying to destroy an invalid thread"); - - llist_unlink (&self->node); - - // get the pthread out of the processing loop - // need to signal to the thread that it should start quitting - // should this be within the section? - LUMIERA_CONDITION_SECTION (cond_sync, &self->signal) - { - REQUIRE (self->state == LUMIERA_THREADSTATE_IDLE, "trying to delete a thread in state other than IDLE (%s)", lumiera_threadstate_names[self->state]); - self->state = LUMIERA_THREADSTATE_SHUTDOWN; - self->function = NULL; - self->arguments = NULL; - LUMIERA_CONDITION_SIGNAL; - } - - int error = pthread_join (self->id, NULL); - ENSURE (0 == error, "pthread_join returned %d:%s", error, strerror (error)); - - // condition has to be destroyed after joining with the thread - lumiera_condition_destroy (&self->signal, &NOBUG_FLAG (threads), NOBUG_CONTEXT); - - return self; -} - - -void -lumiera_thread_delete (LumieraThread self) -{ - TRACE (threads); - lumiera_free (lumiera_thread_destroy (self)); -} - - -LumieraThread -lumiera_thread_self (void) -{ - pthread_once (&lumiera_thread_initialised, lumiera_thread_tls_init); - return pthread_getspecific (lumiera_thread_tls); -} - - - -LumieraThread -lumiera_thread_deadline_set (struct timespec deadline) -{ - TRACE (threads); - LumieraThread self = lumiera_thread_self (); - if (self) - self->deadline = deadline; - return self; -} - - - -LumieraThread -lumiera_thread_deadline_extend (unsigned ms) -{ - TRACE (threads); - LumieraThread self = lumiera_thread_self (); - if (self) - { - struct timespec deadline; - clock_gettime (CLOCK_REALTIME, &deadline); - deadline.tv_sec += ms / 1000; - deadline.tv_nsec += 1000000 * (ms % 1000); - if (deadline.tv_nsec >= 1000000000) - { - deadline.tv_sec += (deadline.tv_nsec / 1000000000); - deadline.tv_nsec %= 1000000000; - } - self->deadline = deadline; - } - - return self; -} - - - -LumieraThread -lumiera_thread_deadline_clear (void) -{ - TRACE (threads); - LumieraThread self = lumiera_thread_self (); - if (self) - { - self->deadline.tv_sec = 0; - self->deadline.tv_nsec = 0; - } - return self; -} - - - -LumieraThread -lumiera_thread_sync_other (LumieraThread other) -{ - TRACE(threads); - - LUMIERA_CONDITION_SECTION (cond_sync, &other->signal) - { - LUMIERA_CONDITION_WAIT (other->state == LUMIERA_THREADSTATE_SYNCING); - other->state = LUMIERA_THREADSTATE_RUNNING; - LUMIERA_CONDITION_SIGNAL; - } - return other; -} - - -LumieraThread -lumiera_thread_sync (void) -{ - TRACE(threads); - - LumieraThread self = lumiera_thread_self (); - REQUIRE(self, "not a lumiera thread"); - - self->state = LUMIERA_THREADSTATE_SYNCING; - lumiera_condition_signal (&self->signal, &NOBUG_FLAG(threads), NOBUG_CONTEXT); - - //////////////////////////////////////////TICKET #843 error handing, maybe timed mutex (using the threads heartbeat timeout, shortly before timeout) - - while (self->state == LUMIERA_THREADSTATE_SYNCING) { - lumiera_condition_wait (&self->signal, &NOBUG_FLAG(threads), self->rh, NOBUG_CONTEXT); - } - - return self; -} - - - -lumiera_err -lumiera_thread_join (LumieraThread thread) -{ - TRACE(threads); - lumiera_err ret = NULL; - - LUMIERA_CONDITION_SECTION (cond_sync, &thread->signal) - { - LUMIERA_CONDITION_WAIT (thread->state == LUMIERA_THREADSTATE_ZOMBIE); - ret = (lumiera_err)thread->arguments; - ERROR_IF (ret, threads, "thread joined with error %s", ret); - - thread->state = LUMIERA_THREADSTATE_JOINED; - LUMIERA_CONDITION_SIGNAL; /* kiss it a last goodbye */ - } - return ret; -} - - - -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/src/vault/threads.h b/src/vault/threads.h deleted file mode 100644 index daf111453..000000000 --- a/src/vault/threads.h +++ /dev/null @@ -1,292 +0,0 @@ -/* - THREADS.h - Helper for managing threads - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -*/ - - -/** @file threads.h - ** Lumiera low-level thread handling framework. - ** Exposed a limited set of operations to deal with parallelism, - ** while threads and thread creation is managed by a threadpool. - ** @todo development in this area is stalled since 2010 - */ - - -#ifndef VAULT_THREADS_H -#define VAULT_THREADS_H - -#include "lib/condition.h" - -#include - - - -typedef struct lumiera_thread_struct lumiera_thread; -typedef lumiera_thread* LumieraThread; - -/** Helper macro used for an enum string trick */ -#define LUMIERA_THREAD_CLASSES \ - /** mostly idle, low latency **/ \ - LUMIERA_THREAD_CLASS(INTERACTIVE) \ - /** busy at average priority **/ \ - LUMIERA_THREAD_CLASS(WORKER) \ - /** busy, soft realtime, high priority **/ \ - LUMIERA_THREAD_CLASS(URGENT) \ - /** high latency, background jobs **/ \ - LUMIERA_THREAD_CLASS(BATCH) \ - /** Something to do when there is really nothing else to do **/ \ - LUMIERA_THREAD_CLASS(IDLE) - -/** Macro for enum string trick: expands as an enum of thread classes */ -#define LUMIERA_THREAD_CLASS(name) LUMIERA_THREADCLASS_##name, - - - -/** - * Thread classes. - * We define some 'classes' of threads for different purposes to abstract - * priorities and other attributes. - */ -enum lumiera_thread_class - { - LUMIERA_THREAD_CLASSES - /** this just denotes the number of classes listed above, - it is used to create arrays **/ - LUMIERA_THREADCLASS_COUNT, /* must be <= 256, thats easy or? */ - - // .. various thread flags follow - /** - * flag to let the decision to run the function in a thread open to the vault. - * depending on load it might decide to run it sequentially. - * This has some constraints: - * The Thread must be very careful with locking, better don't. - * @todo explain synchronisation issues - **/ - LUMIERA_THREAD_OR_NOT = 1<<8, - - /** - * Thread must be joined finally - **/ - LUMIERA_THREAD_JOINABLE = 1<<9 - }; - -#undef LUMIERA_THREAD_CLASS - -// defined in threads.c -extern const char* lumiera_threadclass_names[]; - - -// there is some confusion between the meaning of this -// on one hand it could be used to tell the current state of the thread -// on the other, it is used to tell the thread which state to enter on next iteration -#define LUMIERA_THREAD_STATES \ - LUMIERA_THREAD_STATE(ERROR) \ - LUMIERA_THREAD_STATE(IDLE) \ - LUMIERA_THREAD_STATE(RUNNING) \ - LUMIERA_THREAD_STATE(SYNCING) \ - LUMIERA_THREAD_STATE(WAKEUP) \ - LUMIERA_THREAD_STATE(SHUTDOWN) \ - LUMIERA_THREAD_STATE(ZOMBIE) \ - LUMIERA_THREAD_STATE(JOINED) \ - LUMIERA_THREAD_STATE(STARTUP) - -#define LUMIERA_THREAD_STATE(name) LUMIERA_THREADSTATE_##name, - -/** - * Thread state. - * These are the only states our threads can be in. - */ -typedef enum - { - LUMIERA_THREAD_STATES - } - lumiera_thread_state; - -#undef LUMIERA_THREAD_STATE - -// defined in threads.c -extern const char* lumiera_threadstate_names[]; - -#include "threadpool.h" - - - -/** - * The actual thread data - */ -struct lumiera_thread_struct -{ - llist node; // this should be first for easy casting - - pthread_t id; - - // TODO: maybe this condition variable should be renamed when we have a better understanding of how it will be used - lumiera_condition signal; // control signal, state change signal - - struct timespec deadline; - - struct nobug_resource_user** rh; - - // the following member could have been called "class" except that it would conflict with C++ keyword - // as consequence, it's been decided to leave the type name containing the word "class", - // while all members/variables called "kind" - int kind; - - // this is used both as a command and as a state tracker - lumiera_thread_state state; - void (*function)(void *); - void * arguments; -}; - - -/** - * Create a thread structure. - */ -LumieraThread -lumiera_thread_new (enum lumiera_thread_class kind, - const char* purpose, - struct nobug_flag* flag, - pthread_attr_t* attrs); - -/** - * Destroy and de-initialise a thread structure. - * Memory is not freed by this function. - */ -LumieraThread -lumiera_thread_destroy (LumieraThread self); - -/** - * Actually free the memory used by the thread structure. - * Make sure to destroy the structure first. - */ -void -lumiera_thread_delete (LumieraThread self); - -/** - * Start a thread. - * Threads are implemented as procedures which take a void* and don't return anything. - * When a thread wants to pass something back to the application it should use the void* it got for - * constructing the return. - * * Threads must complete (return from their thread function) - * * They must not call any exit() function. - * * Threads can not be cancelled - * * Threads shall not handle signals (all signals will be disabled for them) unless explicitly acknowledged - * - * @param kind class of the thread to start - * @param function pointer to a function to execute in a thread (returning void, not void* as in pthreads) - * @param arg generic pointer passed to the thread - * @param purpose descriptive name of this thread, used by NoBug - * @param flag NoBug flag used for logging the thread startup and return - */ -LumieraThread -lumiera_thread_run (int kind, - void (*function)(void *), - void * arg, - const char* purpose, - struct nobug_flag* flag); - -/** - * Query the LumieraThread handle of the current thread - * - * - * @return pointer to the (opaque) handle of the current lumiera thread or NULL when this is not a lumiera thread - */ -LumieraThread -lumiera_thread_self (void); - - - -/** - * Set a thread deadline. - * A thread must finish before its deadline is hit. Otherwise it counts as stalled - * which is a fatal error which might pull the application down. - * - * \par Heartbeat and Deadlines. - * - * Any thread can have an optional 'deadline' which must never be hit. - * This deadlines are lazily checked and if hit this is a fatal error which triggers - * an emergency shutdown. Thus threads are obliged to set and extend their deadlines - * accordingly. - */ -LumieraThread -lumiera_thread_deadline_set (struct timespec deadline); - - -/** - * Extend the deadline of a thread - * sets the deadline to \c NOW+ms in future. This can be used to implement a heartbeat. - */ -LumieraThread -lumiera_thread_deadline_extend (unsigned ms); - - -/** - * Clear a thread's deadline - * Threads without deadline will not be checked against deadlocks (this is the default) - */ -LumieraThread -lumiera_thread_deadline_clear (void); - - - -/** - * Synchronise with another threads state. - * This blocks until/unless the other thread reaches a synchronisation point. - * - * \par Thread synchronisation - * The synchronisation primitives act as barrier over 2 threads, any thread reaching - * a synchronisation point first is blocked until the other one reaches it too. - */ -LumieraThread -lumiera_thread_sync_other (LumieraThread other); - -/** - * Synchronise current thread - * - * this blocks until/unless the other thread reaches a synchronisation point - * @return on success pointer to self (opaque), or NULL on error - */ -LumieraThread -lumiera_thread_sync (void); - -// TODO implement timedsync, this is bit tricky because after a timeout, synchronisation points are desynched -// we possibly need some way to reset/resync this -//LumieraThread -//lumiera_thread_timedsync (struct timespec timeout); - - -/** - * Joining threads - * a thread can be set up with the LUMEIRA_THREAD_JOINABLE flag, if so - * then it must be joined finally. Joining clears the error state of the joined thread - * and returns it to the joiner. - * - */ -lumiera_err -lumiera_thread_join (LumieraThread thread); - -#endif /*VAULT_THREADS_H*/ -/* -// Local Variables: -// mode: C -// c-file-style: "gnu" -// indent-tabs-mode: nil -// End: -*/ diff --git a/tests/11locking.tests b/tests/11locking.tests index 7401d4b58..b023c4241 100644 --- a/tests/11locking.tests +++ b/tests/11locking.tests @@ -32,66 +32,3 @@ TEST "recursive mutex section" recursivemutexsection < ,tmp_testfile - -TEST "acquire existing file" acquire_existing < ,tmp_testfile1 -echo testdata > ,tmp_testfile2 -echo testdata > ,tmp_testfile3 - -TEST "acquire 3 files" acquire_existing_3files </dev/null - -TEST "basic mmap" mmap_simple </dev/null - -TEST "use mmap twice" mmap_checkout_twice </dev/null - -TEST "reuse mmap" mmap_checkout_again </dev/null diff --git a/tests/30resourcecollector.tests b/tests/30resourcecollector.tests deleted file mode 100644 index ea916dc58..000000000 --- a/tests/30resourcecollector.tests +++ /dev/null @@ -1,22 +0,0 @@ -TESTING "Resourcecollector" ./test-resourcecollector - - -PLANNED "basic register, destroy" basic </dev/null diff --git a/tests/library/c-lib/test-locking.c b/tests/library/c-lib/test-locking.c index 7703a6a8f..f3d320c07 100644 --- a/tests/library/c-lib/test-locking.c +++ b/tests/library/c-lib/test-locking.c @@ -32,9 +32,6 @@ #include "lib/test/test.h" #include "lib/mutex.h" #include "lib/recmutex.h" -#include "lib/condition.h" -#include "lib/reccondition.h" -#include "lib/rwlock.h" #include #include @@ -157,212 +154,29 @@ TEST (recursivemutexsection) } - -TEST (rwlocksection) -{ - lumiera_rwlock rwlock; - lumiera_rwlock_init (&rwlock, "rwsection", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - - LUMIERA_WRLOCK_SECTION (NOBUG_ON, &rwlock) - { - printf ("write locked section 1\n"); - } - - LUMIERA_RDLOCK_SECTION (NOBUG_ON, &rwlock) - { - printf ("read locked section 2\n"); - } - - lumiera_rwlock_destroy (&rwlock, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); -} - - -TEST (rwlockforgotunlock) -{ - lumiera_rwlock rwlock; - lumiera_rwlock_init (&rwlock, "rwlockforgotunlock", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - - LUMIERA_RDLOCK_SECTION (NOBUG_ON, &rwlock) - { - break; // LOCK_SECTIONS must not be left by a jump - } - - lumiera_rwlock_destroy (&rwlock, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); -} - - - -TEST (rwdeadlockwr) -{ - lumiera_rwlock rwlock; - lumiera_rwlock_init (&rwlock, "rwsection", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - - LUMIERA_WRLOCK_SECTION (NOBUG_ON, &rwlock) - { - printf ("write locked section 1\n"); - LUMIERA_RDLOCK_SECTION (NOBUG_ON, &rwlock) - { - printf ("read locked section 2\n"); - } - } - - lumiera_rwlock_destroy (&rwlock, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); -} - - - -TEST (rwdeadlockrw) -{ - lumiera_rwlock rwlock; - lumiera_rwlock_init (&rwlock, "rwsection", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - - LUMIERA_RDLOCK_SECTION (NOBUG_ON, &rwlock) - { - printf ("read locked section 1\n"); - LUMIERA_WRLOCK_SECTION (NOBUG_ON, &rwlock) - { - printf ("write locked section 2\n"); - } - } - - lumiera_rwlock_destroy (&rwlock, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); -} - - -TEST (conditionops (compiletest only)) -{ - lumiera_condition cond; - lumiera_condition_init (&cond, "conditionsection", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - - LUMIERA_CONDITION_SECTION (NOBUG_ON, &cond) - { - LUMIERA_CONDITION_WAIT(1); - LUMIERA_CONDITION_SIGNAL; - LUMIERA_CONDITION_BROADCAST; - } - - lumiera_condition_destroy (&cond, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); -} - - -TEST (conditionsection) -{ - lumiera_condition cond; - lumiera_condition_init (&cond, "conditionsection", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - - LUMIERA_CONDITION_SECTION (NOBUG_ON, &cond) - { - printf ("condition locked section 1\n"); - } - - LUMIERA_CONDITION_SECTION (NOBUG_ON, &cond) - { - printf ("condition locked section 2\n"); - } - - lumiera_condition_destroy (&cond, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); -} - - - -TEST (conditionforgotunlock) -{ - lumiera_condition cond; - lumiera_condition_init (&cond, "conditionforgotunlock", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - - LUMIERA_CONDITION_SECTION (NOBUG_ON, &cond) - { - break; // CONDITION_SECTIONS must not be left by a jump - } - - lumiera_condition_destroy (&cond, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); -} - - - -TEST (recconditionops (compiletest only)) -{ - lumiera_reccondition reccond; - lumiera_reccondition_init (&reccond, "recconditionsection", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - - LUMIERA_RECCONDITION_SECTION (NOBUG_ON, &reccond) - { - LUMIERA_RECCONDITION_WAIT(1); - LUMIERA_RECCONDITION_SIGNAL; - LUMIERA_RECCONDITION_BROADCAST; - } - - lumiera_reccondition_destroy (&reccond, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); -} - - -TEST (recconditionsection) -{ - lumiera_reccondition reccond; - lumiera_reccondition_init (&reccond, "recconditionsection", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - - LUMIERA_RECCONDITION_SECTION (NOBUG_ON, &reccond) - { - printf ("reccondition locked section 1\n"); - } - - LUMIERA_RECCONDITION_SECTION (NOBUG_ON, &reccond) - { - printf ("reccondition locked section 2\n"); - } - - lumiera_reccondition_destroy (&reccond, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); -} - - -TEST (recconditionforgotunlock) -{ - lumiera_reccondition reccond; - lumiera_reccondition_init (&reccond, "recconditionforgotunlock", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - - LUMIERA_RECCONDITION_SECTION (NOBUG_ON, &reccond) - { - break; // RECCONDITION_SECTIONS must not be left by a jump - } - - lumiera_reccondition_destroy (&reccond, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); -} - -TEST (chainedrecconditionsection) -{ - lumiera_reccondition outer, inner; - lumiera_reccondition_init (&outer, "outer_recconditionsection", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - lumiera_reccondition_init (&inner, "inner_recconditionsection", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - - LUMIERA_RECCONDITION_SECTION (NOBUG_ON, &outer) - { - printf ("outer reccondition locked section\n"); - LUMIERA_RECCONDITION_SECTION_CHAIN (NOBUG_ON, &inner) - { - printf ("inner reccondition locked section\n"); - } - } - lumiera_reccondition_destroy (&outer, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - lumiera_reccondition_destroy (&inner, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); -} - -TEST (nestedrecconditionsection) -{ - lumiera_reccondition outer, inner; - lumiera_reccondition_init (&outer, "outer_recconditionsection", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - lumiera_reccondition_init (&inner, "inner_recconditionsection", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - - LUMIERA_RECCONDITION_SECTION (NOBUG_ON, &outer) - { - printf ("outer reccondition locked section\n"); - LUMIERA_RECCONDITION_SECTION (NOBUG_ON, &inner) - { - printf ("inner reccondition locked section\n"); - } - } - lumiera_reccondition_destroy (&outer, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - lumiera_reccondition_destroy (&inner, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); -} - +/* ====== 10/2023 : partially dismantled + * + * After switching to C++14 Threads and Locking (#1279), + * some backend-services are no longer used... + * - rwlocksection + * - rwlockforgotunlock + * - rwdeadlockwr + * - rwdeadlockrw + * - rwlockdeadlockwr + * - rwlockdeadlockrw + * - conditionops + * - conditionsection + * - conditionforgotunlock + * - condition signaling (planned) + * - condition broadcasting (planned) + * - recconditionops + * - recconditionsection + * - recconditionforgotunlock + * - chainedrecconditionsection + * - nestedrecconditionsection + * - reccondition signaling (planned) + * - reccondition broadcasting (planned) + * + */ TESTS_END diff --git a/tests/library/c-lib/test-mpool.c b/tests/library/c-lib/test-mpool.c deleted file mode 100644 index 231c16955..000000000 --- a/tests/library/c-lib/test-mpool.c +++ /dev/null @@ -1,352 +0,0 @@ -/* - TEST-MPOOL - memory pool for constant sized objects - - Copyright (C) Lumiera.org - 2009, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - -/** @file test-mpool.c - ** C unit test to cover a memory pool custom allocator - ** - ** @warning unfinished implementation as of 2016 - ** @see mpool.h - */ - - -#include "lib/test/test.h" -#include "lib/mpool.h" - -struct teststruct -{ - llist node; - void* ptr[2]; -}; - - -static inline uint32_t mpool_fast_prng () -{ - static uint32_t rnd=0xbabeface; - return rnd = rnd<<1 ^ ((rnd >> 30) & 1) ^ ((rnd>>2) & 1); -} - -static void -dtor (void* o) -{ - ECHO("%d @%p", *(int*)o, o); -} - - - -TESTS_BEGIN - -TEST (basic) -{ - mpool mypool; - mpool_init (&mypool, sizeof(void*), 10, dtor); - ECHO ("initialised"); - - void* element; - element = mpool_alloc (&mypool); - ECHO ("allocated %p", element); - *(int*)element = 0xdeadbabe; - - DUMP(NOBUG_ON, mpool, &mypool, 4, NULL); - - mpool_free (&mypool, element); - ECHO ("freed"); - - DUMP(NOBUG_ON, mpool, &mypool, 4, NULL); - - mpool_destroy (&mypool); - ECHO ("destroyed"); -} - - -TEST (destroy) -{ - mpool mypool; - mpool_init (&mypool, sizeof(void*), 10, dtor); - ECHO ("initialised"); - - void* element; - element = mpool_alloc (&mypool); - ECHO ("allocated %p", element); - *(int*)element = 0xbabeface; - - DUMP(NOBUG_ON, mpool, &mypool, 4, NULL); - - mpool_destroy (&mypool); - ECHO ("destroyed"); -} - - -TEST (clusters) -{ - mpool mypool; - mpool_init (&mypool, sizeof(void*), 2, dtor); - ECHO ("initialised"); - - for (int i = 1; i <= 5; ++i) - { - void* element; - element = mpool_alloc (&mypool); - ECHO ("allocated %p", element); - *(int*)element = i; - } - - DUMP(NOBUG_ON, mpool, &mypool, 4, NULL); - - mpool_destroy (&mypool); - ECHO ("destroyed"); -} - - -TEST (clusters_big) -{ - mpool mypool; - mpool_init (&mypool, sizeof(void*), 200, dtor); - ECHO ("initialised"); - - for (int i = 1; i <= 700; ++i) - { - void* element; - element = mpool_alloc (&mypool); - ECHO ("allocated %p", element); - *(int*)element = i; - } - - DUMP(NOBUG_ON, mpool, &mypool, 4, NULL); - - mpool_destroy (&mypool); - ECHO ("destroyed"); -} - - -TEST (alloc_free) -{ - mpool mypool; - mpool_init (&mypool, 24, 4, dtor); - ECHO ("initialised"); - - void* elem[32]; - - for (int i = 1; i <= 15; ++i) - { - elem[i] = mpool_alloc (&mypool); - *(int*)(elem[i]) = i; - } - ECHO ("allocated"); - - for (int i = 1; i <= 15; i+=3) - { - mpool_free (&mypool, elem[i]); - } - ECHO ("freed some"); - DUMP(NOBUG_ON, mpool, &mypool, 4, NULL); - - mpool_destroy (&mypool); - ECHO ("destroyed"); -} - - -TEST (alloc_free_big) -{ - mpool mypool; - mpool_init (&mypool, 24, 4, dtor); - ECHO ("initialised"); - - void* elem[2000]; - - for (int i = 1; i <= 2000; ++i) - { - elem[i] = mpool_alloc (&mypool); - *(int*)(elem[i]) = i; - } - ECHO ("allocated"); - - for (int i = 1; i <= 2000; i+=3) - { - mpool_free (&mypool, elem[i]); - } - ECHO ("freed some"); - DUMP(NOBUG_ON, mpool, &mypool, 4, NULL); - - DUMP(NOBUG_ON, mpool, &mypool, 4, NULL); - - mpool_destroy (&mypool); - ECHO ("destroyed"); -} - - - -TEST (reserve) -{ -} - - -/* - benchmark mpool vs malloc, first only the allocation/free itself with some necessary llist ops -*/ -TEST (bench_mpool) -{ - mpool mypool; - mpool_init (&mypool, sizeof(struct teststruct), 2000, NULL); - ECHO ("initialised"); - - llist list; - llist_init (&list); - - for (int j = 1; j<=100; ++j) - { - for (int i = 1; i <= 50000; ++i) - { - struct teststruct* element = mpool_alloc (&mypool); - llist_insert_tail (&list, llist_init (&element->node)); - } - - LLIST_WHILE_HEAD (&list, element) - { - llist_unlink_fast_ (element); - mpool_free (&mypool, element); - } - } - - mpool_destroy (&mypool); - ECHO ("destroyed"); -} - - -TEST (bench_malloc) -{ - mpool mypool; - mpool_init (&mypool, sizeof(llist), 2000, NULL); - ECHO ("initialised"); - - llist list; - llist_init (&list); - - for (int j = 100; j; --j) - { - for (int i = 1; i <= 50000; ++i) - { - struct teststruct* element = malloc (sizeof(*element)); - llist_insert_tail (&list, llist_init (&element->node)); - } - - LLIST_WHILE_HEAD (&list, element) - { - llist_unlink_fast_ (element); - free (element); - } - } - - mpool_destroy (&mypool); - ECHO ("destroyed"); -} - - -/* - benchmark mpool vs malloc, try to simulate some slightly more realistic application usage - - allocate list nodes which have 2 data members as payload - - there is a 25% chance at each alloc that the head of the list gets deleted -*/ -TEST (bench_mpool_sim) -{ - mpool mypool; - mpool_init (&mypool, sizeof(struct teststruct), 2000, NULL); - ECHO ("initialised"); - - llist list; - llist_init (&list); - - for (int j = 1; j<=100; ++j) - { - for (int i = 1; i <= 50000; ++i) - { - struct teststruct* element = mpool_alloc (&mypool); - llist_insert_tail (&list, llist_init (&element->node)); - element->ptr[0] = malloc(100+(mpool_fast_prng()%500)); - element->ptr[1] = malloc(100+(mpool_fast_prng()%500)); - - if (!(mpool_fast_prng()%4)) - { - struct teststruct* element = (struct teststruct*)llist_head (&list); - llist_unlink_fast_ (&element->node); - free(element->ptr[0]); - free(element->ptr[1]); - mpool_free (&mypool, element); - } - } - - LLIST_WHILE_HEAD (&list, element) - { - llist_unlink_fast_ (element); - free(((struct teststruct*)element)->ptr[0]); - free(((struct teststruct*)element)->ptr[1]); - mpool_free (&mypool, element); - } - } - - mpool_destroy (&mypool); - ECHO ("destroyed"); -} - - -TEST (bench_malloc_sim) -{ - mpool mypool; - mpool_init (&mypool, sizeof(llist), 2000, NULL); - ECHO ("initialised"); - - llist list; - llist_init (&list); - - for (int j = 100; j; --j) - { - for (int i = 1; i <= 50000; ++i) - { - struct teststruct* element = malloc (sizeof(*element)); - llist_insert_tail (&list, llist_init (&element->node)); - element->ptr[0] = malloc(100+(mpool_fast_prng()%500)); - element->ptr[1] = malloc(100+(mpool_fast_prng()%500)); - - if (!(mpool_fast_prng()%4)) - { - struct teststruct* element = (struct teststruct*)llist_head (&list); - llist_unlink_fast_ (&element->node); - free(element->ptr[0]); - free(element->ptr[1]); - free (element); - } - } - - LLIST_WHILE_HEAD (&list, element) - { - llist_unlink_fast_ (element); - free(((struct teststruct*)element)->ptr[0]); - free(((struct teststruct*)element)->ptr[1]); - free (element); - } - } - - mpool_destroy (&mypool); - ECHO ("destroyed"); -} - - -TESTS_END diff --git a/tests/library/c-lib/test-priqueue.c b/tests/library/c-lib/test-priqueue.c deleted file mode 100644 index ff3c86510..000000000 --- a/tests/library/c-lib/test-priqueue.c +++ /dev/null @@ -1,157 +0,0 @@ -/* - TEST_PRIQUEUE - test the heap based priority queue implementation - - Copyright (C) Lumiera.org - 2011, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - -/** @file test-priqueue.c - ** C unit test to cover a priority queue library implementation - ** @see priqueue.h - */ - - -#include "lib/test/test.h" -#include "lib/priqueue.h" -#include "include/logging.h" - - -/* @note internal helper copied from priqueue.c */ -static inline void* -pq_index (LumieraPriQueue self, unsigned nth) -{ - return (char*)self->queue+self->element_size*nth; -} - - - -void -nobug_priqueue_invariant (LumieraPriQueue self, int depth, const struct nobug_context invariant_context, void* extra) -{ - intptr_t n = 1+(intptr_t)extra; - - intptr_t m=n+n; - - if (self && depth && m <= self->used) - { - INVARIANT_ASSERT (self->cmpfn (pq_index(self, n-1), pq_index(self, m-1)) <= 0, "%d %d", (int)n-1, (int)m-2); - nobug_priqueue_invariant (self, depth-1, invariant_context, (void*)m-1); - - if (mused) - { - INVARIANT_ASSERT (self->cmpfn (pq_index(self, n-1), pq_index(self, m)) <= 0, "%d %d", (int)n-1, (int)m-1); - nobug_priqueue_invariant (self, depth-1, invariant_context, (void*)m); - } - } -} - - -static int -cmpintptr (void* a, void* b) -{ - return *(int*)a - *(int*)b; -} - - - - - - - - - -TESTS_BEGIN - - lumiera_priqueue pq; - - LumieraPriQueue r; - - int data, prev, curr; - - r = lumiera_priqueue_init (&pq, - sizeof (int), - cmpintptr, - NULL, - NULL); - ENSURE (r==&pq); - - - data = 10; - r = lumiera_priqueue_insert (&pq, &data); - ENSURE (r==&pq); - TRACE (test, "inserted %d", data); - - - data = 5; - r = lumiera_priqueue_insert (&pq, &data); - ENSURE (r==&pq); - TRACE (test, "inserted %d", data); - - - data = 15; - r = lumiera_priqueue_insert (&pq, &data); - ENSURE (r==&pq); - TRACE (test, "inserted %d", data); - - data = 20; - r = lumiera_priqueue_insert (&pq, &data); - ENSURE (r==&pq); - TRACE (test, "inserted %d", data); - - - - - for (int i = 0; i < 100000; ++i) - { - data = i; - r = lumiera_priqueue_insert (&pq, &data); - ENSURE (r==&pq); - TRACE (test, "inserted %d", data); - } - - - for (int i = 0; i < 100000; ++i) - { - data = rand()%1000000; - r = lumiera_priqueue_insert (&pq, &data); - ENSURE (r==&pq); - TRACE (test, "inserted %d", data); - } - - NOBUG_INVARIANT(priqueue, &pq, 100, NULL); - - - prev = 0; - for (int i = 0; pq.used; ++i) - { - curr = *(int*)lumiera_priqueue_peek (&pq); - TRACE (test, "TOP: %d", curr); - CHECK (prev <= curr, "priority ordering broken"); - prev = curr; - - r = lumiera_priqueue_remove (&pq); - ENSURE (r==&pq); - } - - - r = lumiera_priqueue_destroy (&pq); - ENSURE (r==&pq); - - - -TESTS_END diff --git a/tests/library/c-lib/test-slist.c b/tests/library/c-lib/test-slist.c deleted file mode 100644 index 4d34f9fd4..000000000 --- a/tests/library/c-lib/test-slist.c +++ /dev/null @@ -1,356 +0,0 @@ -/* - TEST-SLIST - test the linked list lib - - Copyright (C) Lumiera.org - 2009, Anton Yakovlev - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - -/** @file test-slist.c - ** C unit test to cover a single linked list library implementation - */ - - -#include "lib/slist.h" -#include "lib/test/test.h" - -#include -#include - -typedef struct item { - int key; - slist list; -} item_t; - -int cmp( const_SList a, const_SList b ) { - - item_t* x = SLIST_TO_STRUCTP( a, item_t, list ); - item_t* y = SLIST_TO_STRUCTP( b, item_t, list ); - - if ( x -> key < y -> key ) { - return -1; - } - - if ( x -> key > y -> key ) { - return +1; - } - - return 0; - -} - -TESTS_BEGIN - -/* - * 1. Basic: - * SLIST_AUTO( name ) - * void slist_init( SList list ) - * int slist_is_empty( const_SList list ) - * int slist_is_single( const_SList list ) - * int slist_is_head( const_SList list, const_SList head ) - * int slist_is_end( const_SList list, const_SList end ) - * int slist_is_member( const_SList list, const_SList member ) - * int slist_is_before_after( const_SList list, const_SList before, const_SList after ) - */ - -TEST (basic) { - - SLIST_AUTO( listX ); - slist listY; - SLIST_AUTO( nodeA ); - SLIST_AUTO( nodeB ); - - ECHO ("%d", slist_is_end( &listX, &listX ) ); - - slist_init( &listY ); - - ECHO ("%d", slist_is_empty( &listY ) ); - - slist_insert( &listX, &nodeA ); - ECHO ("%d", slist_is_empty( &listX ) ); - ECHO ("%d", slist_is_single( &listX ) ); - ECHO ("%d", slist_is_head( &listX, &nodeA ) ); - ECHO ("%d", slist_is_end( &listX, &nodeA ) ); - ECHO ("%d", slist_is_member( &listX, &nodeA ) ); - ECHO ("%d", slist_is_member( &listX, &nodeB ) ); - - slist_insert( &nodeA, &nodeB ); - ECHO ("%d", slist_is_empty( &listX ) ); - ECHO ("%d", slist_is_single( &listX ) ); - ECHO ("%d", slist_is_head( &listX, &nodeB ) ); - ECHO ("%d", slist_is_end( &listX, &nodeB ) ); - ECHO ("%d", slist_is_member( &listX, &nodeB ) ); - - ECHO ("%d", slist_is_before_after( &listX, &nodeA, &nodeB ) ); - ECHO ("%d", slist_is_before_after( &listX, &nodeB, &nodeA ) ); - -} - -/* - * 2. Insert/delete: - * slist_insert_head( list, element ) - * SList slist_insert( SList head, SList node ) - * SList slist_insert_list( SList xnode, SList ylist ) - * SList slist_insert_range( SList node, SList start, SList end ) - * SList slist_unlink( SList list, SList node ) - */ - -TEST (insert_delete) { - - SLIST_AUTO( listX ); - SLIST_AUTO( nodeA ); - SLIST_AUTO( nodeB ); - SLIST_AUTO( nodeC ); - - slist_insert_head( &listX, &nodeA ); - slist_insert( &nodeA, &nodeB ); - slist_insert( &nodeB, &nodeC ); - ECHO ("%d", slist_next( &listX ) == &nodeA ); - ECHO ("%d", slist_next( &nodeA ) == &nodeB ); - ECHO ("%d", slist_next( &nodeB ) == &nodeC ); - ECHO ("%d", slist_next( &nodeC ) == &listX ); - - slist_unlink( &listX, &nodeA ); - ECHO ("%d", slist_next( &listX ) == &nodeB ); - - slist_insert( &listX, &nodeA ); - ECHO ("%d", slist_next( &listX ) == &nodeA ); - - SLIST_AUTO( listY ); - - slist_insert_list( &listY, &listX ); - ECHO ("%d", slist_is_empty( &listX ) ); - ECHO ("%d", slist_next( &listY ) == &nodeA ); - ECHO ("%d", slist_next( &nodeA ) == &nodeB ); - ECHO ("%d", slist_next( &nodeB ) == &nodeC ); - ECHO ("%d", slist_next( &nodeC ) == &listY ); - - slist_insert_range( &listX, &nodeA, &nodeB ); - ECHO ("%d", slist_next( &listX ) == &nodeA ); - ECHO ("%d", slist_next( &nodeA ) == &nodeB ); - ECHO ("%d", slist_next( &nodeB ) == &listX ); - - ECHO ("%d", slist_is_single( &listY ) ); - ECHO ("%d", slist_next( &listY ) == &nodeC ); - ECHO ("%d", slist_next( &nodeC ) == &listY ); - -} - -/* - * 3. Movements: - * slist_head() - * SList slist_next( const_SList node ) - * SList slist_prev( SList list, SList node ) - * SList slist_advance( SList list, SList node ) - * void slist_forward( SList_ref node ) - */ - -TEST (movement) { - - SLIST_AUTO( listX ); - SLIST_AUTO( nodeA ); - SLIST_AUTO( nodeB ); - SLIST_AUTO( nodeC ); - - slist_insert_head( &listX, &nodeA ); - slist_insert( &nodeA, &nodeB ); - slist_insert( &nodeB, &nodeC ); - - ECHO ("%d", slist_next( &listX ) == &nodeA ); - ECHO ("%d", slist_next( &nodeA ) == &nodeB ); - ECHO ("%d", slist_next( &nodeB ) == &nodeC ); - ECHO ("%d", slist_next( &nodeC ) == &listX ); - - ECHO ("%d", slist_prev( &listX, &listX ) == &nodeC ); - ECHO ("%d", slist_prev( &listX, &nodeC ) == &nodeB ); - ECHO ("%d", slist_prev( &listX, &nodeB ) == &nodeA ); - ECHO ("%d", slist_prev( &listX, &nodeA ) == &listX ); - - slist_advance( &listX, &nodeA ); - ECHO ("%d", slist_next( &listX ) == &nodeB ); - ECHO ("%d", slist_next( &nodeB ) == &nodeA ); - ECHO ("%d", slist_next( &nodeA ) == &nodeC ); - ECHO ("%d", slist_next( &nodeC ) == &listX ); - - SList node = &listX; - slist_forward( &node ); - ECHO ("%d", node == &nodeB ); - -} - -/* - * 4. Enumerations: - * SLIST_TO_STRUCTP( list, type, member ) - * SLIST_FOREACH( list, node ) - * SLIST_FORRANGE( start, end, node ) - * SLIST_WHILE_HEAD( list, head ) - */ - -TEST (enumerations) { - - SLIST_AUTO( list ); - - item_t nodeA = { 'A', { NULL } }; - item_t nodeB = { 'B', { NULL } }; - item_t nodeC = { 'C', { NULL } }; - item_t nodeD = { 'D', { NULL } }; - - slist_init( &nodeA.list ); - slist_init( &nodeB.list ); - slist_init( &nodeC.list ); - slist_init( &nodeD.list ); - - slist_insert( &list, &nodeA.list ); - slist_insert( &nodeA.list, &nodeB.list ); - slist_insert( &nodeB.list, &nodeC.list ); - slist_insert( &nodeC.list, &nodeD.list ); - - SLIST_FOREACH ( &list, node ) { - item_t* item = ( item_t* ) SLIST_TO_STRUCTP( node, item_t, list ); - ECHO ("%c", item -> key ); - } - ECHO ("," ); - - ECHO ("---" ); - - SLIST_FORRANGE ( &nodeB.list, &nodeD.list, node ) { - item_t* item = ( item_t* ) SLIST_TO_STRUCTP( node, item_t, list ); - ECHO ("%c", item -> key ); - } - ECHO ("," ); - - ECHO ("---" ); - - SLIST_WHILE_HEAD ( &list, head ) { - item_t* item = ( item_t* ) SLIST_TO_STRUCTP( head, item_t, list ); - ECHO ("%c ", item -> key ); - slist_unlink( &list, head ); - } - ECHO ("," ); - - ECHO ("%d", slist_is_empty( &list ) ); - -} - -/* - * 5. Counting: - * unsigned slist_count( const_SList list ) - * SList slist_get_nth( SList list, int n ) - * SList slist_get_nth_stop( SList list, int n, const_SList stop ) - */ - -TEST (count) { - - SLIST_AUTO( list ); - SLIST_AUTO( nodeA ); - SLIST_AUTO( nodeB ); - SLIST_AUTO( nodeC ); - - slist_insert( &list, &nodeA ); - slist_insert( &nodeA, &nodeB ); - slist_insert( &nodeB, &nodeC ); - - ECHO ("%u", slist_count( &list ) ); - ECHO ("%d", slist_get_nth( &list, 3 ) == &nodeC ); - ECHO ("%d", slist_get_nth_stop( &list, 3, &nodeC ) == NULL ); - -} - -/* - * 6. Sort: - * SList slist_sort( SList list, slist_cmpfn cmp ) - */ - -TEST (sort) { - - srand( time( NULL ) ); - - SLIST_AUTO( list ); - - unsigned int n = 1000000; - - item_t* items; - if ( ( items = ( item_t* ) malloc( sizeof( item_t ) * n ) ) == NULL ) { - return 1; // ERROR: not enough memory - } - - for ( unsigned int i = 0; i < n; i++ ) { - items[ i ].key = rand(); - slist_init( &items[ i ].list ); - slist_insert( &list, &items[ i ].list ); - } - - slist_sort( &list, cmp ); - - int is_first_cmp = 1; - int prev_key = 0; - - SLIST_FOREACH ( &list, x ) { - item_t* item = SLIST_TO_STRUCTP( x, item_t, list ); - if ( is_first_cmp ) { - is_first_cmp = 0; - } else if ( prev_key > item -> key ) { - return 2; // ERROR: wrong order of elements - } - prev_key = item -> key; - } - - free( items ); - - return 0; - -} - -/* - * 7. Search: - * SList slist_find( const_SList list, const_SList pattern, slist_cmpfn cmp ) - * SList slist_ufind( SList list, const_SList pattern, slist_cmpfn cmp ) - * SList slist_sfind( const_SList list, const_SList pattern, slist_cmpfn cmp ) - */ - -TEST (search) { - - SLIST_AUTO( list ); - - item_t nodeA = { 'A', { NULL } }; - item_t nodeB = { 'B', { NULL } }; - item_t nodeC = { 'C', { NULL } }; - item_t nodeD = { 'D', { NULL } }; - item_t nodeX = { '?', { NULL } }; - - slist_init( &nodeA.list ); - slist_init( &nodeB.list ); - slist_init( &nodeC.list ); - slist_init( &nodeD.list ); - - slist_insert( &list, &nodeA.list ); - slist_insert( &nodeA.list, &nodeB.list ); - slist_insert( &nodeB.list, &nodeC.list ); - slist_insert( &nodeC.list, &nodeD.list ); - - nodeX.key = 'C'; - - ECHO ("%d", slist_find( &list, &nodeX.list, cmp ) == &nodeC.list ); - ECHO ("%d", slist_ufind( &list, &nodeX.list, cmp ) == &nodeC.list ); - ECHO ("%d", slist_next( &nodeC.list ) == &nodeA.list ); - - nodeX.key = 'A'; - ECHO ("%d", slist_sfind( &list, &nodeX.list, cmp ) == NULL ); - -} - -TESTS_END diff --git a/tests/library/sync-locking-test.cpp b/tests/library/sync-locking-test.cpp index 65735e8e1..302325bf7 100644 --- a/tests/library/sync-locking-test.cpp +++ b/tests/library/sync-locking-test.cpp @@ -26,181 +26,124 @@ #include "lib/test/run.hpp" -#include "lib/error.hpp" #include "lib/sync.hpp" #include "lib/thread.hpp" +#include "lib/iter-explorer.hpp" +#include "lib/scoped-collection.hpp" -#include -#include - -using std::bind; - -using std::cout; using test::Test; +using lib::explore; +using std::this_thread::yield; +using std::this_thread::sleep_for; +using std::chrono_literals::operator ""us; namespace lib { namespace test{ - - namespace { // private test classes and data... - - const uint NUM_COUNTERS = 20; ///< number of independent counters to increment in parallel - const uint NUM_THREADS = 10; ///< number of threads trying to increment these counters - const uint MAX_PAUSE = 10000; ///< maximum delay implemented as empty counting loop - const uint MAX_SUM = 1000; ///< trigger when to finish incrementing - const uint MAX_INC = 10; ///< maximum increment on each step + + namespace { // private test classes and data... + + const uint NUM_THREADS = 200; + const uint MAX_RAND_SUMMAND = 100; + + + + /** Helper to verify a contended chain calculation */ + template + class Checker + : public Sync + { + size_t hot_sum_{0}; + size_t control_sum_{0}; + + using Lock = typename Sync::Lock; + + public: + bool + verify() ///< verify test values got handled and accounted + { + Lock guard{this}; + return 0 < hot_sum_ + and control_sum_ == hot_sum_; + } + + uint + createVal() ///< generating test values, remembering the control sum + { + uint val{rand() % MAX_RAND_SUMMAND}; + control_sum_ += val; + return val; + } + + void + addValues (uint a, uint b) ///< to be called concurrently + { + Lock guard{this}; + + hot_sum_ *= 2; + sleep_for (200us); // force preemption + hot_sum_ += 2 * (a+b); + sleep_for (200us); + hot_sum_ /= 2; + } + }; + }// (End) test classes and data.... - class Victim - : public Sync - { - volatile uint cnt_[NUM_COUNTERS]; - volatile uint step_; ///< @note stored as instance variable - - void - pause () - { - Lock guard{this}; // note recursive lock - - for ( uint i=0, lim=(rand() % MAX_PAUSE); i thread_; - void - doIt () + virtual void + run (Arg) { - while (ourVictim.belowLimit()) - ourVictim.inc (rand() % MAX_INC); + CHECK (can_calc_without_Error()); + CHECK (can_calc_without_Error()); + CHECK (not can_calc_without_Error()); } - public: - HavocThread () - : thread_("HavocThread" - , bind (&HavocThread::doIt, this) - ) + template + bool + can_calc_without_Error() { - CHECK (thread_); - } - - ~HavocThread () - { - if (thread_) - thread_.join(); + Checker checksum; // shared variable used by multiple threads + + lib::ScopedCollection threads{NUM_THREADS}; + for (uint i=1; i<=NUM_THREADS; ++i) + threads.emplace ([&checksum, // Note: added values prepared in main thread + a = checksum.createVal(), + b = checksum.createVal()] + { + checksum.addValues (a,b); + }); + + while (explore(threads).has_any()) + yield(); // wait for all threads to terminate + + return checksum.verify(); } }; - } // (End) test classes and data.... - - - - - - - - - - - /******************************************************************//** - * @test create multiple threads, all concurrently trying to increment - * a number of counters with random steps and random pauses. Without - * locking, the likely result will be differing counters. - * But because the class Victim uses an object level monitor to - * guard the mutations, the state should remain consistent. - * - * @see SyncWaiting_test condition based wait/notify - * @see SyncClasslock_test locking a type, not an instance - * @see sync.hpp - */ - class SyncLocking_test : public Test - { - - virtual void - run (Arg) - { - CHECK (ourVictim.checkAllEqual()); - { - HavocThread threads[NUM_THREADS] SIDEEFFECT; - } - // all finished and joined here... - - if (!ourVictim.checkAllEqual()) - { - cout << "Thread locking is broken; internal state got messed up\n" - "NOTE: all counters should be equal and >=" << MAX_SUM << "\n"; - ourVictim.report(); - } - } - - }; - - - - /** Register this test class... */ - LAUNCHER (SyncLocking_test, "unit common"); - - - + + + /** Register this test class... */ + LAUNCHER (SyncLocking_test, "function common"); + + + }} // namespace lib::test diff --git a/tests/library/sync-locking-test_2.cpp b/tests/library/sync-locking-test_2.cpp deleted file mode 100644 index 73cf90cd4..000000000 --- a/tests/library/sync-locking-test_2.cpp +++ /dev/null @@ -1,149 +0,0 @@ -/* - SyncLocking(Test) - check the monitor object based locking - - Copyright (C) Lumiera.org - 2008, Hermann Vosseler - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - -/** @file sync-locking-test.cpp - ** unit test \ref SyncLocking_test - */ - - -#include "lib/test/run.hpp" - -#include "lib/sync.hpp" -#include "lib/thread.hpp" -#include "lib/iter-explorer.hpp" -#include "lib/scoped-collection.hpp" - -using test::Test; -using lib::explore; -using std::this_thread::yield; -using std::this_thread::sleep_for; -using std::chrono_literals::operator ""us; - - -namespace lib { -namespace test{ - - namespace { // private test classes and data... - - const uint NUM_THREADS = 200; - const uint MAX_RAND_SUMMAND = 100; - - - - /** Helper to verify a contended chain calculation */ - template - class Checker - : public Sync - { - size_t hot_sum_{0}; - size_t control_sum_{0}; - - using Lock = typename Sync::Lock; - - public: - bool - verify() ///< verify test values got handled and accounted - { - Lock guard{this}; - return 0 < hot_sum_ - and control_sum_ == hot_sum_; - } - - uint - createVal() ///< generating test values, remembering the control sum - { - uint val{rand() % MAX_RAND_SUMMAND}; - control_sum_ += val; - return val; - } - - void - addValues (uint a, uint b) ///< to be called concurrently - { - Lock guard{this}; - - hot_sum_ *= 2; - sleep_for (200us); // force preemption - hot_sum_ += 2 * (a+b); - sleep_for (200us); - hot_sum_ /= 2; - } - }; - }// (End) test classes and data.... - - - - - - - /******************************************************************//** - * @test verify the object monitor provides locking and to prevent - * data corruption on concurrent modification of shared storage. - * - use a chained calculation with deliberate sleep state - * while holding onto an intermediary result - * - run this calculation contended by a huge number of threads - * - either use locking or no locking - * @see sync.happ - * @see thread.hpp - */ - class SyncLocking_test2 : public Test - { - - virtual void - run (Arg) - { - CHECK (can_calc_without_Error()); - CHECK (can_calc_without_Error()); - CHECK (not can_calc_without_Error()); - } - - - template - bool - can_calc_without_Error() - { - Checker checksum; // shared variable used by multiple threads - - lib::ScopedCollection threads{NUM_THREADS}; - for (uint i=1; i<=NUM_THREADS; ++i) - threads.emplace ([&checksum, // Note: added values prepared in main thread - a = checksum.createVal(), - b = checksum.createVal()] - { - checksum.addValues (a,b); - }); - - while (explore(threads).has_any()) - yield(); // wait for all threads to terminate - - return checksum.verify(); - } - }; - - - - /** Register this test class... */ - LAUNCHER (SyncLocking_test2, "function common"); - - - -}} // namespace lib::test diff --git a/tests/vault/test-filedescriptors.c b/tests/vault/test-filedescriptors.c deleted file mode 100644 index c20a16db7..000000000 --- a/tests/vault/test-filedescriptors.c +++ /dev/null @@ -1,147 +0,0 @@ -/* - TEST-FILEDESCRIPTORS - verify filedescriptor management - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - -/** @file test-filedescriptors.c - ** C unit test to cover filedescriptor management - ** @see filedescriptor.h - */ - - -#include "lib/safeclib.h" - -#include "common/config.h" -#include "vault/backend.h" -#include "vault/filedescriptor.h" - -#include "lib/test/test.h" - -TESTS_BEGIN - -TEST (acquire_existing) -{ - lumiera_backend_init (); - llist node; - llist_init (&node); - LumieraFiledescriptor descriptor = lumiera_filedescriptor_acquire (",tmp_testfile", LUMIERA_FILE_READONLY, &node); - if (descriptor) - { - lumiera_filedescriptor_release (descriptor, ",tmp_testfile", &node); - lumiera_backend_destroy (); - lumiera_config_destroy (); - return 0; - } - else - return 1; -} - - -TEST (acquire_existing_again) -{ - lumiera_backend_init (); - llist node; - llist_init (&node); - LumieraFiledescriptor descriptor = lumiera_filedescriptor_acquire (",tmp_testfile", LUMIERA_FILE_READONLY, &node); - if (descriptor) - { - llist node2; - llist_init (&node2); - LumieraFiledescriptor descriptor2 = lumiera_filedescriptor_acquire (",tmp_testfile", LUMIERA_FILE_READONLY, &node2); - lumiera_filedescriptor_release (descriptor2, ",tmp_testfile", &node2); - lumiera_filedescriptor_release (descriptor, ",tmp_testfile", &node); - lumiera_backend_destroy (); - lumiera_config_destroy (); - return 0; - } - else - return 1; -} - -TEST (acquire_existing_3files) -{ - lumiera_backend_init (); - llist node1; - llist_init (&node1); - LumieraFiledescriptor descriptor1 = lumiera_filedescriptor_acquire (",tmp_testfile1", LUMIERA_FILE_READONLY, &node1); - - llist node2; - llist_init (&node2); - LumieraFiledescriptor descriptor2 = lumiera_filedescriptor_acquire (",tmp_testfile2", LUMIERA_FILE_READONLY, &node2); - - llist node3; - llist_init (&node3); - LumieraFiledescriptor descriptor3 = lumiera_filedescriptor_acquire (",tmp_testfile3", LUMIERA_FILE_READONLY, &node3); - if (descriptor1) - lumiera_filedescriptor_release (descriptor1, ",tmp_testfile1", &node1); - - if (descriptor2) - lumiera_filedescriptor_release (descriptor2, ",tmp_testfile2", &node2); - - if (descriptor3) - lumiera_filedescriptor_release (descriptor3, ",tmp_testfile3", &node3); - - if (descriptor1 && descriptor2 && descriptor3) - { - lumiera_backend_destroy (); - lumiera_config_destroy (); - return 0; - } - else - return 1; -} - -TEST (acquire_create) -{ - lumiera_backend_init (); - llist node; - llist_init (&node); - LumieraFiledescriptor descriptor = lumiera_filedescriptor_acquire (",tmp_testfile", LUMIERA_FILE_CREATE, &node); - if (descriptor) - { - lumiera_filedescriptor_release (descriptor, ",tmp_testfile", &node); - lumiera_backend_destroy (); - lumiera_config_destroy (); - return 0; - } - else - return 1; -} - -TEST (acquire_create_dir) -{ - lumiera_backend_init (); - llist node; - llist_init (&node); - LumieraFiledescriptor descriptor = - lumiera_filedescriptor_acquire (",tmp_testdir/nested/,tmp_testfile", LUMIERA_FILE_CREATE, &node); - if (descriptor) - { - lumiera_filedescriptor_release (descriptor, ",tmp_testdir/nested/,tmp_testfile", &node); - lumiera_backend_destroy (); - lumiera_config_destroy (); - return 0; - } - else - return 1; -} - - -TESTS_END diff --git a/tests/vault/test-filehandles.c b/tests/vault/test-filehandles.c deleted file mode 100644 index 1b5ae33af..000000000 --- a/tests/vault/test-filehandles.c +++ /dev/null @@ -1,126 +0,0 @@ -/* - TEST-FILEHANDLES - test filehandle management - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - -/** @file test-filehandles.c - ** C unit test to cover management for filehandles and the filehandle cache - ** @see filehandlecache.h - */ - - -#include "lib/llist.h" -#include "lib/tmpbuf.h" -#include "common/config.h" - -#include "vault/backend.h" -#include "vault/filehandlecache.h" - -#include "lib/test/test.h" - -TESTS_BEGIN - -TEST ("basic") -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp_testfile", LUMIERA_FILE_CREATE); - - /* get the filehandle */ - int fd = lumiera_file_handle_acquire (file); - - /* we now 'own'it and can use it */ - CHECK (fd > -1); - printf ("got filehandle #%d\n", fd); - - /* put it into aging, can't use anymore */ - lumiera_file_handle_release (file); - - lumiera_file_delete (file); - lumiera_backend_destroy (); - lumiera_config_destroy (); -} - - -TEST ("more") -{ - lumiera_backend_init (); - - LumieraFile files[100]; - int fds[100]; - - /*create 100 files*/ - for (int i=0; i<100; ++i) - { - files[i]= lumiera_file_new (lumiera_tmpbuf_snprintf (256, ",tmpdir/testfile%d", i), LUMIERA_FILE_CREATE); - } - - /* get the filehandles, this gross overallocates filehandles */ - for (int i=0; i<100; ++i) - { - fds[i] = lumiera_file_handle_acquire (files[i]); - CHECK (fds[i] > -1); - printf ("got filehandle #%d\n", fds[i]); - } - - /* put them into aging, can't use anymore */ - for (int i=0; i<100; ++i) - { - lumiera_file_handle_release (files[i]); - } - - /* cleanup */ - for (int i=0; i<100; ++i) - { - lumiera_file_delete (files[i]); - } - - lumiera_backend_destroy (); - lumiera_config_destroy (); -} - - - -TEST (file_locking) -{ - - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp_testfile", LUMIERA_FILE_RECREATE); - - - LumieraFile locked = lumiera_file_wrlock (file); - CHECK (locked); - CHECK (lumiera_file_unlock (locked)); - - lumiera_file_delete (file); - lumiera_backend_destroy (); - lumiera_config_destroy (); -} - - -PLANNED_TEST (file_locking_blocked) -{ -} - - - - - - -TESTS_END diff --git a/tests/vault/test-fileheader.c b/tests/vault/test-fileheader.c deleted file mode 100644 index 243a34141..000000000 --- a/tests/vault/test-fileheader.c +++ /dev/null @@ -1,161 +0,0 @@ -/* - TEST-FILEHEADER - File identification - - Copyright (C) Lumiera.org - 2010, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - -/** @file test-fileheader.c - ** C unit test to verify file identification functions - ** @see fileheader.h - */ - - -#include "vault/backend.h" -#include "vault/fileheader.h" - -#include "lib/test/test.h" - -#include "vault/filehandlecache.h" -extern LumieraFilehandlecache lumiera_fhcache; - -TESTS_BEGIN - - -TEST (create_basic) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-fileheader", LUMIERA_FILE_RECREATE); - - LUMIERA_FILE_WRLOCK_SECTION (NOBUG_ON, file) - { - lumiera_fileheader header = lumiera_fileheader_create (file, "TEST", 0, sizeof (lumiera_fileheader_raw), LUMIERA_FILEHEADER_FLAG_ENDIANESS); - - CHECK (lumiera_error_peek() == NULL); - - ECHO ("fileheader: %s:", (char*)header.header); - - lumiera_fileheader_close (&header, LUMIERA_FILEHEADER_FLAG_CLEAN); - } - - lumiera_file_delete (file); - lumiera_backend_destroy (); -} - - - -TEST (create_nowrite) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-fileheader", LUMIERA_FILE_READONLY); - CHECK(file); - - LUMIERA_FILE_RDLOCK_SECTION (NOBUG_ON, file) - { - lumiera_fileheader header = lumiera_fileheader_create (file, "TEST", 0, sizeof (lumiera_fileheader), NULL); - - CHECK (lumiera_error() == LUMIERA_ERROR_FILEHEADER_NOWRITE); - - lumiera_fileheader_close (&header, LUMIERA_FILEHEADER_FLAG_CLEAN); - } - - lumiera_file_delete (file); - lumiera_backend_destroy (); -} - - - -TEST (acquire_wrongheader) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-fileheader", LUMIERA_FILE_READONLY); - - LUMIERA_FILE_RDLOCK_SECTION (NOBUG_ON, file) - { - lumiera_fileheader header = lumiera_fileheader_open (file, "BADH", - sizeof (lumiera_fileheader), - LUMIERA_FILEHEADER_FLAG_CLEAN, LUMIERA_FILEHEADER_FLAG_CLEAN); - - CHECK (!header.header); - CHECK (lumiera_error() == LUMIERA_ERROR_FILEHEADER_HEADER); - - lumiera_fileheader_close (&header, LUMIERA_FILEHEADER_FLAG_CLEAN); - } - - lumiera_file_delete (file); - lumiera_backend_destroy (); -} - - - -TEST (acquire_basic) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-fileheader", LUMIERA_FILE_READWRITE); - - LUMIERA_FILE_RDLOCK_SECTION (NOBUG_ON, file) - { - lumiera_fileheader header = lumiera_fileheader_open (file, "TEST", - sizeof (lumiera_fileheader), - LUMIERA_FILEHEADER_FLAG_CLEAN, LUMIERA_FILEHEADER_FLAG_CLEAN); - - CHECK (header.header); - CHECK (!lumiera_error()); - - CHECK (lumiera_fileheader_version (&header) == 0); - - ECHO ("fileheader: %s:", (char*)header.header); - - lumiera_fileheader_close (&header, LUMIERA_FILEHEADER_FLAG_CLEAN); - } - - lumiera_file_delete (file); - lumiera_backend_destroy (); - -} - - -TEST (acquire_basic_readonly) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-fileheader", LUMIERA_FILE_READONLY); - - LUMIERA_FILE_RDLOCK_SECTION (NOBUG_ON, file) - { - lumiera_fileheader header = lumiera_fileheader_open (file, "TEST", - sizeof (lumiera_fileheader), - LUMIERA_FILEHEADER_FLAG_CLEAN, NULL); - - CHECK (header.header); - CHECK (!lumiera_error()); - - CHECK (lumiera_fileheader_version (&header) == 0); - - ECHO ("fileheader: %s:", (char*)header.header); - - lumiera_fileheader_close (&header, LUMIERA_FILEHEADER_FLAG_CLEAN); - } - - lumiera_file_delete (file); - lumiera_backend_destroy (); - -} - - - -TESTS_END diff --git a/tests/vault/test-filemmap.c b/tests/vault/test-filemmap.c deleted file mode 100644 index df904a7f5..000000000 --- a/tests/vault/test-filemmap.c +++ /dev/null @@ -1,382 +0,0 @@ -/* - TEST-FILEMAP - test file memory mapping - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - -/** @file test-filemmap.c - ** C unit test to cover memory mapping of file contents - ** @see mmapings.h - */ - - -//#include -//#include - -#include "lib/llist.h" -#include "common/config.h" - -#include "vault/backend.h" -#include "vault/file.h" -#include "vault/filedescriptor.h" -#include "vault/mmapings.h" -#include "vault/mmap.h" - -#include "lib/test/test.h" - -#include -#include -#include -#include - -TESTS_BEGIN - -/* - The first two are not really tests, but some code to check for - filesystem and kernel semantics which might be moved elsewhere someday -*/ -TEST (mmap_semantic) -{ - int fd = open (",mmaptest", O_RDWR|O_CREAT, 0666); - printf ("got fd %d\n", fd); - printf ("error %s\n", strerror (errno)); - int dummy = ftruncate (fd, 8192); - (void)dummy; - TODO ("handle error case better"); - - void* addr = mmap (NULL, - 8192, - PROT_WRITE, - MAP_SHARED, - fd, - 0); - - printf ("mapped at %p\n", addr); - printf ("error %s\n", strerror (errno)); - - int i = 0; - while (++i && ! errno) - { - addr = mmap (NULL, - 4096, - PROT_READ|PROT_WRITE, - MAP_SHARED, - fd, - 0); - - printf ("mapped %d again %p\n", i, addr); - printf ("error %s\n", strerror (errno)); - } -} - -TEST (fd_semantic) -{ - mkdir (",testdir", 0777); errno = 0; - - int i = 0; - while (++i && ! errno) - { - char name[256]; - - sprintf (name, ",testdir/file_%d", i); - int fd = open (name, O_RDWR|O_CREAT, 0666); - printf ("#%d opened %d\n", i, fd); - printf ("error %s\n", strerror (errno)); - - void* addr = mmap (NULL, - 8192, - PROT_WRITE, - MAP_SHARED, - fd, - 0); - printf ("mapped at %p\n", addr); - - close (fd); - } - -} - - -TEST (mmap_missing_chunksize) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-filemmap", LUMIERA_FILE_CREATE); - - LumieraMMapings mmaps = lumiera_file_mmapings (file); - - if (!mmaps) - fprintf (stderr, "%s\n", lumiera_error()); - - lumiera_file_delete (file); - - lumiera_backend_destroy (); - lumiera_config_destroy (); -} - - -TEST (mmap_forget_releasing) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-filemmap", LUMIERA_FILE_CREATE); - lumiera_file_set_chunksize_bias (file, 4096, 0); - - LumieraMMapings mmaps = lumiera_file_mmapings (file); - - LumieraMMap map = lumiera_mmapings_mmap_acquire (mmaps, file, 0, 100); - - (void) map; //lumiera_mmapings_release_mmap (mmaps, &user, mmap); - - lumiera_file_delete (file); - - lumiera_backend_destroy (); - lumiera_config_destroy (); -} - - -TEST (mmap_simple) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-filemmap", LUMIERA_FILE_CREATE); - lumiera_file_set_chunksize_bias (file, 4096, 0); - - LumieraMMapings mmaps = lumiera_file_mmapings (file); - - LumieraMMap map = lumiera_mmapings_mmap_acquire (mmaps, file, 0, 100); - - lumiera_mmapings_release_mmap (mmaps, map); - - lumiera_file_delete (file); - - /* check that the file got truncated to the desired size */ - struct stat st; - stat (",tmp-filemmap", &st); - CHECK (st.st_size == 100); - - lumiera_backend_destroy (); - lumiera_config_destroy (); -} - - - -TEST (mmap_checkout_twice) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-filemmap", LUMIERA_FILE_CREATE); - lumiera_file_set_chunksize_bias (file, 4096, 0); - - LumieraMMapings mmaps = lumiera_file_mmapings (file); - - LumieraMMap map = lumiera_mmapings_mmap_acquire (mmaps, file, 0, 100); - - LumieraMMap map2 = lumiera_mmapings_mmap_acquire (mmaps, file, 0, 100); - - CHECK (map->address == map2->address); - - lumiera_mmapings_release_mmap (mmaps, map); - - lumiera_mmapings_release_mmap (mmaps, map2); - - lumiera_file_delete (file); - - /* check that the file got truncated to the desired size */ - struct stat st; - stat (",tmp-filemmap", &st); - CHECK (st.st_size == 100); - - lumiera_backend_destroy (); - lumiera_config_destroy (); -} - - - -TEST (mmap_checkout_again) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-filemmap", LUMIERA_FILE_CREATE); - lumiera_file_set_chunksize_bias (file, 4096, 0); - - LumieraMMapings mmaps = lumiera_file_mmapings (file); - - LumieraMMap map = lumiera_mmapings_mmap_acquire (mmaps, file, 0, 100); - lumiera_mmapings_release_mmap (mmaps, map); - - LumieraMMap map2 = lumiera_mmapings_mmap_acquire (mmaps, file, 0, 100); - lumiera_mmapings_release_mmap (mmaps, map2); - - lumiera_file_delete (file); - - /* check that the file got truncated to the desired size */ - struct stat st; - stat (",tmp-filemmap", &st); - CHECK (st.st_size == 100); - - lumiera_backend_destroy (); - lumiera_config_destroy (); -} - - - -TEST (mmap_grow_existing_file) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-filemmap", LUMIERA_FILE_READWRITE); - - lumiera_file_set_chunksize_bias (file, 4096, 0); - - LumieraMMapings mmaps = lumiera_file_mmapings (file); - - LumieraMMap map = lumiera_mmapings_mmap_acquire (mmaps, file, 0, 100); - lumiera_mmapings_release_mmap (mmaps, map); - - lumiera_file_delete (file); - - /* check that the file got truncated to the desired size */ - struct stat st; - stat (",tmp-filemmap", &st); - CHECK (st.st_size == 100); - - lumiera_backend_destroy (); - lumiera_config_destroy (); -} - - -TEST (mmap_readonly_file) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-filemmap", LUMIERA_FILE_READONLY); - - lumiera_file_set_chunksize_bias (file, 4096, 0); - - LumieraMMapings mmaps = lumiera_file_mmapings (file); - - LumieraMMap map = lumiera_mmapings_mmap_acquire (mmaps, file, 0, 100); - lumiera_mmapings_release_mmap (mmaps, map); - - lumiera_file_delete (file); - - /* check that the file got truncated to the desired size */ - struct stat st; - stat (",tmp-filemmap", &st); - CHECK (st.st_size == 100); - - lumiera_backend_destroy (); - lumiera_config_destroy (); -} - - - -TEST (file_access) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-filemmap", LUMIERA_FILE_RECREATE); - lumiera_file_set_chunksize_bias (file, 4096, 0); - - LumieraMMap map = lumiera_file_mmap_acquire (file, 10, 100); - - char* addr = lumiera_mmap_address (map, 20); - - strcpy (addr, "test"); - - lumiera_file_release_mmap (file, map); - - lumiera_file_delete (file); - lumiera_backend_destroy (); -} - - - -TEST (exact_mmap) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-filemmap", LUMIERA_FILE_RECREATE); - - LumieraMMap map = lumiera_mmap_new_exact (file, 0, 6); - - char* addr = lumiera_mmap_address (map, 1); - - strcpy (addr, "test"); - - lumiera_mmap_delete (map); - - lumiera_file_delete (file); - lumiera_backend_destroy (); -} - - - -TEST (mmap_section) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-filemmap", LUMIERA_FILE_RECREATE); - lumiera_file_set_chunksize_bias (file, 4096, 0); - - LUMIERA_FILE_MMAP_SECTION (NOBUG_ON, file, 20, 20, addr) - { - strcpy (addr, "mmap section"); - } - - CHECK(lumiera_error_peek() == NULL); - - lumiera_file_delete (file); - lumiera_backend_destroy (); -} - - -TEST (mmap_section_err) -{ - lumiera_backend_init (); - LumieraFile file = lumiera_file_new (",tmp-filemmap", LUMIERA_FILE_RECREATE); - /* forgot to set lumiera_file_chunksize_set (file, 4096); */ - - LUMIERA_FILE_MMAP_SECTION (NOBUG_ON, file, 20, 20, addr) - { - strcpy (addr, "mmap section"); - } - - CHECK(lumiera_error_peek() == NULL); - - lumiera_file_delete (file); - lumiera_backend_destroy (); -} - - -#if 0 -TEST (refactored_usage) -{ - LumieraFile file = lumiera_file_new ("filename", mode); - - LumieraFrameIndex index = lumiera_frameindex_new ("indexfilename", file /*, indexing engine*/); - - LumieraFrame frame = lumiera_frameindex_frame (index, 123); - - //TAG+HINTS could be NEXT|PREV|EXACT|NEAREST|HARD|SOFT - - lumiera_frame_prefetch (index, 123); - - - lumiera_frame_release (frame); - - lumiera_frameindex_delete (index); - - lumiera_file_free (file); -} -#endif - - -TESTS_END diff --git a/tests/vault/test-resourcecollector.c b/tests/vault/test-resourcecollector.c deleted file mode 100644 index ab533380e..000000000 --- a/tests/vault/test-resourcecollector.c +++ /dev/null @@ -1,83 +0,0 @@ -/* - TEST-RESOURCECOLLECTOR - test the resource collector - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - -/** @file test-resourcecollector.c - ** C unit test to cover management of low-level resources for the Vault - ** @see resourcecollector.h - */ - - -#include "vault/resourcecollector.h" - -#include "lib/test/test.h" - -#include -#include - -static enum lumiera_resource_try -test_memory_handler (enum lumiera_resource_try itr, void* data, void* context) -{ - switch (itr) - { - case LUMIERA_RESOURCE_UNREGISTER: - printf ("unregistering memory handler\n"); - break; - - default: - printf ("memory handler got called\n"); - return LUMIERA_RESOURCE_ALL; - } - - return LUMIERA_RESOURCE_NONE; -} - - - - -TESTS_BEGIN - -TEST (basic) -{ - lumiera_resourcecollector_init (); - lumiera_resourcecollector_register_handler (LUMIERA_RESOURCE_MEMORY, test_memory_handler, NULL); - lumiera_resourcecollector_destroy (); -} - -TEST (memory_collection_mockup) -{ - CHECK (argv[2]); - lumiera_resourcecollector_init (); - - lumiera_resourcecollector_register_handler (LUMIERA_RESOURCE_MEMORY, test_memory_handler, NULL); - - size_t size = 1000; - enum lumiera_resource_try iteration = LUMIERA_RESOURCE_ONE; - int trying = atoi (argv[2]); - do { - --trying; - } while (trying && lumiera_resourcecollector_run (LUMIERA_RESOURCE_MEMORY, &iteration, &size)); - - lumiera_resourcecollector_destroy (); -} - - -TESTS_END diff --git a/tests/vault/test-threadpool.c b/tests/vault/test-threadpool.c deleted file mode 100644 index f4ed3a4db..000000000 --- a/tests/vault/test-threadpool.c +++ /dev/null @@ -1,364 +0,0 @@ -/* - TEST-THREADPOOL - test thread pool creation and usage - - Copyright (C) Lumiera.org - 2009, Michael Ploujnikov - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - -/** @file test-threadpool.c - ** C unit test to cover our threadpool implementation - */ - - -#include "lib/test/test.h" - -#include "vault/threadpool.h" -#include "include/logging.h" - -#include -#include -#include -#include - -void is_prime(void * arg) -{ - unsigned long long number = *(unsigned long long *)arg; - unsigned long long prime = 1; - usleep(1); - for (unsigned long long x = number-1; x >= sqrt(number); --x) - { - if ((number % x) == 0) - { - prime = 0; - break; - } - } - *(unsigned long long *)arg = prime; - usleep(1); -} - -void sleep_fn(void * arg) -{ - unsigned int usec = *(int *)arg; - - unsigned int result = usleep (usec); - *(int *)arg = result; -} - - -void other_fn(void * arg) -{ - int input = *(int *)arg; - lumiera_thread_sync (); // the main thread can discard the argument storage - CHECK (input == 42, "input is not 42, but %d", input); - input -= 42; - ECHO ("result is %d", input); -} - -void sleeping_worker_fn(void * arg) -{ - int input = *(int *)arg; - int delay = rand () % 100000; - usleep (delay); - lumiera_thread_sync (); // the main thread can discard the argument storage - input -= 81; - CHECK (input == 42, "result is not 42, but %d", input); -} - -// subtracts 13 from the value -void joinable_worker_fn(void * arg) -{ - int input = *(int *)arg; - lumiera_thread_sync (); // signal that arguments have been received - *(int *)arg = input - 13; -} - -// adds 42 to the value -void joinable_master_fn(void * arg) -{ - int input = *(int *)arg; - lumiera_thread_sync (); - CHECK (input == 42, "input is not 42, but %d", input); - LumieraThread worker = lumiera_thread_run (LUMIERA_THREADCLASS_IDLE - | LUMIERA_THREAD_JOINABLE, - &joinable_worker_fn, - (void *)&input, - "joinable worker thread", - &NOBUG_FLAG (NOBUG_ON)); - lumiera_thread_sync_other (worker); - lumiera_thread_join (worker); - lumiera_thread_sync_other (worker); // wait until the arguments are sent - lumiera_thread_join (worker); // wait until the result has been calculated - CHECK (input == 42-13, "result is not 42-13=29, but %d", input); - input += 42; - *(int *)arg = input; -} - -TESTS_BEGIN - -TEST (threadpool-basic) -{ - lumiera_threadpool_init(); - lumiera_threadpool_destroy(); -} - -TEST (threadpool1) -{ - ECHO("start by initializing the threadpool"); - lumiera_threadpool_init(); - LumieraThread t1 = - lumiera_threadpool_acquire_thread(LUMIERA_THREADCLASS_INTERACTIVE, - "test purpose", - &NOBUG_FLAG(NOBUG_ON)); - // lumiera_threadpool_release_thread(t1); - ECHO("acquired thread 1 %p",t1); - lumiera_threadpool_destroy(); -} - - -TEST (two-thread-acquire) -{ - ECHO("start by initializing the threadpool"); - lumiera_threadpool_init(); - ECHO("acquiring thread 1"); - - LumieraThread t1 = - lumiera_threadpool_acquire_thread(LUMIERA_THREADCLASS_INTERACTIVE, - "test purpose", - &NOBUG_FLAG(NOBUG_ON)); - - ECHO("acquiring thread 2"); - LumieraThread t2 = - lumiera_threadpool_acquire_thread(LUMIERA_THREADCLASS_IDLE, - "test purpose", - &NOBUG_FLAG(NOBUG_ON)); - - ECHO("thread 1 state=%s", lumiera_threadstate_names[t1->state]); - CHECK(LUMIERA_THREADSTATE_IDLE == t1->state); - - ECHO("thread 2 state=%s", lumiera_threadstate_names[t2->state]); - CHECK(LUMIERA_THREADSTATE_IDLE == t2->state); - - LUMIERA_CONDITION_SECTION(NOBUG_ON, &t1->signal) - { - t1->state = LUMIERA_THREADSTATE_WAKEUP; - LUMIERA_CONDITION_SIGNAL; - } - - LUMIERA_CONDITION_SECTION(NOBUG_ON, &t2->signal) - { - t2->state = LUMIERA_THREADSTATE_WAKEUP; - LUMIERA_CONDITION_SIGNAL; - } - - ECHO("cleaning up"); - lumiera_threadpool_destroy(); -} - -TEST (many-sleepy-threads) -{ - - const int threads_per_pool_count = 10; - unsigned int delay = 10000; - - lumiera_threadpool_init(); - LumieraThread threads[threads_per_pool_count*LUMIERA_THREADCLASS_COUNT]; - - for (int kind = 0; kind < LUMIERA_THREADCLASS_COUNT; ++kind) - { - for (int i = 0; i < threads_per_pool_count; ++i) - { - threads[i+kind*threads_per_pool_count] = - lumiera_thread_run(kind, - &sleep_fn, - (void *) &delay, - "just sleep a bit", - &NOBUG_FLAG(NOBUG_ON)); - } - } - - lumiera_threadpool_destroy(); - -} - -TEST (toomany-random-sleepy-threads (compiletest only)) -{ - const int threads_per_pool_count = 500; - unsigned int delay[threads_per_pool_count*LUMIERA_THREADCLASS_COUNT]; - lumiera_threadpool_init(); - LumieraThread threads[threads_per_pool_count*LUMIERA_THREADCLASS_COUNT]; - - for (int kind = 0; kind < LUMIERA_THREADCLASS_COUNT; ++kind) - { - for (int i = 0; i < threads_per_pool_count; ++i) - { - delay[i] = rand() % 1000000; - threads[i+kind*threads_per_pool_count] = - lumiera_thread_run(kind, - &sleep_fn, - (void *) &delay[i], - "just sleep a bit", - &NOBUG_FLAG(NOBUG_ON)); - } - } - lumiera_threadpool_destroy(); -} - -TEST (no-function) -{ - LumieraThread t; - - lumiera_threadpool_init(); - - t = lumiera_thread_run (LUMIERA_THREADCLASS_INTERACTIVE, - NULL, - NULL, - "process my test function", - &NOBUG_FLAG(NOBUG_ON)); - - // cleanup - ECHO("finished waiting"); - lumiera_threadpool_destroy(); -} - -TEST (process-function) -{ - // this is what the scheduler would do once it figures out what function a job needs to run - LumieraThread t; - unsigned long long number = 1307; - - lumiera_threadpool_init(); - - ECHO ("the input to the function is %llu", number); - - t = lumiera_thread_run (LUMIERA_THREADCLASS_INTERACTIVE, - &is_prime, - (void *)&number, //void * arg, - "process my test function", - &NOBUG_FLAG(NOBUG_ON)); // struct nobug_flag* flag) - - // cleanup - lumiera_threadpool_destroy(); - ECHO("the result is %llu", number); - -} - -TEST (many-random-sleepy-threads (compiletest only)) -{ - const int threads_per_pool_count = 10; - unsigned int delay[threads_per_pool_count*LUMIERA_THREADCLASS_COUNT]; - lumiera_threadpool_init(); - LumieraThread threads[threads_per_pool_count*LUMIERA_THREADCLASS_COUNT]; - - for (int kind = 0; kind < LUMIERA_THREADCLASS_COUNT; ++kind) - { - for (int i = 0; i < threads_per_pool_count; ++i) - { - delay[i] = rand() % 1000000; - threads[i+kind*threads_per_pool_count] = - lumiera_thread_run(kind, - &sleep_fn, - (void *) &delay[i], - "just sleep a bit", - &NOBUG_FLAG(NOBUG_ON)); - } - } - lumiera_threadpool_destroy(); -} - -TEST (simple-sync) -{ - lumiera_threadpool_init (); - - int value = 42; - - LumieraThread other = lumiera_thread_run (LUMIERA_THREADCLASS_IDLE, - &other_fn, - (void *)&value, - "other thread", - &NOBUG_FLAG (NOBUG_ON)); - - ECHO ("syncing with the other thread"); - lumiera_thread_sync_other (other); - value += 42; - CHECK (value == 42*2, "value is not equal to 42*2=84, but %d", value); - - lumiera_threadpool_destroy (); -} - -TEST (sync-many) -{ - lumiera_threadpool_init (); - - int value = 1337; - int workers = 100; - LumieraThread threads[workers]; - - for (int i = 0; i < workers; i++) - { - value = 123; - threads[i] = lumiera_thread_run (LUMIERA_THREADCLASS_IDLE, - &sleeping_worker_fn, - (void *)&value, - "worker thread", - &NOBUG_FLAG (NOBUG_ON)); - lumiera_thread_sync_other (threads[i]); - value -= 123; - } - CHECK (value == 0, "final value is not 0, but %d", value); - lumiera_threadpool_destroy (); -} - -TEST (joinable-thread) -{ - int delay = 10000; - lumiera_threadpool_init (); - LumieraThread t = lumiera_thread_run (LUMIERA_THREADCLASS_IDLE - | LUMIERA_THREAD_JOINABLE, - &sleep_fn, - (void *)&delay, - "joinable idle thread", - &NOBUG_FLAG (NOBUG_ON)); - lumiera_thread_join(t); - lumiera_threadpool_destroy (); -} - -TEST (sync-joinable) -{ - // NOTE: this test essentially avoids concurrency with _sync() calls - lumiera_threadpool_init (); - - int value = 42; - - LumieraThread master = lumiera_thread_run (LUMIERA_THREADCLASS_IDLE - | LUMIERA_THREAD_JOINABLE, - &joinable_master_fn, - (void *)&value, - "joinable master thread", - &NOBUG_FLAG (NOBUG_ON)); - - lumiera_thread_sync_other (master); - value = 7732; - - lumiera_thread_join (master); //////////////////////////////TICKET #803 deadlock here - CHECK (value == 42*2-13, "result is not 42*2-12=71, but %d", value); - - lumiera_threadpool_destroy (); -} - -TESTS_END diff --git a/tests/vault/test-threads.c b/tests/vault/test-threads.c deleted file mode 100644 index 75c17c9c6..000000000 --- a/tests/vault/test-threads.c +++ /dev/null @@ -1,166 +0,0 @@ -/* - TEST-THREADS - test thread management - - Copyright (C) Lumiera.org - 2008, Christian Thaeter - - This program is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2 of - the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - -* *****************************************************/ - -/** @file test-threads.c - ** C unit test to cover thread handling helpers - ** Especially, we rely on a specific setup for threads, which allows - ** to manage worker threads in a threadpool - ** @see threads.h - ** @see test-threadpool.c - */ - - -//#include -//#include - -#include "common/config.h" - -#include "include/logging.h" -#include "lib/mutex.h" -#include "vault/threads.h" -#include "lib/test/test.h" - -#include -#include -#include - -void threadfn(void* blah) -{ - (void) blah; - struct timespec wait = {0,300000000}; - - fprintf (stderr, "thread running %s\n", NOBUG_THREAD_ID_GET); - nanosleep (&wait, NULL); - fprintf (stderr, "thread done %s\n", NOBUG_THREAD_ID_GET); -} - - -void threadsyncfn(void* blah) -{ - struct timespec wait = {0,200000000}; - LumieraCondition sync = (LumieraCondition) blah; - - ECHO ("thread starting up %s", NOBUG_THREAD_ID_GET); - LUMIERA_CONDITION_SECTION(cond_sync, sync) - { - ECHO ("send startup signal %s", NOBUG_THREAD_ID_GET); - LUMIERA_CONDITION_SIGNAL; - ECHO ("wait for trigger %s", NOBUG_THREAD_ID_GET); - LUMIERA_CONDITION_WAIT(1); - } - - ECHO ("thread running %s", NOBUG_THREAD_ID_GET); - nanosleep (&wait, NULL); - ECHO ("thread done %s", NOBUG_THREAD_ID_GET); -} - - - -lumiera_mutex testmutex; - -void mutexfn(void* blah) -{ - (void) blah; - struct timespec wait = {0,300000000}; - - LUMIERA_MUTEX_SECTION (NOBUG_ON, &testmutex) - { - fprintf (stderr, "mutex thread running %s\n", NOBUG_THREAD_ID_GET); - nanosleep (&wait, NULL); - fprintf (stderr, "thread done %s\n", NOBUG_THREAD_ID_GET); - } -} - - - -TESTS_BEGIN - -TEST ("simple_thread") -{ - fprintf (stderr, "main before thread %s\n", NOBUG_THREAD_ID_GET); - - lumiera_thread_run (LUMIERA_THREADCLASS_WORKER, - threadfn, - NULL, - argv[1], - NULL); - - struct timespec wait = {0,600000000}; - nanosleep (&wait, NULL); - fprintf (stderr, "main after thread %s\n", NOBUG_THREAD_ID_GET); -} - -TEST ("thread_synced") -{ - lumiera_condition cnd; - lumiera_condition_init (&cnd, "threadsync", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - - LUMIERA_CONDITION_SECTION(cond_sync, &cnd) - { - ECHO ("main before thread %s", NOBUG_THREAD_ID_GET); - - lumiera_thread_run (LUMIERA_THREADCLASS_WORKER, - threadsyncfn, - &cnd, - argv[1], - NULL); - - ECHO ("main wait for thread being ready %s", NOBUG_THREAD_ID_GET); - LUMIERA_CONDITION_WAIT(1); - - ECHO ("main trigger thread %s", NOBUG_THREAD_ID_GET); - LUMIERA_CONDITION_SIGNAL; - - ECHO ("wait for thread end %s", NOBUG_THREAD_ID_GET); - LUMIERA_CONDITION_WAIT(1); - ECHO ("thread ended %s", NOBUG_THREAD_ID_GET); - } - - lumiera_condition_destroy (&cnd, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); -} - - - -TEST ("mutex_thread") -{ - lumiera_mutex_init (&testmutex, "test", &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); - - LUMIERA_MUTEX_SECTION (NOBUG_ON, &testmutex) - { - fprintf (stderr, "main before thread %s\n", NOBUG_THREAD_ID_GET); - - lumiera_thread_run (LUMIERA_THREADCLASS_WORKER, - mutexfn, - NULL, - argv[1], - NULL); - - struct timespec wait = {0,600000000}; - nanosleep (&wait, NULL); - fprintf (stderr, "main after thread %s\n", NOBUG_THREAD_ID_GET); - } - - lumiera_mutex_destroy (&testmutex, &NOBUG_FLAG(NOBUG_ON), NOBUG_CONTEXT); -} - - -TESTS_END diff --git a/wiki/thinkPad.ichthyo.mm b/wiki/thinkPad.ichthyo.mm index 1d5dad9aa..d042ed195 100644 --- a/wiki/thinkPad.ichthyo.mm +++ b/wiki/thinkPad.ichthyo.mm @@ -64292,7 +64292,7 @@ - + @@ -66355,10 +66355,11 @@ - + - - + + + @@ -66629,7 +66630,7 @@ - + @@ -66689,9 +66690,9 @@ - + - + @@ -66711,9 +66712,7 @@ - - - +

daher erscheint es sinnvoll @@ -66753,9 +66752,7 @@ - - - +

Hier gibt es zwar eine Lücke, nämlich die const& auf einen rvalue — aber das hier ist keine general-purpose-Library! @@ -66771,9 +66768,7 @@ - - - +

also entweder ganz oben oder ganz unten, und ansonsten nur noch einen Template-Parameter durchgeben @@ -66790,9 +66785,7 @@ - - - +

man muß nicht speziell auf den Typ des Trägers abstellen; vielmehr kann man dann einen static-cast nachschalten, und der scheitert dann halt gegebenfalls beim Bauen. Das habe ich grade eben genauso beim neuen Thread-Wrapper gemacht, und es hat sich bisher gut bewährt. Spezielle static-asserts sind dann gar nicht notwendig @@ -66809,9 +66802,7 @@ - - - +

man hätte dann also zwei verschiedene APIs für essentiell das Gleiche @@ -66845,9 +66836,7 @@ - - - +

λ-Notation ist inzwischen allgemein bekannt, und hat den Vorteil, daß das Binding nahe am verwendeten Code liegt. Das gilt ganz besonders auch für bool-Flags, und zudem müssen wir uns dann nicht mehr mit Konvertierungen, volatile und Atomics herumschlagen @@ -66860,9 +66849,7 @@ - - - +

Fazit: Nein — denn λ sind bereits der bessere „Support“ @@ -66886,21 +66873,21 @@ - - - + + + - - + + - - + + - - + + - - + + @@ -66911,27 +66898,23 @@ - + - - - +

Im Besonderen #994 kann nun wirklich langsam zugemacht werden! Das schiebe ich nun schon so lange ¾ fertig vor mir her ... und es ist klar daß ich den letzten Schritt (TYPES<TY....>) noch länger vor mir her schiebe, wiewohl das eigentliche Problem effetiv bereits seit 2017 gelöst ist

- + - - - +

hier hatte ich einen »convenience-shortcut« — und der ist broken @@ -66941,15 +66924,13 @@ - + - - - +

hab dummerweise den this-Ptr nicht mehr an dem Punkt; und das will ich auch nicht ändern — sonst hab ich in jeder dieser delegierenden Funktionen das getMonitor(this), und es ist auch inhaltlich nicht besonders präzise, schließlich arbeitet der Guard auf dem Monitor (siehe ClassLock) @@ -66995,9 +66976,7 @@ - - - +

und zwar, weil eine solche anonyme Instanz den umschließenden Scope nicht schützt; sie sieht aber syntaktisch genauso aus wie ein wirksamer Scope-Guard @@ -67008,9 +66987,7 @@ - - - +

...vielmehr wird das Problem tatsächlich gefixt, durch einen try-catch-Block; die spezielle Konstruktor-Variante bleibt damit erhalten, aber wir bieten generell keinen Support mehr für Member-Funktionen, da dies nur für den Konstruktor möglich wäre, aber nich für die frei stehende wait()-Variante. Ohnehin werden nun Lambdas bevorzugt, weil sie meist am Ort der Verwendung definiert werden und damit besser selbsterklärend sind... @@ -67043,9 +67020,7 @@ - - - +

...rein konzeptionell ist es nämlich nicht notwendig, das Lock zu erlangen; aber unser bisheriges API hat dazu gezwungen. Nicht schlimm, aber auch nicht schön @@ -67053,15 +67028,13 @@ - + - - - +

muß dazu getMonitor() public machen @@ -67071,7 +67044,7 @@ - + @@ -67084,11 +67057,17 @@ - - - - - + + + + + + + + + + + @@ -80690,7 +80669,7 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
- + @@ -81637,7 +81616,7 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
- + @@ -81761,16 +81740,16 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
 - - - - - + + + + + - - - + + + @@ -81796,8 +81775,7 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
 - - + @@ -81841,7 +81819,8 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
 - + + @@ -81855,7 +81834,7 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
 - + @@ -81931,8 +81910,9 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
 - - + + + @@ -82109,11 +82089,11 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
 - - - - - + + + + + @@ -82138,12 +82118,14 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
- - + + + + - + @@ -82156,25 +82138,43 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
 - + - + - - - - + + + + + + + + + + + +

+  * thread_loop: ensure no error remains pending. +

+

+  * trigger application shutdown in case of unrecoverable errors +

+

+  * lumiera_threadpool_acquire_thread: error handling (in collaboration with the resourcecollector) +

+

+  * lumiera_thread_sync: error handing, maybe timed mutex (using the threads heartbeat timeout, shortly before timeout) +

+ + - - - - - + + @@ -82183,8 +82183,8 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
 - - + + @@ -83112,9 +83112,9 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
 - - - + + + @@ -83279,7 +83279,7 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
 - + @@ -83686,28 +83686,397 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
 - - + + - - + + + - - + + + + + + + +

+ nicht mehr verwendeter Code +

+
    +
  • + aus namespace vault +
    • +
  • + Support-Library +
    • +
+ + + + + + + + + + + + + + + + + + + +

+ da ich jetzt den Scheduler entwickle +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

+ die Zukunft liegt aber bei IO_URING +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

+ Das Konzept escheint mit mehr wie ein »das könnte vielleicht funktionieren«-Traum. Es wurde nie auch nur ansatzweise geklärt, wie man einerseits Performance durch Ressourcen-Verschwendung gewinnen kann, dann aber im Notfall im Stande sein sollte, diese verschwendeten Resourcen wieder einzusammeln. Und nicht nur das; es wurde nie gezeigt, daß ein solches Konzept zu einem stabilen Betriebszustand führen kann — tatsächlich wäre zu befürchten, daß das regelmäßige Einsammeln von Resourcen die Applikation sogar ausbremst. Als Beispiel führe ich die allseits bekannten Probleme mit Garbage-Collection an; man bedenke nur, was inzwischen alles an Komplexität in den Java-GC eingebaut wurde, um ihn auf ein gutes Performance-Niveau zu heben +

+ + + + + + + + + + + + + + + + + + + + + + + +

+ Tatsächlich laden wir nichts aus dem geplanten Config-System (sondern aus der setup.ini). +

+

+ Habe grade geprüft: selbst der Plugin-Loader hatte zwar eine Abhängigkeit, verwendet dann aber hart gecodete Werte. +

+

+ ALLERDINGS wird lumiera_config_setdefault() verwendet, um den Wert aus der setup.ini zu injizieren +

+ + + + + + + + + + + + + + + + + + + + + + + + + + +

+ lumiera_config_setdefault() wird weiterhin im Plugin-Loader gebraucht +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - + + @@ -83716,9 +84085,7 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
- - - +

Es handelt sich um ein eigenständiges Feature, nämlich daß ein wait abgebrochen wird @@ -83769,9 +84136,7 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
- - - +

...weil es im Konstruktor steht, würde ein Fehler hier den Konstruktor als gescheitert klassifizieren; dadurch würden zwar die Teil-Objekte abgebaut, aber der Objekt-Destruktor nicht aufgerufen. Daher @@ -83785,8 +84150,7 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
- - + @@ -83838,8 +84202,14 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
+ + + + - + + + @@ -83891,7 +84261,7 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
 - + @@ -84163,7 +84533,7 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
- + @@ -88432,7 +88802,7 @@ Date:   Thu Apr 20 18:53:17 2023 +0200
- +