/* JOB-PLANNING.hpp - steps to prepare and build render jobs Copyright (C) Lumiera.org 2012, Hermann Vosseler This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. */ /** @file job-planning.hpp ** The "mechanics" of discovering and planning frame calculation jobs. ** This is a rather abstract chunk of code, to deal especially with the technicalities ** of \em organising the discovery of prerequisites and of joining all the discovered operations ** into a sequence of planning steps. The net result is to present a sequence of job planing ** to the user, while actually encapsulating a depth-first tree exploration, which proceeds on demand. ** ** \par participating elements ** All of these job planning operations are implemented on top of the JobTicket. This is where to look ** for "actual" implementation code. Here, within this header, the following entities cooperate to ** create a simple sequence out of this implementation level tasks: ** - JobPlanningSequence is the entry point for client code: it allows to generate a sequence of jobs ** - JobPlanning is a view on top of all the collected planning information for a single job ** - PlanningState is an iterator, successively exposing a sequence of JobPlanning views ** - #expandPrerequisites(JobPlanning const&) is the operation to explore further prerequisite Jobs recursively ** - PlanningStepGenerator yields the underlying "master beat": a sequence of frame locations to be planned ** ** \par how the PlanningState (sequence) is advanced ** PlanningState is an iterator to expose a sequence of JobPlanning elements. On the implementation level, ** there is always just a single JobPlanning element, which represents the \em current element; this element ** lives as "state core" within the PlanningState object. Advancing to the next JobPlanning element (i.e. to ** consider the next job or prerequisite job to be planned for scheduling) is performed through the iteration ** control API exposed by JobPlanning (the free functions \c checkPoint, \c yield and \c iterNext. Actually, ** these functions are invoked through the depth-first tree exploration performed by JobPlaningSequence. ** The implementation of these invocations can be found within the IterExplorer strategy ** lib::iter_explorer::RecursiveSelfIntegration. The net result is ** - the current element is always accessed through \c yield ** - advancing to the next element happens \em either ** ** - by invoking \c iterNext (when processing a sequence of sibling job prerequisites) ** - by invoking \c integrate (when starting to explore the next level of children) ** ** ** @see DispatcherInterface_test simplified usage examples ** @see JobTicket ** @see Dispatcher ** @see EngineService ** */ #ifndef PROC_ENGINE_JOB_PLANNING_H #define PROC_ENGINE_JOB_PLANNING_H #include "proc/common.hpp" #include "backend/engine/job.h" #include "proc/engine/job-ticket.hpp" #include "proc/engine/frame-coord.hpp" #include "lib/time/timevalue.hpp" #include "lib/iter-explorer.hpp" #include "lib/iter-adapter.hpp" #include "lib/util.hpp" namespace proc { namespace engine { namespace error = lumiera::error; using lib::time::TimeValue; using util::unConst; using util::isnil; /** * View on the execution planning for a single calculation step. * When this view-frontend becomes accessible, behind the scenes all * the necessary information has been pulled and collected from the * low-level model and the relevant rendering/playback configuration. * Typically, clients will materialise this planning into a Job (descriptor) * ready to be entered into the scheduler. * * JobPlanning is indeed a view; the represented planning information is not * persisted (other then in the job to be created). The implementation draws * on a recursive exploration of the corresponding JobTicket, which acts as * a general blueprint for creating jobs within this segment of the timeline. * * @remarks on the implementation level, JobPlanning is used as "state core" * for a PlanningState iterator, to visit and plan subsequently all * the individual operations necessary to render a timeline chunk. */ class JobPlanning { JobTicket::ExplorationState plannedOperations_; FrameCoord point_to_calculate_; public: /** by default create the bottom element of job planning, * which happens to to plan no job at all. It is represented * using an inactive state core (default constructed) */ JobPlanning() { } /** further job planning can be initiated by continuing off a given previous planning state. * This is how the forks are created, expanding into a multitude of prerequisites for * the job in question. */ JobPlanning (JobTicket::ExplorationState const& startingPoint, FrameCoord requestedFrame) : plannedOperations_(startingPoint) , point_to_calculate_(requestedFrame) { } // using the standard copy operations /** cast and explicate this job planning information * to create a frame job descriptor, ready to be scheduled */ operator Job() { if (isnil (plannedOperations_)) throw error::Logic("Attempt to plan a frame-Job based on a missing, " "unspecified, exhausted or superseded job description" ,error::LUMIERA_ERROR_BOTTOM_VALUE); return plannedOperations_->createJobFor (point_to_calculate_); } /** build a new JobPlanning object, * set to explore the prerequisites * at the given planning situation */ JobPlanning discoverPrerequisites() const { if (isnil (plannedOperations_)) return JobPlanning(); else return JobPlanning (plannedOperations_->discoverPrerequisites (point_to_calculate_.channelNr) ,this->point_to_calculate_); } /** integrate another chain of prerequisites into the current evaluation line. * Further evaluation will start to visit prerequisites from the new starting point, * and return to the current evaluation chain later on exhaustion of the side chain. * Especially in case the current evaluation is empty or already exhausted, the * new starting point effectively replaces the current evaluation point */ friend void integrate (JobPlanning const& newStartingPoint, JobPlanning& existingPlan) { if (isnil (existingPlan.plannedOperations_)) { // current evaluation is exhausted: switch to new starting point existingPlan.point_to_calculate_ = newStartingPoint.point_to_calculate_; } existingPlan.plannedOperations_.push (newStartingPoint.plannedOperations_); existingPlan.plannedOperations_.markTreeLocation(); } /* === Iteration control API for IterStateWrapper== */ friend bool checkPoint (JobPlanning const& plan) { return !isnil (plan.plannedOperations_); } friend JobPlanning& yield (JobPlanning const& plan) { REQUIRE (checkPoint (plan)); return unConst(plan); } friend void iterNext (JobPlanning & plan) { plan.plannedOperations_.pullNext(); plan.plannedOperations_.markTreeLocation(); } }; /** * iterator, exposing a sequence of JobPlanning elements */ class PlanningState : public lib::IterStateWrapper { typedef lib::IterStateWrapper _Iter; public: /** inactive evaluation */ PlanningState() : _Iter() { } explicit PlanningState (JobPlanning const& startingPoint) : _Iter(startingPoint) // note: invoking copy ctor on state core { } // using the standard copy operations /* === API for JobPlanningSequence to expand the tree of prerequisites === */ /** attach and integrate the given planning details into this planning state. * Actually the evaluation proceeds depth-first with the other state, * returning to the current position later for further evaluation */ PlanningState & wrapping (JobPlanning const& startingPoint) { integrate (startingPoint, this->stateCore()); return *this; } PlanningState & usingSequence (PlanningState const& prerequisites) { if (isnil (prerequisites)) return *this; else return this->wrapping(*prerequisites); // explanation: PlanningState represents a sequence of successive planning points. // actually this is implemented by switching an embedded JobPlanning element // through a sequence of states. Thus the initial state of an investigation // (which is a JobPlanning) can stand-in for the sequence of prerequisites } /** Extension point to be picked up by ADL. * Provides access for the JobPlanningSequence * for combining and expanding partial results. */ friend PlanningState& build (PlanningState& attachmentPoint) { return attachmentPoint; } }; /** this is the core operation to drive planning ahead: * discover the prerequisites of some operation -- here * "prerequisites" are those operations to be performed * within separate Jobs beforehand. * @note this function is intended to be flat-mapped (">>=") * onto a tree-like monad representing the evaluation process. */ inline PlanningState expandPrerequisites (JobPlanning const& calculationStep) { PlanningState newSubEvaluation( calculationStep.discoverPrerequisites()); return newSubEvaluation; } /** * Abstraction: a Facility to establish frame coordinates * and identify and access the execution plan for this frame. * @see Dispatcher the service interface actually used */ class FrameLocator : public FrameSequencer { public: JobTicket& getJobTicketFor (FrameCoord const& location) { return accessJobTicket (location.modelPort, location.absoluteNominalTime); } bool canContinue (FrameCoord const& location) { return !isEndOfChunk (location.absoluteFrameNumber, location.modelPort); } protected: virtual JobTicket& accessJobTicket (ModelPort, TimeValue nominalTime) =0; virtual bool isEndOfChunk (FrameCnt, ModelPort port) =0; }; /** * Generate a sequence of starting points for Job planning, * based on the underlying frame grid. This sequence will be * used to seed a JobPlanningSequence for generating a chunk * of frame render jobs within a given CalcStream in the player. * Evaluation of that seed will then expand each starting point, * until all prerequisites for those frames are discovered, * resulting in a sequence of Jobs ready to be handed over * to the scheduler for time-bound activation. */ class PlanningStepGenerator { FrameLocator* locationGenerator_; FrameCoord currentLocation_; //////////////////////////////////////////TODO duplicated storage of a FrameCoord record //////////////////////////////////////////TODO nextEvaluation_ is only needed to initialise the "current" sequence //////////////////////////////////////////TODO within the RecursiveSelfIntegration strategy. Maybe this storage could be collapsed? JobPlanning nextEvaluation_; void use_current_location_as_starting_point_for_planning() { JobTicket& processingPlan = locationGenerator_->getJobTicketFor (currentLocation_); nextEvaluation_ = JobPlanning(processingPlan.startExploration() ,currentLocation_); } public: typedef JobPlanning value_type; typedef JobPlanning& reference; typedef JobPlanning * pointer; PlanningStepGenerator(FrameCoord startPoint, FrameLocator& locator) : locationGenerator_(&locator) , currentLocation_(startPoint) { REQUIRE (startPoint.isDefined()); use_current_location_as_starting_point_for_planning(); } // default copyable /* === Iteration control API for IterStateWrapper== */ friend bool checkPoint (PlanningStepGenerator const& gen) { return gen.currentLocation_.isDefined(); } // might indicate end of this planning chunk (or of playback altogether) friend JobPlanning& yield (PlanningStepGenerator const& gen) { ENSURE (checkPoint (gen)); return unConst(gen).nextEvaluation_; } friend void iterNext (PlanningStepGenerator & gen) { if (gen.locationGenerator_->canContinue (gen.currentLocation_)) { gen.currentLocation_ = gen.locationGenerator_->getNextFrame (gen.currentLocation_); gen.use_current_location_as_starting_point_for_planning(); ENSURE (checkPoint (gen)); } else { // indicate end-of playback or a jump to another playback position gen.currentLocation_ = FrameCoord(); } } }; /* type definitions for building the JobPlaningSequence */ typedef PlanningState (*SIG_expandPrerequisites) (JobPlanning const&); typedef lib::IterExplorer JobPlanningChunkStartPoint; typedef JobPlanningChunkStartPoint::FlatMapped::Type ExpandedPlanningSequence; /** * This iterator represents a pipeline to pull planned jobs from. * To dispatch individual frame jobs for rendering, this pipeline is generated * and wired internally such as to interpret the render node definitions. * * \par Explanation of the structure * * The JobPlanningSequence is constructed from several nested layers of functionality * - for the client, it is an iterator, exposing a sequence of JobPlanning elements * - a JobPlanning element allows to add a frame render job to the scheduler * - actually such an element can even be \em converted directly into a Job (descriptor) * - the sequence of such JobPlanning elements (that is, the iterator) is called a PlanningState, * since evaluating this iterator effectively drives the process of job planning ahead * - this planning process is \em implemented as a recursive evaluation and exploration of * a tree of prerequisites; these prerequisites are defined in the JobTicket datastructure * - there is an underlying grid of evaluation starting points, each corresponding to a * single frame. Typically, each frame generates at least two jobs, one for fetching * data, and one for the actual calculations. Depending on the actual render network, * a lot of additional jobs might be necessary * - this basic frame grid is generated by the PlanningStepGenerator, which is * effectively backed by the Dispatcher and thus the render node model. * * @remarks JobPlanningSequence is a monad, and the operation to explore the prerequisites * is applied by the \c >>= (monad flat map operation). This approach allows us * to separate the technicalities of exhausting tree exploration from the actual * "business code" to deal with frame job dependencies */ class JobPlanningSequence : public ExpandedPlanningSequence { public: JobPlanningSequence(engine::FrameCoord startPoint, FrameLocator& locator) : ExpandedPlanningSequence( JobPlanningChunkStartPoint( PlanningStepGenerator(startPoint,locator)) >>= expandPrerequisites) // "flat map" (monad operation) { } }; }} // namespace proc::engine #endif