Group Transformation passes#

group ov_pass_cpp_api

OpenVINO C++ API to work with OpenVINO transformations

Functions

OPENVINO_API void disable_constant_folding (const std::shared_ptr< Node > &node)

this method disables constant folding for given node. Under constant folding we consider ConstantFolding transformation, so other type of constant folding like get_constant_from_source doesn’t work with this attribute. Also before using this attribute please consider two corner cases:

If for sub-graph like ShapeOf->ShapeOf we disable cf for first ShapeOf node, it doesn’t spread to the second ShapeOf, so the entire sub-graph will be folded. (In case if first ShapeOf has exactly one consumer)
If node with disable_constant_folding was replaced with another node, the attribute will be lost because it is not copyable.

OPENVINO_API bool constant_folding_is_disabled (const std::shared_ptr< Node > &node)

Check if constant folding is disabled on Node.

Parameters:: node – Smart pointer to the node.
Returns:: true if attribute constant folding set otherwise false.

OPENVINO_API bool constant_folding_is_disabled (const Node *const node)

Check if constant folding is disabled on Node.

Parameters:: node – Pointer to the node.
Returns:: true if attribute constant folding set otherwise false.

class ConstantFolding : public ov::pass::ModelPass: #include <constant_folding.hpp>

Constant folding iterates over the function and tries to evaluate nodes with constant inputs. Such nodes are then replaced with new Constants containing the result of a folded operation.

class ConvertFP32ToFP16 : public ov::pass::ModelPass: #include <convert_fp32_to_fp16.hpp>

ConvertFP32ToFP16 transformation.

class GraphRewrite : public ov::pass::ModelPass

#include <graph_rewrite.hpp>

GraphRewrite is a container for MatcherPasses that allows to run them on Function in efficient way.

Graph rewrite pass is used for matcher passes execution on Function. To register MatcherPass use

See also

add_matcher<T>(args) method where T is a MatcherPass class. As a default algorithm graph rewrite pass traverse Function in topological order and applies registered matcher passes for each node. But if all registered matcher passes have type based root node in Matcher pattern then efficient mechanism is used to execute them. Matcher pattern root is type based if it’s operation from opset or pattern::op::WrapType. Note: when implementing pattern for Matcher make sure that root node is an operation from opset or has ov::pattern::op::WrapType. That will help GraphRewrite to execute matcher passes more efficient.

Subclassed by ConvertBitwiseToLogical, ov::pass::BackwardGraphRewrite, ov::pass::BidirectionalSequenceDecomposition, ov::pass::CompressFloatConstants, ov::pass::CompressQuantizeWeights, ov::pass::ConcatReduceFusion, ov::pass::ConvertLoopToLSTMSequence, ov::pass::ConvertNmsGatherPathToUnsigned, ov::pass::ConvertReduceToPooling, ov::pass::ConvertReduceToReshape, ov::pass::ConvertSequenceToTensorIterator, ov::pass::ConvertTensorIteratorToSequence, ov::pass::FuseFilteringBoxesBySize, ov::pass::GeluFusion, ov::pass::HSigmoidFusion, ov::pass::HSwishFusion, ov::pass::InitMasks, ov::pass::LSTMCellFusion, ov::pass::LinOpSequenceFusion, ov::pass::MVNFusion, ov::pass::MoveEltwiseUpThroughDataMov, ov::pass::NopElimination, ov::pass::PReluFusion, ov::pass::PadFusion, ov::pass::PropagateMasks, ov::pass::PullThroughReduce, ov::pass::RoPEFusion, ov::pass::SwishFusion, ov::pass::TransposeSinking, ov::pass::low_precision::TypeRelaxedReplacer, ov::pass::transpose_sinking::TSGeneralBackward, ov::pass::transpose_sinking::TSGeneralForward

Public Functions

template<typename T, bool Enabled = true, class ...Args, typename std::enable_if<std::is_base_of<pass::MatcherPass, T>::value, bool>::type = true> inline std::shared_ptr<T> add_matcher(Args&&... args)

Register given transformation class type to GraphRewrite execution list All registered transformations will be executed in a single graph traversal. Example below show the basic usage of pass::GraphRewrite.

pass::Manager manager;
auto anchor = manager.register_pass<GraphRewrite>();
anchor->add_matcher<MatcherPassA>();
anchor->add_matcher<MatcherPassB>();
anchor->set_name("CommonMatchers");
manager.run_passes(f);

For some purposes transformation can be registered and disabled by default.

anchor->add_matcher<MatcherPassB, false>();

Returns:: shared_ptr to the transformation instance

template<typename T, class ...Args, typename std::enable_if<std::is_base_of<pass::GraphRewrite, T>::value, bool>::type = true> inline void add_matcher(Args&&... args)

class ov::pass::LinFusions: public ov::pass::GraphRewrite { public: OPENVINO_RTTI(“LinFusion”); Fusions() { add_matcher<ov::pass::AddFusion>(); add_matcher<ov::pass::MulFusion>(); } };

pass::Manager manager; auto anchor = manager.register_pass<GraphRewrite>(); anchor->add_matcher<LinFusions>(); anchor->add_matcher<OtherFusions>(); anchor->set_name(“CommonFusions”); manager.run_passes(f);

In this case all matcher passes from LinFusions pass will be united with other registered matchers.

virtual void set_pass_config(const std::shared_ptr<PassConfig> &pass_config) override

Set PassConfig for particular transformation instance.

Parameters:: pass_config – is a PassConfig shared_ptr

class LowLatency2 : public ov::pass::ModelPass

#include <low_latency.hpp>

The transformation finds all TensorIterator/Loop layers in the network, processes all back edges that describe a connection between Result and Parameter of the TensorIterator/Loop bodies,and inserts ReadValue and Assign layers at the input and output corresponding to this back edge. Supported platform: CPU.

The example below describes the changes made by the transformation [] - TensorIterator body () - new layer BE - back-edge

before applying the transformation: -> input1[BE_1 -> Parameter -> Layers … -> Result -> BE_1 ]output1->

after applying the transformation: ->(ReadValue)-> input1[BE_1 ->Parameter->Layers …->Result->BE_1]output1 ->(Assign) \ ->… After applying the transformation, the resulting network can be inferred step by step, the states will store between inferences.

class MakeStateful : public ov::pass::ModelPass: #include <make_stateful.hpp>

The transformation replaces the provided pairs Parameter and Result with Memory layers ReadValue and Assign.

class Manager

#include <manager.hpp>

Manager class allows to manage transformation passes.

Public Functions

template<typename T, bool Enable = true, class ...Args> inline std::shared_ptr<T> register_pass(Args&&... args)

pass::Manager manager;
manager.register_pass<MyTransformation>(/* transformation constructor args *&zwj;/);
manager.run_passes(f);

For some purposes transformation can be registered and disabled by default.

manager.register_pass<MyTransformation, false>();

Returns:: shared_ptr to the transformation instance

bool run_passes(const std::shared_ptr<Model> &model)

Runs registered transformations on a given model.

Parameters:: model – Input model
Returns:: Returns true if the model was changed by transformations, false otherwise.

void set_per_pass_validation(bool new_state)

Set flag to enable/disable running Validate pass after executing each registered pass.

Parameters:: new_state – Value “true” enables Validate pass run; “false”, otherwise

inline std::shared_ptr<PassConfig> get_pass_config()

Returns:: PassConfig shared object. This object is used for transformations pipeline configuration. This object allows to disable/enable transformations execution, set callback to particular transformation. For mo details see PassConfig class.

class MatcherPass : public ov::pass::PassBase

#include <matcher_pass.hpp>

MatcherPass is a basic block for pattern based transformations. It describes pattern and action that is applied if pattern is matched.

MatcherPass consists of Matcher and matcher_pass_callback that needs to be implemented and finally registered by using

See also

register_matcher. MatcherPass can be executed on node within

See also

apply method. To run matcher pass on Function use GraphRewrite. In addition MatcherPass provides a way for adding new operations into GraphRewrite execution queue. That means that operations that were created inside transformation callback can be added for matching. To register node use

See also

register_new_node method. GraphRewrite automatically takes registered nodes and put them to execution queue. If multiple nodes were register make sure that they were registered in topological order. Note: when implementing pattern for Matcher make sure that root node is an operation from opset or has ov::pass::pattern::op::WrapType. That will help GraphRewrite to execute matcher passes more efficient.

class PassBase

#include <pass.hpp>

Base class for transformation passes.

Subclassed by ov::pass::MatcherPass, ov::pass::ModelPass

Public Functions

bool get_property(const PassPropertyMask &prop_mask) const: Check if this pass has all the pass properties.

void set_callback(const param_callback &callback)

Set callback for particular transformation type. This method set global callback. For more details see PassConfig class documentation.

Parameters:: callback – lambda function that takes node and returns bool

inline virtual void set_pass_config(const std::shared_ptr<PassConfig> &pass_config)

Set PassConfig for particular transformation instance.

Parameters:: pass_config – is a PassConfig shared_ptr

inline std::shared_ptr<PassConfig> get_pass_config()

Allows to access PassConfig shared instance.

Returns:: Shared instance of PassConfig class

inline bool transformation_callback(const std::shared_ptr<const Node> &node)

Applies callback for given node. By default callback returns false.

Parameters:: node – which will be used inside callback
Returns:: result of callback execution for given node

class ModelPass : public ov::pass::PassBase

#include <pass.hpp>

Base class for Model passes.

class PassConfig

#include <pass_config.hpp>

Class representing a transformations config that is used for disabling/enabling transformations registered inside pass::Manager and also allows to set callback for all transformations or for particular transformation.

When pass::Manager is created all passes registered inside this manager including nested passes will share the same instance of PassConfig class. To work with this class first you need to get shared instance of this class by calling manager.get_pass_config() method. Then you will be able to disable/enable passes based on transformations type_info. For example:

pass::Manager manager;
manager.register_pass<CommonOptimizations>();
auto pass_config = manager.get_pass_config();
pass_config->disable<ConvertGELU>(); // this will disable nested pass inside
                                     // CommonOptimizations pipeline
manager.run_passes(f);

Sometimes it is needed to call transformation inside other transformation manually. And for that case before running transformation you need manually check that this pass is not disabled and then you need to set current PassConfig instance to this transformation. For example:

// Inside MatcherPass callback or inside FunctionPass run_on_function() method
// you need to call get_pass_config() method to get shared instance of PassConfig
auto pass_config = get_pass_config();

// Before running nested transformation you need to check is it disabled or not
if (!pass_config->is_disabled<ConvertGELU>()) {
    auto pass = ConvertGELU();
    pass->set_pass_config(pass_config);
    pass.apply(node);
}

Following this logic inside your transformations you will guaranty that transformations will be executed in a right way.

Public Functions

PassConfig(): Default constructor.

void disable(const DiscreteTypeInfo &type_info)

Disable transformation by its type_info.

Parameters:: type_info – Transformation type_info

template<class T> inline void disable(): Disable transformation by its class type (based on type_info)

void enable(const DiscreteTypeInfo &type_info)

Enable transformation by its type_info.

Parameters:: type_info – Transformation type_info

template<class T> inline void enable(): Enable transformation by its class type (based on type_info)

inline void set_callback(const param_callback &callback): Set callback for all kind of transformations.

template<typename T, class ...Args> inline void set_callback(const param_callback &callback)

Set callback for particular transformation class types.

Example below show how to set callback for one or multiple passes using this method.

pass_config->set_callback<ov::pass::ConvertBatchToSpace,
                          ov::pass::ConvertSpaceToBatch>(
         [](const_node_ptr &node) -> bool {
              // Disable transformations for cases when input shape rank is not
              equal to 4
              const auto input_shape_rank =
              node->get_output_partial_shape(0).rank().get_length();
              if (input_shape_rank != 4) {
                  return false;
              }
              return true;
          });

Note that inside transformations you must provide code that work with this callback. See example below:

if (transformation_callback(node)) {
    return false; // exit from transformation
}

param_callback get_callback(const DiscreteTypeInfo &type_info) const

Get callback for given transformation type_info.

In case if callback wasn’t set for given transformation type then global callback will be returned. But if even global callback wasn’t set then default callback will be returned.

Parameters:: type_info – Transformation type_info

template<class T> inline param_callback get_callback() const

Get callback for given transformation class type.

Returns:: callback lambda function

inline bool is_disabled(const DiscreteTypeInfo &type_info) const

Check either transformation type is disabled or not.

Parameters:: type_info – Transformation type_info
Returns:: true if transformation type was disabled and false otherwise

template<class T> inline bool is_disabled() const

Check either transformation class type is disabled or not.

Returns:: true if transformation type was disabled and false otherwise

inline bool is_enabled(const DiscreteTypeInfo &type_info) const

Check either transformation type is force enabled or not.

Parameters:: type_info – Transformation type_info
Returns:: true if transformation type was force enabled and false otherwise

template<class T> inline bool is_enabled() const

Check either transformation class type is force enabled or not.

Returns:: true if transformation type was force enabled and false otherwise

class SDPAToPagedAttention : public ov::pass::ModelPass: #include <sdpa_to_paged_attention.hpp>

The transformation replaces KV-cache processing part in LLMs by PagedAttention operation.

class Serialize : public ov::pass::ModelPass

#include <serialize.hpp>

Serialize transformation converts ov::Model into IR files.

Attention

dynamic shapes are not supported

class StreamSerialize : public ov::pass::ModelPass

#include <serialize.hpp>

StreamSerialize transformation converts ov::Model into single binary stream.

Attention

dynamic shapes are not supported

struct DataHeader: #include <serialize.hpp>

class StatefulToStateless : public ov::pass::ModelPass: #include <stateful_to_stateless.hpp>

The transformation converts KV cache state back to stateless form.

class Validate : public ov::pass::ModelPass

#include <validate.hpp>

The Validate pass performs sanity checks on attributes and inputs, and computes output shapes and element types for all computation nodes in a given computation graph.

The verification and inference is done via invoking each node’s specific implementation of ov::Node::validate_and_infer_types() function.

By default, the ov::pass::Manager runs this pass after executing every optimization pass. This is to ensure that any update to the graph by an optimization pass does not break the shape and data type requirement on a computation node. This default validation run can be changed via calling the ov::pass::Manager::set_per_pass_validation(bool) function.

class VisualizeTree : public ov::pass::ModelPass: #include <visualize_tree.hpp>

VisualizeTree pass allows to serialize ov::Model to xDot format.