Group Transformation passes#
- group ov_pass_cpp_api
OpenVINO C++ API to work with OpenVINO transformations
Functions
- OPENVINO_API void disable_constant_folding (const std::shared_ptr< Node > &node)
this method disables constant folding for given node. Under constant folding we consider ConstantFolding transformation, so other type of constant folding like
get_constant_from_source
doesn’t work with this attribute. Also before using this attribute please consider two corner cases:If for sub-graph like ShapeOf->ShapeOf we disable cf for first ShapeOf node, it doesn’t spread to the second ShapeOf, so the entire sub-graph will be folded. (In case if first ShapeOf has exactly one consumer)
If node with disable_constant_folding was replaced with another node, the attribute will be lost because it is not copyable.
- OPENVINO_API bool constant_folding_is_disabled (const std::shared_ptr< Node > &node)
Check if constant folding is disabled on Node.
- Parameters:
node – Smart pointer to the node.
- Returns:
true if attribute constant folding set otherwise false.
- OPENVINO_API bool constant_folding_is_disabled (const Node *const node)
Check if constant folding is disabled on Node.
- Parameters:
node – Pointer to the node.
- Returns:
true if attribute constant folding set otherwise false.
-
class ConstantFolding : public ov::pass::ModelPass
- #include <constant_folding.hpp>
Constant folding iterates over the function and tries to evaluate nodes with constant inputs. Such nodes are then replaced with new Constants containing the result of a folded operation.
-
class ConvertFP32ToFP16 : public ov::pass::ModelPass
- #include <convert_fp32_to_fp16.hpp>
ConvertFP32ToFP16 transformation.
-
class GraphRewrite : public ov::pass::ModelPass
- #include <graph_rewrite.hpp>
GraphRewrite is a container for MatcherPasses that allows to run them on Function in efficient way.
Graph rewrite pass is used for matcher passes execution on Function. To register MatcherPass use
See also
add_matcher<T>(args) method where T is a MatcherPass class. As a default algorithm graph rewrite pass traverse Function in topological order and applies registered matcher passes for each node. But if all registered matcher passes have type based root node in Matcher pattern then efficient mechanism is used to execute them. Matcher pattern root is type based if it’s operation from opset or pattern::op::WrapType. Note: when implementing pattern for Matcher make sure that root node is an operation from opset or has ov::pattern::op::WrapType. That will help GraphRewrite to execute matcher passes more efficient.
Subclassed by ConvertBitwiseToLogical, ov::pass::BackwardGraphRewrite, ov::pass::BidirectionalSequenceDecomposition, ov::pass::CompressFloatConstants, ov::pass::CompressQuantizeWeights, ov::pass::ConcatReduceFusion, ov::pass::ConvertLoopToLSTMSequence, ov::pass::ConvertNmsGatherPathToUnsigned, ov::pass::ConvertReduceToPooling, ov::pass::ConvertReduceToReshape, ov::pass::ConvertSequenceToTensorIterator, ov::pass::ConvertTensorIteratorToSequence, ov::pass::FuseFilteringBoxesBySize, ov::pass::GeluFusion, ov::pass::HSigmoidFusion, ov::pass::HSwishFusion, ov::pass::InitMasks, ov::pass::LSTMCellFusion, ov::pass::LinOpSequenceFusion, ov::pass::MVNFusion, ov::pass::MoveEltwiseUpThroughDataMov, ov::pass::NopElimination, ov::pass::PReluFusion, ov::pass::PadFusion, ov::pass::PropagateMasks, ov::pass::PullThroughReduce, ov::pass::RoPEFusion, ov::pass::SwishFusion, ov::pass::TransposeSinking, ov::pass::low_precision::TypeRelaxedReplacer, ov::pass::transpose_sinking::TSGeneralBackward, ov::pass::transpose_sinking::TSGeneralForward
Public Functions
Register given transformation class type to GraphRewrite execution list All registered transformations will be executed in a single graph traversal. Example below show the basic usage of pass::GraphRewrite.
For some purposes transformation can be registered and disabled by default.pass::Manager manager; auto anchor = manager.register_pass<GraphRewrite>(); anchor->add_matcher<MatcherPassA>(); anchor->add_matcher<MatcherPassB>(); anchor->set_name("CommonMatchers"); manager.run_passes(f);
anchor->add_matcher<MatcherPassB, false>();
- Returns:
shared_ptr to the transformation instance
-
template<typename T, class ...Args, typename std::enable_if<std::is_base_of<pass::GraphRewrite, T>::value, bool>::type = true>
inline void add_matcher(Args&&... args) Register passes from GraphRewrite class that contains sequence of matcher passes registered in its ctor. For example:
class ov::pass::LinFusions: public ov::pass::GraphRewrite { public: OPENVINO_RTTI(“LinFusion”); Fusions() { add_matcher<ov::pass::AddFusion>(); add_matcher<ov::pass::MulFusion>(); } };
pass::Manager manager; auto anchor = manager.register_pass<GraphRewrite>(); anchor->add_matcher<LinFusions>(); anchor->add_matcher<OtherFusions>(); anchor->set_name(“CommonFusions”); manager.run_passes(f);
In this case all matcher passes from LinFusions pass will be united with other registered matchers.
Set PassConfig for particular transformation instance.
- Parameters:
pass_config – is a PassConfig shared_ptr
-
class LowLatency2 : public ov::pass::ModelPass
- #include <low_latency.hpp>
The transformation finds all TensorIterator/Loop layers in the network, processes all back edges that describe a connection between Result and Parameter of the TensorIterator/Loop bodies,and inserts ReadValue and Assign layers at the input and output corresponding to this back edge. Supported platform: CPU.
The example below describes the changes made by the transformation [] - TensorIterator body () - new layer BE - back-edge
before applying the transformation: -> input1[BE_1 -> Parameter -> Layers … -> Result -> BE_1 ]output1->
after applying the transformation: ->(ReadValue)-> input1[BE_1 ->Parameter->Layers …->Result->BE_1]output1 ->(Assign) \ ->… After applying the transformation, the resulting network can be inferred step by step, the states will store between inferences.
-
class MakeStateful : public ov::pass::ModelPass
- #include <make_stateful.hpp>
The transformation replaces the provided pairs Parameter and Result with Memory layers ReadValue and Assign.
-
class Manager
- #include <manager.hpp>
Manager class allows to manage transformation passes.
Public Functions
Register given transformation class type to execution list Example below show the basic usage of pass::Manager.
For some purposes transformation can be registered and disabled by default.pass::Manager manager; manager.register_pass<MyTransformation>(/* transformation constructor args *‍/); manager.run_passes(f);
manager.register_pass<MyTransformation, false>();
- Returns:
shared_ptr to the transformation instance
Runs registered transformations on a given model.
- Parameters:
model – Input model
- Returns:
Returns true if the model was changed by transformations, false otherwise.
-
void set_per_pass_validation(bool new_state)
Set flag to enable/disable running Validate pass after executing each registered pass.
- Parameters:
new_state – Value “true” enables Validate pass run; “false”, otherwise
-
inline std::shared_ptr<PassConfig> get_pass_config()
- Returns:
PassConfig shared object. This object is used for transformations pipeline configuration. This object allows to disable/enable transformations execution, set callback to particular transformation. For mo details see PassConfig class.
-
class MatcherPass : public ov::pass::PassBase
- #include <matcher_pass.hpp>
MatcherPass is a basic block for pattern based transformations. It describes pattern and action that is applied if pattern is matched.
MatcherPass consists of Matcher and matcher_pass_callback that needs to be implemented and finally registered by using
See also
register_matcher. MatcherPass can be executed on node within
See also
apply method. To run matcher pass on Function use GraphRewrite. In addition MatcherPass provides a way for adding new operations into GraphRewrite execution queue. That means that operations that were created inside transformation callback can be added for matching. To register node use
See also
register_new_node method. GraphRewrite automatically takes registered nodes and put them to execution queue. If multiple nodes were register make sure that they were registered in topological order. Note: when implementing pattern for Matcher make sure that root node is an operation from opset or has ov::pass::pattern::op::WrapType. That will help GraphRewrite to execute matcher passes more efficient.
Subclassed by ConvertReduceBase, CvtReduceBase, ov::pass::AUGRUCellFusion, ov::pass::AbsSinking, ov::pass::AdaptivePoolToReduce, ov::pass::AddAddFusion, ov::pass::AddFakeQuantizeFusion, ov::pass::AddMultiplyFusion, ov::pass::AlignEltwiseInputRanks, ov::pass::BatchNormDecomposition, ov::pass::BatchToSpaceFusion, ov::pass::BidirectionalGRUSequenceDecomposition, ov::pass::BidirectionalLSTMSequenceDecomposition, ov::pass::BidirectionalRNNSequenceDecomposition, ov::pass::BinarizeWeights, ov::pass::BroadcastConstRangeReplacement, ov::pass::BroadcastElementwiseFusion, ov::pass::BroadcastTransition, ov::pass::ChainedMaximumOptimization, ov::pass::ClampFusion, ov::pass::CompressFloatConstantsImpl, ov::pass::CompressWeightsWithFakeConvert, ov::pass::CompressWeightsWithFakeQuantize, ov::pass::ConcatFusion, ov::pass::ConcatToBroadcast, ov::pass::ConvStridesPropagation, ov::pass::ConvToBinaryConv, ov::pass::ConvertAvgPool14ToAvgPool1, ov::pass::ConvertBatchToSpace, ov::pass::ConvertBitwiseAndToLogicalAnd, ov::pass::ConvertBitwiseNotToLogicalNot, ov::pass::ConvertBitwiseOrToLogicalOr, ov::pass::ConvertBitwiseXorToLogicalXor, ov::pass::ConvertBroadcast3, ov::pass::ConvertBroadcastToTiles, ov::pass::ConvertConvertLike, ov::pass::ConvertConvertPromoteTypes, ov::pass::ConvertDeformableConv8To1, ov::pass::ConvertDepthToSpace, ov::pass::ConvertDetectionOutput1ToDetectionOutput8, ov::pass::ConvertDetectionOutput8ToDetectionOutput1, ov::pass::ConvertDivide, ov::pass::ConvertDivideWithConstant, ov::pass::ConvertEmbeddingBagOffsets15ToEmbeddingBagOffsetsSum3, ov::pass::ConvertEmbeddingBagPacked15ToEmbeddingBagPackedSum3, ov::pass::ConvertGELU, ov::pass::ConvertGP9ToGPIEInternal, ov::pass::ConvertGRUSequenceToTensorIterator, ov::pass::ConvertGather0D, ov::pass::ConvertGather1ToGather7, ov::pass::ConvertGather7ToGather1, ov::pass::ConvertGather7ToGather8, ov::pass::ConvertGather8ToGather7, ov::pass::ConvertGatherToGatherCompressed, ov::pass::ConvertInterpolate11ToInterpolate4, ov::pass::ConvertInterpolate1ToInterpolate4, ov::pass::ConvertLSTMSequenceToTensorIterator, ov::pass::ConvertLoopWithScatterUpdateToLSTMSequence, ov::pass::ConvertLoopWithSlicedInputConcatOutputToLSTMSequence, ov::pass::ConvertMVN1ToMVN6, ov::pass::ConvertMatrixNmsToMatrixNmsIE, ov::pass::ConvertMaxPool14ToMaxPool8, ov::pass::ConvertMaxPool1ToMaxPool8, ov::pass::ConvertMaxPool8ToMaxPool1, ov::pass::ConvertMinimum, ov::pass::ConvertMod, ov::pass::ConvertMulticlassNms8ToMulticlassNms9, ov::pass::ConvertMulticlassNmsToMulticlassNmsIE, ov::pass::ConvertNMS1ToNMS5, ov::pass::ConvertNMS1ToNMS9, ov::pass::ConvertNMS3ToNMS5, ov::pass::ConvertNMS3ToNMS9, ov::pass::ConvertNMS4ToNMS5, ov::pass::ConvertNMS4ToNMS9, ov::pass::ConvertNMS5ToNMS9, ov::pass::ConvertNMS9ToNMSIEInternal, ov::pass::ConvertNMSRotatedToNMSIEInternal, ov::pass::ConvertNMSToNMSIEInternal, ov::pass::ConvertNegative, ov::pass::ConvertPad12ToPad1, ov::pass::ConvertPadToGroupConvolution, ov::pass::ConvertPriorBox8To0, ov::pass::ConvertQuantizeDequantize, ov::pass::ConvertRNNSequenceToTensorIterator, ov::pass::ConvertROIAlign3To9, ov::pass::ConvertROIAlign9To3, ov::pass::ConvertScatterElementsToScatter, ov::pass::ConvertScatterElementsUpdate12ToScatterElementsUpdate3, ov::pass::ConvertScatterNDUpdate15ToScatterNDUpdate3, ov::pass::ConvertShapeOf3, ov::pass::ConvertShuffleChannels3, ov::pass::ConvertSliceScatter, ov::pass::ConvertSoftMax1ToSoftMax8, ov::pass::ConvertSoftMax8ToSoftMax1, ov::pass::ConvertSpaceToBatch, ov::pass::ConvertSpaceToDepth, ov::pass::ConvertSqueeze15ToSqueeze0, ov::pass::ConvertSubtract, ov::pass::ConvertSubtractWithConstant, ov::pass::ConvertTensorIteratorToGRUSequence, ov::pass::ConvertTensorIteratorToLSTMSequence, ov::pass::ConvertTensorIteratorToRNNSequence, ov::pass::ConvertTopK11ToTopK3, ov::pass::ConvertTopK3, ov::pass::ConvertU4WeightsZeroPointToScalar, ov::pass::ConvertXorToLogicalXor, ov::pass::ConvolutionBackpropDataMultiplyFusion, ov::pass::ConvolutionMultiplyFusion, ov::pass::ConvolutionToGroupConvolutionFusion, ov::pass::DeReshapeFullyConnected, ov::pass::DeReshapeMatMul, ov::pass::DepthToSpaceFusion, ov::pass::DilatedConvolutionConverter, ov::pass::DisableDecompressionConvertConstantFolding, ov::pass::DisableRandomUniformConstantFolding, ov::pass::DisableShapeOfConstantFolding, ov::pass::DivideFusion, ov::pass::DropoutWithRandomUniformReplacer, ov::pass::EinsumDecomposition, ov::pass::EliminateConcat, ov::pass::EliminateConvert, ov::pass::EliminateConvertNonZero, ov::pass::EliminateDuplicateTIInputs, ov::pass::EliminateEltwise, ov::pass::EliminateGatherUnsqueeze, ov::pass::EliminateLoopInputsOutputs, ov::pass::EliminateNopBroadcast, ov::pass::EliminatePad, ov::pass::EliminateReduceReshape, ov::pass::EliminateScatterUpdate, ov::pass::EliminateSlice, ov::pass::EliminateSliceBeforeGatherElements, ov::pass::EliminateSplit, ov::pass::EliminateSplitConcat, ov::pass::EliminateSqueeze, ov::pass::EliminateStridedSlice, ov::pass::EliminateStridedSliceByShape, ov::pass::EliminateTranspose, ov::pass::EliminateUnsqueeze, ov::pass::EliminateUnsqueezeGather, ov::pass::EnableDecompressionConvertConstantFolding, ov::pass::EnableShapeOfConstantFolding, ov::pass::EyeDecomposition, ov::pass::FakeQuantizeDecomposition, ov::pass::FakeQuantizeMulFusion, ov::pass::FakeQuantizeReshapeFusion, ov::pass::FlushFP32SubnormalsToZero, ov::pass::FoldSubgraphEmptyInputs, ov::pass::FuseLSTMSequencesToBidirectionalLSTMSequence, ov::pass::FuseReverseLSTMSequence, ov::pass::GLUFusion, ov::pass::GRUCellDecomposition, ov::pass::GRUCellFusion, ov::pass::GatherNopElimination, ov::pass::Gelu7Downgrade, ov::pass::GeluFusionWithErfFour, ov::pass::GeluFusionWithErfOne, ov::pass::GeluFusionWithErfThree, ov::pass::GeluFusionWithErfTwo, ov::pass::GeluFusionWithTanh, ov::pass::GeluFusionWithTanhNoPower, ov::pass::GroupConvolutionBackpropDataMultiplyFusion, ov::pass::GroupConvolutionMultiplyFusion, ov::pass::GroupNormalizationDecomposition, ov::pass::GroupedGatherElimination, ov::pass::HSigmoidDecomposition, ov::pass::HSigmoidFusionWithClampDiv, ov::pass::HSigmoidFusionWithClampMul, ov::pass::HSigmoidFusionWithReluDiv, ov::pass::HSigmoidFusionWithReluMul, ov::pass::HSigmoidFusionWithoutRelu, ov::pass::HSwishDecomposition, ov::pass::HSwishFusionWithClamp, ov::pass::HSwishFusionWithHSigmoid, ov::pass::HSwishFusionWithReluDiv, ov::pass::HSwishFusionWithReluMul, ov::pass::HardSigmoidDecomposition, ov::pass::InitConstMask, ov::pass::InterpolateSequenceFusion, ov::pass::KeepConstAndDecompression, ov::pass::KeepConstantsPrecisionAndAddConverts, ov::pass::LSTMCellDecomposition, ov::pass::LSTMCellFusionWithJointWeights, ov::pass::LSTMCellFusionWithSplitWeights, ov::pass::LabelResolvingThroughSelect, ov::pass::LeakyReluFusion, ov::pass::LogSoftmaxDecomposition, ov::pass::LoraSubgraphFusion, ov::pass::MVN6Decomposition, ov::pass::MVNFusionWithConstantsInside, ov::pass::MVNFusionWithoutConstants, ov::pass::MarkCompressedFloatConstants, ov::pass::MarkDequantizationSubgraph, ov::pass::MarkFloatingPointRange, ov::pass::MarkRopeInputsToKeepInMixedPrecision, ov::pass::MatMulConstTransposesExtraction, ov::pass::MatMulMultiplyFusion, ov::pass::MishFusion, ov::pass::MoveEltwiseUpThroughDataMovPerChannel, ov::pass::MoveEltwiseUpThroughDataMovScalar, ov::pass::MulFakeQuantizeFusion, ov::pass::MultiplyConvolutionBackpropDataFusion, ov::pass::MultiplyConvolutionFusion, ov::pass::MultiplyGroupConvolutionBackpropDataFusion, ov::pass::MultiplyGroupConvolutionFusion, ov::pass::MultiplyMultiplyFusion, ov::pass::NearestNeighborUpsamplingFusion, ov::pass::NonZeroHorizontalFusion, ov::pass::NopBroadcast, ov::pass::NormalizeL2Decomposition, ov::pass::NormalizeL2Fusion, ov::pass::PReluFusionAbsSubMulMulAdd, ov::pass::PReluFusionMultiplyAdd, ov::pass::PReluFusionMultiplySub, ov::pass::PReluFusionNegReluMulAdd, ov::pass::PReluFusionNegativeAdd, ov::pass::PReluFusionNegativeSub, ov::pass::PadFusionAvgPool, ov::pass::PadFusionConvolution, ov::pass::PadFusionConvolutionBackpropData, ov::pass::PadFusionGroupConvolution, ov::pass::PadFusionGroupConvolutionBackpropData, ov::pass::PositionIDsReplacer, ov::pass::PrepareShapeOpsForEliminationAroundBE, ov::pass::PrevSequenceLengthPattern, ov::pass::Proposal1Scales, ov::pass::Proposal4Scales, ov::pass::PullReshapeThroughReduce, ov::pass::PullSqueezeThroughEltwise, ov::pass::PullTransposeThroughFQUp, ov::pass::PullUnsqueezeThroughReduce, ov::pass::RMSFusion, ov::pass::RNNCellDecomposition, ov::pass::RandomUniformFusion, ov::pass::ReduceL1Decomposition, ov::pass::ReduceL2Decomposition, ov::pass::ReduceMerge, ov::pass::ReduceReshapeFusion, ov::pass::ReluFakeQuantizeFusion, ov::pass::RemoveConcatZeroDimInput, ov::pass::RemoveFilteringBoxesBySize, ov::pass::ReplaceConcatReduceByMinOrMax, ov::pass::ReshapeAMatMul, ov::pass::ReshapeBMatMul, ov::pass::ReshapeOptimizations, ov::pass::ReshapePRelu, ov::pass::ReshapeSequenceFusion, ov::pass::ReshapeSinkingMatMul, ov::pass::ReshapeTo1D, ov::pass::RoPEFusionChatGLM, ov::pass::RoPEFusionCosSinPreprocess, ov::pass::RoPEFusionFlux, ov::pass::RoPEFusionGPTJ, ov::pass::RoPEFusionGPTNEOX, ov::pass::RoPEFusionIOSlicing, ov::pass::RoPEFusionPreprocess, ov::pass::RoPEFusionQwen, ov::pass::RoPEShareCosSin, ov::pass::ScaledDotProductAttentionDecomposition, ov::pass::SelectWithOneValueCondition, ov::pass::SequenceFusion, ov::pass::ShapeOfConstFolding, ov::pass::ShuffleChannelsFusion, ov::pass::SimplifyCTCGreedyDecoderSeqLen, ov::pass::SimplifyGatherShapeOf, ov::pass::SimplifySecondInputOfReshape, ov::pass::SkipGatherBeforeTransposeAndReshape, ov::pass::SliceSequenceToSingleSlice, ov::pass::SliceToStridedSlice, ov::pass::SoftPlusDecomposition, ov::pass::SoftPlusFusion, ov::pass::SoftPlusToMishFusion, ov::pass::SoftSignDecomposition, ov::pass::SoftmaxDecomposition, ov::pass::SoftmaxFusion, ov::pass::SpaceToBatchFusion, ov::pass::SplitConcatPairToInterpolateFusion, ov::pass::SplitSqueezeConcatFusion, ov::pass::SqueezeStridedSlice, ov::pass::StateManagementPattern, ov::pass::StridedSliceSqueeze, ov::pass::SubtractFusion, ov::pass::SupportedNodesStridesPropagation, ov::pass::SwishFusionWithBeta, ov::pass::SwishFusionWithSigmoid, ov::pass::SwishFusionWithSigmoidWithBeta, ov::pass::SwishFusionWithoutBeta, ov::pass::TotalSequenceLengthPattern, ov::pass::TransposeConvert, ov::pass::TransposeEltwise, ov::pass::TransposeFQReduction, ov::pass::TransposeFuse, ov::pass::TransposeMatMul, ov::pass::TransposeReduction, ov::pass::TransposeReshapeEliminationForMatmul, ov::pass::TransposeToReshape, ov::pass::UniqueDecomposition, ov::pass::UnsupportedNodesStridesPropagation, ov::pass::WeightsDequantizeToFakeQuantize, ov::pass::WrapInterpolateIntoTransposes, ov::pass::low_precision::ConvertSubtractConstant, ov::pass::low_precision::CreatePrecisionsDependentAttribute< AttributeType, OperationType >, ov::pass::low_precision::LayerTransformation, ov::pass::low_precision::MarkupBias, ov::pass::low_precision::PropagateThroughPrecisionPreserved< AttributeType >, ov::pass::low_precision::PropagateToInput< AttributeType >, ov::pass::low_precision::PullReshapeThroughDequantization, ov::pass::low_precision::PullTransposeThroughDequantization, ov::pass::low_precision::UpdateSharedPrecisionPreserved< AttributeType, ExpectedAttributeType >, ov::pass::transpose_sinking::TSBinaryBackward, ov::pass::transpose_sinking::TSConcatBackward, ov::pass::transpose_sinking::TSCumSumBackward, ov::pass::transpose_sinking::TSDataMovementBackward, ov::pass::transpose_sinking::TSForwardBase, ov::pass::transpose_sinking::TSFuse, ov::pass::transpose_sinking::TSGatherBackward, ov::pass::transpose_sinking::TSInterpolateBackward, ov::pass::transpose_sinking::TSReductionBackward, ov::pass::transpose_sinking::TSResetNoSinkingAttribute, ov::pass::transpose_sinking::TSSliceBackward, ov::pass::transpose_sinking::TSSplitBackward, ov::pass::transpose_sinking::TSSqueezeBackward, ov::pass::transpose_sinking::TSTileBackward, ov::pass::transpose_sinking::TSUnaryBackward, ov::pass::transpose_sinking::TSUnsqueezeBackward
-
class PassBase
- #include <pass.hpp>
Base class for transformation passes.
Subclassed by ov::pass::MatcherPass, ov::pass::ModelPass
Public Functions
-
bool get_property(const PassPropertyMask &prop_mask) const
Check if this pass has all the pass properties.
-
void set_callback(const param_callback &callback)
Set callback for particular transformation type. This method set global callback. For more details see PassConfig class documentation.
- Parameters:
callback – lambda function that takes node and returns bool
Set PassConfig for particular transformation instance.
- Parameters:
pass_config – is a PassConfig shared_ptr
-
inline std::shared_ptr<PassConfig> get_pass_config()
Allows to access PassConfig shared instance.
- Returns:
Shared instance of PassConfig class
Applies callback for given node. By default callback returns false.
- Parameters:
node – which will be used inside callback
- Returns:
result of callback execution for given node
-
bool get_property(const PassPropertyMask &prop_mask) const
-
class ModelPass : public ov::pass::PassBase
- #include <pass.hpp>
Base class for Model passes.
Subclassed by ov::pass::AlignMixedFP32FP16Types, ov::pass::ApplySymbolEquivalence, ov::pass::ChangePlaceholderTypes, ov::pass::CommonOptimizations, ov::pass::ConstantFolding, ov::pass::ConvertCompressedOnlyToLegacy, ov::pass::ConvertFP32ToFP16, ov::pass::ConvertOpSet2ToOpSet1, ov::pass::ConvertOpSet3ToOpSet2, ov::pass::ConvertPrecision, ov::pass::FindBatch, ov::pass::FusedNamesCleanup, ov::pass::GraphRewrite, ov::pass::GroupedSliceToVSplitOptimization, ov::pass::GroupedStridedSliceOptimizer, ov::pass::Hash, ov::pass::InitNodeInfo, ov::pass::LSTMStatesBroadcast, ov::pass::LowLatency2, ov::pass::MOCLegacyTransformations, ov::pass::MOCTransformations, ov::pass::MakeStateful, ov::pass::MarkPrecisionSensitiveShapeOfSubgraphs, ov::pass::MarkSugraphsToKeepInMixedPrecision, ov::pass::OptimizeSymbolsUsedAsValues, ov::pass::PrintModel, ov::pass::Pruning, ov::pass::PushConstantToSubgraph, ov::pass::RemoveMultiSubGraphOpDanglingParamsResults, ov::pass::ResolveNameCollisions, ov::pass::ReverseInputChannelsFusion, ov::pass::ReverseShapeAndTypeInfer, ov::pass::SDPAToPagedAttention, ov::pass::Serialize, ov::pass::SharedOpOptimization, ov::pass::ShrinkWeights, ov::pass::SimplifyShapeOfSubGraph, ov::pass::SmartReshape, ov::pass::StatefulToStateless, ov::pass::StreamSerialize, ov::pass::StridedSliceOptimization, ov::pass::SymbolicOptimizations, ov::pass::SymbolicPropagation, ov::pass::UnrollIf, ov::pass::UnrollTensorIterator, ov::pass::UselessSliceEraser, ov::pass::Validate, ov::pass::VisualizeTree, ov::pass::low_precision::AlignQuantizationIntervals, ov::pass::low_precision::AlignQuantizationParameters, ov::pass::low_precision::MarkupAvgPoolPrecisionPreserved, ov::pass::low_precision::MarkupCanBeQuantized, ov::pass::low_precision::MarkupOptimizations, ov::pass::low_precision::MarkupPrecisions, ov::pass::low_precision::MarkupQuantizationGranularity, ov::pass::low_precision::PropagatePrecisions, ov::pass::low_precision::PropagateSharedValue< AttributeType >, ov::pass::transpose_sinking::TSGeneral
-
class PassConfig
- #include <pass_config.hpp>
Class representing a transformations config that is used for disabling/enabling transformations registered inside pass::Manager and also allows to set callback for all transformations or for particular transformation.
When pass::Manager is created all passes registered inside this manager including nested passes will share the same instance of PassConfig class. To work with this class first you need to get shared instance of this class by calling manager.get_pass_config() method. Then you will be able to disable/enable passes based on transformations type_info. For example:
Sometimes it is needed to call transformation inside other transformation manually. And for that case before running transformation you need manually check that this pass is not disabled and then you need to set current PassConfig instance to this transformation. For example:pass::Manager manager; manager.register_pass<CommonOptimizations>(); auto pass_config = manager.get_pass_config(); pass_config->disable<ConvertGELU>(); // this will disable nested pass inside // CommonOptimizations pipeline manager.run_passes(f);
Following this logic inside your transformations you will guaranty that transformations will be executed in a right way.// Inside MatcherPass callback or inside FunctionPass run_on_function() method // you need to call get_pass_config() method to get shared instance of PassConfig auto pass_config = get_pass_config(); // Before running nested transformation you need to check is it disabled or not if (!pass_config->is_disabled<ConvertGELU>()) { auto pass = ConvertGELU(); pass->set_pass_config(pass_config); pass.apply(node); }
Public Functions
-
PassConfig()
Default constructor.
-
void disable(const DiscreteTypeInfo &type_info)
Disable transformation by its type_info.
- Parameters:
type_info – Transformation type_info
-
template<class T>
inline void disable() Disable transformation by its class type (based on type_info)
-
void enable(const DiscreteTypeInfo &type_info)
Enable transformation by its type_info.
- Parameters:
type_info – Transformation type_info
-
template<class T>
inline void enable() Enable transformation by its class type (based on type_info)
-
inline void set_callback(const param_callback &callback)
Set callback for all kind of transformations.
-
template<typename T, class ...Args>
inline void set_callback(const param_callback &callback) Set callback for particular transformation class types.
Example below show how to set callback for one or multiple passes using this method.
Note that inside transformations you must provide code that work with this callback. See example below:pass_config->set_callback<ov::pass::ConvertBatchToSpace, ov::pass::ConvertSpaceToBatch>( [](const_node_ptr &node) -> bool { // Disable transformations for cases when input shape rank is not equal to 4 const auto input_shape_rank = node->get_output_partial_shape(0).rank().get_length(); if (input_shape_rank != 4) { return false; } return true; });
if (transformation_callback(node)) { return false; // exit from transformation }
-
param_callback get_callback(const DiscreteTypeInfo &type_info) const
Get callback for given transformation type_info.
In case if callback wasn’t set for given transformation type then global callback will be returned. But if even global callback wasn’t set then default callback will be returned.
- Parameters:
type_info – Transformation type_info
-
template<class T>
inline param_callback get_callback() const Get callback for given transformation class type.
- Returns:
callback lambda function
-
inline bool is_disabled(const DiscreteTypeInfo &type_info) const
Check either transformation type is disabled or not.
- Parameters:
type_info – Transformation type_info
- Returns:
true if transformation type was disabled and false otherwise
-
template<class T>
inline bool is_disabled() const Check either transformation class type is disabled or not.
- Returns:
true if transformation type was disabled and false otherwise
-
inline bool is_enabled(const DiscreteTypeInfo &type_info) const
Check either transformation type is force enabled or not.
- Parameters:
type_info – Transformation type_info
- Returns:
true if transformation type was force enabled and false otherwise
-
template<class T>
inline bool is_enabled() const Check either transformation class type is force enabled or not.
- Returns:
true if transformation type was force enabled and false otherwise
-
PassConfig()
-
class SDPAToPagedAttention : public ov::pass::ModelPass
- #include <sdpa_to_paged_attention.hpp>
The transformation replaces KV-cache processing part in LLMs by PagedAttention operation.
-
class Serialize : public ov::pass::ModelPass
- #include <serialize.hpp>
Serialize transformation converts ov::Model into IR files.
- Attention
dynamic shapes are not supported
-
class StreamSerialize : public ov::pass::ModelPass
- #include <serialize.hpp>
StreamSerialize transformation converts ov::Model into single binary stream.
- Attention
dynamic shapes are not supported
-
struct DataHeader
- #include <serialize.hpp>
-
class StatefulToStateless : public ov::pass::ModelPass
- #include <stateful_to_stateless.hpp>
The transformation converts KV cache state back to stateless form.
-
class Validate : public ov::pass::ModelPass
- #include <validate.hpp>
The Validate pass performs sanity checks on attributes and inputs, and computes output shapes and element types for all computation nodes in a given computation graph.
The verification and inference is done via invoking each node’s specific implementation of ov::Node::validate_and_infer_types() function.
By default, the ov::pass::Manager runs this pass after executing every optimization pass. This is to ensure that any update to the graph by an optimization pass does not break the shape and data type requirement on a computation node. This default validation run can be changed via calling the ov::pass::Manager::set_per_pass_validation(bool) function.
-
class VisualizeTree : public ov::pass::ModelPass
- #include <visualize_tree.hpp>
VisualizeTree pass allows to serialize ov::Model to xDot format.