class ngraph::pass::LowLatency2

Overview

The transformation finds all TensorIterator/Loop layers in the network, processes all back edges that describe a connection between Result and Parameter of the TensorIterator body,and inserts ReadValue layer between Parameter and the next layers after this Parameter, and Assign layer after the layers before the Result layer. Supported platforms: CPU, GNA. More…

#include <low_latency.hpp>

class LowLatency2: public ngraph::pass::FunctionPass
{
public:
    // construction

    LowLatency2(bool use_const_initializer = true);

    // methods

    virtual bool run_on_function(std::shared_ptr<ngraph::Function> f);
};

Inherited Members

public:
    // typedefs

    typedef DiscreteTypeInfo type_info_t;

    // methods

    bool get_property(const PassPropertyMask& prop_mask) const;
    void set_name(const std::string& name);
    std::string get_name() const;
    void set_callback(const param_callback& callback);
    virtual void set_pass_config(const std::shared_ptr<PassConfig>& pass_config);
    std::shared_ptr<PassConfig> get_pass_config();
    bool m_transformation_callback(const std::shared_ptr<const Node>& node);
    bool transformation_callback(const std::shared_ptr<const Node>& node);
    virtual const type_info_t& get_type_info() const = 0;
    virtual bool run_on_function(std::shared_ptr<ngraph::Function>) = 0;

Detailed Documentation

The transformation finds all TensorIterator/Loop layers in the network, processes all back edges that describe a connection between Result and Parameter of the TensorIterator body,and inserts ReadValue layer between Parameter and the next layers after this Parameter, and Assign layer after the layers before the Result layer. Supported platforms: CPU, GNA.

The example below describes the changes to the inner part (body, back edges) of the Tensor Iterator layer. [] - TensorIterator body () - new layer

before applying the transformation: back_edge_1 -> [Parameter -> some layers … -> Result ] -> back_edge_1

after applying the transformation: back_edge_1 -> [Parameter -> (ReadValue layer) -> some layers … -> (Assign layer) ] -> Result ] -> back_edge_1

It is recommended to use this transformation in conjunction with the Reshape feature to set sequence dimension to 1 and with the UnrollTensorIterator transformation. For convenience, we have already enabled the unconditional execution of the UnrollTensorIterator transformation when using the LowLatency transformation for CPU, GNA plugins, no action is required here. After applying both of these transformations, the resulting network can be inferred step by step, the states will store between inferences.

The transformation finds all TensorIterator/Loop layers in the network, processes all back edges that describe a connection between Result and Parameter of the TensorIterator/Loop bodies,and inserts ReadValue and Assign layers at the input and output corresponding to this back edge. Supported platforms: CPU, GNA.

The example below describes the changes made by the transformation [] - TensorIterator body () - new layer BE - back-edge

before applying the transformation: -> input1[BE_1 -> Parameter -> Layers … -> Result -> BE_1 ]output1->

after applying the transformation: ->(ReadValue)-> input1[BE_1 ->Parameter->Layers …->Result->BE_1]output1 ->(Assign) ->… After applying the transformation, the resulting network can be inferred step by step, the states will store between inferences.