InferRequest#

interface InferRequest {
    getCompiledModel(): CompiledModel;
    getInputTensor(): Tensor;
    getInputTensor(idx): Tensor;
    getOutputTensor(): Tensor;
    getOutputTensor(idx?): Tensor;
    getTensor(nameOrOutput): Tensor;
    infer(): {
        [outputName: string]: Tensor;
    };
    infer(inputData): {
        [outputName: string]: Tensor;
    };
    infer(inputData): {
        [outputName: string]: Tensor;
    };
    inferAsync(inputData): Promise<{
        [outputName: string]: Tensor;
    }>;
    setInputTensor(tensor): void;
    setInputTensor(idx, tensor): void;
    setOutputTensor(tensor): void;
    setOutputTensor(idx, tensor): void;
    setTensor(name, tensor): void;
}

The InferRequest object is created using CompiledModel.createInferRequest method and is specific for a given deployed model. It is used to make predictions and can be run in asynchronous or synchronous manners.

Methods#

getCompiledModel

  • getCompiledModel(): CompiledModel
    

    It gets the compiled model used by the InferRequest object.

getInputTensor

  • getInputTensor(): Tensor
    

    It gets the input tensor for inference.

    • Parameters:

      • Optional

        idx: number
        
    • Returns: Tensor

      The input tensor for the model. If the model has several inputs, an exception is thrown.

    • Defined in: addon.ts:496

    getInputTensor(idx: number): Tensor
    

    It gets the input tensor for inference.

    • Parameters:

      • Optional

        idx: number
        

        An index of the tensor to get.

    • Returns: Tensor

      A tensor at the specified index. If the tensor with the specified idx is not found, an exception is thrown.

    • Defined in: addon.ts:503

getOutputTensor

  • getOutputTensor(): Tensor
    

    It gets the output tensor for inference.

    • Returns: Tensor

      The output tensor for the model. If the tensor with the specified idx is not found, an exception is thrown.

    • Defined in: addon.ts:509

    getOutputTensor(idx?: number): Tensor
    

    It gets the output tensor for inference.

    • Parameters:

      • Optional

        idx: number
        

        An index of the tensor to get.

    • Returns: Tensor

      A tensor at the specified index. If the tensor with the specified idx is not found, an exception is thrown.

    • Defined in: addon.ts:516

getTensor

  • getTensor(nameOrOutput: string | Output): Tensor;
    

    It gets an input/output tensor for inference. If a tensor with the specified name or port is not found, an exception is thrown.

    • Parameters:

      • nameOrOutput: string | Output

        The name of the tensor or output object.

    • Returns: Tensor

    • Defined in: addon.ts:525

infer

  • infer(): { [outputName: string] : Tensor};
    

    It infers specified input(s) in the synchronous mode. Inputs have to be specified earlier using InferRequest.setTensor or InferRequest.setInputTensor

    • Returns:

      {
      [outputName: string]: Tensor;
      }
      
    • Defined in: addon.ts:460

    infer(inputData): {
        [outputName: string]: Tensor;
    }
    

    It infers specified input(s) in the synchronous mode.

    • Parameters:

      • inputData: {
                   [inputName: string]: Tensor | SupportedTypedArray;
                   }
        

        An object with the key-value pairs where the key is the input name and value can be either a tensor or a TypedArray. TypedArray will be wrapped into Tensor underneath using the input shape and element type of the deployed model.

    • Returns:

      {
      [outputName: string]: Tensor;
      }
      
    • Defined in: addon.ts:468

    infer(inputData): {
        [outputName: string]: Tensor;
    }
    

    It infers specified input(s) in the synchronous mode.

    • Parameters:

      • inputData: Tensor[] | SupportedTypedArray[]
        

        An array with tensors or TypedArrays. TypedArrays will be wrapped into Tensors underneath using the input shape and element type of the deployed model. If the model has multiple inputs, the Tensors and TypedArrays must be passed in the correct order.

    • Returns:

      {
      [outputName: string]: Tensor;
      }
      
    • Defined in: addon.ts:477

inferAsync

  • inferAsync(inputData): Promise<{
        [outputName: string]: Tensor;
    }>
    

    It infers specified input(s) in the asynchronous mode.

    • Parameters:

      • inputData: Tensor[] | {
            [inputName: string]: Tensor;
        }
        

        An object with the key-value pairs where the key is the input name and value is a tensor or an array with tensors. If the model has multiple inputs, the Tensors must be passed in the correct order.

    • Returns:

      Promise<{
       [outputName: string]: Tensor;
      }>
      
    • Defined in: addon.ts:485

setInputTensor

  • setInputTensor(tensor): void
    

    It sets the input tensor to infer models with a single input.

    • Parameters:

      • Tensor

        The input tensor. The element type and shape of the tensor must match the type and size of the model’s input element. If the model has several inputs, an exception is thrown.

    • Returns: void

    • Defined in: addon.ts:532

    setInputTensor(idx, tensor): void
    

    It sets the input tensor to infer.

    • Parameters:

      • idx: number

        The input tensor index. If idx is greater than the number of model inputs, an exception is thrown.

      • Tensor

        The input tensor. The element type and shape of the tensor must match the input element type and size of the model.

    • Returns: void

    • Defined in: addon.ts:540

setOutputTensor

  • setOutputTensor(tensor): void
    

    It sets the output tensor to infer models with a single output.

    • Parameters:

      • Tensor

        The output tensor. The element type and shape of the tensor must match the output element type and size of the model. If the model has several outputs, an exception is thrown.

    • Returns: void

    • Defined in: addon.ts:547

    setOutputTensor(idx, tensor): void
    

    It sets the output tensor to infer.

    • Parameters:

      • idx: number

        The output tensor index.

      • Tensor

        The output tensor. The element type and shape of the tensor must match the output element type and size of the model.

    • Returns: void

    • Defined in: addon.ts:554

setTensor

  • setTensor(name, tensor): void
    

    It sets the input/output tensor to infer.

    • Parameters:

      • name: string

        The input or output tensor name.

      • tensor: Tensor

        The tensor. The element type and shape of the tensor must match the input/output element type and size of the model.

    • Returns: void

    • Defined in: addon.ts:561