openvino.opset17.grouped_matmul#

openvino.opset17.grouped_matmul(mat_a: Node | int | float | ndarray, mat_b: Node | int | float | ndarray, offsets: Node | int | float | ndarray | None = None, name: str | None = None) → Node#

Return a node which performs Grouped Matrix Multiplication for Mixture of Experts (MoE).

Computes multiple matrix multiplications where each group processes a subset of the input data. Two input combinations are supported:

  • Case 1 (2D x 3D), MoE forward pass, requires offsets:

    • mat_a: (total_tokens, K) - rows partitioned by offsets

    • mat_b: (G, N, K) - per-group weights

    • output: (total_tokens, N)

  • Case 2 (3D x 3D), batched uniform, no offsets:

    • mat_a: (G, M, K) - per-group inputs

    • mat_b: (G, N, K) - per-group weights

    • output: (G, M, N)

Parameters:

  • mat_a – The first input tensor.

  • mat_b – The second input tensor with per-group weights.

  • offsets – 1D tensor of cumulative end-offsets of shape (G,) indicating group boundaries. Required for the 2D x 3D case.

  • name – The optional name for the new output node.

Returns:

The new node performing GroupedMatMul operation.