Class ov::op::v17::GroupedMatMul

class GroupedMatMul : public ov::op::Op

Grouped Matrix Multiplication operation for Mixture of Experts (MoE).

Computes multiple matrix multiplications where each group processes a subset of the input data. This operation supports two input combinations:

• Case 1 (2D × 3D): MoE forward pass

  • mat_a: (total_tokens, K) - rows partitioned by offsets

  • mat_b: (G, N, K) - per-group weights (stored transposed)

  • output: (total_tokens, N) - each group’s output in corresponding rows

• Case 2 (3D × 3D): Batched uniform (no offsets needed)

  • mat_a: (G, M, K) - per-group inputs

  • mat_b: (G, N, K) - per-group weights (stored transposed)

  • output: (G, M, N) - per-group outputs

Subclassed by ov::op::internal::GroupedMatMulCompressed

Public Functions

GroupedMatMul(const Output<Node> &mat_a, const Output<Node> &mat_b)

Constructs a GroupedMatMul operation without offsets (3D × 3D case).

Parameters:

• mat_a – First input tensor (G, M, K)

• mat_b – Second input tensor (G, N, K)

GroupedMatMul(const Output<Node> &mat_a, const Output<Node> &mat_b, const Output<Node> &offsets)

Constructs a GroupedMatMul operation with offsets (2D × 3D).

Parameters:

• mat_a – First input tensor

• mat_b – Second input tensor

• offsets – Cumulative offsets tensor of shape (G,) indicating group boundaries. For 2D×3D: partitions rows of mat_a.

virtual void validate_and_infer_types() override

Verifies that attributes and inputs are consistent and computes output shapes and element types. Must be implemented by concrete child classes so that it can be run any number of times.

Throws if the node is invalid.