openvino_genai.OmniPipeline

class openvino_genai.OmniPipeline

Bases: pybind11_object

OmniPipeline — Qwen3-Omni text + speech pipeline.

Composes a VLM pipeline (text generation with hidden-state collection) with a Qwen3-Omni speech pipeline (Talker + CodePredictor + Code2Wav). Each generate call takes two configs: a GenerationConfig text_config (thinker) and an OmniTalkerSpeechConfig talker_speech_config (talker + speech). Speech generation is gated per-call by talker_speech_config.return_audio.

Two construction paths:

• Path-based: OmniPipeline(models_path, device, **properties) loads VLM and speech models from a single directory.

• DI: OmniPipeline(vlm_pipeline, talker) reuses an externally-loaded VLMPipeline and a TalkerBase subclass for independent device choices or custom backends.

Both ctors enforce that the loaded model is Qwen3-Omni capable (model_type == QWEN3_OMNI and enable_audio_output) — non-Omni models throw at construction time.

__init__(*args, **kwargs)

Overloaded function.

1. __init__(self: openvino_genai.py_openvino_genai.OmniPipeline, models_path: os.PathLike | str | bytes, device: str, **kwargs) -> None

   OmniPipeline path-based constructor. models_path (os.PathLike): Path to the folder with exported Qwen3-Omni model files. device (str): Device to run the model on (e.g., CPU, GPU). kwargs: Device properties.

2. __init__(self: openvino_genai.py_openvino_genai.OmniPipeline, vlm: openvino_genai.py_openvino_genai.VLMPipelineBase, talker: openvino_genai.py_openvino_genai.TalkerBase) -> None

   OmniPipeline dependency-injection constructor. Compose a pre-built VLM (thinker) and Talker (speech) so the two stages can use independent devices/properties, or so a custom TalkerBase subclass can be injected. vlm (VLMPipeline): Backing VLM pipeline. Must be a Qwen3-Omni-capable model loaded

   with the continuous-batching backend (attention_backend=PA).

   talker (TalkerBase): Backing speech generator (default impl is Talker).

Methods

__delattr__(name, /)	Implement delattr(self, name).
__dir__()	Default dir() implementation.
__eq__(value, /)	Return self==value.
__format__(format_spec, /)	Default object formatter.
__ge__(value, /)	Return self>=value.
__getattribute__(name, /)	Return getattr(self, name).
__getstate__()	Helper for pickle.
__gt__(value, /)	Return self>value.
__hash__()	Return hash(self).
__init__(*args, **kwargs)	Overloaded function.
__init_subclass__	This method is called when a class is subclassed.
__le__(value, /)	Return self<=value.
__lt__(value, /)	Return self<value.
__ne__(value, /)	Return self!=value.
__new__(**kwargs)
__reduce__()	Helper for pickle.
__reduce_ex__(protocol, /)	Helper for pickle.
__repr__()	Return repr(self).
__setattr__(name, value, /)	Implement setattr(self, name, value).
__sizeof__()	Size of object in memory, in bytes.
__str__()	Return str(self).
__subclasshook__	Abstract classes can override this to customize issubclass().
_pybind11_conduit_v1_
generate(*args, **kwargs)	Overloaded function.
get_talker(self)	Return the underlying TalkerBase.
get_vlm(self)	Return the underlying VLM (thinker) as a VLMPipelineBase.

Attributes

__annotations__

__annotations__ = {}

__class__

alias of pybind11_type

__delattr__(name, /)

Implement delattr(self, name).

__dir__()

Default dir() implementation.

__eq__(value, /)

Return self==value.

__format__(format_spec, /)

Default object formatter.

Return str(self) if format_spec is empty. Raise TypeError otherwise.

__ge__(value, /)

Return self>=value.

__getattribute__(name, /)

Return getattr(self, name).

__getstate__()

Helper for pickle.

__gt__(value, /)

Return self>value.

__hash__()

Return hash(self).

__init__(*args, **kwargs)

Overloaded function.

1. __init__(self: openvino_genai.py_openvino_genai.OmniPipeline, models_path: os.PathLike | str | bytes, device: str, **kwargs) -> None

   OmniPipeline path-based constructor. models_path (os.PathLike): Path to the folder with exported Qwen3-Omni model files. device (str): Device to run the model on (e.g., CPU, GPU). kwargs: Device properties.

2. __init__(self: openvino_genai.py_openvino_genai.OmniPipeline, vlm: openvino_genai.py_openvino_genai.VLMPipelineBase, talker: openvino_genai.py_openvino_genai.TalkerBase) -> None

   OmniPipeline dependency-injection constructor. Compose a pre-built VLM (thinker) and Talker (speech) so the two stages can use independent devices/properties, or so a custom TalkerBase subclass can be injected. vlm (VLMPipeline): Backing VLM pipeline. Must be a Qwen3-Omni-capable model loaded

   with the continuous-batching backend (attention_backend=PA).

   talker (TalkerBase): Backing speech generator (default impl is Talker).

__init_subclass__()

This method is called when a class is subclassed.

The default implementation does nothing. It may be overridden to extend subclasses.

__le__(value, /)

Return self<=value.

__lt__(value, /)

Return self<value.

__ne__(value, /)

Return self!=value.

__new__(**kwargs)

__reduce__()

Helper for pickle.

__reduce_ex__(protocol, /)

Helper for pickle.

__repr__()

Return repr(self).

__setattr__(name, value, /)

Implement setattr(self, name, value).

__sizeof__()

Size of object in memory, in bytes.

__str__()

Return str(self).

__subclasshook__()

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

_pybind11_conduit_v1_()

generate(*args, **kwargs)

Overloaded function.

1. generate(self: openvino_genai.py_openvino_genai.OmniPipeline, prompt: str, images: collections.abc.Sequence[openvino._pyopenvino.Tensor] = [], videos: collections.abc.Sequence[openvino._pyopenvino.Tensor] = [], videos_metadata: collections.abc.Sequence[openvino_genai.py_openvino_genai.VideoMetadata] = [], audios: collections.abc.Sequence[openvino._pyopenvino.Tensor] = [], text_config: openvino_genai.py_openvino_genai.GenerationConfig | None = None, talker_speech_config: openvino_genai.py_openvino_genai.OmniTalkerSpeechConfig | None = None, streamer: collections.abc.Callable[[str], int | None] | openvino_genai.py_openvino_genai.StreamerBase | None = None, speech_streamer: collections.abc.Callable[[openvino._pyopenvino.Tensor], int | None] | openvino_genai.py_openvino_genai.OmniSpeechStreamerBase | None = None) -> openvino_genai.py_openvino_genai.OmniDecodedResults

   Generate text and (optionally) speech from a flat prompt.

   param prompt:

   Input prompt

   type prompt:

   str

   param images:

   image tensors to be prepended to the prompt

   type images:

   list[ov.Tensor]

   param videos:

   video tensors to be prepended to the prompt

   type videos:

   list[ov.Tensor]

   param videos_metadata:

   metadata for each video (fps, frames_indices). Must be empty or have the same length as videos.

   type videos_metadata:

   list[VideoMetadata]

   param audios:

   audio tensors to be prepended to the prompt

   type audios:

   list[ov.Tensor]

   param text_config:

   thinker text-decode config. None = use the VLM’s default GenerationConfig loaded from generation_config.json.

   type text_config:

   GenerationConfig | None

   param talker_speech_config:

   talker + speech-output config. None = a default- constructed OmniTalkerSpeechConfig (return_audio=True, model-default speaker).

   type talker_speech_config:

   OmniTalkerSpeechConfig | None

   param streamer:

   optional streamer for text tokens.

   type streamer:

   Callable[[str], bool] | StreamerBase | None

   param speech_streamer:

   optional callback or OmniSpeechStreamerBase to receive audio chunks during speech generation. Lambda receives ov.Tensor [1, 1, N_samples] and returns StreamingStatus (or bool/None).

   type speech_streamer:

   Callable[[ov.Tensor], StreamingStatus | bool | None] | OmniSpeechStreamerBase | None

   return:

   OmniDecodedResults with speech_result.waveforms populated when talker_speech_config.return_audio is True.

   rtype:

   OmniDecodedResults

2. generate(self: openvino_genai.py_openvino_genai.OmniPipeline, history: openvino_genai.py_openvino_genai.ChatHistory, images: collections.abc.Sequence[openvino._pyopenvino.Tensor] = [], videos: collections.abc.Sequence[openvino._pyopenvino.Tensor] = [], videos_metadata: collections.abc.Sequence[openvino_genai.py_openvino_genai.VideoMetadata] = [], audios: collections.abc.Sequence[openvino._pyopenvino.Tensor] = [], text_config: openvino_genai.py_openvino_genai.GenerationConfig | None = None, talker_speech_config: openvino_genai.py_openvino_genai.OmniTalkerSpeechConfig | None = None, streamer: collections.abc.Callable[[str], int | None] | openvino_genai.py_openvino_genai.StreamerBase | None = None, speech_streamer: collections.abc.Callable[[openvino._pyopenvino.Tensor], int | None] | openvino_genai.py_openvino_genai.OmniSpeechStreamerBase | None = None) -> openvino_genai.py_openvino_genai.OmniDecodedResults

   Generate text and (optionally) speech from a chat history. Same parameter semantics as the prompt overload.

   param history:

   Chat history

   type history:

   ChatHistory

   param videos_metadata:

   metadata for each video (fps, frames_indices). Must be empty or have the same length as videos.

   type videos_metadata:

   list[VideoMetadata]

get_talker(self: openvino_genai.py_openvino_genai.OmniPipeline) → openvino_genai.py_openvino_genai.TalkerBase

Return the underlying TalkerBase. Speaker enumeration and embedding retrieval live here: pipe.get_talker().list_speakers(), pipe.get_talker().get_speaker_embedding(name).

get_vlm(self: openvino_genai.py_openvino_genai.OmniPipeline) → openvino_genai.py_openvino_genai.VLMPipelineBase

Return the underlying VLM (thinker) as a VLMPipelineBase. Useful for inspecting model metadata or reusing the same VLM across pipelines via the DI constructor.