Install OpenVINO™ GenAI#
OpenVINO GenAI is a new flavor of OpenVINO, aiming to simplify running inference of generative AI models. It hides the complexity of the generation process and minimizes the amount of code required. You can now provide a model and input context directly to OpenVINO, which performs tokenization of the input text, executes the generation loop on the selected device, and returns the generated text. For a quickstart guide, refer to the GenAI API Guide.
To see GenAI in action, check the Jupyter notebooks: LLM-powered Chatbot and LLM Instruction-following pipeline.
The OpenVINO GenAI flavor is available for installation via PyPI and Archive distributions. A detailed guide on how to build OpenVINO GenAI is available in the OpenVINO GenAI repository.
PyPI Installation#
To install the GenAI flavor of OpenVINO via PyPI, follow the standard installation steps, but use the openvino-genai package instead of openvino:
python -m pip install openvino-genai
Archive Installation#
The OpenVINO GenAI archive package includes the OpenVINO™ Runtime and Tokenizers. To install the GenAI flavor of OpenVINO from an archive file, follow the standard installation steps for your system but instead of using the vanilla package file, download the one with OpenVINO GenAI:
Linux#
curl -L https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/2024.5/linux/openvino_genai_ubuntu24_2024.5.0.0_x86_64.tar.gz --output openvino_genai_2024.5.0.0.tgz
tar -xf openvino_genai_2024.5.0.0.tgz
curl -L https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/2024.5/linux/openvino_genai_ubuntu22_2024.5.0.0_x86_64.tar.gz --output openvino_genai_2024.5.0.0.tgz
tar -xf openvino_genai_2024.5.0.0.tgz
curl -L https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/2024.5/linux/openvino_genai_ubuntu20_2024.5.0.0_x86_64.tar.gz --output openvino_genai_2024.5.0.0.tgz
tar -xf openvino_genai_2024.5.0.0.tgz
curl -L https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/2024.5/linux/openvino_genai_ubuntu20_2024.5.0.0_arm64.tar.gz -O openvino_genai_2024.5.0.0.tgz
tar -xf openvino_genai_2024.5.0.0.tgz
Windows#
cd <user_home>/Downloads
curl -L https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/2024.5/windows/openvino_genai_windows_2024.5.0.0_x86_64.zip --output openvino_genai_2024.5.0.0.zip
macOS#
curl -L https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/2024.5/macos/openvino_genai_macos_12_6_2024.5.0.0_x86_64.tar.gz --output openvino_genai_2024.5.0.0.tgz
tar -xf openvino_genai_2024.5.0.0.tgz
curl -L https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/2024.5/macos/openvino_genai_macos_12_6_2024.5.0.0_arm64.tar.gz --output openvino_genai_2024.5.0.0.tgz
tar -xf openvino_genai_2024.5.0.0.tgz