Write a Client Application

OpenVINO™ Model Server supports multiple APIs, for easy integration with systems using one of them for inference. The APIs are:

• KServe API for inference

• OpenAI API for text generation.

• OpenAI API for embeddings

• Cohere API for reranking

KServe API works on both gRPC and REST interfaces. The REST API endpoints for generative use cases support both streamed and unary responses.

Check the following articles to learn more about the supported APIs:

• KServe gRPC API

• KServe REST API

• OpenAI chat completions API

• OpenAI completions API

• OpenAI embeddings API

• Cohere rerank API

• OpenAI images generations API

• OpenAI speech recognition API

• OpenAI speech generation API

In this section you can find short code samples to interact with OpenVINO Model Server endpoints via:

• KServe API

• Generative AI clients