Write a Client Application

OpenVINO™ Model Server supports multiple APIs, for easy integration with systems using one of them for inference. The APIs are:

• one compatible with TensorFlow Serving,

• KServe API for inference

• OpenAI API for text generation.

• OpenAI API for embeddings

• Cohere API for reranking

Both TFS and KServe APIs work on gRPC and REST interfaces. The REST API endpoints for generative use cases support both streamed and unary responses.

Check the following articles to learn more about the supported APIs:

• TensorFlow Serving gRPC API

• KServe gRPC API

• TensorFlow Serving REST API

• KServe REST API

• OpenAI chat completions API

• OpenAI completions API

• OpenAI embeddings API

• Cohere rerank API

In this section you can find short code samples to interact with OpenVINO Model Server endpoints via:

• TensorFlow Serving API

• KServe API

• Generative AI clients