Single-Model Mode

Learn about the structure of a Model Repository before running the Docker image.

Launch Model Server by running the following command:

docker run -d --rm -v <models_repository>:/models -p 9000:9000 -p 9001:9001 openvino/model_server:latest \
--model_path <path_to_model> --model_name <model_name> --port 9000 --rest_port 9001 --log_level DEBUG

Configuration Arguments for Running Model Server:

–rm

remove the container when exiting the Docker container

-d

runs the container in the background

-v

defines how to mount the model folder in the Docker container

-p

exposes the model serving port outside the Docker container

openvino/model_server:latest

represents the image name; the ovms binary is the Docker entry point
varies by tag and build process - see tags: https://hub.docker.com/r/openvino/model_server/tags/ for a full tag list.

–model_path

model location, which can be:
a Docker container path that is mounted during start-up
a Google Cloud Storage path gs://<bucket>/<model_path>
an AWS S3 path s3://<bucket>/<model_path>
an Azure blob path az://<container>/<model_path>

–model_name

the name of the model in the model_path

–port

the gRPC server port

–rest_port

the REST server port

Notes

  • Publish the container’s port to your host’s open ports.

  • In the command above, port 9000 is exposed for gRPC and port 9001 is exposed for REST API calls.

  • For preparing and saving models to serve with OpenVINO Model Server refer to the Model Repository article.

  • Add model_name for the client gRPC/REST API calls.