Single-Model Mode

Learn about the structure of a Model Repository before running the Docker image.

Launch Model Server by running the following command:

docker run -d --rm -v <models_repository>:/models -p 9000:9000 -p 9001:9001 openvino/model_server:latest \
--model_path <path_to_model> --model_name <model_name> --port 9000 --rest_port 9001 --log_level DEBUG


mkdir -p models/resnet/1
wget -P models/resnet/1
wget -P models/resnet/1

docker run -d --rm -v ${PWD}/models:/models -p 9000:9000 -p 9001:9001 openvino/model_server:latest \
--model_path /models/resnet/ --model_name resnet --port 9000 --rest_port 9001 --log_level DEBUG

Configuration Arguments for Running Model Server:


remove the container when exiting the Docker container


runs the container in the background


defines how to mount the model folder in the Docker container


exposes the model serving port outside the Docker container


represents the image name; the ovms binary is the Docker entry point
varies by tag and build process - see tags: for a full tag list.


model location, which can be:
a Docker container path that is mounted during start-up
a Google Cloud Storage path gs://<bucket>/<model_path>
an AWS S3 path s3://<bucket>/<model_path>
an Azure blob path az://<container>/<model_path>


the name of the model in the model_path


the gRPC server port


the REST server port


  • Publish the container’s port to your host’s open ports.

  • In the command above, port 9000 is exposed for gRPC and port 9001 is exposed for REST API calls.

  • For preparing and saving models to serve with OpenVINO Model Server refer to the Model Repository article.

  • Add model_name for the client gRPC/REST API calls.