Dynamic Input Parameters¶
Models served by the OpenVINO Model Server can be configured to accept data with different batch sizes and in different shapes. There are multiple ways of enabling dynamic inputs for the model:
dynamic batch size with a demuliplexer - create a simple pipeline that splits data of any batch size and performs inference on each element in the batch separately.
dynamic batch size with automatic model reloading - configure the Model Server to reload the model each time it receives a request with a batch size other than what is currently set.
dynamic shape with automatic model reloading - configure the Model Server to reload the model each time the model receives a request with data in shape other than what is currently set.
dynamic input shape with a custom node - create a simple pipeline by pairing your model with a custom node that performs data preprocessing and provides the model with data in an acceptable shape.
dynamic input shape with binary input format - send data in binary format (JPEG or PNG encoded), so the Model Server will adjust the input during data decoding.