Implement any CPU layer, that is not support by OpenVINO yet, as a shared library.
Leverage the OpenVINO model caching feature to speed up subsequent model loading on a target device.
Custom Model Loader¶
Write your own custom model loader based on a predefined interface and load it similar to a dynamic library.
Securing Model Server with NGINX¶
Protect network endpoints with traffic encription and client authorization using a reverse proxy