Class ov::ICacheManager#

class ICacheManager#

This class represents private interface for Cache Manager.

Subclassed by ov::runtime::SingleFileStorage

Public Types

using StreamWriter = std::function<void(std::ostream&)>#

Function passing created output stream.

using CompiledBlobVariant = std::variant<const ov::Tensor, std::reference_wrapper<std::istream>>#

Variant type for compiled blob representation.

using StreamReader = std::function<void(CompiledBlobVariant&)>#

Function passing created input stream.

Public Functions

virtual ~ICacheManager() = default#

Default destructor.

virtual void write_cache_entry(const std::string &id, StreamWriter writer) = 0#

Callback when OpenVINO intends to write model to cache.

Client needs to call create std::ostream object and call writer(ostream) Otherwise, model will not be cached

Parameters:

  • id – Id of cache (hash of the model)

  • writer – Lambda function to be called when stream is created

virtual void read_cache_entry(const std::string &id, bool enable_mmap, StreamReader reader) = 0#

Callback when OpenVINO intends to read model from cache.

Client needs to call create std::istream object and call reader(istream) Otherwise, model will not be read from cache and will be loaded as usual

Parameters:

  • id – Id of cache (hash of the model)

  • enable_mmap – use mmap or ifstream to read model file

  • reader – Lambda function to be called when input stream is created

virtual void remove_cache_entry(const std::string &id) = 0#

Callback when OpenVINO intends to remove cache entry.

Client needs to perform appropriate cleanup (e.g. delete a cache file)

Parameters:

id – Id of cache (hash of the model)