Class ov::pass::ConvertPagedAttnInputs#
-
class ConvertPagedAttnInputs : public ov::pass::MatcherPass#
Set precision and shape of KV cache in PagedAttn op based runtime options.
-
struct KVCacheConfig#
-
struct KVCacheConfig#
Site Navigation
Section Navigation