openvino_genai.GenerationConfig#
- class openvino_genai.GenerationConfig#
Bases:
pybind11_objectGenerationConfig parameters max_length: the maximum length the generated tokens can have. Corresponds to the length of the input prompt +
max_new_tokens. Its effect is overridden by max_new_tokens, if also set.
max_new_tokens: the maximum numbers of tokens to generate, excluding the number of tokens in the prompt. max_new_tokens has priority over max_length. ignore_eos: if set to true, then generation will not stop even if <eos> token is met. eos_token_id: token_id of <eos> (end of sentence)
Beam search specific parameters: num_beams: number of beams for beam search. 1 disables beam search. num_beam_groups: number of groups to divide num_beams into in order to ensure diversity among different groups of beams. diversity_penalty: value is subtracted from a beam’s score if it generates the same token as any beam from other group at a particular time. length_penalty: exponential penalty to the length that is used with beam-based generation. It is applied as an exponent to
the sequence length, which in turn is used to divide the score of the sequence. Since the score is the log likelihood of the sequence (i.e. negative), length_penalty > 0.0 promotes longer sequences, while length_penalty < 0.0 encourages shorter sequences.
num_return_sequences: the number of sequences to return for grouped beam search decoding. no_repeat_ngram_size: if set to int > 0, all ngrams of that size can only occur once. stop_criteria: controls the stopping condition for grouped beam search. It accepts the following values:
“openvino_genai.StopCriteria.EARLY”, where the generation stops as soon as there are num_beams complete candidates; “openvino_genai.StopCriteria.HEURISTIC” is applied and the generation stops when is it very unlikely to find better candidates; “openvino_genai.StopCriteria.NEVER”, where the beam search procedure only stops when there cannot be better candidates (canonical beam search algorithm).
Random sampling parameters: temperature: the value used to modulate token probabilities for random sampling. top_p: if set to float < 1, only the smallest set of most probable tokens with probabilities that add up to top_p or higher are kept for generation. top_k: the number of highest probability vocabulary tokens to keep for top-k-filtering. do_sample: whether or not to use multinomial random sampling that add up to top_p or higher are kept. repetition_penalty: the parameter for repetition penalty. 1.0 means no penalty.
- __init__(*args, **kwargs)#
Overloaded function.
__init__(self: openvino_genai.py_generate_pipeline.GenerationConfig, json_path: str) -> None
path where generation_config.json is stored
__init__(self: openvino_genai.py_generate_pipeline.GenerationConfig, **kwargs) -> None
Methods
__delattr__(name, /)Implement delattr(self, name).
__dir__()Default dir() implementation.
__eq__(value, /)Return self==value.
__format__(format_spec, /)Default object formatter.
__ge__(value, /)Return self>=value.
__getattribute__(name, /)Return getattr(self, name).
__gt__(value, /)Return self>value.
__hash__()Return hash(self).
__init__(*args, **kwargs)Overloaded function.
This method is called when a class is subclassed.
__le__(value, /)Return self<=value.
__lt__(value, /)Return self<value.
__ne__(value, /)Return self!=value.
__new__(**kwargs)Helper for pickle.
__reduce_ex__(protocol, /)Helper for pickle.
__repr__()Return repr(self).
__setattr__(name, value, /)Implement setattr(self, name, value).
Size of object in memory, in bytes.
__str__()Return str(self).
Abstract classes can override this to customize issubclass().
is_beam_search(self)set_eos_token_id(self, arg0)Attributes
- __class__#
alias of
pybind11_type
- __delattr__(name, /)#
Implement delattr(self, name).
- __dir__()#
Default dir() implementation.
- __eq__(value, /)#
Return self==value.
- __format__(format_spec, /)#
Default object formatter.
- __ge__(value, /)#
Return self>=value.
- __getattribute__(name, /)#
Return getattr(self, name).
- __gt__(value, /)#
Return self>value.
- __hash__()#
Return hash(self).
- __init__(*args, **kwargs)#
Overloaded function.
__init__(self: openvino_genai.py_generate_pipeline.GenerationConfig, json_path: str) -> None
path where generation_config.json is stored
__init__(self: openvino_genai.py_generate_pipeline.GenerationConfig, **kwargs) -> None
- __init_subclass__()#
This method is called when a class is subclassed.
The default implementation does nothing. It may be overridden to extend subclasses.
- __le__(value, /)#
Return self<=value.
- __lt__(value, /)#
Return self<value.
- __ne__(value, /)#
Return self!=value.
- __new__(**kwargs)#
- __reduce__()#
Helper for pickle.
- __reduce_ex__(protocol, /)#
Helper for pickle.
- __repr__()#
Return repr(self).
- __setattr__(name, value, /)#
Implement setattr(self, name, value).
- __sizeof__()#
Size of object in memory, in bytes.
- __str__()#
Return str(self).
- __subclasshook__()#
Abstract classes can override this to customize issubclass().
This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).
- property diversity_penalty#
- property do_sample#
- property eos_token_id#
- property frequency_penalty#
- property ignore_eos#
- is_beam_search(self: openvino_genai.py_generate_pipeline.GenerationConfig) bool#
- property length_penalty#
- property max_length#
- property max_new_tokens#
- property min_new_tokens#
- property no_repeat_ngram_size#
- property num_beam_groups#
- property num_beams#
- property num_return_sequences#
- property presence_penalty#
- property repetition_penalty#
- property rng_seed#
- set_eos_token_id(self: openvino_genai.py_generate_pipeline.GenerationConfig, arg0: int) None#
- property stop_criteria#
- property temperature#
- property top_k#
- property top_p#