Cohere API rerank endpoint#

API Reference#

OpenVINO Model Server includes now the rerank endpoint based on Cohere API. Please see the Cohere rerank API Reference for more information on the API. The endpoint can be accessed via a path:

http://server_name:port/v3/rerank

In the cohere client library specify the base_url=http://server_name:port/v3.

Example request#

curl http://localhost:8000/v3/rerank \
  -H "Content-Type: application/json" \
  -d '{"model": "BAAI/bge-reranker-large", \
  "query": "Hello", "documents":["Welcome","Farewell"]}'

Example response#

{
  "results": [
    {
      "index": 0,
      "relevance_score": 0.3886180520057678
    },
    {
      "index": 1,
      "relevance_score": 0.0055549247190356255
    }
  ]
}

Request#

Param

OpenVINO Model Server

Cohere

Type

Description

model

string (required)

Name of the model to use. Name assigned to a servable configured to run a reranking model.

query

string (required)

Text to compare the similarity with the documents

documents

⚠️

list of strings

Documents to rerank

top_n

integer

Limit response to n most similar

return_documents

boolean

Return the documents in the response

Unsupported params from Cohere service:#

  • max_chunks_per_doc - configurable on the server side in the graph parameters.

  • rank_fields

  • documents in a form of a dictionary. A list of strings is allowed.

Response#

Param

OpenVINO Model Server

Cohere

Type

Description

results.index

integer

index of the document

results.relevance_score

float in a range 0-1

relevance score with the query

results.document

string

Assessed document

id

integer

Response index

metadata

string

Model name

Unsupported params from OpenAI service:#

  • id

  • metadata

Error handling#

Endpoint can raise an error related to incorrect request in the following conditions:

  • incorrect format of any of the fields based on the schema

  • Amount of documents exceeds allowed configured value - default 100

  • Number of chunks needed to split any of the input documents exceed the configured value - default 10

References#

End to end demo with rerank endpoint

Code snippets