Rank

In machine learning, ranking is the process of ordering a set of items based on their relevance to a specific query.
It is often used in information retrieval, recommendation systems, and search engines to provide a personalized and optimized user experience.
The goal of ranking is to present the most relevant items to the user, based on their preferences and behavior.

Send Requests via Python Client

The rank method of the Model object takes a text or image data as query and a list of candidates as input and returns a list of reordered candidates as well as their scores as output.
You can also wrap the text or image and candidates using DocArray.

Plain Input

When using plain input with the rank method, the candidates can be passed as text or image data.
The text or image reference should be passed as a single data entry, while the candidates can be passed as a list of data entries. Similar to the encode method, each image data can be a path to an image, bytes of an image, or an array that represents an image.

from inference_client import Client

client = Client(token='<your access token>')
model = client.get_model('<model of your selection>')

candidates = [
    'an image about dogs',
    'an image about cats',
    'an image about birds',
]

# Rank against image
image = 'path/to/image.jpg'
result = model.rank(image=image, candidates=candidates)

# Rank against text
text = 'a sentence describing the beautiful nature'
result = model.rank(text=text, candidates=candidates)

[('an image about cats', {'clip_score_cosine': {'value': 0.3,'op_name': 'cosine',},'clip_score': {'value': 0.6, 'op_name': 'softmax'},},
('an image about dogs', {'clip_score_cosine': {'value': 0.2,'op_name': 'cosine',},'clip_score': {'value': 0.4, 'op_name': 'softmax'},},
('an image about birds', {'clip_score_cosine': {'value': 0.1,'op_name': 'cosine',},'clip_score': {'value': 0.2, 'op_name': 'softmax'},}]

The result will be a list of tuples, where each tuple contains the candidate and its score.

📘
We try our best to guess the type of the input data, but there are certain cases where the image path or url cannot be distinguished from a text string and the input will be treated as text.
To make sure the input is treated as an image, please use DocumentArray input instead.

DocumentArray Input

The rank method also supports DocumentArray inputs.
DocArray is a library for representing, sending and storing multi-model data, which is perfect for Machine Learning applications.

When using DocumentArray input with the rank method, the reference and candidates can be passed as DocumentArray objects or lists of Document objects. First construct a cross-modal Document where the root contains an image and .matches contain sentences to rerank. You can construct a text-to-image rerank as below:

from inference_client import Client
from jina import DocumentArray, Document

client = Client(token='<your access token>')
model = client.get_model('<model of your selection>')

doc = Document(
    text='a photo of conference room',
    matches=[
        Document(uri='path/to/image1.jpg'),
        Document(uri='path/to/image2.jpg'),
        Document(uri='path/to/image3.jpg'),
    ],
)

# A list of Document objects
result = model.rank(docs=[doc])

# A DocumentArray object
result = model.rank(docs=DocumentArray([doc]))

The result will be a DocumentArray object with the same structure as the input, but with the .matches reordered and scored. You can access the matches and their scores using the matches and scores attributes of the Document object like below:

for match in result[0].matches:
    print(match.uri, match.scores)

path/to/image2.jpg {'clip_score_cosine': {'value': 0.3,'op_name': 'cosine',},'clip_score': {'value': 0.6, 'op_name': 'softmax'},}
path/to/image1.jpg {'clip_score_cosine': {'value': 0.2,'op_name': 'cosine',},'clip_score': {'value': 0.4, 'op_name': 'softmax'},}
path/to/image3.jpg {'clip_score_cosine': {'value': 0.1,'op_name': 'cosine',},'clip_score': {'value': 0.2, 'op_name': 'softmax'},}

You can refer to the DocArray documentation to learn more about how to construct a text Document or an image Document.
You can also learn more about how to construct a Nested Structure Document.

Sending Requests via JavaScript Client

To perform ranking using the JavaScript client, you need to provide a single reference text or image that will be used for ranking, along with a list of text_candidates or image_candidates to be reordered based on their similarity to the reference. The image can be represented using path, bytes, or array.

import Client from 'inference-client';

const client = new Client('<your auth token>');
const model = await client.getModel'<model of your selection>');

const result1 = await model.rank({ 
  	text: 'hello world', 
  	text_candidates: ['hello Jina', 'hello Ziniu'] 
});
const result2 = await model.rank({
  	text: 'green field and blue sky',
	  image_candidates: ['https://picsum.photos/id/254/200', 'https://picsum.photos/id/255/200'],
});
const result3 = await model.rank({
  	image: 'https://picsum.photos/id/254/200',
	  text_candidates: ['green field and blue sky', 'black trees'],
});
const result4 = await model.rank({
  	image: 'https://picsum.photos/id/251/200',
	  image_candidates: ['https://picsum.photos/id/254/200', 'https://picsum.photos/id/255/200'],
});

The returned result will be the reordered list of text or image candidates based on their similarity to the reference.

Plain HTTP Requests via cURL

In addition to using the Inference Client package for ranking tasks, you can also send requests directly using cURL command-line tool. This approach can be useful if you prefer working with command-line tools or need to integrate the encoding process into scripts or automation workflows.

First you need to copy the HTTP address of the Inference API you created at Jina AI Cloud. You can find the endpoint in the detail page of the Inference API.

To rank, just send a POST request to the /rank endpoint with your access token and the data in the request body. Notice the /post endpoint at the end of the address.

curl \
  -X POST https://<your-inference-address>-http.wolf.jina.ai/post \
  -H 'Content-Type: application/json' \
  -H 'Authorization: <your access token>' \
  -d '{"data": [
      {"uri": "https://picsum.photos/200",
      "matches": [{"text": "an image about dogs"},
      {"text": "an image about cats"},
      {"text": "an image about birds"}]}],
      "exec_endpoint": "/rank"}'

The response will be a JSON object containing the candidates of the input data in the order of scores similar to the following. Some fields are omitted for brevity.

{
  "header": {
    "requestId": "2387690283814ad5a03fa20a6e0e4955",
    "execEndpoint": "/rank"
  },
  "parameters": {},
  "routes": [],
  "data": [
    {
      "id": "b7757ff1de2c348e909885b97abc3210",
      "uri": "https://picsum.photos/200",
      "embedding": [
        0.12070998549461365,
        0.08413087576627731,
        ...
      ],
      "matches": [
        {
          "id": "e5f157d9550357b18f00d1410083131e",
          "mime_type": "text/plain",
          "text": "an image about dogs",
          "scores": {
            "clip_score": {
              "value": 0.4190603792667389,
              "op_name": "softmax",
            },
            "clip_score_cosine": {
              "value": 0.22395944595336914,
              "op_name": "cosine",
            }
          },
        },
        {
          "id": "76a6357ec0ee4e4eb599cd959ec0ab53",
          "mime_type": "text/plain",
          "text": "an image about birds",
          "scores": {
            "clip_score": {
              "value": 0.3944833278656006,
              "op_name": "softmax",
            },
            "clip_score_cosine": {
              "value": 0.22335505485534668,
              "op_name": "cosine",
            }
          },
        },
        {
          "id": "535487cdb4eae8fd1a74c6d5a71b88c7",
          "mime_type": "text/plain",
          "text": "an image about cats",
          "scores": {
            "clip_score": {
              "value": 0.18645627796649933,
              "op_name": "softmax",
            },
            "clip_score_cosine": {
              "value": 0.2158612608909607,
              "op_name": "cosine",
            }
          }
        }
      ]
    }
  ]
}