Machine translation demo

This tutorial is also available as a Jupyter notebook that can be cloned directly from GitHub. See the installation guide for instructions to run this tutorial locally on Windows, Linux or macOS. To run without installing anything, click the launch binder button.

This demo utilizes Intel’s pre-trained model that translates from English to German. More information about the model can be found here.

This model encodes sentences using the SentecePieceBPETokenizer from HuggingFace. The tokenizer vocabulary is downloaded automatically with the OMZ tool.

Inputs The model’s input is a sequence of up to 150 tokens with the following structure: <s> + tokenized sentence + <s> + <pad> (<pad> tokens pad the remaining blank spaces).

Ouput After the inference, we have a sequence of up to 200 tokens. The structure is the same as the one for the input.

!pip install -q "openvino-dev=2022.3"

ERROR: Invalid requirement: 'openvino-dev=2022.3'
Hint: = is not a valid operator. Did you mean == ?

import time
from openvino.runtime import Core
import numpy as np
import itertools
from tokenizers import SentencePieceBPETokenizer

Downloading model

The following command will download the model to the current directory. Make sure you have run pip install openvino-dev beforehand.

! omz_downloader --name  machine-translation-nar-en-de-0002

################|| Downloading machine-translation-nar-en-de-0002 ||################

========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-416/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/tokenizer_tgt/merges.txt


========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-416/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/tokenizer_tgt/vocab.json


========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-416/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/tokenizer_src/merges.txt


========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-416/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/tokenizer_src/vocab.json


========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-416/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/FP32/machine-translation-nar-en-de-0002.xml


========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-416/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/FP32/machine-translation-nar-en-de-0002.bin


========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-416/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/FP16/machine-translation-nar-en-de-0002.xml


========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-416/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/FP16/machine-translation-nar-en-de-0002.bin

Load and configure the model

The model is now available in the intel/ folder. Below, we load and configure its inputs and outputs.

core = Core()
model = core.read_model('intel/machine-translation-nar-en-de-0002/FP32/machine-translation-nar-en-de-0002.xml')
compiled_model = core.compile_model(model)
input_name = "tokens"
output_name = "pred"
model.output(output_name)
max_tokens = model.input(input_name).shape[1]

Load tokenizers

NLP models usually take a list of tokens as standard input. A token is a single word converted to some integer. To provide the proper input, we need the vocabulary for such mapping. We use ‘merges.txt’ to find out what sequences of letters form a token. ‘vocab.json’ specifies the mapping between tokens and integers.

The input needs to be transformed into a token sequence the model understands, and the output must be transformed into a sentence that is human readable.

Initialize the tokenizer for the input src_tokenizer and the output tgt_tokenizer.

src_tokenizer = SentencePieceBPETokenizer.from_file(
    'intel/machine-translation-nar-en-de-0002/tokenizer_src/vocab.json',
    'intel/machine-translation-nar-en-de-0002/tokenizer_src/merges.txt'
)
tgt_tokenizer = SentencePieceBPETokenizer.from_file(
    'intel/machine-translation-nar-en-de-0002/tokenizer_tgt/vocab.json',
    'intel/machine-translation-nar-en-de-0002/tokenizer_tgt/merges.txt'
)

Perform translation

The following function translates a sentence in English to German.

def translate(sentence: str) -> str:
    """
    Tokenize the sentence using the downloaded tokenizer and run the model,
    whose output is decoded into a human readable string.

    :param sentence: a string containing the phrase to be translated
    :return: the translated string
    """
    # Remove leading and trailing white spaces
    sentence = sentence.strip()
    assert len(sentence) > 0
    tokens = src_tokenizer.encode(sentence).ids
    # Transform the tokenized sentence into the model's input format
    tokens = [src_tokenizer.token_to_id('<s>')] + \
        tokens + [src_tokenizer.token_to_id('</s>')]
    pad_length = max_tokens - len(tokens)

    # If the sentence size is less than the maximum allowed tokens,
    # fill the remaining tokens with '<pad>'.
    if pad_length > 0:
        tokens = tokens + [src_tokenizer.token_to_id('<pad>')] * pad_length
    assert len(tokens) == max_tokens, "input sentence is too long"
    encoded_sentence = np.array(tokens).reshape(1, -1)

    # Perform inference
    enc_translated = compiled_model({input_name: encoded_sentence})
    output_key = compiled_model.output(output_name)
    enc_translated = enc_translated[output_key][0]

    # Decode the sentence
    sentence = tgt_tokenizer.decode(enc_translated)

    # Remove <pad> tokens, as well as '<s>' and '</s>' tokens which mark the
    # beginning and ending of the sentence.
    for s in ['</s>', '<s>', '<pad>']:
        sentence = sentence.replace(s, '')

    # Transform sentence into lower case and join words by a white space
    sentence = sentence.lower().split()
    sentence = " ".join(key for key, _ in itertools.groupby(sentence))
    return sentence

Translate the sentence

The following function is a basic loop that translates sentences.

def run_translator():
    """
    Run the translation in real time, reading the input from the user.
    This function prints the translated sentence and the time
    spent during inference.
    :return:
    """
    while True:
        input_sentence = input()
        if input_sentence == "":
            break

        start_time = time.perf_counter()
        translated = translate(input_sentence)
        end_time = time.perf_counter()
        print(f'Translated: {translated}')
        print(f'Time: {end_time - start_time:.2f}s')

# uncomment the following line for a real time translation of your input
# run_translator()

Test your translation

Run the following cell with an English sentence to have it translated to German

sentence = "My name is openvino"
print(f'Translated: {translate(sentence)}')

Translated: mein name ist openvino.