Machine translation demo¶
This tutorial is also available as a Jupyter notebook that can be cloned directly from GitHub. See the installation guide for instructions to run this tutorial locally on Windows, Linux or macOS. To run without installing anything, click the launch binder button.
This demo utilizes Intel’s pre-trained model that translates from English to German. More information about the model can be found here.
This model encodes sentences using the SentecePieceBPETokenizer from HuggingFace. The tokenizer vocabulary is downloaded automatically with the OMZ tool.
Inputs¶
The model’s input is a sequence of up to 150 tokens with the following
structure: <s> + tokenized sentence + <s> + <pad>
(<pad> tokens pad the remaining blank spaces).
Ouput¶
After the inference, we have a sequence of up to 200 tokens. The structure is the same as the one for the input.
import time
from openvino.runtime import Core
import numpy as np
import itertools
from tokenizers import SentencePieceBPETokenizer
Downloading model¶
The following command will download the model to the current directory.
Please, make sure you have run pip install openvino-dev beforehand.
! omz_downloader --name machine-translation-nar-en-de-0002
################|| Downloading machine-translation-nar-en-de-0002 ||################
========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-275/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/tokenizer_tgt/merges.txt
========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-275/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/tokenizer_tgt/vocab.json
========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-275/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/tokenizer_src/merges.txt
========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-275/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/tokenizer_src/vocab.json
========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-275/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/FP32/machine-translation-nar-en-de-0002.xml
========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-275/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/FP32/machine-translation-nar-en-de-0002.bin
========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-275/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/FP16/machine-translation-nar-en-de-0002.xml
========== Downloading /opt/home/k8sworker/cibuilds/ov-notebook/OVNotebookOps-275/.workspace/scm/ov-notebook/notebooks/221-machine-translation/intel/machine-translation-nar-en-de-0002/FP16/machine-translation-nar-en-de-0002.bin
Load and configure the model¶
The model is now available in the intel/ folder. Below we load and
configure its inputs and outputs.
core = Core()
model = core.read_model('intel/machine-translation-nar-en-de-0002/FP32/machine-translation-nar-en-de-0002.xml')
compiled_model = core.compile_model(model)
input_name = "tokens"
output_name = "pred"
model.output(output_name)
max_tokens = model.input(input_name).shape[1]
Load tokenizers¶
NLP models usually take a list of tokens as standard input. A token is a single word converted to some integer. To provide the proper input, we need the vocabulary for such mapping. We use ‘merges.txt’ to inform us what sequences of letters form a token. ‘vocab.json’ specifies the mapping between tokens and integers.
The input needs to be transformed into a token sequence the model understands, and the ouput must be transformed into a sentence that is human readable.
Initialize the tokenizer for the input src_tokenizer and the output
tgt_tokenizer.
src_tokenizer = SentencePieceBPETokenizer.from_file(
'intel/machine-translation-nar-en-de-0002/tokenizer_src/vocab.json',
'intel/machine-translation-nar-en-de-0002/tokenizer_src/merges.txt'
)
tgt_tokenizer = SentencePieceBPETokenizer.from_file(
'intel/machine-translation-nar-en-de-0002/tokenizer_tgt/vocab.json',
'intel/machine-translation-nar-en-de-0002/tokenizer_tgt/merges.txt'
)
Perform translation¶
The following function translates a sentence in English to German.
def translate(sentence: str) -> str:
"""
Tokenize the sentence using the downloaded tokenizer and run the model,
whose output is decoded into a human readable string.
:param sentence: a string containing the phrase to be translated
:return: the translated string
"""
# Remove leading and trailing white spaces
sentence = sentence.strip()
assert len(sentence) > 0
tokens = src_tokenizer.encode(sentence).ids
# Transform the tokenized sentence into the model's input format
tokens = [src_tokenizer.token_to_id('<s>')] + \
tokens + [src_tokenizer.token_to_id('</s>')]
pad_length = max_tokens - len(tokens)
# If the sentence size is less than the maximum allowed tokens,
# fill the remaining tokens with '<pad>'.
if pad_length > 0:
tokens = tokens + [src_tokenizer.token_to_id('<pad>')] * pad_length
assert len(tokens) == max_tokens, "input sentence is too long"
encoded_sentence = np.array(tokens).reshape(1, -1)
# Perform inference
enc_translated = compiled_model({input_name: encoded_sentence})
output_key = compiled_model.output(output_name)
enc_translated = enc_translated[output_key][0]
# Decode the sentence
sentence = tgt_tokenizer.decode(enc_translated)
# Remove <pad> tokens, as well as '<s>' and '</s>' tokens which mark the
# beginning and ending of the sentence.
for s in ['</s>', '<s>', '<pad>']:
sentence = sentence.replace(s, '')
# Transform sentence into lower case and join words by a white space
sentence = sentence.lower().split()
sentence = " ".join(key for key, _ in itertools.groupby(sentence))
return sentence
Translate the sentence¶
The following function is a basic loop that translates sentences.
def run_translator():
"""
Run the translation in real time, reading the input from the user.
This function prints the translated sentence and the time
spent during inference.
:return:
"""
while True:
input_sentence = input()
if input_sentence == "":
break
start_time = time.perf_counter()
translated = translate(input_sentence)
end_time = time.perf_counter()
print(f'Translated: {translated}')
print(f'Time: {end_time - start_time:.2f}s')
# uncomment the following line for a real time translation of your input
# run_translator()
Test your translation¶
Run the following cell with an English sentence to have it translated to German
sentence = "My name is openvino"
print(f'Translated: {translate(sentence)}')
Translated: mein name ist openvino.