Blog #2: Finding and Using Pretrained Models from HuggingFace

Date: Oct 19, 2024

Tag: Python, Machine Learning

Finding a Decent Pre-trained Model from HuggingFace

HuggingFace is currently the best place to find open-source pre-trained machine learning models. It's sort of like a popular code repository for machine learning models.

We can use HuggingFace to tune the pre-trained model and deploy it on serverless services or use their inference API for prototyping. However, our goal is to run it locally.

My Requirements

• Task: Translation
• Languages: Multiple languages
• License: Apache 2.0, MIT
• Decent Average Score: BLEU (Bilingual Evaluation Understudy) - the higher, the better.

After going through the models' documentation, I chose: Helsinki-NLP/opus-mt-mul-en.

There are filters on the model pages. Make sure to filter based on your requirement and remember to check the option for the correct license suitable for you.

Running a Machine Learning Model Locally

We will use Python to run the model. Keep in mind that I am using WSL2 with Ubuntu 22 LTS and Python 3.10

Local Python Environment

Before we start, it is recommended to create a virtual env for our project. Why ? Because , the more Python projects we have, the more likely it is that we need to work with different versions of Python libraries, or even Python itself. Different versions of libraries for one project can break compatibility in another project. I tend to use venv within the project folder and use pip freeze to check the project's packages. Alternatively, Conda is another option.

Find out how to create, activate, and deactivate a virtual environment here.

Deep Learning Library

We have many DLL today.Popular ones include PyTorch and TensorFlow. HuggingFace initially supported PyTorch but at present supports TensorFlow as well. (We will use TensorFlow, as I want to explore TensorFlow Lite later for its better Android support)

We will be using a CPU only version of TensorFlow (I have an older NVIDIA GTX card which is not officially supported).

Installing TensorFlow

Installation details for TensorFlow can be found here.

Note: This will install the latest TensorFlow onto the system and this will cause some or the other backward compatibility issue with HuggingFace Transformer. Check out the ‘Installing HugginFace Transformers’ section to install a Transformer with compactable TensorFlow.

Note: Avoid using Conda for TensorFlow as it may not have the latest stable version.

For CPU users:

pip install --upgrade pip
pip install tensorflow

Verifying CPU-only Installation of TensorFlow

python3 -c "import tensorflow as tf; print(tf.reduce_sum(tf.random.normal([1000, 1000])))"

If a tensor is returned, the installation is successful.

Installing HuggingFace Transformers

• HuggingFace Transformers is a library that simplifies the process of building and training language models or including them into custom applications.
• Note: If you install and validate HuggingFace transformer, you will find that it is built upon pytorch as the underlying Machine Learning framework.
• You can find the official installation guide here.

For CPU-support only, we can conveniently install HugginFace Transformers and TensorFlow 2.0 using the below command (This is my recommendation to install TensorFlow and HuggingFace Transformer on local which require CPU only support of TensorFlow):

pip install 'transformers[tf-cpu]'

Finally, check if HuggingFace Transformers has been properly installed by running the following command. (It will download a pretrained model and run it)

python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('we love you'))"

Running the Helsinki-NLP/Opus-mt-mul-en Model Locally

Note: You can find the code used in this blog here.

We can copy the template for the HuggingFace Pipeline which simplifies using models for inference. According to HuggingFace documentation, pipelines abstract complex code, providing a simple API for several tasks. Even if we don't have experience with a specific modality or aren't familiar with the underlying code behind the models, you can still use them for inference with the pipeline().

Note: Remember that the code provided will most likely be set to use PyTorch.

Coding it up as a transformer

Code :

Output :

Note: Details on TFAutoModel can be followed up here.

Note: Details on AutoTokenizer can be followed up here. The tokenizer returns a dictionary with three important items:

• input_ids are the indices corresponding to each token in the sentence.
• attention_mask indicates whether a token should be attended to or not.
• token_type_ids identifies which sequence a token belongs to when there is more than one sequence.

Improving Translation Accuracy ?

Finetuning the model is beyond the scope of this blog. But if we prepend the language to the input text as below we will get a more accurate result.

However, based on the MarianMT Model documentation, we don’t need to prepend the source language in multilingual mode, leading to possible confusion or a bug ? Unfortunately I was not able to find an answer for this.

Coding it up using HuggingFace Pipeline

Code :

(I have prepend the language to the input text to improve the translation.)

Output :

In conclusion , using the pipeline is easier for inference than the traditional method, although the latter offers more tweaking flexibility.