Download HuggingFace Model

listing down the steps involved..

As a first step, lets get started by creating a virtual environment

python3 -m venv myenv
source /bin/active/myenv

After activating the virtual environment myenvwe need to get started with the packages installation. Below are the packages to be installed.

pip3 install torch tensorflow flax sentencepiece

the above packages if skipped will not be possible to download the model from transformers library. Hence we need to install pytorch, tensforflow, flax and also sentencepiece..!

Open up a jupyter notebook

jupyter notebook

This will open up a new tab, create a new .ipynb notebook and then will be able to import some transformer packages, for eg.,

from transformers import AutoModelForSequenceClassification,AutoTokenizer
import warnings, logging
warnings.simplefilter('ignore')
logging.disable(logging.WARNING)

After successful import of the transformers, we can actually specify the model which we wanted to download.

model_nm = 'microsoft/deberta-v3-small'
model = AutoModelForSequenceClassification.from_pretrained(model_nm, return_dict=True)
tokenizer = AutoTokenizer.from_pretrained(model_nm)

We can provide the absolute path to save the model

save_path = '/Users/<name>/<model-downloads>/podeberta_v3_small_pretrained_model_pytorch'

!mkdir {save_path}

We can save both the model and tokenizer in the same path

model.save_pretrained(save_path)
tokenizer.save_pretrained(save_path)

If we list down the files it saved -

!ls {save_path}

added_tokens.json       special_tokens_map.json tokenizer_config.json
config.json             spm.model
model.safetensors       tokenizer.json

Thats how simple to download a hugging face model and save local to disk.