Convert a Hugging Face Model to Cerebras Model Zoo

This guide uses the BertForPretraining class as an example. You’ll download the checkpoint and config files, convert the checkpoint, verify the output, and then use the newly converted model.

​

Download Checkpoint and Config Files

First, you need to download the model’s checkpoint and configuration files from Hugging Face.

Here’s how you can do it for BertForPreTraining:

from transformers import BertForPreTraining

# Replace 'bert-base-uncased' with the model you are interested in
model = BertForPreTraining.from_pretrained("bert-base-uncased")
model.save_pretrained("bert_checkpoint")

This code will save two files in the bert_checkpoint directory:

• config.json: The model’s configuration file.

• pytorch_model.bin: The model’s weights.

​

Convert Checkpoint to Cerebras Format

Now that you have the necessary files, you can convert them to a format compatible with the Cerebras Model Zoo. Use the provided conversion script in the Cerebras Model Zoo toolkit. Here’s the command:

# Navigate to the directory containing the convert_checkpoint.py script
cd <modelzoo path>/modelzoo/common/model_utils/

# Run the conversion
python convert_checkpoint.py convert \
   --model bert \  # Specify the model type
   --src-fmt hf \  # Source format (Hugging Face)
   --tgt-fmt cs-2.0 \  # Target format (Cerebras)
   bert_checkpoint/pytorch_model.bin \  # Input checkpoint file
   --config bert_checkpoint/config.json  # Configuration file

Replace <modelzoo path> with the actual path to the Model Zoo directory on your system.

​

Verify the Output

After running the conversion script, check the bert_checkpoint directory for the output files:

• pytorch_model_to_cs-2.0.mdl: The converted model checkpoint.

• config_to_cs-2.0.yaml: The converted configuration file.

​

Use the Converted Model

With the converted files, you can now use the Cerebras Model Zoo tools and workflows to further train or deploy the model on Cerebras hardware.