---
tags:
- babylm
- language-model
- gpt-bert
- multilingual
license: mit
---
# babybabellm-multismall

This repository contains checkpoints for the **multilingual small (tier 1)** variant of **BabyBabeLLM**. This is a multilingual BabyLM trained on Tier 1 languages from the multilingual BabyLM corpus (Jumelet et al 2025).

## Files
- `*_15_16.bin` – main model weights  
- `*_15_16_ema.bin` – EMA smoothed weights  
- `*_15_16_state_dict.bin` – PyTorch state dict  
- `pytorch_model.bin` – extracted EMA weights (for AutoModel)  

## Usage
```python
from transformers import AutoModel, AutoTokenizer
repo = "suchirsalhan/babybabellm-multismall"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModel.from_pretrained(repo)
inputs = tokenizer("Hello world!", return_tensors="pt")
outputs = model(**inputs)
```
## Notes
- These are research checkpoints trained on BabyLM-style data.
- Model naming: `multismall` indicates the language/config variant.