FineData

AI & ML interests

We release large pre-training datasets to accelerate open LLM development. Part of the Hugging Face Science team (hf.co/science)

Recent Activity

craffel  authored a paper about 9 hours ago

TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior

eliebak  submitted a paper 7 days ago

SonicMoE: Accelerating MoE with IO and Tile-aware Optimizations

hynky  new activity 11 days ago

HuggingFaceFW/finepdfs:Which language detector did you use

View all activity

Papers

FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

View all Papers

Team members 18