reedmayhew

/

Llama-3.1-8B-claude-3.7-sonnet-reasoning-distilled

like 4

Hello!

Do you mind clarifying what you mean by "source"? If you mean the dataset that was used to train the models, it's a custom dataset which I've uploaded to my profile here:
https://huggingface.co/datasets/reedmayhew/claude-3.7-sonnet-reasoning

If you mean the code that was used to train the models, I used the Unsloth Colab notebooks. I made several adjustments in order to get them to train properly, so I've made a duplicate copy to share with you:

https://colab.research.google.com/drive/1zypzS3uDS3_zWLjR0iVT1UkvnUp6-_6L?usp=sharing

Make sure at the end to input your HF token and username/desired model name.

If you want different size models, just change model_name = "unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit" with another META model, such as Llama 3.2 3B or 1B. Other architectures require different Unsloth notebooks.

Hope that helps!

Hey ;

Source is the full precision version of the model, in bfloat16, float16 or float32 in "safetensors" format.
If memory serves, I think you can set unsloth to create the "full source" version, VS a quant like Q8_0.
This step is after the "quant" section in the notebook - again , memory a bit fuzzy here.

In your Colab:

Saving, loading fine-tuned models
To save the final model as LoRA adapters, either use Huggingface's push_to_hub for an online save or save_pretrained for a local save.
[NOTE] This ONLY saves the LoRA adapters, and not the full model. To save to 16bit or GGUF, scroll down!

"Section: Save 16bit" would produce the "Source Files".

Another option:

"Save the LORA to HF" -> This can be used as an adapter, which is then merged with a model which can also create "source" files.
This method is not as strong as generating the source files directly; but can generate source files.

HMM.; viewing the colab, it appears if the 16 bit (full source) option is missing, unless they mean "16 bit GGUF" ("f16" option vs "Q8_0").
Might want to contact Unsloth about this, as the source version in safetensors would be the best option.

"Source" version can then be used to create GGUF (and Imatrix GGUFs), EXL2, AWQ and other quants as well as full merges using Mergekit.
Hope this helps ;

Thanks for the explination! I'm still new to fine-tuning and unfamiliar with some of the terms.

I don't believe I saved the source files, unfortunately, since I was running this through Colab quickly to generate the GGUF files for my own personal use. I didn't expect people to have an interest in them! I'll definitely upload the source files moving forward.

I'll add back the source cell to the Colab, as I should be able to grab it from the original Unsloth notebook.