Thinking models and finetune healing
Are there plans to release the thinking counter part of this model too?
Also, there have been some research into the benefits of doing a brief round of fine-tuning post-abliteration to help the model heal any loss that resulted in the lost paths, which often yields even better performance than the original model had since now it doesn't have the previous restrictions but has full access to all its remaining paths. Might be worth considering doing that with one of the highly curated public datasets available here to help your models stand above the rest.
It is very difficult to ablate models with thinking-class patterns using traditional ablation methods; perhaps we will try fine-tuning to achieve ablation.
@huihui-ai
, thanks for all these abliterated models!
It looks like you were able to abliterate 32b-thinking model. Would it be possible to abliterate 30b-a3b-thinking as well?
I'd really appreciate it. Thanks!