|
|
--- |
|
|
license: apache-2.0 |
|
|
base_model: |
|
|
- mistralai/Mistral-7B-Instruct-v0.2 |
|
|
language: |
|
|
- en |
|
|
pipeline_tag: text-generation |
|
|
library_name: transformers |
|
|
--- |
|
|
# Mistral-7B-Instruct-v0.2-8bit-abliterated-layer18 |
|
|
|
|
|
This model was abliterated by computing a refusal vector an 8-bit bitsandbytes quant, and then applying the vector to the full weight model. |
|
|
Abliteration was performed locally using a CUDA GPU, the VRAM memory consumption appeared to be constrained to be under 12GB. |
|
|
|
|
|
Layer 18 was selected for derivation of the refusal direction, as measurements of the refusal direction magnitude, signal-to-noise ratio, and angle between the means of the "harmful" and "harmless" directions suggested that intervention based on this layer would be relatively efficient and effective. |
|
|
|
|
|
No additional fine-tuning was performed on these weights. Repair is required for proper use. |
|
|
|
|
|
The code used can be found on Github at [https://github.com/jim-plus/llm-abliteration](https://github.com/jim-plus/llm-abliteration). |
|
|
|
|
|
(My prior attempt relied on default values within the codebase, which turned out to be less effective than this intervention.) |
