🎤 LLaMA2 K-Pop Q&A Model (Fine-tuned)
This model is a fine-tuned version of meta-llama/Llama-2-7b-hf using PEFT (LoRA) on a custom 20-example K-Pop Q&A dataset.
✨ Overview
- Task: Text generation (K-Pop Question Answering)
- Dataset: Manually written 20 K-Pop themed examples
- Training Setup: LoRA-based fine-tuning on Google Colab Free Tier (15 GB GPU)
- Use Case: Lightweight model for educational/demo use related to K-Pop fan Q&A
📚 Dataset Description
Each example is structured as: Example:
Question: Who is the leader of BTS?
Context: BTS is a popular South Korean boy band formed in 2013.
Answer: RM is the leader of BTS.
🧪 Training Details
| Parameter | Value |
|---|---|
| Base model | LLaMA2 7B HF |
| Finetuning method | PEFT (LoRA) |
| Epochs | 1 |
| Batch Size | 1 |
| Max Length | 512 tokens |
| Optimizer | AdamW |
| Learning Rate | 2e-4 |
| GPU Used | Free Google Colab |
| Layers Updated | Only 1 (rest frozen) |
| Quantization | 4-bit (bitsandbytes) |
🧾 How to Use
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
model = AutoModelForCausalLM.from_pretrained("Tammy7777777/kpop-llama2-finetuned", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Tammy7777777/kpop-llama2-finetuned")
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
prompt = "Question: Who is the leader of BTS?\nContext: BTS is a South Korean boy band formed in 2013.\nAnswer:"
print(pipe(prompt, max_new_tokens=64)[0]['generated_text'])
📈 Evaluation
Base model output: Often generic or unrelated
Fine-tuned model: More aligned, provides specific K-Pop answers
Method: Manual comparison of predictions before & after fine-tuning
📦 Model Architecture
Based on meta-llama/Llama-2-7b-hf
Only one transformer layer fine-tuned using LoRA
Efficient for few-shot adaptation
⚠️ Limitations
Trained on a very small (20-example) dataset
May hallucinate or overfit on known examples
Not suitable for production use or factually sensitive topics
👤Author
Tamanna Sheikh (@Tammy7777777)
📝 License
This model is released under the MIT license for research and educational use.