abhinavsunil/lexai-gemma2b-lora

LexAI - Gemma-2B (LoRA Fine-Tuned for Indian Legal Instruction)

This is a LoRA fine-tuned version of Google's Gemma-2B model on a custom dataset of Indian legal instructions, writ petitions, bail applications, and more.

The model is part of the LexAI initiative — a lightweight legal language model assistant that can be integrated with voice systems like Whisper ASR and Coqui TTS.

---

🧠 Model Details

• Base Model: google/gemma-2b
• LoRA Config:
  • r=8
  • alpha=16
  • dropout=0.05
  • Target Modules: ["q_proj", "v_proj"]
• Quantization: 8-bit (via bitsandbytes)
• Fine-Tuned Using: Hugging Face PEFT + Transformers
• Training Epochs: 3
• Batch Size: 2 (accumulated over 4 steps)
• Device: Google Colab T4 (12GB VRAM)

---

📚 Training Data

Custom JSONL dataset with 1,000+ legal examples formatted like:

### Instruction:
What is a bail application?

### Input:

### Response:
A bail application is a request made by a defendant seeking release from custody while the case is pending...