---
language: en
tags:
- question-answering
- squad
- transformers
datasets:
- squad
metrics:
- exact_match
- f1
model-index:
- name: HariomSahu/distilbert-squad-qa
  results:
  - task:
      type: question-answering
      name: Question Answering
    dataset:
      name: SQuAD
      type: squad
    metrics:
    - type: exact_match
      value: N/A
    - type: f1
      value: 85.3016055407403
---

# distilbert-base-uncased fine-tuned on SQuAD

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the SQuAD dataset.

## Training Details

### Training Hyperparameters

- **Model**: distilbert-base-uncased
- **Dataset**: SQuAD
- **Optimizer**: adamw
- **Learning Rate Scheduler**: cosine_with_restarts
- **Learning Rate**: 3e-05
- **Batch Size**: 16 per device
- **Total Batch Size**: 64
- **Epochs**: 5 (with early stopping)
- **Weight Decay**: 0.01
- **Warmup Ratio**: 0.06
- **Max Gradient Norm**: 1.0

### Early Stopping

- **Patience**: 4
- **Metric**: f1
- **Best Epoch**: 3

## Usage

```python
from transformers import AutoTokenizer, AutoModelForQuestionAnswering

tokenizer = AutoTokenizer.from_pretrained("HariomSahu/distilbert-squad-qa")
model = AutoModelForQuestionAnswering.from_pretrained("HariomSahu/distilbert-squad-qa")

# Example usage
question = "What is the capital of France?"
context = "France is a country in Europe. Its capital city is Paris."

inputs = tokenizer(question, context, return_tensors="pt")
outputs = model(**inputs)

# Get answer
start_scores, end_scores = outputs.start_logits, outputs.end_logits
start_index = start_scores.argmax()
end_index = end_scores.argmax()
answer = tokenizer.decode(inputs["input_ids"][0][start_index:end_index+1])
print(f"Answer: {answer}")
```

## Evaluation Results

The model achieved the following results on the evaluation set:

- **Exact Match**: 76.9253
- **F1 Score**: 85.2786

## Training Configuration Hash

Config Hash: fe08f7bd

This hash can be used to reproduce the exact training configuration.