---
tags:
- image-captioning
- medical-imaging
- vision-language
- radiology
datasets:
- eltorio/ROCOv2-radiology
language:
- en
metrics:
- wer
base_model: microsoft/git-base
---

# Medical Image Captioning Model

This model is fine-tuned for medical image captioning using the ROCOv2 radiology dataset.

## Model Description

- **Base Model**: microsoft/git-base
- **Training Data**: ROCOv2 radiology images
- **Task**: Generate descriptive captions for medical/radiology images
- **Language**: English

## Usage

```python
from transformers import AutoProcessor, AutoModelForCausalLM
from PIL import Image

# Load model and processor
processor = AutoProcessor.from_pretrained("WafaaFraih/medical-image-captioning-roco")
model = AutoModelForCausalLM.from_pretrained("WafaaFraih/medical-image-captioning-roco")

# Load and process image
image = Image.open("path_to_medical_image.jpg")
inputs = processor(images=image, return_tensors="pt")

# Generate caption
generated_ids = model.generate(pixel_values=inputs.pixel_values, max_length=100)
caption = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(caption)
```

## Training Details

- **Training Samples**: 1800
- **Validation Samples**: 200
- **Epochs**: 10
- **Batch Size**: 8
- **Learning Rate**: 5e-05

## Evaluation

Evaluated using Word Error Rate (WER) metric on medical image descriptions.

## Limitations

- Trained on a subset of ROCOv2 dataset
- Performance may vary on different imaging modalities
- Should not be used for clinical diagnosis without expert validation

## Citation

If you use this model, please cite:

```bibtex
@misc{medical-image-captioning,
  author = {WafaaFraih},
  title = {Medical Image Captioning Model},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/WafaaFraih/medical-image-captioning-roco}
}
```

## Acknowledgments

- Base model: Microsoft GIT
- Dataset: ROCOv2 Radiology Dataset