marianmt_syr_voc_eastern / README.md

johnlockejrr

Update README.md

535725a verified 5 days ago

preview code

raw

history blame contribute delete

8.68 kB

metadata

language:
  - syr
tags:
  - translation
  - syriac
  - vocalization
  - diacritics
  - biblical
  - eastern-syriac
  - mossul
  - seq2seq
  - marianmt
license: apache-2.0
datasets:
  - custom
metrics:
  - bleu
  - chrf
  - accuracy
base_model: Helsinki-NLP/opus-mt-tc-bible-big-sem-en
model-index:
  - name: johnlockejrr/marianmt_syr_voc_eastern
    results:
      - task:
          type: text-to-text-generation
          name: Eastern Syriac Vocalization
        dataset:
          name: Eastern Syriac Vocalization Dataset
          type: custom
        metrics:
          - type: bleu
            value: 62.41
            name: BLEU Score
          - type: chrf
            value: 87.98
            name: chrF Score
          - type: accuracy
            value: 58.81
            name: Character Accuracy

MarianMT Eastern Syriac Vocalization Model

A fine-tuned MarianMT model for automatic Eastern Syriac (Mossul Bible) vocalization, converting consonantal (unvocalized) Syriac text to fully vocalized text with diacritical marks.

Model Description

This model is fine-tuned from Helsinki-NLP/opus-mt-tc-bible-big-sem-en to perform Eastern Syriac vocalization—the task of adding diacritical marks (vowels) to consonantal Syriac text. The model is specifically trained on Eastern Syriac texts, and is optimized for the Eastern Syriac vocalization system.

Key Features

Single-direction model: Converts consonantal Syriac (>>syr_cons<<) to vocalized Eastern Syriac (>>syr_voc<<)
Eastern Syriac optimized: Trained specifically on Eastern Syriac texts (Mossul edition) and Digital Syriac Corpus texts vocalized in Eastern Syriac
High performance: Achieves 62.41 BLEU, 87.98 chrF, and 58.81% character accuracy on test set
Biblical and corpus text optimized: Trained on Eastern Syriac Bible texts (Mossul edition) and Digital Syriac Corpus texts

Model Details

Model Information

Architecture: MarianMT (Transformer-based sequence-to-sequence)
Base Model: Helsinki-NLP/opus-mt-tc-bible-big-sem-en
Parameters: 240,944,128 (~241M)
Vocabulary Size: 61,025 tokens
Language Tags:
- Source: >>syr_cons<< (consonantal Syriac)
- Target: >>syr_voc<< (vocalized Eastern Syriac)

Training Data

Training Examples: 26,633
Validation Examples: 3,097
Test Examples: 1,239
Total: 30,969 sentence pairs
Source:
- Eastern Syriac Bible texts (Mossul edition)
- Digital Syriac Corpus texts vocalized in Eastern Syriac
Format: Consonantal and vocalized Eastern Syriac pairs

Training Configuration

Batch Size: 4
Effective Batch Size: 16 (with gradient accumulation)
Learning Rate: 1e-5
Max Input/Target Length: 512 tokens
Training Steps: 66,000 (early stopping)
Epochs: 39.64
Optimizer: AdamW with cosine learning rate schedule
Precision: bfloat16
Early Stopping: Based on validation metrics
Training Time: ~2 days, 13 hours, 51 minutes

Performance

Best Validation Metrics (Epoch 36.04)

BLEU: 63.04
chrF: 88.19
Character Accuracy: 57.84%
Validation Loss: 0.0769

Final Test Metrics

BLEU: 62.41
chrF: 87.98
Character Accuracy: 58.81%
Test Loss: 0.0782

Usage

Direct Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("johnlockejrr/marianmt_syr_voc_eastern")
model = AutoModelForSeq2SeqLM.from_pretrained("johnlockejrr/marianmt_syr_voc_eastern")

# Input: consonantal Syriac text
text = "ܒܪܫܝܬ ܐܝܬܘܗܝ ܗܘܐ ܡܠܬܐ"

# Add language tag
input_text = f">>syr_cons<< {text}"

# Tokenize
inputs = tokenizer(input_text, return_tensors="pt", padding=True, truncation=True, max_length=512)

# Generate
outputs = model.generate(**inputs, max_length=512, num_beams=4, length_penalty=0.6)

# Decode
vocalized = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(vocalized)

Using the Pipeline

from transformers import pipeline

vocalizer = pipeline("text2text-generation", 
                     model="johnlockejrr/marianmt_syr_voc_eastern",
                     tokenizer="johnlockejrr/marianmt_syr_voc_eastern")

# Input text (consonantal)
text = "ܒܪܫܝܬ ܐܝܬܘܗܝ ܗܘܐ ܡܠܬܐ"
input_text = f">>syr_cons<< {text}"

# Vocalize
result = vocalizer(input_text, max_length=512, num_beams=4, length_penalty=0.6)
print(result[0]['generated_text'])

Text Normalization

The model expects input text to be normalized to NFC (Normalization Form Composed) Unicode format. The model automatically handles this, but for best results, ensure your input text is properly normalized:

import unicodedata

def normalize_text(text: str) -> str:
    """Normalize text to NFC format."""
    return unicodedata.normalize("NFC", text)

# Normalize input before processing
text = normalize_text("ܒܪܫܝܬ ܐܝܬܘܗܝ")

Input Cleaning

For optimal results, input text should contain only consonantal Syriac characters. The model is designed to work with raw consonantal text, but it can handle text with some punctuation. For best performance, remove vocalization marks from input text if present.

Generation Parameters

Recommended generation parameters:

num_beams: 4 (beam search for better quality)
length_penalty: 0.6 (encourages longer outputs)
early_stopping: True
max_length: 512 (matches training configuration)
do_sample: False (deterministic generation)

Limitations and Bias

Dialect Specificity: This model is trained specifically on Eastern Syriac (Mossul edition). Performance may vary on Western Syriac or other Syriac dialects.
Domain Specificity: This model is trained primarily on biblical and corpus Syriac texts. Performance may vary on other domains (e.g., modern Syriac, poetry, prose).
Single Direction: The model only vocalizes consonantal text. It does not perform the reverse operation (removing vocalization).
Length Constraints: Maximum input/output length is 512 tokens. Longer texts should be split into smaller segments.
Character Accuracy: While BLEU and chrF scores are high, character-level accuracy is ~59%, meaning some diacritical marks may be missing or incorrect in complex cases.

Training Procedure

Training Infrastructure

Hardware: GPU (CUDA)
Training Time: ~2 days, 13 hours, 51 minutes
Framework: Hugging Face Transformers
Evaluation Frequency: Every 1,000 steps

Preprocessing

Text normalized to NFC Unicode format
Language tags (>>syr_cons<< and >>syr_voc<<) added to tokenizer vocabulary
Tokenization using SentencePiece (inherited from base model)

Hyperparameters

{
  "learning_rate": 1e-5,
  "batch_size": 4,
  "gradient_accumulation_steps": 4,
  "num_epochs": 100,
  "max_input_length": 512,
  "max_target_length": 512,
  "warmup_steps": 1000,
  "weight_decay": 0.01,
  "eval_steps": 1000,
  "save_steps": 1000,
  "save_total_limit": 3
}

Evaluation

The model is evaluated using three metrics:

BLEU Score: Measures n-gram precision between generated and reference text
chrF Score: Character-level F-score, more lenient than BLEU
Character Accuracy: Exact character match percentage

Evaluation Results

Metric	Validation (Best)	Test (Final)
BLEU	63.04	62.41
chrF	88.19	87.98
Char Acc	57.84%	58.81%
Loss	0.0769	0.0782

Citation

If you use this model, please cite:

@misc{marianmt_syr_voc_eastern,
  title={MarianMT Eastern Syriac Vocalization Model},
  author={johnlockejrr},
  year={2025},
  howpublished={\url{https://huggingface.co/johnlockejrr/marianmt_syr_voc_eastern}},
  note={Fine-tuned from Helsinki-NLP/opus-mt-tc-bible-big-sem-en. Trained on Eastern Syriac Bible texts (Mossul) and Digital Syriac Corpus texts.}
}

Acknowledgments

Base Model: Helsinki-NLP/opus-mt-tc-bible-big-sem-en by the Helsinki NLP team
Framework: Hugging Face Transformers
Training Framework: MarianMT architecture
Training Data: Eastern Syriac Bible texts (Mossul edition) and Digital Syriac Corpus texts

Model Card Contact

For questions, issues, or contributions, please open an issue on the model repository.

License

This model is released under the Apache 2.0 license, consistent with the base model.