Upload 16 files

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+source.spm filter=lfs diff=lfs merge=lfs -text
+target.spm filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,268 @@
----
-license: mit
----
+---
+language:
+- syr
+tags:
+- translation
+- syriac
+- vocalization
+- diacritics
+- biblical
+- eastern-syriac
+- mossul
+- seq2seq
+- marianmt
+license: apache-2.0
+datasets:
+- custom
+metrics:
+- bleu
+- chrf
+- accuracy
+base_model: Helsinki-NLP/opus-mt-tc-bible-big-sem-en
+model-index:
+- name: johnlockejrr/marianmt_syr_voc_eastern
+  results:
+  - task:
+      type: text-to-text-generation
+      name: Eastern Syriac Vocalization
+    dataset:
+      name: Eastern Syriac Vocalization Dataset
+      type: custom
+    metrics:
+    - type: bleu
+      value: 62.41
+      name: BLEU Score
+    - type: chrf
+      value: 87.98
+      name: chrF Score
+    - type: accuracy
+      value: 58.81
+      name: Character Accuracy
+---
+
+# MarianMT Eastern Syriac Vocalization Model
+
+A fine-tuned MarianMT model for automatic Eastern Syriac (Mossul dialect) vocalization, converting consonantal (unvocalized) Syriac text to fully vocalized text with diacritical marks.
+
+## Model Description
+
+This model is fine-tuned from [`Helsinki-NLP/opus-mt-tc-bible-big-sem-en`](https://huggingface.co/Helsinki-NLP/opus-mt-tc-bible-big-sem-en) to perform Eastern Syriac vocalization—the task of adding diacritical marks (vowels) to consonantal Syriac text. The model is specifically trained on **Eastern Syriac** texts, including the Mossul dialect, and is optimized for the Eastern Syriac vocalization system.
+
+### Key Features
+
+- **Single-direction model**: Converts consonantal Syriac (`>>syr_cons<<`) to vocalized Eastern Syriac (`>>syr_voc<<`)
+- **Eastern Syriac optimized**: Trained specifically on Eastern Syriac texts (Mossul dialect) and Digital Syriac Corpus texts vocalized in Eastern Syriac
+- **High performance**: Achieves 62.41 BLEU, 87.98 chrF, and 58.81% character accuracy on test set
+- **Biblical and corpus text optimized**: Trained on Eastern Syriac Bible texts (Mossul) and Digital Syriac Corpus texts
+
+## Model Details
+
+### Model Information
+
+- **Architecture**: MarianMT (Transformer-based sequence-to-sequence)
+- **Base Model**: `Helsinki-NLP/opus-mt-tc-bible-big-sem-en`
+- **Parameters**: 240,944,128 (~241M)
+- **Vocabulary Size**: 61,025 tokens
+- **Language Tags**: 
+  - Source: `>>syr_cons<<` (consonantal Syriac)
+  - Target: `>>syr_voc<<` (vocalized Eastern Syriac)
+
+### Training Data
+
+- **Training Examples**: 26,633
+- **Validation Examples**: 3,097
+- **Test Examples**: 1,239
+- **Total**: 30,969 sentence pairs
+- **Source**: 
+  - Eastern Syriac Bible texts (Mossul dialect)
+  - Digital Syriac Corpus texts vocalized in Eastern Syriac
+- **Format**: Consonantal and vocalized Eastern Syriac pairs
+
+### Training Configuration
+
+- **Batch Size**: 4
+- **Effective Batch Size**: 16 (with gradient accumulation)
+- **Learning Rate**: 1e-5
+- **Max Input/Target Length**: 512 tokens
+- **Training Steps**: 66,000 (early stopping)
+- **Epochs**: 39.64
+- **Optimizer**: AdamW with cosine learning rate schedule
+- **Precision**: bfloat16
+- **Early Stopping**: Based on validation metrics
+- **Training Time**: ~2 days, 13 hours, 51 minutes
+
+### Performance
+
+#### Best Validation Metrics (Epoch 36.04)
+- **BLEU**: 63.04
+- **chrF**: 88.19
+- **Character Accuracy**: 57.84%
+- **Validation Loss**: 0.0769
+
+#### Final Test Metrics
+- **BLEU**: **62.41**
+- **chrF**: **87.98**
+- **Character Accuracy**: **58.81%**
+- **Test Loss**: 0.0782
+
+## Usage
+
+### Direct Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
+
+tokenizer = AutoTokenizer.from_pretrained("johnlockejrr/marianmt_syr_voc_eastern")
+model = AutoModelForSeq2SeqLM.from_pretrained("johnlockejrr/marianmt_syr_voc_eastern")
+
+# Input: consonantal Syriac text
+text = "ܒܪܫܝܬ ܐܝܬܘܗܝ ܗܘܐ ܡܠܬܐ"
+
+# Add language tag
+input_text = f">>syr_cons<< {text}"
+
+# Tokenize
+inputs = tokenizer(input_text, return_tensors="pt", padding=True, truncation=True, max_length=512)
+
+# Generate
+outputs = model.generate(**inputs, max_length=512, num_beams=4, length_penalty=0.6)
+
+# Decode
+vocalized = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(vocalized)
+```
+
+### Using the Pipeline
+
+```python
+from transformers import pipeline
+
+vocalizer = pipeline("text2text-generation", 
+                     model="johnlockejrr/marianmt_syr_voc_eastern",
+                     tokenizer="johnlockejrr/marianmt_syr_voc_eastern")
+
+# Input text (consonantal)
+text = "ܒܪܫܝܬ ܐܝܬܘܗܝ ܗܘܐ ܡܠܬܐ"
+input_text = f">>syr_cons<< {text}"
+
+# Vocalize
+result = vocalizer(input_text, max_length=512, num_beams=4, length_penalty=0.6)
+print(result[0]['generated_text'])
+```
+
+### Text Normalization
+
+The model expects input text to be normalized to NFC (Normalization Form Composed) Unicode format. The model automatically handles this, but for best results, ensure your input text is properly normalized:
+
+```python
+import unicodedata
+
+def normalize_text(text: str) -> str:
+    """Normalize text to NFC format."""
+    return unicodedata.normalize("NFC", text)
+
+# Normalize input before processing
+text = normalize_text("ܒܪܫܝܬ ܐܝܬܘܗܝ")
+```
+
+### Input Cleaning
+
+For optimal results, input text should contain only consonantal Syriac characters. The model is designed to work with raw consonantal text, but it can handle text with some punctuation. For best performance, remove vocalization marks from input text if present.
+
+## Generation Parameters
+
+Recommended generation parameters:
+
+- **num_beams**: 4 (beam search for better quality)
+- **length_penalty**: 0.6 (encourages longer outputs)
+- **early_stopping**: True
+- **max_length**: 512 (matches training configuration)
+- **do_sample**: False (deterministic generation)
+
+## Limitations and Bias
+
+- **Dialect Specificity**: This model is trained specifically on Eastern Syriac (Mossul dialect). Performance may vary on Western Syriac or other Syriac dialects.
+- **Domain Specificity**: This model is trained primarily on biblical and corpus Syriac texts. Performance may vary on other domains (e.g., modern Syriac, poetry, prose).
+- **Single Direction**: The model only vocalizes consonantal text. It does not perform the reverse operation (removing vocalization).
+- **Length Constraints**: Maximum input/output length is 512 tokens. Longer texts should be split into smaller segments.
+- **Character Accuracy**: While BLEU and chrF scores are high, character-level accuracy is ~59%, meaning some diacritical marks may be missing or incorrect in complex cases.
+
+## Training Procedure
+
+### Training Infrastructure
+
+- **Hardware**: GPU (CUDA)
+- **Training Time**: ~2 days, 13 hours, 51 minutes
+- **Framework**: Hugging Face Transformers
+- **Evaluation Frequency**: Every 1,000 steps
+
+### Preprocessing
+
+- Text normalized to NFC Unicode format
+- Language tags (`>>syr_cons<<` and `>>syr_voc<<`) added to tokenizer vocabulary
+- Tokenization using SentencePiece (inherited from base model)
+
+### Hyperparameters
+
+```json
+{
+  "learning_rate": 1e-5,
+  "batch_size": 4,
+  "gradient_accumulation_steps": 4,
+  "num_epochs": 100,
+  "max_input_length": 512,
+  "max_target_length": 512,
+  "warmup_steps": 1000,
+  "weight_decay": 0.01,
+  "eval_steps": 1000,
+  "save_steps": 1000,
+  "save_total_limit": 3
+}
+```
+
+## Evaluation
+
+The model is evaluated using three metrics:
+
+1. **BLEU Score**: Measures n-gram precision between generated and reference text
+2. **chrF Score**: Character-level F-score, more lenient than BLEU
+3. **Character Accuracy**: Exact character match percentage
+
+### Evaluation Results
+
+| Metric | Validation (Best) | Test (Final) |
+|--------|-------------------|-------------|
+| BLEU   | 63.04             | 62.41       |
+| chrF   | 88.19             | 87.98       |
+| Char Acc | 57.84%          | 58.81%      |
+| Loss   | 0.0769            | 0.0782      |
+
+## Citation
+
+If you use this model, please cite:
+
+```bibtex
+@misc{marianmt_syr_voc_eastern,
+  title={MarianMT Eastern Syriac Vocalization Model},
+  author={johnlockejrr},
+  year={2025},
+  howpublished={\url{https://huggingface.co/johnlockejrr/marianmt_syr_voc_eastern}},
+  note={Fine-tuned from Helsinki-NLP/opus-mt-tc-bible-big-sem-en. Trained on Eastern Syriac Bible texts (Mossul) and Digital Syriac Corpus texts.}
+}
+```
+
+## Acknowledgments
+
+- **Base Model**: [Helsinki-NLP/opus-mt-tc-bible-big-sem-en](https://huggingface.co/Helsinki-NLP/opus-mt-tc-bible-big-sem-en) by the Helsinki NLP team
+- **Framework**: [Hugging Face Transformers](https://github.com/huggingface/transformers)
+- **Training Framework**: MarianMT architecture
+- **Training Data**: Eastern Syriac Bible texts (Mossul dialect) and Digital Syriac Corpus texts
+
+## Model Card Contact
+
+For questions, issues, or contributions, please open an issue on the model repository.
+
+## License
+
+This model is released under the Apache 2.0 license, consistent with the base model.
+

added_tokens.json

@@ -0,0 +1,4 @@
+{
+  ">>syr_cons<<": 61023,
+  ">>syr_voc<<": 61024
+}

all_results.json

@@ -0,0 +1,15 @@
+{
+    "epoch": 39.63973569605046,
+    "eval_bleu": 62.40825505874667,
+    "eval_char_accuracy": 58.8069928121235,
+    "eval_chrf": 87.98365764199096,
+    "eval_loss": 0.07816814631223679,
+    "eval_runtime": 1054.196,
+    "eval_samples_per_second": 1.175,
+    "eval_steps_per_second": 0.294,
+    "total_flos": 6.129316686672691e+16,
+    "train_loss": 0.06632431999797171,
+    "train_runtime": 222681.6466,
+    "train_samples_per_second": 11.96,
+    "train_steps_per_second": 0.748
+}

config.json

@@ -0,0 +1,40 @@
+{
+  "activation_dropout": 0.0,
+  "activation_function": "relu",
+  "architectures": [
+    "MarianMTModel"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 0,
+  "classifier_dropout": 0.0,
+  "d_model": 1024,
+  "decoder_attention_heads": 16,
+  "decoder_ffn_dim": 4096,
+  "decoder_layerdrop": 0.0,
+  "decoder_layers": 6,
+  "decoder_start_token_id": 61022,
+  "decoder_vocab_size": 61025,
+  "dropout": 0.1,
+  "encoder_attention_heads": 16,
+  "encoder_ffn_dim": 4096,
+  "encoder_layerdrop": 0.0,
+  "encoder_layers": 6,
+  "eos_token_id": 928,
+  "forced_eos_token_id": null,
+  "init_std": 0.02,
+  "is_encoder_decoder": true,
+  "max_length": null,
+  "max_position_embeddings": 1024,
+  "model_type": "marian",
+  "normalize_embedding": false,
+  "num_beams": null,
+  "num_hidden_layers": 6,
+  "pad_token_id": 61022,
+  "scale_embedding": true,
+  "share_encoder_decoder_embeddings": true,
+  "static_position_embeddings": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.52.4",
+  "use_cache": true,
+  "vocab_size": 61025
+}

generation_config.json

@@ -0,0 +1,15 @@
+{
+  "bad_words_ids": [
+    [
+      61022
+    ]
+  ],
+  "bos_token_id": 0,
+  "decoder_start_token_id": 61022,
+  "eos_token_id": 928,
+  "forced_eos_token_id": 928,
+  "max_length": 512,
+  "num_beams": 4,
+  "pad_token_id": 61022,
+  "transformers_version": "4.52.4"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d81fdd0b92fda214802a146298b7890f77deaad2d16570bf65635b392518e836
+size 955661620

model_info.json

@@ -0,0 +1,27 @@
+{
+  "model_name": "Helsinki-NLP/opus-mt-tc-bible-big-sem-en",
+  "direction": "syr_cons→syr_voc (East Syriac)",
+  "vocabulary_size": 61025,
+  "model_parameters": 240944128,
+  "training_config": {
+    "dataset_path": "datasets/syriac_vocalization_dataset_east",
+    "output_dir": "./syr_voc_model_east",
+    "model_name": "Helsinki-NLP/opus-mt-tc-bible-big-sem-en",
+    "batch_size": 4,
+    "learning_rate": 1e-05,
+    "num_epochs": 100,
+    "max_input_length": 512,
+    "max_target_length": 512,
+    "eval_steps": 1000,
+    "save_steps": 1000,
+    "save_total_limit": 3,
+    "use_bf16": true,
+    "skip_evaluation": false,
+    "seed": 42,
+    "resume_from_checkpoint": null,
+    "weight_decay": 0.01,
+    "logging_steps": 100,
+    "warmup_steps": 1000,
+    "gradient_accumulation_steps": 4
+  }
+}

source.spm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4027f4026e47c9ade0b5dde7d4ffc2f662b0ca832c062d43d7cef521fb6cea71
+size 848358

special_tokens_map.json

@@ -0,0 +1,21 @@
+{
+  "additional_special_tokens": [
+    {
+      "content": ">>syr_cons<<",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": ">>syr_voc<<",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    }
+  ],
+  "eos_token": "</s>",
+  "pad_token": "<pad>",
+  "unk_token": "<unk>"
+}

target.spm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4fd6a59a41075b1f42d514de1da61a3b419c61f95342b5dd0dbff73f37317086
+size 804382

test_results.json

@@ -0,0 +1,10 @@
+{
+    "epoch": 39.63973569605046,
+    "eval_bleu": 62.40825505874667,
+    "eval_char_accuracy": 58.8069928121235,
+    "eval_chrf": 87.98365764199096,
+    "eval_loss": 0.07816814631223679,
+    "eval_runtime": 1054.196,
+    "eval_samples_per_second": 1.175,
+    "eval_steps_per_second": 0.294
+}

tokenizer_config.json

@@ -0,0 +1,59 @@
+{
+  "added_tokens_decoder": {
+    "928": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1186": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "61022": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "61023": {
+      "content": ">>syr_cons<<",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "61024": {
+      "content": ">>syr_voc<<",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    ">>syr_cons<<",
+    ">>syr_voc<<"
+  ],
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "extra_special_tokens": {},
+  "model_max_length": 512,
+  "pad_token": "<pad>",
+  "separate_vocabs": false,
+  "source_lang": "sem",
+  "sp_model_kwargs": {},
+  "target_lang": "en",
+  "tokenizer_class": "MarianTokenizer",
+  "unk_token": "<unk>"
+}

train_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 39.63973569605046,
+    "total_flos": 6.129316686672691e+16,
+    "train_loss": 0.06632431999797171,
+    "train_runtime": 222681.6466,
+    "train_samples_per_second": 11.96,
+    "train_steps_per_second": 0.748
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b01016a389e29fee8f2865cb4c0566bb9c1e50225de41c3564b94f0e8aadf521
+size 5841