Upload all models and assets for ang (latest)

README.md

@@ -36,7 +36,7 @@ metrics:
     value: 4.012
   - name: best_isotropy
     type: isotropy
-    value: 0.8005
+    value: 0.7896
   - name: vocabulary_size
     type: vocab
     value: 0
@@ -99,32 +99,32 @@ We analyze tokenizers, n-gram models, Markov chains, vocabulary statistics, and
 
 Below are sample sentences tokenized with each vocabulary size:
 
-**Sample 1:** `Valladolid is burg on Spēnum. Valladolid hæfþ 319,943 lēoda. on Castillan`
+**Sample 1:** `Grēat Coldūn () is þorp in þæm East Þriding, se is Eoferƿicscire dǣl, on Englum....`
 
 | Vocab | Tokens | Count |
 |-------|--------|-------|
-| 8k | `▁val lad ol id ▁is ▁burg ▁on ▁spēnum . ▁val ... (+18 more)` | 28 |
-| 16k | `▁val lad ol id ▁is ▁burg ▁on ▁spēnum . ▁val ... (+17 more)` | 27 |
-| 32k | `▁val ladol id ▁is ▁burg ▁on ▁spēnum . ▁val ladol ... (+14 more)` | 24 |
-| 64k | `▁valladolid ▁is ▁burg ▁on ▁spēnum . ▁valladolid ▁hæfþ ▁ 3 ... (+10 more)` | 20 |
+| 8k | `▁grēat ▁c old ūn ▁() ▁is ▁þorp ▁in ▁þæm ▁east ... (+15 more)` | 25 |
+| 16k | `▁grēat ▁c old ūn ▁() ▁is ▁þorp ▁in ▁þæm ▁east ... (+15 more)` | 25 |
+| 32k | `▁grēat ▁cold ūn ▁() ▁is ▁þorp ▁in ▁þæm ▁east ▁þriding ... (+14 more)` | 24 |
+| 64k | `▁grēat ▁cold ūn ▁() ▁is ▁þorp ▁in ▁þæm ▁east ▁þriding ... (+14 more)` | 24 |
 
-**Sample 2:** `Cicġan () is burg on Cile. Þǣr oneardiaþ 161,953 lēoda (þæs gēares). His mearc i...`
+**Sample 2:** `Lingua Franca Nova is gehugod sprǣc. Utweardlice bendas elefen.org gereord`
 
 | Vocab | Tokens | Count |
 |-------|--------|-------|
-| 8k | `▁c ic ġ an ▁() ▁is ▁burg ▁on ▁cile . ... (+33 more)` | 43 |
-| 16k | `▁cic ġan ▁() ▁is ▁burg ▁on ▁cile . ▁þǣr ▁oneardiaþ ... (+31 more)` | 41 |
-| 32k | `▁cic ġan ▁() ▁is ▁burg ▁on ▁cile . ▁þǣr ▁oneardiaþ ... (+30 more)` | 40 |
-| 64k | `▁cicġan ▁() ▁is ▁burg ▁on ▁cile . ▁þǣr ▁oneardiaþ ▁ ... (+28 more)` | 38 |
+| 8k | `▁l ing ua ▁franc a ▁nov a ▁is ▁geh ug ... (+11 more)` | 21 |
+| 16k | `▁l ing ua ▁franc a ▁nova ▁is ▁geh ug od ... (+10 more)` | 20 |
+| 32k | `▁ling ua ▁franca ▁nova ▁is ▁gehugod ▁sprǣc . ▁utweardlice ▁bendas ... (+5 more)` | 15 |
+| 64k | `▁lingua ▁franca ▁nova ▁is ▁gehugod ▁sprǣc . ▁utweardlice ▁bendas ▁ele ... (+4 more)` | 14 |
 
-**Sample 3:** `Welwīc () is þorp in þæm East Þriding, se is Eoferƿicscire dǣl, on Englum. Heo h...`
+**Sample 3:** `Andreas Iǣxcūn ƿæs se seofoða Foresittend þāra Geānlǣhtra Rīca, fram þǣm gēare ō...`
 
 | Vocab | Tokens | Count |
 |-------|--------|-------|
-| 8k | `▁wel wīc ▁() ▁is ▁þorp ▁in ▁þæm ▁east ▁þriding , ... (+21 more)` | 31 |
-| 16k | `▁wel wīc ▁() ▁is ▁þorp ▁in ▁þæm ▁east ▁þriding , ... (+21 more)` | 31 |
-| 32k | `▁wel wīc ▁() ▁is ▁þorp ▁in ▁þæm ▁east ▁þriding , ... (+21 more)` | 31 |
-| 64k | `▁welwīc ▁() ▁is ▁þorp ▁in ▁þæm ▁east ▁þriding , ▁se ... (+20 more)` | 30 |
+| 8k | `▁andreas ▁i ǣ x c ūn ▁ƿæs ▁se ▁seof oða ... (+17 more)` | 27 |
+| 16k | `▁andreas ▁iǣx c ūn ▁ƿæs ▁se ▁seofoða ▁foresittend ▁þāra ▁geānlǣhtra ... (+14 more)` | 24 |
+| 32k | `▁andreas ▁iǣx c ūn ▁ƿæs ▁se ▁seofoða ▁foresittend ▁þāra ▁geānlǣhtra ... (+14 more)` | 24 |
+| 64k | `▁andreas ▁iǣxcūn ▁ƿæs ▁se ▁seofoða ▁foresittend ▁þāra ▁geānlǣhtra ▁rīca , ... (+12 more)` | 22 |
 
 
 ### Key Findings
@@ -274,27 +274,27 @@ Below are text samples generated from each word-based Markov chain model:
 
 **Context Size 1:**
 
-1. `and strætham is eoferƿicscire dǣl on wyrse ġearum ac bowser jr americanisc auto maker grady thomas`
-2. `on pictocusċīre`
-3. `is burg þes geþoftede rīce and gescyldnesse kowane mutum yana da vinci chapter 93 dead rǣdinge`
+1. `and bedældede hine in þǣm geānedum rīcum þā protest sang rocc and sīþe hrēðcyninges hām to`
+2. `on francum in þæm miclum burgum and his ƿæter hit hê hê willgesweostor shes laid back`
+3. `is unesco æfter déaðe drepe þrōƿade heorosƿeng heardn ond sēo hēafodmearc iesuitisces rǣses it was f...`
 
 **Context Size 2:**
 
-1. `on þǣm ġerēfscipe þæs fōresittende sam dōnde swā þurh sūþsið fǣmneland wǣre and ǣrende genōg land fo...`
-2. `in þǣm geānlǣhtum rīcum fram þǣm sericus gārsecge hit stent is 60 mīle geddoburg be sūðƿesten on`
-3. `in þæm suþernan dæle þæs geānedan cynerīces grēatre brytene cynerīce ƿæs þēodland in ƿesternre europ...`
+1. `on þǣm fylle þǣm þe nāhwæþer ne þā ġeānedan land sculon ne ǣniġ land sceal ætfōn oþþe`
+2. `in þǣm indiscum lande uttar pradesh þæt land þæt ƿæs corēan independence activist politicians and jo...`
+3. `in þæm east þriding se is eoferƿicscire dǣl on englum hit hæfþ 11 351 būendas on eoferwicscīre`
 
 **Context Size 3:**
 
-1. `td valign top efencasere mid numeriane murdered td valign top td valign top td oþ 269 td valign`
-2. `is þorp in þæm east þriding se is eoferƿicscire dǣl on englum on eoferwicscīre þæs geānedan cynerīce...`
-3. `eoferwicscīre þæs geānedan cynerīces on wēalum þrēoscyte is hēo and hæfþ twegen ecgas onmang beorgum...`
+1. `td valign top ualentinianus ii td valign top td to 297 td valign top co emperor with honorius`
+2. `is þorp in soria on castile and leóne in spēonlande and þorpas on sorie`
+3. `eoferwicscīre þæs geānedan cynerīces and hēafodman þæs behealdenda hēapes siþðan mǣdmōnaþ he is gebē...`
 
 **Context Size 4:**
 
 1. `on eoferwicscīre þæs geānedan cynerīces`
-2. `is eoferƿicscire dǣl on englum hit hæfþ 318 buend on eoferwicscīre þæs geānedan cynerīces`
-3. `eoferƿicscire dǣl on englum on eoferwicscīre þæs geānedan cynerīces`
+2. `is eoferƿicscire dǣl on englalande on eoferwicscīre þæs geānedan cynerīces`
+3. `eoferƿicscire dǣl on englum mid grēatum hǣþfelda ġesċieppaþ hie þone burgsċipe of hǣþfelda on eoferw...`
 
 
 ### Generated Text Samples (Subword-based)
@@ -303,27 +303,27 @@ Below are text samples generated from each subword-based Markov chain model:
 
 **Context Size 1:**
 
-1. `_bor.rm_frsh_ded`
-2. `es_a_urīþofophob`
-3. `nftaner,_a_gavst`
+1. `_htofunes_anōre_`
+2. `e_c_weaþǣfyn_sca`
+3. `n_þeal_wun_berie`
 
 **Context Size 2:**
 
-1. `e_wærr_ofiret_be_`
-2. `and:_gēac_(144_75`
-3. `n_en_polan_on_þæs`
+1. `e_of_fi_94oðbe_tw`
+2. `an_thoseadand_īeg`
+3. `n_nīƿ_mesprytt,_þ`
 
 **Context Size 3:**
 
-1. `and_womericanada_r`
-2. `nd_þā_illinge._þā_`
-3. `an_60_1,000_ause_ƿ`
+1. `and_und_ofher_mā_s`
+2. `nd_titutede_him._h`
+3. `an_asscran_betwa_ǣ`
 
 **Context Size 4:**
 
-1. `and_(oððe_stre_in_n`
-2. `_and_scot_ƿæs_þēodi`
-3. `_on_mererīca,_a_cli`
+1. `and_belalan_(mother`
+2. `_and_ġecosta_tƿiste`
+3. `_on_þā_habbað_nofgo`
 
 
 ### Key Findings
@@ -428,18 +428,18 @@ Below are text samples generated from each subword-based Markov chain model:
 
 | Model | Dimension | Isotropy | Semantic Density | Alignment R@1 | Alignment R@10 |
 |-------|-----------|----------|------------------|---------------|----------------|
-| **mono_32d** | 32 | 0.8005 🏆 | 0.3579 | N/A | N/A |
-| **mono_64d** | 64 | 0.4615 | 0.3163 | N/A | N/A |
-| **mono_128d** | 128 | 0.1318 | 0.3128 | N/A | N/A |
-| **aligned_32d** | 32 | 0.8005 | 0.3430 | 0.0360 | 0.2720 |
-| **aligned_64d** | 64 | 0.4615 | 0.3200 | 0.0780 | 0.3300 |
-| **aligned_128d** | 128 | 0.1318 | 0.2993 | 0.0840 | 0.3840 |
+| **mono_32d** | 32 | 0.7896 | 0.3585 | N/A | N/A |
+| **mono_64d** | 64 | 0.4746 | 0.3175 | N/A | N/A |
+| **mono_128d** | 128 | 0.1353 | 0.3004 | N/A | N/A |
+| **aligned_32d** | 32 | 0.7896 🏆 | 0.3555 | 0.0300 | 0.2480 |
+| **aligned_64d** | 64 | 0.4746 | 0.3090 | 0.0860 | 0.3400 |
+| **aligned_128d** | 128 | 0.1353 | 0.3041 | 0.1280 | 0.4020 |
 
 ### Key Findings
 
-- **Best Isotropy:** mono_32d with 0.8005 (more uniform distribution)
-- **Semantic Density:** Average pairwise similarity of 0.3249. Lower values indicate better semantic separation.
-- **Alignment Quality:** Aligned models achieve up to 8.4% R@1 in cross-lingual retrieval.
+- **Best Isotropy:** aligned_32d with 0.7896 (more uniform distribution)
+- **Semantic Density:** Average pairwise similarity of 0.3242. Lower values indicate better semantic separation.
+- **Alignment Quality:** Aligned models achieve up to 12.8% R@1 in cross-lingual retrieval.
 - **Recommendation:** 128d aligned for best cross-lingual performance
 
 ---
@@ -461,17 +461,19 @@ These are the most productive prefixes and suffixes identified by sampling the v
 #### Productive Prefixes
 | Prefix | Examples |
 |--------|----------|
-| `-ge` | geƿinnes, geendebyrded, genered |
+| `-ge` | geondrīcisce, gebold, gemyndgung |
 
 #### Productive Suffixes
 | Suffix | Examples |
 |--------|----------|
-| `-e` | ohthere, tide, crulande |
-| `-an` | iuliscan, weardan, ċiriċeburnan |
-| `-es` | geƿinnes, laurentes, yankees |
-| `-um` | stōwum, sǣfōrum, betweonum |
-| `-de` | tide, crulande, īeglande |
-| `-ng` | hūselhālgung, āmang, rising |
+| `-e` | ārwurðnysse, cǣġe, farende |
+| `-s` | celebrations, villages, annivs |
+| `-es` | villages, ides, missiles |
+| `-an` | þēodacynewīsan, hāligan, europiscan |
+| `-um` | dorsætum, maniȝum, elpendum |
+| `-de` | farende, ungeƿilde, bestandende |
+| `-en` | ƿriten, eċġen, hyrneġen |
+| `-on` | edmonton, huffington, aragon |
 
 ### 6.3 Bound Stems (Lexical Roots)
 
@@ -479,18 +481,18 @@ Bound stems are high-frequency subword units that are semantically cohesive but
 
 | Stem | Cohesion | Substitutability | Examples |
 |------|----------|------------------|----------|
-| `mani` | 2.03x | 43 contexts | amani, maniġ, maniȝ |
-| `enne` | 1.97x | 48 contexts | fenne, agenne, etenne |
-| `ster` | 1.84x | 59 contexts | buster, easter, noster |
-| `wear` | 1.97x | 43 contexts | weard, wearþ, wearð |
-| `unge` | 1.85x | 46 contexts | tunge, tunges, hunger |
-| `tion` | 2.26x | 19 contexts | action, motion, nation |
-| `inga` | 1.78x | 34 contexts | þinga, minga, ðinga |
-| `ning` | 1.70x | 35 contexts | cyning, ininga, cining |
-| `aste` | 1.76x | 27 contexts | ēaste, easte, taste |
-| `ynin` | 2.28x | 11 contexts | cynin, cyning, cyninȝ |
-| `afod` | 1.85x | 18 contexts | heafod, ƿafode, hēafod |
-| `nisc` | 1.56x | 27 contexts | denisc, cinisc, dēnisc |
+| `enne` | 2.04x | 48 contexts | fenne, etenne, cenneþ |
+| `mani` | 2.03x | 43 contexts | amani, maniȝ, maniġ |
+| `wear` | 1.91x | 43 contexts | wearð, wearg, weard |
+| `ster` | 1.67x | 59 contexts | sister, ēaster, faster |
+| `unge` | 1.77x | 46 contexts | tunge, tunges, jungen |
+| `tion` | 2.19x | 19 contexts | motion, nation, action |
+| `inga` | 1.72x | 34 contexts | þinga, minga, ðinga |
+| `ning` | 1.64x | 35 contexts | mining, cining, cyning |
+| `aste` | 1.69x | 27 contexts | taste, easte, ēaste |
+| `ynin` | 2.21x | 11 contexts | cynin, cyning, cyninȝ |
+| `afod` | 1.82x | 18 contexts | hēafod, heafod, ƿafode |
+| `nisc` | 1.49x | 27 contexts | rūnisc, denisc, dēnisc |
 
 ### 6.4 Affix Compatibility (Co-occurrence)
 
@@ -498,12 +500,14 @@ This table shows which prefixes and suffixes most frequently co-occur on the sam
 
 | Prefix | Suffix | Frequency | Examples |
 |--------|--------|-----------|----------|
-| `-ge` | `-e` | 75 words | gearore, gegaderode |
-| `-ge` | `-de` | 27 words | gegaderode, gelytlode |
-| `-ge` | `-um` | 20 words | getremincum, gearum |
-| `-ge` | `-an` | 19 words | gearan, gesecan |
-| `-ge` | `-es` | 18 words | gearƿes, geferscipes |
-| `-ge` | `-ng` | 7 words | geswutelung, geþrang |
+| `-ge` | `-e` | 79 words | geƿorhte, geƿǣre |
+| `-ge` | `-en` | 35 words | getimbroden, geferræden |
+| `-ge` | `-de` | 35 words | geanede, gehiersomode |
+| `-ge` | `-s` | 29 words | genus, geardas |
+| `-ge` | `-an` | 20 words | gegildan, gemæccan |
+| `-ge` | `-um` | 20 words | gerādum, germanicum |
+| `-ge` | `-es` | 17 words | geofones, geānlǣhtes |
+| `-ge` | `-on` | 9 words | gestaðoledon, gestrēon |
 
 ### 6.5 Recursive Morpheme Segmentation
 
@@ -511,21 +515,21 @@ Using **Recursive Hierarchical Substitutability**, we decompose complex words in
 
 | Word | Suggested Split | Confidence | Stem |
 |------|-----------------|------------|------|
-| gereordes | **`ge-reord-es`** | 6.0 | `reord` |
-| geƿealdes | **`ge-ƿeald-es`** | 6.0 | `ƿeald` |
-| gehealdan | **`ge-heald-an`** | 6.0 | `heald` |
-| gefeohtes | **`ge-feoht-es`** | 6.0 | `feoht` |
-| foresittendlican | **`foresittendlic-an`** | 4.5 | `foresittendlic` |
-| hundgēares | **`hundgēar-es`** | 4.5 | `hundgēar` |
-| ƿīntrēoƿum | **`ƿīntrēoƿ-um`** | 4.5 | `ƿīntrēoƿ` |
-| cræftigum | **`cræftig-um`** | 4.5 | `cræftig` |
-| bryttiscan | **`bryttisc-an`** | 4.5 | `bryttisc` |
-| speliġendhūses | **`speliġendhūs-es`** | 4.5 | `speliġendhūs` |
-| russiscum | **`russisc-um`** | 4.5 | `russisc` |
-| regollicum | **`regollic-um`** | 4.5 | `regollic` |
-| atlantiscan | **`atlantisc-an`** | 4.5 | `atlantisc` |
-| ƿealdende | **`ƿealden-de`** | 4.5 | `ƿealden` |
-| gegaderunge | **`ge-gaderunge`** | 4.5 | `gaderunge` |
+| gehƿilcum | **`ge-hƿilc-um`** | 6.0 | `hƿilc` |
+| gefeahten | **`ge-feaht-en`** | 6.0 | `feaht` |
+| underbyrigum | **`underbyrig-um`** | 4.5 | `underbyrig` |
+| geþoftscipe | **`ge-þoftscipe`** | 4.5 | `þoftscipe` |
+| sanghordes | **`sanghord-es`** | 4.5 | `sanghord` |
+| gesweoster | **`ge-sweoster`** | 4.5 | `sweoster` |
+| russlandes | **`russland-es`** | 4.5 | `russland` |
+| þēodisclandes | **`þēodiscland-es`** | 4.5 | `þēodiscland` |
+| gestrēonum | **`ge-strē-on-um`** | 4.5 | `strē` |
+| drȳġelandes | **`drȳġeland-es`** | 4.5 | `drȳġeland` |
+| drēamhordes | **`drēamhord-es`** | 4.5 | `drēamhord` |
+| andweardum | **`andweard-um`** | 4.5 | `andweard` |
+| engliscan | **`englisc-an`** | 4.5 | `englisc` |
+| stǣrlican | **`stǣrlic-an`** | 4.5 | `stǣrlic` |
+| bedæleden | **`bedæled-en`** | 4.5 | `bedæled` |
 
 ### 6.6 Linguistic Interpretation
 
@@ -759,4 +763,4 @@ MIT License - Free for academic and commercial use.
 ---
 *Generated by Wikilangs Models Pipeline*
 
-*Report Date: 2026-01-03 14:10:12*
+*Report Date: 2026-01-03 16:22:13*

models/embeddings/aligned/ang_128d.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:916924f4cae8ca68c9b328270e6c8f5d750e1f5f8a934a8f14b2a9ed44bfa5e9
+oid sha256:4042fcd252c39668ffd282b54a7959049dca6a2fa2494d370faeffe4781a8f96
 size 1034408961

models/embeddings/aligned/ang_128d.projection.npy

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:25502450dc6c859927a71989b5700de37d1474c28657905e5d57935efab0ce21
+oid sha256:8cef629d215fae2a8d4f6d0757920e964a035b8e6ddc4ce58bc93bd2a48cd3ee
 size 65664

models/embeddings/aligned/ang_32d.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bf60165a1c710ff1513a37905cbab9e4938cc02aa22cdf5c36d13a954072c8da
+oid sha256:7386137c71165a0deab23014e63a9dab73ec6a8ba0a1f42c0ee53ddb3189b496
 size 258728193

models/embeddings/aligned/ang_32d.projection.npy

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ddf7e9dc22e45732b340414482d0aa30bca9a7225905b66b922786d632cfb556
+oid sha256:0ccb3f646a1f22afcafedaa07234038b8eb0a81333e2639d062d8e005227b599
 size 4224

models/embeddings/aligned/ang_64d.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ca061364183fd6d9b77c2b10561dad103ed397dde669eca49fcc861ac7e34566
+oid sha256:43f6d782da9fc3ba7f04cc21fd7e473b4bf33cf249fa39d5f48d9b3fd8683cdd
 size 517288449

models/embeddings/aligned/ang_64d.projection.npy

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a318bf6e939ba9d157a6589439149031e127db782751369e9a4fb9bca1368f69
+oid sha256:32cfa410833fe9bfa29b598a89a4320b14c48b98059aa50a02d59ad523514af2
 size 16512

models/embeddings/monolingual/ang_128d.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:916924f4cae8ca68c9b328270e6c8f5d750e1f5f8a934a8f14b2a9ed44bfa5e9
+oid sha256:4042fcd252c39668ffd282b54a7959049dca6a2fa2494d370faeffe4781a8f96
 size 1034408961

models/embeddings/monolingual/ang_32d.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bf60165a1c710ff1513a37905cbab9e4938cc02aa22cdf5c36d13a954072c8da
+oid sha256:7386137c71165a0deab23014e63a9dab73ec6a8ba0a1f42c0ee53ddb3189b496
 size 258728193

models/embeddings/monolingual/ang_64d.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ca061364183fd6d9b77c2b10561dad103ed397dde669eca49fcc861ac7e34566
+oid sha256:43f6d782da9fc3ba7f04cc21fd7e473b4bf33cf249fa39d5f48d9b3fd8683cdd
 size 517288449

models/subword_markov/ang_markov_ctx1_subword.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3c51cb623755a1753881542eeb56f63d0e3cca6b26b36201d37c4673561c4964
-size 66968
+oid sha256:a3d0e9296297f8dedcde600098131e69215f012f86ea97793a8e1080e8128c44
+size 66921

models/subword_markov/ang_markov_ctx2_subword.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:75c24d757e1f8ff20404bf43a0494cdc7025f3099ecf2c5fd0ded6196982f719
-size 360335
+oid sha256:29bbca0b68aa28374aeeb514ff149d22cae2ccc2374dcc906d83f03a4bc5ba01
+size 365734

models/subword_markov/ang_markov_ctx3_subword.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:92be48c01deb0d03b413930a4303e8ea68fa19ad34847e3364801ebb65d661f3
-size 1378085
+oid sha256:395c5ce0a7f4fcc034376cb9661ac1b61aa5fe4083f091797c2c5f7a19ac9da3
+size 1376530

models/subword_markov/ang_markov_ctx4_subword.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:63d5bae5a9064ea8d0b55a7323a00e10fd3cc817a992d548d92596656acd9a23
-size 3780418
+oid sha256:eb6b7720d371619b051d6a22c2b70ac3c30be0259c80a7665e5d7cb3583fd0d2
+size 3782214

models/subword_ngram/ang_2gram_subword.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3de4ae1fde9e711415a305f0cff7d028d9d6dbb8e9784696204bb787c90dd202
-size 40740
+oid sha256:2c29aed8334b7b6162a0bf48ffc4670ac15eaee132268818982da5a677631114
+size 40739

models/subword_ngram/ang_3gram_subword.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3c21515a66cb038262024984e5d43dfebb139b67620b893fda4f9839a52c0dc7
-size 290609
+oid sha256:1afdcf65c813b813b9f7167b71a86e2ed39b8472d93eb94894396d88f272f0c9
+size 290617

models/subword_ngram/ang_4gram_subword.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6a666f8631a2c316853af5db1a673829c50f10adee5b2020606062eb57ed52f6
-size 1209850
+oid sha256:66b000da624816424df2f6e6a6553cfc1cc8c61a7abb8f07d8a679486bfaecdc
+size 1209861

models/subword_ngram/ang_5gram_subword.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7c3056ca0bf5781dd09261541479f028b29a9cc8d5a5a6f2bd27261e1805e569
-size 2555553
+oid sha256:d02d41d0aeffee68af91ecb738ccd5e09afea80746ffbd0c2b4587fe7124c75c
+size 2557033

models/tokenizer/ang_tokenizer_16k.model

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:040cdd2ba84e292b44535ad7e47be3d9d4dea1f384c04801879d77cd0895da82
+oid sha256:7acdfd4c40e61701b3c752556981df6787ce11c50ebb3526c7a9fc756f4ad69f
 size 507562

models/tokenizer/ang_tokenizer_32k.model

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:11310c4f4880a83c7e0121967baf71b21fdb39d161532062e4d76550c057a832
+oid sha256:1b247988006b8533aac3130c934db31dad60edd556804ac0cb222338b7f1a996
 size 785162

models/tokenizer/ang_tokenizer_64k.model

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d5e8fac427c909ddf6ed2f1d118ab76e763479cf812ec71e84f5a78d4c94f50d
+oid sha256:3e8a3d118894e1a975ad1fff32919ea29007260b0c52dade56cef8a1339ad50e
 size 1395192

models/tokenizer/ang_tokenizer_8k.model

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:de84f48cb9fa027b88c36715acd20bb110a824676f2c89684ff9ac4bc44afca9
+oid sha256:8cda0d4bd94f33872e22279abe24e04c56a98809d2caef495cf50e1306614ba7
 size 372530

models/word_markov/ang_markov_ctx1_word.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f9404cf8e75c7dafaa37bb38566f58ecbfbcde523666ebe80d07fb25ebd8ff43
-size 2834503
+oid sha256:2d984506f93bc12a615d11f8673288f8b486297370aea808062a825b039b007b
+size 2832246

models/word_markov/ang_markov_ctx2_word.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dd85fc8f75ac79620c6acdaafd11303ab1e84fccbf3610c168b1fc640627a885
-size 5756661
+oid sha256:44a93c0e23326da84f4adfb9922498a0150b7776c64ad13512e2c8cac1f0b09b
+size 5756090

models/word_markov/ang_markov_ctx3_word.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fcce366c6306e35d8fc2a154588f0dc2d60b1cad2e9b8745210fd703040cf2ec
-size 7380283
+oid sha256:58b09f9cf4546dfd1942066e7985ab66b49cd7d5de114ab4d137b46a255cb02d
+size 7368188

models/word_markov/ang_markov_ctx4_word.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:baebcc70f7df9c5b12bac7c718db37f7325d96c1bbf341f4e669cc2d6ab223f8
-size 8221898
+oid sha256:e5e3d039cc0295413db5595736159f713ab902083e1161c02a9fbebec5add409
+size 8219077

models/word_ngram/ang_2gram_word.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c152efe031001c5114e91ed2cef367af2fdd50d4da5544809c5ae8cbcc106504
-size 108294
+oid sha256:53842c7854c682bf6b33d37af0123022da18d7f0a3209576dbc3578b43182725
+size 107141

models/word_ngram/ang_3gram_word.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:01ccbf2a73755bb852ec26c495f01344430dc76928525d7aa4c2ceb4cf736bdd
-size 110659
+oid sha256:b5a29ef33847dff6003ed529dd6a0eeb6991cb931c8c1d52f463c3daeca18c95
+size 110953

models/word_ngram/ang_4gram_word.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bcc923ecbf09b8736c1afc198a7cd786085e0b469ccc980707742b62183aa38a
-size 221788
+oid sha256:f53e17ffa9d4a1f899d7ed060df75a23e530911203da730f48b1050f37a35fc8
+size 222381

models/word_ngram/ang_5gram_word.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:506caac829f6d1545248ae1d0df0166b41dbef522f6beb4b192ce2672c70d194
-size 165316
+oid sha256:c9fb9acd2ed808ea08685fbe13baaef8c3a0ebbd981d5dd368178cbd489e818e
+size 165202