Kabyle (kab) subword embeddings
| Vocab size | vocab | model | 25 dim | 50 dim | 100 dim | 200 dim | 300 dim |
|---|---|---|---|---|---|---|---|
| 1000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 3000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 5000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 10000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
Training corpus sample, encoded with different BPE vocabulary sizes
| Vocab size | kabwiki sample |
|---|---|
| original | ḍḍeεfan ay yeḍεef yedles deg tmura n tmazγa (maghreb) u ladγa deg ldzayer d ay-n ay d-yurew wemnaε ay ttwamenεent waṭas n tγawsiwin seg seg wasmi ay u caker iqeddec deg usudu aɣelnaw n yilsawen ed tɣermiwin tugmiḍin (inalco) n paris seg useggas n 0000, deg wemkan n andré basset ed lionel galand. tagduda tamegdayt n kungu (ign, ue, oit, insee, sq, pr, re)kungu (pr)kungu-kincasa (sq)kungu (kincasa) (ss) tagduda tamegdayt n kungu (prm) |
| 1000 | ▁ ḍ ḍ e ε fan ▁ay ▁ye ḍ ε ef ▁yed les ▁deg ▁tm ura ▁n ▁t ma z γ a ▁( ma gh r eb ) ▁u ▁la d γ a ▁deg ▁ldzayer ▁d ▁ay - n ▁ay ▁d - y ur ew ▁wem na ε ▁ay ▁tt w am en ε ent ▁waṭas ▁n ▁t γ aw s iwin ▁seg ▁seg ▁was mi ▁ay ▁u ▁c ak er ▁iq edd ec ▁deg ▁us ud u ▁aɣ el na w ▁n ▁yi l s awen ▁ed ▁t ɣer mi win ▁tug mi ḍ in ▁( in al c o ) ▁n ▁paris ▁seg ▁useggas ▁n ▁0000, ▁deg ▁wem kan ▁n ▁an d ré ▁b ass et ▁ed ▁l ion el ▁g al and . ▁tag duda ▁tam eg d ayt ▁n ▁k un gu ▁( ig n , ▁u e , ▁o it , ▁in se e , ▁s q , ▁pr , ▁r e ) k un gu ▁( p r ) k un gu - k in c as a ▁( s q ) k un gu ▁( k in c as a ) ▁( ss ) ▁tag duda ▁tam eg d ayt ▁n ▁k un gu ▁( p r m ) |
| 3000 | ▁ḍ ḍ e ε fan ▁ay ▁yeḍ ε ef ▁yedles ▁deg ▁tmura ▁n ▁tma z γ a ▁( ma gh reb ) ▁u ▁lad γ a ▁deg ▁ldzayer ▁d ▁ay - n ▁ay ▁d - yur ew ▁wem na ε ▁ay ▁tt w amen ε ent ▁waṭas ▁n ▁t γ awsiwin ▁seg ▁seg ▁wasmi ▁ay ▁u ▁c ak er ▁iqedd ec ▁deg ▁us ud u ▁aɣelnaw ▁n ▁yil s awen ▁ed ▁t ɣer mi win ▁tug mi ḍin ▁( in al co ) ▁n ▁paris ▁seg ▁useggas ▁n ▁0000, ▁deg ▁wemkan ▁n ▁and ré ▁b ass et ▁ed ▁l ion el ▁g al and . ▁tagduda ▁tam eg dayt ▁n ▁k un gu ▁( ign , ▁ue , ▁oit , ▁insee , ▁sq , ▁pr , ▁re ) k un gu ▁( pr ) k un gu - kin c as a ▁( s q ) k un gu ▁( kin c as a ) ▁( ss ) ▁tagduda ▁tam eg dayt ▁n ▁k un gu ▁( pr m ) |
| 5000 | ▁ḍḍ e ε fan ▁ay ▁yeḍ ε ef ▁yedles ▁deg ▁tmura ▁n ▁tma z γ a ▁( ma gh reb ) ▁u ▁lad γ a ▁deg ▁ldzayer ▁d ▁ay - n ▁ay ▁d - yur ew ▁wem na ε ▁ay ▁tt w amen ε ent ▁waṭas ▁n ▁t γ awsiwin ▁seg ▁seg ▁wasmi ▁ay ▁u ▁c ak er ▁iqeddec ▁deg ▁us ud u ▁aɣelnaw ▁n ▁yil sawen ▁ed ▁t ɣer mi win ▁tug mi ḍin ▁( in al co ) ▁n ▁paris ▁seg ▁useggas ▁n ▁0000, ▁deg ▁wemkan ▁n ▁and ré ▁b ass et ▁ed ▁l ion el ▁gal and . ▁tagduda ▁tameg dayt ▁n ▁k un gu ▁( ign , ▁ue , ▁oit , ▁insee , ▁sq , ▁pr , ▁re ) kun gu ▁( pr ) kun gu - kin c as a ▁( s q ) kun gu ▁( kin c as a ) ▁( ss ) ▁tagduda ▁tameg dayt ▁n ▁k un gu ▁( pr m ) |
| 10000 | ▁ḍḍ e ε fan ▁ay ▁yeḍ ε ef ▁yedles ▁deg ▁tmura ▁n ▁tmaz γ a ▁( ma gh reb ) ▁u ▁lad γ a ▁deg ▁ldzayer ▁d ▁ay - n ▁ay ▁d - yur ew ▁wem na ε ▁ay ▁tt w amen ε ent ▁waṭas ▁n ▁t γ awsiwin ▁seg ▁seg ▁wasmi ▁ay ▁u ▁caker ▁iqeddec ▁deg ▁us udu ▁aɣelnaw ▁n ▁yil sawen ▁ed ▁tɣer miwin ▁tug mi ḍin ▁( in alco ) ▁n ▁paris ▁seg ▁useggas ▁n ▁0000, ▁deg ▁wemkan ▁n ▁and ré ▁basset ▁ed ▁l ion el ▁gal and . ▁tagduda ▁tamegdayt ▁n ▁kungu ▁( ign , ▁ue , ▁oit , ▁insee , ▁sq , ▁pr , ▁re ) kun gu ▁( pr ) kun gu - kin cas a ▁( sq ) kun gu ▁( kin cas a ) ▁( ss ) ▁tagduda ▁tamegdayt ▁n ▁kungu ▁( pr m ) |