Tsonga (ts) subword embeddings
| Vocab size | vocab | model | 25 dim | 50 dim | 100 dim | 200 dim | 300 dim |
|---|---|---|---|---|---|---|---|
| 1000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 3000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 5000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
Training corpus sample, encoded with different BPE vocabulary sizes
| Vocab size | tswiki sample |
|---|---|
| original | nhokonho, kambe lowu tiviwaka tanihi vuvabyi bya hansen (hd), i bya nkarhi wo leha ntluleto wa byona wu vangiwa hi bakitheriya ya kuni tinxakaxaka ta mabindzu hikuya hi mafambisele ya tiko na ndhawu, kambe tinxaka leti tolovelekeke hileti: * bindzu ra munhu-un'we: muxaka lowu wut matluka ya mbvuva mbvuva i murhi lowu milaka kunwana na kunwana. murhi lowu loko wu byariwile ekaya wu tirha tanihi ndzhuti na ku sivela mabubutsa lok |
| 1000 | ▁n ho kon ho , ▁kambe ▁lowu ▁tiv iwaka ▁tanihi ▁vuvabyi ▁bya ▁han s en ▁( h d ), ▁i ▁bya ▁nkarhi ▁wo ▁le ha ▁nt lu le to ▁wa ▁b yona ▁wu ▁vang iwa ▁hi ▁ba ki the ri ya ▁ya ▁kun i ▁tin xaka xaka ▁ta ▁mab indzu ▁hiku ya ▁hi ▁ma fa mb is ele ▁ya ▁tiko ▁na ▁nd hawu , ▁kambe ▁tin xaka ▁leti ▁to lo vele keke ▁hi le ti : ▁* ▁b indzu ▁ra ▁munhu - un ' we : ▁mu xaka ▁lowu ▁wu t ▁matluka ▁ya ▁mb vu va ▁mb vu va ▁i ▁murhi ▁lowu ▁mi laka ▁kun wana ▁na ▁kun wana . ▁murhi ▁lowu ▁loko ▁wu ▁bya riwile ▁eka ya ▁wu ▁ti rha ▁tanihi ▁ndz hu ti ▁na ▁ku ▁si vela ▁ma bu bu tsa ▁lo k |
| 3000 | ▁n ho konho , ▁kambe ▁lowu ▁tiviwaka ▁tanihi ▁vuvabyi ▁bya ▁han sen ▁( h d ), ▁i ▁bya ▁nkarhi ▁wo ▁leha ▁ntluleto ▁wa ▁byona ▁wu ▁vangiwa ▁hi ▁ba ki theriya ▁ya ▁kuni ▁tinxaka xaka ▁ta ▁mabindzu ▁hikuya ▁hi ▁mafambisele ▁ya ▁tiko ▁na ▁ndhawu , ▁kambe ▁tinxaka ▁leti ▁tolovelekeke ▁hi le ti : ▁* ▁bindzu ▁ra ▁munhu - un ' we : ▁muxaka ▁lowu ▁wu t ▁matluka ▁ya ▁mb vuva ▁mb vuva ▁i ▁murhi ▁lowu ▁mi laka ▁kun wana ▁na ▁kun wana . ▁murhi ▁lowu ▁loko ▁wu ▁bya riwile ▁ekaya ▁wu ▁tirha ▁tanihi ▁ndzhuti ▁na ▁ku ▁sivela ▁ma bu bu tsa ▁lo k |
| 5000 | ▁n ho konho , ▁kambe ▁lowu ▁tiviwaka ▁tanihi ▁vuvabyi ▁bya ▁han sen ▁( h d ), ▁i ▁bya ▁nkarhi ▁wo ▁leha ▁ntluleto ▁wa ▁byona ▁wu ▁vangiwa ▁hi ▁ba kitheriya ▁ya ▁kuni ▁tinxaka xaka ▁ta ▁mabindzu ▁hikuya ▁hi ▁mafambisele ▁ya ▁tiko ▁na ▁ndhawu , ▁kambe ▁tinxaka ▁leti ▁tolovelekeke ▁hile ti : ▁* ▁bindzu ▁ra ▁munhu - un ' we : ▁muxaka ▁lowu ▁wu t ▁matluka ▁ya ▁mbvuva ▁mbvuva ▁i ▁murhi ▁lowu ▁milaka ▁kunwana ▁na ▁kunwana . ▁murhi ▁lowu ▁loko ▁wu ▁bya riwile ▁ekaya ▁wu ▁tirha ▁tanihi ▁ndzhuti ▁na ▁ku ▁sivela ▁ma bu bu tsa ▁lo k |