Twi (tw) subword embeddings
| Vocab size | vocab | model | 25 dim | 50 dim | 100 dim | 200 dim | 300 dim |
|---|---|---|---|---|---|---|---|
| 1000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 3000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 5000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
Training corpus sample, encoded with different BPE vocabulary sizes
| Vocab size | twwiki sample |
|---|---|
| original | macedonia mmrahy0 b0dwa fie kesie abadwemfɔɔ nam oman no mmra a 0fa sukuu ho so na wɔdi bue sukuu yi wɔ afe apem aduonu wɔtwe (0000) mu . adesuafoɔ a onini no buee n'anum, aku nyaa akokoduro de ne nsa sin no hyεε n'anum. onini no kaa nano too mu bεyɛ sima kakra na ɔsan buee n'anum. aku yii ne nsa fi asantehene: asantehene a, ote asika dwa kokoo soo, ebeye mfie edu nie na obedii ne wofa, otumfuo opoku ware oto so mmienu adee. asanteman do won ahenf |
| 1000 | ▁macedonia ▁mm ra hy 0 ▁b 0 dwa ▁fie ▁k esie ▁aba d w em f ɔ ɔ ▁na m ▁oman ▁no ▁mm ra ▁a ▁0 fa ▁sukuu ▁ho ▁so ▁na ▁wɔ di ▁b ue ▁sukuu ▁yi ▁wɔ ▁afe ▁apem ▁aduon u ▁wɔ twe ▁( 0000 ) ▁mu ▁. ▁adesua foɔ ▁a ▁onini ▁no ▁bu ee ▁n ' anum , ▁aku ▁nyaa ▁ak o ko d uro ▁de ▁ne ▁nsa ▁s in ▁no ▁hy εε ▁n ' anum . ▁onini ▁no ▁kaa ▁nan o ▁too ▁mu ▁b ε yɛ ▁si ma ▁ka kra ▁na ▁ɔ san ▁bu ee ▁n ' anum . ▁aku ▁yi i ▁ne ▁nsa ▁fi ▁asantehene : ▁asantehene ▁a , ▁o te ▁asi ka ▁dwa ▁ko ko o ▁so o , ▁e be ye ▁m fie ▁edu ▁n ie ▁na ▁o be dii ▁ne ▁wo fa , ▁o tum fuo ▁o po ku ▁w are ▁o to ▁so ▁mmienu ▁ad ee . ▁asanteman ▁do ▁won ▁ahen f |
| 3000 | ▁macedonia ▁mmra hy 0 ▁b 0 dwa ▁fie ▁kesie ▁aba dwemfɔɔ ▁nam ▁oman ▁no ▁mmra ▁a ▁0 fa ▁sukuu ▁ho ▁so ▁na ▁wɔdi ▁bue ▁sukuu ▁yi ▁wɔ ▁afe ▁apem ▁aduonu ▁wɔtwe ▁( 0000) ▁mu ▁. ▁adesua foɔ ▁a ▁onini ▁no ▁buee ▁n ' anum , ▁aku ▁nyaa ▁ak okod uro ▁de ▁ne ▁nsa ▁sin ▁no ▁hy εε ▁n ' anum . ▁onini ▁no ▁kaa ▁nano ▁too ▁mu ▁b ε yɛ ▁sima ▁kakra ▁na ▁ɔsan ▁buee ▁n ' anum . ▁aku ▁yii ▁ne ▁nsa ▁fi ▁asantehene : ▁asantehene ▁a , ▁ote ▁asika ▁dwa ▁kokoo ▁soo , ▁ebeye ▁mfie ▁edu ▁nie ▁na ▁o bedii ▁ne ▁wofa , ▁otumfuo ▁opoku ▁ware ▁oto ▁so ▁mmienu ▁adee . ▁asanteman ▁do ▁won ▁ahen f |
| 5000 | ▁macedonia ▁mmrahy 0 ▁b 0 dwa ▁fie ▁kesie ▁abadwemfɔɔ ▁nam ▁oman ▁no ▁mmra ▁a ▁0 fa ▁sukuu ▁ho ▁so ▁na ▁wɔdi ▁bue ▁sukuu ▁yi ▁wɔ ▁afe ▁apem ▁aduonu ▁wɔtwe ▁(0000) ▁mu ▁. ▁adesuafoɔ ▁a ▁onini ▁no ▁buee ▁n ' anum , ▁aku ▁nyaa ▁akokoduro ▁de ▁ne ▁nsa ▁sin ▁no ▁hy εε ▁n ' anum . ▁onini ▁no ▁kaa ▁nano ▁too ▁mu ▁b ε yɛ ▁sima ▁kakra ▁na ▁ɔsan ▁buee ▁n ' anum . ▁aku ▁yii ▁ne ▁nsa ▁fi ▁asantehene : ▁asantehene ▁a , ▁ote ▁asika ▁dwa ▁kokoo ▁soo , ▁ebeye ▁mfie ▁edu ▁nie ▁na ▁obedii ▁ne ▁wofa , ▁otumfuo ▁opoku ▁ware ▁oto ▁so ▁mmienu ▁adee . ▁asanteman ▁do ▁won ▁ahen f |