Pedi (nso) subword embeddings

Vocab size vocab model 25 dim 50 dim 100 dim 200 dim 300 dim
1000 vocab model txt | bin
bokeh | umap | matrix 		txt | bin
bokeh | umap | matrix 		txt | bin
bokeh | umap | matrix 		txt | bin
bokeh | umap | matrix 		txt | bin
bokeh | umap | matrix
3000 vocab model txt | bin
bokeh | umap | matrix 		txt | bin
bokeh | umap | matrix 		txt | bin
bokeh | umap | matrix 		txt | bin
bokeh | umap | matrix 		txt | bin
bokeh | umap | matrix
5000 vocab model txt | bin
bokeh | umap | matrix 		txt | bin
bokeh | umap | matrix 		txt | bin
bokeh | umap | matrix 		txt | bin
bokeh | umap | matrix 		txt | bin
bokeh | umap | matrix

View Pedi subword embeddings in TensorFlow Projector

2-D UMAP plots

Embedding matrix plots

Training corpus sample, encoded with different BPE vocabulary sizes

Vocab sizensowiki sample
original mokgwa wa boraro o swana le wa bobedi, efela baraloki bao ba sa lobang papadi (mokgwa wa pele) ba tšwela pele.
seat ya alfred nzo ke mount ayliff. setšhaba se 000,000, e 000,000 ba bulela sethosa (0000 palo ya batho). district code ke dc00.
during the boer war, the gillitts family farm held a garrison of british troops, who built a small dam for water and a camp in what is now camp road.
1000 ▁mo kgwa ▁wa ▁boraro ▁o ▁swana ▁le ▁wa ▁bobedi , ▁e f ela ▁ba ra loki ▁bao ▁ba ▁sa ▁lo ba ng ▁pa pa di ▁( mo kgwa ▁wa ▁pele ) ▁ba ▁tšw ela ▁pele .
▁se at ▁ya ▁al f red ▁n z o ▁ke ▁mo un t ▁a y li f f . ▁setšhaba ▁se ▁000, 000, ▁e ▁000, 000 ▁ba ▁bu lela ▁set ho sa ▁(0000 ▁palo ▁ya ▁batho ). ▁district ▁co de ▁ke ▁d c 00 .
▁du ring ▁the ▁bo er ▁wa r , ▁the ▁g il lit ts ▁fa mi ly ▁far m ▁he ld ▁a ▁ga r ri s on ▁of ▁b ri ti sh ▁t ro op s , ▁w ho ▁bu il t ▁a ▁s ma ll ▁d am ▁for ▁wa ter ▁and ▁a ▁ca mp ▁in ▁w h at ▁is ▁no w ▁ca mp ▁ro ad .
3000 ▁mokgwa ▁wa ▁boraro ▁o ▁swana ▁le ▁wa ▁bobedi , ▁e fela ▁baraloki ▁bao ▁ba ▁sa ▁lo ba ng ▁papadi ▁( mo kgwa ▁wa ▁pele ) ▁ba ▁tšwela ▁pele .
▁seat ▁ya ▁alfred ▁nzo ▁ke ▁mo unt ▁a y li ff . ▁setšhaba ▁se ▁000,000, ▁e ▁000,000 ▁ba ▁bulela ▁sethosa ▁(0000 ▁palo ▁ya ▁batho ). ▁district ▁code ▁ke ▁dc 00.
▁during ▁the ▁bo er ▁war , ▁the ▁gillitts ▁family ▁farm ▁he ld ▁a ▁ga r ris on ▁of ▁british ▁t ro op s , ▁who ▁built ▁a ▁small ▁dam ▁for ▁water ▁and ▁a ▁ca mp ▁in ▁wh at ▁is ▁now ▁ca mp ▁road .
5000 ▁mokgwa ▁wa ▁boraro ▁o ▁swana ▁le ▁wa ▁bobedi , ▁efela ▁baraloki ▁bao ▁ba ▁sa ▁loba ng ▁papadi ▁( mo kgwa ▁wa ▁pele ) ▁ba ▁tšwela ▁pele .
▁seat ▁ya ▁alfred ▁nzo ▁ke ▁mo unt ▁a yli ff . ▁setšhaba ▁se ▁000,000, ▁e ▁000,000 ▁ba ▁bulela ▁sethosa ▁(0000 ▁palo ▁ya ▁batho ). ▁district ▁code ▁ke ▁dc 00.
▁during ▁the ▁bo er ▁war , ▁the ▁gillitts ▁family ▁farm ▁he ld ▁a ▁ga r ris on ▁of ▁british ▁t ro op s , ▁who ▁built ▁a ▁small ▁dam ▁for ▁water ▁and ▁a ▁camp ▁in ▁wh at ▁is ▁now ▁camp ▁road .