Tumbuka (tum) subword embeddings
| Vocab size | vocab | model | 25 dim | 50 dim | 100 dim | 200 dim | 300 dim |
|---|---|---|---|---|---|---|---|
| 1000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 3000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
| 5000 | vocab | model |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
txt |
bin
bokeh | umap | matrix |
Training corpus sample, encoded with different BPE vocabulary sizes
| Vocab size | tumwiki sample |
|---|---|
| original | the dance for "bang bang bang," choreographed by parris goebel, was one of the most popular dances for the year 0000. their stage fashion is also a hi thumb thumb india (hi, bh, gom, mai, mr & pi: भारत, as: ভাৰত, bpy: ভারত, gu: ભારત, kn: ಭಾರತ, ks: بًارت, ml: ഇന്ത്യ, or: ଭାରତ, pa: ਭਾਰਤ, sa: भारतम्, sd big bang performing in 0000 in january 0000, yg entertainment began releasing teasers for big bang's fifth korean ep ''alive'', which was released in |
| 1000 | ▁the ▁d ance ▁for ▁" bang ▁bang ▁bang ," ▁ch ore og ra ph ed ▁by ▁p ar r is ▁g o e bel , ▁was ▁one ▁of ▁the ▁most ▁popular ▁d an c es ▁for ▁the ▁year ▁0000. ▁their ▁sta ge ▁f as h ion ▁is ▁also ▁a ▁h i ▁thumb ▁thumb ▁in di a ▁( h i , ▁b h , ▁g om , ▁ma i , ▁m r ▁& ▁p i : ▁ भ ा र त , ▁as : ▁ ভ া ৰ ত , ▁b p y : ▁ ভ া র ত , ▁gu : ▁ ભ ાર ત , ▁k n : ▁ ಭಾರತ , ▁k s : ▁ ب ً ا ر ت , ▁ml : ▁ ഇന ് ത ് യ , ▁or : ▁ ଭାରତ , ▁pa : ▁ ਭ ਾ ਰ ਤ , ▁s a : ▁ भ ा र त म ् , ▁s d ▁big ▁bang ▁perform ing ▁in ▁0000 ▁in ▁j an uary ▁0000, ▁y g ▁ent ert ain ment ▁be g an ▁releas ing ▁t e as ers ▁for ▁big ▁bang ' s ▁f if th ▁korean ▁ep ▁'' ali ve '', ▁which ▁was ▁released ▁in |
| 3000 | ▁the ▁dance ▁for ▁" bang ▁bang ▁bang ," ▁choreograph ed ▁by ▁par ris ▁go e bel , ▁was ▁one ▁of ▁the ▁most ▁popular ▁d ances ▁for ▁the ▁year ▁0000. ▁their ▁stage ▁fashion ▁is ▁also ▁a ▁hi ▁thumb ▁thumb ▁india ▁( hi , ▁bh , ▁g om , ▁ma i , ▁m r ▁& ▁p i : ▁ भा रत , ▁as : ▁ ভা ৰ ত , ▁b p y : ▁ ভা র ত , ▁gu : ▁ ભ ાર ત , ▁kn : ▁ ಭಾರತ , ▁k s : ▁ب ً ارت , ▁ml : ▁ ഇന ് ത ് യ , ▁or : ▁ ଭାରତ , ▁pa : ▁ ਭ ਾ ਰ ਤ , ▁sa : ▁ भा रत म ् , ▁s d ▁big ▁bang ▁performing ▁in ▁0000 ▁in ▁january ▁0000, ▁yg ▁entertainment ▁be gan ▁releasing ▁te as ers ▁for ▁big ▁bang ' s ▁fifth ▁korean ▁ep ▁'' alive '', ▁which ▁was ▁released ▁in |
| 5000 | ▁the ▁dance ▁for ▁" bang ▁bang ▁bang ," ▁choreographed ▁by ▁par ris ▁go e bel , ▁was ▁one ▁of ▁the ▁most ▁popular ▁d ances ▁for ▁the ▁year ▁0000. ▁their ▁stage ▁fashion ▁is ▁also ▁a ▁hi ▁thumb ▁thumb ▁india ▁( hi , ▁bh , ▁g om , ▁ma i , ▁mr ▁& ▁pi : ▁भारत , ▁as : ▁ভাৰত , ▁b p y : ▁ভারত , ▁gu : ▁ ભ ાર ત , ▁kn : ▁ ಭಾರತ , ▁k s : ▁ب ً ارت , ▁ml : ▁ ഇന ് ത ് യ , ▁or : ▁ ଭାରତ , ▁pa : ▁ ਭ ਾ ਰ ਤ , ▁sa : ▁भारत म ् , ▁s d ▁big ▁bang ▁performing ▁in ▁0000 ▁in ▁january ▁0000, ▁yg ▁entertainment ▁began ▁releasing ▁teas ers ▁for ▁big ▁bang ' s ▁fifth ▁korean ▁ep ▁'' alive '', ▁which ▁was ▁released ▁in |