English (en) subword embeddings

Vocab size vocab model 25 dim 50 dim 100 dim 200 dim 300 dim
1000 vocab model txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
3000 vocab model txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
5000 vocab model txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
10000 vocab model txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
25000 vocab model txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
50000 vocab model txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
100000 vocab model txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
200000 vocab model txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix
txt | bin
bokeh | umap | matrix

Training corpus sample, encoded with different BPE vocabulary sizes

Vocab sizeenwiki sample
original the diets of the wealthy were rich in sugars, which promoted periodontal disease. despite the flattering physiques portrayed on tomb walls, the overwe
the first project proposed and worked on was a period film to be called ''dora-heita'', but when this was deemed too expensive, attention shifted to '
neutralization with a base weaker than the acid results in a weakly acidic salt. an example is the weakly acidic ammonium chloride, which is produced
1000 ▁the ▁d iet s ▁of ▁the ▁we al th y ▁were ▁r ich ▁in ▁su g ars , ▁which ▁p rom ot ed ▁per i od on t al ▁dis e ase . ▁des p ite ▁the ▁fl at ter ing ▁ph ys i qu es ▁port ra y ed ▁on ▁to m b ▁w all s , ▁the ▁over we
▁the ▁first ▁pro ject ▁pro pos ed ▁and ▁work ed ▁on ▁was ▁a ▁per i od ▁film ▁to ▁be ▁called ▁'' d or a - he it a '', ▁but ▁when ▁this ▁was ▁de em ed ▁to o ▁exp ens ive , ▁att ent ion ▁sh if ted ▁to ▁'
▁ne ut ral iz ation ▁with ▁a ▁b ase ▁we ak er ▁than ▁the ▁ac id ▁res ult s ▁in ▁a ▁we ak ly ▁ac id ic ▁s al t . ▁an ▁ex amp le ▁is ▁the ▁we ak ly ▁ac id ic ▁am m on ium ▁ch l or ide , ▁which ▁is ▁prod uced
3000 ▁the ▁d iet s ▁of ▁the ▁we alth y ▁were ▁rich ▁in ▁sug ars , ▁which ▁promot ed ▁period on tal ▁dis e ase . ▁despite ▁the ▁fl at ter ing ▁phys i qu es ▁port ray ed ▁on ▁tom b ▁wall s , ▁the ▁over we
▁the ▁first ▁project ▁pro posed ▁and ▁worked ▁on ▁was ▁a ▁period ▁film ▁to ▁be ▁called ▁'' d or a - he ita '', ▁but ▁when ▁this ▁was ▁de em ed ▁too ▁exp ensive , ▁att ention ▁sh if ted ▁to ▁'
▁ne ut ral ization ▁with ▁a ▁base ▁we aker ▁than ▁the ▁ac id ▁result s ▁in ▁a ▁we ak ly ▁ac id ic ▁sal t . ▁an ▁example ▁is ▁the ▁we ak ly ▁ac id ic ▁am mon ium ▁ch lor ide , ▁which ▁is ▁produced
5000 ▁the ▁d iet s ▁of ▁the ▁we alth y ▁were ▁rich ▁in ▁sug ars , ▁which ▁promot ed ▁period on tal ▁disease . ▁despite ▁the ▁flat ter ing ▁phys iques ▁portray ed ▁on ▁tom b ▁wall s , ▁the ▁over we
▁the ▁first ▁project ▁proposed ▁and ▁worked ▁on ▁was ▁a ▁period ▁film ▁to ▁be ▁called ▁'' d ora - he ita '', ▁but ▁when ▁this ▁was ▁de em ed ▁too ▁exp ensive , ▁attention ▁sh if ted ▁to ▁'
▁ne ut ral ization ▁with ▁a ▁base ▁we aker ▁than ▁the ▁acid ▁results ▁in ▁a ▁we ak ly ▁acid ic ▁sal t . ▁an ▁example ▁is ▁the ▁we ak ly ▁acid ic ▁am mon ium ▁ch lor ide , ▁which ▁is ▁produced
10000 ▁the ▁diet s ▁of ▁the ▁wealthy ▁were ▁rich ▁in ▁sug ars , ▁which ▁promoted ▁period on tal ▁disease . ▁despite ▁the ▁flat tering ▁phys iques ▁portrayed ▁on ▁tomb ▁walls , ▁the ▁over we
▁the ▁first ▁project ▁proposed ▁and ▁worked ▁on ▁was ▁a ▁period ▁film ▁to ▁be ▁called ▁'' d ora - he ita '', ▁but ▁when ▁this ▁was ▁deemed ▁too ▁expensive , ▁attention ▁shif ted ▁to ▁'
▁neutral ization ▁with ▁a ▁base ▁we aker ▁than ▁the ▁acid ▁results ▁in ▁a ▁weak ly ▁acid ic ▁salt . ▁an ▁example ▁is ▁the ▁weak ly ▁acid ic ▁am mon ium ▁chlor ide , ▁which ▁is ▁produced
25000 ▁the ▁diet s ▁of ▁the ▁wealthy ▁were ▁rich ▁in ▁sug ars , ▁which ▁promoted ▁period on tal ▁disease . ▁despite ▁the ▁flat tering ▁phys iques ▁portrayed ▁on ▁tomb ▁walls , ▁the ▁over we
▁the ▁first ▁project ▁proposed ▁and ▁worked ▁on ▁was ▁a ▁period ▁film ▁to ▁be ▁called ▁'' d ora - he ita '', ▁but ▁when ▁this ▁was ▁deemed ▁too ▁expensive , ▁attention ▁shifted ▁to ▁'
▁neutral ization ▁with ▁a ▁base ▁weaker ▁than ▁the ▁acid ▁results ▁in ▁a ▁weak ly ▁acid ic ▁salt . ▁an ▁example ▁is ▁the ▁weak ly ▁acid ic ▁ammon ium ▁chloride , ▁which ▁is ▁produced
50000 ▁the ▁diets ▁of ▁the ▁wealthy ▁were ▁rich ▁in ▁sugars , ▁which ▁promoted ▁period on tal ▁disease . ▁despite ▁the ▁flat tering ▁phys iques ▁portrayed ▁on ▁tomb ▁walls , ▁the ▁over we
▁the ▁first ▁project ▁proposed ▁and ▁worked ▁on ▁was ▁a ▁period ▁film ▁to ▁be ▁called ▁'' d ora - he ita '', ▁but ▁when ▁this ▁was ▁deemed ▁too ▁expensive , ▁attention ▁shifted ▁to ▁'
▁neutral ization ▁with ▁a ▁base ▁weaker ▁than ▁the ▁acid ▁results ▁in ▁a ▁weakly ▁acidic ▁salt . ▁an ▁example ▁is ▁the ▁weakly ▁acidic ▁ammonium ▁chloride , ▁which ▁is ▁produced
100000 ▁the ▁diets ▁of ▁the ▁wealthy ▁were ▁rich ▁in ▁sugars , ▁which ▁promoted ▁periodontal ▁disease . ▁despite ▁the ▁flattering ▁phys iques ▁portrayed ▁on ▁tomb ▁walls , ▁the ▁over we
▁the ▁first ▁project ▁proposed ▁and ▁worked ▁on ▁was ▁a ▁period ▁film ▁to ▁be ▁called ▁'' dora - he ita '', ▁but ▁when ▁this ▁was ▁deemed ▁too ▁expensive , ▁attention ▁shifted ▁to ▁'
▁neutralization ▁with ▁a ▁base ▁weaker ▁than ▁the ▁acid ▁results ▁in ▁a ▁weakly ▁acidic ▁salt . ▁an ▁example ▁is ▁the ▁weakly ▁acidic ▁ammonium ▁chloride , ▁which ▁is ▁produced