Commit

e92daaa

I switched the LM's output layer to adaptive softmax and used the freed-up compute to widen the main body. Frequent words reached the same loss as a model with about twice the compute, but shrinking rare words to 28 dimensions caused a big loss, so overall the difference is almost zero.
Translated from Japanese

commit e92daaa