摘果みかん

@mikan-919

Hello

Commit activity

214 commits in the last year

longest streak 9 dayscurrent 9

2,535 XPMaximum rank (UID 0)

Today

I wrote down the language's design policy. By laying out side by side what is guaranteed by types and what is decided by the connection with the runtime environment, it became clear where to make judgments when adding features.
Translated from Japanese
WASMと受け渡し規約をコンパイラーから受け取って、JavaScriptと型宣言はhibanaのライブラリで作る形にした。接続コードを出すかどうかでWASM側のエラーの読み方まで変わっていたので、そこも揃えた
hibanaのビルドスクリプトから公開関数を選んで、WASMと接続コードと型宣言を出せるようにした。生成したものをどこに書くかはスクリプトに置く
語の綴りのbyte n-gramを埋め込みに足してみた。幅256で計算量1.28倍相当、まれな語だけなら幅512より良くなったけど、損失の大半は頻出語なので全体はそこまで動かない
Made it possible to also generate TypeScript declarations from the type information used by the WASM connection code. The success and failure of Result are clearly distinguished on the caller side too.
Translated from Japanese
Even if input can be validated in WASM, if it can't return a Result, it took extra effort to pass along the reason for failure. I made it so the JavaScript side can receive the success value and failure reason as-is.
Translated from Japanese
irisout 0.3.8を公開した。npmから取れるまでに導入検査の待ち時間を使い切ったので、取得できるようになってから公開ジョブだけ再実行した
I switched the LM's output layer to adaptive softmax and used the freed-up compute to widen the main body. Frequent words reached the same loss as a model with about twice the compute, but shrinking rare words to 28 dimensions caused a big loss, so overall the difference is…
Translated from Japanese
Summarized the changelog for irisout 0.3.8, and tried putting the packaged artifacts into a separate Hono app. Verified initial render and state updates on the dev server, and also tested the operations that use the API on the production server.
Translated from Japanese
Allowing refined types to be written on values coming from outside was being treated as if the condition had been checked, so I closed that off. When I traced the direction in which values flow in, the callback arguments were the same.
Translated from Japanese
I tried distilling a width-256 LM from a width-768 one, but when I measured it with a ruler calibrated for the changed width, it only came out to the equivalent of 1.22x the compute. There isn't much room left to draw out of a small model through training tricks, and the capacity…
Translated from Japanese
If there was an apostrophe in the body of the macro, it was stopping at the string reading before reaching the macro. When I tried placing a normal string next to it, there were also cases where only the lexical analysis passed.
Translated from Japanese
I couldn't use int conditions for number division and remainder, so I made it also check the conditions before conversion. Without Z3, the outputs of the 4 methods matched.
Translated from Japanese
fla.modules.FusedLinearCrossEntropyLossが5060Tiで94msかかってて、1stepの6割が語彙の出力だった。torch.compileした素直なCEだと44msになるけど、ベンチのブレが0.005から0.157に広がって判定できず。段階1はここで止める
KDAを局所注意に置き換えて3-gram表を外した構成、文脈8192で全文脈attentionの0.896/0.897/0.901倍だった。目標0.90を3回目だけ0.001超えて不採択。乱数入力だとn-gram表の時間がブレるので実データで測ることにしたのは正解だった
言語モデルの比較計画を書いた。学習した長さと評価する長さをそろえて、今使っているモデルにも勝てるかを測りたい
言語モデルを学習時の4倍の文脈で比べた。予測の差が広がったので同じ文章を短く区切って見直したら、差の大半は比較相手の損失が上がった分だった
hibanaのビルドスクリプト用にcompile_fileとwrite_bytesだけ戻した。hibana build.hb を実行するとJS・WASM・ネイティブを書き出せる。生成側のコードから呼ばれてたらコンパイルエラーにしてる
hibanaで割り算の割る数が0じゃないことを型検査で証明させるようにした。添字の範囲の証明がもうあったのでそれに乗せるだけで済んだ。Z3なしの証明器だと i >= 2 から i != 0 が出せなかったので 0 < i も試すようにしたのと、number の != 0 も示せるようにした
In hibana, there were cases where just changing the name of a trait changed which method gets called. When there are multiple candidates, I made it report an error down to the declaration's position.
Translated from Japanese