Skip to content

Support mruby 4.0 (RITE0400) bytecodeFeature/rite0400 - #88

Closed
bash0C7 wants to merge 21 commits into
mrubyedge:masterfrom
bash0C7:feature/rite0400
Closed

bash0C7 wants to merge 21 commits into
mrubyedge:masterfrom
bash0C7:feature/rite0400

Conversation

@bash0C7

@bash0C7 bash0C7 commented Sep 10, 2026

Copy link
Copy Markdown

Title: Support mruby 4.0 (RITE0400) bytecode

Loads and runs the RITE0400 chunks emitted by mruby 4.0. The opcode table is picked from the RITE header's major version ("03" / "04"), so 3.x behaviour is untouched; any other major is rejected with rite::Error::UnsupportedVersion.

Area Change
New opcodes GETIDX0 MATCHERR SSEND0 SEND0 BLKCALL RETSELF RETNIL RETTRUE RETFALSE ADDILV SUBILV TDEF SDEF
Renamed opcodes LOADI8 / LOADTRUE / LOADFALSE map onto the existing LOADI / LOADT / LOADF implementations
ENTER n1 flag (MRB_ASPEC_NOBLOCK): passing a block to def f(&nil) raises ArgumentError
Prelude NoMatchingPatternError under StandardError, raised by MATCHERR
Fixtures The harness compiles with mec-mrbc-sys 3.3, which cannot emit 4.0 chunks, so tests/fixtures/mruby40.mrb is compiled with mruby-compiler2 and checked in; per-opcode coverage uses hand-built IREPs
Still unimplemented The 17 opcodes already unimplemented on master (ARGARY, ASET, GETCV, EXT1-3, …) stay unimplemented in 4.0 as well
Before After
rite::insn::FETCH_TABLE RiteVersion::{V3,V4}::decode(byte) -> (OpCode, FetchFn)
OpCode, 106 variants 119 variants; TryFrom<u8> keeps the 3.x numbering
rite::Error::UnsupportedVersion, rite::Rite.version
debug_eval_insn(insns) debug_eval_insn(insns, version)
IREP unchanged

@udzura

udzura commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

docs/table.html の更新も考慮していただき、大変助かります。 👍

Comment thread mrubyedge/tests/fixtures/mruby40.rb Outdated
@@ -0,0 +1,67 @@
# Reaches every opcode mruby 4.0 added except MATCHERR; compiled into mruby40.mrb, see tests/rite0400.rs.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

「ファイルを見ればわかる」という状態を維持したいので、fixtureを作らないでください。
mrbのようなバイナリファイルもこれ以上追加しないでください。

Comment thread mrubyedge/tests/opcodes40.rs Outdated
@@ -0,0 +1,791 @@
// The opcodes mruby 4.0 (RITE0400) added, as hand-built IREPs, since the harness compiles with mruby 3.3.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mrubyedge/tests の配下は、以下のフォーマットのテスト形式を守ってください。

  1. Rubyのコードを直接記述する。
  2. そのコードをコンパイルする。
  3. それをロード、実行する(toplevel、関数呼び出しどちらも許容)。
  4. 返却値をRustのバリューにして検査する。

今回は、対応するバイトコードを出力するRubyコードがテストケースになると思います。

IREPを直接記述するのは原則禁止です。

(例外もある可能性はありますが、それらは古いテストで置き換え予定、またはしかたないエッジケースのはず)

Comment thread mrubyedge/tests/rite0400.rs Outdated

use mrubyedge::rite::insn::{OpCode, RiteVersion};

const MRUBY40: &[u8] = include_bytes!("fixtures/mruby40.mrb");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

上のコメントの通り、直接テストコードにRubyコードを記述してください。 include_bytes! は必要ないはずです。

@udzura

udzura commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

@bash0C7 bashさんの式神(AI)へのレビューコメントをしたので確認お願いします!
ご面倒かと思いますが🙏

@udzura

udzura commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

ドキュメントがないのはすいません(このPRでAIの指示を追加しても構いません)

bash0C7 and others added 5 commits September 12, 2026 00:23
mruby 4.0 renumbered the opcode table from index 36 and added thirteen
opcodes, so a 4.0 chunk cannot be decoded with the 3.x numbering and a 3.x
chunk cannot be decoded with this one. rite::load now refuses any header
that does not say 04, and the tests compile their Ruby with
mruby-compiler2, which emits it.

The table and the opcodes have to land together: 4.0's codegen reaches
TDEF, SSEND0 and the RET* family for ordinary Ruby, so the existing suite
would hit unimplemented arms if either half came first. The same goes for
three paths 4.0's codegen walks that 3.3's did not:

- a rescue handler opens with GETGV $! rather than EXCEPT, and reading a
  global that was never assigned answered an internal error instead of nil
- a single begin/ensure emits three overlapping catch handlers, so the
  table has to keep each handler's range and be searched the way vm.c's
  catch_handler_find does, at the raise, innermost first
- the ensure path runs RESCUE against the nil that EXCEPT leaves behind,
  where the VM assumed an exception

MATCHERR raises NoMatchingPatternError, which the prelude now defines
under StandardError.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
mruby 4.0 widened ENTER's operand to 24 bits and put MRB_ASPEC_NOBLOCK in
the new top bit, which the compiler sets for def m(&nil). vm.c raises
ArgumentError "no block accepted" when a block arrives anyway.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
consume_ensure_block stashed every error into vm.exception and carried on to
the RAISEIF it stops at, so op_jmpuw returned Ok, the run loop never started
unwinding, and the catch handler search was skipped. The exception surfaced
as a Rust error out of vm.run() instead of being rescued by the Ruby around
it. mruby aborts the clause at the raise.

Real exceptions now leave as Err, which is the ingress the run loop already
has for them. Error::Break and Error::BlockReturn keep the old handling:
they are control flow, not exceptions, and the Break arm reads a breadcrumb
that the ensure clause never pushed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
examples/hi.mrb and examples/simple.mrb are both RITE0300, so rite::load
refuses them now. Nothing but the docs read them, their names do not match
their contents, and their sources are not in the tree.

The doc example and the README read a path instead of bundling a blob, and
the IREP example drops no_run so cargo test --doc runs it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
bash0C7 and others added 16 commits September 12, 2026 14:43
A prefix widens the next instruction's first operand, its second, or both.
Skipping one and reading the instruction that follows at the narrow width
misreads every byte after it, so a chunk with more than 255 registers or
symbols was decoded into a different program: a 300-symbol source reached
ARYPUSH, an opcode it never contained, and a 250-local one reached a byte
that is not an opcode at all.

fetch_next consumes the prefix as part of the instruction it belongs to,
following the FETCH_*_1/_2/_3 rules in mruby's include/mruby/opcode.h. The
operand table becomes a table of shapes so one fetch serves every width, and
Fetched carries u16 where an operand can be widened. Registers grow past 256
on demand, because a frame that needs EXT1 needs more registers than the
fixed array had.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
The record is a one-byte length followed by that many bytes plus two, but
the loader read a two-byte big-endian length and then that many bytes
again. One ordinary integer literal wide enough to need a bignum made
rite::load walk off the end of the buffer and panic, in a function whose
signature says it returns an error.

mrbc 3.3 was built without MRB_USE_BIGINT, so no chunk the old test
harness compiled ever carried the tag. mruby-compiler2 emits it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
The test named for the both-operands prefix put the EXT1 body and the
EXT2 body next to each other at the top level, which emits those two
prefixes separately and never the third. It passed without covering
anything.

A prefix that widens both operands needs one instruction whose register
and symbol numbers are each above 255, so the body is a method holding
250 locals and 256 instance variables. Breaking the EXT3 arm of
fetch_shape now fails this test and no other.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
dump.c writes the value as two big-endian 32-bit halves, high first,
and load reads it back the same way, so the record is one big-endian
i64. The loader read it little-endian, and every integer literal too
wide for 32 bits came out as a different number: 4294967296123 ran as
8863084070557646848.

The existing pool test asserted the tag and not the value, so nothing
caught it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
read_binary_header refuses a chunk whose first four bytes are not RITE
and one whose minor version is newer than the format this reads, then
bounds the section scan by the size the header carries. The loader
checked only the major version and walked to the end of whatever buffer
it was handed, so a chunk with bytes appended after the end marker was
rejected and one with a corrupt ident was accepted.

An unknown section ident now ends the scan and returns what was read,
which is what read_irep does; it used to be an error, and one that
printed the whole buffer to stderr on the way out.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
Both were read as C strings, so a zero byte anywhere inside one ended
the value early and the loader rejected the chunk as malformed. mruby
writes the length first and copies that many bytes; "a\0b" is an
ordinary string literal and its chunk would not load.

Pool strings and symbol names become byte vectors, which is what they
are on the wire. Local variable names in the LVAR section keep their
C string type: that section stores names, not values.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
Class variables, a constant assigned through a scope, a symbol built
from an interpolated string, the two array splat forms, the two hash
spread forms and a bare super all compile to opcodes that had no arm
and stopped the program with unimplemented!.

Modules gain a place to keep class variables, which they did not have.
ARGARY reads the argument shape out of its packed operand and collects
the frame's arguments, walking the environment chain when the super is
inside a block; SUPER learns the argument count of 15 that means those
arguments arrived as one array.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
interpret_insn returns a Result and then called unimplemented! for any
opcode without an arm, so a chunk carrying one took the process down
rather than handing the caller an error. The seven opcodes still
without an arm are ones mruby's compiler never emits, but a chunk from
elsewhere can hold them.

Also covers the opcodes the suite compiled but never reached: a constant
assigned at the top level, a singleton class body, and a method defined
after 256 symbols, where the compiler stops folding the definition into
TDEF and emits TCLASS, METHOD and DEF.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
The branch had accumulated thirty lines of prose explaining code that
the code and the test names already say, a COVERAGE row describing the
test files rather than what the crate supports, a note in Cargo.toml
about a dependency that the dependency line states, and a doctest that
had quietly been switched from no_run to running.

One comment in docs/table.html pointed at FETCH_TABLE, which this
branch deleted; it goes back to the single line master had.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
The opcode table still marked the nine this branch implemented as
missing, and neither it nor COVERAGE said anything about the five that
stay unimplemented or why. Five have no site in mruby's code generator,
or a site the parser never lets through, so no Ruby reaches them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
Every length in the IREP and LVAR sections was read and then used to
slice the buffer unchecked, so a chunk whose length field had been
damaged took the process down from inside a function that returns a
Result. Setting any two adjacent bytes of a small chunk to 0xff found
thirty-nine such positions.

The reads go through helpers that return TooShort instead. This also
covers the symbol length of 0xffff that mruby writes for an empty
symbol slot: its code generator never leaves one, so no Ruby produces
a chunk carrying it, and the loader no longer dies if one arrives.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
Two opcodes mruby 4.0 emits took the process down.

RETURN_BLK carries a return out of a block, and the compiler emits it
for any return that leaves a begin or a loop, block or not. The handler
asked for the enclosing block's environment and unwrapped it, so
`def m; begin; return 1; ensure; 2; end; end` crashed. mruby returns
normally when the frame has no environment.

SDEF defines a singleton method on any receiver, but the singleton
class was only ever built for a class, so a module could hold no
singleton methods: `module M; def self.x; end; end` crashed, and so did
a `class << self` body inside one. Modules now keep the same back
reference to their singleton class that classes keep.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
Two scripts and the tests that drive them.

coverage/opcodes.rb holds 51 standalone programs. Between them they
compile to 112 of the 119 opcodes mruby 4.0 numbers; the other seven
are named at the top as unreachable, because the code generator has no
site that emits them or the parser rejects the shape that would. Each
section names the opcodes compiling it emits and the value running it
gives, and the test checks both, so a section that stops covering what
it claims fails instead of passing quietly.

coverage/language.rb is the sibling: 98 programs over the control
structures, argument shapes, class and module forms, literals and core
methods. Five are marked broken, each naming the answer Ruby gives and
the answer this VM gives instead. The test holds those to the wrong
answer, so fixing one fails the test and says to move it back.

Both files are valid Ruby end to end; the markers are comments. The
tests read them at run time rather than embedding them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
The review asks that everything under mrubyedge/tests write its Ruby
directly. These two are whole Ruby programs that an editor should treat
as Ruby, so they sit beside the crate instead and the tests read them
at run time.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
Two opcodes read their operands wrongly and took the process down.

A call with a splat or a double splat arrives as one array or one hash
with an argument count of 15, which the send path read as fifteen
registers. `pair(*spread)` walked off the frame. The count is now read
for what it means and the packed values are laid out one per register,
growing the register file when the caller's frame was sized for the
packed form.

BLKPUSH takes the same packed operand ARGARY does, naming where the
block sits and which frame holds it. The handler ignored the operand,
took the count off the callinfo and unwrapped it, so a yield inside a
block -- where there is no callinfo -- crashed instead of reaching out
to the enclosing method's block.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
Ten sections of coverage/language.rb name an answer this VM does not
give. Each has its own test carrying the reason and marked ignored, so
an ordinary run counts them as skipped and names them, and
`cargo test -- --ignored` checks every one is still wrong. Fixing one
fails its test and says to move the section back.

Five are new: a parameter after a rest parameter, a multiple
assignment with a rest in the middle, a constant two namespaces deep,
! on false, and next inside an ensure.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FfoC7uMVr3d3VaxWFnaPFT
@udzura

udzura commented Sep 13, 2026

Copy link
Copy Markdown
Collaborator

本PRなのですが、変更が大きくなりすぎ、なおかつメンテナが全てを理解するのが難しい状態になっているので、以下の方針で対応します。

  • 適切な粒度で収まっているコミットからブランチを切り直す
    • おそらく 445635d が良さそうですが今一度検討します
  • RITE0400 のバイトコード対応と、そのバイトコードが生成されるコードのテストの内容に止める
    • その際、稀にしか出てこないバイトコードであればテストをオミットし、別途対応する可能性がある

また、本来対応しなといけない箇所をまとめて対応していただいてるのも観測していますが、1つのPRにまとめるべきではないと考えています。
変更箇所後日参照し、cherry-pickあるいは翻案して、 each 要対応案件ごとに取り込む予定です。

「each 要対応案件ごと」のPRを切り出して作成していただくのはもちろん大歓迎です。図々しいのですが。

本PRはクローズします。

@udzura udzura closed this Sep 13, 2026
@bash0C7

bash0C7 commented Sep 13, 2026

Copy link
Copy Markdown
Author

お手数おかけします。勇み足でした🙏
小さい単位では機会みて個別にPR出させていただきます。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants