symbol.c: copy the name of a symbol that symbol GC can free - #7020
Merged
Conversation
`Symbol#to_s` and `Symbol#name` hand the symbol table's name buffer to
`mrb_str_new_static()`, which for a name too long to embed builds a
`RSTR_NOFREE` string pointing straight at that buffer. Symbol GC does not
treat such a string as a reference to the symbol: `sym_gc_mark_object()`
looks for `mrb_symbol_p()` values in classes, ivars, arrays, hashes, env
and the stack, and `MRB_TT_STRING` falls into its `default: break;`. A
string is the one kind of object that can keep a symbol's name alive
without holding the symbol, and it is exactly the case that is not
scanned, so the sweep frees the buffer under a live string.
```ruby
str = ("q" * 40).to_sym.to_s
6000.times { |i| "junk-symbol-with-a-long-name-#{i}".to_sym }
GC.start
a = []; 3000.times { |i| a << "Z" * 60 + i.to_s }
p str
# CRuby: "qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq"
# mruby: "junk-symbol-with-a-long-name-4615\x00qqqqqq"
```
Copy the name when the symbol is one symbol GC can reclaim, and keep the
zero copy path for everything else. Only a dynamic symbol gets an
individual `mrb_malloc()`; a presym name is static data, a literal name
comes from the symbol pool and an inline symbol carries its name in the
value, so none of those is ever freed. A name short enough to embed was
already copied by `str_init_embed()`, so the cost is one allocation per
`to_s` of a long dynamic symbol, and symbols from source literals keep
sharing the buffer.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughDynamic symbol names are now copied before symbol garbage collection can reclaim their storage. A regression test verifies that ChangesDynamic symbol name ownership
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Symbol#to_sandSymbol#namereturn a string that shares the symbol's name buffer,which symbol GC can free.
For a dynamic symbol whose name is longer than
RSTRING_EMBED_LEN_MAX,Symbol#to_sreturns a
RSTR_NOFREEstring pointing straight into the symbol table's name buffer.Symbol GC does not treat such a string as a reference to the symbol, so it can sweep the
symbol and
mrb_free()the buffer while the string is still alive. Every later read ofthat string is a use after free.
"qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq""junk-symbol-with-a-long-name-4615\x00qqqqqq"The exact garbage varies by run: it is whatever allocation happened to reuse the block.
Here the freed buffer was taken by a later symbol's packed name, so the string reports
that symbol's name plus the packed length byte.
Symbol#namehas the same problem, and so does anything that keeps the result ofmrb_obj_as_string()for a symbol.MatchDatamakes it easy to hit by accident, becauseit keeps the subject string for the lifetime of
$~:Cause
mrb_sym_str()hands the raw name pointer tomrb_str_new_static(), andstr_new_static()copies only when the name fits inline:That was safe while symbol names lived forever. It no longer is: a dynamic symbol gets an
individual
mrb_malloc()and symbol GC frees it in phase 3 ofmrb_symbol_gc():Nothing connects the two.
sym_gc_mark_object()walks classes, ivars, arrays, hashes, envand the stack looking for
mrb_symbol_p()values, andMRB_TT_STRINGfalls into itsdefault: break;. A string is the one kind of object that can keep a symbol's name alivewithout holding the symbol, and it is exactly the case that is not scanned.
The name length is what decides between safe and corrupt, at
RSTRING_EMBED_LEN_MAX(27 on a 64-bit build):
"qqqqqqqqqqqqqqqqqqqqqqqqqq""qqqqqqqqqqqqqqqqqqqqqqqqqqq""junk-symbol-with-a-long-name"MRB_SYMBOL_MAXdefaults to 4096, so a default build is affected; onlyMRB_SYMBOL_MAX 0is immune. Presym and literal symbols are safe, because their names are either static data
or pool allocated and are never individually freed.
Fix
Copy the name in
mrb_sym_str()andsym_name()when the symbol is one symbol GC canreclaim, and keep the zero copy path for everything else. Only a dynamic symbol gets an
individual allocation: a presym name is static data, a literal name comes from the symbol
pool and an inline symbol carries its name in the value, so none of those is ever freed.
The cost is one allocation per
to_sof a long dynamic symbol. Short names were alreadycopied by
str_init_embed(), and symbols from source literals, which is nearly all ofthem in practice, keep sharing the buffer.
Alternatives considered
Marking from the string side would mean recognising, during symbol GC, that a
RSTR_NOFREEstring's pointer lands inside the symbol table, then mapping it back to a symbol index.
That is a linear scan per string over
mrb->symtblfor a case that copying avoidsoutright, and it does not compose with
str_init_nofree()strings that legitimately pointat static data.
Freezing the returned string does not help either: the problem is the lifetime of the
buffer, not mutation of the string.
Where the fix belongs
This came up twice in review, on #6993 and on #6995, both times as "a
MatchDataholding asymbol derived string can outlive the symbol". Neither PR was the right place. The same use
after free reproduces with
Symbol#to_salone and no regexp anywhere, and patchingmatch_operand()in mruby-regexp would close theRegexp#match(:sym)route while leavingSymbol#to_s,Symbol#name,"#{sym}"and every othermrb_obj_as_string()caller open.Verification
test/t/symbol.rbgains a test that fails on master and passes with the patch:rake testpasses on a plain host build (1943 assertions, 0 KO, 0 crash) and onbuild_config/clang-asan.rb(2120 assertions, 0 KO, 0 crash).Under ASan the use after free is reported directly. The shortest repro needs a small
MRB_SYMBOL_MAXso the sweep runs early, and in that shape the tombstone guard from #7018has to be in place first, otherwise the null deref in
migrate_to_hash_table()landsbefore this bug can be read back. With
MRB_SYMBOL_MAX=16and that guard applied:With the patch the same build prints
"dynamic-target-name-aaaaaaaaaaaa"and ASan issilent. The two changes are independent; only that shortcut repro is shared.
Summary by CodeRabbit
Bug Fixes
Symbol#to_sandSymbol#nameso returned strings remain valid after dynamic symbol garbage collection and memory reuse.Tests