Skip to content

string.h: hold a string's encoding as an index, not as one bit #7169

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
matz merged 1 commit into from
Aug 14, 2026
Merged

string.h: hold a string's encoding as an index, not as one bit #7169

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions include/mruby/string.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,12 +51,22 @@ struct RStringEmbed {
#define MRB_STR_EMBED_LEN_BITS 5
#define MRB_STR_EMBED_LEN_MASK (((1 << MRB_STR_EMBED_LEN_BITS) - 1) << MRB_STR_EMBED_LEN_SHIFT)

#define MRB_STR_BINARY 16
#define MRB_STR_SINGLE_BYTE 32
/* bits 6..10 are the embedded length, so 11 is the first one free */
/* bit 4 is free, and bits 6..10 are the embedded length, so 11 is the first
one free above it */
#define MRB_STR_VALID_ENC 2048
#define MRB_STR_BROKEN_ENC 4096

/* Where in the flags word the encoding index sits. Two bits name four
encodings, which is more than the two a build carries now; widening them is
for whenever a third is carried. They sit above the flags rather than in
bit 4, the bit the index leaves behind, because a pair needs two bits in a
row and MRB_STR_SINGLE_BYTE holds the one beside it. Moving them down is
for whenever the word is laid out afresh. */
#define MRB_STR_ENCODING_SHIFT 13
#define MRB_STR_ENCODING_BITS 2
#define MRB_STR_ENCODING_MASK (((1 << MRB_STR_ENCODING_BITS) - 1) << MRB_STR_ENCODING_SHIFT)

Comment thread
coderabbitai[bot] marked this conversation as resolved.
#define RSTR_EMBED_P(s) ((s)->flags & MRB_STR_EMBED)
#define RSTR_SET_EMBED_FLAG(s) ((s)->flags |= MRB_STR_EMBED)
#define RSTR_SET_EMBED_LEN(s, n) do {\
Expand Down Expand Up @@ -138,10 +148,10 @@ struct RStringEmbed {
#endif

#define RSTR_ENCODING(s) \
(((s)->flags & MRB_STR_BINARY) ? MRB_STR_ENCODING_BINARY : MRB_STR_ENCODING_DEFAULT)
(((s)->flags & MRB_STR_ENCODING_MASK) >> MRB_STR_ENCODING_SHIFT)
#define RSTR_ENCODING_SET(s, e) \
((s)->flags = ((s)->flags & ~MRB_STR_BINARY) | \
(((e) == MRB_STR_ENCODING_BINARY) ? MRB_STR_BINARY : 0))
((s)->flags = ((s)->flags & ~MRB_STR_ENCODING_MASK) | \
((e) << MRB_STR_ENCODING_SHIFT))
#define RSTR_BINARY_P(s) (RSTR_ENCODING(s) == MRB_STR_ENCODING_BINARY)
/* A copy of a string is read the way the string it copies is, so the encoding
travels with the bytes rather than being left behind on the original. */
Expand Down
2 changes: 1 addition & 1 deletion mrbgems/mruby-string-ext/src/string.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ int_chr_binary(mrb_state *mrb, mrb_value num)
it the byte spells no UTF-8 character at all, so the string is not one
character per byte and saying that it is left every reader of the flag
handing the byte back as a character. What it is instead is a string read
by bytes, which is what MRB_STR_BINARY says. */
by bytes, which is what MRB_STR_ENCODING_BINARY names. */
if (cp < 0x80) {
RSTR_CODERANGE_SET(mrb_str_ptr(str), MRB_STR_CODERANGE_7BIT);
}
Expand Down
Loading