string.h: hold a string's encoding as an index, not as one bit - #7169

RSTR_ENCODING() and RSTR_ENCODING_SET() name an index into the encodings a build carries, but what they read and write is a single bit:

#define RSTR_ENCODING(s) \
  (((s)->flags & MRB_STR_BINARY) ? MRB_STR_ENCODING_BINARY : MRB_STR_ENCODING_DEFAULT)
#define RSTR_ENCODING_SET(s, e) \
  ((s)->flags = ((s)->flags & ~MRB_STR_BINARY) | \
                (((e) == MRB_STR_ENCODING_BINARY) ? MRB_STR_BINARY : 0))

With two encodings the name and the thing agree. With a third they part: the setter asks whether it was handed MRB_STR_ENCODING_BINARY, so an index it does not know for compiles and comes back as the default, at every site that writes it and with nothing to catch it.

This PR gives the index bits of its own. What a build carries does not change: binary, and UTF-8 where MRB_UTF8_STRING is set.

What goes into include/mruby/string.h

#define MRB_STR_ENCODING_SHIFT 13
#define MRB_STR_ENCODING_BITS 2
#define MRB_STR_ENCODING_MASK (((1 << MRB_STR_ENCODING_BITS) - 1) << MRB_STR_ENCODING_SHIFT)

#define RSTR_ENCODING(s) \
  (((s)->flags & MRB_STR_ENCODING_MASK) >> MRB_STR_ENCODING_SHIFT)
#define RSTR_ENCODING_SET(s, e) \
  ((s)->flags = ((s)->flags & ~MRB_STR_ENCODING_MASK) | \
                ((e) << MRB_STR_ENCODING_SHIFT))

The three constants take the shape MRB_STR_EMBED_LEN_SHIFT / _BITS / _MASK already use for the other field in the word.

MRB_STR_BINARY goes with the bit. Since 4848467 put the last writers on the accessors, every mention of it on master is the definition, the two accessor bodies above, and one comment. Nothing reads it and nothing writes it:

$ git grep -n MRB_STR_BINARY        # on master
include/mruby/string.h:54:#define MRB_STR_BINARY    16
include/mruby/string.h:141:  (((s)->flags & MRB_STR_BINARY) ? MRB_STR_ENCODING_BINARY : MRB_STR_ENCODING_DEFAULT)
include/mruby/string.h:143:  ((s)->flags = ((s)->flags & ~MRB_STR_BINARY) | \
include/mruby/string.h:144:                (((e) == MRB_STR_ENCODING_BINARY) ? MRB_STR_BINARY : 0))
mrbgems/mruby-string-ext/src/string.c:27:     by bytes, which is what MRB_STR_BINARY says. */

$ git grep -n MRB_STR_BINARY        # on this branch
$

RSTR_BINARY_P() reads the field now, and the comment names MRB_STR_ENCODING_BINARY.

Two bits hold four encodings, twice what a build carries. Widening them is for whenever a third is carried; this PR is about the shape of the answer, not about the set of answers.

Where the field sits

The field sits above the flags rather than in bit 4 that it leaves behind: a pair needs two bits in a row and MRB_STR_SINGLE_BYTE holds bit 5. flags is 20 bits wide (include/mruby/object.h), so the pair is inside it with five to spare.

Where the field finally sits is for whenever the word is laid out afresh, once the three coderange bits fold into a field of their own and bits 4 and 5 are a pair again.

What does not change

Every write writes what it wrote. The callers hand RSTR_ENCODING_SET() an index already, and the two indices they hand it land in the field as the bit landed in bit 4, so no call site changes and no string answers anything different.

Index 0 is still what a zero filled string says, so the path every string is made on stores nothing. RSTR_BINARY_P() still answers 1, and its readers are untouched. The index still stops short of the dump format: src/dump.c writes IREP_TT_STR and the bytes and no string flags at all.

Outside the header, the one line that changes is a comment in mruby-string-ext that named MRB_STR_BINARY.

MRB_STR_BINARY leaves the public header

include/mruby/string.h is a public header, so an out of tree gem that names the bit stops compiling. That is the point: the bit is gone rather than renamed, and code that wrote it wrote into a place the field no longer reads. An alias would have to be either 16, which now names a bit nothing looks at, or MRB_STR_ENCODING_BINARY, which is 1 and cannot be OR'ed into flags. Either one keeps flags |= MRB_STR_BINARY compiling while it stops meaning anything, which is the failure this PR is written to prevent. A compile error names the lines to change.

Names in this family have left the header before: RSTR_SET_BINARY_FLAG() and its two companions in 4848467, the unused unset macros in c18cd60, and the ASCII flag renamed to MRB_STR_SINGLE_BYTE in 57fd0ed.

Generated code

gcc -O3, against the same objects built from master. Each of the four builds ci/gcc-clang makes:

The objects that differ are the ones that name the accessors: src/string.o, mruby-string-ext, mruby-sprintf, mruby-regexp, mruby-string-bitops, and mruby-encoding where the build carries it. Comparison is objdump -d over every .o in the build; every other object is identical instruction for instruction.

The growth is the immediate. On x86-64 a mask below bit 8 fits one byte, and this one does not:

   and    $0x10,%eax          before, 3 bytes
   and    $0x6000,%eax        after,  5 bytes

so each read and each write grows a few bytes. Moving the field into bits 4 and 5 later takes it back under a byte.

Testing

rake -m test over ci/gcc-clang, all four builds green, 0 KO, 0 crash, 0 warnings:

byte-string is the build without mruby-encoding, where strings index by byte. Binary cannot share index 0 with the default even there, since String#b and Integer#chr mark strings binary and mruby-regexp reads the mark to decide whether to walk a subject a byte at a time; the two indices stay apart, and that build's String#b and regexp tests are what pin it.

No test accompanies the change. No string answers anything different, so there is nothing new to pin.

Not in this PR

The coderange bits do not move, the set of encodings a build carries does not grow, and the field does not go to its final position.