perf: in-place to_json, integer-valued double fast path, SWAR integer parsing, in-register escape handling - #2799

Three independent optimizations to the reflection serializer and on-demand integer parsing. Output is byte-identical to master in every benchmark and fuzz case; each change is motivated by a profile and ships with a dedicated correctness harness.

Rebased on current master (#2795, #2798, #2804, #2805 all merged since the first revision; everything below is re-measured against b3072d2, 5 alternating co-built rounds, medians). Following the escaper comparison in the comments and the interest there, this PR now also carries a fourth change: the in-register escape handling grafted into #2795's escape_block structure. Details in section 4.

_{gcc 16.1, aarch64/NEON, -O3. Baseline and patched trees co-built in one container, 5 alternating rounds, medians (same-invocation A/B only; cross-run numbers drift on this hardware). twitter deserialize and the reused-string_builder serialization paths are unchanged (±noise).}

1. to_json(value, std::string&) builds in place

Profile of to_json on twitter (taken before #2795; the copy share only grows with the faster escaper): ~43% of the time goes to copying or managing bytes that were already produced:

write_string_escaped  █████████████████░░░░░░░░░  41%
memcpy / growth       ███████████░░░░░░░░░░░░░░░  28%   ← this change
to_json wrapper       ██████░░░░░░░░░░░░░░░░░░░░  15%   ← this change

The old implementation builds in a private buffer (doubling from 1 KB, each doubling a new[]+memcpy), then copies the whole result again into the destination:

// before: every output byte written ~3x
string_builder b(initial_capacity);
append(b, z);
s.assign(view);          // full copy of the output

// after: the destination's storage is the build buffer
string_builder b(s, initial_capacity);   // new string-backed mode
append(b, z);
s.resize(view.size());   // shrink, no copy; capacity kept for the next call

A caller that reuses the string pays zero allocations and zero copies, which is the common production pattern. to_json_string builds into a local and move-returns. The owned mode is untouched on its hot path (the two new members are cold; a cached buf pointer keeps codegen branch-free).

Caveats: destination must not alias the serialized value (documented); under -fno-exceptions, std::string growth failure aborts exactly as it would in caller code.

2. Integer-valued doubles take the integer writer

double fields holding exact integers (ids, counts, epoch timestamps) are everywhere in real JSON, and the shortest round-trip formatting of such a value below 10^15 is just its integer digits plus ".0":

if (d > -1.0e15 && d < 1.0e15) {          // false for NaN, so the cast is safe
  int64_t iv = static_cast<int64_t>(d);
  if (static_cast<double>(iv) == d && !(iv == 0 && std::signbit(d))) {
    p = internal::write_uint_jeaiii(p, mag);   // ~5x faster than the float path
    *p++ = '.'; *p++ = '0';

Previously ~2.4× behind yyjson on this data class, now slightly ahead. Full-precision doubles pay 1–2% for the check.

The bound is 10^15, not 2^53: to_chars switches to scientific notation at 16 significant digits. An exhaustive identity sweep (2,403,929 cases: powers of 2/10, trailing-zero grids, the ±2^53 band, random integer-valued doubles) proves the fast path byte-identical to to_chars over the whole gate. The sweep caught an earlier 2^53 version of this patch and is worth keeping as a test.

Bundled fix: with SIMDJSON_ENABLE_NAN_INF=0 (the default), append(NaN) used to reach to_chars and emit 2.696539702293474e+308, valid JSON with silently wrong data. It now emits null. Happy to switch to returning an error if preferred.

3. SWAR fast path for get_int64 / get_uint64

Profile of citm deserialize (baseline): stage1 40%, struct deserialization 32%, and inside the latter, digit-at-a-time integer parsing. The code declines SWAR with a comment: “we don't use is_made_of_eight_digits_fast because large integers like 123456789 are rare.” In id-heavy documents they are the norm:

// gate = the validity check itself: one 8-byte load (in bounds via
// SIMDJSON_PADDING); short numbers fall through to the byte loop unchanged
if (simdjson_likely(!is_made_of_eight_digits_fast(src))) { return parse_unsigned(src); }
uint64_t i = parse_eight_digits_unrolled(src);           // 8 digits per step
if (is_made_of_eight_digits_fast(src + 8)) { i = i * 100000000 + ...; }

9/16/19-digit integers: +49% / +101% / +81% in isolation; citm deserialize +5.8%; twitter neutral. On targets that set SIMDJSON_SWAR_NUMBER_PARSING to 0 (big-endian, where parse_eight_digits_unrolled is not safe) both functions fall back to the byte loop. Trade-off, disclosed: documents of exclusively 2-5-digit integers measure −8 to −13% on the gate in a synthetic micro; both real-document benchmarks are net positive or neutral.

Correctness: a differential fuzz (15,528 adversarial cases: int64/uint64 boundaries ±1, 20-digit overflows, leading zeros, adjacent numbers straddling the 16-byte window, non-minified spacing) produces bit-identical (value, error) pairs against baseline.

4. Escape-heavy blocks are consumed in registers

Follow-up to the escaper comparison in the comments (and thanks @fior512 for the offer to help; this is the merge of the two approaches). #2795's structure is unchanged for clean blocks and sub-16-byte strings. When a block does contain escapes, it is now blind-stored and its whole mask is consumed in registers: after each escape expansion the remaining lanes are repositioned with a shuffle (vqtbl1q_u8 on NEON, a one-time spill on SSE2) instead of escape_block's per-run copies.

escape_store16(out, word);                    // clean prefix lands in place
out = escape_block_hot(word, src, out, i, escape_bitmask(flags));
// hot: per escape, emit + reposition the tail from the register; one load
// and one store per 16 input bytes at any escape density

Escape-heavy strings gain +32-59% at every length (one escape per 16 bytes); twitter serialization gains ~6% across the builder variants. Disclosed trade-off: long strings with sparse escapes (one per ~333 bytes) measure 5-11% slower in a synthetic sweep, because the out-of-line hot call re-materializes the vector constants per escape-bearing block; documents like twitter/citm come out ahead (see table). I explored keeping the hot path inline plus a length split (four variants measured); each traded a different band. Happy to share that data in a follow-up if there is appetite for per-block strategy selection.

The blind store needs one vector of slack past the worst-case expansion, so the escape capacity checks move from 2 + 6*len to 6*len + 18 (and analogues at all five call sites); escape_tight_capacity_boundary exercises exactly those budgets under ASan, alongside #2804's exhaustive escaper test which also passes.

Testing. The second commit adds escape_tight_capacity_boundary to builder_string_builder_tests: dense escape patterns at every block-boundary length/offset, built with initial_capacity = 1 so every growth lands on an exact pos+needed boundary, byte-compared against a scalar reference. It guards the #2795 escaper's capacity handling and passes under ASan. The other harnesses (double-format identity sweep, number-parse differential fuzz) are standalone; glad to contribute them as unit tests too.

Scope. All numbers are gcc 16 / aarch64; x86 validation welcome. The three changes are independent, happy to split if easier to review.