Skip to content

Encode setBody(String) bodies with String.getBytes - #2371

Merged
hyperxpro merged 1 commit into
AsyncHttpClient:mainfrom
maygemdev:claude/string-body-getbytes-40977c
Nov 5, 2026
Merged

hyperxpro merged 1 commit into
AsyncHttpClient:mainfrom
maygemdev:claude/string-body-getbytes-40977c

Conversation

@pavel-ptashyts

Copy link
Copy Markdown
Contributor

Summary

  • setBody(String) request bodies are now encoded with body.getBytes(charset) instead of charset.encode(CharBuffer.wrap(body)).
  • The bytes sent are the same as before. Encoding is 3 to 20 times faster for ASCII and Latin-1 text.
  • Form params, digest hashing and HTTP/2-specific code are unchanged.

Problem

CharBuffer.wrap(String) returns a buffer with no backing array. The JDK encoder therefore takes its per-char buffer loop (virtual get/put with bounds checks), not the array loop or the String.getBytes ASCII/Latin-1 fast paths. CharsetEncoder.encode also starts at 1.1 bytes per char and grows by copying. This runs once per request attempt, on the caller thread, and costs O(body length).

Change

NettyRequestFactory.body(...) builds a NettyByteArrayBody from getBytes(charset). Before, it built a NettyByteBufferBody around the encoded ByteBuffer. Both are NettyDirectBody: same content length, unpooled heap ByteBuf, no content-type override. The HTTP/1.1 and HTTP/2 paths and replays read the body from the DefaultFullHttpRequest content, so they behave the same.

Both APIs replace malformed and unmappable input with the charset's default replacement. NettyRequestFactoryStringBodyTest checks byte-for-byte equality through newNettyRequest for every charset the JDK can encode, with unpaired surrogates, BOM and replacement chars, and UTF-16/UTF-32 (BOM writing). The same sweep in a standalone check on JDK 17, 21 and 25 showed 0 differences.

Netty's ByteBufUtil.writeUtf8 with an exact-size array was also tried and rejected: for a high surrogate followed by a valid pair it emits ??? where the JDK emits ? plus the pair.

HttpUtils.urlEncodeFormParams is left alone. It encodes a pooled StringBuilder, not a String, so getBytes would need an extra toString().

Benchmark

StringBodyEncodeBenchmark (new), JDK 17, UTF-8, -f 2 -wi 3 -i 5 -prof gc. Length is in chars. mixed is ASCII JSON ending in one emoji, so the String is stored as UTF-16.

text length ns/op before -> after B/op before -> after
ascii 256 586 -> 29 520 -> 336
ascii 4096 8,933 -> 304 4,744 -> 4,176
ascii 65536 137,408 -> 6,324 72,328 -> 65,616
latin1 256 908 -> 227 1,160 -> 928
latin1 4096 12,819 -> 4,483 13,832 -> 13,296
latin1 65536 198,104 -> 59,588 216,584 -> 211,272
cyrillic 256 1,399 -> 449 1,160 -> 1,336
cyrillic 4096 9,958 -> 8,161 13,832 -> 19,832
cyrillic 65536 151,404 -> 125,907 216,584 -> 315,864
mixed 256 590 -> 112 520 -> 1,128
mixed 4096 14,094 -> 1,858 4,744 -> 16,488
mixed 65536 160,041 -> 30,291 72,328 -> 262,248

Trade-off: for a String stored as UTF-16 (any char above U+00FF), getBytes(UTF_8) encodes into a scratch array of 3 bytes per char and then trims it. Transient allocation therefore grows for that text (same on JDK 21 and 25). Encoding is still faster, and the body kept for the request is exact-size; before, it was the encoder's oversized buffer.

The host was shared with other builds, so take the timings as indicative. B/op is deterministic.

AI disclosure

Claude Code on behalf of Pavel Ptashyts. The commit includes Co-Authored-By per AGENTS.md.

Test plan

  • NettyRequestFactoryStringBodyTest: 4 tests pass.
  • mvnw.cmd -B -ntp -Dgpg.skip=true clean verify on JDK 17 (no JDK 11 was available on the host): 1,861 tests, 0 failures, 0 errors, 35 skipped; Revapi passed. Source/target is 11. JDK 11 coverage comes from CI.

🤖 Generated with Claude Code

A String request body was encoded with
charset.encode(CharBuffer.wrap(body)). The wrapped CharBuffer has no
backing array, so the JDK encoder walks it one char at a time through
virtual get/put calls, and CharsetEncoder.encode starts at 1.1 bytes
per char and grows by copying. String.getBytes encodes straight from
the String's own array and uses the JDK's ASCII and Latin-1 fast paths.

Both replace malformed and unmappable input with the charset's default
replacement, so the bytes on the wire do not change. A test checks this
for every charset the JDK can encode, including unpaired surrogates and
charsets that write a byte order mark.

JMH StringBodyEncodeBenchmark on JDK 17, UTF-8: 4 KB ASCII goes from
8.9 us to 0.3 us and 64 KB Latin-1 from 198 us to 60 us. For a String
stored as UTF-16 (any char above U+00FF) getBytes encodes into a 3 bytes
per char scratch array and trims it, so transient allocation grows
(64 KB Cyrillic 217 KB -> 316 KB/op) while encoding is still faster and
the retained body is exact-size instead of oversized.

Claude Code on behalf of Pavel Ptashyts

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hyperxpro
hyperxpro merged commit e26f841 into AsyncHttpClient:main Nov 5, 2026
13 checks passed
@pavel-ptashyts
pavel-ptashyts deleted the claude/string-body-getbytes-40977c branch November 6, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants