Repository navigation
Conversation
NettyRequestFactory added three header values in forms that cost work on every request. A body-less POST, PUT or PATCH set Content-Length from an Integer, which Netty turns into a new "0" String each time. A body's length was boxed into a Long before the same conversion. The configured User-Agent was a String, so HttpHeadersEncoder took its char-by-char setCharSequence path instead of the AsciiString bulk copy. Use HttpHeaderValues.ZERO, setInt for lengths that fit in an int (HttpHeaders has no setLong, so larger lengths keep the old path), and convert the User-Agent to an AsciiString once in the constructor. The bytes on the wire are unchanged: AsciiString maps chars the same way the String path does, and the validating request headers still reject an illegal User-Agent on every request. The value is now read from the config once per factory, as the cookie encoder already is. RequestHeaderValueBenchmark, JMH 1.37, JDK 17.0.2, 3 forks, -prof gc: - Content-Length 0: 817.5 -> 769.5 B/op - body Content-Length: 841.5 -> 817.5 B/op - User-Agent value write: 21.3 +- 6.2 -> 11.1 +- 2.6 ns/op, no alloc Whole-request times were within noise on a loaded host. Local `mvnw clean verify` ran on JDK 17 (no JDK 11 on the host): 1865 tests, 0 failures; Revapi passed. Claude Code on behalf of Pavel Ptashyts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hyperxpro
approved these changes
Nov 6, 2026
pavel-ptashyts
added a commit
to maygemdev/async-http-client
that referenced
this pull request
Nov 6, 2026
Keeps the branch current with main at f9eafb9 (includes AsyncHttpClient#2372 and AsyncHttpClient#2380). Merged without conflicts. Claude Code on behalf of Pavel Ptashyts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hyperxpro
pushed a commit
that referenced
this pull request
Nov 7, 2026
This is the first of the two follow-ups promised on #2362 ([discussion](#2362 (comment))). A compressed HTTP/1.1 response body that ends in the middle of its compressed stream now fails the request, instead of completing as if it were whole. Netty's `HttpContentDecompressor` behaves like `main` here, so this is a behavior change and gets its own PR. ## Problem A body can end early, for example when the connection closes in the middle of a close-delimited body. On `main` such a body completed normally: - gzip: a member cut off in its header, data or trailer completed, so its CRC32 and length were never checked. - deflate: a stream that never finished completed. - br, zstd and snappy: Netty's `Decompressor` was never told that the input had ended, so its own truncation check never ran. ## Change Once a body has been decompressed in full, `Http1ContentDecompressor` checks that it ends where a gzip member or deflate stream ends, and `PullDecompression` calls `Decompressor.endOfInput()`, which throws for a truncated br, zstd or snappy stream. Output the decompressor still produces after that waits for a resume like any other part. The check only runs once held input and output have been handed on, also after a suspension and when the connection closes while input is held back. A truncated body fails the request with a `DecompressionException`. For zstd, Netty wraps zstd-jni's `ZstdIOException`, and since AHC fails a request with the innermost cause, that looked like a failed connection and went to the `IOExceptionFilter`s, where a replaying filter could leave the first connection open. `PullDecompression` now throws a plain `DecompressionException("Truncated zstd stream")` instead and keeps zstd-jni's exception as a suppressed one. Still accepted: - a body without any bytes, for every coding (`endOfInput()` is only called once input was added, and only in `NEED_INPUT`) - concatenated gzip members - a few bytes after a complete gzip member, fewer than a gzip header, which `JdkZlibDecoder` leaves unread Not checked: - a body nobody wants, such as the rest of a response drained for a replay, or one whose exchange is over. That includes a body too short to have produced any output, so the end of the body checks again whether it is wanted. - a last chunk that failed to parse, so that its own failure surfaces. A premature close of a `Content-Length` or chunked body produces no `LastHttpContent` at all. - bodies left to Netty's `HttpContentDecompressor`: br, zstd and snappy with a Netty older than 4.2.17, zstd at a non-positive `io.netty.compression.defaultMaxForwardBytes`, and transfer codings such as `Transfer-Encoding: gzip` - HTTP/2 response bodies Some cuts also go unnoticed. snappy has no end marker, so a body cut between two of its chunks looks complete. A zstd body of several frames completes when it is cut inside a frame that started in the same chunk as the end of the previous one, because zstd-jni only notices the new frame on a fresh read. The class Javadoc and the README describe all of this. The second follow-up, output left over when the connection closes, comes separately. ## Tests Fail on `main`, pass here: - `Http1DecompressionTruncationTest.aBodyThatTheConnectionEndsInTheMiddleOfItsStreamFails`: through the client, a close-delimited body still decodes in full for all five codings, and the same body cut short by 3 bytes fails with a `DecompressionException`. - `Http1DecompressionTruncationTest.aCutOffBodyIsNotPassedToTheIOExceptionFilters`: a cut-off `Content-Length` body with a replaying `IOExceptionFilter` fails with a `DecompressionException`, the filter is not called, no second request is sent and the client closes the connection. - `Http1ContentDecompressorTest.decodesLikeNettyForEveryBodyAndSplit`: the truncated gzip and deflate bodies (header, header name, data, missing trailer, partial trailer, second member, zlib, raw, missing Adler-32, one byte) now fail with `Truncated gzip stream` or `Truncated deflate stream` across all splits and buffer kinds, with no `IOException` in the chain. Every other body still decodes like Netty. - `Http1ContentDecompressorTest.truncatedBrZstdAndSnappyBodiesFailAndBodiesWithoutBytesEnd`: br, zstd and snappy bodies cut in half or by one byte fail, and bodies without bytes still end. - `Http1ContentDecompressorSuspensionTest.aTruncatedBodyFailsAtItsEndAfterTheResumes` (gzip and zstd) and `aTruncatedBodyHeldWhenTheConnectionClosesFails`: the parts before the cut are still handed on, then the body fails. - `reusesTheInflaterAcrossResponsesOnOneConnection` and `aBufferThatCannotGrowFailsLikeNetty` sent cut-off bodies on purpose and now expect the failure. Pass on `main` too, and fail if the end of the body stops checking whether it is wanted: - `Http1DecompressionTruncationTest.aReplayIsNotFailedByTheCutOffBodyItDrains`: a `ResponseFilter` replays a 503 whose gzip body is only 6 bytes, and the replay still gets its 200. - `Http1ContentDecompressorSuspensionTest.aShortBodyThatIsNotWantedEndsEvenIfItIsCutOff`: the first 1 to 12 bytes of a body that is discarded, drained or whose exchange is over end without a failure, for every coding. Each of these fails without the code it covers: - `aFailedLastChunkStaysFailed`, for the check of the last chunk's decoder result - `outputThatOnlyFollowsTheEndOfInputIsHandedOn` and `outputThatOnlyFollowsTheEndOfInputWaitsForResumes`, for output that only comes after `endOfInput()` - `reusesTheInflaterAcrossResponsesOnOneConnection`, for the reset of the gzip member state between responses `./mvnw -B -ntp -Dgpg.skip=true clean verify` on JDK 11, merged with `main` at `f9eafb982` (so with #2375 and #2380): 1,960 tests, 0 failures, 0 errors, 11 skipped, Revapi passed. That run predates a test-only fix in the new client test. The `Http1ContentDecompressor*Test`, `Http2ContentDecompressorTest` and `Http1Decompression*Test` classes also pass on JDK 17, 21 and 25, 101 tests each. Claude Code on behalf of Matthias Kurz. The commits carry `Co-Authored-By` as `AGENTS.md` asks. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
NettyRequestFactoryset three header values in forms that cost work on every HTTP/1.1 request:Content-Lengthfrom anIntegerconstant. Netty's header value converter callstoString(), so every such request allocated a new"0"string.Longbefore the same conversion.User-Agentis aString, soHttpHeadersEncoder.writeAsciiencoded it char by char throughsetCharSequenceinstead of theAsciiStringbulk copy.HeaderEncodeBenchmarkmeasured the same effect for header names (Intern known request header names to static AsciiString #2206).Changes:
Content-Length: 0usesHttpHeaderValues.ZERO.intgoes throughsetInt.HttpHeadershas nosetLong, so larger lengths keep the old path.User-Agentis converted to anAsciiStringonce, in the constructor.Behaviour notes
AsciiStringmaps chars the same way theStringencode path does: Latin-1 chars become their own byte, and anything above U+00FF becomes?. A test pins this.User-Agent(CR, LF and the like) with the sameIllegalArgumentException, on every request, as before.User-Agentis now read from the config once per factory, the way the cookie encoder already is. Only a customAsyncHttpClientConfigwhosegetUserAgent()changes between calls would notice.DefaultAsyncHttpClientConfigis immutable.Benchmark
New
RequestHeaderValueBenchmark: builds and encodes a request with the factory's own headers, with each value in its old and new form. JMH 1.37, JDK 17.0.2, Windows,-prof gc, 3 forks, 5 x 1 s warmup, 10 x 1 s measurement.Content-Length: 0Content-Length: 1536User-Agentvalue (writeAsciibranch only)The host was 75-92% busy, so whole-request times (~800 ns/op) stayed within noise. Allocation is the reliable figure. The
User-Agentgain was measured on the encoder's two branches in isolation. Neither branch allocates.Tests
NettyRequestFactoryHeaderValuesTestencodes the factory's requests withHttpRequestEncoderand checks:User-Agentis sent as configured;User-Agentproduces the same bytes as theStringvalue did;User-Agentwith CRLF is still rejected on every request;User-Agentwins over the configured one;Content-Length: 0is sent for body-less POST/PUT/PATCH, and noContent-Lengthfor GET;Integer.MAX_VALUE.Verification
mvnw.cmd -B -ntp clean verify -Dgpg.skip=trueon JDK 17.0.2 (Windows): 1,919 tests, 0 failures, 0 errors, 35 skipped. Error Prone, NullAway and Revapi passed. Local verify ran on JDK 17 because there is no JDK 11 on this machine; CI covers JDK 11.-Dgpg.skipis set only because gpg is not installed here.RedirectConnectionUsageTesttimed out once under load. It fails the same way on unchangedmainand passed in the next runs.Claude Code on behalf of Pavel Ptashyts
🤖 Generated with Claude Code