Issue/544 reactive streams support - #963
Conversation
There was a problem hiding this comment.
The alternative is queuing up elements in the subscriber, but the risk with that is overflowing the queue (dropping elements is even worse) when the consumer is much slower than the producer. Also, note this implementation is only used when the provider doesn't support reactive streams.
There was a problem hiding this comment.
Also, note this implementation is only used when the provider doesn't support reactive streams.
I'd rather throw an UnsupportedOperationException then. :)
There was a problem hiding this comment.
Mmm, I'm looking at the implementation of FeedableBodyGenerator and realise I probably don't need blocking. Forgive me :-)
|
Hi Mirko, I think your |
There was a problem hiding this comment.
What if the buffer doesn't have an array?
There was a problem hiding this comment.
Oh, is that possible? Any idea how I should handle that? (should I create the array myself, or fail?)
There was a problem hiding this comment.
Oh, is that possible?
Of course: DirectBuffers, composites, etc. That's what hasArray() is for.
See
FeedableBodyGenerator, you can put a ByteBuffer into another.
There was a problem hiding this comment.
@dotta the netty in action cover this:).composites could have array if it have only one element and the element have array or it was empty.
There was a problem hiding this comment.
@hepin1989 Don't confuse Netty's ByteBuf and Java's ByteBuffer :)
There was a problem hiding this comment.
@slandelle haha,sorry,I don't see the whole context:).
|
Hi @slandelle, thanks a lot for the feedback!
Do you mean the implementation of |
Yeah, they serve the same purpose, so I guess existing code could be reuse/extended/wrapped. |
There was a problem hiding this comment.
@jroper I'm not so convinced about requesting only one element at a time. Any thought?
There was a problem hiding this comment.
Considering this is a fallback implementation, I think it's fine. Another option, seeing as the provider is going to end up buffering infinitely anyway, is you can invoke subscription.request(Long.MAX_VALUE) in onSubscribe, this is allowed according to reactive streams spec 3.17.
|
@slandelle I think you should be more pleased now (I for sure am :-)). |
There was a problem hiding this comment.
For consistency, I'd rather remove the I prefix and prefix the default implementation with Simple (or better if you can think of something).
There was a problem hiding this comment.
Please change copyright to "the AsyncHttpClient Project".
a7b7b08 to
f91bac1
Compare
|
@slandelle I believe I've addressed all comments. The only pending one is #963 (comment), which you are discussing with @jroper. Let me know if you have other comments, or if I should rebase and squash commits. |
|
@slandelle I'm happy to squash commits and rebase if you give me your bless :-) In the meanwhile, I'll update Play master to take advantage of the changes in this PR. Whohoo! |
|
@dotta Sorry, had some unexpected things to deal with first... Yes, could you please squash and rebase? Let's merge this so everyone can move forward and get everything stable in the next weeks so we can release 2.0.0. |
|
Whohooo! I'm on it, thanks! :-) |
a373a21 to
71c658c
Compare
Issue/544 reactive streams support
|
@slandelle Thank you for merging! I'm updating the Play codebase to use AHC master, will let you know if I hit any issue. |
|
Great! |
|
I believe it's November 2015, but I'll let @jroper confirm. Once I've verified that things work properly on the Play side, I'd really appreciate if you could release a 2.0.0 milestone (in case you would like to wait a bit more before cutting a 2.0.0 final). |
|
I can easily cut an alpha. I'm still considering some refactorings (maybe Proxies) + I might want to break a few things to improve DNS handling in Gatling. |
|
Sounds good. I'll ping you soon :-) |
|
@slandelle I've a branch in Play that is now using the new |
|
I just released 2.0.0-alpha9. |
|
Thanks @slandelle! |
|
@slandelle Thank you! |
## Summary - Add a thread-safe `ResponseBodyControl` callback after final response headers. - Support suspending, resuming, and cancelling HTTP/1.1 and HTTP/2 response bodies without coupling AHC to a streaming API. - Pause the network read timeout while reads are intentionally suspended while leaving the request timeout active. - Keep suspended HTTP/2 streams independent and cover transport backpressure, cancellation, timeouts, and connection reuse. ## Motivation AHC 3 removed `StreamedAsyncHandler` and its Reactive Streams integration in pull request [#1843](#1843). That removal avoids coupling AHC to a particular streaming library, but `AsyncHandler` by itself has no way to stop transport reads while a downstream consumer has no demand. [Play WS](https://github.com/playframework/play-ws) is the driving consumer for this change. Play WS needs transport backpressure to preserve its existing Pekko Streams and Reactive Streams response APIs while upgrading to AHC 3. A working adapter on a currently local Play WS development branch turns this control into a single-subscriber Reactive Streams publisher and has been tested against this AHC branch. This pull request adds only the transport primitive. Streaming-library policy and dependencies remain in Play WS, so AHC does not regain a dependency on Reactive Streams or JDK Flow. ## Semantics - `AsyncHandler.onResponseBodyStart` runs after final headers and before body parts, including for a response with no body. - Calls to the supplied control are thread-safe, idempotent, and ignored after response completion. - `suspend()` stops requesting new transport data, although body parts already read may still be delivered. - `resume()` permits transport reads again. - Returning `State.ABORT` is the synchronous callback-time way to stop processing. A handler can retain the control and call `cancel()` when an asynchronous decision is made after the callback returns. - Cancelling an HTTP/1.1 body closes its connection when bytes may remain unread; cancellation after terminal content has been received can reuse a keep-alive connection. Cancelling an HTTP/2 body closes only its stream. - Fully consumed responses retain the existing HTTP/1.1 pooling and HTTP/2 parent-connection reuse behavior. - Suspension pauses only the network read timeout. The request timeout remains active; if the request timeout is disabled, an application that never resumes or cancels can retain the exchange and its transport resources indefinitely. ## HTTP/2 flow control Connections with no actively suspended response retain Netty's normal connection-level receive-window accounting. The default 65,535-byte connection window therefore continues to cap unconsumed flow-controlled DATA across all streams for users that never call `suspend()`. When the first response on a connection is suspended, AHC returns connection-level credit that has already accumulated and continues returning that shared credit as DATA arrives. This prevents the suspended stream from exhausting the connection window and starving sibling streams. Per-stream credit remains consumption-driven at all times, so each suspended stream is still bounded by its own receive window. Multiple simultaneous suspensions are counted, and normal connection accounting resumes after the last one ends. Credit already returned and data already queued cannot be revoked, and the controller tracks credit returned early so later application consumption does not return it twice. During an active suspension, aggregate buffering can still scale with the number of concurrent streams; the relevant controls are `http2InitialWindowSize`, `http2MaxConcurrentStreams`, and the connection limits. The defaults do not impose a hard client-side aggregate bound during that interval: the initial per-stream window is 16 MiB and `http2MaxConcurrentStreams = -1` leaves concurrency server-controlled. A rough upper-bound estimate is connection count times effective concurrent streams times the initial window, excluding network and decoder overhead. Applications requiring a finite policy must configure these values together. A hard aggregate byte budget is a separate design and is outside this pull request. Netty's connection auto-refill state is private and fixed when `DefaultHttp2LocalFlowController` is constructed, so it cannot be enabled only for the lifetime of a suspension through composition or subclassing. The package-private `SuspensionAwareHttp2LocalFlowController` therefore adapts `DefaultHttp2LocalFlowController` from Netty 4.2.17.Final, preserving its normal behavior while making connection refill suspension-scoped. This adaptation has a maintenance cost: AHC owns the copied flow-control logic and every Netty upgrade must compare it with the corresponding upstream implementation for correctness and security fixes. The exact source version and that obligation are recorded in the class Javadoc. An upstream Netty API that permits connection auto-refill to be changed at runtime would provide the exit path and allow AHC to remove the adaptation; no such API exists in Netty 4.2.17.Final. The auto-refill mode requires a custom `Http2Connection`. Netty's builder treats `server()` and `connection()` as mutually exclusive, so `ClientHttp2FrameCodecBuilder` supplies the connection through the protected builder API and overrides `isServer()` to retain client mode. The explicit `gracefulShutdownTimeoutMillis(0)` is not a new shutdown policy. `Http2FrameCodecBuilder.forClient()` selects zero through its package-private client constructor; the subclass must use the protected no-argument constructor, so it sets zero explicitly to preserve the existing client-factory behavior. ## Scope and commit structure The API/lifecycle work and the HTTP/2 independence work are kept as separate logical commits, but they belong in one pull request. Without the HTTP/2 work, a suspended response can consume the shared connection window and block unrelated sibling streams, so the public control would not have correct multiplexed behavior. Follow-up review fixes are also split into focused commits covering exchange ownership, terminal cleanup, terminal HTTP/1.1 cancellation, bodyless responses, interim responses, indefinite-suspension diagnostics, and suspension-scoped HTTP/2 refill. ## History checked - Issue [#544](#544) originally identified the lack of `AsyncHandler` backpressure, and pull request [#963](#963) addressed it by adding Reactive Streams support. - Issues [#1233](#1233) and [#1721](#1721) document the interaction between downstream demand and read timeouts in the former streamed handler. - Pull request [#1843](#1843) removed `StreamedAsyncHandler` for AHC 3. - Discussion [#1925](#1925) asks how to migrate streamed consumers to AHC 3; the maintainer response declines restoring Reactive Streams because other libraries provide that policy and maintaining it adds overhead. - Focused GitHub issue, pull-request, discussion, and local history searches found no existing AHC 3 proposal for a streaming-library-neutral suspend/resume/cancel response-body control. ## Compatibility - `AsyncHandler.onResponseBodyStart` is a new Java `default` method, so existing handler implementations remain source- and binary-compatible and retain their previous behavior unless they override it. - `ResponseBodyControl` is a new public interface. - `NettyResponseBodyControl` is public only to support AHC's cross-package transport integration and is marked `@ApiStatus.Internal`; consumers should depend on `ResponseBodyControl` instead. - HTTP/1.1 informational responses from 102 through 199 are now treated as interim and no longer reach `onStatusReceived` or `onHeadersReceived`; this matches the existing HTTP/2 behavior and prevents a 103 Early Hints response from completing the exchange before the final response. Existing 100 Continue handling and 101 protocol switching are preserved. - Cancelling HTTP/1.1 processing from a terminal trailer or body callback now reuses a keep-alive connection because `LastHttpContent` has already been received; it previously closed that fully read connection. - The full JDK 11 verification, including Revapi, passes. ## AI disclosure OpenAI Codex on behalf of Matthias Kurz. The commits include `Co-Authored-By: OpenAI Codex <codex@openai.com>` per `AGENTS.md`. ## Test plan - [x] On the first commit alone, the new HTTP/2 sibling-stream test reproduced shared connection-window starvation: a sibling response did not progress while the first response remained suspended. - [x] The focused `ResponseBodyControlTest`, `Http2ResponseBodyControlTest`, `NettyResponseFutureTest`, `TimeoutTimerTaskTest`, `SuspensionAwareHttp2LocalFlowControllerTest`, and `Continue100InterceptorTest` suites pass 38 tests: 12 HTTP/1.1 control tests, 6 HTTP/2 integration tests, 9 future-lifecycle tests, 4 timeout-task tests, 5 flow-controller accounting tests, and 2 HTTP/1.1 Continue tests. - [x] The flow-controller tests cover unchanged connection accounting without suspension, connection-only refill during suspension, DATA received before the suspension callback, overlapping suspensions, and avoiding double credit after normal accounting resumes. - [x] `JAVA_HOME=<jdk-11> ./mvnw clean verify`: BUILD SUCCESS for the full reactor, including tests, Javadocs, coverage, and Revapi. - [x] The locally published AHC snapshot passes the full Play WS Scala 2.13 and Scala 3.3.8 test matrices on Java 17 and Java 21: 183 integration tests and 76 unit tests pass in each combination, with 2 expected pending tests. - [x] Play WS code validation, documentation, Scala 2 and Scala 3 MiMa checks, and dependency-tree verification pass against the locally published AHC branch. --------- Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Aayush Atharva <24762260+hyperxpro@users.noreply.github.com>
Here is an initial proposal to fix #544.
I'm entirely new to both Netty and AHC, so if you notice anything weird it's very likely that I haven't understood something :-)
Other than that, something is puzzling me with the tests I wrote. Basically, if I change theCHUNK_SIZEin https://github.com/AsyncHttpClient/async-http-client/compare/AsyncHttpClient:master...dotta:issue/544-reactive-streams-support?expand=1#diff-5133f4e3fed98f7a99081a1ae643c4c0R104 to a value around20000(or lower), the tests start to fail because there is a mismatch between the received bytes and the expected ones. I honestly have no good explanation for this, but I thought it may have something to do with the internals of Netty (or maybe a config flag?). I'd really appreciate it if someone more experienced with Netty could have a look.EDIT: This issue is now fixed after using RxJava instead of providing my own implementation of
Publisherin the test :-)This PR also includes @jroper commit to add support for handling response bodies.