Repository navigation
Re-arm an early read timeout for what is left - #2377
Merged
hyperxpro merged 4 commits intoNov 7, 2026
Merged
Conversation
When the read timeout fires but the channel was read from since it was armed, the task re-armed itself through startReadTimeout, which always armed a whole readTimeout. The remaining window was computed and then dropped, so an exchange could sit idle for up to twice the read timeout after its last read before being failed. Pass the remaining duration through a startReadTimeout overload and arm with it, capped at a whole read timeout so a wall clock stepping back cannot stretch it. The request-timeout-first check now compares the delay actually armed: with the full value it also dropped a read timeout that was due well before the request timeout. The suspended body branch still re-arms with a whole window on purpose, since suspension pauses the read timeout rather than counting against it. The task now reads the holder's clock, the same wall clock as before, so a test can drive it without letting time pass. Local verification ran on JDK 17 (no JDK 11 on the host); CI covers JDK 11. Claude Code on behalf of Pavel Ptashyts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of AsyncHttpClient#2377 pointed out that nothing failed if the cap at a whole read timeout was dropped, and that the unit test never reached the branch that expires the exchange. Add a case where the wall clock steps back, which re-arms for exactly one read timeout (the uncapped code re-arms for 1600 ms), and a case with no read in the window, which fails the future with a TimeoutException and arms nothing more. Claude Code on behalf of Pavel Ptashyts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hyperxpro
requested changes
Nov 6, 2026
Review of AsyncHttpClient#2377 noted that the cap at a whole read timeout does not stop a wall clock stepping back from stretching the timeout: expiry is still decided on the wall clock, so without a read after the step the exchange fails late by the whole step. What the cap does is keep any single re-arm from outlasting a window, so say that, and name the test after it. Also state in the test constructor's comment that millisClock must be the clock touch() stamps lastTouch with, now that the read timeout reads it too. Claude Code on behalf of Pavel Ptashyts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pavel-ptashyts
added a commit
to maygemdev/async-http-client
that referenced
this pull request
Nov 7, 2026
Second review round on the shared request/read entry. Telling a stale handle from a live one by asking the scheduler whether an entry had run was a guess on an event loop: a loop marks an entry done only after it returns, so the code read "due" as "has run", and a successor that was already due when it was recorded was dropped as stale. cancel() then could not take it out. Each arming now takes a number from a per-task counter before it schedules, and an entry is recorded only if no later arming has begun, which says exactly what the guess approximated. The shared entry re-armed for what was left of the read window with no upper bound. A wall clock stepping back made that remainder longer than the request deadline, so the entry was armed for the deadline and the read timeout was not checked again before it. The remainder is now capped at a whole window, as the separate read timeout task does since AsyncHttpClient#2377. Also covers the read timeout starting while the entry re-arms. Claude Code on behalf of Pavel Ptashyts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hyperxpro
approved these changes
Nov 7, 2026
hyperxpro
pushed a commit
that referenced
this pull request
Nov 9, 2026
## Summary Every attempt builds a `TimeoutsHolder`, arms its request timeout and, once the request is written, its read timeout, then cancels both when the exchange ends. When the read timeout is shorter than the request timeout, that is two timer entries per attempt, and almost every exchange cancels both unfired. On the client's `HashedWheelTimer` each entry is an increment of a client-wide `AtomicLong` and an offer to a client-wide MPSC queue, each cancel an offer to a second one, and the timer's single thread drains them all. With `useEventLoopTimeouts` each is a scheduled task added to and removed from the loop. Changes: 1. **`RequestTimeoutsArmBenchmark`** measures one attempt's timeout lifecycle on a real `HashedWheelTimer`, with the read timeout shorter and not shorter than the request timeout. **`RequestTimeoutsFireBenchmark`** measures an exchange that outlives its read window: the entry that checks the read timeout fires one or five times, finding a recent read and arming again each time. 2. **Private timeout guards in field updaters.** The two `AtomicBoolean`s of `TimeoutsHolder` and the suspension-warning flag of `ReadTimeoutTimerTask` become volatile ints behind static `AtomicIntegerFieldUpdater`s. `TimeoutTimerTask.done` stays an `AtomicBoolean`: it is a protected field of a public class, and replacing it is the API break that closed #2263. 3. **One timer entry per attempt.** When the read timeout is the shorter one, `start()` arms the request timeout's entry for when a read timeout would be due. Once the read timeout starts it rides on that entry instead of taking its own. On firing, the entry fails the exchange with a read timeout when one is due, and otherwise re-arms for whichever deadline comes next: what is left of the read window, capped at a whole window and by the request deadline. 4. **Fixes from the review rounds below**, each with a test that fails without it: - the request deadline that the shared entry re-arms and checks against is kept on the monotonic clock (an absolute deadline straight off the exchange's start), so a wall clock step cannot move it; - each arming takes a number from a per-task counter, and an entry is recorded only if no later arming has begun, so an arming that loses the race against its own firing cannot hide the newer entry from `cancel()`; the handle is one field, taken atomically on cancel; - if the timer rejects the re-armed entry (a user timer that is full or stopped), the exchange is failed instead of being left with no deadline, but only while this holder is still the future's and the future is not done; - the monotonic clock is read only where the deadline is needed (an entry armed early, or `useEventLoopTimeouts`), so the default attempt costs no more than on `main`; - an entry that fires early reads the monotonic clock once, both to check the request deadline and to time the next entry; it skips the read-state CAS once the read timeout is covered, and writes the early flag only when the flag changes. Without these, five fires per exchange ran slower than `main`'s separate read entry. ## Behaviour notes - Same timeouts, same messages, same exceptions. - A read timeout that has not started when the shared entry fires (a slow connect or upload) goes back to an entry of its own, as before. That costs one fire and re-arm that `main` does not have. - An exchange that outlives its read window re-arms the shared entry for what is left of the window, capped at a whole window. #2377 re-arms `main`'s separate read entry the same way. Each fire arms one entry, as each fire of the read entry does on `main`, and the request timeout's own entry is gone. - An entry that fires past the request deadline (a wheel tick can land after it) delivers the request timeout. - A suspended response body still holds off the read timeout for a whole window, and the indefinite-suspension warning is unchanged. - Absolute deadlines, `rehomeOn`, `useEventLoopTimeouts` and cancel without pinning the future are kept; their existing tests pass unchanged. - No public API change: Revapi passes. New members are private or package-private; `TimeoutsHolder.cancel()` keeps its signature. ## Relation to #2377 #2377 makes the separate read timeout entry re-arm for what is left of its window rather than a whole one. It has since merged into `main`. This branch still merges into `main` cleanly (`git merge-tree` against f5ad541), and no merge of `main` was pushed here. Before #2377 merged, both PRs' timeout tests passed on the merged tree. The shared entry here re-arms for the remaining window the same way, capped at a whole window as #2377 caps it. ## Benchmark `RequestTimeoutsArmBenchmark.armAndCancel`, request timeout 60 s. JMH 1.37, JDK 17.0.2, Windows, `-prof gc`, 3 forks, 3 x 2 s warmup, 5 x 2 s measurement, `main` against 12291df. The later commits change the firing path; on the arming path they only skip a write of a flag that did not change. Read timeout 10 s (two entries on `main`, one here): | Threads | `main` (ops/us) | This PR (ops/us) | Bytes/op | | ---: | ---: | ---: | ---: | | 1 | 2.76 +- 0.02 | 3.57 +- 0.03 | 391 -> 223 | | 8 | 5.35 +- 1.13 | 10.71 +- 3.11 | 392 -> 225 | | 32 | 1.74 +- 0.52 | 3.36 +- 0.96 | 395 -> 226 | Read timeout 60 s, the default (one entry either way): | Threads | `main` (ops/us) | This PR (ops/us) | Bytes/op | | ---: | ---: | ---: | ---: | | 1 | 3.99 +- 0.03 | 3.97 +- 0.02 | 247 -> 223 | | 8 | 7.43 +- 4.59 | 11.08 +- 2.89 | 249 -> 225 | | 32 | 3.91 +- 1.05 | 4.32 +- 1.07 | 249 -> 226 | `RequestTimeoutsFireBenchmark.armFireAndCancel`, read timeout 10 s, request timeout 60 s, same settings, `main` against 44b7bc3. The benchmark fires the entry from its own thread by taking it out of the wheel (a cancel) and running its task. Both builds pay that cancel once per fire. At one thread, `main` and this branch ran alternately, twice each: | Threads | Fires | `main` (ops/us) | This PR (ops/us) | Bytes/op | | ---: | ---: | ---: | ---: | ---: | | 1 | 1 | 1.67 +- 0.32, 1.71 +- 0.32 | 1.99 +- 0.33, 1.88 +- 0.34 | 455 -> 287 | | 1 | 5 | 0.76 +- 0.20, 0.61 +- 0.12 | 0.99 +- 0.10, 0.89 +- 0.14 | 709 -> 541 | | 8 | 1 | 1.71 +- 1.20 | 5.07 +- 0.82 | 459 -> 290 | | 8 | 5 | 1.27 +- 0.17 | 1.31 +- 0.45 | 716 -> 550 | | 32 | 1 | 1.23 +- 0.39 | 1.44 +- 0.53 | 460 -> 292 | | 32 | 5 | 0.51 +- 0.20 | 0.67 +- 0.27 | 722 -> 553 | The host was shared with other builds while these ran, so the multi-threaded rows have wide error bars; allocation is the reliable figure there. ## Tests New `RequestTimeoutTimerTaskTest` (26 tests) drives a holder with fake wall and monotonic clocks and a recording timer and event loop. It covers one entry serving both timeouts, re-arming for the rest of the read window, delivery of read and request timeouts from the shared entry, a read timeout that starts late or while the entry re-arms, absolute deadlines, wall clock steps both ways, suspended bodies, `rehomeOn` before the read timeout starts, cancellation after an early fire on the timer and on a loop, an arming that loses the race to its own firing, a re-arm the timer rejects (with a live, a replaced and a detached holder), a request timeout of `Long.MAX_VALUE` ms whose deadline wraps on the monotonic clock, a completed exchange fired inside its read window, the default attempt not reading the monotonic clock, and an early fire reading it once. `./mvnw clean verify -Dgpg.skip=true` passes on JDK 17 (1965 tests; no JDK 11 and no gpg on the host); CI covers JDK 11. ## Maintainer review Eight comments from hyperxpro, written against c8df6c3. At 12291df, five were already fixed: the deadline on the monotonic clock with subtraction, the one-window cap, the `Long.MAX_VALUE` overflow, the atomic take of the handle in `cancelArmed()`, and the abort on a rejected re-arm. The `done.set(false)` ordering test already existed (`aReadTimeoutStartingWhileTheEntryReArmsGetsItsOwnEntry`). This push adds the rest: the overflow test, the completed-exchange test fired inside the window, `RequestTimeoutsFireBenchmark`, and the per-fire trims it led to. Both new review tests fail when the code they guard is reverted, and so does the existing ordering test when `done.set(false)` moves below the re-arm. ## Second-agent review Both reviewers ran from fresh CLI sessions, read-only, on `git diff upstream/main...HEAD`, with the intent and constraints in the prompt. | Reviewer | Agent and model | Effort | Commit reviewed | Result | | --- | --- | --- | --- | --- | | Guided (against `AGENTS.md`) | Codex `gpt-6-astra` | medium | 12291df | No actionable findings | | Independent (no checklist) | Claude Code `claude-opus-5-5` | medium | 12291df | No actionable findings; two optional notes, no change made | Seven rounds in all (c8df6c3, f2cddcc, 3daa3b7, cbb6a78, fe9fc58, e678f09, 12291df). Every actionable finding of the earlier rounds was fixed in the commits listed under change 4, except these, which were not changed: - **`startReadTimeout(task)` still compares the read timeout against the request deadline on the wall clock.** Inherited from `main`. #2377, now merged, rewrote the same line. Changing it here would need `main` merged into this branch, so it is left for a follow-up after this PR. - **Check-then-act window when a rejected re-arm aborts the exchange.** After the ownership check passes, a redirect or completion can still replace the holder or pool the channel before `abort` runs. Every timeout delivery on `main` has the same window (`expire()` checks and then aborts with the future's current channel from the timer thread); closing it means serializing timeout delivery with attempt replacement on the channel's event loop, a redesign outside this PR. The path is reached only when a user-supplied timer rejects an entry. - **`start()` sets the pending read state with a plain write** (round 6, independent reviewer, low). A read timeout started on a new holder before `start()` runs can leave both a separate read entry and the early entry; `main` double-arms the read timeout in the same race, and the two entries run on the same scheduler, so the overlap that would let one drop the other does not arise outside a `rehomeOn`. - **`rehomeOn` javadoc** still says it is a no-op once the request timeout has fired; an early firing of the shared entry is not that, and the next re-arm reads the new executor anyway. Wording only. ## Follow-up idea, not in this PR A per-connection lazy deadline for pooled HTTP/1.1 connections, in the style of `IdleStateHandler`: one entry per connection, re-armed only when it fires early or a new deadline is earlier, so a keep-alive request touches neither the timer nor a `TimeoutsHolder`. A throwaway prototype put the per-request cost near zero, but it is a new synchronization protocol at the exchange handover on a pooled channel, so it should wait for an end-to-end keep-alive benchmark that shows how much of a whole request this cost is. Claude Code on behalf of Pavel Ptashyts 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When the read timeout fires but the channel was read from since it was armed,
ReadTimeoutTimerTaskcomputes how much of the window is left and then re-arms throughTimeoutsHolder.startReadTimeout, which always armed a wholereadTimeout. An exchange could therefore sit idle for up to twice the read timeout after its last read before being failed.startReadTimeout(task, delay)overload and arm with the remaining duration, capped at a whole read timeout so that no single re-arm outlasts a window, whatever the wall clock did. Expiry itself is still decided on the wall clock.touch()stampslastTouchwith) so a test can drive it without letting time pass.No public API change.
History
Before f045f40 (2015) the task re-armed with
durationBeforeCurrentReadTimeout; moving re-arming intoTimeoutsHolder.startReadTimeoutdropped the remainder. No existing issue or PR covers this.Tests
ReadTimeoutTimerTaskTestdrives the task with a fake clock and a mockedTimerand reads the re-arm delay off the timer:[1000, 400];maingives[1000, 1000]);maindrops it);[1000, 6000]without theMath.min);TimeoutExceptionand nothing more is armed.mvnw.cmd clean verify -Dgpg.skip=truepasses locally on JDK 17 at f2a45f0 (no JDK 11 on the host): 1943 tests, 0 failures, Revapi green. CI covers JDK 11.Second-agent review
Final pass on f2a45f0 (diff
upstream/main...HEAD, base f9eafb9), both run read-only in fresh sessions:AGENTS.md)gpt-6-astraclaude-opus-5-5The first pass on b05899c was also clean apart from two optional test suggestions (a backward clock step and the expiry branch), both added above.
Optional nits from the final pass, not applied:
TimeoutsHolder.millisTime()partly repeats the test constructor's comment. Kept: it is the one a reader of the accessor sees, and the reviewer itself suggested leaving it.any()and verified withany(TimeUnit.class). Both match; purely stylistic.Claude Code on behalf of Pavel Ptashyts
🤖 Generated with Claude Code