fix: address book bucket overflow, decoder panic, stale rollback caches, protoio byte reader, negative-number queries - #6043
Open
gomesalexandre wants to merge 3 commits into
Conversation
…es, protoio byte reader, negative-number queries
Five independent, small correctness bugs found across peer-discovery
bookkeeping and internal libraries:
- p2p/pex: addToNewBucket/addToOldBucket used `>` instead of `>=` against
their bucket size constants, admitting one entry past the documented
capacity. Worse, when moveToOld's overflow-recovery path demotes the
oldest entry of a full old bucket back to a new bucket, removeFromBucket
never resets BucketType before addToNewBucket's isOld() guard runs, so
the demotion always fails and the address is silently dropped from the
book (only logged, never surfaced).
- libs/json: decodeReflect's 64-bit-integer guard read bz[0] and
bz[len(bz)-1] without checking len(bz) >= 2 first, so a 1-byte JSON
value ("\"") panics on the following bz[1:len(bz)-1] slice. The
sibling time-value guard 30 lines up already has the len(bz) < 2
check this one is missing. Reachable via unauthenticated RPC
(e.g. GET /block?height="); caught by RecoverAndLogHandler, so this
is a wrong-status/log-noise bug, not a node crash.
- store: DeleteLatestBlock (the `cometbft rollback` operator path)
deletes the block/commit/seen-commit rows but never deletes the
extended-commit ("EC:") row, and never evicts any of the three LRU
commit caches. After a rollback + resync with a different block at
the same height, LoadBlockExtendedCommit can return the stale
pre-rollback value straight from cache.
- libs/protoio: byteReader.ReadByte mishandled two io.Reader return
shapes that are legal per the interface's own doc: (0, nil) surfaced
a stale/fabricated byte instead of retrying, and (n>0, io.EOF)
discarded an already-read byte along with the error. Fixed by
looping on (0, nil) and returning the byte immediately whenever
n == 1, regardless of the accompanying error.
- libs/pubsub/query: extractNum's regex only matched `^\d+(\.\d+)?`,
so a negative event attribute value (e.g. "-5") extracted an empty
string, parseNumber returned an error, and every numeric comparison
operator (=, <, <=, >, >=) against it silently evaluated to false.
Fixed by allowing an optional leading '-'.
Each fix ships with a genuine red-before/green-after regression test
(verified via git stash against the unfixed source, not just written
and assumed correct) plus the corresponding CHANGELOG.md entry.
A separate, genuinely security-sensitive finding from the same
scouting pass (an unrecovered nil-pointer dereference in the mempool
recheck path) is intentionally NOT included here and is being routed
through coordinated disclosure instead of a public PR.
…or, and DoS bound Codex's adversarial pass on the previous commit surfaced several real, concrete issues, all addressed here: - p2p/pex: a single demotion in moveToOld only brings a full old bucket down to exactly oldBucketSize, one entry short of the room a promotion needs -- a bucket a pre-fix build already left at oldBucketSize+2 (or more) would still reject the promotee's final re-add, silently orphaning it (BucketType=old, zero bucket membership, uncounted, unreachable). Changed the recovery to keep demoting until there's room, and added a last-resort fallback to keep the address reachable as new if that still somehow fails. The identical issue existed in addToNewBucket's single-expiry-then-add pattern for a legacy oversized new bucket; changed to loop there too. Added regression tests for both, independently verified red against the prior commit's fix and green against this one. - libs/protoio: bounded the (0, nil) retry loop added in the previous commit with the same maxConsecutiveEmptyReads=100 -> io.ErrNoProgress pattern bufio.Reader itself uses, so a reader that persistently returns (0, nil) can't hang ReadByte forever. Added a goroutine+timeout regression test proving the unbounded version genuinely hangs and the bounded version doesn't. (Reviewed and kept as-is: returning the byte with nil error whenever n==1, deferring any accompanying non-EOF error to the next call, exactly matches bufio.Reader's own fill()/readErr() pattern -- not a regression.) - store: strengthened the DeleteLatestBlock regression coverage to check the extended-commit DB row is actually gone (not just that the cache was evicted, which a subsequent overwrite could have hidden either way), and added seen-commit cache eviction coverage alongside the existing extended-commit cache test. - libs/pubsub/query: removed a TestNegativeNumbers subtest asserting `delta.value = 0` is false -- true both before and after the fix (-5 != 0 either way), so it couldn't actually distinguish red from green. Replaced with a comment explaining why "=" isn't independently testable via query syntax (the query literal itself can't be negative, a separate pre-existing scanner limitation) even though the fix covers it via the same parseNumber the ordering-operator tests do exercise. Not changed, disclosed instead: Load*Commit's DB-read-then-cache-Add sequence isn't done under bs.mtx, so a concurrent DeleteLatestBlock racing an in-flight Load call could theoretically repopulate a cache entry just after this fix evicts it. This race predates this PR -- every existing cache-eviction call site in this file (e.g. PruneBlocks) has the identical gap -- and fixing it would mean broader locking changes across the store package, out of scope for this housekeeping PR. Flagging it for a maintainer rather than silently expanding scope.
gomesalexandre
marked this pull request as ready for review
September 1, 2026 21:47
Contributor
|
PR author is not in the allowed authors list. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What it says on the box
Five independent, small correctness bugs found while auditing peer-discovery bookkeeping and a handful of internal libraries — none of these are crashes or DoS vectors on their own. (A separate, genuinely security-sensitive nil-pointer-dereference finding in the mempool recheck path came out of the same audit; it is intentionally not in this PR and is being routed through coordinated disclosure instead.)
p2p/pex — address book bucket capacity + overflow-recovery losing addresses
addToNewBucket/addToOldBucketused>instead of>=against their bucket-size constants (newBucketSize/oldBucketSize, both 64), so a bucket already at capacity would still admit a 65th entry.Worse: when
moveToOld's overflow-recovery path demotes the oldest entry of a full old bucket back to a new bucket,removeFromBucketnever resets the entry'sBucketType.addToNewBucket'sisOld()consistency guard then rejects the demotion outright — silently, only logged — and sinceremoveFromBuckethad already dropped the address fromaddrLookup, it's gone from the book entirely.Receipts (
p2p/pex/addrbook_bucket_overflow_test.go):Both fail with the predicted symptoms when run against the unfixed source (verified via
git stashbefore writing the fix — not just written and assumed correct).A Codex adversarial review of the first pass caught that this fix was itself incomplete for legacy state: a book persisted by a pre-fix build can already have a bucket at 65+ entries. Against that state, a single demotion only brings the bucket down to exactly 64 — one short of the room the original promotee still needs on its final re-add attempt — so the promotee itself ended up orphaned (
BucketType=old, zero bucket membership, uncounted, unreachable). Fixed by looping the demotion until there's actually room, with a last-resort fallback that keeps the address reachable asnewrather than losing it if that still somehow fails. The identical shape existed inaddToNewBucket's single-expiry-then-add pattern for a legacy oversized new bucket (would stay at 65 forever); fixed the same way. Both have dedicated regression tests, independently verified red against the Codex-reviewed first-pass fix and green against the final one:Full package:
go test ./p2p/pex/...→ok(11.1s, 25 tests).libs/json — 1-byte input panics the 64-bit-integer decode guard
decodeReflect's int64/uint64 branch checksbz[0] != '"' || bz[len(bz)-1] != '"'without first checkinglen(bz) >= 2. For a 1-byte input this reads the same byte twice, the guard passes (both comparisons happen to be false when that byte is"), and the followingbz[1:len(bz)-1]slice becomesbz[1:0]— panic. The sibling time-value guard 30 lines up already has thelen(bz) < 2check this one is missing.Reachable via unauthenticated RPC (e.g.
GET /block?height="); caught byRecoverAndLogHandler, so this is a wrong-status-code + log-noise bug, not a node crash — stated plainly, not oversold.Fix: add the missing
len(bz) < 2 ||. Receipt:Fails with the exact panic above against the unfixed source. Codex found no issue in this fix or its test.
store — DeleteLatestBlock (the
cometbft rollbackpath) leaks the extended-commit row and never evicts commit cachesDeleteLatestBlockdeletes the block/commit/seen-commit/meta rows but never deletes theEC:(extended-commit) row, and never evicts any of the three LRU commit caches (blockCommitCache,seenCommitCache,blockExtendedCommitCache). After a rollback + resync with a different block landing at the same height,LoadBlockExtendedCommitcan return the stale pre-rollback value straight from cache — the DB itself gets correctly overwritten by the resync, but the cache doesn't know that.Admin-triggered (requires an operator running
cometbft rollback), not remotely reachable — stated as such.Fix: delete the
EC:key in the batch, and evict all three caches for the height. Receipts:The first test proves the cache-staleness scenario end to end via
LoadBlockExtendedCommit. Codex correctly flagged that this alone doesn't prove the DB row is actually deleted (a resave overwrites it regardless of whether the delete ran) — addedTestDeleteLatestBlockDeletesExtendedCommitRow, which reads the raw DB key directly with no resync involved, and extended coverage toLoadSeenCommit's cache too. All three independently verified red against upstreammain(the real pre-fix source, not just the first-pass commit) and green after.Disclosed, not fixed — flagging for a maintainer rather than expanding scope:
Load*Commit's DB-read-then-cache-Addsequence isn't performed underbs.mtx. A concurrentDeleteLatestBlockracing an in-flightLoad*Commitcall could theoretically repopulate a cache entry just after this fix evicts it. This race predates this PR — every existing cache-eviction call site in this file (e.g.PruneBlocks's ownblockExtendedCommitCache.Remove) has the identical gap — and closing it properly means broader locking discipline changes across the wholestorepackage, out of scope for a housekeeping PR like this one.Full package:
go test ./store/...→ok(2.1s, 15 tests).libs/protoio — byteReader mishandles two legal
io.Readerreturn shapes(0, nil)— legal per theio.Readerdoc ("nothing happened yet") — surfaced a stale or fabricated zero byte instead of retrying.(n>0, io.EOF)— also legal, explicitly documented — discarded an already-successfully-read byte along with the error. All existing tests usebytes.Buffer, which happens to never produce either shape.Fix: loop on
(0, nil); return the byte immediately whenevern == 1regardless of the accompanying error (deferring any non-EOF error to the next call). This exactly matchesbufio.Reader's ownfill()/readErr()pattern in the standard library — verified by reading the real stdlib source, not assumed. Receipts:Codex flagged (correctly) that an unbounded
(0, nil)retry loop could hang forever against a pathological reader. Bounded it with the identicalmaxConsecutiveEmptyReads = 100 → io.ErrNoProgresspatternbufio.Readeritself uses.TestByteReaderGivesUpOnPersistentZeroByteReaderuses a goroutine + 5s test timeout to directly prove the unbounded version genuinely hangs (confirmed by temporarily reverting just the bound and watching the test time out) and the bounded version returns cleanly.Full package:
go test ./libs/protoio/...→ok(1.3s, 9 tests).libs/pubsub/query — numeric operators never match a negative event attribute value
extractNum's regex was^\d+(\.\d+)?— no support for a leading-. A negative event attribute (e.g."-5") extracted an empty string,parseNumberreturned an error, and=/</<=/>/>=all share the sameerr == nil && ...guard — so every comparison against a negative attribute silently evaluated tofalse.Fix:
^-?\d+(\.\d+)?. Receipts:Note on scope: the query literal itself still can't be negative — the grammar's own number scanner doesn't accept a leading
-in query text at all, a separate pre-existing limitation this PR doesn't touch. Tests compare only against a non-negative literal (0), which is sufficient to isolate this fix. Codex correctly caught that an earlier draft of this test included adelta.value = 0case that wasfalseboth before and after the fix (-5 != 0either way) — removed it and left a comment explaining why=isn't independently provable through the query-syntax path even though the fix covers it via the sameparseNumberthe ordering-operator tests do exercise.Full package:
go test ./libs/pubsub/...→ok(all three subpackages).Review process
Ran Codex (gpt-5.6-sol) as a synchronous adversarial reviewer against the first-pass commit before pushing. It surfaced six concrete, real findings — two led to genuine additional fixes (the legacy-state bucket handling, the busy-spin bound), three led to test-quality improvements (the DB-row-vs-cache distinction, the misleading
=subtest, the seen-commit coverage), and one (thebyteReadern=1+err handling) was investigated against the real Go stdlib and confirmed to already be correct, matchingbufio.Reader's own idiom — kept as-is with the reasoning disclosed above rather than silently ignored or blindly changed.What's not in this PR
A separate, genuinely security-sensitive finding from the same audit pass — an unrecovered nil-pointer dereference in the mempool recheck path (
findNextEntryMatching), reachable under a specific recheck-timeout + stale-response timing condition — is deliberately excluded. Publishing the exact trigger conditions in a public diff before a fix ships would be irresponsible for a consensus node's mempool; it's being routed through coordinated disclosure instead.