Repository navigation
[Fix] getLength: compute the length of a ranged fs.ReadStream correctly - #614
Open
OsamaAnsar wants to merge 1 commit into
Open
OsamaAnsar wants to merge 1 commit into
OsamaAnsar wants to merge 1 commit into
Conversation
…rectly
`_lengthRetriever` only used `end + 1 - start` when both `start` and `end`
were set, and otherwise fell back to the full file size minus `start`.
- `fs.createReadStream(file, { end: N })` without `start` reads `N + 1`
bytes (node honors `end` on its own, so the old TODO is stale), but
`getLength` reported the whole file size.
- An `end` past the end of the file was not clamped to the file size, so
the reported length was larger than the body that gets sent.
- A `start` past the end of the file produced a negative length.
A wrong Content-Length results in a hung or a truncated request. Always
`stat` the file (or `fstat` the descriptor for a stream that was created
from an `fd` and has no `path`), clamp `end` to the file size, and never
report a negative length.
Add an integration test that compares `getLength` with the size of the
serialized body for a file that is big enough for these cases to matter.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
_lengthRetrievercomputes the length of anfs.ReadStreampart. It only usedend + 1 - startwhen bothstartandendwere set, and otherwise usedstat.size - start. That gives a wronggetLength()(and so a wrongContent-Length, which means a hung or a truncated upload) in these cases:{ end: N }withoutstart: node honorsendon its own (the stale TODO blamed a node bug from 2014), but the whole file size was reported.endpast the end of the file: not clamped to the file size.startpast the end of the file: a negative length was reported.Repro
A 1000 byte file, each stream appended to a form,
getLength()compared with the number of bytes the form actually serializes (Node v24.19.0, Windows):The cases that were already correct (no range,
{start: 100},{start: 100, end: 199},{start: 0, end: 199},{start: 100, end: Infinity},{start: 999, end: 999},{end: 5000}) still match.Fix
Always
statthe file, clampend + 1to the file size and never report a negative length:The old fast path for
start+endis gone, since it can not know whetherendis past EOF without the file size. A stream that was created from an existingfdhas nopath(fs.createReadStream(null, { fd, start, end })), and the fast path used to be the only thing that madegetLength()work for those with a range, so for them the descriptor isfstated instead. I also removed the stale TODO.Tests
Added
test/integration/test-ranged-filestream-length.js. It usesunicycle.jpg(19806 bytes), which is big enough for these cases to matter (the existingtest-ranged-filestream.jsuses a 21 byte file, and only setsoptions.endwhen it is truthy). For 13 ranges, plus anfdstream without apath, it asserts that the stream provides the expected number of file bytes, and thatgetLength()equals the size of the serialized body.Verified that the test fails without the fix: reverting only the
lib/form_data.jschange makes the test throw (start and end, both past the end of the file: getLength() does not match the serialized body,5114 !== 213), and with the change restored it passes.On this branch (Windows, Node v24.19.0),
npm testexits 0:npm run lintreports 0 errors (32 warnings, the same count as onmaster), the suite reports0 errors in 30 files, and thenpm audit --productionstep reportsfound 0 vulnerabilities. The browser tests (npm run browser) were not run.