Skip to content

[Fix] getLength: compute the length of a ranged fs.ReadStream correctly - #614

Open
OsamaAnsar wants to merge 1 commit into
form-data:masterfrom
OsamaAnsar:fix/ranged-readstream-length
Open

OsamaAnsar wants to merge 1 commit into
form-data:masterfrom
OsamaAnsar:fix/ranged-readstream-length

Conversation

@OsamaAnsar

Copy link
Copy Markdown

What was wrong

_lengthRetriever computes the length of an fs.ReadStream part. It only used end + 1 - start when both start and end were set, and otherwise used stat.size - start. That gives a wrong getLength() (and so a wrong Content-Length, which means a hung or a truncated upload) in these cases:

  • { end: N } without start: node honors end on its own (the stale TODO blamed a node bug from 2014), but the whole file size was reported.
  • end past the end of the file: not clamped to the file size.
  • start past the end of the file: a negative length was reported.

Repro

A 1000 byte file, each stream appended to a form, getLength() compared with the number of bytes the form actually serializes (Node v24.19.0, Windows):

                                          before                                      after
{end: 199}                                getLength()=1224  actual body bytes=424     getLength()=424   actual body bytes=424
{start: 900, end: 5000}                   getLength()=4325  actual body bytes=324     getLength()=324   actual body bytes=324
{start: 0, end: 5000}                     getLength()=5225  actual body bytes=1224    getLength()=1224  actual body bytes=1224
{end: 0}                                  getLength()=1224  actual body bytes=225     getLength()=225   actual body bytes=225
{start: 2000}                             getLength()=-776  actual body bytes=224     getLength()=224   actual body bytes=224

The cases that were already correct (no range, {start: 100}, {start: 100, end: 199}, {start: 0, end: 199}, {start: 100, end: Infinity}, {start: 999, end: 999}, {end: 5000}) still match.

Fix

Always stat the file, clamp end + 1 to the file size and never report a negative length:

var start = value.start ? value.start : 0;
var end = value.end != null && value.end !== Infinity ? Math.min(value.end + 1, stat.size) : stat.size;
callback(null, Math.max(0, end - start));

The old fast path for start + end is gone, since it can not know whether end is past EOF without the file size. A stream that was created from an existing fd has no path (fs.createReadStream(null, { fd, start, end })), and the fast path used to be the only thing that made getLength() work for those with a range, so for them the descriptor is fstated instead. I also removed the stale TODO.

Tests

Added test/integration/test-ranged-filestream-length.js. It uses unicycle.jpg (19806 bytes), which is big enough for these cases to matter (the existing test-ranged-filestream.js uses a 21 byte file, and only sets options.end when it is truthy). For 13 ranges, plus an fd stream without a path, it asserts that the stream provides the expected number of file bytes, and that getLength() equals the size of the serialized body.

Verified that the test fails without the fix: reverting only the lib/form_data.js change makes the test throw (start and end, both past the end of the file: getLength() does not match the serialized body, 5114 !== 213), and with the change restored it passes.

On this branch (Windows, Node v24.19.0), npm test exits 0: npm run lint reports 0 errors (32 warnings, the same count as on master), the suite reports 0 errors in 30 files, and the npm audit --production step reports found 0 vulnerabilities. The browser tests (npm run browser) were not run.

…rectly

`_lengthRetriever` only used `end + 1 - start` when both `start` and `end`
were set, and otherwise fell back to the full file size minus `start`.

- `fs.createReadStream(file, { end: N })` without `start` reads `N + 1`
  bytes (node honors `end` on its own, so the old TODO is stale), but
  `getLength` reported the whole file size.
- An `end` past the end of the file was not clamped to the file size, so
  the reported length was larger than the body that gets sent.
- A `start` past the end of the file produced a negative length.

A wrong Content-Length results in a hung or a truncated request. Always
`stat` the file (or `fstat` the descriptor for a stream that was created
from an `fd` and has no `path`), clamp `end` to the file size, and never
report a negative length.

Add an integration test that compares `getLength` with the size of the
serialized body for a file that is big enough for these cases to matter.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant