perf: cut CSS/HTML parser memory and speed up long-token scanning - #21504
Merged
Conversation
A `<template>` element and a raw-text element never coexist on one node, so store a template's content-fragment ref in the existing `_nodeContentEnds` slot gated by a new `FLAG_HAS_TEMPLATE` bit, dropping one Int32 AST node column (~46 vs ~50 bytes/node of capacity). Behaviour-preserving and parse time unchanged (measured neutral); ~6% less AST typed-array memory.
…column A block rule's `blockEnd` always equals its `end` (the parser sets both to the `}` position) and is `-1` when there is no block (`blockStart === -1`), so store only `blockStart` and derive `blockEnd`. Drops the `_aux2` Int32 column across all nodes: ~21% less retained memory for `parseA*` trees, and slightly faster (one fewer column to grow/copy/snapshot). Behaviour-preserving.
`consumeAStringToken` / `consumeAUrlToken` read one JS char per code point, so a long `data:` URI or base64 string costs thousands of reads. Add two module-scope sticky regexes whose negated classes match exactly the loops' per-char terminators, and fast-forward over the ordinary run via `test()` (advances `lastIndex`, allocates nothing). ~45-65% faster on data-URI-heavy CSS, ~5% on ordinary CSS; behaviour-preserving (token boundaries and url content ranges unchanged).
The comment state read one JS char per code point while the data / RAWTEXT / RCDATA / script-data states already fast-forward with a memoized `indexOf`. Give the comment state the same treatment (a `nextHyphen` memo alongside `nextLt` / `nextNul`), so long comments (license banners, commented-out blocks) skip their body in native scans. ~80% faster on comment-heavy HTML, ~20% even when the comment is hyphen-dense; behaviour-preserving.
The bogus-comment (`<?…>`, `<!…>`, `</bad>`) and plaintext states still read one JS char per code point while every other long-scan tokenizer state fast-forwards with `indexOf`. Give them the same treatment — a `nextGt` memo for bogus comments and a single `indexOf` for the NUL-or-EOF plaintext body — so long processing instructions and large `<plaintext>` bodies skip in native scans. ~40% / ~27% faster on those inputs, neutral otherwise; behaviour-preserving.
🦋 Changeset detectedLatest commit: ba8064b The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
|
This PR is packaged and the instant preview is available (fc5459f). Install it locally:
npm i -D webpack@https://pkg.pr.new/webpack@fc5459f
yarn add -D webpack@https://pkg.pr.new/webpack@fc5459f
pnpm add -D webpack@https://pkg.pr.new/webpack@fc5459f |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #21504 +/- ##
==========================================
- Coverage 93.57% 93.57% -0.01%
==========================================
Files 619 619
Lines 73232 73242 +10
Branches 21105 21117 +12
==========================================
+ Hits 68526 68534 +8
- Misses 4706 4708 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Two lint errors on lines added by this branch: the CSS url fast-forward class used `\x` control escapes (no-control-regex / unicorn/no-hex-escape / escape-case), and the HTML content-end JSDoc put text on the `/**` line (jsdoc/multiline-blocks). Switch the class to `\u` escapes with a justified `no-control-regex` disable (the url token's terminators genuinely include the control range and DEL), and reflow the JSDoc. No behavior change.
Merging this PR will degrade performance by 12.14%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Memory | benchmark "css-modules", scenario '{"name":"mode-production","mode":"production"}' |
6.9 MB | 9.8 MB | -28.77% |
| ❌ | Memory | benchmark "many-modules-commonjs", scenario '{"name":"mode-production","mode":"production"}' |
7.6 MB | 9.8 MB | -22.39% |
| ⚡ | Memory | benchmark "many-chunks-esm", scenario '{"name":"mode-production","mode":"production"}' |
9.6 MB | 7.8 MB | +22.69% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing perf/css-html-parser-memory-and-scan (ba8064b) with main (61d4136)
Restore the two pre-existing changesets (035 HTML, 050 CSS) that this branch had folded into, and add a single dedicated changeset covering the two column drops and the native token-body scans. Resolves the changeset-bot "no changeset" flag and keeps this PR's release note separate from the earlier PRs' entries.
cspell doesn't know the plural noun "printables"; reword to "non-printable code points", matching the file's existing `_isNonPrintableCodePoint` wording.
Contributor
Types CoverageCoverage after merging perf/css-html-parser-memory-and-scan into main will be
Coverage Report |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The experimental CSS (
lib/css/syntax.js) and HTML (lib/html/syntax.js) struct-of-arrays parsers carried two AST node columns that are always derivable from other stored fields, and several tokenizer states scanned long token bodies one JS char at a time while their sibling states already fast-forward withindexOf. This PR drops the redundant columns and gives the remaining char loops the same native-scan treatment — one idea applied to each parser (drop a derivable column; scan long runs natively).Measured with standalone benchmarks against the parser modules (interleaved A/B, median of 23; before = the tree prior to these commits): CSS
data:-URI-heavy −48.6%, HTML comment-heavy −81%, PI/bogus-comment −49%, large<plaintext>−29%, and every other workload neutral-or-faster (no regressions). Memory: retained CSSparseA*columns −11.8%, HTML peak columns −5.9% (one Int32 column each). Refs n/a.What kind of change does this PR introduce?
perf
Did you add tests for your changes?
No new tests — every change is behavior-preserving, and the changed lines are exercised by the existing
html5libtree-construction conformance suite and theCssSyntax/HtmlSyntaxunit tests (templates, comments, processing instructions, plaintext, strings,url()tokens, block rules). Each change was additionally validated by differential tree-serialization against the pre-change parser across the relevant edge cases.Does this PR introduce a breaking change?
No.
If relevant, what needs to be documented once your changes are merged or what have you already documented?
n/a — internal parser changes; no public API, config, or output surface changes.
Use of AI
Yes. Claude (via Claude Code) analyzed both SoA parsers to find divergences and optimization opportunities, measured every candidate for CPU and memory against the pre-change parser, and discarded the ones that did not hold up (an ident-sequence regex that regressed short idents, a widened dispatch table, intern-name lowercasing,
parseA*pre-sizing, an iterative walk). The five changes here are the ones that measured as clear, regression-free wins, each verified behavior-preserving by differential testing. Analysis and implementation were AI-assisted as described.🤖 Generated with Claude Code
https://claude.ai/code/session_016RA5wBcgNR7YdwtAnb3ysm
Generated by Claude Code