Skip to content

fix(muya): apply pandoc's tex_math_dollars constraints to $…$ (#5446) - #5449

Merged
Jocs merged 1 commit into
developfrom
fix/muya-tex-math-dollars-constraints
Sep 18, 2026
Merged

Jocs merged 1 commit into
developfrom
fix/muya-tex-math-dollars-constraints

Conversation

@Jocs

@Jocs Jocs commented Sep 18, 2026

Copy link
Copy Markdown
Member

Task 1 of #5446 β€” the one task there that fixes reported breakage under default settings, with no new preference.

The problem

muya has two independent $ math tokenizers, guarded differently, so the same document parsed differently while editing and on export:

Revenue rose from $13B to $24B.
  live editor  β†’  "$13B to $" goes to KaTeX
  HTML export  β†’  literal text

a $13B to $24B and $x+y$ b
  live editor  β†’  "$13B to $" is a formula, the real $x+y$ is missed
  HTML export  β†’  one formula spanning "$13B to $24B and $x+y$"

This is the substance of #2002 (2020), #4231 (closed unmerged) and #5243 (which reports binary-patching app.asar as a workaround): not a missing preference, but a tokenizer looser than the one it is supposed to agree with.

The change

Both rules now carry pandoc's three tex_math_dollars constraints β€” a non-space right after the opening $, a non-space right before the closing $, and no digit right after the closing $ β€” and both leave $$…$$ display math unconstrained, as pandoc does.

  • packages/muya/src/inlineRenderer/rules.ts β€” the live-editor inline_math rule. $(?=\S) mirrors how strong and em already spell the opening constraint in that same file; the other two ride a leading veto, because they must not apply to $$…$$ ($$ E=MC^2 $$ is padded on both sides).
  • packages/muya/src/utils/marked/extensions/math.ts β€” the import / export / clipboard rule, same shape minus bare newlines.

The marked extension's start() hint had to change too: it tested only the first $ on the line and gave up when that one did not open a formula, which with the tighter rule hid every later formula on the line. It now walks the candidates.

Capture-group numbering is unchanged in both (1 = marker, 2 = content), which the lexer's isLengthEven(to[3]) guard and the marked tokenizer both depend on.

Behaviour

input before (editor / export) after (both)
Revenue rose from $13B to $24B. $13B to $ / β€” β€”
It costs $5 and $10 total. $5 and $ / β€” β€”
price is $20, tax $3. $20, tax $ / β€” β€”
a $13B to $24B and $x+y$ b $13B to $, $x+y$ / one span over the lot $x+y$
$ x$, $x $ math / math β€”
$x+y$, $1+1$, $y = \$10000$ math / math unchanged
$$ E=MC^2 $$, $$a$$ $$b$$, $$x$y$$, $$a$$2 unchanged unchanged

One behaviour follows from aligning the two sides: on export a $ may now open a formula mid-word (word$E=mc^2$word), which the editor and pandoc have always allowed. A user-facing switch for dollar math is #5446's task 2 (texMathDollars), not this PR.

Tests

  • packages/muya/src/inlineRenderer/__tests__/texMathDollars.spec.ts β€” one corpus run through both tokenizers, so their agreement is what is pinned rather than each side separately.
  • packages/muya/e2e/tests/inline/tex-math-dollars.spec.ts β€” the editor the symptom was reported against. Verified as a positive control: both cases fail on develop (1 stray KaTeX for the currency prose, 2 formulas instead of 1 for the mixed line) and pass here.

Also run: full muya unit suite (250 files / 1857 tests), CommonMark 0.31 + GFM 0.29 conformance (1347), lint, lint:types, check-circular, and the muya Playwright suite. The #5365 backtracking guard ('$$' + '\\'.repeat(131072) + '$') still completes in ~1 ms.

Closes #2002
Closes #5243

πŸ€– Generated with Claude Code

`$…$` had no flanking rules in the live editor, so prose like
"Revenue rose from $13B to $24B." handed `$13B to $` to KaTeX while
typing. The marked-based import/export path guarded the same syntax
differently, so the two tokenizers disagreed about the same document:
the editor found `$13B to $`, the export found one formula spanning
`$13B to $24B and $x+y$`.

Both now apply pandoc's three `tex_math_dollars` rules β€” a non-space
right after the opening `$`, a non-space right before the closing `$`,
and no digit right after it β€” and leave `$$…$$` display math
unconstrained, as pandoc does.

On the marked path the start hint also had to change: it tested only
the first `$` on the line and gave up when that one did not open a
formula, which with the tighter rule hid every later formula on the
line. It now walks the candidates.

One behaviour follows from aligning the two: on export a `$` may open
a formula mid-word (`word$E=mc^2$word`), which the editor and pandoc
have always allowed.

The new unit spec runs one corpus through both tokenizers, so their
agreement is what it pins; the e2e covers the editor the symptom was
reported against.

Task 1 of #5446.

Closes #2002
Closes #5243

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Jocs
Jocs merged commit 7f8de79 into develop Sep 18, 2026
7 checks passed
@Jocs
Jocs deleted the fix/muya-tex-math-dollars-constraints branch September 18, 2026 16:00
Jocs added a commit that referenced this pull request Sep 21, 2026
…5483) (#5484)

* feat(desktop): follow the math preferences in source mode (#5483)

The `markdown-math` mode has never read a preference. `codeMirror/index.ts`
registers it at module load and `sourceCode.vue` asked for it by name, so the
source view delegated `$…$` to stex whether or not `texMathDollars` was on.

That went unnoticed because the preference ships enabled, but #5446 added
three more math syntaxes and they ship disabled β€” the same shortcut would be
wrong in the opposite direction for them. So the mode now takes its regions
from the mode spec, which is where the delimiters that follow can join.

Visible change, and the intended one: with `texMathDollars` off, the source
view stops treating `$` as a delimiter, matching what the editor has done
since #5450. That is what #2002 and #5243 asked for β€” the latter reported
patching app.asar to get it.

The factory's defaults reproduce the old behaviour, so a caller still passing
the bare string `'markdown-math'` is unaffected. A fresh CodeMirror instance
reads the preferences at mount (`index.vue` builds the source view behind a
`v-if`, so every entry is a new one); the watch covers a preference changed
while the view is already open, mirroring the one #5111 added for
`sourceCodeLineNumbers`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(desktop): apply pandoc's dollar-math constraints to the source view guard (#5483)

The inline `$…$` open guard said it mirrored Muya's `inline_math` rule, and
pointed at `src/muya/lib/parser/rules.js` β€” a path that belongs to the engine
this editor no longer uses. It drifted further when #5449 gave the editor
pandoc's three `tex_math_dollars` constraints, which the guard never gained.

So the source view still opened a math span on `Revenue rose from $13B to
$24B.` while the editor read it as prose β€” the currency case #2002 and #5243
reported, on the very input #5446 used to describe the problem.

The guard now carries the same three constraints: a non-space right after the
opener, a non-space right before the closer, and no digit right after it. It
still requires a closer on the line, which is this view's own rule rather than
pandoc's: CodeMirror tokenizes per line and the multiplexer's state survives
across them, so an unclosed opener would render the rest of the file as LaTeX.

The guard only decides whether to enter stex. The region's closer is the plain
string `$` and leaves on the next one without re-checking; a closer regexp
would need the text before it, which `StringStream.match` does not expose.

The existing currency case passed throughout: the old guard opened its span at
`$5` and closed it before `only`, which is what the assertion looked for. It
now checks the span contents instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(desktop): highlight the backslash TeX math delimiters in source mode (#5483)

`\(…\)`, `\[…\]`, `\\(…\\)` and `\\[…\\]` render as formulas in the WYSIWYG
view since #5481 but were plain prose in the source view β€” and worse than
plain: `markdown-math` exists because the markdown mode reads the underscores
in `\(\text{F}_\text{A}\)` as emphasis (#4121), which it was still doing for
these.

Each pair now gets a stex region, riding the preference that governs it.

Notes:

* The double-backslash regions are registered first. The multiplexer takes the
  first spec matching at the cursor, and the literal `\(` matches at the second
  character of `\\(`.
* The inline forms need their own open guards, not `INLINE_MATH_OPEN`: part of
  that one requires the character before the closer not to be a backslash, and
  for `\)` it is one. The content rules come from Muya's own regexes, including
  the two escape rules that differ between the extensions, so both views draw
  the same span. Display math legitimately spans lines, so like `$$` it carries
  no guard.
* Known limit, commented at the guards: `\\(x\)` β€” a double-backslash opener
  closed with a single-backslash closer β€” is read as math when only the single
  extension is on. The multiplexer searches a regexp against the rest of the
  line, so a guard cannot see the backslash in front of the opener. The shape
  is invalid in both dialects, and the well-formed `\\(x\\)` is unaffected.
* `- \[TODO\] item` becomes a formula once the extension is on. That is pandoc's
  documented drawback β€” the extension "precludes escaping `(` and `[`" β€” the
  editor agrees, and it is why both preferences ship disabled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(desktop): trim the markdown-math mode to the comments that earn it

The file had grown to 42% comments, several of them restating the code or
recording archaeology (why the outer mode is gfm, what `setMode` used to
resolve to) rather than anything a reader needs.

The three delimiter pairs also carried three copies of the same two region
literals. Folding them into one helper removes the duplication and, with it,
the comments that existed to explain it β€” including the note about display
regions preceding inline ones, which was never load-bearing.

What stays is what the code cannot say: why this mode exists at all, the
constraints each open guard encodes and where they come from, why the
backslash guards cannot reuse the dollar one, the ordering the multiplexer
demands, and the one shape it reads differently from the editor.

156 lines to 111; no behaviour change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(desktop): stop a backslash math region claiming the other one's block (#5483)

With only `texMathSingleBackslash` on, a `\\[…\\]` block highlighted as math β€”
the very syntax that preference does not cover. The editor reads `\\` there as
an escaped backslash and finds no formula, so the two views disagreed exactly
where this change set out to make them agree.

The display openers span lines, so unlike the inline ones they carry no closer
lookahead. `\[` was a plain string, and `\\[` contains it one character in: the
multiplexer found it at that offset and opened a region that ran to the
matching `\]`, swallowing the block whichever way the double-backslash
preference was set.

Every opener now refuses a preceding backslash. That needs the openers to be
regexps, and regexps could not see what precedes them: `indexOf` in our
vendored `multiplexingMode` searched `string.slice(from)`, hiding the text
before the cursor. It now searches the whole line from an offset, which keeps
lookbehind working β€” at the cost of `^` in an opener anchoring to the start of
the line rather than to the cursor, noted there. No opener in this codebase is
anchored.

The same guard retires the limitation the previous commit documented: `\\(x\)`
is no longer read as single-backslash math either.

The regression test sets its own preferences rather than leaning on the test
before it β€” with the extension off the assertion is vacuous, which is how a
first version of it passed against the unfixed code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(desktop): set the math delimiters back from the formula in source mode (#5483)

`$`, `\(`, `\[` and their closers carried the multiplexer's `delimStyle`
classes but no rule ever painted them, so a delimiter was as loud as the
formula it wrapped β€” unlike the WYSIWYG view, which greys its own markers.

They now keep the surrounding colour at reduced opacity. Fading what is
inherited rather than naming a hue is what makes this work across the three
CodeMirror themes the app ships and the thirty-odd MarkText palettes behind
them; no single colour could sit on both `#ffffff` and `#161616`.

0.65 is the lowest value that clears 3:1 β€” the threshold this sort of
de-emphasised punctuation is held to β€” in every shipped theme. Measured:
light 7.0, graphite 6.85, ulysses 6.72, tokyo-night-light 6.0, oxocarbon-dark
7.28, synthwave-84 6.57, material-dark 5.43, one-dark 3.67. one-dark sets the
floor because its body text only reaches 6.57:1, so anything fainter
disappears there long before it does anywhere else.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(desktop): give GitHub inline math its own source-view region (#5483)

`` $`e=mc^2`$ `` opens on `$`, so the dollar rule claimed the span and handed
the backticks to stex as part of the formula β€” the same mis-reading #5457 fixed
in the editor, still on show in the source view, and unaffected by the
`texMathGfm` preference either way.

It now has its own region, offered ahead of the dollar ones because its opener
is the longer of the two. With the extension off the dollar rule keeps the span
exactly as before, which is what `tex_math_dollars` alone prescribes and what
the editor does.

The three delimiter pairs were building their regions through one helper that
took a display pair and an inline pair together; gfm has no display half, so
the helper splits into `displayRegion` and `inlineRegion` and each extension
pushes what it actually has.

Out of scope, and unchanged: ```` ```math ```` is a fenced code block rather
than a delimiter pair, so it reaches stex through a different route than a
multiplexer region.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: trim the source-mode math comments to what the code cannot say

144 comment lines across the four files this PR touches, down to 102. What
went was restatement and detail a reader does not need to act on: how
`StringStream.match` hides context, why display openers need the guard more
than inline ones, that inner modes are imported eagerly.

Two comments had drifted from the code they describe β€” later edits inserted
the gfm opener between the backslash comment and its constants, and the
delimiter-contrast note between the currency comment and its test. Both are
back above what they explain, and one had picked up doubled backslashes on the
way through a script.

Comments only; no behaviour change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Wangqkoe pushed a commit to Wangqkoe/marktext that referenced this pull request Oct 21, 2026
…text#5446) (marktext#5449)

`$…$` had no flanking rules in the live editor, so prose like
"Revenue rose from $13B to $24B." handed `$13B to $` to KaTeX while
typing. The marked-based import/export path guarded the same syntax
differently, so the two tokenizers disagreed about the same document:
the editor found `$13B to $`, the export found one formula spanning
`$13B to $24B and $x+y$`.

Both now apply pandoc's three `tex_math_dollars` rules β€” a non-space
right after the opening `$`, a non-space right before the closing `$`,
and no digit right after it β€” and leave `$$…$$` display math
unconstrained, as pandoc does.

On the marked path the start hint also had to change: it tested only
the first `$` on the line and gave up when that one did not open a
formula, which with the tighter rule hid every later formula on the
line. It now walks the candidates.

One behaviour follows from aligning the two: on export a `$` may open
a formula mid-word (`word$E=mc^2$word`), which the editor and pandoc
have always allowed.

The new unit spec runs one corpus through both tokenizers, so their
agreement is what it pins; the e2e covers the editor the symptom was
reported against.

Task 1 of marktext#5446.

Closes marktext#2002
Closes marktext#5243

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 7f8de79)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Setting to turn off math parsing ($...$). Users are patching the app to get it Allow disable inline math format ($)

1 participant