feat: add CJK friendly emphasis extension #1059
Open
+248
−9
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ollpu
reviewed
Nov 3, 2025
pulldown-cmark/src/firstpass.rs
Outdated
Comment on lines
2255
to
2265
| #[inline] | ||
| fn previous_two_chars(s: &str, ix: usize) -> (Option<char>, Option<char>) { | ||
| let mut iter = s[..ix].chars(); | ||
| let mut prev_prev = None; | ||
| let mut prev = None; | ||
| while let Some(ch) = iter.next() { | ||
| prev_prev = prev; | ||
| prev = Some(ch); | ||
| } | ||
| (prev, prev_prev) | ||
| } |
There was a problem hiding this comment.
This iterates through the full string, which makes emphasis parsing O(n^2), as caught by CI.
The previous implementation uses .chars().last(), which takes advantage of DoubleEndedIterator. Also, I wouldn't put #[inline] on an internal function unless a benchmark indicates it helps.
Suggested change
| #[inline] | |
| fn previous_two_chars(s: &str, ix: usize) -> (Option<char>, Option<char>) { | |
| let mut iter = s[..ix].chars(); | |
| let mut prev_prev = None; | |
| let mut prev = None; | |
| while let Some(ch) = iter.next() { | |
| prev_prev = prev; | |
| prev = Some(ch); | |
| } | |
| (prev, prev_prev) | |
| } | |
| fn previous_two_chars(s: &str, ix: usize) -> (Option<char>, Option<char>) { | |
| let mut iter = s[..ix].chars().rev(); | |
| let prev = iter.next(); | |
| let prev_prev = iter.next(); | |
| (prev, prev_prev) | |
| } |
There was a problem hiding this comment.
Related, maybe this should not take ix, so that it's the caller's responsibility to slice the string.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
CommonMark has a problem that the following emphasis marks
**are not recognized as emphasis marks in CJK.This pull request introduces support for CJK-friendly emphasis handling in the Markdown parser, aligning with the CommonMark CJK-friendly amendments specification.
It adds a new option to enable CJK-friendly emphasis parsing, updates the delimiter run logic to properly handle CJK characters and punctuation, and includes comprehensive tests to verify the new behavior. By default, the feature is disabled to maintain backward compatibility.
In addition to the specification, I also refer to the Tips for Implementers and Concrete ranges of each terms in tats-u/markdown-cjk-friendly for implementation.