bug: Marked treats tabs and spaces differently in lists #3784

akilesh1706 · 2025-10-03T16:25:05Z

Marked version: 11.0.0 (bug reported against 11.0.0). Fix implemented on master.

Markdown flavor: CommonMark / GitHub Flavored Markdown (GFM)

Description

When a list item used a tab after the list marker (e.g. 1.\tText or -\tText), and there was preceding text (a paragraph) before the list, the tokenizer sometimes treated the item content as an indented code block (leading four spaces) instead of normal list content. This produced

...

inside

for the tab-using list, while the same list using spaces produced normal

Text

.

This PR fixes that inconsistency so tabs after the marker are interpreted the same as spaces regardless of text preceding the list.

Fixes the inconsistent tab handling in list items that could make tabbed list items render as indented code blocks.
Files changed (key):

Expectation
Given input (tabs or spaces after marker):

I am using spaces after the number:

Some Text
Some Text
Some Text
I am using tabs after the number: 1. Some Text 2. Some Text 3. Some Text

Expected HTML (both cases):

(And similar for unordered lists)

Result (before this PR)

Lists that used tabs after the marker were sometimes parsed as indented code blocks and rendered as:

```
Some Text 
```

while the same lists using spaces were parsed/rendered correctly as simple

Some Text

.

What was attempted / what this PR does

Normalizes the first line of a list item by replacing tabs with the same tab normalization used elsewhere in the codebase (tabs -> 4 spaces), then computes indent and slices content from the normalized string so the slice length is correct when tabs are used.
Ensures indent detection replaces tabs with spaces consistently when finding the first non-space character.
Adds tests to cover:
ordered lists using spaces vs tabs,
ordered lists that follow a paragraph (the reported scenario),
unordered lists using spaces vs tabs.

vercel · 2025-10-03T16:25:10Z

@akilesh1706 is attempting to deploy a commit to the MarkedJS Team on Vercel.

A member of the Team first needs to authorize it.

gemini-code-assist · 2025-10-03T16:25:19Z

Summary of Changes

Hello @akilesh1706, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a parsing inconsistency in the Marked library where list items using tabs after their markers were sometimes misinterpreted as indented code blocks, particularly when following a paragraph. The changes normalize tab characters to spaces during the initial processing of list item lines and refine indent calculation, ensuring that Marked consistently renders list items whether tabs or spaces are used after the marker. This improves the robustness and predictability of markdown rendering.

Highlights

Inconsistent List Parsing: Fixed a bug where Marked inconsistently parsed list items, treating content after a tab-separated marker as an indented code block instead of normal list content, especially when preceded by a paragraph.
Tab Normalization: Implemented consistent normalization of tabs to 4 spaces within list item parsing logic, ensuring uniform handling of tabs and spaces after list markers.
Improved Indent Calculation: Enhanced the indent calculation for list items to correctly account for tabs by treating them as 4 spaces when determining the first non-space character.
Comprehensive Testing: Added new unit tests covering ordered and unordered lists with both tabs and spaces after markers, including the specific scenario of lists following a paragraph, to validate the fix.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request correctly addresses an inconsistency in how tabs are handled within lists, and the new tests provide good coverage for the fix. However, I've identified a critical indentation error that appears to be accidental and would break list parsing. Additionally, I've provided a couple of suggestions to improve the clarity and efficiency of the updated tokenizer logic.

src/Tokenizer.ts

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

vercel · 2025-10-04T04:52:42Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
marked-website	Ready	Preview	Comment	Oct 4, 2025 4:52am

test/unit/marked.test.js

UziTech · 2025-10-04T06:15:09Z

test/unit/marked.test.js

+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+
+const mdPath = path.join(__dirname, '../specs/new/list_with_tabs.md');


this is not needed. the files in the new folder are already ran in other tests

akilesh1706 · 2025-10-04T13:52:50Z

Necessary changes done, requesting for review.

UziTech

tests don't pass

UziTech · 2025-10-16T01:43:18Z

@akilesh1706 are you planning on continuing to work on this?

akilesh1706 · 2025-10-16T01:44:54Z

@akilesh1706 are you planning on continuing to work on this?

Yes, I'm resolving the CI test passes.

bug: Marked treats tabs and spaces differently in lists

d6c5c82

gemini-code-assist bot reviewed Oct 3, 2025

View reviewed changes

src/Tokenizer.ts Outdated Show resolved Hide resolved

src/Tokenizer.ts Outdated Show resolved Hide resolved

src/Tokenizer.ts Outdated Show resolved Hide resolved

akilesh1706 and others added 3 commits October 3, 2025 22:02

Update src/Tokenizer.ts

2ee3a33

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Update src/Tokenizer.ts

7448424

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Update src/Tokenizer.ts

951f251

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

vercel bot deployed to Preview October 4, 2025 04:52 View deployment

UziTech linked an issue Oct 4, 2025 that may be closed by this pull request

Marked treats tabs and spaces differently in lists #3126

Open

UziTech requested changes Oct 4, 2025

View reviewed changes

test/unit/marked.test.js Show resolved Hide resolved

akilesh1706 added 2 commits October 4, 2025 11:06

created list_with_tabs.md and edited marked.test.js

dd933de

Merge branch 'master' of https://github.com/akilesh1706/marked

b950c97

UziTech requested changes Oct 4, 2025

View reviewed changes

Update marked.test.js

a7a6bb7

UziTech requested changes Oct 7, 2025

View reviewed changes

UziTech mentioned this pull request Oct 15, 2025

Marked treats tabs and spaces differently in lists #3126

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

bug: Marked treats tabs and spaces differently in lists #3784

bug: Marked treats tabs and spaces differently in lists #3784

akilesh1706 commented Oct 3, 2025

Uh oh!

vercel bot commented Oct 3, 2025

Uh oh!

gemini-code-assist bot commented Oct 3, 2025

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vercel bot commented Oct 4, 2025 •

edited

Loading

Uh oh!

Uh oh!

UziTech Oct 4, 2025

Uh oh!

akilesh1706 commented Oct 4, 2025

Uh oh!

UziTech left a comment •

edited

Loading

Uh oh!

UziTech commented Oct 16, 2025

Uh oh!

akilesh1706 commented Oct 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bug: Marked treats tabs and spaces differently in lists #3784

Are you sure you want to change the base?

bug: Marked treats tabs and spaces differently in lists #3784

Conversation

akilesh1706 commented Oct 3, 2025

Description

Expected HTML (both cases):

Result (before this PR)

What was attempted / what this PR does

Uh oh!

vercel bot commented Oct 3, 2025

Uh oh!

gemini-code-assist bot commented Oct 3, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vercel bot commented Oct 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

UziTech Oct 4, 2025

Choose a reason for hiding this comment

Uh oh!

akilesh1706 commented Oct 4, 2025

Uh oh!

UziTech left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

UziTech commented Oct 16, 2025

Uh oh!

akilesh1706 commented Oct 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vercel bot commented Oct 4, 2025 •

edited

Loading

UziTech left a comment •

edited

Loading