Skip to content

gh-88843: Do not allow to parse line breaks between quotes in sniffer regexes.#27256

Closed
ptokarski wants to merge 1 commit into
python:mainfrom
ptokarski:fix_csv_sniffer_regexes
Closed

gh-88843: Do not allow to parse line breaks between quotes in sniffer regexes.#27256
ptokarski wants to merge 1 commit into
python:mainfrom
ptokarski:fix_csv_sniffer_regexes

Conversation

@ptokarski

@ptokarski ptokarski commented Jul 20, 2021

Copy link
Copy Markdown

bpo-44677: Do not allow to parse line breaks between quotes in CSV sniffer regexes

@the-knights-who-say-ni

This comment was marked as outdated.

@github-actions

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Aug 20, 2021
@CoolCat467

Copy link
Copy Markdown
Contributor

ptokarski has signed CLA, but piotr-profound has not.

@github-actions github-actions Bot removed the stale Stale PR or inactive for long period of time. label Feb 19, 2022
@hugovk

This comment was marked as resolved.

@github-actions

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Aug 11, 2022
@StanFromIreland StanFromIreland changed the title Do not allow to parse line breaks between quotes in sniffer regexes. gh-88843: Do not allow to parse line breaks between quotes in sniffer regexes. Oct 3, 2025
@github-actions github-actions Bot removed the stale Stale PR or inactive for long period of time. label Apr 9, 2026
@github-actions

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label May 10, 2026
@serhiy-storchaka

Copy link
Copy Markdown
Member

Thank you for the patch, and sorry for the long delay.

On the main branch the sniffer was rewritten (gh-83273), so these regular expressions no longer exist there and the sample from the issue is detected correctly now.

For the maintenance branches this change is too risky: a quoted field may legitimately contain a line break, and forbidding that in the pattern loses the evidence for the real delimiter. On 4000 generated samples it changes the guess for 1.4% of them, and in all of those cases the new guess is worse — for example '"multi\nline"\tit\'s\nit\'s\t"multi\nline"\n' is detected as '\t' before the change and as 'l' after it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting review stale Stale PR or inactive for long period of time.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants