Skip to content

Read tzical files with the UTF-8 iCalendar charset - #1604

Open
fhgffy wants to merge 2 commits into
dateutil:masterfrom
fhgffy:fix/tzical-utf8-files
Open

fhgffy wants to merge 2 commits into
dateutil:masterfrom
fhgffy:fix/tzical-utf8-files

Conversation

@fhgffy

@fhgffy fhgffy commented Oct 9, 2026

Copy link
Copy Markdown

Summary of changes

tzical(filename) currently decodes iCalendar files with the locale encoding. On a cp936 Windows installation, a UTF-8 TZID such as Café becomes Caf茅. Open filename inputs explicitly as UTF-8, as required by the existing API documentation, using io.open() for the supported Python 2 versions. Add a file-based regression, author entry, and news fragment.

Closes #1207

Validation: the regression fails before the change and passes after it; 41 tzical tests passed with one platform skip. The complete Windows suite has the same four existing localized tzres-name failures as an unmodified-base run (1,999 vs 1,998 passing tests). Darker/isort/Black, whitespace/debug hooks, and sdist/wheel builds passed.

Pull Request Checklist

  • Changes have tests
  • Authors have been added to AUTHORS.md
  • News fragment added in changelog.d

@Wkhanex Wkhanex left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked file and stream inputs with Café, 東京, München and an ASCII TZID, using both CRLF and LF line endings. All eight cases retained the exact ID and expected UTC offset. A spy on io.open confirmed that filename inputs explicitly pass encoding='utf-8', while caller-owned StringIO objects remained open after parsing. A ninth case surfaced UnicodeDecodeError for malformed UTF-8 instead of accepting corrupted text.

These were standalone Linux checks, so they verify the explicit-encoding path without claiming a Windows run.

I used Python 3.12.14 against commit 7e9b819. These were focused checks, not a full-suite run.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tzical uses the wrong encoding on Windows

2 participants