Skip to content

gh-162080: Fix UnicodeEncodeError for non-ASCII MIME-Version header - #162195

Open
xgauravyaduvanshii wants to merge 1 commit into
python:mainfrom
xgauravyaduvanshii:fix/email-mime-version-non-ascii-162080
Open

xgauravyaduvanshii wants to merge 1 commit into
python:mainfrom
xgauravyaduvanshii:fix/email-mime-version-non-ascii-162080

Conversation

@xgauravyaduvanshii

@xgauravyaduvanshii xgauravyaduvanshii commented Nov 18, 2026 •

Copy link
Copy Markdown

Fixes #162080

A message whose MIME-Version header contains an undecodable (non-ASCII) byte parsed without error under policy.default, but as_bytes() / as_string() raised UnicodeEncodeError:

import email
from email import policy

msg = email.message_from_bytes(b'MIME-Version: 1.\xe9\n\nbody\n', policy=policy.default)
msg.as_bytes()  # UnicodeEncodeError: 'utf-8' codec can't encode character '\udce9' ...

Root cause: parse_mime_version() appended the invalid text as a bare ValueTerminal(..., 'xtext') and never called _validate_xtext(), so no UndecodableBytesDefect was registered and _refold_parse_tree() chose utf-8 instead of unknown-8bit for the encoded word.

Fix: run _validate_xtext() on the four invalid-text terminals created in parse_mime_version() (invalid major digits, trailing text with no dot, invalid minor digits, excess text after the version). The header is now refolded as MIME-Version: 1.=?unknown-8bit?q?=E9?=, as for other structured headers, and carries an UndecodableBytesDefect.

Tests:

  • Added 3 regression tests in Lib/test/test_email/test_headerregistry.py (TestMIMEVersionHeader): non-ASCII byte in the minor version, in the major version, and in excess text after the version.
  • Full test_email suite on a build of this branch: 1,826 tests run, SUCCESS (2 skipped).
  • The repro from the issue now returns b'MIME-Version: 1.=?unknown-8bit?q?=E9?=\n\nbody\n' from as_bytes() instead of raising.

…ader

A MIME-Version header containing undecodable (non-ASCII) bytes parsed without error but could not be serialized again: as_bytes() and as_string() raised UnicodeEncodeError. parse_mime_version() now runs _validate_xtext() on the invalid-text terminals it creates, so an UndecodableBytesDefect is registered and the header is refolded using the unknown-8bit charset, as for other structured headers.

Signed-off-by: Gaurav Kumar Yadav <xgauravyaduvanshii@gmail.com>
@xgauravyaduvanshii
xgauravyaduvanshii requested a review from a team as a code owner November 18, 2026 04:24
@python-cla-bot

Copy link
Copy Markdown

The following commit authors need to sign the Contributor License Agreement:

CLA not signed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

email: as_bytes() / as_string() raise UnicodeEncodeError for a MIME-Version header containing non-ASCII bytes

1 participant