Skip to content

bpo-41152: Revise setting idlelib.iomenu.encoding #21206

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
terryjreedy wants to merge 1 commit into from
Closed

bpo-41152: Revise setting idlelib.iomenu.encoding #21206

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 29 additions & 33 deletions Lib/idlelib/iomenu.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,52 +13,48 @@
import idlelib
from idlelib.config import idleConf

if idlelib.testing: # Set True by test.test_idle to avoid setlocale.
if idlelib.testing or sys.platform == 'win32':
# When testing (set by test_idle), avoid setlocale changing environment.
encoding = 'utf-8'
errors = 'surrogateescape'
else:
# Try setting the locale, so that we can find out
# what encoding to use
# Try setting the locale to find out what encoding to use.
try:
import locale
locale.setlocale(locale.LC_CTYPE, "")
except (ImportError, locale.Error):
pass

if sys.platform == 'win32':
encoding = 'utf-8'
errors = 'surrogateescape'
else:
try:
# Different things can fail here: the locale module may not be
# loaded, it may not offer nl_langinfo, or CODESET, or the
# resulting codeset may be unknown to Python. We ignore all
# these problems, falling back to ASCII.
locale_encoding = locale.nl_langinfo(locale.CODESET)
if locale_encoding:
codecs.lookup(locale_encoding)
except (NameError, AttributeError, LookupError):
# Try getdefaultlocale: it parses environment variables,
# which may give a clue. Unfortunately, getdefaultlocale has
# bugs that can cause ValueError.
try:
# Different things can fail here: the locale module may not be
# loaded, it may not offer nl_langinfo, or CODESET, or the
# resulting codeset may be unknown to Python. We ignore all
# these problems, falling back to ASCII
locale_encoding = locale.nl_langinfo(locale.CODESET)
locale_encoding = locale.getdefaultlocale()[1]
if locale_encoding:
codecs.lookup(locale_encoding)
except (NameError, AttributeError, LookupError):
# Try getdefaultlocale: it parses environment variables,
# which may give a clue. Unfortunately, getdefaultlocale has
# bugs that can cause ValueError.
try:
locale_encoding = locale.getdefaultlocale()[1]
if locale_encoding:
codecs.lookup(locale_encoding)
except (ValueError, LookupError):
pass
except (ValueError, LookupError):
pass

if locale_encoding:
encoding = locale_encoding.lower()
errors = 'strict'
else:
# POSIX locale or macOS
encoding = 'ascii'
errors = 'surrogateescape'
# Encoding is used in multiple files; locale_encoding nowhere.
# The only use of 'encoding' below is in _decode as initial value
# of deprecated block asking user for encoding.
# Perhaps use elsewhere should be reviewed.
if locale_encoding:
encoding = locale_encoding.lower()
errors = 'strict'
else:
# POSIX locale or macOS
encoding = 'ascii'
errors = 'surrogateescape'
# Encoding is used in multiple files; locale_encoding nowhere.
# The only use of 'encoding' below is in _decode as initial value
# of deprecated block asking user for encoding.
# Perhaps use elsewhere should be reviewed.

coding_re = re.compile(r'^[ \t\f]*#.*?coding[:=][ \t]*([-\w.]+)', re.ASCII)
blank_re = re.compile(r'^[ \t\f]*(?:[#\r\n]|$)', re.ASCII)
Expand Down