Skip to content

Speed up number formatting by caching locale and pattern parsing - #1437

Open
DicksonWu654 wants to merge 4 commits into
python-babel:masterfrom
DicksonWu654:perf/number-formatting
Open

DicksonWu654 wants to merge 4 commits into
python-babel:masterfrom
DicksonWu654:perf/number-formatting

Conversation

@DicksonWu654

Copy link
Copy Markdown

A typical format_decimal(x, locale='de_DE') or format_currency(x, 'EUR', format='¤#,##0.00', locale=...) call parses the same locale identifier and often the same number pattern every time. This PR removes that repeated work in four small commits. Each one can be reviewed (or dropped) on its own:

  1. core: cache parse_locale() results. A 1024-entry lru_cache keyed by the exact (identifier, sep) strings. The result is a tuple of strings. Locale objects are still created on every Locale.parse() call, so nothing mutable is shared. str subclasses and non-string inputs take the uncached path, so their behaviour and error messages don't change. Exceptions are never cached.
  2. core: try the exact locale before defining the Locale.parse fallbacks. Locale.parse() created two closures on every call, but they're only needed when the locale doesn't exist. It now tries cls(*parts) first. The likely-subtag fallback code is unchanged.
  3. numbers: cache parsed number patterns. Uses the same approach as babel.dates.parse_pattern, which already keeps a 1024-entry lru_cache. Here the cache holds the parsed constructor arguments, which are all strings, tuples, ints and None. NumberPattern is mutable, so a new instance is still built on every call. This helps any call that passes a custom format= string. Locale default patterns are already stored as NumberPattern objects and skip the cache.
  4. numbers: skip quote cleanup when the result has no quotes. The re.sub that strips pattern quotes can't match without an apostrophe. Skipping it saves a regex call and a lambda on almost every formatted number.

Both caches are bounded, and their keys are the function's only inputs. Neither result depends on locale data or global state.

Benchmarks

The workload, for each of 12 locales (en_US, de_DE, fr_FR, ja_JP, ar_EG, hi_IN, pt_BR, ru_RU, es_ES, zh_Hans_CN, fi_FI, de_CH), is: format_decimal with the default pattern and with '#,##0.00', format_currency with the default pattern and with '¤#,##0.00', format_percent, format_compact_decimal, and parse_decimal. That's 84 calls per run. Each sample was a separate process pinned to one core on Linux. I ran master and this branch in alternating order and give the median per-pair ratio:

Python locale passed as master this PR speedup (range)
3.12.12 string 1.244 ms 1.005 ms 1.24x (1.21–1.26x)
3.12.12 Locale object 0.901 ms 0.773 ms 1.17x (1.10–1.24x)
3.15.2 string 1.285 ms 1.052 ms 1.23x (1.20–1.25x)
3.15.2 Locale object 0.914 ms 0.790 ms 1.16x (1.15–1.19x)
3.9.25 string 2.182 ms 1.775 ms 1.23x (1.21–1.25x)

Eight pairs on 3.12 and 3.15, six on 3.9. Per commit, on the string-locale workload (Python 3.12, 8 pairs each): parse_locale cache 1.056x (1.023–1.090x), Locale.parse reorder 1.030x (1.009–1.036x), number-pattern cache 1.080x (1.063–1.139x), quote check 1.055x (1.020–1.084x). When callers pass Locale objects, Locale.parse returns early and only the two numbers commits apply.

Checks

  • I compared master and this branch in separate processes and checked that the output was byte-identical. The comparison covered parse_locale and Locale.parse (with and without likely-subtag resolution, both separators) for every sixth CLDR locale plus aliases, POSIX/C, modifiers, encodings and invalid identifiers. It also covered format_decimal, format_currency, format_currency(format_type='name'), format_percent, format_scientific, format_compact_decimal, format_compact_currency and parse_decimal across those locales, 19 numbers (including NaN, ±Infinity, -0.0 and 40-digit Decimals) and 24 patterns (quotes, '', currency signs, per-mille, exponents, negative subpatterns and invalid patterns). Each case ran twice so that cache hits were compared too, and I mutated a returned NumberPattern between parses. That's 452,106 results, 8,776 of them exceptions, all identical on Python 3.9, 3.12 and 3.15.
  • New tests check that cached results aren't shared: a mutated Locale or NumberPattern doesn't affect the next call, and pickling still works. They also cover str subclasses, unhashable and invalid inputs (same exceptions as before), the cache size limits, Locale subclasses whose constructors return other objects or raise, and quote handling after pattern mutation and currency-symbol replacement.
  • The full test suite passes on Python 3.9, 3.12 and 3.15, and pre-commit run --all-files passes.

Overlap: #1380 rewrites the quote cleanup that commit 4 touches. If #1380 is merged first, commit 4 can be dropped or redone on top of it. #1382 and #1370 change parse_pattern internals; they apply cleanly on this branch and work with the cache.

🤖 Generated with Claude Code

DicksonWu654 and others added 4 commits November 6, 2026 20:31
Every formatting call that receives a locale identifier string parses it
again. Cache the parsed tuple in a 1024-entry LRU keyed by the exact
(identifier, sep) strings; the result is an immutable tuple, Locale objects
are still created per call, and str subclasses and invalid inputs take the
uncached path so errors are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Locale.parse() created two helper closures on every call even though they
are only needed when the requested locale does not exist. Try the exact
locale first; the fallback path is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Custom format strings (e.g. format_decimal(x, "#,##0.00")) are re-parsed
on every call. Cache the parsed constructor arguments (all immutable) in a
1024-entry LRU keyed by the pattern string, as babel.dates already does for
date patterns. A new NumberPattern is still returned on every call, since
NumberPattern is mutable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The quote-removal regex cannot match a string without an apostrophe, which
is the common case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@akx akx added the AI Assisted label Nov 9, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants