Skip to content

re.Scanner lexicon patterns with capturing groups abort the program with a traceback #142220

@eli-schwartz

Description

@eli-schwartz

Bug report

Bug description:

Seen in e.g. https://bugs.gentoo.org/967023

This was caused by backporting #140944 into 3.13.10

Installing calibre environment module: /var/tmp/portage/app-text/calibre-8.15.0/image/usr/lib/python3.13/site-packages/init_calibre.py
Traceback (most recent call last):
  File "/var/tmp/portage/app-text/calibre-8.15.0/work/calibre-8.15.0/setup.py", line 130, in <module>
    sys.exit(main())
             ~~~~^^
  File "/var/tmp/portage/app-text/calibre-8.15.0/work/calibre-8.15.0/setup.py", line 115, in main
    command.run_all(opts)
    ~~~~~~~~~~~~~~~^^^^^^
  File "/var/tmp/portage/app-text/calibre-8.15.0/work/calibre-8.15.0/setup/__init__.py", line 250, in run_all
    self.run_cmd(self, opts)
    ~~~~~~~~~~~~^^^^^^^^^^^^
  File "/var/tmp/portage/app-text/calibre-8.15.0/work/calibre-8.15.0/setup/__init__.py", line 244, in run_cmd
    cmd.run(opts)
    ~~~~~~~^^^^^^
  File "/var/tmp/portage/app-text/calibre-8.15.0/work/calibre-8.15.0/setup/install.py", line 142, in run
    self.run_postinstall()
    ~~~~~~~~~~~~~~~~~~~~^^
  File "/var/tmp/portage/app-text/calibre-8.15.0/work/calibre-8.15.0/setup/install.py", line 168, in run_postinstall
    from calibre.linux import PostInstall
  File "/var/tmp/portage/app-text/calibre-8.15.0/work/calibre-8.15.0/src/calibre/linux.py", line 16, in <module>
    from calibre.customize.ui import all_input_formats
  File "/var/tmp/portage/app-text/calibre-8.15.0/work/calibre-8.15.0/src/calibre/customize/ui.py", line 30, in <module>
    from calibre.customize.builtins import plugins as builtin_plugins
  File "/var/tmp/portage/app-text/calibre-8.15.0/work/calibre-8.15.0/src/calibre/customize/builtins.py", line 737, in <module>
    from calibre.devices.kobo.driver import KOBO, KOBOTOUCH
  File "/var/tmp/portage/app-text/calibre-8.15.0/work/calibre-8.15.0/src/calibre/devices/kobo/driver.py", line 26, in <module>
    from calibre.devices.kobo.books import Book, ImageWrapper, KTCollectionsBookList
  File "/var/tmp/portage/app-text/calibre-8.15.0/work/calibre-8.15.0/src/calibre/devices/kobo/books.py", line 14, in <module>
    from calibre.ebooks.metadata.book.formatter import SafeFormat
  File "/var/tmp/portage/app-text/calibre-8.15.0/work/calibre-8.15.0/src/calibre/ebooks/metadata/book/formatter.py", line 10, in <module>
    from calibre.utils.formatter import TemplateFormatter
  File "/var/tmp/portage/app-text/calibre-8.15.0/work/calibre-8.15.0/src/calibre/utils/formatter.py", line 1692, in <module>
    class TemplateFormatter(string.Formatter):
    ...<398 lines>...
                self.restore_state(state)
  File "/var/tmp/portage/app-text/calibre-8.15.0/work/calibre-8.15.0/src/calibre/utils/formatter.py", line 1758, in TemplateFormatter
    arg_parser = re.Scanner([
                (r',', lambda x,t: ''),
                (r'.*?((?<!\\),)', lambda x,t: t[:-1]),
                (r'.*?\)', lambda x,t: t[:-1]),
        ])
  File "/usr/lib/python3.13/re/__init__.py", line 404, in __init__
    raise ValueError("Cannot use capturing groups in re.Scanner")
ValueError: Cannot use capturing groups in re.Scanner```

The code in question dates back to 2010: kovidgoyal/calibre@deddac1
So probably doesn't segfault (there is only one capturing group, not multiple). It seems a bit extreme to backport such a breaking change to the tenth version of an existing already released version, and not giving application authors the chance to test with CPython release candidates before declaring compatibility with a new major version of CPython.

If something needs to be backported, it should probably be an urgent deprecation warning instead.

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Metadata

Metadata

Assignees

No one assigned

    Labels

    type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions