Skip to content

gh-131918: Add _ThreadLocalSqliteConnection in dbm.sqlite#131920

Draft
meghprkh wants to merge 5 commits into
python:mainfrom
meghprkh:dbm-sqlite3-multithreaded
Draft

gh-131918: Add _ThreadLocalSqliteConnection in dbm.sqlite#131920
meghprkh wants to merge 5 commits into
python:mainfrom
meghprkh:dbm-sqlite3-multithreaded

Conversation

@meghprkh

@meghprkh meghprkh commented Mar 30, 2025

Copy link
Copy Markdown

I am a first-time contributor and I am not sure about the quality of the code at all, review comments will be appreciated :)

Will try to add a test later. Should this be back-ported to Python 3.13?

@ghost

ghost commented Mar 30, 2025

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-app

bedevere-app Bot commented Mar 30, 2025

Copy link
Copy Markdown

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@erlend-aasland

Copy link
Copy Markdown
Contributor

Should this be back-ported to Python 3.13?

IMO, yes.

@@ -0,0 +1 @@
Fixes dbm.sqlite3 for multi-threaded use-cases by using thread-local connections.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Fixes dbm.sqlite3 for multi-threaded use-cases by using thread-local connections.
Make :mod:`dbm.sqlite3` thread-safe by using thread-local database connections.

@meghprkh

Copy link
Copy Markdown
Author

Hmm this fix doesnt really work with close, but is closing sqlite connections opened in WAL mode necesssary?

As in currently the close method closes connections from other threads, which would give the same error.

@brianschubert

brianschubert commented Mar 30, 2025

Copy link
Copy Markdown
Contributor

Not too familiar with sqlite3 thread-safety, but does this need to consider the case where sqlite3.threadsafety is 0, which (I think) would make forming connections from multiple threads unsafe?

There should probably be a docs update to explain exactly what thread-safety guarantees this is making (and if they depend on sqlite3.threadsafety?)

@meghprkh

Copy link
Copy Markdown
Author

Very weirdly, on my system (Fedora Linux), the sqlite3.threadsafety is set to 3 by default, but I still seem to be getting the error

@erlend-aasland
erlend-aasland marked this pull request as draft March 31, 2025 06:01
Comment thread Lib/dbm/sqlite3.py
wrthread = ref(thread, thread_deleted)
try:
conn = sqlite3.connect(self._uri, autocommit=True, uri=True)
self._conn[id(thread)] = conn

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
self._conn[id(thread)] = conn
self._conn[thread.native_id] = conn

Hmm why not using native_id?

@python-cla-bot

python-cla-bot Bot commented Apr 6, 2025

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

@github-actions

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale Stale PR or inactive for long period of time.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants