Skip to content

history clear/trim fails on Windows: PermissionError on history.sqlite (open SQLite handle blocks unlink) #15241

@sjack2

Description

@sjack2

Summary

On Windows, ipython history clear (and ipython history trim whenever the DB exceeds --keep) fails with PermissionError: [WinError 32] when trying to delete history.sqlite. The SQlite connection opened earlier in HistoryTrim.start() is never closed, so the file is still locked when hist_file.unlink() runs.

unix does not hit this because unlinking an open file is permitted there.

Environment

  • IPython 9.12.0
  • Python 3.14.4 (likely affects all 3.x - see Root cause)
  • Windows 11 (10.0.26200)
  • Installed via miniconda

Steps to reproduce

  1. On Windows, populate history.sqlite with at least one row beyond --keep (e.g. open ipython, run a couple of expressions, exit).
  2. Run ipython history clear -f (or ipython history trim --keep 1 against a larger DB).

Expected behavior

history database is cleared/trimmed, leaving a single history.sqlite with the appropriate row count.

Actual behavior

Really delete all ipython history?  y
Trimming history to the most recent 0 entries.
Traceback (most recent call last):
  File "...\Scripts\ipython-script.py", line 9, in <module>
    sys.exit(start_ipython())
  ...
  File "...\IPython\core\historyapp.py", line 110, in start
    hist_file.unlink()
  File "...\Lib\pathlib\__init__.py", line 1042, in unlink
    os.unlink(self)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\...\\.ipython\\profile_default\\history.sqlite'

Side effect: every failed run leaves behind a history.sqlite.newN file (numbered to avoid collisions at line 77-80), so debris accumulates.

Root cause

In IPython/core/historyapp.py, HistoryTrim.start() uses:

with sqlite3.connect(hist_file) as con:
    ...

Python's sqlite3.Connection.__exit__ only commits or rolls back the transaction -- it does not close the connection. From the stdlib docs:

The context manager neither implicitly opens a new transaction nor closes the connection. If you need a closing context manager, consider using contextlib.closing().

So con stays open through hist_file.unlink() at line 110, and Windows refuses to delete a file that has open handles.

There is a second instance at line 81 with new_db = sqlite3.connect(new_hist_file) that would similarly block the subsequent new_hist_file.rename(hist_file) at line 112 once the unlink is fixed.

Suggested fix

Wrap both connections in try/finally so they are closed before any filesystem mutation:

-        with sqlite3.connect(hist_file) as con:
+        con = sqlite3.connect(hist_file)
+        try:

             # Grab the recent history from the current database.
             inputs = list(con.execute('SELECT session, line, source, source_raw FROM '
                                     'history ORDER BY session DESC, line DESC LIMIT ?', (self.keep+1,)))
             ...
                 sessions = list(con.execute('SELECT session, start, end, num_cmds, remark FROM '
                                             'sessions WHERE session >= ?', (first_session,)))
+        finally:
+            con.close()

         # Create the new history database.
         ...
-        with sqlite3.connect(new_hist_file) as new_db:
+        new_db = sqlite3.connect(new_hist_file)
+        try:
             new_db.execute("""CREATE TABLE IF NOT EXISTS sessions ...""")
             ...
                     new_db.executemany('insert into output_history values (?,?,?)', outputs)
+        finally:
+            new_db.close()

Confirmed working locally: ipython history clear -f against a populated DB now exits cleanly, leaves a single history.sqlite, no .new debris.

Related

Same class of bug (open SQLite handle blocking a Windows file op): #14985.

Happy to open a PR with the fix if helpful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions