Mercurial > p > roundup > code

annotate scripts/dump_dbm_sessions_db.py @ 6610:db3f0ba75b4a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Change checkpoint_data and restore_connection_on_error to subtransaction checkpoint_data and restore_connection_on_error used to commit() and rollback() the db connection. This causes additional I/O and load. Changed them to use 'SAVEPOINT name' and 'ROLLBACK TO name' to get a faster method for handling errors within a tranaction. One thing to note is that postgresql (unlike SQL std) doesn't overwrite an older savepoint with he same name. It keeps all savepoints but only rolls back to the newest one with a given name. This could be a resource issue. I left a commented out release statement in case somebody runs into an issue due to too many savepoints. I expect it to slow down the import but....
author John Rouillard <rouilj@ieee.org>
date Sat, 29 Jan 2022 11:29:36 -0500
parents 61481d7bbb07
children 1188bb423f92
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6571
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
1 #! /usr/bin/env python3
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
2 """Usage: dump_dbm_sessions_db.py [filename]
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
3
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
4 Simple script to dump the otks and sessions dbm databases. Dumps
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
5 sessions db in current directory if no argument is given.
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
6
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
7 Dump format:
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
8
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
9 key: <timestamp> data
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
10
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
11 where <timestamp> is the human readable __timestamp decoded from the
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
12 data object.
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
13
6577
61481d7bbb07 fix syntax error
John Rouillard <rouilj@ieee.org>
parents: 6571
diff changeset
14 """
6571
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
15
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
16 import dbm, marshal, sys
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
17 from datetime import datetime
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
18
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
19 try:
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
20 file = sys.argv[1]
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
21 except IndexError:
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
22 file="sessions"
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
23
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
24 try:
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
25 db = dbm.open(file)
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
26 except Exception:
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
27 print("Unable to open database: %s"%file)
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
28 exit(1)
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
29
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
30 k = db.firstkey()
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
31 while k is not None:
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
32 d = marshal.loads(db[k])
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
33 t = datetime.fromtimestamp(d['__timestamp'])
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
34 print("%s: %s %s"%(k, t, d))
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
35 k = db.nextkey(k)
Roundup Issue Tracker: http://roundup-tracker.org/