Mercurial > p > roundup > code

annotate scripts/dump_dbm_sessions_db.py @ 8556:dd0445649244

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
bug(perf): put legacy '_file' last in tuple to speed up match
author John Rouillard <rouilj@ieee.org>
date Mon, 06 Apr 2026 01:30:55 -0400
parents 9ba04f37896f
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6571
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
1 #! /usr/bin/env python3
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
2 """Usage: dump_dbm_sessions_db.py [filename]
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
3
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
4 Simple script to dump the otks and sessions dbm databases. Dumps
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
5 sessions db in current directory if no argument is given.
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
6
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
7 Dump format:
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
8
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
9 key: <timestamp> data
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
10
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
11 where <timestamp> is the human readable __timestamp decoded from the
6799
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
12 data object. Data object is dumped in json format. With pretty print
6571
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
13
6799
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
14 key:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
15 <timestamp>
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
16 {
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
17 key: val,
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
18 ...
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
19 }
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
20
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
21 if data is not a python object, print will be key: data or
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
22 key:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
23 data
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
24
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
25 if pretty printed.
6577
61481d7bbb07 fix syntax error
John Rouillard <rouilj@ieee.org>
parents: 6571
diff changeset
26 """
6571
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
27
7780
9ba04f37896f chore: lint cleanups
John Rouillard <rouilj@ieee.org>
parents: 6799
diff changeset
28 import argparse
9ba04f37896f chore: lint cleanups
John Rouillard <rouilj@ieee.org>
parents: 6799
diff changeset
29 import dbm
9ba04f37896f chore: lint cleanups
John Rouillard <rouilj@ieee.org>
parents: 6799
diff changeset
30 import json
9ba04f37896f chore: lint cleanups
John Rouillard <rouilj@ieee.org>
parents: 6799
diff changeset
31 import marshal
9ba04f37896f chore: lint cleanups
John Rouillard <rouilj@ieee.org>
parents: 6799
diff changeset
32 import os
9ba04f37896f chore: lint cleanups
John Rouillard <rouilj@ieee.org>
parents: 6799
diff changeset
33 import sys
6571
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
34 from datetime import datetime
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
35
7780
9ba04f37896f chore: lint cleanups
John Rouillard <rouilj@ieee.org>
parents: 6799
diff changeset
36
6799
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
37 def indent(text, amount, ch=" "):
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
38 """ Found at: https://stackoverflow.com/a/8348914
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
39 """
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
40 padding = amount * ch
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
41 return ''.join(padding+line for line in text.splitlines(True))
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
42
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
43 def print_marshal(k):
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
44 d = marshal.loads(db[k])
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
45 try:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
46 t = datetime.fromtimestamp(d['__timestamp'])
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
47 except (KeyError, TypeError):
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
48 # TypeError raised if marshalled data is not a dict (list, tuple etc)
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
49 t = "no_timestamp"
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
50 if args.pretty:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
51 print("%s:\n %s\n%s"%(k, t, indent(json.dumps(
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
52 d, sort_keys=True, indent=4), 4)))
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
53 else:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
54 print("%s: %s %s"%(k, t, d))
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
55
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
56 def print_raw(k):
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
57 if args.pretty:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
58 print("%s:\n %s"%(k, db[k]))
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
59 else:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
60 print("%s: %s"%(k, db[k]))
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
61
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
62 parser = argparse.ArgumentParser(
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
63 description='Dump DBM files used by Roundup in storage order.')
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
64 parser.add_argument('-k', '--key', action="append",
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
65 help='dump the entry for a key, can be used multiple times.')
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
66 parser.add_argument('-K', '--keysonly', action='store_true',
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
67 help='print the database keys, sorted in byte order.')
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
68 parser.add_argument('-p', '--pretty', action='store_true',
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
69 help='pretty print the output rather than printing on one line.')
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
70 parser.add_argument('file', nargs='?',
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
71 help='file to be dumped ("sessions" if not provided)')
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
72 args = parser.parse_args()
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
73
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
74 if args.file:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
75 file = args.file
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
76 else:
6571
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
77 file="sessions"
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
78
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
79 try:
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
80 db = dbm.open(file)
6799
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
81 except Exception as e:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
82 print("Unable to open database for %s: %s"%(file, e))
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
83 try:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
84 os.stat(file)
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
85 print(" perhaps file is invalid or was created with a different version of Python?")
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
86 except OSError:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
87 # the file does exist on disk.
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
88 pass
7780
9ba04f37896f chore: lint cleanups
John Rouillard <rouilj@ieee.org>
parents: 6799
diff changeset
89 sys.exit(1)
6799
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
90
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
91 if args.keysonly:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
92 for k in sorted(db.keys()):
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
93 print("%s"%k)
7780
9ba04f37896f chore: lint cleanups
John Rouillard <rouilj@ieee.org>
parents: 6799
diff changeset
94 sys.exit(0)
6799
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
95
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
96 if args.key:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
97 for k in args.key:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
98 try:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
99 print_marshal(k)
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
100 except (ValueError):
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
101 print_raw(k)
7780
9ba04f37896f chore: lint cleanups
John Rouillard <rouilj@ieee.org>
parents: 6799
diff changeset
102 sys.exit(0)
6571
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
103
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
104 k = db.firstkey()
cd408eb748dd Add small utility script for dumping dbm based databases.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
105 while k is not None:
6799
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
106 try:
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
107 print_marshal(k)
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
108 except (ValueError): # ValueError marshal.loads failed
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
109 print_raw(k)
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
110
1188bb423f92 Script dumps dbm files without marshalled data like _ids
John Rouillard <rouilj@ieee.org>
parents: 6577
diff changeset
111 k = db.nextkey(k)
Roundup Issue Tracker: http://roundup-tracker.org/