Mercurial > p > roundup > code

annotate roundup/backends/back_sqlite.py @ 7965:6763813d9d34

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
issue2551350 - Python changes for 3.12 with roundup 2.3.0 cgitb.py Fix change in pydoc.html.header() signature. It dropped foreground and background color arguments in 3.11 and newer. Also enable test code for the html function.
author John Rouillard <rouilj@ieee.org>
date Tue, 14 May 2024 21:27:28 -0400
parents e3975f679bf1
children 95f91b6f0386
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
1 """Implements a backend for SQLite.
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
2
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3 See https://pysqlite.sourceforge.net/ for pysqlite info
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
4
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
5
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
6 NOTE: we use the rdbms_common table creation methods which define datatypes
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
7 for the columns, but sqlite IGNORES these specifications.
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
8 """
2005
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
9 __docformat__ = 'restructuredtext'
fc52d57c6c3e documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents: 1911
diff changeset
10
6932
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
11 import logging
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
12 import os
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
13 import shutil
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
14 import time
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
15
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
16
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
17 from roundup import hyperdb, date, password
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
18 from roundup.backends import rdbms_common
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
19 from roundup.backends import sessions_sqlite
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
20 from roundup.backends import sessions_dbm
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
21
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
22 try:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
23 from roundup.backends import sessions_redis
6820
feb175271f3e redis import failure on python2 causes crash
John Rouillard <rouilj@ieee.org>
parents: 6814
diff changeset
24 except ImportError:
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
25 sessions_redis = None
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
26
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5319
diff changeset
27 from roundup.anypy.strings import uany2s
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
28
3728
b476fef16ccc fixed support for pysqlite2 (version 2.1.0 is the minimum version supported)
Richard Jones <richard@users.sourceforge.net>
parents: 3718
diff changeset
29 sqlite_version = None
3654
f5201bbbc505 update for latest version of pysqlite ([SF#1487098]; patch 1534227)
Richard Jones <richard@users.sourceforge.net>
parents: 3609
diff changeset
30 try:
4231
23ba17d01a24 re-order sqlite imports to handle multiple installed versions (issue 2550570)
Richard Jones <richard@users.sourceforge.net>
parents: 4091
diff changeset
31 import sqlite3 as sqlite
23ba17d01a24 re-order sqlite imports to handle multiple installed versions (issue 2550570)
Richard Jones <richard@users.sourceforge.net>
parents: 4091
diff changeset
32 sqlite_version = 3
3654
f5201bbbc505 update for latest version of pysqlite ([SF#1487098]; patch 1534227)
Richard Jones <richard@users.sourceforge.net>
parents: 3609
diff changeset
33 except ImportError:
3718
0d561b24ceff support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents: 3693
diff changeset
34 try:
0d561b24ceff support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents: 3693
diff changeset
35 from pysqlite2 import dbapi2 as sqlite
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
36 if sqlite.version_info < (2, 1, 0):
3794
f5ddd1639a15 *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 3752
diff changeset
37 raise ValueError('pysqlite2 minimum version is 2.1.0+ '
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
38 '- %s found' % sqlite.version)
3728
b476fef16ccc fixed support for pysqlite2 (version 2.1.0 is the minimum version supported)
Richard Jones <richard@users.sourceforge.net>
parents: 3718
diff changeset
39 sqlite_version = 2
3718
0d561b24ceff support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents: 3693
diff changeset
40 except ImportError:
7921
e3975f679bf1 issue2551302 - Remove support for sqlite version 1 from back_sqlite.py
John Rouillard <rouilj@ieee.org>
parents: 7699
diff changeset
41 raise ValueError("Unable to import sqlite3 or sqlite 2.")
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
42
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
43
2633
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2514
diff changeset
44 def db_exists(config):
2736
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
45 return os.path.exists(os.path.join(config.DATABASE, 'db'))
2633
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2514
diff changeset
46
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
47
2633
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2514
diff changeset
48 def db_nuke(config):
2736
402d6d556558 postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents: 2721
diff changeset
49 shutil.rmtree(config.DATABASE)
2633
a9e1fff1e793 I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents: 2514
diff changeset
50
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
51
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
52 class Database(rdbms_common.Database):
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
53 """Sqlite DB backend implementation
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
54
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
55 attributes:
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
56 dbtype:
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
57 holds the value for the type of db. It is used by indexer to
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
58 identify the database type so it can import the correct indexer
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
59 module when using native text search mode.
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
60 """
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
61
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
62 # char to use for positional arguments
7921
e3975f679bf1 issue2551302 - Remove support for sqlite version 1 from back_sqlite.py
John Rouillard <rouilj@ieee.org>
parents: 7699
diff changeset
63 arg = '?'
3048
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3008
diff changeset
64
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
65 dbtype = "sqlite"
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5067
diff changeset
66
3048
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3008
diff changeset
67 # used by some code to switch styles of query
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3008
diff changeset
68 implements_intersect = 1
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3008
diff changeset
69
5175
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5143
diff changeset
70 # used in generic backend to determine if db supports
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5143
diff changeset
71 # 'DOUBLE PRECISION' for floating point numbers. Note that sqlite
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5143
diff changeset
72 # already has double precision as its standard 'REAL' type. So this
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5143
diff changeset
73 # is set to False here.
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5143
diff changeset
74
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5143
diff changeset
75 implements_double_precision = False
e1e40674a0bc Implement double-precision Number
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5143
diff changeset
76
2168
7e595abb781e better sqlite columns
Richard Jones <richard@users.sourceforge.net>
parents: 2144
diff changeset
77 hyperdb_to_sql_datatypes = {
6932
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
78 hyperdb.String : 'VARCHAR(255)', # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
79 hyperdb.Date : 'VARCHAR(30)', # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
80 hyperdb.Link : 'INTEGER', # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
81 hyperdb.Interval : 'VARCHAR(255)', # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
82 hyperdb.Password : 'VARCHAR(255)', # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
83 hyperdb.Boolean : 'BOOLEAN', # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
84 hyperdb.Number : 'REAL', # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
85 hyperdb.Integer : 'INTEGER', # noqa: E203
2168
7e595abb781e better sqlite columns
Richard Jones <richard@users.sourceforge.net>
parents: 2144
diff changeset
86 }
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
87 hyperdb_to_sql_value = {
6932
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
88 hyperdb.String : str, # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
89 hyperdb.Date : lambda x: x.serialise(), # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
90 hyperdb.Link : int, # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
91 hyperdb.Interval : str, # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
92 hyperdb.Password : str, # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
93 hyperdb.Boolean : int, # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
94 hyperdb.Integer : int, # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
95 hyperdb.Number : lambda x: x, # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
96 hyperdb.Multilink : lambda x: x, # used in journal marshalling, # noqa: E203
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
97 }
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
98 sql_to_hyperdb_value = {
6932
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
99 hyperdb.String : uany2s, # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
100 hyperdb.Date : lambda x: date.Date(str(x)), # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
101 hyperdb.Link : str, # XXX numeric ids # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
102 hyperdb.Interval : date.Interval, # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
103 hyperdb.Password : lambda x: password.Password(encrypted=x), # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
104 hyperdb.Boolean : int, # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
105 hyperdb.Integer : int, # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
106 hyperdb.Number : rdbms_common._num_cvt, # noqa: E203
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
107 hyperdb.Multilink : lambda x: x, # used in journal marshalling, # noqa: E203
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
108 }
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
109
6933
e138f5cf432a fix comment on use of sqlite for session db's.
John Rouillard <rouilj@ieee.org>
parents: 6932
diff changeset
110 # We can use DBM, redis or SQLite for managing session info and
e138f5cf432a fix comment on use of sqlite for session db's.
John Rouillard <rouilj@ieee.org>
parents: 6932
diff changeset
111 # one-time keys:
e138f5cf432a fix comment on use of sqlite for session db's.
John Rouillard <rouilj@ieee.org>
parents: 6932
diff changeset
112 # For SQL database storage of this info we have to create separate
e138f5cf432a fix comment on use of sqlite for session db's.
John Rouillard <rouilj@ieee.org>
parents: 6932
diff changeset
113 # databases for Otk and Session because SQLite doesn't support
e138f5cf432a fix comment on use of sqlite for session db's.
John Rouillard <rouilj@ieee.org>
parents: 6932
diff changeset
114 # concurrent connections to the same database.
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
115 def getSessionManager(self):
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
116 if not self.Session:
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
117 if self.config.SESSIONDB_BACKEND == "redis":
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
118 if sessions_redis is None:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
119 self.Session = sessions_sqlite.Sessions(self)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
120 raise ValueError("[redis] session is set, but "
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
121 "redis module is not found")
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
122 self.Session = sessions_redis.Sessions(self)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
123 elif self.config.SESSIONDB_BACKEND == "anydbm":
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
124 self.Session = sessions_dbm.Sessions(self)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
125 else:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
126 self.Session = sessions_sqlite.Sessions(self)
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
127 return self.Session
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
128
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
129 def getOTKManager(self):
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
130 if not self.Otk:
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
131 if self.config.SESSIONDB_BACKEND == "redis":
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
132 if sessions_redis is None:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
133 self.Session = sessions_sqlite.OneTimeKeys(self)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
134 raise ValueError("[redis] session is set, but "
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
135 "redis is not found")
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
136 self.Otk = sessions_redis.OneTimeKeys(self)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
137 elif self.config.SESSIONDB_BACKEND == "anydbm":
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
138 self.Otk = sessions_dbm.OneTimeKeys(self)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
139 else:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
140 self.Otk = sessions_sqlite.OneTimeKeys(self)
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
141 return self.Otk
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
142
6804
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
143 def sql_open_connection(self, dbname=None):
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
144 """Open a standard, non-autocommitting connection.
2374
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2273
diff changeset
145
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2273
diff changeset
146 pysqlite will automatically BEGIN TRANSACTION for us.
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
147 """
2764
d3b3f1b3d59e sqlite database creation requires existing directory.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2736
diff changeset
148 # make sure the database directory exists
d3b3f1b3d59e sqlite database creation requires existing directory.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2736
diff changeset
149 # database itself will be created by sqlite if needed
d3b3f1b3d59e sqlite database creation requires existing directory.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2736
diff changeset
150 if not os.path.isdir(self.config.DATABASE):
d3b3f1b3d59e sqlite database creation requires existing directory.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2736
diff changeset
151 os.makedirs(self.config.DATABASE)
d3b3f1b3d59e sqlite database creation requires existing directory.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2736
diff changeset
152
6804
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
153 if dbname:
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
154 db = os.path.join(self.config.DATABASE, 'db-' + dbname)
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
155 else:
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
156 db = os.path.join(self.config.DATABASE, 'db')
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
157 logging.getLogger('roundup.hyperdb').info('open database %r' % db)
7921
e3975f679bf1 issue2551302 - Remove support for sqlite version 1 from back_sqlite.py
John Rouillard <rouilj@ieee.org>
parents: 7699
diff changeset
158 conn = sqlite.connect(db, timeout=self.config.RDBMS_SQLITE_TIMEOUT)
e3975f679bf1 issue2551302 - Remove support for sqlite version 1 from back_sqlite.py
John Rouillard <rouilj@ieee.org>
parents: 7699
diff changeset
159 conn.row_factory = sqlite.Row
4091
09e79cbeb827 force sqlite3 in py2.6+ to treat our stored text as UTF-8
Richard Jones <richard@users.sourceforge.net>
parents: 4076
diff changeset
160
4240
289f249ba192 compatibilty fix for pysqlite2 / unicode
Richard Jones <richard@users.sourceforge.net>
parents: 4231
diff changeset
161 # pysqlite2 / sqlite3 want us to store Unicode in the db but
289f249ba192 compatibilty fix for pysqlite2 / unicode
Richard Jones <richard@users.sourceforge.net>
parents: 4231
diff changeset
162 # that's not what's been done historically and it's definitely
289f249ba192 compatibilty fix for pysqlite2 / unicode
Richard Jones <richard@users.sourceforge.net>
parents: 4231
diff changeset
163 # not what the other backends do, so we'll stick with UTF-8
289f249ba192 compatibilty fix for pysqlite2 / unicode
Richard Jones <richard@users.sourceforge.net>
parents: 4231
diff changeset
164 if sqlite_version in (2, 3):
4091
09e79cbeb827 force sqlite3 in py2.6+ to treat our stored text as UTF-8
Richard Jones <richard@users.sourceforge.net>
parents: 4076
diff changeset
165 conn.text_factory = str
09e79cbeb827 force sqlite3 in py2.6+ to treat our stored text as UTF-8
Richard Jones <richard@users.sourceforge.net>
parents: 4076
diff changeset
166
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
167 cursor = conn.cursor()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
168 return (conn, cursor)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
169
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
170 def open_connection(self):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
171 # ensure files are group readable and writable
3609
f2fda3e6fc8b umask is now configurable (with the same 0002 default)
Richard Jones <richard@users.sourceforge.net>
parents: 3347
diff changeset
172 os.umask(self.config.UMASK)
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
173
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
174 (self.conn, self.cursor) = self.sql_open_connection()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents: 2077
diff changeset
175
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
176 try:
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
177 self.load_dbschema()
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5175
diff changeset
178 except sqlite.DatabaseError as error:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
179 if str(error) != 'no such table: schema':
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
180 raise
2075
b1704ba7be41 make mysql / postgresql work again. beginnings of otk/session store in rdbmses
Richard Jones <richard@users.sourceforge.net>
parents: 2073
diff changeset
181 self.init_dbschema()
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
182 self.sql('create table schema (schema varchar)')
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
183 self.sql('create table ids (name varchar, num integer)')
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
184 self.sql('create index ids_name_idx on ids(name)')
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
185 self.create_version_2_tables()
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6050
diff changeset
186 self._add_fts5_table()
6917
fba76e0bba98 set journal mode to wal when creating db.
John Rouillard <rouilj@ieee.org>
parents: 6916
diff changeset
187 # Set journal mode to WAL.
6930
a96a239db0d9 Set all sqlite db's to WAL mode on creation
John Rouillard <rouilj@ieee.org>
parents: 6917
diff changeset
188 self.sql_commit() # close out rollback journal/transaction
a96a239db0d9 Set all sqlite db's to WAL mode on creation
John Rouillard <rouilj@ieee.org>
parents: 6917
diff changeset
189 self.sql('pragma journal_mode=wal') # set wal
a96a239db0d9 Set all sqlite db's to WAL mode on creation
John Rouillard <rouilj@ieee.org>
parents: 6917
diff changeset
190 self.sql_commit() # close out rollback and commit wal change
2073
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
191
261c2e6ceb1e *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 2005
diff changeset
192 def create_version_2_tables(self):
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
193 self.sql('create table otks (otk_key varchar, '
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
194 'otk_value varchar, otk_time integer)')
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
195 self.sql('create index otks_key_idx on otks(otk_key)')
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
196 self.sql('create table sessions (session_key varchar, '
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
197 'session_time integer, session_value varchar)')
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
198 self.sql('create index sessions_key_idx on '
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
199 'sessions(session_key)')
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
200
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
201 # full-text indexing store
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
202 self.sql('CREATE TABLE __textids (_class varchar, '
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
203 '_itemid varchar, _prop varchar, _textid'
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
204 ' integer primary key) ')
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
205 self.sql('CREATE TABLE __words (_word varchar, '
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
206 '_textid integer)')
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
207 self.sql('CREATE INDEX words_word_ids ON __words(_word)')
3858
bb30bbfc7cdd Indexing fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3794
diff changeset
208 self.sql('CREATE INDEX words_by_id ON __words (_textid)')
bb30bbfc7cdd Indexing fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3794
diff changeset
209 self.sql('CREATE UNIQUE INDEX __textids_by_props ON '
bb30bbfc7cdd Indexing fixes.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3794
diff changeset
210 '__textids (_class, _itemid, _prop)')
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
211 sql = 'insert into ids (name, num) values (%s,%s)' % (
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
212 self.arg, self.arg)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
213 self.sql(sql, ('__textids', 1))
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
214
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
215 def add_new_columns_v2(self):
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
216 # update existing tables to have the new actor column
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
217 tables = self.database_schema['tables']
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
218 for classname, spec in self.classes.items():
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4240
diff changeset
219 if classname in tables:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
220 dbspec = tables[classname]
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
221 self.update_class(spec, dbspec, force=1, adding_v2=1)
2137
c49495585c44 Fix for sqlite backend migration. Change Cookie -> SimpleCookie
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
222 # we've updated - don't try again
c49495585c44 Fix for sqlite backend migration. Change Cookie -> SimpleCookie
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
223 tables[classname] = spec.schema()
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
224
2721
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2645
diff changeset
225 def fix_version_3_tables(self):
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2645
diff changeset
226 # NOOP - no restriction on column length here
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2645
diff changeset
227 pass
1cd01cf106e1 extend OTK and session table value cols to TEXT [SF#1031271]
Richard Jones <richard@users.sourceforge.net>
parents: 2645
diff changeset
228
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6050
diff changeset
229 def _add_fts5_table(self):
7699
e10d0b262abd fix: diagnose/report use of SQLite without FTS5 support
John Rouillard <rouilj@ieee.org>
parents: 6933
diff changeset
230 try:
e10d0b262abd fix: diagnose/report use of SQLite without FTS5 support
John Rouillard <rouilj@ieee.org>
parents: 6933
diff changeset
231 self.sql('CREATE virtual TABLE __fts USING fts5(_class, '
6932
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
232 '_itemid, _prop, _textblob)')
7699
e10d0b262abd fix: diagnose/report use of SQLite without FTS5 support
John Rouillard <rouilj@ieee.org>
parents: 6933
diff changeset
233 except sqlite.OperationalError:
e10d0b262abd fix: diagnose/report use of SQLite without FTS5 support
John Rouillard <rouilj@ieee.org>
parents: 6933
diff changeset
234 available_options = self.cursor.execute(
e10d0b262abd fix: diagnose/report use of SQLite without FTS5 support
John Rouillard <rouilj@ieee.org>
parents: 6933
diff changeset
235 'pragma compile_options;').fetchall()
e10d0b262abd fix: diagnose/report use of SQLite without FTS5 support
John Rouillard <rouilj@ieee.org>
parents: 6933
diff changeset
236 if 'ENABLE_FTS5' in [opt['compile_options'] for opt
e10d0b262abd fix: diagnose/report use of SQLite without FTS5 support
John Rouillard <rouilj@ieee.org>
parents: 6933
diff changeset
237 in available_options]:
e10d0b262abd fix: diagnose/report use of SQLite without FTS5 support
John Rouillard <rouilj@ieee.org>
parents: 6933
diff changeset
238 # sqlite supports FTS5 something else has gone wrong
e10d0b262abd fix: diagnose/report use of SQLite without FTS5 support
John Rouillard <rouilj@ieee.org>
parents: 6933
diff changeset
239 raise
e10d0b262abd fix: diagnose/report use of SQLite without FTS5 support
John Rouillard <rouilj@ieee.org>
parents: 6933
diff changeset
240 else:
e10d0b262abd fix: diagnose/report use of SQLite without FTS5 support
John Rouillard <rouilj@ieee.org>
parents: 6933
diff changeset
241 # report a useful error message
e10d0b262abd fix: diagnose/report use of SQLite without FTS5 support
John Rouillard <rouilj@ieee.org>
parents: 6933
diff changeset
242 raise NotImplementedError(
e10d0b262abd fix: diagnose/report use of SQLite without FTS5 support
John Rouillard <rouilj@ieee.org>
parents: 6933
diff changeset
243 "This version of SQLite was not built with support "
e10d0b262abd fix: diagnose/report use of SQLite without FTS5 support
John Rouillard <rouilj@ieee.org>
parents: 6933
diff changeset
244 "for FTS5. SQLite version: %s" % sqlite.sqlite_version)
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6050
diff changeset
245
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6050
diff changeset
246 def fix_version_6_tables(self):
6599
39189dd94f2c issue2551189 - increase size of words in full text index.
John Rouillard <rouilj@ieee.org>
parents: 6588
diff changeset
247 # note sqlite has no limit on column size so v6 fixes
39189dd94f2c issue2551189 - increase size of words in full text index.
John Rouillard <rouilj@ieee.org>
parents: 6588
diff changeset
248 # to __words._word length are not needed.
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6050
diff changeset
249 # Add native full-text indexing table
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6050
diff changeset
250 self._add_fts5_table()
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6050
diff changeset
251
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
252 def update_class(self, spec, old_spec, force=0, adding_v2=0):
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
253 """ Determine the differences between the current spec and the
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
254 database version of the spec, and update where necessary.
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
255
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
256 If 'force' is true, update the database anyway.
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
257
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
258 SQLite doesn't have ALTER TABLE, so we have to copy and
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
259 regenerate the tables with the new schema.
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
260 """
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
261 new_spec = spec.schema()
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
262 new_spec[1].sort()
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
263 old_spec[1].sort()
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
264 if not force and new_spec == old_spec:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
265 # no changes
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
266 return 0
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
267
4421
67bef70ab9b9 - more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4420
diff changeset
268 logging.getLogger('roundup.hyperdb').info(
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
269 'update_class %s' % spec.classname)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
270
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
271 # detect multilinks that have been removed, and drop their table
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
272 old_has = {}
2137
c49495585c44 Fix for sqlite backend migration. Change Cookie -> SimpleCookie
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
273 for name, prop in old_spec[1]:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
274 old_has[name] = 1
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
275 if name in spec.properties or not \
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
276 isinstance(prop, hyperdb.Multilink):
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
277 continue
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
278 # it's a multilink, and it's been removed - drop the old
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
279 # table. First drop indexes.
3347
9020fe58c51c more errors. backport candidate
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents: 3310
diff changeset
280 self.drop_multilink_table_indexes(spec.classname, name)
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
281 sql = 'drop table %s_%s' % (spec.classname, prop)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
282 self.sql(sql)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
283
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
284 # now figure how we populate the new table
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
285 if adding_v2:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
286 fetch = ['_activity', '_creation', '_creator']
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
287 else:
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
288 fetch = ['_actor', '_activity', '_creation', '_creator']
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
289 properties = spec.getprops()
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
290 for propname, _x in new_spec[1]:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
291 prop = properties[propname]
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
292 if isinstance(prop, hyperdb.Multilink):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4240
diff changeset
293 if propname not in old_has:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
294 # we need to create the new table
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
295 self.create_multilink_table(spec, propname)
2137
c49495585c44 Fix for sqlite backend migration. Change Cookie -> SimpleCookie
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
296 elif force:
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
297 tn = '%s_%s' % (spec.classname, propname)
2137
c49495585c44 Fix for sqlite backend migration. Change Cookie -> SimpleCookie
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
298 # grabe the current values
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
299 sql = 'select linkid, nodeid from %s' % tn
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
300 self.sql(sql)
2137
c49495585c44 Fix for sqlite backend migration. Change Cookie -> SimpleCookie
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
301 rows = self.cursor.fetchall()
c49495585c44 Fix for sqlite backend migration. Change Cookie -> SimpleCookie
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
302
c49495585c44 Fix for sqlite backend migration. Change Cookie -> SimpleCookie
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
303 # drop the old table
c49495585c44 Fix for sqlite backend migration. Change Cookie -> SimpleCookie
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
304 self.drop_multilink_table_indexes(spec.classname, propname)
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
305 sql = 'drop table %s' % tn
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
306 self.sql(sql)
2137
c49495585c44 Fix for sqlite backend migration. Change Cookie -> SimpleCookie
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
307
c49495585c44 Fix for sqlite backend migration. Change Cookie -> SimpleCookie
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
308 # re-create and populate the new table
c49495585c44 Fix for sqlite backend migration. Change Cookie -> SimpleCookie
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
309 self.create_multilink_table(spec, propname)
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
310 sql = """insert into %s (linkid, nodeid) values
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
311 (%s, %s)""" % (tn, self.arg, self.arg)
2137
c49495585c44 Fix for sqlite backend migration. Change Cookie -> SimpleCookie
Richard Jones <richard@users.sourceforge.net>
parents: 2098
diff changeset
312 for linkid, nodeid in rows:
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
313 self.sql(sql, (int(linkid), int(nodeid)))
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4240
diff changeset
314 elif propname in old_has:
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
315 # we copy this col over from the old table
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
316 fetch.append('_'+propname)
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
317
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
318 # select the data out of the old table
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
319 fetch.append('id')
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
320 fetch.append('__retired__')
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
321 fetchcols = ','.join(fetch)
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
322 cn = spec.classname
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
323 sql = 'select %s from _%s' % (fetchcols, cn)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
324 self.sql(sql)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
325 olddata = self.cursor.fetchall()
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
326
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
327 # TODO: update all the other index dropping code
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
328 self.drop_class_table_indexes(cn, old_spec[0])
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
329
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
330 # drop the old table
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
331 self.sql('drop table _%s' % cn)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
332
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
333 # create the new table
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
334 self.create_class_table(spec)
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
335
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
336 if olddata:
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
337 inscols = ['id', '_actor', '_activity', '_creation',
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
338 '_creator', '__retired__']
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
339 for propname, _x in new_spec[1]:
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
340 prop = properties[propname]
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
341 if isinstance(prop, hyperdb.Multilink):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
342 continue
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
343 elif isinstance(prop, hyperdb.Interval):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
344 inscols.append('_'+propname)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
345 inscols.append('__'+propname+'_int__')
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4240
diff changeset
346 elif propname in old_has:
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
347 # we copy this col over from the old table
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
348 inscols.append('_'+propname)
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
349
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
350 # do the insert of the old data - the new columns will have
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
351 # NULL values
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
352 args = ','.join([self.arg for x in inscols])
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
353 cols = ','.join(inscols)
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
354 sql = 'insert into _%s (%s) values (%s)' % (cn, cols, args)
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
355 for entry in olddata:
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
356 d = []
4774
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4421
diff changeset
357 retired_id = None
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
358 for name in inscols:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
359 # generate the new value for the Interval int column
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
360 if name.endswith('_int__'):
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
361 name = name[2:-6]
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
362 if sqlite_version in (2, 3):
3718
0d561b24ceff support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents: 3693
diff changeset
363 try:
0d561b24ceff support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents: 3693
diff changeset
364 v = hyperdb.Interval(entry[name]).as_seconds()
0d561b24ceff support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents: 3693
diff changeset
365 except IndexError:
0d561b24ceff support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents: 3693
diff changeset
366 v = None
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4240
diff changeset
367 elif name in entry:
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
368 v = hyperdb.Interval(entry[name]).as_seconds()
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
369 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
370 v = None
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
371 elif sqlite_version in (2, 3):
3718
0d561b24ceff support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents: 3693
diff changeset
372 try:
0d561b24ceff support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents: 3693
diff changeset
373 v = entry[name]
0d561b24ceff support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents: 3693
diff changeset
374 except IndexError:
0d561b24ceff support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents: 3693
diff changeset
375 v = None
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
376 else:
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
377 v = None
4774
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4421
diff changeset
378 if name == 'id':
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4421
diff changeset
379 retired_id = v
6932
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
380 elif (name == '__retired__' and retired_id and
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
381 v not in ['0', 0]):
4774
3adff0fb0207 Fixed issue2550595: Allow migrating from roundup 0.x to 1.4
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4421
diff changeset
382 v = retired_id
2217
98d3bf8ffb19 store Intervals as two columns (and other fixes
Richard Jones <richard@users.sourceforge.net>
parents: 2191
diff changeset
383 d.append(v)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
384 self.sql(sql, tuple(d))
2077
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
385
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
386 return 1
3e0961d6d44d Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents: 2075
diff changeset
387
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
388 def sql_close(self):
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
389 """ Squash any error caused by us already having closed the
1236
dd52bf10f934 Bug fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
390 connection.
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
391 """
1236
dd52bf10f934 Bug fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
392 try:
2374
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2273
diff changeset
393 self.conn.close()
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5175
diff changeset
394 except sqlite.ProgrammingError as value:
2374
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2273
diff changeset
395 if str(value) != 'close failed - Connection is closed.':
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2273
diff changeset
396 raise
1236
dd52bf10f934 Bug fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
397
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
398 def sql_rollback(self):
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
399 """ Squash any error caused by us having closed the connection (and
1236
dd52bf10f934 Bug fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
400 therefore not having anything to roll back)
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
401 """
1236
dd52bf10f934 Bug fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
402 try:
dd52bf10f934 Bug fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
403 self.conn.rollback()
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5175
diff changeset
404 except sqlite.ProgrammingError as value:
1236
dd52bf10f934 Bug fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
405 if str(value) != 'rollback failed - Connection is closed.':
dd52bf10f934 Bug fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
406 raise
dd52bf10f934 Bug fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 1195
diff changeset
407
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
408 def __repr__(self):
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
409 return '<roundlite 0x%x>' % id(self)
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
410
5319
62de601bdf6f Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5318
diff changeset
411 def sql_commit(self):
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
412 """ Actually commit to the database.
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
413
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
414 Ignore errors if there's nothing to commit.
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
415 """
6916
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
416 def list_dir(dir):
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
417 import os
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
418 files = os.listdir(self.dir)
6932
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
419 # ['db-journal', 'files', 'db']
6916
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
420 for entry in [''] + files:
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
421 path = self.dir + '/' + entry
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
422 stat = os.stat(path)
6932
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
423 print("file: %s, uid: %s, gid: %s, mode: %o" % (
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
424 path, stat.st_uid, stat.st_gid, stat.st_mode))
6916
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
425
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
426 # Getting sqlite3.OperationalError: disk I/O error
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
427 # in CI. It happens intermittently. Try to get more
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
428 # info about what is happening and retry the commit.
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
429 # Some possibilities:
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
430 # -journal file not writable
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
431 # file has disappeared
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
432 #
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
433 # Note after exception self.conn.in_transaction is False
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
434 # but was True before failed commit(). Retry succeeds,
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
435 # but I am not sure it actually does anything.
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
436 # for retry in range(2):
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
437 try:
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
438 self.conn.commit()
6916
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
439 except sqlite.OperationalError as error:
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
440 if str(error) != 'disk I/O error':
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
441 raise
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
442 list_dir(self.dir)
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
443 raise
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5175
diff changeset
444 except sqlite.DatabaseError as error:
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
445 if str(error) != 'cannot commit - no transaction is active':
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
446 raise
6916
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
447 # else:
fad6b3a272cd Add instrumentation to debug I/O error in CI that I don't get locally
John Rouillard <rouilj@ieee.org>
parents: 6820
diff changeset
448 # break # out of loop if no exception
2374
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2273
diff changeset
449 # open a new cursor for subsequent work
31cb1014300c Switch to using sqlite's own locking mechanisms...
Richard Jones <richard@users.sourceforge.net>
parents: 2273
diff changeset
450 self.cursor = self.conn.cursor()
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
451
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1836
diff changeset
452 def sql_index_exists(self, table_name, index_name):
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
453 self.sql('pragma index_list(%s)' % table_name)
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1836
diff changeset
454 for entry in self.cursor.fetchall():
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1836
diff changeset
455 if entry[1] == index_name:
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1836
diff changeset
456 return 1
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1836
diff changeset
457 return 0
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1836
diff changeset
458
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
459 # old-skool id generation
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
460 def newid(self, classname):
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
461 """ Generate a new id for the given class
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
462 """
5143
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
463
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
464 # Prevent other processes from reading while we increment.
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
465 # Otherwise multiple processes can end up with the same
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
466 # new id and hilarity results.
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
467 #
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
468 # Defeat pysqlite's attempts to do locking by setting
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
469 # isolation_level to None. Pysqlite can commit
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
470 # on it's own even if we don't want it to end the transaction.
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
471 # If we rewrite to use another sqlite library like apsw we
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
472 # don't have to deal with this autocommit/autotransact foolishness.
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
473 self.conn.isolation_level = None
5143
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
474 # Manage the transaction locks manually.
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
475 self.sql("BEGIN IMMEDIATE")
5143
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
476
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
477 # get the next ID
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
478 sql = 'select num from ids where name=%s' % self.arg
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
479 self.sql(sql, (classname, ))
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
480 newid = int(self.cursor.fetchone()[0])
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
481
5143
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
482 # leave the next larger number as the next newid
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
483 sql = 'update ids set num=num+1 where name=%s' % self.arg
5143
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
484 vals = (classname,)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
485 self.sql(sql, vals)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
486
5143
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
487 # reset pysqlite's auto transact stuff to default since the
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
488 # rest of the code expects it.
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
489 self.conn.isolation_level = ''
5143
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
490 # commit writing the data, clearing locks for other processes
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
491 # and create a new cursor to the database.
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
492 self.sql_commit()
5143
9cebf686c552 issue2550727: db.newid is broken with sqlite.
John Rouillard <rouilj@ieee.org>
parents: 5096
diff changeset
493
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
494 # return as string
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
495 return str(newid)
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
496
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
497 def setid(self, classname, setid):
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
498 """ Set the id counter: used during import of database
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
499
3306
ae1093684ef5 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 3163
diff changeset
500 We add one to make it behave like the sequences in postgres.
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
501 """
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
502 sql = 'update ids set num=%s where name=%s' % (self.arg, self.arg)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
503 vals = (int(setid)+1, classname)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
504 self.sql(sql, vals)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
505
3306
ae1093684ef5 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 3163
diff changeset
506 def clear(self):
3310
3518d1ffd940 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 3306
diff changeset
507 rdbms_common.Database.clear(self)
3306
ae1093684ef5 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 3163
diff changeset
508 # set the id counters to 0 (setid adds one) so we start at 1
ae1093684ef5 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 3163
diff changeset
509 for cn in self.classes.keys():
ae1093684ef5 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 3163
diff changeset
510 self.setid(cn, 0)
ae1093684ef5 merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents: 3163
diff changeset
511
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
512 def create_class(self, spec):
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
513 rdbms_common.Database.create_class(self, spec)
6932
ff7f6c0bbdae flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6930
diff changeset
514 sql = 'insert into ids (name, num) values (%s, %s)' % (
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
515 self.arg, self.arg)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
516 vals = (spec.classname, 1)
2514
091711fb2f8c Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents: 2374
diff changeset
517 self.sql(sql, vals)
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2094
diff changeset
518
7921
e3975f679bf1 issue2551302 - Remove support for sqlite version 1 from back_sqlite.py
John Rouillard <rouilj@ieee.org>
parents: 7699
diff changeset
519 def load_journal(self, classname, cols, nodeid):
e3975f679bf1 issue2551302 - Remove support for sqlite version 1 from back_sqlite.py
John Rouillard <rouilj@ieee.org>
parents: 7699
diff changeset
520 """We need to turn the sqlite3.Row into a tuple so it can be
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
521 unpacked"""
7921
e3975f679bf1 issue2551302 - Remove support for sqlite version 1 from back_sqlite.py
John Rouillard <rouilj@ieee.org>
parents: 7699
diff changeset
522 l = rdbms_common.Database.load_journal(self,
e3975f679bf1 issue2551302 - Remove support for sqlite version 1 from back_sqlite.py
John Rouillard <rouilj@ieee.org>
parents: 7699
diff changeset
523 classname, cols, nodeid)
e3975f679bf1 issue2551302 - Remove support for sqlite version 1 from back_sqlite.py
John Rouillard <rouilj@ieee.org>
parents: 7699
diff changeset
524 cols = range(5)
e3975f679bf1 issue2551302 - Remove support for sqlite version 1 from back_sqlite.py
John Rouillard <rouilj@ieee.org>
parents: 7699
diff changeset
525 return [[row[col] for col in cols] for row in l]
3718
0d561b24ceff support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents: 3693
diff changeset
526
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
527
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
528 class sqliteClass:
5869
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
529 def filter(self, *args, **kw):
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
530 """ If there's NO matches to a fetch, sqlite returns NULL
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
531 instead of nothing
4076
59d02edfaa9c Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents: 3858
diff changeset
532 """
5869
16e1255b16cf Implement limit and offset for filter
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5867
diff changeset
533 return [f for f in rdbms_common.Class.filter(self, *args, **kw) if f]
1165
14467c765167 sqlite backend!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
534
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
535
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
536 class Class(sqliteClass, rdbms_common.Class):
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
537 pass
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
538
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
539
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
540 class IssueClass(sqliteClass, rdbms_common.IssueClass):
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
541 pass
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
542
6050
19a58adf5d1f Flake8: whitepace; don't import base64; unused loop var gets leading _
John Rouillard <rouilj@ieee.org>
parents: 5869
diff changeset
543
1911
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
544 class FileClass(sqliteClass, rdbms_common.FileClass):
f5c804379c85 fixed ZRoundup - mostly changes to classic template
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
545 pass
1168
94620e088e3a fixes to the rdbms backends
Richard Jones <richard@users.sourceforge.net>
parents: 1165
diff changeset
546
2645
8250c63c3963 remove space at EOL, add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2633
diff changeset
547 # vim: set et sts=4 sw=4 :
Roundup Issue Tracker: http://roundup-tracker.org/