Mercurial > p > roundup > code
comparison test/db_test_base.py @ 3634:57c66056ffe4
Implemented what I'll call for now "transitive searching"...
...using the filter method. The first idea was mentioned on the
roundup-users mailing list:
http://article.gmane.org/gmane.comp.bug-tracking.roundup.user/6909
We can now search for items which link transitively to other classes
using filter. An example is searching for all items where a certain user
has added a message in the last week:
db.issue.filter (None, {'messages.author' : '42', 'messages.date' : '.-1w;'})
or more readable (but not exactly semantically equivalent, if you're
searching for multiple users in this way it will fail, because string
searches are ANDed): {'messages.author.username':'ralf', ...
We can even extend this further, look for all items that were changed by
users belonging to a certain department (having the same supervisor -- a
property that is not in the user class in standard roundup) in the last
week, the filterspec would be:
{'messages.author.supervisor' : '42', 'messages.date' : '.-1w;'}
If anybody wants to suggest another name instead of transitive
searching, you're welcome.
I've implemented a generic method for this in hyperdb.py -- the backend
now implements _filter in this case. With the generic method, anydbm and
metakit should work (anydbm is tested, metakit breaks for other
reasons). A backend may chose to implement the real transitive filter
itself. This was done for rdbms_common.py. It now has an implementation
of filter that supports transitive searching by creating one big join in
the generated SQL query.
I've added several new regression tests to test for the new features.
All the tests (not just the new ones) run through on python2.3 and
python2.4 with postgres, mysql, sqlite, anydbm -- but metakit was
already broken when I started.
I've generated a tag before commit called 'rsc_before_transitive_search'
and will create the 'after' tag after this commit, so you can merge out
my changes if you don't like them -- if you like them I can remove the
tags.
.-- Ralf
| author | Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net> |
|---|---|
| date | Sat, 08 Jul 2006 18:28:18 +0000 |
| parents | 7b25567f0f54 |
| children | 53987aa153d2 |
comparison
equal
deleted
inserted
replaced
| 3633:a292054b4393 | 3634:57c66056ffe4 |
|---|---|
| 13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS | 13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS |
| 14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" | 14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" |
| 15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, | 15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, |
| 16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. | 16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. |
| 17 # | 17 # |
| 18 # $Id: db_test_base.py,v 1.69 2006-04-27 01:39:47 richard Exp $ | 18 # $Id: db_test_base.py,v 1.70 2006-07-08 18:28:18 schlatterbeck Exp $ |
| 19 | 19 |
| 20 import unittest, os, shutil, errno, imp, sys, time, pprint, sets | 20 import unittest, os, shutil, errno, imp, sys, time, pprint, sets |
| 21 | 21 |
| 22 from roundup.hyperdb import String, Password, Link, Multilink, Date, \ | 22 from roundup.hyperdb import String, Password, Link, Multilink, Date, \ |
| 23 Interval, DatabaseError, Boolean, Number, Node | 23 Interval, DatabaseError, Boolean, Number, Node |
| 67 status = module.Class(db, "status", name=String()) | 67 status = module.Class(db, "status", name=String()) |
| 68 status.setkey("name") | 68 status.setkey("name") |
| 69 priority = module.Class(db, "priority", name=String(), order=String()) | 69 priority = module.Class(db, "priority", name=String(), order=String()) |
| 70 priority.setkey("name") | 70 priority.setkey("name") |
| 71 user = module.Class(db, "user", username=String(), password=Password(), | 71 user = module.Class(db, "user", username=String(), password=Password(), |
| 72 assignable=Boolean(), age=Number(), roles=String()) | 72 assignable=Boolean(), age=Number(), roles=String(), |
| 73 supervisor=Link('user')) | |
| 73 user.setkey("username") | 74 user.setkey("username") |
| 74 file = module.FileClass(db, "file", name=String(), type=String(), | 75 file = module.FileClass(db, "file", name=String(), type=String(), |
| 75 comment=String(indexme="yes"), fooz=Password()) | 76 comment=String(indexme="yes"), fooz=Password()) |
| 76 file_nidx = module.FileClass(db, "file_nidx", content=String(indexme='no')) | 77 file_nidx = module.FileClass(db, "file_nidx", content=String(indexme='no')) |
| 77 issue = module.IssueClass(db, "issue", title=String(indexme="yes"), | 78 issue = module.IssueClass(db, "issue", title=String(indexme="yes"), |
| 78 status=Link("status"), nosy=Multilink("user"), deadline=Date(), | 79 status=Link("status"), nosy=Multilink("user"), deadline=Date(), |
| 79 foo=Interval(), files=Multilink("file"), assignedto=Link('user'), | 80 foo=Interval(), files=Multilink("file"), assignedto=Link('user'), |
| 80 priority=Link('priority')) | 81 priority=Link('priority')) |
| 81 stuff = module.Class(db, "stuff", stuff=String()) | 82 stuff = module.Class(db, "stuff", stuff=String()) |
| 82 session = module.Class(db, 'session', title=String()) | 83 session = module.Class(db, 'session', title=String()) |
| 83 msg = module.FileClass(db, "msg", | 84 msg = module.FileClass(db, "msg", date=Date(), |
| 84 author=Link("user", do_journal='no')) | 85 author=Link("user", do_journal='no')) |
| 85 session.disableJournalling() | 86 session.disableJournalling() |
| 86 db.post_init() | 87 db.post_init() |
| 87 if create: | 88 if create: |
| 88 user.create(username="admin", roles='Admin', | 89 user.create(username="admin", roles='Admin', |
| 947 | 948 |
| 948 def testFilteringID(self): | 949 def testFilteringID(self): |
| 949 ae, filt = self.filteringSetup() | 950 ae, filt = self.filteringSetup() |
| 950 ae(filt(None, {'id': '1'}, ('+','id'), (None,None)), ['1']) | 951 ae(filt(None, {'id': '1'}, ('+','id'), (None,None)), ['1']) |
| 951 ae(filt(None, {'id': '2'}, ('+','id'), (None,None)), ['2']) | 952 ae(filt(None, {'id': '2'}, ('+','id'), (None,None)), ['2']) |
| 952 ae(filt(None, {'id': '10'}, ('+','id'), (None,None)), []) | 953 ae(filt(None, {'id': '100'}, ('+','id'), (None,None)), []) |
| 953 | 954 |
| 954 def testFilteringNumber(self): | 955 def testFilteringNumber(self): |
| 955 self.filteringSetup() | 956 self.filteringSetup() |
| 956 ae, filt = self.assertEqual, self.db.user.filter | 957 ae, filt = self.assertEqual, self.db.user.filter |
| 957 ae(filt(None, {'age': '1'}, ('+','id'), (None,None)), ['3']) | 958 ae(filt(None, {'age': '1'}, ('+','id'), (None,None)), ['3']) |
| 1090 ae(filt(None, {}, ('+','deadline'), ('-','priority')), | 1091 ae(filt(None, {}, ('+','deadline'), ('-','priority')), |
| 1091 ['3', '4', '2', '1']) | 1092 ['3', '4', '2', '1']) |
| 1092 ae(filt(None, {}, ('-','deadline'), ('-','priority')), | 1093 ae(filt(None, {}, ('-','deadline'), ('-','priority')), |
| 1093 ['4', '3', '1', '2']) | 1094 ['4', '3', '1', '2']) |
| 1094 | 1095 |
| 1096 def filteringSetupTransitiveSearch(self): | |
| 1097 u_m = {} | |
| 1098 k = 1 | |
| 1099 for user in ( | |
| 1100 {'username': 'ceo', 'age': 129}, | |
| 1101 {'username': 'grouplead1', 'age': 29, 'supervisor': '3'}, | |
| 1102 {'username': 'grouplead2', 'age': 29, 'supervisor': '3'}, | |
| 1103 {'username': 'worker1', 'age': 25, 'supervisor' : '4'}, | |
| 1104 {'username': 'worker2', 'age': 26, 'supervisor' : '4'}, | |
| 1105 {'username': 'worker3', 'age': 27, 'supervisor' : '5'}, | |
| 1106 {'username': 'worker4', 'age': 28, 'supervisor' : '5'}, | |
| 1107 {'username': 'worker5', 'age': 29, 'supervisor' : '5'}): | |
| 1108 u = self.db.user.create(**user) | |
| 1109 u_m [u] = self.db.msg.create(author = u, content = ' ' | |
| 1110 , date = date.Date ('2006-01-%s' % k)) | |
| 1111 k += 1 | |
| 1112 iss = self.db.issue | |
| 1113 for issue in ( | |
| 1114 {'title': 'ts1', 'status': '2', 'assignedto': '6', | |
| 1115 'priority': '3', 'messages' : [u_m ['6']], 'nosy' : ['4']}, | |
| 1116 {'title': 'ts2', 'status': '1', 'assignedto': '6', | |
| 1117 'priority': '3', 'messages' : [u_m ['6']], 'nosy' : ['5']}, | |
| 1118 {'title': 'ts4', 'status': '2', 'assignedto': '7', | |
| 1119 'priority': '3', 'messages' : [u_m ['7']]}, | |
| 1120 {'title': 'ts5', 'status': '1', 'assignedto': '8', | |
| 1121 'priority': '3', 'messages' : [u_m ['8']]}, | |
| 1122 {'title': 'ts6', 'status': '2', 'assignedto': '9', | |
| 1123 'priority': '3', 'messages' : [u_m ['9']]}, | |
| 1124 {'title': 'ts7', 'status': '1', 'assignedto': '10', | |
| 1125 'priority': '3', 'messages' : [u_m ['10']]}, | |
| 1126 {'title': 'ts8', 'status': '2', 'assignedto': '10', | |
| 1127 'priority': '3', 'messages' : [u_m ['10']]}, | |
| 1128 {'title': 'ts9', 'status': '1', 'assignedto': '10', | |
| 1129 'priority': '3', 'messages' : [u_m ['10'], u_m ['9']]}): | |
| 1130 self.db.issue.create(**issue) | |
| 1131 return self.assertEqual, self.db.issue.filter | |
| 1132 | |
| 1133 def testFilteringTransitiveLinkUser(self): | |
| 1134 ae, filt = self.filteringSetupTransitiveSearch() | |
| 1135 ufilt = self.db.user.filter | |
| 1136 ae(ufilt(None, {'supervisor.username': 'ceo'}, ('+','username')), | |
| 1137 ['4', '5']) | |
| 1138 ae(ufilt(None, {'supervisor.supervisor.username': 'ceo'}, | |
| 1139 ('+','username')), ['6', '7', '8', '9', '10']) | |
| 1140 ae(ufilt(None, {'supervisor.supervisor': '3'}, ('+','username')), | |
| 1141 ['6', '7', '8', '9', '10']) | |
| 1142 ae(ufilt(None, {'supervisor.supervisor.id': '3'}, ('+','username')), | |
| 1143 ['6', '7', '8', '9', '10']) | |
| 1144 ae(ufilt(None, {'supervisor.username': 'grouplead1'}, ('+','username')), | |
| 1145 ['6', '7']) | |
| 1146 ae(ufilt(None, {'supervisor.username': 'grouplead2'}, ('+','username')), | |
| 1147 ['8', '9', '10']) | |
| 1148 ae(ufilt(None, {'supervisor.username': 'grouplead2', | |
| 1149 'supervisor.supervisor.username': 'ceo'}, ('+','username')), | |
| 1150 ['8', '9', '10']) | |
| 1151 | |
| 1152 def testFilteringTransitiveLinkIssue(self): | |
| 1153 ae, filt = self.filteringSetupTransitiveSearch() | |
| 1154 ae(filt(None, {'assignedto.supervisor.username': 'grouplead1'}, | |
| 1155 ('+','id')), ['1', '2', '3']) | |
| 1156 ae(filt(None, {'assignedto.supervisor.username': 'grouplead2'}, | |
| 1157 ('+','id')), ['4', '5', '6', '7', '8']) | |
| 1158 ae(filt(None, {'assignedto.supervisor.username': 'grouplead2', | |
| 1159 'status': '1'}, ('+','id')), ['4', '6', '8']) | |
| 1160 ae(filt(None, {'assignedto.supervisor.username': 'grouplead2', | |
| 1161 'status': '2'}, ('+','id')), ['5', '7']) | |
| 1162 ae(filt(None, {'assignedto.supervisor.username': ['grouplead2'], | |
| 1163 'status': '2'}, ('+','id')), ['5', '7']) | |
| 1164 ae(filt(None, {'assignedto.supervisor': ['4', '5'], 'status': '2'}, | |
| 1165 ('+','id')), ['1', '3', '5', '7']) | |
| 1166 | |
| 1167 def testFilteringTransitiveMultilink(self): | |
| 1168 ae, filt = self.filteringSetupTransitiveSearch() | |
| 1169 ae(filt(None, {'messages.author.username': 'grouplead1'}, | |
| 1170 ('+','id')), []) | |
| 1171 ae(filt(None, {'messages.author': '6'}, | |
| 1172 ('+','id')), ['1', '2']) | |
| 1173 ae(filt(None, {'messages.author.id': '6'}, | |
| 1174 ('+','id')), ['1', '2']) | |
| 1175 ae(filt(None, {'messages.author.username': 'worker1'}, | |
| 1176 ('+','id')), ['1', '2']) | |
| 1177 ae(filt(None, {'messages.author': '10'}, | |
| 1178 ('+','id')), ['6', '7', '8']) | |
| 1179 ae(filt(None, {'messages.author': '9'}, | |
| 1180 ('+','id')), ['5', '8']) | |
| 1181 ae(filt(None, {'messages.author': ['9', '10']}, | |
| 1182 ('+','id')), ['5', '6', '7', '8']) | |
| 1183 ae(filt(None, {'messages.author': ['8', '9']}, | |
| 1184 ('+','id')), ['4', '5', '8']) | |
| 1185 ae(filt(None, {'messages.author': ['8', '9'], 'status' : '1'}, | |
| 1186 ('+','id')), ['4', '8']) | |
| 1187 ae(filt(None, {'messages.author': ['8', '9'], 'status' : '2'}, | |
| 1188 ('+','id')), ['5']) | |
| 1189 ae(filt(None, {'messages.author': ['8', '9', '10'], | |
| 1190 'messages.date': '2006-01-07.21:00;2006-01-10'}, ('+','id')), | |
| 1191 ['6', '7', '8']) | |
| 1192 ae(filt(None, {'nosy.supervisor.username': 'ceo'}, | |
| 1193 ('+','id')), ['1', '2']) | |
| 1194 | |
| 1095 # XXX add sorting tests for other types | 1195 # XXX add sorting tests for other types |
| 1096 # XXX test auditors and reactors | 1196 # XXX test auditors and reactors |
| 1097 | 1197 |
| 1098 def testImportExport(self): | 1198 def testImportExport(self): |
| 1099 # use the filtering setup to create a bunch of items | 1199 # use the filtering setup to create a bunch of items |
