annotate roundup/actions.py @ 6433:c1d3fbcdbfbd

issue2551142 - Import of retired node ... unique constraint failure. Title: Import of retired node with username after active node fails with unique constraint failure. More fixes needed for mysql and postgresql. mysql: add unique constraint for (keyvalue, __retired__) when creating class in the database. On schema change if class is changed, remove the unique constraint too. upgrade version of rdbms database from 5 to 6 to add constraint to all version 5 databases that were created as version 5 and didn't get the unique constraint. Make no changes on version 5 databases upgraded from version 4, the upgrade process to 5 added the constraint. Make no changes to other databases (sqlite, postgres) during upgrade from version 5 to 6. postgres: Handle the exception raised on unique constraint violation. The exception invalidates the database connection so it can't be used to recover from the exception. Added two new database methods: checkpoint_data - performs a db.commit under postgres does nothing on other backends restore_connection_on_error - does a db.rollback on postgres, does nothing on other backends with the rollback() done on the connection I can use the database connection to fixup the import that failed on the unique constraint. This makes postgres slower but without the commit after every imported object, the rollback will delete all the entries done up to this point. Trying to figure out how to make the caller do_import batch and recover from this failure is beyond me. Also dismissed having to process the export csv file before importing. Pushing that onto a user just seems wrong. Also since import/export isn't frequently done the lack of surprise on having a failing import and reduced load/frustration for the user seems worth it. Also the import can be run in verbose mode where it prints out a row as it is processed, so it may take a while, ut the user can get feedback. db_test-base.py: add test for upgrade from 5 to 6.
author John Rouillard <rouilj@ieee.org>
date Thu, 10 Jun 2021 12:52:05 -0400
parents 48a1f919f894
children 4c9acc580769
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4083
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
1 #
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
2 # Copyright (C) 2009 Stefan Seefeld
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
3 # All rights reserved.
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
4 # For license terms see the file COPYING.txt.
5604
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
5 # Actions used in REST and XMLRPC APIs
4083
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
6 #
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
7
5071
a7541077cf12 Remove 'import *' statement from actions.py
John Kristensen <john@jerrykan.com>
parents: 4357
diff changeset
8 from roundup.exceptions import Unauthorised
4083
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
9 from roundup import hyperdb
6003
48a1f919f894 Flake8 cleanups.
John Rouillard <rouilj@ieee.org>
parents: 5604
diff changeset
10
4083
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
11
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
12 class Action:
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
13 def __init__(self, db, translator):
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
14 self.db = db
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
15 self.translator = translator
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
16
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
17 def handle(self, *args):
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
18 """Action handler procedure"""
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
19 raise NotImplementedError
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
20
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
21 def execute(self, *args):
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
22 """Execute the action specified by this object."""
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
23
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
24 self.permission(*args)
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
25 return self.handle(*args)
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
26
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
27 def permission(self, *args):
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
28 """Check whether the user has permission to execute this action.
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
29
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
30 If not, raise Unauthorised."""
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
31
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
32 pass
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
33
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
34 def gettext(self, msgid):
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
35 """Return the localized translation of msgid"""
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
36 return self.translator.gettext(msgid)
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
37
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
38 _ = gettext
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
39
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
40
5604
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
41 class PermCheck(Action):
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
42 def permission(self, designator):
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
43
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
44 classname, itemid = hyperdb.splitDesignator(designator)
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
45 perm = self.db.security.hasPermission
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
46
6003
48a1f919f894 Flake8 cleanups.
John Rouillard <rouilj@ieee.org>
parents: 5604
diff changeset
47 if not perm('Retire', self.db.getuid(), classname=classname,
48a1f919f894 Flake8 cleanups.
John Rouillard <rouilj@ieee.org>
parents: 5604
diff changeset
48 itemid=itemid):
5604
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
49 raise Unauthorised(self._('You do not have permission to retire '
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
50 'or restore the %(classname)s class.')
6003
48a1f919f894 Flake8 cleanups.
John Rouillard <rouilj@ieee.org>
parents: 5604
diff changeset
51 % locals())
48a1f919f894 Flake8 cleanups.
John Rouillard <rouilj@ieee.org>
parents: 5604
diff changeset
52
5604
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
53
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
54 class Retire(PermCheck):
4083
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
55
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
56 def handle(self, designator):
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
57
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
58 classname, itemid = hyperdb.splitDesignator(designator)
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
59
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
60 # make sure we don't try to retire admin or anonymous
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
61 if (classname == 'user' and
6003
48a1f919f894 Flake8 cleanups.
John Rouillard <rouilj@ieee.org>
parents: 5604
diff changeset
62 self.db.user.get(itemid, 'username') in ('admin', 'anonymous')):
4357
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4125
diff changeset
63 raise ValueError(self._(
13b3155869e0 Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents: 4125
diff changeset
64 'You may not retire the admin or anonymous user'))
4083
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
65
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
66 # do the retire
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
67 self.db.getclass(classname).retire(itemid)
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
68 self.db.commit()
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
69
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
70
5604
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
71 class Restore(PermCheck):
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
72
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
73 def handle(self, designator):
4083
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
74
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
75 classname, itemid = hyperdb.splitDesignator(designator)
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents:
diff changeset
76
5604
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
77 # do the restore
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
78 self.db.getclass(classname).restore(itemid)
ed02a1e0aa5d Fix actions
Ralf Schlatterbeck <rsc@runtux.com>
parents: 5071
diff changeset
79 self.db.commit()

Roundup Issue Tracker: http://roundup-tracker.org/