Mercurial > p > roundup > code
annotate roundup/cgi/TAL/TALDefs.py @ 6433:c1d3fbcdbfbd
issue2551142 - Import of retired node ... unique constraint failure.
Title: Import of retired node with username after active node fails
with unique constraint failure.
More fixes needed for mysql and postgresql.
mysql: add unique constraint for (keyvalue, __retired__) when
creating class in the database.
On schema change if class is changed, remove the unique
constraint too.
upgrade version of rdbms database from 5 to 6 to add constraint
to all version 5 databases that were created as version 5
and didn't get the unique constraint. Make no changes
on version 5 databases upgraded from version 4, the upgrade
process to 5 added the constraint. Make no changes
to other databases (sqlite, postgres) during upgrade from
version 5 to 6.
postgres: Handle the exception raised on unique constraint violation.
The exception invalidates the database connection so it
can't be used to recover from the exception.
Added two new database methods:
checkpoint_data - performs a db.commit under postgres
does nothing on other backends
restore_connection_on_error - does a db.rollback on
postgres, does nothing on other
backends
with the rollback() done on the connection I can use the
database connection to fixup the import that failed on the
unique constraint. This makes postgres slower but without the
commit after every imported object, the rollback will delete
all the entries done up to this point.
Trying to figure out how to make the caller do_import batch
and recover from this failure is beyond me.
Also dismissed having to process the export csv file before
importing. Pushing that onto a user just seems wrong. Also
since import/export isn't frequently done the lack of
surprise on having a failing import and reduced
load/frustration for the user seems worth it. Also the import
can be run in verbose mode where it prints out a row as it is
processed, so it may take a while, ut the user can get
feedback.
db_test-base.py: add test for upgrade from 5 to 6.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Thu, 10 Jun 2021 12:52:05 -0400 |
| parents | 7172c201dec2 |
| children |
| rev | line source |
|---|---|
| 1049 | 1 ############################################################################## |
| 2 # | |
| 3 # Copyright (c) 2001, 2002 Zope Corporation and Contributors. | |
| 4 # All Rights Reserved. | |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
5 # |
| 1049 | 6 # This software is subject to the provisions of the Zope Public License, |
| 7 # Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution. | |
| 8 # THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED | |
| 9 # WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED | |
| 10 # WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS | |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
11 # FOR A PARTICULAR PURPOSE. |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
12 # |
| 1049 | 13 ############################################################################## |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
14 # Modifications for Roundup: |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
15 # 1. commented out ITALES references |
| 1049 | 16 """ |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
17 Common definitions used by TAL and METAL compilation an transformation. |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
18 """ |
| 1049 | 19 |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
20 #from ITALES import ITALESErrorInfo |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
21 |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
22 TAL_VERSION = "1.4" |
| 1049 | 23 |
| 24 XML_NS = "http://www.w3.org/XML/1998/namespace" # URI for XML namespace | |
| 25 XMLNS_NS = "http://www.w3.org/2000/xmlns/" # URI for XML NS declarations | |
| 26 | |
| 27 ZOPE_TAL_NS = "http://xml.zope.org/namespaces/tal" | |
| 28 ZOPE_METAL_NS = "http://xml.zope.org/namespaces/metal" | |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
29 ZOPE_I18N_NS = "http://xml.zope.org/namespaces/i18n" |
| 1049 | 30 |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
31 # This RE must exactly match the expression of the same name in the |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
32 # zope.i18n.simpletranslationservice module: |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
33 NAME_RE = "[a-zA-Z_][-a-zA-Z0-9_]*" |
| 1049 | 34 |
| 35 KNOWN_METAL_ATTRIBUTES = [ | |
| 36 "define-macro", | |
| 37 "use-macro", | |
| 38 "define-slot", | |
| 39 "fill-slot", | |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
40 "slot", |
| 1049 | 41 ] |
| 42 | |
| 43 KNOWN_TAL_ATTRIBUTES = [ | |
| 44 "define", | |
| 45 "condition", | |
| 46 "content", | |
| 47 "replace", | |
| 48 "repeat", | |
| 49 "attributes", | |
| 50 "on-error", | |
| 51 "omit-tag", | |
| 52 "tal tag", | |
| 53 ] | |
| 54 | |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
55 KNOWN_I18N_ATTRIBUTES = [ |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
56 "translate", |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
57 "domain", |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
58 "target", |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
59 "source", |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
60 "attributes", |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
61 "data", |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
62 "name", |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
63 ] |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
64 |
|
5265
63868084b8bb
Python 2 and 3 support. Convert Exception to BaseException. TAL and
John Rouillard <rouilj@ieee.org>
parents:
2348
diff
changeset
|
65 class TALError(BaseException): |
| 1049 | 66 |
| 67 def __init__(self, msg, position=(None, None)): | |
| 68 assert msg != "" | |
| 69 self.msg = msg | |
| 70 self.lineno = position[0] | |
| 71 self.offset = position[1] | |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
72 self.filename = None |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
73 |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
74 def setFile(self, filename): |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
75 self.filename = filename |
| 1049 | 76 |
| 77 def __str__(self): | |
| 78 result = self.msg | |
| 79 if self.lineno is not None: | |
| 80 result = result + ", at line %d" % self.lineno | |
| 81 if self.offset is not None: | |
| 82 result = result + ", column %d" % (self.offset + 1) | |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
83 if self.filename is not None: |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
84 result = result + ', in file %s' % self.filename |
| 1049 | 85 return result |
| 86 | |
| 87 class METALError(TALError): | |
| 88 pass | |
| 89 | |
| 90 class TALESError(TALError): | |
| 91 pass | |
| 92 | |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
93 class I18NError(TALError): |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
94 pass |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
95 |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
96 |
| 1049 | 97 class ErrorInfo: |
| 98 | |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
99 #__implements__ = ITALESErrorInfo |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
100 |
| 1049 | 101 def __init__(self, err, position=(None, None)): |
| 102 if isinstance(err, Exception): | |
| 103 self.type = err.__class__ | |
| 104 self.value = err | |
| 105 else: | |
| 106 self.type = err | |
| 107 self.value = None | |
| 108 self.lineno = position[0] | |
| 109 self.offset = position[1] | |
| 110 | |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
111 |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
112 |
| 1049 | 113 import re |
| 114 _attr_re = re.compile(r"\s*([^\s]+)\s+([^\s].*)\Z", re.S) | |
| 115 _subst_re = re.compile(r"\s*(?:(text|structure)\s+)?(.*)\Z", re.S) | |
| 116 del re | |
| 117 | |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
118 def parseAttributeReplacements(arg, xml): |
| 1049 | 119 dict = {} |
| 120 for part in splitParts(arg): | |
| 121 m = _attr_re.match(part) | |
| 122 if not m: | |
|
5377
12fe83f90f0d
Python 3 preparation: use repr() instead of ``.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5265
diff
changeset
|
123 raise TALError("Bad syntax in attributes: " + repr(part)) |
| 1049 | 124 name, expr = m.group(1, 2) |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
125 if not xml: |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
126 name = name.lower() |
|
5381
0942fe89e82e
Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents:
5377
diff
changeset
|
127 if name in dict: |
|
5377
12fe83f90f0d
Python 3 preparation: use repr() instead of ``.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5265
diff
changeset
|
128 raise TALError("Duplicate attribute name in attributes: " + repr(part)) |
| 1049 | 129 dict[name] = expr |
| 130 return dict | |
| 131 | |
| 132 def parseSubstitution(arg, position=(None, None)): | |
| 133 m = _subst_re.match(arg) | |
| 134 if not m: | |
|
5377
12fe83f90f0d
Python 3 preparation: use repr() instead of ``.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5265
diff
changeset
|
135 raise TALError("Bad syntax in substitution text: " + repr(arg), position) |
| 1049 | 136 key, expr = m.group(1, 2) |
| 137 if not key: | |
| 138 key = "text" | |
| 139 return key, expr | |
| 140 | |
| 141 def splitParts(arg): | |
| 142 # Break in pieces at undoubled semicolons and | |
| 143 # change double semicolons to singles: | |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
144 arg = arg.replace(";;", "\0") |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
145 parts = arg.split(';') |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
146 parts = [p.replace("\0", ";") for p in parts] |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
147 if len(parts) > 1 and not parts[-1].strip(): |
| 1049 | 148 del parts[-1] # It ended in a semicolon |
| 149 return parts | |
| 150 | |
| 151 def isCurrentVersion(program): | |
| 152 version = getProgramVersion(program) | |
| 153 return version == TAL_VERSION | |
| 154 | |
| 155 def getProgramMode(program): | |
| 156 version = getProgramVersion(program) | |
|
5420
7172c201dec2
Python 3 preparation: avoid obsolete types.*Type names.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5381
diff
changeset
|
157 if (version == TAL_VERSION and isinstance(program[1], tuple) and |
| 1049 | 158 len(program[1]) == 2): |
| 159 opcode, mode = program[1] | |
| 160 if opcode == "mode": | |
| 161 return mode | |
| 162 return None | |
| 163 | |
| 164 def getProgramVersion(program): | |
| 165 if (len(program) >= 2 and | |
|
5420
7172c201dec2
Python 3 preparation: avoid obsolete types.*Type names.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5381
diff
changeset
|
166 isinstance(program[0], tuple) and len(program[0]) == 2): |
| 1049 | 167 opcode, version = program[0] |
| 168 if opcode == "version": | |
| 169 return version | |
| 170 return None | |
| 171 | |
|
2348
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
172 import re |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
173 _ent1_re = re.compile('&(?![A-Z#])', re.I) |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
174 _entch_re = re.compile('&([A-Z][A-Z0-9]*)(?![A-Z0-9;])', re.I) |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
175 _entn1_re = re.compile('&#(?![0-9X])', re.I) |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
176 _entnx_re = re.compile('&(#X[A-F0-9]*)(?![A-F0-9;])', re.I) |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
177 _entnd_re = re.compile('&(#[0-9][0-9]*)(?![0-9;])') |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
178 del re |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
179 |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
180 def attrEscape(s): |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
181 """Replace special characters '&<>' by character entities, |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
182 except when '&' already begins a syntactically valid entity.""" |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
183 s = _ent1_re.sub('&', s) |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
184 s = _entch_re.sub(r'&\1', s) |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
185 s = _entn1_re.sub('&#', s) |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
186 s = _entnx_re.sub(r'&\1', s) |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
187 s = _entnd_re.sub(r'&\1', s) |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
188 s = s.replace('<', '<') |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
189 s = s.replace('>', '>') |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
190 s = s.replace('"', '"') |
|
8c2402a78bb0
beginning getting ZPT up to date: TAL first
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
191 return s |
