Mercurial > p > roundup > code
annotate test/test_pythonexpr.py @ 7668:5b41018617f2
fix: out of memory error when importing under postgresql
If you try importing more than 20k items under postgresql you can run
out of memory:
psycopg2.errors.OutOfMemory: out of shared memory
HINT: You might need to increase max_locks_per_transaction.
Tuning memory may help, it's unknown at this point.
This checkin forces a commit to the postgres database after 10,000
rows have been added. This clears out the savepoints for each row and
starts a new transaction.
back_postgresql.py:
Implement commit mechanism in checkpoint_data(). Add two class level
attributes for tracking the number of savepoints and the limit when
the commit should happen.
roundup_admin.py:
implement pragma and dynamically create the config item
RDBMS_SAVEPOINT_LIMIT used by checkpoint_data.
Also fixed formatting of descriptions when using pragma list in
verbose mode.
admin_guide.txt, upgrading.txt:
Document change and use of pragma savepoint_limit in roundup-admin
for changing the default of 10,000.
test/db_test_base.py:
add some more asserts. In existing testAdminImportExport, set the
savepoint limit to 5 to test setting method and so that the commit
code will be run by existing tests. This provides coverage, but
does not actually test that the commit is done every 5 savepoints
8-(. The verification of every 5 savepoints was done manually
using a pdb breakpoint just before the commit.
acknowledgements.txt:
Added 2.4.0 section mentioning Norbert as he has done a ton of
testing with much larger datasets than I can test with.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Thu, 19 Oct 2023 16:11:25 -0400 |
| parents | e70885fe72a4 |
| children |
| rev | line source |
|---|---|
|
5676
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
1 """ |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
2 In Python 3, sometimes TAL "python:" expressions that refer to |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
3 variables but not all variables are recognized. That is in Python 2.7 |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
4 all variables used in a TAL "python:" expression are recognized as |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
5 references. In Python 3.5 (perhaps earlier), some TAL "python:" |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
6 expressions refer to variables but the reference generates an error |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
7 like this: |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
8 |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
9 <class 'NameError'>: name 'some_tal_variable' is not defined |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
10 |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
11 even when the variable is defined. Output after this message lists the |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
12 variable and its value. |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
13 """ |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
14 |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
15 import unittest |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
16 |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
17 from roundup.cgi.PageTemplates.PythonExpr import PythonExpr as PythonExprClass |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
18 |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
19 class ExprTest(unittest.TestCase): |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
20 def testExpr(self): |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
21 expr = '[x for x in context.assignedto ' \ |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
22 'if x.realname not in user_realnames]' |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
23 pe = PythonExprClass('test', expr, None) |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
24 # Looking at the expression, only context and user_realnames are |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
25 # external variables. The names assignedto and realname are members, |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
26 # and x is local. |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
27 required_names = ['context', 'user_realnames'] |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
28 got_names = pe._f_varnames |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
29 for required_name in required_names: |
|
e70885fe72a4
issue2551026: template variable not defined even though it is.
John Rouillard <rouilj@ieee.org>
parents:
diff
changeset
|
30 self.assertIn(required_name, got_names) |
