view test/test_jinja2.py @ 7668:5b41018617f2

fix: out of memory error when importing under postgresql If you try importing more than 20k items under postgresql you can run out of memory: psycopg2.errors.OutOfMemory: out of shared memory HINT: You might need to increase max_locks_per_transaction. Tuning memory may help, it's unknown at this point. This checkin forces a commit to the postgres database after 10,000 rows have been added. This clears out the savepoints for each row and starts a new transaction. back_postgresql.py: Implement commit mechanism in checkpoint_data(). Add two class level attributes for tracking the number of savepoints and the limit when the commit should happen. roundup_admin.py: implement pragma and dynamically create the config item RDBMS_SAVEPOINT_LIMIT used by checkpoint_data. Also fixed formatting of descriptions when using pragma list in verbose mode. admin_guide.txt, upgrading.txt: Document change and use of pragma savepoint_limit in roundup-admin for changing the default of 10,000. test/db_test_base.py: add some more asserts. In existing testAdminImportExport, set the savepoint limit to 5 to test setting method and so that the commit code will be run by existing tests. This provides coverage, but does not actually test that the commit is done every 5 savepoints 8-(. The verification of every 5 savepoints was done manually using a pdb breakpoint just before the commit. acknowledgements.txt: Added 2.4.0 section mentioning Norbert as he has done a ton of testing with much larger datasets than I can test with.
author John Rouillard <rouilj@ieee.org>
date Thu, 19 Oct 2023 16:11:25 -0400
parents d26921b851c3
children
line wrap: on
line source

#-*- encoding: utf-8 -*-
""" Testing the jinja2 templating engine of roundup-tracker.

Copyright: 2016 Intevation GmbH.
Author: Bernhard E. Reiter <bernhard@intevation.de>

This module is Free Software under the Roundup licensing of 1.5,
see the COPYING.txt file coming with Roundup.

Just a test file template for now.
"""
import shutil # only, needed for tearDown. TODO: Remove when refactored.
import unittest

from . import db_test_base

TESTSUITE_IDENTIFIER='jinja2'

class TestCase_Zero(unittest.TestCase):
    def test_zero(self):
        self.assertEqual(True, True)


class Jinja2Test(object):
    """Sets up and tears down an instance with database contents.

    Setup and teardown modelled after the use of db_test_base
    by several modules like test_xmlrpc and test_userauditor.

    TODO: Should probably be moved to a base case in db_test_base.py.
    """

    backend = None  # can be used to create tests per backend, see test_xmlrpc

    def setUp(self):
        self.dirname = '_test_' + TESTSUITE_IDENTIFIER
        self.instance = db_test_base.setupTracker(self.dirname, self.backend)
        self.db = self.instance.open('admin')

    def tearDown(self):
        self.db.close()
        try:
            shutil.rmtree(self.dirname)
        except OSError as error:
            if error.errno not in (errno.ENOENT, errno.ESRCH): raise

    def test_zero(self):
        """Do nothing just make sure that setup and teardown works."""
        pass


# only using one database backend for now, not sure if doing all
# backends will keep the test focussed enough to be useful for the used
# computing time. Would be okay to change in the future.
class anydbmJinja2Test(Jinja2Test, unittest.TestCase):
    backend = 'anydbm'

# vim: ts=4 et sts=4 sw=4 ai :



Roundup Issue Tracker: http://roundup-tracker.org/