view test/pytest_patcher.py @ 7668:5b41018617f2

fix: out of memory error when importing under postgresql If you try importing more than 20k items under postgresql you can run out of memory: psycopg2.errors.OutOfMemory: out of shared memory HINT: You might need to increase max_locks_per_transaction. Tuning memory may help, it's unknown at this point. This checkin forces a commit to the postgres database after 10,000 rows have been added. This clears out the savepoints for each row and starts a new transaction. back_postgresql.py: Implement commit mechanism in checkpoint_data(). Add two class level attributes for tracking the number of savepoints and the limit when the commit should happen. roundup_admin.py: implement pragma and dynamically create the config item RDBMS_SAVEPOINT_LIMIT used by checkpoint_data. Also fixed formatting of descriptions when using pragma list in verbose mode. admin_guide.txt, upgrading.txt: Document change and use of pragma savepoint_limit in roundup-admin for changing the default of 10,000. test/db_test_base.py: add some more asserts. In existing testAdminImportExport, set the savepoint limit to 5 to test setting method and so that the commit code will be run by existing tests. This provides coverage, but does not actually test that the commit is done every 5 savepoints 8-(. The verification of every 5 savepoints was done manually using a pdb breakpoint just before the commit. acknowledgements.txt: Added 2.4.0 section mentioning Norbert as he has done a ton of testing with much larger datasets than I can test with.
author John Rouillard <rouilj@ieee.org>
date Thu, 19 Oct 2023 16:11:25 -0400
parents 1c94afabb2cb
children
line wrap: on
line source

"""
The following code was taken from:

    https://github.com/pytest-dev/pytest/issues/568#issuecomment-216569420

to resolve a bug with using pytest.mark.skip(). Once the bug is resolved in
pytest this file can be removed along with all the wrapper mark_class()
references in the other test files.
"""
import types


def mark_class(marker):
    '''Workaround for https://github.com/pytest-dev/pytest/issues/568'''
    def copy_func(f):
        try:
            return types.FunctionType(f.__code__, f.__globals__,
                                      name=f.__name__, argdefs=f.__defaults__,
                                      closure=f.__closure__)
        except AttributeError:
            return types.FunctionType(f.func_code, f.func_globals,
                                      name=f.func_name,
                                      argdefs=f.func_defaults,
                                      closure=f.func_closure)

    def mark(cls):
        if isinstance(cls, types.FunctionType):
            return marker(copy_func(cls))

        for method in dir(cls):
            if method.startswith('test'):
                f = copy_func(getattr(cls, method))
                setattr(cls, method, marker(f))
        return cls
    return mark

Roundup Issue Tracker: http://roundup-tracker.org/