http://hg.code.sf.net:8000/p/roundup/code/atom-log/tip/roundup/backends/back_postgresql.py Mercurial Repository: p/roundup/code: roundup/backends/back_postgresql.py history 2025-03-01T18:55:54+01:00 Generate savepoint only if necessary http://hg.code.sf.net:8000/p/roundup/code/#changeset-a81a3cd067fa00f483078fcabbba56d1ac7eee6b Ralf Schlatterbeck rsc@runtux.com 2025-03-01T18:55:54+01:00 2025-03-01T18:55:54+01:00
changeset a81a3cd067fa
branch
bookmark
tag
user Ralf Schlatterbeck <rsc@runtux.com>
description Generate savepoint only if necessary

Now some methods got an additional 'allow_abort' parameter. By default
this is True. When False the postgres backend generates a savepoint.
The methods are called with allow_abort=False from some of the cgi
methods which can produce a traceback when called with data from the
web-interface.
files
Fix filter_iter so that exception is caught http://hg.code.sf.net:8000/p/roundup/code/#changeset-45ec660eb7f7c748218df23b21140f3635e8b4a1 Ralf Schlatterbeck rsc@runtux.com 2025-02-27T10:05:38+01:00 2025-02-27T10:05:38+01:00
changeset 45ec660eb7f7
branch
bookmark
tag
user Ralf Schlatterbeck <rsc@runtux.com>
description Fix filter_iter so that exception is caught
files
issue2551376: Fix tracebacks in item templates http://hg.code.sf.net:8000/p/roundup/code/#changeset-82a26ea1afdf59e43c8fd228ecfc0bdd8349fb5f Ralf Schlatterbeck rsc@runtux.com 2025-02-26T11:24:13+01:00 2025-02-26T11:24:13+01:00
changeset 82a26ea1afdf
branch
bookmark
tag
user Ralf Schlatterbeck <rsc@runtux.com>
description issue2551376: Fix tracebacks in item templates
files
backout bbc99def147a. All the other backends were programmed to accomodate the skip one nature of postgres sequences http://hg.code.sf.net:8000/p/roundup/code/#changeset-cfa7d43a36583d32bb80c59c78a8fca94a2cdba5 John Rouillard rouilj@ieee.org 2025-01-20T16:32:39-05:00 2025-01-20T16:32:39-05:00
changeset cfa7d43a3658
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description backout bbc99def147a. All the other backends were programmed to accomodate the skip one nature of postgres sequences
files
fix: issue2550815 - roundup-admin import for postgres causes id to be skipped. http://hg.code.sf.net:8000/p/roundup/code/#changeset-bbc99def147a23879dc4ce3955b867ca2759f374 John Rouillard rouilj@ieee.org 2025-01-19T19:37:48-05:00 2025-01-19T19:37:48-05:00
changeset bbc99def147a
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: issue2550815 - roundup-admin import for postgres causes id to be skipped.

Fixed code so that the id used to set the sequence
will be returned and not skipped. (Tom Ekberg diagnosed and
supplied the fix.)
files
fix(db): Make using pg_service work again. http://hg.code.sf.net:8000/p/roundup/code/#changeset-8147f6deac9ffc0582e831d77491868f96effcd5 John Rouillard rouilj@ieee.org 2023-12-28T15:13:42-05:00 2023-12-28T15:13:42-05:00
changeset 8147f6deac9f
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix(db): Make using pg_service work again.

When I did the merge of schema support I broke pg_service.conf support
by replacing get_database_name with db_schema_split. This commit
fixes it.

Also this commit returns the schema if one is specified in
pg_service.conf.

back_postgresql.py:

Replace calls to db_schema_split() with get_database_schema_names()
(new name for get_database_name()). Rename db_schema_split to
_db_schema_split. It now returns a tuple (dbname, schema) rather
than a list. It is used only by get_database_schema_names() which
also returns tuples.

get_database_schema_names() can also get schema info for the service

(if present) as specified by pg_service.conf.

Add get_database_user() to get the user from either RDBMS_USER or
pg_service.conf. (User needed for creating schema, so not needed
before schema patch.

import re at the top of file and remove lower import.

Remove some schema code from db_command as it's not needed. The
database conection is done to either postgresql or template1
existing databases. This command never connects to the roundp
specified db.

test/test_postgresql.py:

Reorganize top level imports, add import os. Replace import of
db_schema_split with get_database_schema_names. Also replace calls
to db_schema_split.

Create new Opener for the service file. Set PGSERVICEFILE to point
to test/pg_service.conf.

Add three new classes to test Service:

1) using regular db
2) using schema within db
3) Unable to parse schema name from pg_service.conf.

The last doesn't need a db. Number 1 and 2 reuse the tests in ROTest
to verify db connectivity.

test/pg_service.conf:

three service connections for: db only, db and schema, and incorrectly
specified schema test cases.

doc/upgrading.txt:

updated to current status. Included example schema definition in
service file.
files
feat: issue2550852 - support using a specified PostgreSQL db schema http://hg.code.sf.net:8000/p/roundup/code/#changeset-3071db43bfb6d957b2dab43ae0f9df0e98f86002 John Rouillard rouilj@ieee.org 2023-12-27T22:52:14-05:00 2023-12-27T22:52:14-05:00
changeset 3071db43bfb6
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat: issue2550852 - support using a specified PostgreSQL db schema

Finally after 7 years this is closed.

roundup/backends/back_postgresql.py:
Support use of schema when specified in RDBMS_NAME. Stuart McGraws
code is finally merged 8-).

test/test_postgresql.py, test/conftest.py:
Run all postgresql tests in the schema db as well.
Also make sure that db_nuke raises an error when trying to delete
the schema test database. Conftest defines pg_schema mark that can
be used to exclude schema tests with pytest -m "not pg_schema".

roundup/configuration.py:
change doc on RDBMS_NAME to include db.schema form.

.travis.yml, .github/workflows/ci-test.yml:
create schema test db; add user for testing with schema; grant new
user create privs for schema.

doc/installation.txt:
Reference to roundup-admin init deleting schema added.

doc/mysql.txt doc/postgresql.txt:
New documentation on psql/mysql commands to set up a production db.

doc/upgrading.txt:
mention schema support, also document service setting for
selecting connection from pg_service.conf.

doc/reference.txt:
update config.ini documentation for RDBMS_NAME.
files
fix: make regexp a raw string http://hg.code.sf.net:8000/p/roundup/code/#changeset-3da452f4a3acb6243723f89bdca8acbf4684f65d John Rouillard rouilj@ieee.org 2023-12-23T21:59:08-05:00 2023-12-23T21:59:08-05:00
changeset 3da452f4a3ac
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: make regexp a raw string

So I don't have to escape the \s sequences.
files
fix: typo in variable name http://hg.code.sf.net:8000/p/roundup/code/#changeset-2eb976e1de2a3be876d1751f957a6f5c8f3f9dbf John Rouillard rouilj@ieee.org 2023-12-21T09:19:08-05:00 2023-12-21T09:19:08-05:00
changeset 2eb976e1de2a
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: typo in variable name
files
fix: spelling fix in comment. http://hg.code.sf.net:8000/p/roundup/code/#changeset-0ccc6d3ba9f287ffcea82254a298b77c0a44c9bf John Rouillard rouilj@ieee.org 2023-12-19T08:56:14-05:00 2023-12-19T08:56:14-05:00
changeset 0ccc6d3ba9f2
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: spelling fix in comment.
files
fix: remove 'public' schema from __words table in fix_version_3_tables http://hg.code.sf.net:8000/p/roundup/code/#changeset-95f0002e85c45bebd9e09561791e6d77394860ac John Rouillard rouilj@ieee.org 2023-12-18T20:20:15-05:00 2023-12-18T20:20:15-05:00
changeset 95f0002e85c4
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: remove 'public' schema from __words table in fix_version_3_tables

If you are using a different schema than public, fix_version_3_tables
crashes on the migration/setup.

Nothing else n the postgresql code uses an explicit public schema.

This removes an impediment for those wanting to use schemas rather
than databases as the primary organizational structure.

See: issue2551299

Also see: issue2550852 for code to allow use of schemas as a first
class config parameter.
files
fix: figure out dbname when using pg_service. http://hg.code.sf.net:8000/p/roundup/code/#changeset-b41750bf9f033b1e7dc39032c8fa9ffee9a1f71a John Rouillard rouilj@ieee.org 2023-12-18T16:56:49-05:00 2023-12-18T16:56:49-05:00
changeset b41750bf9f03
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: figure out dbname when using pg_service.

See: https://issues.roundup-tracker.org/msg7890 where drop database
is missing the dbname.
files
feat(db): support using postgresql service connection file http://hg.code.sf.net:8000/p/roundup/code/#changeset-4af0d235b5709fef74ad266a14fe243260d48199 John Rouillard rouilj@ieee.org 2023-11-07T12:11:37-05:00 2023-11-07T12:11:37-05:00
changeset 4af0d235b570
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat(db): support using postgresql service connection file

Add new service rdbms config option to set the service name to be used
with a postgresql service connection file.

This can be done using the PGSERVICE environment variable for a single
instance tracker server. For a multi-instance server this per-tracker
config option is needed.

Note that settings (host, user, (db)name...) in config.ini file will
override the service connection file setting. Also setting PGSERVICE
and service will use the service setting.
files
fix: out of memory error when importing under postgresql http://hg.code.sf.net:8000/p/roundup/code/#changeset-5b41018617f2984f188ec7f1516cfcd970a8b9ba John Rouillard rouilj@ieee.org 2023-10-19T16:11:25-04:00 2023-10-19T16:11:25-04:00
changeset 5b41018617f2
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: out of memory error when importing under postgresql

If you try importing more than 20k items under postgresql you can run
out of memory:

psycopg2.errors.OutOfMemory: out of shared memory
HINT: You might need to increase max_locks_per_transaction.

Tuning memory may help, it's unknown at this point.

This checkin forces a commit to the postgres database after 10,000
rows have been added. This clears out the savepoints for each row and
starts a new transaction.

back_postgresql.py:

Implement commit mechanism in checkpoint_data(). Add two class level
attributes for tracking the number of savepoints and the limit when
the commit should happen.

roundup_admin.py:

implement pragma and dynamically create the config item
RDBMS_SAVEPOINT_LIMIT used by checkpoint_data.

Also fixed formatting of descriptions when using pragma list in
verbose mode.

admin_guide.txt, upgrading.txt:

Document change and use of pragma savepoint_limit in roundup-admin
for changing the default of 10,000.

test/db_test_base.py:

add some more asserts. In existing testAdminImportExport, set the
savepoint limit to 5 to test setting method and so that the commit
code will be run by existing tests. This provides coverage, but
does not actually test that the commit is done every 5 savepoints
8-(. The verification of every 5 savepoints was done manually
using a pdb breakpoint just before the commit.

acknowledgements.txt:

Added 2.4.0 section mentioning Norbert as he has done a ton of
testing with much larger datasets than I can test with.
files
flake8 fixes http://hg.code.sf.net:8000/p/roundup/code/#changeset-b5062cb5c2a2e56f1ae6952bbb7e331829509a23 John Rouillard rouilj@ieee.org 2022-09-08T13:41:26-04:00 2022-09-08T13:41:26-04:00
changeset b5062cb5c2a2
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description flake8 fixes
files
fix missed renaming from revision 98d3bf8ffb19 done 2004-04-18 http://hg.code.sf.net:8000/p/roundup/code/#changeset-b0dbc13a835a582d27054f10289dc0248dfb35b4 John Rouillard rouilj@ieee.org 2022-09-04T00:43:29-04:00 2022-09-04T00:43:29-04:00
changeset b0dbc13a835a
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix missed renaming from revision 98d3bf8ffb19 done 2004-04-18
files
- issue2551223 - fix timestamp truncation in mysql and postgresql http://hg.code.sf.net:8000/p/roundup/code/#changeset-bdd28b244839c257336c4eefc7ba45593437d1d3 John Rouillard rouilj@ieee.org 2022-07-25T17:20:20-04:00 2022-07-25T17:20:20-04:00
changeset bdd28b244839
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description - issue2551223 - fix timestamp truncation in mysql and postgresql

The data types used to represent timestamps in pg and mysql for
ephemeral tables: sessions and otks don't have enough signifcant
digits to work. As a result the timestamps are rounduped (up/down)
rsuling in the stored timestamp being 2 minutes (pg) or 2-3
hours(mysql) off from what it should be.

Modify db schema to use a numeric type that preserves more significant
figures. Implement schema upgrade. Document need for upgrade in
upgrading.txt.

Write tests for schema upgrade.

Implement test for updateTimestamp method on BasicDatabase that showed
this issue in the first place. Write overrides for test for
anydbm/memorydb which store timestamp properly or not at all.
files
Change checkpoint_data and restore_connection_on_error to subtransaction http://hg.code.sf.net:8000/p/roundup/code/#changeset-db3f0ba75b4ab01314df61a5210f65d1d6a5352e John Rouillard rouilj@ieee.org 2022-01-29T11:29:36-05:00 2022-01-29T11:29:36-05:00
changeset db3f0ba75b4a
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Change checkpoint_data and restore_connection_on_error to subtransaction

checkpoint_data and restore_connection_on_error used to commit() and
rollback() the db connection. This causes additional I/O and load.

Changed them to use 'SAVEPOINT name' and 'ROLLBACK TO name' to get a
faster method for handling errors within a tranaction.

One thing to note is that postgresql (unlike SQL std) doesn't
overwrite an older savepoint with he same name. It keeps all
savepoints but only rolls back to the newest one with a given name.
This could be a resource issue. I left a commented out release
statement in case somebody runs into an issue due to too many
savepoints. I expect it to slow down the import but....
files
Allow Roundup to use PostgreSQL database native full text search http://hg.code.sf.net:8000/p/roundup/code/#changeset-0d99ae7c8de62fdc4561e7f0a8a6b38da86e34cd John Rouillard rouilj@ieee.org 2022-01-27T19:48:48-05:00 2022-01-27T19:48:48-05:00
changeset 0d99ae7c8de6
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Allow Roundup to use PostgreSQL database native full text search

back_postgreql.py - schema version changes for schema version 7.

configuration.py - added indexer_language checks for postgresql. Hardcoded
list for now.

Docs admin_guide and upgrading

Tests.


This also restructures the version upgrade tests for the rdbms
backends. They can run all of them now as the proper cascade is
developed to roll back changes to version 6.
files
issue2551189 - increase size of words in full text index. http://hg.code.sf.net:8000/p/roundup/code/#changeset-39189dd94f2cc9d0b460685b20b3fc077d59f7ec John Rouillard rouilj@ieee.org 2022-01-26T15:04:09-05:00 2022-01-26T15:04:09-05:00
changeset 39189dd94f2c
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551189 - increase size of words in full text index.

Increased indexed word maxlength to 50

DB migration code is written and tests work.

Restructured some tests to allow for code reuse.

Docs.

If this passes CI without errors 2551189 should be done. However,
testing on my system generates errors. Encoding (indexer unicode
russian unicode string invalid) and collation errors (utf8_bin not
valid) when running under python2. No issues with python3 and I
haven't changed code that should cause these since the last successful
build in CI. So if this fails in CI we will have more checkins.
files
issue2551189 - increase text search maxlength http://hg.code.sf.net:8000/p/roundup/code/#changeset-e70e2789bc2c7a554e59d54918bf31d0a2b588a5 John Rouillard rouilj@ieee.org 2022-01-25T13:22:00-05:00 2022-01-25T13:22:00-05:00
changeset e70e2789bc2c
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551189 - increase text search maxlength

This removes I think all the magic references to 25 and 30 (varchar
size) and replaces them with references to maxlength or maxlength+5.

I am not sure why the db column is 5 characters larger than the size
of what should be the max size of a word, but I'll keep the buffer
of 5 as making it 1/5 the size of maxlength makes less sense.

Also added tests for fts search in templating which were missing.
Added postgres, mysql and sqlite native indexing backends in which to
test fts. Added fts test to native-fts as well to make sure it's
working.

I want to commit this now for CI.

Todo:

add test cases for the use of FTS in the csv output in
actions.py. There is no test coverage of the match case there.

change maxlength to a higher value (50) as requested in the ticket.

Modify existing extremewords test cases to allow words > 25 and < 51

write code to migrate column sizes for mysql and postgresql to match
maxlength I will roll this into the version 7 schema update that
supports use of database fts support.
files
Add words_both_idx to newly created databases. http://hg.code.sf.net:8000/p/roundup/code/#changeset-4f8fc55730e1d675b30cc63805045458c0ce7b98 John Rouillard rouilj@ieee.org 2022-01-22T01:33:06-05:00 2022-01-22T01:33:06-05:00
changeset 4f8fc55730e1
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Add words_both_idx to newly created databases.

The changes to the otk and session table done by fix_version_3_tables
look like they were rolled into create_version_2_tables, but the new
index creation wasn't. So call self.fix_version_3_tables() to add the
index.
files
issue2551142 - Import of retired node ... unique constraint failure. http://hg.code.sf.net:8000/p/roundup/code/#changeset-c1d3fbcdbfbd3a23b25c9ac9b6f090229d63b768 John Rouillard rouilj@ieee.org 2021-06-10T12:52:05-04:00 2021-06-10T12:52:05-04:00
changeset c1d3fbcdbfbd
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551142 - Import of retired node ... unique constraint failure.

Title: Import of retired node with username after active node fails
with unique constraint failure.

More fixes needed for mysql and postgresql.

mysql: add unique constraint for (keyvalue, __retired__) when
creating class in the database.

On schema change if class is changed, remove the unique
constraint too.

upgrade version of rdbms database from 5 to 6 to add constraint
to all version 5 databases that were created as version 5
and didn't get the unique constraint. Make no changes
on version 5 databases upgraded from version 4, the upgrade
process to 5 added the constraint. Make no changes
to other databases (sqlite, postgres) during upgrade from
version 5 to 6.

postgres: Handle the exception raised on unique constraint violation.
The exception invalidates the database connection so it
can't be used to recover from the exception.

Added two new database methods:

checkpoint_data - performs a db.commit under postgres
does nothing on other backends
restore_connection_on_error - does a db.rollback on
postgres, does nothing on other
backends

with the rollback() done on the connection I can use the
database connection to fixup the import that failed on the
unique constraint. This makes postgres slower but without the
commit after every imported object, the rollback will delete
all the entries done up to this point.

Trying to figure out how to make the caller do_import batch
and recover from this failure is beyond me.

Also dismissed having to process the export csv file before
importing. Pushing that onto a user just seems wrong. Also
since import/export isn't frequently done the lack of
surprise on having a failing import and reduced
load/frustration for the user seems worth it. Also the import
can be run in verbose mode where it prints out a row as it is
processed, so it may take a while, ut the user can get
feedback.

db_test-base.py: add test for upgrade from 5 to 6.
files
Use server-side cursor for postgres in some cases http://hg.code.sf.net:8000/p/roundup/code/#changeset-6a6b4651be1f53bd98ce5c430c3bf9c355da4fb9 Ralf Schlatterbeck rsc@runtux.com 2021-03-04T12:55:21+01:00 2021-03-04T12:55:21+01:00
changeset 6a6b4651be1f
branch
bookmark
tag
user Ralf Schlatterbeck <rsc@runtux.com>
description Use server-side cursor for postgres in some cases

In filter, filter_iter, and _materialize_multilinks, use named cursor
with postgresql. This turns of client-side cursor handling and avoids
*large* roundup process (or wsgi process) in case of large results.
Fixes issue2551114.
files
issue2551040: New release of psycopg2 drops support for psycopg1 http://hg.code.sf.net:8000/p/roundup/code/#changeset-5cb6e6b594b00eac69386a2419931c99af3753a8 John Rouillard rouilj@ieee.org 2019-06-01T15:47:34-04:00 2019-06-01T15:47:34-04:00
changeset 5cb6e6b594b0
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551040: New release of psycopg2 drops support for psycopg1

First try at suport for psycopg2. Change .travis.yml to install newest
psycopg2 without psycopg1 support.
files
Fix commits although a Reject exception is raised http://hg.code.sf.net:8000/p/roundup/code/#changeset-62de601bdf6fd224fead26c33b585af5a05e7d80 Ralf Schlatterbeck rsc@runtux.com 2018-04-20T18:46:28+02:00 2018-04-20T18:46:28+02:00
changeset 62de601bdf6f
branch
bookmark
tag
user Ralf Schlatterbeck <rsc@runtux.com>
description Fix commits although a Reject exception is raised

Fix the problem that changes are committed to the database (due to
commits to otk handling) even when a Reject exception occurs. The fix
implements separate database connections for otk/session handling and
normal database operation.
files
Use Python-3-compatible 'as' syntax for except statements http://hg.code.sf.net:8000/p/roundup/code/#changeset-198b6e810c67c4e7e38f1aa41edbc864783fe6e6 Eric S. Raymond esr@thyrsus.com 2017-08-24T22:21:37-04:00 2017-08-24T22:21:37-04:00
changeset 198b6e810c67
branch
bookmark
tag
user Eric S. Raymond <esr@thyrsus.com>
description Use Python-3-compatible 'as' syntax for except statements

Many raise statements near these are also fixed.
So are two ivorrect file encoding marks ('utf8'->'utf-8').
files
issue2550853 - better error handling and cleanup on some postgres http://hg.code.sf.net:8000/p/roundup/code/#changeset-57452bc6d98941ee058aaf3489febd5da2422ad6 John Rouillard rouilj@ieee.org 2016-07-02T21:15:23-04:00 2016-07-02T21:15:23-04:00
changeset 57452bc6d989
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2550853 - better error handling and cleanup on some postgres
tests by Stuart McGraw.

My postgres tests passed both before and after I applied his patch,
but the code change seems rational.

issue2086536 - back_postgresql: fixing pg_command and prefering
psycopg2. Patch done by Philipp Gortan (mephinet). His patch
also improves handling of retryable errors. Applied and
edited by John Rouillard. Edits included removing support for
psycopg1. See:
https://sourceforge.net/p/roundup/mailman/message/32855027/
for rational for dropping it

Again all 137 postgres tests pass.
files
- issue2550636, issue2550909: Added support for Whoosh indexer. http://hg.code.sf.net:8000/p/roundup/code/#changeset-e74c3611b1387156cc22a7e18d0d5048b6961f8a John Rouillard rouilj@ieee.org 2016-06-25T20:10:03-04:00 2016-06-25T20:10:03-04:00
changeset e74c3611b138
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description - issue2550636, issue2550909: Added support for Whoosh indexer.
Also adds new config.ini setting called indexer to select
indexer. See ``doc/upgrading.txt`` for details. Initial patch
done by David Wolever. Patch modified (see ticket or below for
changes), docs updated and committed.

I have an outstanding issue with test/test_indexer.py. I have to
comment out all imports and tests for indexers I don't have (i.e.
mysql, postgres) otherwise no tests run.

With that change made, dbm, sqlite (rdbms), xapian and whoosh indexes
are all passing the indexer tests.

Changes summary:
1) support native back ends dbm and rdbms. (original patch only fell
through to dbm)
2) Developed whoosh stopfilter to not index stopwords or words outside
the the maxlength and minlength limits defined in index_common.py.
Required to pass the extremewords test_indexer test. Also I
removed a call to .lower on the input text as the tokenizer I chose
automatically does the lowercase.
3) Added support for max/min length to find. This was needed to pass
extremewords test.
4) Added back a call to save_index in add_text. This allowed all but
two tests to pass.
5) Fixed a call to:
results = searcher.search(query.Term("identifier", identifier))
which had an extra parameter that is an error under current whoosh.
6) Set limit=None in search call for find() otherwise it only return
10 items. This allowed it to pass manyresults test

Also due to changes in the roundup code removed the call in
indexer_whoosh to

from roundup.anypy.sets_ import set

since we use the python builtin set.
files
New rdbms configuration option 'isolation_level' http://hg.code.sf.net:8000/p/roundup/code/#changeset-05c857e5dbed8261415450c21a2558a508daf4da Ralf Schlatterbeck rsc@runtux.com 2014-04-06T11:11:04+02:00 2014-04-06T11:11:04+02:00
changeset 05c857e5dbed
branch
bookmark
tag
user Ralf Schlatterbeck <rsc@runtux.com>
description New rdbms configuration option 'isolation_level'

See discussion in http://issues.roundup-tracker.org/issue2550806
files
Closed issue2550805 (Postgresql should search title case insensitive), by Tom Ekberg. http://hg.code.sf.net:8000/p/roundup/code/#changeset-4a017661e414f539e5ab7a4fcc413023bb222655 Bernhard Reiter bernhard@intevation.de 2013-05-10T23:24:10+02:00 2013-05-10T23:24:10+02:00
changeset 4a017661e414
branch
bookmark
tag
user Bernhard Reiter <bernhard@intevation.de>
description Closed issue2550805 (Postgresql should search title case insensitive), by Tom Ekberg.
files
Remove keyword expansions from CVS. All regression tests passed afterwards. http://hg.code.sf.net:8000/p/roundup/code/#changeset-6e3e4f24c75376f61ae0bf0e9ee334567585c38e Eric S. Raymond esr@thyrsus.com 2011-11-16T09:51:38-05:00 2011-11-16T09:51:38-05:00
changeset 6e3e4f24c753
branch
bookmark
tag
user Eric S. Raymond <esr@thyrsus.com>
description Remove keyword expansions from CVS. All regression tests passed afterwards.
files
The PostgreSQL backend quotes database names now for CREATE and DROP... http://hg.code.sf.net:8000/p/roundup/code/#changeset-6467fd9a3afd71c0752ee86cbfa6cd8993534965 Bernhard Reiter Bernhard.Reiter@intevation.de 2011-07-01T15:17:32+00:00 2011-07-01T15:17:32+00:00
changeset 6467fd9a3afd
branch
bookmark
tag
user Bernhard Reiter <Bernhard.Reiter@intevation.de>
description The PostgreSQL backend quotes database names now for CREATE and DROP...

...enabling more exotic tracker names. Closes issue2550497.
Thanks to Sebastian Harl for providing the patch.
files
PostgreSQL backend minor improvement: http://hg.code.sf.net:8000/p/roundup/code/#changeset-f6c49df25048dd54fb603c086589a73007398d4d Bernhard Reiter Bernhard.Reiter@intevation.de 2011-07-01T15:05:09+00:00 2011-07-01T15:05:09+00:00
changeset f6c49df25048
branch
bookmark
tag
user Bernhard Reiter <Bernhard.Reiter@intevation.de>
description PostgreSQL backend minor improvement:

database creation less likely to fail for PostgreSQL versions >= 8.1
as the table "postgres" is used by default. Closes
issue2550543. Thanks to Kai Storbeck for the patch.
files
Configuration issue: http://hg.code.sf.net:8000/p/roundup/code/#changeset-4f353d71d7166f639636b352f5caec25bb231ff4 Ralf Schlatterbeck schlatterbeck@users.sourceforge.net 2011-03-16T11:26:50+00:00 2011-03-16T11:26:50+00:00
changeset 4f353d71d716
branch
bookmark
tag
user Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
description Configuration issue:

On some postgresql 8.4 installations (notably on debian squeeze) the
default template database used for database creation doesn't match the
needed character encoding UTF8 -- a new config option 'template' in
the rdbms section now allows specification of the template. You know
you need this option if you get the error message:

psycopg2.DataError: new encoding (UTF8) is incompatible with the
encoding of the template database (SQL_ASCII)
HINT: Use the same encoding as in the template database, or use
template0 as template.
files
- more logger fixes, sorry for the noise. http://hg.code.sf.net:8000/p/roundup/code/#changeset-67bef70ab9b956f0af82e6eff9a743fd5d84187c Ralf Schlatterbeck schlatterbeck@users.sourceforge.net 2010-09-17T06:47:58+00:00 2010-09-17T06:47:58+00:00
changeset 67bef70ab9b9
branch
bookmark
tag
user Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
description - more logger fixes, sorry for the noise.
files
- more logger fixes -- use correct hierarchical logger names... http://hg.code.sf.net:8000/p/roundup/code/#changeset-9655a1b65974dfe2d1e2270fa419a067829ef72a Ralf Schlatterbeck schlatterbeck@users.sourceforge.net 2010-09-16T07:44:50+00:00 2010-09-16T07:44:50+00:00
changeset 9655a1b65974
branch
bookmark
tag
user Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
description - more logger fixes -- use correct hierarchical logger names...

...always starting with "roundup." -- otherwise logger configuration
changed in changeset
2010-08-09T03:36:06Z!schlatterbeck@users.sourceforge.net will not work
(e.g. turning debugging on)
files
Beginnings of a big code cleanup / modernisation to make 2to3 happy http://hg.code.sf.net:8000/p/roundup/code/#changeset-13b3155869e050964ca244b2fc0e586e7918a3f3 Richard Jones richard@users.sourceforge.net 2010-02-22T05:26:57+00:00 2010-02-22T05:26:57+00:00
changeset 13b3155869e0
branch
bookmark
tag
user Richard Jones <richard@users.sourceforge.net>
description Beginnings of a big code cleanup / modernisation to make 2to3 happy
files
remove unused, deprecated import http://hg.code.sf.net:8000/p/roundup/code/#changeset-5e3991670011a323e6ad42d00226b596dc71ab0f Richard Jones richard@users.sourceforge.net 2009-08-10T03:42:17+00:00 2009-08-10T03:42:17+00:00
changeset 5e3991670011
branch
bookmark
tag
user Richard Jones <richard@users.sourceforge.net>
description remove unused, deprecated import
files
Fix for postgres 8.3 compatibility (and bug) (patch [SF#2030479]) http://hg.code.sf.net:8000/p/roundup/code/#changeset-732a37da3a1071b17e5d827b873ac1cc1e0d28b0 Richard Jones richard@users.sourceforge.net 2008-08-07T05:50:03+00:00 2008-08-07T05:50:03+00:00
changeset 732a37da3a10
branch
bookmark
tag
user Richard Jones <richard@users.sourceforge.net>
description Fix for postgres 8.3 compatibility (and bug) (patch [SF#2030479])
files
add self to Sessions.set http://hg.code.sf.net:8000/p/roundup/code/#changeset-97ae174f7a0e742c53a249993d06c6ba1273d2b6 Justus Pendleton jpend@users.sourceforge.net 2007-09-28T15:15:06+00:00 2007-09-28T15:15:06+00:00
changeset 97ae174f7a0e
branch
bookmark
tag
user Justus Pendleton <jpend@users.sourceforge.net>
description add self to Sessions.set

Richard noticed my earlier change broke things. Third try's the
charm?
files
assorted pyflakes fixes http://hg.code.sf.net:8000/p/roundup/code/#changeset-21d3d7eeea8ce005759a795b1b441cc4479c1140 Justus Pendleton jpend@users.sourceforge.net 2007-09-27T06:12:57+00:00 2007-09-27T06:12:57+00:00
changeset 21d3d7eeea8c
branch
bookmark
tag
user Justus Pendleton <jpend@users.sourceforge.net>
description assorted pyflakes fixes

I (re-)discovered pyflakes today and ran it over to roundup
code to see what it had to say. It reported a ton of "imported
module foo but you're not actually using it" warnings and a couple
of actual errors that this patch fixes.

The code was wrong but it was mostly stuff that never gets
executed...like hyperdb.Choice.
files
fix vim modelines http://hg.code.sf.net:8000/p/roundup/code/#changeset-416606b09b278d370dda45a41231a703ee082eee Justus Pendleton jpend@users.sourceforge.net 2007-09-25T19:49:19+00:00 2007-09-25T19:49:19+00:00
changeset 416606b09b27
branch
bookmark
tag
user Justus Pendleton <jpend@users.sourceforge.net>
description fix vim modelines
files
safer session cleanup on postgres backends http://hg.code.sf.net:8000/p/roundup/code/#changeset-c8899c4bf6add168d78bd005e370a0130e825310 Justus Pendleton jpend@users.sourceforge.net 2007-09-25T02:44:50+00:00 2007-09-25T02:44:50+00:00
changeset c8899c4bf6ad
branch
bookmark
tag
user Justus Pendleton <jpend@users.sourceforge.net>
description safer session cleanup on postgres backends

The logic is all from Erik Forsberg, so thank him (or blame him ;) for it
I just moved it into the postgres backend to make it a little cleaner and
am committing it after making sure it seems to work.

This fixes [SF#1703116]
files
Seems like not all versions of postgres-drivers contain the string "ERROR"... http://hg.code.sf.net:8000/p/roundup/code/#changeset-1d47f70a99bfa06a5922aa4505dd2dd51ff6b779 Ralf Schlatterbeck schlatterbeck@users.sourceforge.net 2007-08-09T08:58:42+00:00 2007-08-09T08:58:42+00:00
changeset 1d47f70a99bf
branch
bookmark
tag
user Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
description Seems like not all versions of postgres-drivers contain the string "ERROR"...

...in error-messages. In that case creating the database (when it
already exists) fails. This fixes the problem for me.
files
Indexing fixes. http://hg.code.sf.net:8000/p/roundup/code/#changeset-bb30bbfc7cdda4caa7010d3056b88b020786aaef Ralf Schlatterbeck schlatterbeck@users.sourceforge.net 2007-06-21T07:35:51+00:00 2007-06-21T07:35:51+00:00
changeset bb30bbfc7cdd
branch
bookmark
tag
user Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
description Indexing fixes.

- Add new rdbms-indexes to the tables for the full-text index. These
speed up adding new entries and finding old ones to remove.
- Remove getprops method from FileClass in backends/rdbms_common: This
forced indexing for the content property even if the user decided to
turn it off for a FileClass instance.
files
fix error detection for non-EN locales of postgres [SF#1592249] http://hg.code.sf.net:8000/p/roundup/code/#changeset-f1363e19121ae74666124ade59c59727b6be0c50 Richard Jones richard@users.sourceforge.net 2006-11-09T00:55:33+00:00 2006-11-09T00:55:33+00:00
changeset f1363e19121a
branch
bookmark
tag
user Richard Jones <richard@users.sourceforge.net>
description fix error detection for non-EN locales of postgres [SF#1592249]
files
better conflict retry in postgresql backend [SF#1552809] http://hg.code.sf.net:8000/p/roundup/code/#changeset-5770f1802cd09e15ce96f5f4a71a3bf6d6a0c3e0 Richard Jones richard@users.sourceforge.net 2006-10-03T23:28:51+00:00 2006-10-03T23:28:51+00:00
changeset 5770f1802cd0
branch
bookmark
tag
user Richard Jones <richard@users.sourceforge.net>
description better conflict retry in postgresql backend [SF#1552809]

fix time log example [SF#1554630]
files
change as per last message in patch [SF#1429391] http://hg.code.sf.net:8000/p/roundup/code/#changeset-50add98cbbac2adc1bad6044c2faaf69e3de73ac Richard Jones richard@users.sourceforge.net 2006-10-03T23:15:09+00:00 2006-10-03T23:15:09+00:00
changeset 50add98cbbac
branch
bookmark
tag
user Richard Jones <richard@users.sourceforge.net>
description change as per last message in patch [SF#1429391]
files
Postgres backend allows transaction collisions to be ignored when... http://hg.code.sf.net:8000/p/roundup/code/#changeset-ff9f4ca424542977700f022594dba3a54569e07a Richard Jones richard@users.sourceforge.net 2006-08-29T04:20:50+00:00 2006-08-29T04:20:50+00:00
changeset ff9f4ca42454
branch
bookmark
tag
user Richard Jones <richard@users.sourceforge.net>
description Postgres backend allows transaction collisions to be ignored when...

...committing clenup in the sessions database
files
Null-value sorting fixes: http://hg.code.sf.net:8000/p/roundup/code/#changeset-4d9adb8bc3b102509b665924874cf69edd506e4b Ralf Schlatterbeck schlatterbeck@users.sourceforge.net 2006-08-23T12:57:10+00:00 2006-08-23T12:57:10+00:00
changeset 4d9adb8bc3b1
branch
bookmark
tag
user Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
description Null-value sorting fixes:

- Fixed sorting of NULL values for the PostgreSQL backend. Generic
implementation of additional sort-key for NULL values, can be reused
by other (future) rdbms backends.
- Added a test for NULL value sorting and removed an override for the
Interval test for the PostgreSQL backend which had a special case for
NULL values
files
update for latest version of psycopg2 (patch-ish [SF#429391]) http://hg.code.sf.net:8000/p/roundup/code/#changeset-bfb12092830955049baf626ecd02fedd28750eea Richard Jones richard@users.sourceforge.net 2006-08-11T00:44:00+00:00 2006-08-11T00:44:00+00:00
changeset bfb120928309
branch
bookmark
tag
user Richard Jones <richard@users.sourceforge.net>
description update for latest version of psycopg2 (patch-ish [SF#429391])
files
enforce correct encoding of PostgreSQL backend [SF#1374235] http://hg.code.sf.net:8000/p/roundup/code/#changeset-addbe69ff58b39c05d340fe1288c1ed870f9f627 Richard Jones richard@users.sourceforge.net 2006-01-20T02:27:12+00:00 2006-01-20T02:27:12+00:00
changeset addbe69ff58b
branch
bookmark
tag
user Richard Jones <richard@users.sourceforge.net>
description enforce correct encoding of PostgreSQL backend [SF#1374235]
files
merge from maint-0-7 http://hg.code.sf.net:8000/p/roundup/code/#changeset-3518d1ffd9409ece98c0d5fae5b489c0b920729a Richard Jones richard@users.sourceforge.net 2005-05-02T05:48:59+00:00 2005-05-02T05:48:59+00:00
changeset 3518d1ffd940
branch
bookmark
tag
user Richard Jones <richard@users.sourceforge.net>
description merge from maint-0-7
files
[maint-0.8] merge from maint-0-7 http://hg.code.sf.net:8000/p/roundup/code/#changeset-088c1792b7f62be0ef56fa33794e58bd9b7cfdd3 Richard Jones richard@users.sourceforge.net 2005-05-02T05:48:59+00:00 2005-05-02T05:48:59+00:00
changeset 088c1792b7f6
branch maint-0.8
bookmark
tag
user Richard Jones <richard@users.sourceforge.net>
description merge from maint-0-7
files
[maint-0.7] fix RDBMS clear() so it resets all class itemid counters http://hg.code.sf.net:8000/p/roundup/code/#changeset-c7c25f2103b23f0c8aa9b12411ce288ebd42f5cb Richard Jones richard@users.sourceforge.net 2005-05-02T05:44:56+00:00 2005-05-02T05:44:56+00:00
changeset c7c25f2103b2
branch maint-0.7
bookmark
tag 0.7.12
user Richard Jones <richard@users.sourceforge.net>
description fix RDBMS clear() so it resets all class itemid counters
files
[maint-0.7] fix RDBMS clear() so it resets all class itemid counters http://hg.code.sf.net:8000/p/roundup/code/#changeset-e074c641cb5eadb6fb224d609c280c108339d713 Richard Jones richard@users.sourceforge.net 2005-05-02T01:03:41+00:00 2005-05-02T01:03:41+00:00
changeset e074c641cb5e
branch maint-0.7
bookmark
tag
user Richard Jones <richard@users.sourceforge.net>
description fix RDBMS clear() so it resets all class itemid counters
files
[maint-0.8] merge from HEAD http://hg.code.sf.net:8000/p/roundup/code/#changeset-e1da7b5b04abf9880c0cb834dc868ffd59c556d1 Richard Jones richard@users.sourceforge.net 2005-02-14T02:55:31+00:00 2005-02-14T02:55:31+00:00
changeset e1da7b5b04ab
branch maint-0.8
bookmark
tag
user Richard Jones <richard@users.sourceforge.net>
description merge from HEAD
files
Python 2.3 minimum version - bye bye roundup.rlog, you had a short life. http://hg.code.sf.net:8000/p/roundup/code/#changeset-57b60bda9473826c2c97b92f46f8f4b18ccd8900 Richard Jones richard@users.sourceforge.net 2005-02-14T02:48:12+00:00 2005-02-14T02:48:12+00:00
changeset 57b60bda9473
branch
bookmark
tag
user Richard Jones <richard@users.sourceforge.net>
description Python 2.3 minimum version - bye bye roundup.rlog, you had a short life.
files
[maint-0.8] merge from HEAD http://hg.code.sf.net:8000/p/roundup/code/#changeset-406862712b7d9af45a995a066e06c630dfb50a39 Richard Jones richard@users.sourceforge.net 2005-01-13T05:05:12+00:00 2005-01-13T05:05:12+00:00
changeset 406862712b7d
branch maint-0.8
bookmark
tag
user Richard Jones <richard@users.sourceforge.net>
description merge from HEAD
files