annotate doc/postgresql.txt @ 6433:c1d3fbcdbfbd

issue2551142 - Import of retired node ... unique constraint failure. Title: Import of retired node with username after active node fails with unique constraint failure. More fixes needed for mysql and postgresql. mysql: add unique constraint for (keyvalue, __retired__) when creating class in the database. On schema change if class is changed, remove the unique constraint too. upgrade version of rdbms database from 5 to 6 to add constraint to all version 5 databases that were created as version 5 and didn't get the unique constraint. Make no changes on version 5 databases upgraded from version 4, the upgrade process to 5 added the constraint. Make no changes to other databases (sqlite, postgres) during upgrade from version 5 to 6. postgres: Handle the exception raised on unique constraint violation. The exception invalidates the database connection so it can't be used to recover from the exception. Added two new database methods: checkpoint_data - performs a db.commit under postgres does nothing on other backends restore_connection_on_error - does a db.rollback on postgres, does nothing on other backends with the rollback() done on the connection I can use the database connection to fixup the import that failed on the unique constraint. This makes postgres slower but without the commit after every imported object, the rollback will delete all the entries done up to this point. Trying to figure out how to make the caller do_import batch and recover from this failure is beyond me. Also dismissed having to process the export csv file before importing. Pushing that onto a user just seems wrong. Also since import/export isn't frequently done the lack of surprise on having a failing import and reduced load/frustration for the user seems worth it. Also the import can be run in verbose mode where it prints out a row as it is processed, so it may take a while, ut the user can get feedback. db_test-base.py: add test for upgrade from 5 to 6.
author John Rouillard <rouilj@ieee.org>
date Thu, 10 Jun 2021 12:52:05 -0400
parents ba82fb74f806
children a0a100453eb9
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6167
81ae33038ec5 more index entries.
John Rouillard <rouilj@ieee.org>
parents: 6163
diff changeset
1 .. index:: postgres; deployment notes
6163
c2fd254c9257 Doc updates/indexing.
John Rouillard <rouilj@ieee.org>
parents: 5756
diff changeset
2
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3 ==========================
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4 PostgreSQL/psycopg Backend
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
5 ==========================
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
6
3059
213468d6e7c9 Add installation documentation for tsearch2.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2860
diff changeset
7 This are notes about PostqreSQL backend based on the psycopg adapter for
213468d6e7c9 Add installation documentation for tsearch2.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2860
diff changeset
8 Roundup issue tracker.
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
9
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
10 Prerequisites
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
11 =============
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
12
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
13 To use PostgreSQL as backend for storing roundup data, you should
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
14 additionally install:
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
15
5756
e48b039b0ec0 issue2550966: fix suboptimal links in docs.
John Rouillard <rouilj@ieee.org>
parents: 4718
diff changeset
16 1. PostgreSQL 8.x or higher - https://www.postgresql.org/
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
17
1875
3ffb7b56172a *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
18 2. The psycopg python interface to PostgreSQL:
3666
8304a1c1cc98 encourage use of latest psycopg
Richard Jones <richard@users.sourceforge.net>
parents: 3500
diff changeset
19
6262
ba82fb74f806 Update psycopg link to: https://www.psycopg.org/
John Rouillard <rouilj@ieee.org>
parents: 6167
diff changeset
20 https://www.psycopg.org/
3666
8304a1c1cc98 encourage use of latest psycopg
Richard Jones <richard@users.sourceforge.net>
parents: 3500
diff changeset
21
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
22
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1875
diff changeset
23 Running the PostgreSQL unit tests
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1875
diff changeset
24 =================================
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1875
diff changeset
25
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1875
diff changeset
26 The user that you're running the tests as will need to be able to access
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1875
diff changeset
27 the postgresql database on the local machine and create and drop
4513
6a32a2fb95b4 Docs update for mysql and postgresl: Mentioned test/db_test_base.py because
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4512
diff changeset
28 databases. See the config values in 'test/db_test_base.py'
6a32a2fb95b4 Docs update for mysql and postgresl: Mentioned test/db_test_base.py because
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4512
diff changeset
29 about which database connection, name and user will be used.
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1875
diff changeset
30
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1875
diff changeset
31
2594
c0d6d5004464 rdbms doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
32 Credit
c0d6d5004464 rdbms doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
33 ======
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
34
2594
c0d6d5004464 rdbms doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
35 The postgresql backend was originally submitted by Federico Di Gregorio
c0d6d5004464 rdbms doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
36 <fog@initd.org>
c0d6d5004464 rdbms doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
37

Roundup Issue Tracker: http://roundup-tracker.org/