annotate doc/postgresql.txt @ 5525:bb7865241f8a

Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975). The roundup-admin export and import commands are used for migrating between different database backends. It is desirable that they should be usable also for migrations between Python 2 and Python 3, and in some cases (e.g. with the anydbm backend) this may be required. To be usable for such migrations, the format of the generated CSV files needs to be stable, meaning the same as currently used with Python 2. The export process uses repr() to produce the fields in the CSV files and eval() to convert them back to Python data structures. repr() of strings with non-ASCII characters produces different results for Python 2 and Python 3. This patch adds repr_export and eval_import functions to roundup/anypy/strings.py which provide the required operations that are just repr() and eval() in Python 2, but are more complicated in Python 3 to use data representations compatible with Python 2. These functions are then used in the required places for export and import. repr() and eval() are also used in storing the dict of changed values in the journal for the RDBMS backends. It is similarly desirable that the database be compatible between Python 2 and Python 3, so that export and import do not need to be used for a migration between Python versions for non-anydbm back ends. Thus, this patch changes rdbms_common.py in the places involved in storing journals in the database, not just in those involved in import/export. Given this patch, import/export with non-ASCII characters appear based on some limited testing to work across Python versions, and an instance using the sqlite backend appears to be compatible between Python versions without needing import/export, *if* the sessions/otks databases (which use anydbm) are deleted when changing Python version.
author Joseph Myers <jsm@polyomino.org.uk>
date Sun, 02 Sep 2018 23:48:04 +0000
parents 76b71c9bd50d
children e48b039b0ec0
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 ==========================
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 PostgreSQL/psycopg Backend
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3 ==========================
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4
3059
213468d6e7c9 Add installation documentation for tsearch2.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2860
diff changeset
5 This are notes about PostqreSQL backend based on the psycopg adapter for
213468d6e7c9 Add installation documentation for tsearch2.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2860
diff changeset
6 Roundup issue tracker.
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
8 Prerequisites
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
9 =============
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
10
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
11 To use PostgreSQL as backend for storing roundup data, you should
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
12 additionally install:
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
13
4718
76b71c9bd50d Updated recommended version infos for psycopg and PostgreSQL.
Bernhard Reiter <bernhard@intevation.de>
parents: 4513
diff changeset
14 1. PostgreSQL 8.x or higher - http://www.postgresql.org/
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
15
1875
3ffb7b56172a *** empty log message ***
Richard Jones <richard@users.sourceforge.net>
parents: 1873
diff changeset
16 2. The psycopg python interface to PostgreSQL:
3666
8304a1c1cc98 encourage use of latest psycopg
Richard Jones <richard@users.sourceforge.net>
parents: 3500
diff changeset
17
8304a1c1cc98 encourage use of latest psycopg
Richard Jones <richard@users.sourceforge.net>
parents: 3500
diff changeset
18 http://initd.org/software/initd/psycopg
8304a1c1cc98 encourage use of latest psycopg
Richard Jones <richard@users.sourceforge.net>
parents: 3500
diff changeset
19
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1875
diff changeset
21 Running the PostgreSQL unit tests
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1875
diff changeset
22 =================================
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1875
diff changeset
23
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1875
diff changeset
24 The user that you're running the tests as will need to be able to access
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1875
diff changeset
25 the postgresql database on the local machine and create and drop
4513
6a32a2fb95b4 Docs update for mysql and postgresl: Mentioned test/db_test_base.py because
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4512
diff changeset
26 databases. See the config values in 'test/db_test_base.py'
6a32a2fb95b4 Docs update for mysql and postgresl: Mentioned test/db_test_base.py because
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4512
diff changeset
27 about which database connection, name and user will be used.
1906
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1875
diff changeset
28
f255363e6d97 PostgreSQL backend lands.
Richard Jones <richard@users.sourceforge.net>
parents: 1875
diff changeset
29
2594
c0d6d5004464 rdbms doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
30 Credit
c0d6d5004464 rdbms doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
31 ======
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
32
2594
c0d6d5004464 rdbms doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
33 The postgresql backend was originally submitted by Federico Di Gregorio
c0d6d5004464 rdbms doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
34 <fog@initd.org>
c0d6d5004464 rdbms doc updates
Richard Jones <richard@users.sourceforge.net>
parents: 1906
diff changeset
35

Roundup Issue Tracker: http://roundup-tracker.org/