annotate roundup/scripts/roundup_demo.py @ 5525:bb7865241f8a

Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975). The roundup-admin export and import commands are used for migrating between different database backends. It is desirable that they should be usable also for migrations between Python 2 and Python 3, and in some cases (e.g. with the anydbm backend) this may be required. To be usable for such migrations, the format of the generated CSV files needs to be stable, meaning the same as currently used with Python 2. The export process uses repr() to produce the fields in the CSV files and eval() to convert them back to Python data structures. repr() of strings with non-ASCII characters produces different results for Python 2 and Python 3. This patch adds repr_export and eval_import functions to roundup/anypy/strings.py which provide the required operations that are just repr() and eval() in Python 2, but are more complicated in Python 3 to use data representations compatible with Python 2. These functions are then used in the required places for export and import. repr() and eval() are also used in storing the dict of changed values in the journal for the RDBMS backends. It is similarly desirable that the database be compatible between Python 2 and Python 3, so that export and import do not need to be used for a migration between Python versions for non-anydbm back ends. Thus, this patch changes rdbms_common.py in the places involved in storing journals in the database, not just in those involved in import/export. Given this patch, import/export with non-ASCII characters appear based on some limited testing to work across Python versions, and an instance using the sqlite backend appears to be compatible between Python versions without needing import/export, *if* the sessions/otks databases (which use anydbm) are deleted when changing Python version.
author Joseph Myers <jsm@polyomino.org.uk>
date Sun, 02 Sep 2018 23:48:04 +0000
parents 4cf48ff01e04
children d09a1d6a3bd9
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
1 #! /usr/bin/env python
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
2 #
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
3 # Copyright 2004 Richard Jones (richard@mechanicalcat.net)
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
4 #
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
5
4766
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
6 DEFAULT_HOME = './demo'
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
7 DEFAULT_TEMPLATE = 'classic'
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
8
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
9
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
10 import sys
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
11
4766
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
12
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
13 # --- patch sys.path to make sure 'import roundup' finds correct version
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
14 import os.path as osp
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
15
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
16 thisdir = osp.dirname(osp.abspath(__file__))
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
17 rootdir = osp.dirname(osp.dirname(thisdir))
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
18 if (osp.exists(thisdir + '/__init__.py') and
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
19 osp.exists(rootdir + '/roundup/__init__.py')):
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
20 # the script is located inside roundup source code
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
21 sys.path.insert(0, rootdir)
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
22 # --/
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
23
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
24
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
25 from roundup import admin, configuration, demo, instance
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
26 from roundup.i18n import _
5401
4cf48ff01e04 Python 3 preparation: replace raw_input uses.
Joseph Myers <jsm@polyomino.org.uk>
parents: 4766
diff changeset
27 from roundup.anypy.my_input import my_input
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
28
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
29 def run():
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
30 home = DEFAULT_HOME
4108
01e8be352fb8 Allow user to choose the tracker template for demo.
Stefan Seefeld <stefan@seefeld.name>
parents: 2777
diff changeset
31 template = DEFAULT_TEMPLATE
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
32 nuke = sys.argv[-1] == 'nuke'
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
33 # if there is no tracker in home, force nuke
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
34 try:
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
35 instance.open(home)
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
36 except configuration.NoConfigError:
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
37 nuke = 1
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
38 # if we are to create the tracker, prompt for home
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
39 if nuke:
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
40 if len(sys.argv) > 2:
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
41 backend = sys.argv[-2]
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
42 else:
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
43 backend = 'anydbm'
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
44 # FIXME: i'd like to have an option to abort the tracker creation
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
45 # say, by entering a single dot. but i cannot think of
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
46 # appropriate prompt for that.
5401
4cf48ff01e04 Python 3 preparation: replace raw_input uses.
Joseph Myers <jsm@polyomino.org.uk>
parents: 4766
diff changeset
47 home = my_input(
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
48 _('Enter directory path to create demo tracker [%s]: ') % home)
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
49 if not home:
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
50 home = DEFAULT_HOME
4108
01e8be352fb8 Allow user to choose the tracker template for demo.
Stefan Seefeld <stefan@seefeld.name>
parents: 2777
diff changeset
51 templates = admin.AdminTool().listTemplates().keys()
5401
4cf48ff01e04 Python 3 preparation: replace raw_input uses.
Joseph Myers <jsm@polyomino.org.uk>
parents: 4766
diff changeset
52 template = my_input(
4108
01e8be352fb8 Allow user to choose the tracker template for demo.
Stefan Seefeld <stefan@seefeld.name>
parents: 2777
diff changeset
53 _('Enter tracker template to use (one of (%s)) [%s]: ') %
01e8be352fb8 Allow user to choose the tracker template for demo.
Stefan Seefeld <stefan@seefeld.name>
parents: 2777
diff changeset
54 (','.join(templates),template))
01e8be352fb8 Allow user to choose the tracker template for demo.
Stefan Seefeld <stefan@seefeld.name>
parents: 2777
diff changeset
55 if not template:
01e8be352fb8 Allow user to choose the tracker template for demo.
Stefan Seefeld <stefan@seefeld.name>
parents: 2777
diff changeset
56 template = DEFAULT_TEMPLATE
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
57 # install
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
58 demo.install_demo(home, backend,
4108
01e8be352fb8 Allow user to choose the tracker template for demo.
Stefan Seefeld <stefan@seefeld.name>
parents: 2777
diff changeset
59 admin.AdminTool().listTemplates()[template]['path'])
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
60 # run
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
61 demo.run_demo(home)
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
62
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
63 if __name__ == '__main__':
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
64 run()
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
65
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
66 # vim: set et sts=4 sw=4 :

Roundup Issue Tracker: http://roundup-tracker.org/