annotate roundup/scripts/roundup_demo.py @ 6433:c1d3fbcdbfbd

issue2551142 - Import of retired node ... unique constraint failure. Title: Import of retired node with username after active node fails with unique constraint failure. More fixes needed for mysql and postgresql. mysql: add unique constraint for (keyvalue, __retired__) when creating class in the database. On schema change if class is changed, remove the unique constraint too. upgrade version of rdbms database from 5 to 6 to add constraint to all version 5 databases that were created as version 5 and didn't get the unique constraint. Make no changes on version 5 databases upgraded from version 4, the upgrade process to 5 added the constraint. Make no changes to other databases (sqlite, postgres) during upgrade from version 5 to 6. postgres: Handle the exception raised on unique constraint violation. The exception invalidates the database connection so it can't be used to recover from the exception. Added two new database methods: checkpoint_data - performs a db.commit under postgres does nothing on other backends restore_connection_on_error - does a db.rollback on postgres, does nothing on other backends with the rollback() done on the connection I can use the database connection to fixup the import that failed on the unique constraint. This makes postgres slower but without the commit after every imported object, the rollback will delete all the entries done up to this point. Trying to figure out how to make the caller do_import batch and recover from this failure is beyond me. Also dismissed having to process the export csv file before importing. Pushing that onto a user just seems wrong. Also since import/export isn't frequently done the lack of surprise on having a failing import and reduced load/frustration for the user seems worth it. Also the import can be run in verbose mode where it prints out a row as it is processed, so it may take a while, ut the user can get feedback. db_test-base.py: add test for upgrade from 5 to 6.
author John Rouillard <rouilj@ieee.org>
date Thu, 10 Jun 2021 12:52:05 -0400
parents d09a1d6a3bd9
children 062a54eeb0a1
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
1 #! /usr/bin/env python
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
2 #
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
3 # Copyright 2004 Richard Jones (richard@mechanicalcat.net)
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
4 #
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
5
4766
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
6 DEFAULT_HOME = './demo'
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
7 DEFAULT_TEMPLATE = 'classic'
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
8
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
9
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
10 import sys
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
11
4766
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
12
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
13 # --- patch sys.path to make sure 'import roundup' finds correct version
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
14 import os.path as osp
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
15
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
16 thisdir = osp.dirname(osp.abspath(__file__))
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
17 rootdir = osp.dirname(osp.dirname(thisdir))
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
18 if (osp.exists(thisdir + '/__init__.py') and
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
19 osp.exists(rootdir + '/roundup/__init__.py')):
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
20 # the script is located inside roundup source code
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
21 sys.path.insert(0, rootdir)
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
22 # --/
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
23
86ef4ab17dc5 Run scripts (roundup_admin.py, ...) directly from checkout.
anatoly techtonik <techtonik@gmail.com>
parents: 4108
diff changeset
24
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
25 from roundup import admin, configuration, demo, instance
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
26 from roundup.i18n import _
5401
4cf48ff01e04 Python 3 preparation: replace raw_input uses.
Joseph Myers <jsm@polyomino.org.uk>
parents: 4766
diff changeset
27 from roundup.anypy.my_input import my_input
6042
d09a1d6a3bd9 flake8 whitespace fixes; add version_check
John Rouillard <rouilj@ieee.org>
parents: 5401
diff changeset
28 from roundup import version_check
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
29
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
30 def run():
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
31 home = DEFAULT_HOME
4108
01e8be352fb8 Allow user to choose the tracker template for demo.
Stefan Seefeld <stefan@seefeld.name>
parents: 2777
diff changeset
32 template = DEFAULT_TEMPLATE
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
33 nuke = sys.argv[-1] == 'nuke'
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
34 # if there is no tracker in home, force nuke
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
35 try:
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
36 instance.open(home)
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
37 except configuration.NoConfigError:
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
38 nuke = 1
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
39 # if we are to create the tracker, prompt for home
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
40 if nuke:
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
41 if len(sys.argv) > 2:
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
42 backend = sys.argv[-2]
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
43 else:
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
44 backend = 'anydbm'
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
45 # FIXME: i'd like to have an option to abort the tracker creation
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
46 # say, by entering a single dot. but i cannot think of
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
47 # appropriate prompt for that.
5401
4cf48ff01e04 Python 3 preparation: replace raw_input uses.
Joseph Myers <jsm@polyomino.org.uk>
parents: 4766
diff changeset
48 home = my_input(
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
49 _('Enter directory path to create demo tracker [%s]: ') % home)
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
50 if not home:
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
51 home = DEFAULT_HOME
4108
01e8be352fb8 Allow user to choose the tracker template for demo.
Stefan Seefeld <stefan@seefeld.name>
parents: 2777
diff changeset
52 templates = admin.AdminTool().listTemplates().keys()
5401
4cf48ff01e04 Python 3 preparation: replace raw_input uses.
Joseph Myers <jsm@polyomino.org.uk>
parents: 4766
diff changeset
53 template = my_input(
4108
01e8be352fb8 Allow user to choose the tracker template for demo.
Stefan Seefeld <stefan@seefeld.name>
parents: 2777
diff changeset
54 _('Enter tracker template to use (one of (%s)) [%s]: ') %
6042
d09a1d6a3bd9 flake8 whitespace fixes; add version_check
John Rouillard <rouilj@ieee.org>
parents: 5401
diff changeset
55 (','.join(templates), template))
4108
01e8be352fb8 Allow user to choose the tracker template for demo.
Stefan Seefeld <stefan@seefeld.name>
parents: 2777
diff changeset
56 if not template:
01e8be352fb8 Allow user to choose the tracker template for demo.
Stefan Seefeld <stefan@seefeld.name>
parents: 2777
diff changeset
57 template = DEFAULT_TEMPLATE
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
58 # install
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
59 demo.install_demo(home, backend,
6042
d09a1d6a3bd9 flake8 whitespace fixes; add version_check
John Rouillard <rouilj@ieee.org>
parents: 5401
diff changeset
60 admin.AdminTool().listTemplates()[template]['path'])
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
61 # run
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
62 demo.run_demo(home)
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
63
6042
d09a1d6a3bd9 flake8 whitespace fixes; add version_check
John Rouillard <rouilj@ieee.org>
parents: 5401
diff changeset
64
2777
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
65 if __name__ == '__main__':
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
66 run()
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
67
baaf90070dc4 instant-gratification script for binary distributions
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
68 # vim: set et sts=4 sw=4 :

Roundup Issue Tracker: http://roundup-tracker.org/