annotate roundup/cgi/TAL/README.txt @ 5525:bb7865241f8a

Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975). The roundup-admin export and import commands are used for migrating between different database backends. It is desirable that they should be usable also for migrations between Python 2 and Python 3, and in some cases (e.g. with the anydbm backend) this may be required. To be usable for such migrations, the format of the generated CSV files needs to be stable, meaning the same as currently used with Python 2. The export process uses repr() to produce the fields in the CSV files and eval() to convert them back to Python data structures. repr() of strings with non-ASCII characters produces different results for Python 2 and Python 3. This patch adds repr_export and eval_import functions to roundup/anypy/strings.py which provide the required operations that are just repr() and eval() in Python 2, but are more complicated in Python 3 to use data representations compatible with Python 2. These functions are then used in the required places for export and import. repr() and eval() are also used in storing the dict of changed values in the journal for the RDBMS backends. It is similarly desirable that the database be compatible between Python 2 and Python 3, so that export and import do not need to be used for a migration between Python versions for non-anydbm back ends. Thus, this patch changes rdbms_common.py in the places involved in storing journals in the database, not just in those involved in import/export. Given this patch, import/export with non-ASCII characters appear based on some limited testing to work across Python versions, and an instance using the sqlite backend appears to be compatible between Python versions without needing import/export, *if* the sessions/otks databases (which use anydbm) are deleted when changing Python version.
author Joseph Myers <jsm@polyomino.org.uk>
date Sun, 02 Sep 2018 23:48:04 +0000
parents b9988e118055
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1049
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 TAL - Template Attribute Language
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 ---------------------------------
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4 This is an implementation of TAL, the Zope Template Attribute
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
5 Language. For TAL, see the Zope Presentation Templates ZWiki:
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
6
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7 http://dev.zope.org/Wikis/DevSite/Projects/ZPT/FrontPage
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
8
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
9 It is not a Zope product nor is it designed exclusively to run inside
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
10 of Zope, but if you have a Zope checkout that includes
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
11 Products/ParsedXML, its Expat parser will be used.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
12
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
13 Prerequisites
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
14 -------------
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
15
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
16 You need:
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
17
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
18 - A recent checkout of Zope2; don't forget to run the wo_pcgi.py
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
19 script to compile everything. (See above -- this is now optional.)
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
21 - A recent checkout of the Zope2 product ParsedXML, accessible
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
22 throught <Zope2>/lib/python/Products/ParsedXML; don't forget to run
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
23 the setup.py script to compiles Expat. (Again, optional.)
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
24
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
25 - Python 1.5.2; the driver script refuses to work with other versions
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
26 unless you specify the -n option; this is done so that I don't
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
27 accidentally use Python 2.x features.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
28
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
29 - Create a .path file containing proper module search path; it should
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
30 point the <Zope2>/lib/python directory that you want to use.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
31
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
32 How To Play
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
33 -----------
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
34
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
35 (Don't forget to edit .path, see above!)
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
36
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
37 The script driver.py takes an XML file with TAL markup as argument and
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
38 writes the expanded version to standard output. The filename argument
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
39 defaults to tests/input/test01.xml.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
40
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
41 Regression test
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
42 ---------------
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
43
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
44 There are unit test suites in the 'tests' subdirectory; these can be
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
45 run with tests/run.py. This should print the testcase names plus
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
46 progress info, followed by a final line saying "OK". It requires that
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
47 ../unittest.py exists.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
48
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
49 There are a number of test files in the 'tests' subdirectory, named
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
50 tests/input/test<number>.xml and tests/input/test<number>.html. The
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
51 Python script ./runtest.py calls driver.main() for each test file, and
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
52 should print "<file> OK" for each one. These tests are also run as
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
53 part of the unit test suites, so tests/run.py is all you need.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
54
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
55 What's Here
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
56 -----------
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
57
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
58 DummyEngine.py simple-minded TALES execution engine
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
59 TALInterpreter.py class to interpret intermediate code
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
60 TALGenerator.py class to generate intermediate code
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
61 XMLParser.py base class to parse XML, avoiding DOM
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
62 TALParser.py class to parse XML with TAL into intermediate code
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
63 HTMLTALParser.py class to parse HTML with TAL into intermediate code
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
64 HTMLParser.py HTML-parsing base class
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
65 driver.py script to demonstrate TAL expansion
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
66 timer.py script to time various processing phases
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
67 setpath.py hack to set sys.path and import ZODB
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
68 __init__.py empty file that makes this directory a package
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
69 runtest.py Python script to run file-comparison tests
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
70 ndiff.py helper for runtest.py to produce diffs
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
71 tests/ drectory with test files and output
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
72 tests/run.py Python script to run all tests
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
73
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
74 Author and License
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
75 ------------------
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
76
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
77 This code is written by Guido van Rossum (project lead), Fred Drake,
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
78 and Tim Peters. It is owned by Digital Creations and can be
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
79 redistributed under the Zope Public License.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
80
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
81 TO DO
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
82 -----
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
83
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
84 (See also http://www.zope.org/Members/jim/ZPTIssueTracker .)
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
85
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
86 - Need to remove leading whitespace and newline when omitting an
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
87 element (either through tal:replace with a value of nothing or
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
88 tal:condition with a false condition).
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
89
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
90 - Empty TAL/METAL attributes are ignored: tal:replace="" is ignored
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
91 rather than causing an error.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
92
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
93 - HTMLTALParser.py and TALParser.py are silly names. Should be
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
94 HTMLTALCompiler.py and XMLTALCompiler.py (or maybe shortened,
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
95 without "TAL"?)
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
96
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
97 - Should we preserve case of tags and attribute names in HTML?

Roundup Issue Tracker: http://roundup-tracker.org/