Mercurial > p > roundup > code
annotate roundup/cgi/engine_jinja2.py @ 6593:e70e2789bc2c
issue2551189 - increase text search maxlength
This removes I think all the magic references to 25 and 30 (varchar
size) and replaces them with references to maxlength or maxlength+5.
I am not sure why the db column is 5 characters larger than the size
of what should be the max size of a word, but I'll keep the buffer
of 5 as making it 1/5 the size of maxlength makes less sense.
Also added tests for fts search in templating which were missing.
Added postgres, mysql and sqlite native indexing backends in which to
test fts. Added fts test to native-fts as well to make sure it's
working.
I want to commit this now for CI.
Todo:
add test cases for the use of FTS in the csv output in
actions.py. There is no test coverage of the match case there.
change maxlength to a higher value (50) as requested in the ticket.
Modify existing extremewords test cases to allow words > 25 and < 51
write code to migrate column sizes for mysql and postgresql to match
maxlength I will roll this into the version 7 schema update that
supports use of database fts support.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Tue, 25 Jan 2022 13:22:00 -0500 |
| parents | ec1f725f5c91 |
| children | 07ce4e4110f5 |
| rev | line source |
|---|---|
|
4743
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
1 """ |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
2 Experimental Jinja2 support for Roundup. It will become less |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
3 experimental when it is completely clear what information is |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
4 passed to template, and when the info is limited to the sane |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
5 minimal set (to avoid Roundup state changes from template). |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
6 |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
7 [ ] fallback mechanizm to use multiple templating engines in |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
8 parallel and aid in incremental translation from one |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
9 engine to another |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
10 |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
11 [ ] define a place for templates |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
12 probably |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
13 TRACKER_HOME/templates/jinja2 |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
14 with |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
15 TRACKER_HOME/templates/INFO.txt |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
16 describing how the dir was created, for example |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
17 "This is a copy of 'classic' template from ..." |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
18 also template fallback mechanizm for multi-engine |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
19 configuration |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
20 [ ] backward compatibility - if no engine is explicitly |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
21 specified, use TRACKER_HOME/html directory |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
22 [ ] copy TEMPLATES-INFO.txt to INFO.txt |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
23 [ ] implement VERSION file in environment for auto |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
24 upgrade |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
25 |
|
4748
e339583eae8e
templating: rename precompileTemplates to precompile
anatoly techtonik <techtonik@gmail.com>
parents:
4743
diff
changeset
|
26 [ ] precompile() is a stub |
|
4743
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
27 |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
28 [ ] add {{ debug() }} dumper to inspect available variables |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
29 https://github.com/mitsuhiko/jinja2/issues/174 |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
30 """ |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
31 |
|
5376
64b05e24dbd8
Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5015
diff
changeset
|
32 from __future__ import print_function |
|
4743
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
33 import jinja2 |
|
4751
a8c95abaf3fb
Work in progress - template based on jinja2, bootstrap and classic schema
Pradip Caulagi <caulagi@gmail.com>
parents:
4750
diff
changeset
|
34 import gettext |
|
5516
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
35 import mimetypes |
|
5449
ddf1cf299ebc
only encode output on Python 2
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5416
diff
changeset
|
36 import sys |
|
4743
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
37 |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
38 # http://jinja.pocoo.org/docs/api/#loaders |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
39 |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
40 from roundup.cgi.templating import context, LoaderBase, TemplateBase |
|
5416
56c9bcdea47f
Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5376
diff
changeset
|
41 from roundup.anypy.strings import s2u |
|
4743
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
42 |
|
6031
0a357d3b3557
Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents:
5516
diff
changeset
|
43 |
|
4743
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
44 class Jinja2Loader(LoaderBase): |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
45 def __init__(self, dir): |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
46 self._env = jinja2.Environment( |
|
5516
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
47 loader=jinja2.FileSystemLoader(dir), |
|
6054
ffaf89a4a9d9
use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents:
6032
diff
changeset
|
48 extensions=['jinja2.ext.i18n'], |
|
ffaf89a4a9d9
use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents:
6032
diff
changeset
|
49 autoescape=True |
|
5516
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
50 ) |
|
4743
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
51 |
|
5015
85484d35f1a2
issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents:
4772
diff
changeset
|
52 # Adding a custom filter that can transform roundup's vars to unicode |
|
85484d35f1a2
issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents:
4772
diff
changeset
|
53 # This is necessary because jinja2 can only deal with unicode objects |
|
85484d35f1a2
issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents:
4772
diff
changeset
|
54 # and roundup uses utf-8 for the internal representation. |
|
85484d35f1a2
issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents:
4772
diff
changeset
|
55 # The automatic conversion will assume 'ascii' and fail sometime. |
|
85484d35f1a2
issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents:
4772
diff
changeset
|
56 # Analysed with roundup 1.5.0 and jinja 2.7.1. See issue2550811. |
|
5516
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
57 self._env.filters["u"] = s2u |
|
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
58 |
|
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
59 def _find(self, tplname): |
|
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
60 for extension in ('', '.html', '.xml'): |
|
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
61 try: |
|
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
62 filename = tplname + extension |
|
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
63 return self._env.get_template(filename) |
|
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
64 except jinja2.TemplateNotFound: |
|
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
65 continue |
|
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
66 |
|
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
67 return None |
|
5015
85484d35f1a2
issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents:
4772
diff
changeset
|
68 |
|
4743
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
69 def check(self, tplname): |
|
5516
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
70 return bool(self._find(tplname)) |
|
4743
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
71 |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
72 def load(self, tplname): |
|
5516
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
73 tpl = self._find(tplname) |
|
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
74 pt = Jinja2ProxyPageTemplate(tpl) |
|
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
75 pt.content_type = mimetypes.guess_type(tpl.filename)[0] or 'text/html' |
|
91391a366d92
fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5449
diff
changeset
|
76 return pt |
|
4743
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
77 |
|
4748
e339583eae8e
templating: rename precompileTemplates to precompile
anatoly techtonik <techtonik@gmail.com>
parents:
4743
diff
changeset
|
78 def precompile(self): |
|
4743
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
79 pass |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
80 |
|
6031
0a357d3b3557
Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents:
5516
diff
changeset
|
81 |
|
4743
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
82 class Jinja2ProxyPageTemplate(TemplateBase): |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
83 def __init__(self, template): |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
84 self._tpl = template |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
85 |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
86 def render(self, client, classname, request, **options): |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
87 # [ ] limit the information passed to the minimal necessary set |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
88 c = context(client, self, classname, request) |
|
6054
ffaf89a4a9d9
use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents:
6032
diff
changeset
|
89 |
|
ffaf89a4a9d9
use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents:
6032
diff
changeset
|
90 c.update({'options': options, |
|
ffaf89a4a9d9
use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents:
6032
diff
changeset
|
91 'gettext': lambda s: s2u(client.gettext(s)), |
|
ffaf89a4a9d9
use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents:
6032
diff
changeset
|
92 'ngettext': lambda s, p, n: s2u(client.ngettext(s, p, n))}) |
|
5449
ddf1cf299ebc
only encode output on Python 2
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5416
diff
changeset
|
93 s = self._tpl.render(c) |
|
6031
0a357d3b3557
Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents:
5516
diff
changeset
|
94 return s if sys.version_info[0] > 2 else \ |
|
0a357d3b3557
Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents:
5516
diff
changeset
|
95 s.encode(client.STORAGE_CHARSET, ) |
|
4743
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
96 |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
97 def __getitem__(self, name): |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
98 # [ ] figure out what are these for |
|
6031
0a357d3b3557
Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents:
5516
diff
changeset
|
99 raise NotImplementedError |
|
6032
238dafbe521c
Flake8 couple of more whitespace changes.
John Rouillard <rouilj@ieee.org>
parents:
6031
diff
changeset
|
100 # return self._pt[name] |
|
4743
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
101 |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
102 def __getattr__(self, name): |
|
2d6959f1d2df
templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff
changeset
|
103 # [ ] figure out what are these for |
|
6031
0a357d3b3557
Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents:
5516
diff
changeset
|
104 raise NotImplementedError |
|
6032
238dafbe521c
Flake8 couple of more whitespace changes.
John Rouillard <rouilj@ieee.org>
parents:
6031
diff
changeset
|
105 # return getattr(self._pt, name) |
