Mercurial > p > roundup > code

annotate roundup/cgi/engine_jinja2.py @ 8575:b1024bf0d9f7

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
feature: add nonceless/tokenless CSRF protection Add tokenless CSRF protection following: https://words.filippo.io/csrf/ Must be enabled using use_tokenless_csrf_protection in config.ini. By default it's off. If enabled the older csrf_* settings are ignored. The allowed_api_origins setting is still used for Origin comparisons. This should also improve performance as a nonce isn't required so generating random nonce and saving it to the otks database is eliminated. doc/admin_guide.txt, doc/reference.txt doc/upgrading.txt doc updates. roundup/configuration.py add use_tokenless_csrf_protection setting. move allowed_api_origins directly after use_tokenless_csrf_protection and before the older csrf_* settings. It's used by both of them. Rewrite description of allowed_api_origins as its applied to all URLs with tokenless protection, not just API URLs. roundup/anypy/urllib_.py import urlsplit, it is used in new code. urlparse() is less efficient and splits params out of the path component. Since Roundup doesn't require that params be split from the path. I expect future patch will replace urlparse() with urlsplit() globally and not need urlparse(). roundup/cgi/client.py add handle_csrf_tokenless() and call from handle_csrf() if use_tokenless_csrf_protection is enabled. refactor code that expires csrf tokens when used with the wrong methods (i.e. GET) into expire_exposed_keys(). Call same from handle_csrf and handle_csrf_tokenless. Also improve logging if this happens including both Referer and Origin headers if available. Arguably we dont care about CSRF tokens exposed via GET/HEAD/OPTIONS in the tokenless case, but this cleans them up in case the admin has to switch back. At some future date we can delete all the nonce based CSRF from 2018. Update handle_csrf() docstring about calling/returning handle_csrf_tokenless() when enabled. Call expire_exposed_keys(method) if token is supplied with wrong method. roundup/cgi/templating.py disable nonce generation/save and always return "0" when use_tokenless_csrf_protection enabled.
author John Rouillard <rouilj@ieee.org>
date Sun, 19 Apr 2026 20:50:07 -0400
parents 9c3ec0a5c7fc
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
1 """
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
2 Experimental Jinja2 support for Roundup. It will become less
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
3 experimental when it is completely clear what information is
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
4 passed to template, and when the info is limited to the sane
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
5 minimal set (to avoid Roundup state changes from template).
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
6
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
7 [ ] fallback mechanizm to use multiple templating engines in
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
8 parallel and aid in incremental translation from one
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
9 engine to another
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
10
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
11 [ ] define a place for templates
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
12 probably
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
13 TRACKER_HOME/templates/jinja2
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
14 with
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
15 TRACKER_HOME/templates/INFO.txt
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
16 describing how the dir was created, for example
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
17 "This is a copy of 'classic' template from ..."
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
18 also template fallback mechanizm for multi-engine
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
19 configuration
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
20 [ ] backward compatibility - if no engine is explicitly
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
21 specified, use TRACKER_HOME/html directory
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
22 [ ] copy TEMPLATES-INFO.txt to INFO.txt
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
23 [ ] implement VERSION file in environment for auto
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
24 upgrade
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
25
4748
e339583eae8e templating: rename precompileTemplates to precompile
anatoly techtonik <techtonik@gmail.com>
parents: 4743
diff changeset
26 [ ] precompile() is a stub
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
27
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
28 [ ] add {{ debug() }} dumper to inspect available variables
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
29 https://github.com/mitsuhiko/jinja2/issues/174
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
30 """
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
31
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
32 import jinja2
5516
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
33 import mimetypes
5449
ddf1cf299ebc only encode output on Python 2
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5416
diff changeset
34 import sys
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
35
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
36 # http://jinja.pocoo.org/docs/api/#loaders
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
37
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
38 from roundup.cgi.templating import context, LoaderBase, TemplateBase
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5376
diff changeset
39 from roundup.anypy.strings import s2u
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
40
6031
0a357d3b3557 Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents: 5516
diff changeset
41
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
42 class Jinja2Loader(LoaderBase):
7775
b8e63e65d9a8 chore: replace use of dir with template_dir.
John Rouillard <rouilj@ieee.org>
parents: 7228
diff changeset
43 def __init__(self, template_dir):
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
44 self._env = jinja2.Environment(
7775
b8e63e65d9a8 chore: replace use of dir with template_dir.
John Rouillard <rouilj@ieee.org>
parents: 7228
diff changeset
45 loader=jinja2.FileSystemLoader(template_dir),
6054
ffaf89a4a9d9 use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents: 6032
diff changeset
46 extensions=['jinja2.ext.i18n'],
ffaf89a4a9d9 use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents: 6032
diff changeset
47 autoescape=True
5516
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
48 )
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
49
5015
85484d35f1a2 issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents: 4772
diff changeset
50 # Adding a custom filter that can transform roundup's vars to unicode
85484d35f1a2 issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents: 4772
diff changeset
51 # This is necessary because jinja2 can only deal with unicode objects
85484d35f1a2 issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents: 4772
diff changeset
52 # and roundup uses utf-8 for the internal representation.
85484d35f1a2 issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents: 4772
diff changeset
53 # The automatic conversion will assume 'ascii' and fail sometime.
85484d35f1a2 issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents: 4772
diff changeset
54 # Analysed with roundup 1.5.0 and jinja 2.7.1. See issue2550811.
5516
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
55 self._env.filters["u"] = s2u
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
56
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
57 def _find(self, tplname):
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
58 for extension in ('', '.html', '.xml'):
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
59 try:
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
60 filename = tplname + extension
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
61 return self._env.get_template(filename)
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
62 except jinja2.TemplateNotFound:
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
63 continue
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
64
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
65 return None
5015
85484d35f1a2 issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents: 4772
diff changeset
66
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
67 def check(self, tplname):
5516
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
68 return bool(self._find(tplname))
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
69
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
70 def load(self, tplname):
5516
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
71 tpl = self._find(tplname)
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
72 pt = Jinja2ProxyPageTemplate(tpl)
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
73 pt.content_type = mimetypes.guess_type(tpl.filename)[0] or 'text/html'
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
74 return pt
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
75
4748
e339583eae8e templating: rename precompileTemplates to precompile
anatoly techtonik <techtonik@gmail.com>
parents: 4743
diff changeset
76 def precompile(self):
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
77 pass
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
78
6031
0a357d3b3557 Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents: 5516
diff changeset
79
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
80 class Jinja2ProxyPageTemplate(TemplateBase):
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
81 def __init__(self, template):
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
82 self._tpl = template
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
83
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
84 def render(self, client, classname, request, **options):
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
85 # [ ] limit the information passed to the minimal necessary set
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
86 c = context(client, self, classname, request)
6054
ffaf89a4a9d9 use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents: 6032
diff changeset
87
ffaf89a4a9d9 use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents: 6032
diff changeset
88 c.update({'options': options,
ffaf89a4a9d9 use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents: 6032
diff changeset
89 'gettext': lambda s: s2u(client.gettext(s)),
ffaf89a4a9d9 use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents: 6032
diff changeset
90 'ngettext': lambda s, p, n: s2u(client.ngettext(s, p, n))})
5449
ddf1cf299ebc only encode output on Python 2
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5416
diff changeset
91 s = self._tpl.render(c)
6031
0a357d3b3557 Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents: 5516
diff changeset
92 return s if sys.version_info[0] > 2 else \
0a357d3b3557 Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents: 5516
diff changeset
93 s.encode(client.STORAGE_CHARSET, )
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
94
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
95 def __getitem__(self, name):
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
96 # [ ] figure out what are these for
6031
0a357d3b3557 Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents: 5516
diff changeset
97 raise NotImplementedError
6032
238dafbe521c Flake8 couple of more whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6031
diff changeset
98 # return self._pt[name]
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
99
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
100 def __getattr__(self, name):
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
101 # [ ] figure out what are these for
6031
0a357d3b3557 Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents: 5516
diff changeset
102 raise NotImplementedError
6032
238dafbe521c Flake8 couple of more whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6031
diff changeset
103 # return getattr(self._pt, name)
Roundup Issue Tracker: http://roundup-tracker.org/