annotate roundup/cgi/engine_jinja2.py @ 8411:ef1ea918b07a reauth-confirm_id

feat(security): Add user confirmation/reauth for sensitive changes Auditors can raise Reauth(reason) exception to require the user to enter a token (e.g. account password) to verify the user is performing the change. Naming is subject to change. actions.py: New ReauthAction class handler and verifyPassword() method for overriding if needed. client.py: Handle Reauth exception by calling Client:reauth() method. Default client:reauth method. Add 'reauth' action declaration. exceptions.py: Define and document Reauth exception as a subclass of RoundupCGIException. templating.py: Define method utils.embed_form_fields(). The original form making a change to the database has a lot of form fields. These need to be resubmitted to Roundup as part of the form submission that verifies the user's password. This method turns all non file form fields into type=hidden inputs. It escapes the names and values to prevent XSS. For file form fields, it base64 encodes the contents and puts them in hidden pre blocks. The pre blocks have data attributes for the filename, filetype and the original field name. (Note the original field name is not used.) This stops the file content data (maybe binary e.g. jpegs) from breaking the html page. The reauth template runs JavaScript that turns the encoded data inside the pre tags back into a file. Then it adds a multiple file input control to the page and attaches all the files to it. This file input is submitted with the rest of the fields. _generic.reauth.html (multiple tracker templates): Generates a form with id=reauth_form to: display any message from the Reauth exception to the user (e.g. why user is asked to auth). get the user's password submit the form embed all the form data that triggered the reauth recreate any file data that was submitted as part of the form and generate a new file input to push the data to the back end It has the JavaScript routine (as an IIFE) that regenerates a file input without user intervention. All the TAL based tracker templates use the same form. There is also one for the jinja2 template. The JavaScript for both is the same. reference.txt: document embed_form_fields utility method. upgrading.txt: initial upgrading docs. TODO: Finalize naming. I am leaning toward ConfirmID rather than Reauth. Still looking for a standard name for this workflow. Externalize the javascript in _generic.reauth.html to a seperate file and use utils.readfile() to embed it or change the script to load it from a @@file url. Clean up upgrading.txt with just steps to implement and less feature detail/internals. Document internals/troubleshooting in reference.txt. Add tests using live server.
author John Rouillard <rouilj@ieee.org>
date Mon, 11 Aug 2025 14:01:12 -0400
parents b8e63e65d9a8
children 9c3ec0a5c7fc
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
1 """
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
2 Experimental Jinja2 support for Roundup. It will become less
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
3 experimental when it is completely clear what information is
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
4 passed to template, and when the info is limited to the sane
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
5 minimal set (to avoid Roundup state changes from template).
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
6
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
7 [ ] fallback mechanizm to use multiple templating engines in
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
8 parallel and aid in incremental translation from one
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
9 engine to another
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
10
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
11 [ ] define a place for templates
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
12 probably
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
13 TRACKER_HOME/templates/jinja2
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
14 with
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
15 TRACKER_HOME/templates/INFO.txt
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
16 describing how the dir was created, for example
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
17 "This is a copy of 'classic' template from ..."
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
18 also template fallback mechanizm for multi-engine
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
19 configuration
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
20 [ ] backward compatibility - if no engine is explicitly
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
21 specified, use TRACKER_HOME/html directory
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
22 [ ] copy TEMPLATES-INFO.txt to INFO.txt
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
23 [ ] implement VERSION file in environment for auto
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
24 upgrade
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
25
4748
e339583eae8e templating: rename precompileTemplates to precompile
anatoly techtonik <techtonik@gmail.com>
parents: 4743
diff changeset
26 [ ] precompile() is a stub
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
27
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
28 [ ] add {{ debug() }} dumper to inspect available variables
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
29 https://github.com/mitsuhiko/jinja2/issues/174
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
30 """
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
31
5376
64b05e24dbd8 Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5015
diff changeset
32 from __future__ import print_function
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
33 import jinja2
5516
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
34 import mimetypes
5449
ddf1cf299ebc only encode output on Python 2
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5416
diff changeset
35 import sys
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
36
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
37 # http://jinja.pocoo.org/docs/api/#loaders
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
38
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
39 from roundup.cgi.templating import context, LoaderBase, TemplateBase
5416
56c9bcdea47f Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5376
diff changeset
40 from roundup.anypy.strings import s2u
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
41
6031
0a357d3b3557 Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents: 5516
diff changeset
42
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
43 class Jinja2Loader(LoaderBase):
7775
b8e63e65d9a8 chore: replace use of dir with template_dir.
John Rouillard <rouilj@ieee.org>
parents: 7228
diff changeset
44 def __init__(self, template_dir):
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
45 self._env = jinja2.Environment(
7775
b8e63e65d9a8 chore: replace use of dir with template_dir.
John Rouillard <rouilj@ieee.org>
parents: 7228
diff changeset
46 loader=jinja2.FileSystemLoader(template_dir),
6054
ffaf89a4a9d9 use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents: 6032
diff changeset
47 extensions=['jinja2.ext.i18n'],
ffaf89a4a9d9 use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents: 6032
diff changeset
48 autoescape=True
5516
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
49 )
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
50
5015
85484d35f1a2 issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents: 4772
diff changeset
51 # Adding a custom filter that can transform roundup's vars to unicode
85484d35f1a2 issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents: 4772
diff changeset
52 # This is necessary because jinja2 can only deal with unicode objects
85484d35f1a2 issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents: 4772
diff changeset
53 # and roundup uses utf-8 for the internal representation.
85484d35f1a2 issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents: 4772
diff changeset
54 # The automatic conversion will assume 'ascii' and fail sometime.
85484d35f1a2 issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents: 4772
diff changeset
55 # Analysed with roundup 1.5.0 and jinja 2.7.1. See issue2550811.
5516
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
56 self._env.filters["u"] = s2u
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
57
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
58 def _find(self, tplname):
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
59 for extension in ('', '.html', '.xml'):
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
60 try:
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
61 filename = tplname + extension
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
62 return self._env.get_template(filename)
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
63 except jinja2.TemplateNotFound:
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
64 continue
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
65
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
66 return None
5015
85484d35f1a2 issue2550811 20% fix: jinja2 template engine now has an example how to use non-ascii unicode contents with a custom filter ('|u').
Bernhard Reiter <bernhard@intevation.de>
parents: 4772
diff changeset
67
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
68 def check(self, tplname):
5516
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
69 return bool(self._find(tplname))
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
70
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
71 def load(self, tplname):
5516
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
72 tpl = self._find(tplname)
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
73 pt = Jinja2ProxyPageTemplate(tpl)
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
74 pt.content_type = mimetypes.guess_type(tpl.filename)[0] or 'text/html'
91391a366d92 fixed issue2550967: support .xml files in addition to .html in the
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5449
diff changeset
75 return pt
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
76
4748
e339583eae8e templating: rename precompileTemplates to precompile
anatoly techtonik <techtonik@gmail.com>
parents: 4743
diff changeset
77 def precompile(self):
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
78 pass
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
79
6031
0a357d3b3557 Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents: 5516
diff changeset
80
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
81 class Jinja2ProxyPageTemplate(TemplateBase):
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
82 def __init__(self, template):
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
83 self._tpl = template
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
84
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
85 def render(self, client, classname, request, **options):
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
86 # [ ] limit the information passed to the minimal necessary set
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
87 c = context(client, self, classname, request)
6054
ffaf89a4a9d9 use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents: 6032
diff changeset
88
ffaf89a4a9d9 use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents: 6032
diff changeset
89 c.update({'options': options,
ffaf89a4a9d9 use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents: 6032
diff changeset
90 'gettext': lambda s: s2u(client.gettext(s)),
ffaf89a4a9d9 use jinja2 i18n extension
Christof Meerwald <cmeerw@cmeerw.org>
parents: 6032
diff changeset
91 'ngettext': lambda s, p, n: s2u(client.ngettext(s, p, n))})
5449
ddf1cf299ebc only encode output on Python 2
Christof Meerwald <cmeerw@cmeerw.org>
parents: 5416
diff changeset
92 s = self._tpl.render(c)
6031
0a357d3b3557 Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents: 5516
diff changeset
93 return s if sys.version_info[0] > 2 else \
0a357d3b3557 Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents: 5516
diff changeset
94 s.encode(client.STORAGE_CHARSET, )
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
95
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
96 def __getitem__(self, name):
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
97 # [ ] figure out what are these for
6031
0a357d3b3557 Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents: 5516
diff changeset
98 raise NotImplementedError
6032
238dafbe521c Flake8 couple of more whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6031
diff changeset
99 # return self._pt[name]
4743
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
100
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
101 def __getattr__(self, name):
2d6959f1d2df templating: proof of concept for Jinja2 support. Select 'jinja2'
anatoly techtonik <techtonik@gmail.com>
parents:
diff changeset
102 # [ ] figure out what are these for
6031
0a357d3b3557 Flake8 fixes: Raise NotImplementedError not NotImplemented; whitespace
John Rouillard <rouilj@ieee.org>
parents: 5516
diff changeset
103 raise NotImplementedError
6032
238dafbe521c Flake8 couple of more whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6031
diff changeset
104 # return getattr(self._pt, name)

Roundup Issue Tracker: http://roundup-tracker.org/