Mercurial > p > roundup > code
annotate roundup/cgi/TAL/talgettext.py @ 8575:b1024bf0d9f7
feature: add nonceless/tokenless CSRF protection
Add tokenless CSRF protection following:
https://words.filippo.io/csrf/
Must be enabled using use_tokenless_csrf_protection in config.ini. By
default it's off. If enabled the older csrf_* settings are ignored.
The allowed_api_origins setting is still used for Origin comparisons.
This should also improve performance as a nonce isn't required so
generating random nonce and saving it to the otks database is
eliminated.
doc/admin_guide.txt, doc/reference.txt doc/upgrading.txt
doc updates.
roundup/configuration.py
add use_tokenless_csrf_protection setting.
move allowed_api_origins directly after
use_tokenless_csrf_protection and before the older csrf_* settings.
It's used by both of them.
Rewrite description of allowed_api_origins as its applied to all
URLs with tokenless protection, not just API URLs.
roundup/anypy/urllib_.py
import urlsplit, it is used in new code.
urlparse() is less efficient and splits params out of the path
component.
Since Roundup doesn't require that params be split from the path. I
expect future patch will replace urlparse() with urlsplit() globally
and not need urlparse().
roundup/cgi/client.py
add handle_csrf_tokenless() and call from handle_csrf() if
use_tokenless_csrf_protection is enabled.
refactor code that expires csrf tokens when used with the wrong
methods (i.e. GET) into expire_exposed_keys(). Call same from
handle_csrf and handle_csrf_tokenless. Also improve logging if this
happens including both Referer and Origin headers if available.
Arguably we dont care about CSRF tokens exposed via
GET/HEAD/OPTIONS in the tokenless case, but this cleans them up in
case the admin has to switch back. At some future date we can
delete all the nonce based CSRF from 2018.
Update handle_csrf() docstring about calling/returning
handle_csrf_tokenless() when enabled. Call
expire_exposed_keys(method) if token is supplied with wrong method.
roundup/cgi/templating.py
disable nonce generation/save and always return "0" when
use_tokenless_csrf_protection enabled.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Sun, 19 Apr 2026 20:50:07 -0400 |
| parents | 9c3ec0a5c7fc |
| children |
| rev | line source |
|---|---|
| 2352 | 1 #!/usr/bin/env python |
| 2 ############################################################################## | |
| 3 # | |
| 4 # Copyright (c) 2002 Zope Corporation and Contributors. | |
| 5 # All Rights Reserved. | |
| 6 # | |
| 7 # This software is subject to the provisions of the Zope Public License, | |
| 8 # Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution. | |
| 9 # THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED | |
| 10 # WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED | |
| 11 # WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS | |
| 12 # FOR A PARTICULAR PURPOSE. | |
| 13 # | |
| 14 ############################################################################## | |
| 15 # Modifications for Roundup: | |
| 16 # 1. commented out ITALES references | |
|
2385
85cbfc4a5946
escape newlines
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2382
diff
changeset
|
17 # 2. escape quotes and line feeds in msgids |
|
2802
e816a232f988
don't collect empty msgids
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2385
diff
changeset
|
18 # 3. don't collect empty msgids |
| 2352 | 19 |
| 20 """Program to extract internationalization markup from Page Templates. | |
| 21 | |
| 22 Once you have marked up a Page Template file with i18n: namespace tags, use | |
| 23 this program to extract GNU gettext .po file entries. | |
| 24 | |
| 25 Usage: talgettext.py [options] files | |
| 26 Options: | |
| 27 -h / --help | |
| 28 Print this message and exit. | |
| 29 -o / --output <file> | |
| 30 Output the translation .po file to <file>. | |
| 31 -u / --update <file> | |
| 32 Update the existing translation <file> with any new translation strings | |
| 33 found. | |
| 34 """ | |
| 35 | |
| 36 import sys | |
| 37 import time | |
| 38 import getopt | |
| 39 import traceback | |
| 40 | |
|
4572
3a407f0d5dad
Add import for version info
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4570
diff
changeset
|
41 from roundup import __version__ |
| 2352 | 42 from roundup.cgi.TAL.HTMLTALParser import HTMLTALParser |
| 43 from roundup.cgi.TAL.TALInterpreter import TALInterpreter | |
| 44 from roundup.cgi.TAL.DummyEngine import DummyEngine | |
| 45 #from ITALES import ITALESEngine | |
| 46 from roundup.cgi.TAL.TALDefs import TALESError | |
| 47 | |
| 48 pot_header = '''\ | |
| 49 # SOME DESCRIPTIVE TITLE. | |
| 50 # Copyright (C) YEAR ORGANIZATION | |
| 51 # FIRST AUTHOR <EMAIL@ADDRESS>, YEAR. | |
| 52 # | |
| 53 msgid "" | |
| 54 msgstr "" | |
| 55 "Project-Id-Version: PACKAGE VERSION\\n" | |
| 56 "POT-Creation-Date: %(time)s\\n" | |
| 57 "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\\n" | |
| 58 "Last-Translator: FULL NAME <EMAIL@ADDRESS>\\n" | |
| 59 "Language-Team: LANGUAGE <LL@li.org>\\n" | |
| 60 "MIME-Version: 1.0\\n" | |
| 61 "Content-Type: text/plain; charset=CHARSET\\n" | |
| 62 "Content-Transfer-Encoding: ENCODING\\n" | |
| 63 "Generated-By: talgettext.py %(version)s\\n" | |
| 64 ''' | |
| 65 | |
| 66 NLSTR = '"\n"' | |
| 67 | |
| 68 def usage(code, msg=''): | |
| 69 # Python 2.1 required | |
|
5376
64b05e24dbd8
Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5248
diff
changeset
|
70 print(__doc__, file=sys.stderr) |
| 2352 | 71 if msg: |
|
5376
64b05e24dbd8
Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5248
diff
changeset
|
72 print(msg, file=sys.stderr) |
| 2352 | 73 sys.exit(code) |
| 74 | |
| 75 | |
| 76 class POTALInterpreter(TALInterpreter): | |
| 77 def translate(self, msgid, default, i18ndict=None, obj=None): | |
| 78 # XXX is this right? | |
| 79 if i18ndict is None: | |
| 80 i18ndict = {} | |
| 81 if obj: | |
| 82 i18ndict.update(obj) | |
| 83 # XXX Mmmh, it seems that sometimes the msgid is None; is that really | |
| 84 # possible? | |
| 85 if msgid is None: | |
| 86 return None | |
| 87 # XXX We need to pass in one of context or target_language | |
| 88 return self.engine.translate(msgid, self.i18nContext.domain, i18ndict, | |
| 89 position=self.position, default=default) | |
| 90 | |
| 91 | |
| 92 class POEngine(DummyEngine): | |
| 93 #__implements__ = ITALESEngine | |
| 94 | |
| 95 def __init__(self, macros=None): | |
| 96 self.catalog = {} | |
| 97 DummyEngine.__init__(self, macros) | |
| 98 | |
| 99 def evaluate(*args): | |
| 100 return '' # who cares | |
| 101 | |
| 102 def evaluatePathOrVar(*args): | |
| 103 return '' # who cares | |
| 104 | |
| 105 def evaluateSequence(self, expr): | |
| 106 return (0,) # dummy | |
| 107 | |
| 108 def evaluateBoolean(self, expr): | |
| 109 return True # dummy | |
| 110 | |
| 111 def translate(self, msgid, domain=None, mapping=None, default=None, | |
| 112 # XXX position is not part of the ITALESEngine | |
| 113 # interface | |
| 114 position=None): | |
| 115 | |
|
2802
e816a232f988
don't collect empty msgids
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2385
diff
changeset
|
116 if not msgid: return 'x' |
|
e816a232f988
don't collect empty msgids
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2385
diff
changeset
|
117 |
| 2352 | 118 if domain not in self.catalog: |
| 119 self.catalog[domain] = {} | |
| 120 domain = self.catalog[domain] | |
| 121 | |
| 122 if msgid not in domain: | |
| 123 domain[msgid] = [] | |
| 124 domain[msgid].append((self.file, position)) | |
| 125 return 'x' | |
| 126 | |
| 127 | |
| 128 class UpdatePOEngine(POEngine): | |
| 129 """A slightly-less braindead POEngine which supports loading an existing | |
| 130 .po file first.""" | |
| 131 | |
| 132 def __init__ (self, macros=None, filename=None): | |
| 133 POEngine.__init__(self, macros) | |
| 134 | |
| 135 self._filename = filename | |
| 136 self._loadFile() | |
| 137 self.base = self.catalog | |
| 138 self.catalog = {} | |
| 139 | |
| 140 def __add(self, id, s, fuzzy): | |
| 141 "Add a non-fuzzy translation to the dictionary." | |
| 142 if not fuzzy and str: | |
| 143 # check for multi-line values and munge them appropriately | |
| 144 if '\n' in s: | |
| 145 lines = s.rstrip().split('\n') | |
| 146 s = NLSTR.join(lines) | |
| 147 self.catalog[id] = s | |
| 148 | |
| 149 def _loadFile(self): | |
| 150 # shamelessly cribbed from Python's Tools/i18n/msgfmt.py | |
| 151 # 25-Mar-2003 Nathan R. Yergler (nathan@zope.org) | |
| 152 # 14-Apr-2003 Hacked by Barry Warsaw (barry@zope.com) | |
| 153 | |
| 154 ID = 1 | |
| 155 STR = 2 | |
| 156 | |
| 157 try: | |
| 158 lines = open(self._filename).readlines() | |
|
5248
198b6e810c67
Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents:
4572
diff
changeset
|
159 except IOError as msg: |
|
5376
64b05e24dbd8
Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5248
diff
changeset
|
160 print(msg, file=sys.stderr) |
| 2352 | 161 sys.exit(1) |
| 162 | |
| 163 section = None | |
| 164 fuzzy = False | |
| 165 | |
| 166 # Parse the catalog | |
| 167 lno = 0 | |
| 168 for l in lines: | |
| 169 lno += True | |
| 170 # If we get a comment line after a msgstr, this is a new entry | |
| 171 if l[0] == '#' and section == STR: | |
| 172 self.__add(msgid, msgstr, fuzzy) | |
| 173 section = None | |
| 174 fuzzy = False | |
| 175 # Record a fuzzy mark | |
| 176 if l[:2] == '#,' and l.find('fuzzy'): | |
| 177 fuzzy = True | |
| 178 # Skip comments | |
| 179 if l[0] == '#': | |
| 180 continue | |
| 181 # Now we are in a msgid section, output previous section | |
| 182 if l.startswith('msgid'): | |
| 183 if section == STR: | |
| 184 self.__add(msgid, msgstr, fuzzy) | |
| 185 section = ID | |
| 186 l = l[5:] | |
| 187 msgid = msgstr = '' | |
| 188 # Now we are in a msgstr section | |
| 189 elif l.startswith('msgstr'): | |
| 190 section = STR | |
| 191 l = l[6:] | |
| 192 # Skip empty lines | |
| 193 if not l.strip(): | |
| 194 continue | |
| 195 # XXX: Does this always follow Python escape semantics? | |
| 196 l = eval(l) | |
| 197 if section == ID: | |
| 198 msgid += l | |
| 199 elif section == STR: | |
| 200 msgstr += '%s\n' % l | |
| 201 else: | |
|
5376
64b05e24dbd8
Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5248
diff
changeset
|
202 print('Syntax error on %s:%d' % (infile, lno), |
|
64b05e24dbd8
Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5248
diff
changeset
|
203 'before:', file=sys.stderr) |
|
64b05e24dbd8
Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5248
diff
changeset
|
204 print(l, file=sys.stderr) |
| 2352 | 205 sys.exit(1) |
| 206 # Add last entry | |
| 207 if section == STR: | |
| 208 self.__add(msgid, msgstr, fuzzy) | |
| 209 | |
| 210 def evaluate(self, expression): | |
| 211 try: | |
| 212 return POEngine.evaluate(self, expression) | |
| 213 except TALESError: | |
| 214 pass | |
| 215 | |
| 216 def evaluatePathOrVar(self, expr): | |
| 217 return 'who cares' | |
| 218 | |
| 219 def translate(self, msgid, domain=None, mapping=None, default=None, | |
| 220 position=None): | |
| 221 if msgid not in self.base: | |
| 222 POEngine.translate(self, msgid, domain, mapping, default, position) | |
| 223 return 'x' | |
| 224 | |
| 225 | |
| 226 def main(): | |
| 227 try: | |
| 228 opts, args = getopt.getopt( | |
| 229 sys.argv[1:], | |
| 230 'ho:u:', | |
| 231 ['help', 'output=', 'update=']) | |
|
5248
198b6e810c67
Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents:
4572
diff
changeset
|
232 except getopt.error as msg: |
| 2352 | 233 usage(1, msg) |
| 234 | |
| 235 outfile = None | |
| 236 engine = None | |
| 237 update_mode = False | |
| 238 for opt, arg in opts: | |
| 239 if opt in ('-h', '--help'): | |
| 240 usage(0) | |
| 241 elif opt in ('-o', '--output'): | |
| 242 outfile = arg | |
| 243 elif opt in ('-u', '--update'): | |
| 244 update_mode = True | |
| 245 if outfile is None: | |
| 246 outfile = arg | |
| 247 engine = UpdatePOEngine(filename=arg) | |
| 248 | |
| 249 if not args: | |
|
5376
64b05e24dbd8
Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5248
diff
changeset
|
250 print('nothing to do') |
| 2352 | 251 return |
| 252 | |
| 253 # We don't care about the rendered output of the .pt file | |
| 254 class Devnull: | |
| 255 def write(self, s): | |
| 256 pass | |
| 257 | |
| 258 # check if we've already instantiated an engine; | |
| 259 # if not, use the stupidest one available | |
| 260 if not engine: | |
| 261 engine = POEngine() | |
| 262 | |
| 263 # process each file specified | |
| 264 for filename in args: | |
| 265 try: | |
| 266 engine.file = filename | |
| 267 p = HTMLTALParser() | |
| 268 p.parseFile(filename) | |
| 269 program, macros = p.getCode() | |
| 270 POTALInterpreter(program, macros, engine, stream=Devnull(), | |
| 271 metal=False)() | |
| 272 except: # Hee hee, I love bare excepts! | |
|
5376
64b05e24dbd8
Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5248
diff
changeset
|
273 print('There was an error processing', filename) |
| 2352 | 274 traceback.print_exc() |
| 275 | |
| 276 # Now output the keys in the engine. Write them to a file if --output or | |
| 277 # --update was specified; otherwise use standard out. | |
| 278 if (outfile is None): | |
| 279 outfile = sys.stdout | |
| 280 else: | |
|
5383
6fbb7d52e38f
Python 3 preparation: use open() instead of file().
Joseph Myers <jsm@polyomino.org.uk>
parents:
5376
diff
changeset
|
281 outfile = open(outfile, update_mode and "a" or "w") |
| 2352 | 282 |
| 283 catalog = {} | |
| 284 for domain in engine.catalog.keys(): | |
| 285 catalog.update(engine.catalog[domain]) | |
| 286 | |
| 287 messages = catalog.copy() | |
| 288 try: | |
| 289 messages.update(engine.base) | |
| 290 except AttributeError: | |
| 291 pass | |
| 292 if '' not in messages: | |
|
5376
64b05e24dbd8
Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5248
diff
changeset
|
293 print(pot_header % {'time': time.ctime(), |
|
64b05e24dbd8
Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5248
diff
changeset
|
294 'version': __version__}, file=outfile) |
| 2352 | 295 |
| 296 # XXX: You should not sort by msgid, but by filename and position. (SR) | |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5383
diff
changeset
|
297 msgids = sorted(catalog.keys()) |
| 2352 | 298 for msgid in msgids: |
| 299 positions = catalog[msgid] | |
| 300 for filename, position in positions: | |
| 301 outfile.write('#: %s:%s\n' % (filename, position[0])) | |
| 302 | |
|
2385
85cbfc4a5946
escape newlines
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2382
diff
changeset
|
303 outfile.write('msgid "%s"\n' |
|
85cbfc4a5946
escape newlines
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2382
diff
changeset
|
304 % msgid.replace('"', '\\"').replace("\n", '\\n"\n"')) |
| 2352 | 305 outfile.write('msgstr ""\n') |
| 306 outfile.write('\n') | |
| 307 | |
| 308 | |
| 309 if __name__ == '__main__': | |
| 310 main() |