annotate roundup/cgi/accept_language.py @ 5710:0b79bfcb3312

Add support for making an idempotent POST. This allows retrying a POST that was interrupted. It involves creating a post once only (poe) url /rest/data/<class>/@poe/<random_token>. This url acts the same as a post to /rest/data/<class>. However once the @poe url is used, it can't be used for a second POST. To make these changes: 1) Take the body of post_collection into a new post_collection_inner function. Have post_collection call post_collection_inner. 2) Add a handler for POST to rest/data/class/@poe. This will return a unique POE url. By default the url expires after 30 minutes. The POE random token is only good for a specific user and is stored in the session db. 3) Add a handler for POST to rest/data/<class>/@poe/<random token>. The random token generated in 2 is validated for proper class (if token is not generic) and proper user and must not have expired. If everything is valid, call post_collection_inner to process the input and generate the new entry. To make recognition of 2 stable (so it's not confused with rest/data/<:class_name>/<:item_id>), removed @ from Routing::url_to_regex. The current Routing.execute method stops on the first regular expression to match the URL. Since item_id doesn't accept a POST, I was getting 405 bad method sometimes. My guess is the order of the regular expressions is not stable, so sometime I would get the right regexp for /data/<class>/@poe and sometime I would get the one for /data/<class>/<item_id>. By removing the @ from the url_to_regexp, there was no way for the item_id case to match @poe. There are alternate fixes we may need to look at. If a regexp matches but the method does not, return to the regexp matching loop in execute() looking for another match. Only once every possible match has failed should the code return a 405 method failure. Another fix is to implement a more sophisticated mechanism so that @Routing.route("/data/<:class_name>/<:item_id>/<:attr_name>", 'PATCH') has different regexps for matching <:class_name> <:item_id> and <:attr_name>. Currently the regexp specified by url_to_regex is used for every component. Other fixes: Made failure to find any props in props_from_args return an empty dict rather than throwing an unhandled error. Make __init__ for SimulateFieldStorageFromJson handle an empty json doc. Useful for POSTing to rest/data/class/@poe with an empty document. Testing: added testPostPOE to test/rest_common.py that I think covers all the code that was added. Documentation: Add doc to rest.txt in the "Client API" section titled: Safely Re-sending POST". Move existing section "Adding new rest endpoints" in "Client API" to a new second level section called "Programming the REST API". Also a minor change to the simple rest client moving the header setting to continuation lines rather than showing one long line.
author John Rouillard <rouilj@ieee.org>
date Sun, 14 Apr 2019 21:07:11 -0400
parents b00cd44fea16
children 936275dfe1fa
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3426
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
1 """Parse the Accept-Language header as defined in RFC2616.
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
2
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
3 See http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.4
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
4 for details. This module should follow the spec.
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
5 Author: Hernan M. Foffani (hfoffani@gmail.com)
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
6 Some use samples:
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
7
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
8 >>> parse("da, en-gb;q=0.8, en;q=0.7")
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
9 ['da', 'en_gb', 'en']
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
10 >>> parse("en;q=0.2, fr;q=1")
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
11 ['fr', 'en']
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
12 >>> parse("zn; q = 0.2 ,pt-br;q =1")
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
13 ['pt_br', 'zn']
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
14 >>> parse("es-AR")
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
15 ['es_AR']
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
16 >>> parse("es-es-cat")
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
17 ['es_es_cat']
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
18 >>> parse("")
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
19 []
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
20 >>> parse(None)
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
21 []
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
22 >>> parse(" ")
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
23 []
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
24 >>> parse("en,")
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
25 ['en']
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
26 """
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
27
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
28 import re
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
29 import heapq
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
30
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
31 # regexp for languange-range search
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
32 nqlre = "([A-Za-z]+[-[A-Za-z]+]*)$"
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
33 # regexp for languange-range search with quality value
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
34 qlre = "([A-Za-z]+[-[A-Za-z]+]*);q=([\d\.]+)"
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
35 # both
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
36 lre = re.compile(nqlre + "|" + qlre)
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
37
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
38 whitespace = ' \t\n\r\v\f'
5439
b00cd44fea16 Python 3 preparation: update string translate method call in cgi/accept_language.py.
Joseph Myers <jsm@polyomino.org.uk>
parents: 4362
diff changeset
39 try:
b00cd44fea16 Python 3 preparation: update string translate method call in cgi/accept_language.py.
Joseph Myers <jsm@polyomino.org.uk>
parents: 4362
diff changeset
40 # Python 3.
b00cd44fea16 Python 3 preparation: update string translate method call in cgi/accept_language.py.
Joseph Myers <jsm@polyomino.org.uk>
parents: 4362
diff changeset
41 remove_ws = (str.maketrans('', '', whitespace),)
b00cd44fea16 Python 3 preparation: update string translate method call in cgi/accept_language.py.
Joseph Myers <jsm@polyomino.org.uk>
parents: 4362
diff changeset
42 except AttributeError:
b00cd44fea16 Python 3 preparation: update string translate method call in cgi/accept_language.py.
Joseph Myers <jsm@polyomino.org.uk>
parents: 4362
diff changeset
43 # Python 2.
b00cd44fea16 Python 3 preparation: update string translate method call in cgi/accept_language.py.
Joseph Myers <jsm@polyomino.org.uk>
parents: 4362
diff changeset
44 remove_ws = (None, whitespace)
3426
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
45
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
46 def parse(language_header):
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
47 """parse(string_with_accept_header_content) -> languages list"""
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
48
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
49 if language_header is None: return []
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
50
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
51 # strip whitespaces.
5439
b00cd44fea16 Python 3 preparation: update string translate method call in cgi/accept_language.py.
Joseph Myers <jsm@polyomino.org.uk>
parents: 4362
diff changeset
52 lh = language_header.translate(*remove_ws)
3426
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
53
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
54 # if nothing, return
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
55 if lh == "": return []
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
56
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
57 # split by commas and parse the quality values.
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
58 pls = [lre.findall(x) for x in lh.split(',')]
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
59
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
60 # drop uncomformant
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
61 qls = [x[0] for x in pls if len(x) > 0]
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
62
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
63 # use a heap queue to sort by quality values.
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
64 # the value of each item is 1.0 complement.
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
65 pq = []
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
66 for l in qls:
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
67 if l[0] != '':
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
68 heapq.heappush(pq, (0.0, l[0]))
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
69 else:
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
70 heapq.heappush(pq, (1.0-float(l[2]), l[1]))
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
71
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
72 # get the languages ordered by quality
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
73 # and replace - by _
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
74 return [x[1].replace('-','_') for x in pq]
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
75
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
76 if __name__ == "__main__":
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
77 import doctest
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
78 doctest.testmod()
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
79
52f89836d05b Parse the Accept-Language header as defined in RFC2616.
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
80 # vim: set et sts=4 sw=4 :

Roundup Issue Tracker: http://roundup-tracker.org/