Mercurial > p > roundup > code

annotate roundup/rate_limit.py @ 7587:8f29e4ea05ce

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
fix: issue2551278 - datetime.datetime.utcnow deprecation. Replace calls with equivalent that produces timezone aware dates rather than naive dates. Also some flake8 fixes for test/rest_common.py.
author John Rouillard <rouilj@ieee.org>
date Tue, 25 Jul 2023 16:30:10 -0400
parents 69a35d164a69
children 5fbd3af526bd
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
5717
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
1 # Originaly from
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
2 # https://smarketshq.com/implementing-gcra-in-python-5df1f11aaa96?gi=4b9725f99bfa
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
3 # with imports, modifications for python 2, implementation of
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
4 # set/get_tat and marshaling as string, support for testonly
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
5 # and status method.
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
6
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
7 from datetime import timedelta, datetime
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
8
7587
8f29e4ea05ce fix: issue2551278 - datetime.datetime.utcnow deprecation.
John Rouillard <rouilj@ieee.org>
parents: 5996
diff changeset
9 from roundup.anypy.datetime_ import utcnow
8f29e4ea05ce fix: issue2551278 - datetime.datetime.utcnow deprecation.
John Rouillard <rouilj@ieee.org>
parents: 5996
diff changeset
10
5996
69a35d164a69 Make rate_limit.py pass flake8.
John Rouillard <rouilj@ieee.org>
parents: 5937
diff changeset
11
5739
e225f403cc35 Run pylint and clean up it's issues. Also fix comment.
John Rouillard <rouilj@ieee.org>
parents: 5722
diff changeset
12 class RateLimit: # pylint: disable=too-few-public-methods
5717
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
13 def __init__(self, count, period):
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
14 self.count = count
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
15 self.period = period
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
16
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
17 @property
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
18 def inverse(self):
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
19 return self.period.total_seconds() / self.count
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
20
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
21
5722
2f116ba7e7cf Rename Store class in rate_limit.py to Gcra. The name Store makes no
John Rouillard <rouilj@ieee.org>
parents: 5717
diff changeset
22 class Gcra:
5739
e225f403cc35 Run pylint and clean up it's issues. Also fix comment.
John Rouillard <rouilj@ieee.org>
parents: 5722
diff changeset
23 def __init__(self):
e225f403cc35 Run pylint and clean up it's issues. Also fix comment.
John Rouillard <rouilj@ieee.org>
parents: 5722
diff changeset
24 self.memory = {}
5717
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
25
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
26 def get_tat(self, key):
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
27 # This should return a previous tat for the key or the current time.
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
28 if key in self.memory:
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
29 return self.memory[key]
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
30 else:
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
31 return datetime.min
5996
69a35d164a69 Make rate_limit.py pass flake8.
John Rouillard <rouilj@ieee.org>
parents: 5937
diff changeset
32
5717
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
33 def set_tat(self, key, tat):
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
34 self.memory[key] = tat
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
35
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
36 def get_tat_as_string(self, key):
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
37 # get value as string:
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
38 # YYYY-MM-DDTHH:MM:SS.mmmmmm
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
39 # to allow it to be marshalled/unmarshaled
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
40 if key in self.memory:
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
41 return self.memory[key].isoformat()
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
42 else:
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
43 return datetime.min.isoformat()
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
44
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
45 def set_tat_as_string(self, key, tat):
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
46 # Take value as string and unmarshall:
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
47 # YYYY-MM-DDTHH:MM:SS.mmmmmm
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
48 # to datetime
5996
69a35d164a69 Make rate_limit.py pass flake8.
John Rouillard <rouilj@ieee.org>
parents: 5937
diff changeset
49 self.memory[key] = datetime.strptime(tat, "%Y-%m-%dT%H:%M:%S.%f")
5717
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
50
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
51 def update(self, key, limit, testonly=False):
5739
e225f403cc35 Run pylint and clean up it's issues. Also fix comment.
John Rouillard <rouilj@ieee.org>
parents: 5722
diff changeset
52 '''Determine if the item associated with the key should be
5717
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
53 rejected given the RateLimit limit.
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
54 '''
7587
8f29e4ea05ce fix: issue2551278 - datetime.datetime.utcnow deprecation.
John Rouillard <rouilj@ieee.org>
parents: 5996
diff changeset
55 now = utcnow()
5717
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
56 tat = max(self.get_tat(key), now)
5996
69a35d164a69 Make rate_limit.py pass flake8.
John Rouillard <rouilj@ieee.org>
parents: 5937
diff changeset
57 separation = (tat - now).total_seconds()
5717
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
58 max_interval = limit.period.total_seconds() - limit.inverse
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
59 if separation > max_interval:
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
60 reject = True
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
61 else:
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
62 reject = False
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
63 if not testonly:
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
64 new_tat = max(tat, now) + timedelta(seconds=limit.inverse)
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
65 self.set_tat(key, new_tat)
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
66 return reject
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
67
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
68 def status(self, key, limit):
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
69 '''Return status suitable for displaying as headers:
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
70 X-RateLimit-Limit: calls allowed per period. Period/window
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
71 is not specified in any api I found.
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
72 X-RateLimit-Limit-Period: Non standard. Defines period in
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
73 seconds for RateLimit-Limit.
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
74 X-RateLimit-Remaining: How many calls are left in this window.
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
75 X-RateLimit-Reset: window ends in this many seconds (not an
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
76 epoch timestamp) and all RateLimit-Limit calls are
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
77 available again.
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
78 Retry-After: if user's request fails, this is the next time there
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
79 will be at least 1 available call to be consumed.
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
80 '''
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
81
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
82 ret = {}
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
83 tat = self.get_tat(key)
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
84 # static defined headers according to limit
5937
5d0873a4de4a fix rate limit headers - were ints/floats need to be strings
John Rouillard <rouilj@ieee.org>
parents: 5739
diff changeset
85 # all values are strings as that is required when used as headers
5d0873a4de4a fix rate limit headers - were ints/floats need to be strings
John Rouillard <rouilj@ieee.org>
parents: 5739
diff changeset
86 ret['X-RateLimit-Limit'] = str(limit.count)
5d0873a4de4a fix rate limit headers - were ints/floats need to be strings
John Rouillard <rouilj@ieee.org>
parents: 5739
diff changeset
87 ret['X-RateLimit-Limit-Period'] = str(
5d0873a4de4a fix rate limit headers - were ints/floats need to be strings
John Rouillard <rouilj@ieee.org>
parents: 5739
diff changeset
88 int(
5d0873a4de4a fix rate limit headers - were ints/floats need to be strings
John Rouillard <rouilj@ieee.org>
parents: 5739
diff changeset
89 limit.period.total_seconds())
5d0873a4de4a fix rate limit headers - were ints/floats need to be strings
John Rouillard <rouilj@ieee.org>
parents: 5739
diff changeset
90 )
5717
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
91
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
92 # status of current limit as of now
7587
8f29e4ea05ce fix: issue2551278 - datetime.datetime.utcnow deprecation.
John Rouillard <rouilj@ieee.org>
parents: 5996
diff changeset
93 now = utcnow()
5717
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
94
5996
69a35d164a69 Make rate_limit.py pass flake8.
John Rouillard <rouilj@ieee.org>
parents: 5937
diff changeset
95 current_count = int((limit.period - (tat - now)).total_seconds() /
69a35d164a69 Make rate_limit.py pass flake8.
John Rouillard <rouilj@ieee.org>
parents: 5937
diff changeset
96 limit.inverse)
69a35d164a69 Make rate_limit.py pass flake8.
John Rouillard <rouilj@ieee.org>
parents: 5937
diff changeset
97 ret['X-RateLimit-Remaining'] = str(min(current_count, limit.count))
5717
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
98
5739
e225f403cc35 Run pylint and clean up it's issues. Also fix comment.
John Rouillard <rouilj@ieee.org>
parents: 5722
diff changeset
99 # tat_in_epochsec = (tat - datetime(1970, 1, 1)).total_seconds()
5717
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
100 seconds_to_tat = (tat - now).total_seconds()
5937
5d0873a4de4a fix rate limit headers - were ints/floats need to be strings
John Rouillard <rouilj@ieee.org>
parents: 5739
diff changeset
101 ret['X-RateLimit-Reset'] = str(max(seconds_to_tat, 0))
5996
69a35d164a69 Make rate_limit.py pass flake8.
John Rouillard <rouilj@ieee.org>
parents: 5937
diff changeset
102 ret['X-RateLimit-Reset-date'] = "%s" % tat
69a35d164a69 Make rate_limit.py pass flake8.
John Rouillard <rouilj@ieee.org>
parents: 5937
diff changeset
103 ret['Now'] = str((now - datetime(1970, 1, 1)).total_seconds())
69a35d164a69 Make rate_limit.py pass flake8.
John Rouillard <rouilj@ieee.org>
parents: 5937
diff changeset
104 ret['Now-date'] = "%s" % now
5717
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
105
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
106 if self.update(key, limit, testonly=True):
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
107 # A new request would be rejected if it was processes.
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
108 # The user has to wait until an item is dequeued.
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
109 # One item is dequeued every limit.inverse seconds.
5937
5d0873a4de4a fix rate limit headers - were ints/floats need to be strings
John Rouillard <rouilj@ieee.org>
parents: 5739
diff changeset
110 ret['Retry-After'] = str(int(limit.inverse))
5996
69a35d164a69 Make rate_limit.py pass flake8.
John Rouillard <rouilj@ieee.org>
parents: 5937
diff changeset
111 ret['Retry-After-Timestamp'] = "%s" % \
69a35d164a69 Make rate_limit.py pass flake8.
John Rouillard <rouilj@ieee.org>
parents: 5937
diff changeset
112 (now + timedelta(seconds=limit.inverse)) # noqa: E127
5717
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
113 else:
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
114 # if we are not rejected, the user can post another
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
115 # attempt immediately.
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
116 # Do we even need this header if not rejected?
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
117 # RFC implies this is used with a 503 (or presumably
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
118 # 429 which may postdate the rfc). So if no error, no header?
5937
5d0873a4de4a fix rate limit headers - were ints/floats need to be strings
John Rouillard <rouilj@ieee.org>
parents: 5739
diff changeset
119 # ret['Retry-After'] = '0'
5d0873a4de4a fix rate limit headers - were ints/floats need to be strings
John Rouillard <rouilj@ieee.org>
parents: 5739
diff changeset
120 # ret['Retry-After-Timestamp'] = str(ret['Now-date'])
5717
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
121 pass
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
122
cad18de2b988 issue2550949: Rate limit password guesses/login attempts.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
123 return ret
Roundup Issue Tracker: http://roundup-tracker.org/