Mercurial > p > roundup > code
view roundup/rate_limit.py @ 8472:224ccb8b49ca
refactor: change some classes to use __slots__
Speed up access to and reduce size of some low level classes. A few
classes in security.py, rest.py are heavily used. But for all, it
prevents adding random properties to lower level classes that people
shouldn't be mucking with. While doing this I found some test cases
accessing an invalid property name and this change caused the cases to
crash.
admin.py:
Use new method Role.props_dict() and Permission.props_dict() where
original code just referenced __dict__ when printing Role/Permission.
mlink_expr.py:
Add slots to multiple classes.
Classes Binary and Unary set real properties/attributes. Classes that
inherit from them (Equals, Empty, Not, Or, And) define empty slots
tuple to eliminate need for __dict__.
Class Expression also gets a slot.
rate_limit.py:
RateLimit and Gcra classes get slots.
A couple of pep8 fixes: sort imports, remove trailing spaces on a
line, remove unused noqa comment.
rest.py:
Add slots to class SimulateFieldStorageFromJson and FsValue
classes. The memory savings from this could be useful as well as
speedier access to the attributes.
security.py:
Add slots to Permission class. To prevent conflict between slot
limit_perm_to_props_only and the class variable of the same name,
rename the class variable to limit_perm_to_props_only_default.
Also define method props_dict() to allow other code to get a dict to
iterate over when checking permissions.
Add slots to class Role along with props_dict() method.
Add slots to class Security. Also have to add explicit __dict__ slot
to support test override of the hasPermission() method. Add
props_dict() method, currently unused, but added for symmetry.
support.py:
TruthDict and PrioList gets slots.
test/test_cgi.py:
Fix incorrect setting of permission property. Was setting
permissions. So testing may not have been doing what we thought it
was. Multiple places found with this typo.
Remove setting of permissions in some places where it should
have no effect on the test and looks like it was just copypasta.
test/test_xmlrpc.py
Remove setting of permissions in some places where it should
have no effect on the test and looks like it was just copypasta.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Mon, 03 Nov 2025 00:13:04 -0500 |
| parents | 5fbd3af526bd |
| children |
line wrap: on
line source
# Originaly from # https://smarketshq.com/implementing-gcra-in-python-5df1f11aaa96?gi=4b9725f99bfa # with imports, modifications for python 2, implementation of # set/get_tat and marshaling as string, support for testonly # and status method. from datetime import datetime, timedelta try: # used by python 3.11 and newer use tz aware dates from datetime import UTC dt_min = datetime.min.replace(tzinfo=UTC) # start of unix epoch dt_epoch = datetime(1970, 1, 1, tzinfo=UTC) fromisoformat = datetime.fromisoformat except ImportError: # python 2.7 and older than 3.11 - use naive dates dt_min = datetime.min dt_epoch = datetime(1970, 1, 1) def fromisoformat(date): # only for naive dates return datetime.strptime(date, "%Y-%m-%dT%H:%M:%S.%f") from roundup.anypy.datetime_ import utcnow class RateLimit: # pylint: disable=too-few-public-methods __slots__ = ("count", "period") def __init__(self, count, period): self.count = count self.period = period @property def inverse(self): return self.period.total_seconds() / self.count class Gcra: __slots__ = ("memory",) def __init__(self): self.memory = {} def get_tat(self, key): # This should return a previous tat for the key or the current time. if key in self.memory: return self.memory[key] else: return dt_min def set_tat(self, key, tat): self.memory[key] = tat def get_tat_as_string(self, key): # get value as string: # YYYY-MM-DDTHH:MM:SS.mmmmmm # to allow it to be marshalled/unmarshaled if key in self.memory: return self.memory[key].isoformat() else: return dt_min.isoformat() def set_tat_as_string(self, key, tat): # Take value as string and unmarshall: # YYYY-MM-DDTHH:MM:SS.mmmmmm # to datetime self.memory[key] = fromisoformat(tat) def update(self, key, limit, testonly=False): '''Determine if the item associated with the key should be rejected given the RateLimit limit. ''' now = utcnow() tat = max(self.get_tat(key), now) separation = (tat - now).total_seconds() max_interval = limit.period.total_seconds() - limit.inverse if separation > max_interval: reject = True else: reject = False if not testonly: new_tat = max(tat, now) + timedelta(seconds=limit.inverse) self.set_tat(key, new_tat) return reject def status(self, key, limit): '''Return status suitable for displaying as headers: X-RateLimit-Limit: calls allowed per period. Period/window is not specified in any api I found. X-RateLimit-Limit-Period: Non standard. Defines period in seconds for RateLimit-Limit. X-RateLimit-Remaining: How many calls are left in this window. X-RateLimit-Reset: window ends in this many seconds (not an epoch timestamp) and all RateLimit-Limit calls are available again. Retry-After: if user's request fails, this is the next time there will be at least 1 available call to be consumed. ''' ret = {} tat = self.get_tat(key) # static defined headers according to limit # all values are strings as that is required when used as headers ret['X-RateLimit-Limit'] = str(limit.count) ret['X-RateLimit-Limit-Period'] = str( int( limit.period.total_seconds()) ) # status of current limit as of now now = utcnow() current_count = int((limit.period - (tat - now)).total_seconds() / limit.inverse) ret['X-RateLimit-Remaining'] = str(min(current_count, limit.count)) # tat_in_epochsec = (tat - datetime(1970, 1, 1)).total_seconds() seconds_to_tat = (tat - now).total_seconds() ret['X-RateLimit-Reset'] = str(max(seconds_to_tat, 0)) ret['X-RateLimit-Reset-date'] = "%s" % tat ret['Now'] = str((now - dt_epoch).total_seconds()) ret['Now-date'] = "%s" % now if self.update(key, limit, testonly=True): # A new request would be rejected if it was processes. # The user has to wait until an item is dequeued. # One item is dequeued every limit.inverse seconds. ret['Retry-After'] = str(int(limit.inverse)) ret['Retry-After-Timestamp'] = "%s" % \ (now + timedelta(seconds=limit.inverse)) else: # if we are not rejected, the user can post another # attempt immediately. # Do we even need this header if not rejected? # RFC implies this is used with a 503 (or presumably # 429 which may postdate the rfc). So if no error, no header? # ret['Retry-After'] = '0' # ret['Retry-After-Timestamp'] = str(ret['Now-date']) pass return ret
