Mercurial > p > roundup > code
view test/test_hypothesis.py @ 7853:03c1b7ae3a68
issue2551328/issue2551264 unneeded next link and total_count incorrect
Fix: issue2551328 - REST results show next link if number of
results is a multiple of page size. (Found by members of
team 3 in the UMass-Boston CS682 Spring 2024 class.)
issue2551264 - REST X-Total-Count header and @total_size
count incorrect when paginated
These issues arose because we retrieved the exact number of rows
from the database as requested by the user using the @page_size
parameter. With this changeset, we retrieve up to 10 million + 1
rows from the database. If the total number of rows exceeds 10
million, we set the total_count indicators to -1 as an invalid
size. (The max number of requested rows (default 10 million +1)
can be modified by the admin through interfaces.py.)
By retrieving more data than necessary, we can calculate the
total count by adding @page_index*@page_size to the number of
rows returned by the query.
Furthermore, since we return more than @page_size rows, we can
determine the existence of a row at @page_size+1 and use that
information to determine if a next link should be
provided. Previously, a next link was returned if @page_size rows
were retrieved.
This change does not guarantee that the user will get @page_size
rows returned. Access policy filtering occurs after the rows are
returned, and discards rows inaccessible by the user.
Using the current @page_index/@page_size it would be difficult to
have the roundup code refetch data and make sure that a full
@page_size set of rows is returned. E.G. @page_size=100 and 5 of
them are dropped due to access restrictions. We then fetch 10
items and add items 1-4 and 6 (5 is inaccessible). There is no
way to calculate the new database offset at:
@page_index*@page_size + 6 from the URL. We would need to add an
@page_offset=6 or something.
This could work since the client isn't adding 1 to @page_index to
get the next page. Thanks to HATEOAS, the client just uses the
'next' url. But I am not going to cross that bridge without a
concrete use case.
This can also be handled client side by merging a short response
with the next response and re-paginating client side.
Also added extra index markers to the docs to highlight use of
interfaces.py.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Mon, 01 Apr 2024 09:57:16 -0400 |
| parents | ce740d9a7d8d |
| children | 9ff94a2e8c82 |
line wrap: on
line source
import unittest import pytest pytest.importorskip("hypothesis") # ruff: noqa: E402 from hypothesis import example, given, settings from hypothesis.strategies import binary, none, one_of, sampled_from, text from roundup.anypy.strings import b2s, s2b, s2u, u2s # ruff: noqa: I001 - yes I know I am using \ to continue the line... from roundup.password import PasswordValueError, encodePassword, \ h64decode, h64encode def Identity(x): return x _max_examples = 1000 class HypoTestStrings(unittest.TestCase): @given(text()) @settings(max_examples=_max_examples) def test_b2s(self, utf8_bytes): self.assertEqual(b2s(utf8_bytes.encode("utf-8")), utf8_bytes) @given(text()) @settings(max_examples=_max_examples) def test_s2b(self, s): self.assertTrue(isinstance(s2b(s), bytes)) @given(text()) @settings(max_examples=_max_examples) @example("\U0001F600 hi there") # smiley face emoji def test_s2u_u2s_invertable(self, s): self.assertEqual(u2s(s2u(s)), s) class HypoTestPassword(unittest.TestCase): @given(binary()) @example(b"") @settings(max_examples=_max_examples) def test_h64encode_h64decode(self, s): self.assertEqual(h64decode(h64encode(s)), s) @given(one_of(none(), text()), sampled_from(("PBKDF2S5", "PBKDF2", "SSHA", "SHA", "MD5", "crypt", "plaintext", "zot"))) @example("asd\x00df", "crypt") @settings(max_examples=_max_examples) def test_encodePassword(self, password, scheme): if scheme == "crypt" and password and "\x00" in password: with self.assertRaises(ValueError) as e: encodePassword(password, scheme) self.assertEqual(e.exception.args[0], "embedded null character") elif scheme == "plaintext": if password is not None: self.assertEqual(encodePassword(password, scheme), password) else: self.assertEqual(encodePassword(password, scheme), "") elif scheme == "zot": with self.assertRaises(PasswordValueError) as e: encodePassword(password, scheme) self.assertEqual(e.exception.args[0], "Unknown encryption scheme 'zot'") else: # it shouldn't throw anything. pw = encodePassword(password, scheme) # verify format if scheme in ["PBKDF2S5", "PBKDF2"]: # 1000$XbSsijELEQbZZb1LlD7CFuotF/8$DdtssSlm.e self.assertRegex(pw, r"^\d{4,8}\$.{27}\$.*") elif scheme == "SSHA": # vqDbjvs8rhrS1AJxHYEGGXQW3x7STAPgo7uCtnw4GYgU7FN5VYbZxccQYCC0eXOxSipLbtgBudH1vDRMNlG0uw== self.assertRegex(pw, r"^[^=]*={0,3}$") elif scheme == "SHA": # da39a3ee5e6b4b0d3255bfef95601890afd80709' self.assertRegex(pw, r"^[a-z0-9]{40}$") elif scheme == "MD5": # d41d8cd98f00b204e9800998ecf8427e' self.assertRegex(pw, r"^[a-z0-9]{32}$") elif scheme == "crypt": # WqzFDzhi8MmoU self.assertRegex(pw, r"^[A-Za-z0-9./]{13}$") else: self.assertFalse("Unknown scheme: %s, val: %s" % (scheme, pw))
