Mercurial > p > roundup > code
view test/test_userauditor.py @ 7853:03c1b7ae3a68
issue2551328/issue2551264 unneeded next link and total_count incorrect
Fix: issue2551328 - REST results show next link if number of
results is a multiple of page size. (Found by members of
team 3 in the UMass-Boston CS682 Spring 2024 class.)
issue2551264 - REST X-Total-Count header and @total_size
count incorrect when paginated
These issues arose because we retrieved the exact number of rows
from the database as requested by the user using the @page_size
parameter. With this changeset, we retrieve up to 10 million + 1
rows from the database. If the total number of rows exceeds 10
million, we set the total_count indicators to -1 as an invalid
size. (The max number of requested rows (default 10 million +1)
can be modified by the admin through interfaces.py.)
By retrieving more data than necessary, we can calculate the
total count by adding @page_index*@page_size to the number of
rows returned by the query.
Furthermore, since we return more than @page_size rows, we can
determine the existence of a row at @page_size+1 and use that
information to determine if a next link should be
provided. Previously, a next link was returned if @page_size rows
were retrieved.
This change does not guarantee that the user will get @page_size
rows returned. Access policy filtering occurs after the rows are
returned, and discards rows inaccessible by the user.
Using the current @page_index/@page_size it would be difficult to
have the roundup code refetch data and make sure that a full
@page_size set of rows is returned. E.G. @page_size=100 and 5 of
them are dropped due to access restrictions. We then fetch 10
items and add items 1-4 and 6 (5 is inaccessible). There is no
way to calculate the new database offset at:
@page_index*@page_size + 6 from the URL. We would need to add an
@page_offset=6 or something.
This could work since the client isn't adding 1 to @page_index to
get the next page. Thanks to HATEOAS, the client just uses the
'next' url. But I am not going to cross that bridge without a
concrete use case.
This can also be handled client side by merging a short response
with the next response and re-paginating client side.
Also added extra index markers to the docs to highlight use of
interfaces.py.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Mon, 01 Apr 2024 09:57:16 -0400 |
| parents | 89aa919997c0 |
| children |
line wrap: on
line source
import os, unittest, shutil from .db_test_base import setupTracker from .test_dates import skip_pytz class UserAuditorTest(unittest.TestCase): def setUp(self): self.dirname = '_test_user_auditor' self.instance = setupTracker(self.dirname) self.db = self.instance.open('admin') self.db.tx_Source = "cli" self.db.user.create(username='kyle', address='kyle@example.com', realname='Kyle Broflovski', roles='User') def tearDown(self): self.db.close() try: shutil.rmtree(self.dirname) except OSError as error: if error.errno not in (errno.ENOENT, errno.ESRCH): raise def testBadTimezones(self): self.assertRaises(ValueError, self.db.user.create, username='eric', timezone='24') userid = self.db.user.lookup('kyle') self.assertRaises(ValueError, self.db.user.set, userid, timezone='3000') self.assertRaises(ValueError, self.db.user.set, userid, timezone='24') self.assertRaises(ValueError, self.db.user.set, userid, timezone='-24') self.assertRaises(ValueError, self.db.user.set, userid, timezone='-3000') @skip_pytz def testBadTimezonesPyTZ(self): userid = self.db.user.lookup('kyle') self.assertRaises(KeyError, self.db.user.set, userid, timezone='MiddleOf/Nowhere') def testGoodTimezones(self): self.db.user.create(username='test_user01', timezone='12') userid = self.db.user.lookup('kyle') # TODO: roundup should accept non-integer offsets since those are valid # this is the offset for Tehran, Iran #self.db.user.set(userid, timezone='3.5') self.db.user.set(userid, timezone='-23') self.db.user.set(userid, timezone='23') self.db.user.set(userid, timezone='0') @skip_pytz def testGoodTimezonesPyTZ(self): userid = self.db.user.lookup('kyle') self.db.user.create(username='test_user02', timezone='MST') self.db.user.set(userid, timezone='US/Eastern') def testBadEmailAddresses(self): userid = self.db.user.lookup('kyle') self.assertRaises(ValueError, self.db.user.set, userid, address='kyle @ example.com') self.assertRaises(ValueError, self.db.user.set, userid, address='one@example.com,two@example.com') self.assertRaises(ValueError, self.db.user.set, userid, address='weird@@example.com') self.assertRaises(ValueError, self.db.user.set, userid, address='embedded\nnewline@example.com') # verify that we check alternates as well self.assertRaises(ValueError, self.db.user.set, userid, alternate_addresses='kyle @ example.com') # make sure we accept local style addresses self.db.user.set(userid, address='kyle') # verify we are case insensitive self.db.user.set(userid, address='kyle@EXAMPLE.COM') def testUniqueEmailAddresses(self): self.db.user.create(username='kenny', address='kenny@example.com', alternate_addresses='sp_ken@example.com') self.assertRaises(ValueError, self.db.user.create, username='test_user01', address='kenny@example.com') uid = self.db.user.create(username='eric', address='eric@example.com') self.assertRaises(ValueError, self.db.user.set, uid, address='kenny@example.com') # make sure we check alternates self.assertRaises(ValueError, self.db.user.set, uid, address='kenny@example.com') self.assertRaises(ValueError, self.db.user.set, uid, address='sp_ken@example.com') self.assertRaises(ValueError, self.db.user.set, uid, alternate_addresses='kenny@example.com') def testBadRoles(self): userid = self.db.user.lookup('kyle') self.assertRaises(ValueError, self.db.user.set, userid, roles='BadRole') self.assertRaises(ValueError, self.db.user.set, userid, roles='User,BadRole') def testGoodRoles(self): userid = self.db.user.lookup('kyle') # make sure we handle commas in weird places self.db.user.set(userid, roles='User,') self.db.user.set(userid, roles=',User') # make sure we strip whitespace self.db.user.set(userid, roles=' User ') # check for all-whitespace (treat as no role) self.db.user.set(userid, roles=' ') def testBadUsernames(self): ''' ky,le raises: ValueError: Username/Login Name must consist only of the letters a-z (any case), digits 0-9 and the symbols: @._-!+% ''' for name in [ "ky'le", "ky<br>le", "ky,le" ]: with self.assertRaises(ValueError) as ctx: self.db.user.create(username=name, address='kyle@example.com', realname='Kyle Broflovski', roles='User') self.assertEqual(str(ctx.exception), "Username/Login Name must " "consist only of the letters a-z (any case), " "digits 0-9 and the symbols: @._-!+%") self.db.user.create(username='rouilj-1+mya_ddr@users.example.com', address='kyle1@example.com', realname='Kyle Broflovski', roles='User') # vim: filetype=python sts=4 sw=4 et si
