Mercurial > p > roundup > code
view test/wsgi_liveserver.py @ 7853:03c1b7ae3a68
issue2551328/issue2551264 unneeded next link and total_count incorrect
Fix: issue2551328 - REST results show next link if number of
results is a multiple of page size. (Found by members of
team 3 in the UMass-Boston CS682 Spring 2024 class.)
issue2551264 - REST X-Total-Count header and @total_size
count incorrect when paginated
These issues arose because we retrieved the exact number of rows
from the database as requested by the user using the @page_size
parameter. With this changeset, we retrieve up to 10 million + 1
rows from the database. If the total number of rows exceeds 10
million, we set the total_count indicators to -1 as an invalid
size. (The max number of requested rows (default 10 million +1)
can be modified by the admin through interfaces.py.)
By retrieving more data than necessary, we can calculate the
total count by adding @page_index*@page_size to the number of
rows returned by the query.
Furthermore, since we return more than @page_size rows, we can
determine the existence of a row at @page_size+1 and use that
information to determine if a next link should be
provided. Previously, a next link was returned if @page_size rows
were retrieved.
This change does not guarantee that the user will get @page_size
rows returned. Access policy filtering occurs after the rows are
returned, and discards rows inaccessible by the user.
Using the current @page_index/@page_size it would be difficult to
have the roundup code refetch data and make sure that a full
@page_size set of rows is returned. E.G. @page_size=100 and 5 of
them are dropped due to access restrictions. We then fetch 10
items and add items 1-4 and 6 (5 is inaccessible). There is no
way to calculate the new database offset at:
@page_index*@page_size + 6 from the URL. We would need to add an
@page_offset=6 or something.
This could work since the client isn't adding 1 to @page_index to
get the next page. Thanks to HATEOAS, the client just uses the
'next' url. But I am not going to cross that bridge without a
concrete use case.
This can also be handled client side by merging a short response
with the next response and re-paginating client side.
Also added extra index markers to the docs to highlight use of
interfaces.py.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Mon, 01 Apr 2024 09:57:16 -0400 |
| parents | e9760702bf0c |
| children | f6923d2ba9a5 |
line wrap: on
line source
# -*- coding: utf-8 -*- """ wsgi-liveserver provides a simple LiverServerTestCase class that can be used to help start a web server in the background to serve a WSGI compliant application for use with testing. Generally it will be used in conjuction with something like Selenium to perform a series of functional tests using a browser. Licensed under the GNU GPL v3 Copyright (c) 2013 John Kristensen (unless explicitly stated otherwise). """ import threading import socket import unittest from wsgiref.simple_server import make_server, WSGIRequestHandler __author__ = 'John Kristensen' __version__ = '0.3.1' __license__ = 'GPLv3' class QuietHandler(WSGIRequestHandler): def log_request(*args, **kwargs): pass class LiveServerTestCase(unittest.TestCase): port_range = (8080, 8090) def create_app(self): """Create your wsgi app and return it.""" raise NotImplementedError def __call__(self, result=None): """ Do some custom setup stuff and then hand off to TestCase to do its thing. """ try: self._pre_setup() super(LiveServerTestCase, self).__call__(result) finally: self._post_teardown() def url_base(self): """Return the url of the test server.""" return 'http://{0}:{1}'.format(self.host, self.port) def _pre_setup(self): """Setup and start the test server in the background.""" self._server = None self.host = 'localhost' self.port = self.port_range[0] self._thread = None # Get the app self.app = self.create_app() # Cycle through the port range to find a free port while self._server is None and self.port <= self.port_range[1]: try: self._server = make_server(self.host, self.port, self.app, handler_class=QuietHandler) except socket.error: self.port += 1 # No free port, raise an exception if self._server is None: raise socket.error('Ports {0}-{1} are all already in use'.format( *self.port_range)) # Start the test server in the background self._thread = threading.Thread(target=self._server.serve_forever) self._thread.start() def _post_teardown(self): """Stop the test server.""" if self._thread is not None: self._server.shutdown() self._server.server_close() self._thread.join() del self._server
