http://hg.code.sf.net:8000/p/roundup/code/atom-log/tip/test/test_cgi.py Mercurial Repository: p/roundup/code: test/test_cgi.py history 2026-04-08T21:39:40-04:00 chore: remove __future print_funcion from code. http://hg.code.sf.net:8000/p/roundup/code/#changeset-9c3ec0a5c7fc88acb8a65632ecc13b2d52380314 John Rouillard rouilj@ieee.org 2026-04-08T21:39:40-04:00 2026-04-08T21:39:40-04:00
changeset 9c3ec0a5c7fc
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description chore: remove __future print_funcion from code.

Not needed as of Python 3.
files
issue2551413 - Broken MultiLink columns in CSV export (take 2) http://hg.code.sf.net:8000/p/roundup/code/#changeset-166cb2632315149c28c8414fa84d9eaceb0bdfda John Rouillard rouilj@ieee.org 2025-12-15T00:04:16-05:00 2025-12-15T00:04:16-05:00
changeset 166cb2632315
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551413 - Broken MultiLink columns in CSV export (take 2)

Changed how I solved this. Restored the original line that cmeerw took
out, but use the 'id' field rather than the 'name' field. The if
statements folowing the line change it to the 'name' field
(realname if it's a user object): if there is one.

Updated the tests to test for this error and exercise the code. I had
to change the test to create/add messages to an issue. This required
that I suppress the sending of nosy messages using SENDMAILDEBUG env
var.
files
refactor: change some classes to use __slots__ http://hg.code.sf.net:8000/p/roundup/code/#changeset-224ccb8b49ca54569350edb5905b19185a97b092 John Rouillard rouilj@ieee.org 2025-11-03T00:13:04-05:00 2025-11-03T00:13:04-05:00
changeset 224ccb8b49ca
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description refactor: change some classes to use __slots__

Speed up access to and reduce size of some low level classes. A few
classes in security.py, rest.py are heavily used. But for all, it
prevents adding random properties to lower level classes that people
shouldn't be mucking with. While doing this I found some test cases
accessing an invalid property name and this change caused the cases to
crash.

admin.py:

Use new method Role.props_dict() and Permission.props_dict() where
original code just referenced __dict__ when printing Role/Permission.

mlink_expr.py:

Add slots to multiple classes.

Classes Binary and Unary set real properties/attributes. Classes that
inherit from them (Equals, Empty, Not, Or, And) define empty slots
tuple to eliminate need for __dict__.

Class Expression also gets a slot.

rate_limit.py:

RateLimit and Gcra classes get slots.

A couple of pep8 fixes: sort imports, remove trailing spaces on a
line, remove unused noqa comment.

rest.py:

Add slots to class SimulateFieldStorageFromJson and FsValue
classes. The memory savings from this could be useful as well as
speedier access to the attributes.

security.py:

Add slots to Permission class. To prevent conflict between slot
limit_perm_to_props_only and the class variable of the same name,
rename the class variable to limit_perm_to_props_only_default.
Also define method props_dict() to allow other code to get a dict to
iterate over when checking permissions.

Add slots to class Role along with props_dict() method.

Add slots to class Security. Also have to add explicit __dict__ slot
to support test override of the hasPermission() method. Add
props_dict() method, currently unused, but added for symmetry.

support.py:

TruthDict and PrioList gets slots.

test/test_cgi.py:

Fix incorrect setting of permission property. Was setting
permissions. So testing may not have been doing what we thought it
was. Multiple places found with this typo.

Remove setting of permissions in some places where it should
have no effect on the test and looks like it was just copypasta.

test/test_xmlrpc.py

Remove setting of permissions in some places where it should
have no effect on the test and looks like it was just copypasta.
files
fix(web): issue2551406 - dont crash when handed invalid @template=a|b|c http://hg.code.sf.net:8000/p/roundup/code/#changeset-b07165add61b439e585d2e04064a3021628aea81 John Rouillard rouilj@ieee.org 2025-06-02T08:52:39-04:00 2025-06-02T08:52:39-04:00
changeset b07165add61b
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix(web): issue2551406 - dont crash when handed invalid @template=a|b|c

Only two template cases (ok and error) are handled. Presence of second
'|' caused crash. Discovered/patch provided by Christof Meerwald
(cmeerw).
files
issue2551391 - checkboxes and radiobutton inputs get wrong id's. http://hg.code.sf.net:8000/p/roundup/code/#changeset-669dfccca89860ec425a3ece589811ef15696df8 John Rouillard rouilj@ieee.org 2025-01-16T02:32:58-05:00 2025-01-16T02:32:58-05:00
changeset 669dfccca898
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551391 - checkboxes and radiobutton inputs get wrong id's.

Actually it breaks automatic id assignment for all inputs.

Inputs now get an automatic id assignment that matches the name.
It can be overridden by supplting an id parameter in the call to
the field() method.

This is also a partial fix for issue1513369. I think it obsoletes the
changes to templating.py.
files
fix: issue2551387 - TypeError: not indexable. http://hg.code.sf.net:8000/p/roundup/code/#changeset-05d8806b25ada1d891922f8d091f163af654d6a7 John Rouillard rouilj@ieee.org 2025-01-12T12:34:52-05:00 2025-01-12T12:34:52-05:00
changeset 05d8806b25ad
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: issue2551387 - TypeError: not indexable.

Fix crash due to uninitialized list element on a (Mini)FieldStorage
when unexpected input is posted via wsgi. This doesn't happen when
running roundup-server. It might happen under other front ends.

Moved the code that sets '.list = [] if .list == None' to the main
flow. Added an exception hander that logs the value of self.form if
self.form.list raises an AttributeError. This exception should never
happen if I understand the code correctly (but I probably don't).

Fixed a number of test cases that were broken because I was calling
Client and passing '[]' rather than a cgi.formStorage object.

Added test cases:

create a FileStorage (self.form) with .list = None.

check AttributeError exception and verify logging.

Problem reported and debugged by Christof Meerwald.
files
fix(api): issue2551384. Verify REST authorization earlier http://hg.code.sf.net:8000/p/roundup/code/#changeset-35beff3168834a58b57bea4ac44ac499ba7942ab John Rouillard rouilj@ieee.org 2025-01-09T09:30:08-05:00 2025-01-09T09:30:08-05:00
changeset 35beff316883
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix(api): issue2551384. Verify REST authorization earlier

To reduce the ability of bad actors to spam (DOS) the REST endpoint
with bad data and generate logs meant for debugging, modify the flow
in client.py's REST handler to verify authorization earlier.

If the anonymous user is allowed to use REST, this won't make a
difference for a DOS attempt. The templates don't enable REST for the
anonymous user by default. Most admins don't change this.

The validation order for REST requests has been changed.

CORS identfied an handled
User authorization to use REST (return 403 on failure)
REST request validated (Origin header valid etc.) (return 400 for
bad request)

Incorrectly formatted CORS preflight requests (e.g. missing Origin
header) that are not recogized as a CORS request can now return HTTP
status 403 as well as status 400 (when anonymous is allowed
access). Note all CORS preflights are sent without authentication so
appear as anonymous requests.

The tests were updated to compensate, but it is not obvious to me from
specs what the proper evaulation order/return codes should be for this
case. Both 403/400 are failures and cause CORS to fail so there should
be no difference but...
files
test: fix failing test setup for change in PBKDF2 rounds. http://hg.code.sf.net:8000/p/roundup/code/#changeset-0242cf22ef74fb30db0add07b98b5c87b9a77556 John Rouillard rouilj@ieee.org 2024-12-30T21:17:59-05:00 2024-12-30T21:17:59-05:00
changeset 0242cf22ef74
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description test: fix failing test setup for change in PBKDF2 rounds.
files
fix(web): issue2551356. Add etag header for not-modified (304) request. http://hg.code.sf.net:8000/p/roundup/code/#changeset-e84d4585b16db20b43bc45bb9ef6bf7397124311 John Rouillard rouilj@ieee.org 2024-12-10T16:06:13-05:00 2024-12-10T16:06:13-05:00
changeset e84d4585b16d
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix(web): issue2551356. Add etag header for not-modified (304) request.

When a 304 is returned to a conditional request for a static file,
print an ETag for the response.

ETag was always sent with a 200 response.

This also adds initial support for if-none-match conditional requests
for static files.

Changes:

Refactors the if-modified-since code out to a method.

It moves a file stat call from serve_static_file to _serve_file
so that an etag can be generated by both serve_static_file and
serve_file which call _serve_file.

Tests added. This does not test the codepath where serve_file pulls
content from the database rather than from a local file on disk.

Test mocking _serve_file changed to account for 5th argument to serve_file

BREAKING CHANGE:

function signature for client.py-Client::_serve_file() now has 5 not 4
parameters (added etag param). Since this is a "hidden" method I am
not too worried about it.
files
[permission-performance] Fix failing test due to mokey patching http://hg.code.sf.net:8000/p/roundup/code/#changeset-603aa730b067384c5b94f94453419e6b3e0457b2 Ralf Schlatterbeck rsc@runtux.com 2024-10-22T16:10:29+02:00 2024-10-22T16:10:29+02:00
changeset 603aa730b067
branch permission-performance
bookmark
tag
user Ralf Schlatterbeck <rsc@runtux.com>
description Fix failing test due to mokey patching

An earlier test monkey-patched the permission checks which subsequently
failed in another test.
files
test: correct checksum for changed template http://hg.code.sf.net:8000/p/roundup/code/#changeset-e44b65651012cbd190b73b128caa942e4ba59ad9 John Rouillard rouilj@ieee.org 2024-07-09T09:58:01-04:00 2024-07-09T09:58:01-04:00
changeset e44b65651012
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description test: correct checksum for changed template

A change I made to disable spellcheck in user.item.html caused the
render funxtion to generate a diferent sha1 checksum. The test looks
for the checksum. Update to the new checkum after the change.
files
fix(security): fix CVE-2024-39124, CVE-2024-39124, and CVE-2024-39125 http://hg.code.sf.net:8000/p/roundup/code/#changeset-28aa76443f58bc1605a9933bb4ea4d599c97af5a John Rouillard rouilj@ieee.org 2024-07-09T09:07:09-04:00 2024-07-09T09:07:09-04:00
changeset 28aa76443f58
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix(security): fix CVE-2024-39124, CVE-2024-39124, and CVE-2024-39125

Directions for fixing:

* `CVE-2024-39124`_ - :ref:`classhelpers (_generic.help.html) are
vulnerable to an XSS attack. <CVE-2024-39124>` Requires fixing
tracker homes.
* `CVE-2024-39125`_ - :ref:`if Referer header is set to a script
tag, it will be executed. <CVE-2024-39125>` Fixed in release 2.4.0,
directions available for fixing in prior versions.
* `CVE-2024-39126`_ - :ref:`PDF, XML and SVG files downloaded from
an issue can contain embedded JavaScript which is
executed. <CVE-2024-39126>` Fixed in release 2.4.0, directions
available for fixing in prior versions.

prior to 2.4.0 release this weekend that fixes the last two CVE's.
files
issue2551334 - get test suite running under windows http://hg.code.sf.net:8000/p/roundup/code/#changeset-470616e644149ef8283f020f41f762cf47a8352e John Rouillard rouilj@ieee.org 2024-04-27T23:19:51-04:00 2024-04-27T23:19:51-04:00
changeset 470616e64414
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551334 - get test suite running under windows

Fix test_cgi under windows. Few classes of errors:

1) comparing paths with a/b (unix) to a\b (windows). Use normpath to
fix it. Also change regexps used for path matching to use either
\ or / for directory separators.

2) database not being closed preventing deletion of test case
directory. Windows doesn't allow deletion of open files.
In some cases replaced calling client.inner_mail() with main()
because main() makes sure to close the database. In other cases
assigned self.db = client.db beacuse client reopens the database
and closes the original self.db. Reassigning allows tearDown to
close the last opened handle to a db.

3) commit the admin password to the database. If it's not commited
calling the code sometimes comes up with a different admin
password. Not sure why we don't see this on linux.

4) run commit() on database so that sqlite databases can be closed
and deleted. Unit tests don't call the main entry points that
have finally clauses to close the databases properly, so do it in
the test.

5) split tests that try to resolve symbolic links in the template
directory to a separate method. Under windows user needs special
permissions to creae symbolic links, so I report the method is
skipped if creating a link fails.
files
test: map assertRegexpMatches to assertRegex for python2 http://hg.code.sf.net:8000/p/roundup/code/#changeset-a430339f55e63268fee7a2d907c2df0efa58b1bd John Rouillard rouilj@ieee.org 2024-03-26T15:01:10-04:00 2024-03-26T15:01:10-04:00
changeset a430339f55e6
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description test: map assertRegexpMatches to assertRegex for python2
files
test: add test for bare % in expanded file. http://hg.code.sf.net:8000/p/roundup/code/#changeset-e90be54708e91c65128e46092861c8da0dfd7b16 John Rouillard rouilj@ieee.org 2024-03-26T14:51:50-04:00 2024-03-26T14:51:50-04:00
changeset e90be54708e9
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description test: add test for bare % in expanded file.
files
issue2551270 - Better templating support for JavaScript http://hg.code.sf.net:8000/p/roundup/code/#changeset-219fc5804345cfe3775b2613e96185c719dcc270 John Rouillard rouilj@ieee.org 2024-03-26T14:15:46-04:00 2024-03-26T14:15:46-04:00
changeset 219fc5804345
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551270 - Better templating support for JavaScript

Add (templating) utils.readfile(file, optional=False) and
utils.expandfile(file, token_dict=None, optional=False). Allows
reading an external file (e.g. JavaScript) and inserting it using
tal:contents or equivalent jinja function. expandfile allows setting
a dictionary and tokens in the file of the form "%(token_name)s"
will be replaced in the file with the values from the dict.

See method doc blocks or reference.txt for more info.

Also reordered table in references.txt to be case sensitive
alphabetic. Added a paragraph on using python's help() to get
method/function/... documention blocks.

in templating.py _find method. Added explicit return None calls to all
code paths. Also added internationalization method to the
TemplatingUtils class. Fixed use of 'property' hiding python builtin
of same name.

Added tests for new TemplatingUtils framework to use for testing existing
utils.
files
fix: issue2551193 - Fix roundup for removal of cgi and cgitb ... http://hg.code.sf.net:8000/p/roundup/code/#changeset-978285986b2c9d73f4a39d745ab6a3bbab58af3d John Rouillard rouilj@ieee.org 2023-07-24T17:49:58-04:00 2023-07-24T17:49:58-04:00
changeset 978285986b2c
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: issue2551193 - Fix roundup for removal of cgi and cgitb ...

standard python modules (and FieldStorage/MiniFieldStorage).

Vendor cgi.py and modify imports.

Details:

roundup/anypy/cgi_.py
import that accesses a working cgi.py. All imports dealing with cgi
now use cgi_.

roundup/anypy/vendored/cgi.py
vendored version 2.6 of cgi.py from:
https://pypi.org/project/legacy-cgi/

CHANGES.txt
change note added

COPYING.txt
added license for cgi.py

doc/rest.txt
change example to use cgi_

doc/upgrading.txt
doc removal and how to rework local code using cgi.py.

frontends/roundup.cgi
remove unneeded cgi import

roundup/cgi/actions.py
roundup/cgi/apache.py
roundup/cgi/client.py
roundup/cgi/templating.py
roundup/cgi/TAL/TALGenerator.py
test/db_test_base.py
test/rest_common.py
test/test_cgi.py
remove import cgi and replace with from roundup.anypy.cgi_ import
cgi

test/test_actions.py
test/test_templating.py
modify import to get *FieldStorage

test/test_admin.py
test/test_hyperdbvals.py
test/test_xmlrpc.py
remove unneeded cgi import
files
Fix round check/settings in needs_migration http://hg.code.sf.net:8000/p/roundup/code/#changeset-8b2287d850c8f5d73aeb27bb499d3d5847bb3740 John Rouillard rouilj@ieee.org 2023-02-26T16:17:37-05:00 2023-02-26T16:17:37-05:00
changeset 8b2287d850c8
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Fix round check/settings in needs_migration

Support test rounds in needs_migration

Two test were missing os.environ seting to have them use config
setting.
files
issue2551251 - migrate pbkdf2 passwords ... test fixes and doc update http://hg.code.sf.net:8000/p/roundup/code/#changeset-1549c7e74ef842f2f22ed5a0aecc0c9cc9edfd27 John Rouillard rouilj@ieee.org 2023-02-24T23:47:28-05:00 2023-02-24T23:47:28-05:00
changeset 1549c7e74ef8
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551251 - migrate pbkdf2 passwords ... test fixes and doc update

Fixed a couple of tests where calls to needs_migration() was missing
its config parameter.

Documented need to update config.ini's password_pbkdf2_default_rounds.
files
Fix test failure when run alone. http://hg.code.sf.net:8000/p/roundup/code/#changeset-5487882ff17ada07593d5f38fbdaef2e410b6673 John Rouillard rouilj@ieee.org 2023-02-24T17:15:29-05:00 2023-02-24T17:15:29-05:00
changeset 5487882ff17a
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Fix test failure when run alone.

testPasswordMigration was failing when run pytest -k
testPasswordMigration. Was always failing on third test.
When run as part of whole FormTestCase class it worked.

Fixed it by:

Using cl.db after call to actions.LoginAction.

Make sure to set cl.db.Otk = self.db.Otk before calling LoginAction.

Add real Otk database and not a mock to a couple of db's.

Without this the third element in the loop (crypt at this point) still
has the old vale when eading from db. I think this is due to the
original cl.db being closed inside LoginAction (to re-open db using
new user's credentials cl.db.ptimize is not set). I think the old
self.db is returning the cached values. At least that's my guess.
files
Add 'Access-Control-Expose-Headers' to a couple of tests. http://hg.code.sf.net:8000/p/roundup/code/#changeset-ed63b6d358387124ed4195d15ab020b6ca63bf99 John Rouillard rouilj@ieee.org 2023-02-23T17:20:41-05:00 2023-02-23T17:20:41-05:00
changeset ed63b6d35838
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Add 'Access-Control-Expose-Headers' to a couple of tests.
files
improve REST interface security http://hg.code.sf.net:8000/p/roundup/code/#changeset-89a59e46b3af7bb327ebf079c34e22bbbff8bbf2 John Rouillard rouilj@ieee.org 2023-02-23T12:01:33-05:00 2023-02-23T12:01:33-05:00
changeset 89a59e46b3af
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description improve REST interface security

When using REST, we reflect the client's origin. If the wildcard '*'
is used in allowed_api_origins all origins are allowed. When this is
done, it also added an 'Access-Control-Allow-Credentials: true'
header.

This Credentials header should not be added if the site is matched
only by '*'. This header should be provided only for explicit origins
(e.g. https://example.org) not for the wildcard.

This is now fixed for CORS preflight OPTIONS request as well as normal
GET, PUT, DELETE, POST, PATCH and OPTIONS requests.

A missing Access-Control-Allow-Credentials will prevent the tracker
from being accessed using credentials. This prevents an unauthorized
third party web site from using a user's credentials to access
information in the tracker that is not publicly available.

Added test for this specific case.

In addition, allowed_api_origins can include explicit origins in
addition to '*'. '*' must be first in the list.

Also adapted numerous tests to work with these changes.

Doc updates.
files
fix test; string for json object has extra space under python2. http://hg.code.sf.net:8000/p/roundup/code/#changeset-f614176903d00863b33011f6ecfce511de4335e8 John Rouillard rouilj@ieee.org 2023-02-21T23:06:15-05:00 2023-02-21T23:06:15-05:00
changeset f614176903d0
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix test; string for json object has extra space under python2.

So compensate by comparing json parsed into objects and compensate
with a different content-length between py2 and py3..
files
Refactor rejecting requests; update tests, xfail test http://hg.code.sf.net:8000/p/roundup/code/#changeset-1181157d7ceca212bcbe865f9df6cbfced40918d John Rouillard rouilj@ieee.org 2023-02-21T22:35:58-05:00 2023-02-21T22:35:58-05:00
changeset 1181157d7cec
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Refactor rejecting requests; update tests, xfail test

Added new Client::reject_request method. Deployed throughout
handle_rest() method.

Fix tests to compensate for consistent formatting of errors.

Mark testRestOriginValidation test xfail. Code needed to implement it
fully is only partly written.

Tests for OPTIONS request on a bad attribute and valid and invalid
origin tests added.
files
issue2551203 - Add support for CORS preflight request http://hg.code.sf.net:8000/p/roundup/code/#changeset-9a1f5e496e6c4dbd311434a9636bb3ffa20b01d6 John Rouillard rouilj@ieee.org 2022-06-07T09:39:35-04:00 2022-06-07T09:39:35-04:00
changeset 9a1f5e496e6c
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551203 - Add support for CORS preflight request

Add support for unauthenticated CORS preflight and fix headers for
CORS.

client.py:

pass through unauthenticated CORS preflight to rest backend. Normal
rest OPTION handlers (including tracker defined extensions) can
see and handle the request.

make some error cases return error json with crrect mime type rather
than plain text tracebacks.

create new functions to verify origin and referer that filter using
allowed origins setting.

remove tracker base url from error message is referer is not at an
allowed origin.

rest.py:

fix up OPTION methods handlers to include
Access-Control-Allow-Methods that are the same as the Allow
header.

set cache to one week for all Access-Control headers for CORS
preflight only.

remove self.client.setHeader("Access-Control-Allow-Origin", "*") and
set Access-Control-Allow-Origin to the client supplied origin if
it passes allowed origin checks. Required for CORS otherwise data
isn't available to caller. Set for all responses.


set Vary header now includes Origin as responses can differ based on
Origin for all responses.

set Access-Control-Allow-Credentials to true on all responses.

test_liveserver.py:

run server with setting to enforce origin csrf header check

run server with setting to enforce x-requested-with csrf header check

run server with setting for allowed_api_origins

requests now set required csrf headers

test preflight request on collections

check new headers and Origin is no longer '*'


rewrite all compression checks to use a single method with argument
to use different compression methods. Reduce a lot of code
duplication and makes updating for new headers easier.


test_cgi:

test new error messages in client.py

account for new headers

test preflight and new code paths
files
issue2551205: Add support for specifying valid origins for api: xmlrpc/rest http://hg.code.sf.net:8000/p/roundup/code/#changeset-ab2ed11c021eec91ca1a624ee4ee259e2282741d John Rouillard rouilj@ieee.org 2022-05-17T17:18:51-04:00 2022-05-17T17:18:51-04:00
changeset ab2ed11c021e
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551205: Add support for specifying valid origins for api: xmlrpc/rest

We now have an allow list to filter the hosts allowed to do api
requests. An element of this allow list must match the http ORIGIN
header exactly or the rest/xmlrpc CORS request will result in an
error.

The tracker host is always allowed to do a request.
files
Fix modification of Cache_Control http://hg.code.sf.net:8000/p/roundup/code/#changeset-da6c9050a79e2bc68bcb21f6919306ac9f46a674 Ralf Schlatterbeck rsc@runtux.com 2022-05-02T14:25:15+02:00 2022-05-02T14:25:15+02:00
changeset da6c9050a79e
branch
bookmark
tag
user Ralf Schlatterbeck <rsc@runtux.com>
description Fix modification of Cache_Control

The previous code modified the Cache_Control dictionary in the Client
class (in one of the cgi tests) which made a later test in
test_liveserver fail
files
Add date column to CSV output - test date display code. http://hg.code.sf.net:8000/p/roundup/code/#changeset-154f286061e21737e390080459b68b57e602b84d John Rouillard rouilj@ieee.org 2022-01-26T17:30:59-05:00 2022-01-26T17:30:59-05:00
changeset 154f286061e2
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Add date column to CSV output - test date display code.

Test date formatting code. Uses date.Date mock/override so creation
date is reproducible.
files
Fix csv export with text search. test csv export Sqlite FTS syntax error http://hg.code.sf.net:8000/p/roundup/code/#changeset-65336409738c7c63fe9fd6fd767e398f75b405cc John Rouillard rouilj@ieee.org 2022-01-26T16:48:06-05:00 2022-01-26T16:48:06-05:00
changeset 65336409738c
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Fix csv export with text search. test csv export Sqlite FTS syntax error
files
issue2551189 - increase size of words in full text index. http://hg.code.sf.net:8000/p/roundup/code/#changeset-39189dd94f2cc9d0b460685b20b3fc077d59f7ec John Rouillard rouilj@ieee.org 2022-01-26T15:04:09-05:00 2022-01-26T15:04:09-05:00
changeset 39189dd94f2c
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551189 - increase size of words in full text index.

Increased indexed word maxlength to 50

DB migration code is written and tests work.

Restructured some tests to allow for code reuse.

Docs.

If this passes CI without errors 2551189 should be done. However,
testing on my system generates errors. Encoding (indexer unicode
russian unicode string invalid) and collation errors (utf8_bin not
valid) when running under python2. No issues with python3 and I
haven't changed code that should cause these since the last successful
build in CI. So if this fails in CI we will have more checkins.
files
issue2551189 - increase text search maxlength http://hg.code.sf.net:8000/p/roundup/code/#changeset-e70e2789bc2c7a554e59d54918bf31d0a2b588a5 John Rouillard rouilj@ieee.org 2022-01-25T13:22:00-05:00 2022-01-25T13:22:00-05:00
changeset e70e2789bc2c
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551189 - increase text search maxlength

This removes I think all the magic references to 25 and 30 (varchar
size) and replaces them with references to maxlength or maxlength+5.

I am not sure why the db column is 5 characters larger than the size
of what should be the max size of a word, but I'll keep the buffer
of 5 as making it 1/5 the size of maxlength makes less sense.

Also added tests for fts search in templating which were missing.
Added postgres, mysql and sqlite native indexing backends in which to
test fts. Added fts test to native-fts as well to make sure it's
working.

I want to commit this now for CI.

Todo:

add test cases for the use of FTS in the csv output in
actions.py. There is no test coverage of the match case there.

change maxlength to a higher value (50) as requested in the ticket.

Modify existing extremewords test cases to allow words > 25 and < 51

write code to migrate column sizes for mysql and postgresql to match
maxlength I will roll this into the version 7 schema update that
supports use of database fts support.
files
Summary: Add test cases for sqlite fts http://hg.code.sf.net:8000/p/roundup/code/#changeset-91ab3e0ffcd0d5550274ffda207241843ee6a545 John Rouillard rouilj@ieee.org 2022-01-23T18:57:45-05:00 2022-01-23T18:57:45-05:00
changeset 91ab3e0ffcd0
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Summary: Add test cases for sqlite fts

Add support for using the FTS5 full text query engine for sqlite.

Also stubbed out some sections for adding postgresql FTS support as
well.

Added nee indexer type native-fts. It is not selected by default. The
indexer=native is used if no indexer is set. This prevents an upgrade
from seeming to wipe out the native index if upgraded and
indexer=native is not explicitly set.

Docs updated. Also changed section headers to sentence case for the
current release notes.

Indexing backend can control if the full text search phrase is broken
into a list of words or passed intact. For backends with query
languages (sqlite and can be enabled for whoosh and xapian) we do not
want the phrase "tokenized" on whitespace.

This also updates the rdbms database version to version 7 to add FTS
table. I will be using the same version when I add postgresql. If
somebody runs this version on postgresql, they will have to manually
add the fts tables for postgresql if they want to use it.

Added a new renderError method to client. This allows errors to be
reported still using page.html rather than raw html. It also supports
templates for any error code. If no template for the error code
(e.g. 400) is found, the error in raw html with no page frame is
shown.

New IndexerQueryError exception to pass back message about query syntax
errors.
files
Ignore blank lines when editing class via CSV http://hg.code.sf.net:8000/p/roundup/code/#changeset-ada96db8ec62a59631d71a1a5d1603efb72234d0 John Rouillard rouilj@ieee.org 2021-06-11T18:09:21-04:00 2021-06-11T18:09:21-04:00
changeset ada96db8ec62
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Ignore blank lines when editing class via CSV
files
Fix issue2551129 - Template not found return 500 and traceback http://hg.code.sf.net:8000/p/roundup/code/#changeset-b35a50d0289086d5d2c4e23257395d9dec5bd47d John Rouillard rouilj@ieee.org 2021-04-18T20:54:48-04:00 2021-04-18T20:54:48-04:00
changeset b35a50d02890
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Fix issue2551129 - Template not found return 500 and traceback

Handle traceback caused when requested @template is not found.
Moved scope of try to include call to self.selectTemplate.
Patch provided by Cedric Krier.

Additional patch to make this case return 400 error since it
is a client caused error. Test case added.
files
Move mocknull from test to roundup/test http://hg.code.sf.net:8000/p/roundup/code/#changeset-f2c31f5ec50bd58de8d268a70f161fd9bf69d074 Ralf Schlatterbeck rsc@runtux.com 2021-03-31T15:18:38+02:00 2021-03-31T15:18:38+02:00
changeset f2c31f5ec50b
branch
bookmark
tag
user Ralf Schlatterbeck <rsc@runtux.com>
description Move mocknull from test to roundup/test
files
Fix roundup/test http://hg.code.sf.net:8000/p/roundup/code/#changeset-58817c3bf47111b723b3fca5ef988f5ba9a54f8a Ralf Schlatterbeck rsc@runtux.com 2021-03-30T14:16:28+02:00 2021-03-30T14:16:28+02:00
changeset 58817c3bf471
branch
bookmark
tag
user Ralf Schlatterbeck <rsc@runtux.com>
description Fix roundup/test

Move the test-detectors in tx_Source_detector.py to roundup/test for two
reasons: It's used in the memorydb convenience functions and it may be
useful in other tests. Make the prefix a paramter of the convenience
functions to be usable in other tests.
files
Replace http:....roundup-tracker.org with https. http://hg.code.sf.net:8000/p/roundup/code/#changeset-bdcccd2b21413e6897d91c9c72173e8aad9d869e John Rouillard rouilj@ieee.org 2020-09-09T21:37:16-04:00 2020-09-09T21:37:16-04:00
changeset bdcccd2b2141
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Replace http:....roundup-tracker.org with https.
files
Quote all exported CSV data http://hg.code.sf.net:8000/p/roundup/code/#changeset-15fd91fd3c4c316bcf2db75c9a019f7836f5ec07 John Rouillard rouilj@ieee.org 2020-06-08T16:18:21-04:00 2020-06-08T16:18:21-04:00
changeset 15fd91fd3c4c
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Quote all exported CSV data

Quote all non-numeric data in csv export functions. Report that a
title like '=a2+b3' could be interpreted as a function in Excel and
executed. csv.writer now includes quoting=csv.QUOTE_NONNUMERIC to
generate quoted values for all fields. This should make the string
starting with = be interpreted as a string and not a formula.
files
issue2551019 needs to be handled in the action code itself, not the WSGI handler http://hg.code.sf.net:8000/p/roundup/code/#changeset-f74d078cfd9adbeb3164545dedeff0602fdfa973 Christof Meerwald cmeerw@cmeerw.org 2020-02-08T00:29:13+00:00 2020-02-08T00:29:13+00:00
changeset f74d078cfd9a
branch
bookmark
tag
user Christof Meerwald <cmeerw@cmeerw.org>
description issue2551019 needs to be handled in the action code itself, not the WSGI handler
for Python 3 we always need to encode the output in the client character set
files
Support setting cache-control headers for static files http://hg.code.sf.net:8000/p/roundup/code/#changeset-54d0080769f95ebb5cfb4ad728578f280796d361 John Rouillard rouilj@ieee.org 2019-11-21T20:50:56-05:00 2019-11-21T20:50:56-05:00
changeset 54d0080769f9
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Support setting cache-control headers for static files

Control how to cache static files. Can control by mime type or
filename. Needs to use interfaces.py mechanism to configure.
See customization.txt file in the interfaces.py section.

Also added docs for using interfacs.py and a few examples.
files
Clean up SENDMAILDEBUG in test case. http://hg.code.sf.net:8000/p/roundup/code/#changeset-fefdf5f97c50ae25d775f8efc5b58ed26e62bf90 John Rouillard rouilj@ieee.org 2019-11-09T16:53:42-05:00 2019-11-09T16:53:42-05:00
changeset fefdf5f97c50
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Clean up SENDMAILDEBUG in test case.
files
- issue2550920 - Optionally detect duplicate username at registration. http://hg.code.sf.net:8000/p/roundup/code/#changeset-71c68961d9f4171f782b87a462347313a123967d John Rouillard rouilj@ieee.org 2019-11-09T16:33:42-05:00 2019-11-09T16:33:42-05:00
changeset 71c68961d9f4
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description - issue2550920 - Optionally detect duplicate username at registration.

Added config option to allow detection of duplicate username when the
user tries to register. Previously user was rejected when dupliate
name found at confirmation step.

Optional as it can make username guessing easier.

Testing is in place for this.

Also attempted to make the unfriendly error message:

'node with key "username" exists'

into a translatable friendly error:

"Username 'username' already exists."

This is missing any test. It is also fragile as I capture the
ValueError exception and see that the exception matches:

'node with key "username" exists'

If it does reassert the friendly message. Otherwise just re-raise
existing exception. If the "node with key..." message is translated
the friendly override will not trigger.
files
issue2550919 - Anti-bot signup using 4 second delay http://hg.code.sf.net:8000/p/roundup/code/#changeset-fe334430ca07c8d9d45af20bbfb52e1fdd7f6fb4 John Rouillard rouilj@ieee.org 2019-11-09T00:30:37-05:00 2019-11-09T00:30:37-05:00
changeset fe334430ca07
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2550919 - Anti-bot signup using 4 second delay

Took the code by erik forsberg and massaged it into the core.

So this is no longer needed in the tracker.

Updated devel and responsive trackers to remove timestamp.py and
update input field name.

Docs, changes and tests complete. Hopefully these tracker changes
won't cause an issue for other tests.
files
issue2550925 strip HTTP_PROXY environment variable http://hg.code.sf.net:8000/p/roundup/code/#changeset-b40059d7036f03bf3a802cae4564217a723962c5 John Rouillard rouilj@ieee.org 2019-10-13T17:45:06-04:00 2019-10-13T17:45:06-04:00
changeset b40059d7036f
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2550925 strip HTTP_PROXY environment variable

if deployed as CGI and client sends an http PROXY
header, the tainted HTTP_PROXY environment variable is created. It
can affect calls using requests package or curl. A roundup admin
would have to write detectors/extensions that use these mechanisms.
Not exploitable in default config.

See: https://httpoxy.org/
files
Change microcopy for missing csrf to follow mismatched csrf. Fix tests. http://hg.code.sf.net:8000/p/roundup/code/#changeset-26cd8e8bbed3beaea558ac0559efca4f6e10fdb1 John Rouillard rouilj@ieee.org 2019-07-15T20:59:12-04:00 2019-07-15T20:59:12-04:00
changeset 26cd8e8bbed3
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Change microcopy for missing csrf to follow mismatched csrf. Fix tests.
files
More extensive EditCSV testing. http://hg.code.sf.net:8000/p/roundup/code/#changeset-bd6d41f21a5afc675b8f51eebba1887c9f477adb John Rouillard rouilj@ieee.org 2019-06-14T21:26:19-04:00 2019-06-14T21:26:19-04:00
changeset bd6d41f21a5a
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description More extensive EditCSV testing.
Uses test object to cover all types defined by database.
Also check for an error with a row that's too short.
CSV starts with blank line to verify that it recognises header line.
Changes password, date, int, link and other fields.
Deletes row and verified that it is retired.

Make actions.py properly handle emptying of password field.
files
Extra test of < and > inside quotes. http://hg.code.sf.net:8000/p/roundup/code/#changeset-39a5f40ae4d47f54c2c0b674edc6d130f9bd2c5f John Rouillard rouilj@ieee.org 2019-06-12T17:26:19-04:00 2019-06-12T17:26:19-04:00
changeset 39a5f40ae4d4
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Extra test of < and > inside quotes.
files
Replace deprecated assertEquals with assertEqual and failUnlessRaises http://hg.code.sf.net:8000/p/roundup/code/#changeset-95a366d460654c841c28b1d7d6ba45cda60b1779 John Rouillard rouilj@ieee.org 2019-06-10T21:27:17-04:00 2019-06-10T21:27:17-04:00
changeset 95a366d46065
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Replace deprecated assertEquals with assertEqual and failUnlessRaises
with assertRaises.
files
Replace assertEquals (depricated) with assertEqual. http://hg.code.sf.net:8000/p/roundup/code/#changeset-68b0c1767b50c4e892ac7250ac057c2e725c9586 John Rouillard rouilj@ieee.org 2019-06-10T20:14:14-04:00 2019-06-10T20:14:14-04:00
changeset 68b0c1767b50
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Replace assertEquals (depricated) with assertEqual.
files
Got an xpass on testPasswordMigration, so removing xfail marker. http://hg.code.sf.net:8000/p/roundup/code/#changeset-3f00269f329759308f69a3179d4e0f3e46bea662 John Rouillard rouilj@ieee.org 2019-06-07T11:35:55-04:00 2019-06-07T11:35:55-04:00
changeset 3f00269f3297
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Got an xpass on testPasswordMigration, so removing xfail marker.
Not sure what got fixed, but travis is reporting the same.
files
Mark the failing test I deactivated as xfail to make it easier for http://hg.code.sf.net:8000/p/roundup/code/#changeset-abb9fdb02228587449346ea176489cd79c31ad89 John Rouillard rouilj@ieee.org 2019-05-11T21:22:48-04:00 2019-05-11T21:22:48-04:00
changeset abb9fdb02228
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Mark the failing test I deactivated as xfail to make it easier for
people to run the test and try to debug it.
files
Deactivate failing test till I can get somebody to look at it. I want http://hg.code.sf.net:8000/p/roundup/code/#changeset-071625b5b7c07543601b06ec474d92173f7a811f John Rouillard rouilj@ieee.org 2019-05-11T20:53:06-04:00 2019-05-11T20:53:06-04:00
changeset 071625b5b7c0
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Deactivate failing test till I can get somebody to look at it. I want
the code coverage stats to look at for my changes and I can't get that
while tests are failing.

This test loops over 4 depricated encryption schemes. It looks like it
fails on whatever encryption item is in the third location. When
looking at the action code, it looks like on the third iteration, it
is getting the replacement properly encrypted string, so it doesn't do
a change to the database. However the test harness is still seeing the
badly encrypted string. I don't know is there is a cache issue or
something here.
files
Add test for @pretty=false format change. http://hg.code.sf.net:8000/p/roundup/code/#changeset-92c1864d5dd2fe644854d4d0afb96afc7252bcaa John Rouillard rouilj@ieee.org 2019-04-10T18:04:36-04:00 2019-04-10T18:04:36-04:00
changeset 92c1864d5dd2
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Add test for @pretty=false format change.
files
Change output comparison from strings to comparison on python http://hg.code.sf.net:8000/p/roundup/code/#changeset-f90a534cb1127316a33517e761964271ff149ac8 John Rouillard rouilj@ieee.org 2019-04-08T22:45:47-04:00 2019-04-08T22:45:47-04:00
changeset f90a534cb112
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Change output comparison from strings to comparison on python
dicts. String representation of the json data had fields in different
order causing failure for python3 compared to python2.
files
Adding tests for csrf protection for rest. Also test disabling of http://hg.code.sf.net:8000/p/roundup/code/#changeset-b1ab8bd18e797c758bf404d57c93e6b9269a5363 John Rouillard rouilj@ieee.org 2019-04-08T21:46:21-04:00 2019-04-08T21:46:21-04:00
changeset b1ab8bd18e79
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Adding tests for csrf protection for rest. Also test disabling of
pretty printing.
files
python2/python3 normalization. When exporting CSV, sort lists as they http://hg.code.sf.net:8000/p/roundup/code/#changeset-9689d1bf9bb023d72e40a4fa4ba9ec1d944659a2 John Rouillard rouilj@ieee.org 2019-03-17T19:00:43-04:00 2019-03-17T19:00:43-04:00
changeset 9689d1bf9bb0
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description python2/python3 normalization. When exporting CSV, sort lists as they
are ordered differently in python3 vs python2. Also python 3 list
element order seems to not be stable/repeatable between runs. Sometimes
the tests would pass sometimes they wouldn't.
files
assert_ is depricated. Replacing with assertTrue to reduce logs in travisci. http://hg.code.sf.net:8000/p/roundup/code/#changeset-f8893e1cde0d05513a328bd4698dac10d94805b3 John Rouillard rouilj@ieee.org 2019-03-15T20:29:00-04:00 2019-03-15T20:29:00-04:00
changeset f8893e1cde0d
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description assert_ is depricated. Replacing with assertTrue to reduce logs in travisci.
files
[maint-1.6] issue2551023: Fix CSRF headers for use with wsgi and cgi. The http://hg.code.sf.net:8000/p/roundup/code/#changeset-8e3df461d316944b1ab0eafc431307ecc4973b3f John Rouillard rouilj@ieee.org 2019-02-27T21:47:39-05:00 2019-02-27T21:47:39-05:00
changeset 8e3df461d316
branch maint-1.6
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551023: Fix CSRF headers for use with wsgi and cgi. The
env variable array used - separators rather than _. Compare:
HTTP_X-REQUESTED-WITH to HTTP_X_REQUESTED_WITH. The last is
correct. Also fix roundup-server to produce the latter form. (Patch
by Cédric Krier)
files
issue2551023: Fix CSRF headers for use with wsgi and cgi. The http://hg.code.sf.net:8000/p/roundup/code/#changeset-b3618882f9063f761c6df2ca6f31e164350314ef John Rouillard rouilj@ieee.org 2019-02-27T21:47:39-05:00 2019-02-27T21:47:39-05:00
changeset b3618882f906
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551023: Fix CSRF headers for use with wsgi and cgi. The
env variable array used - separators rather than _. Compare:
HTTP_X-REQUESTED-WITH to HTTP_X_REQUESTED_WITH. The last is
correct. Also fix roundup-server to produce the latter form. (Patch
by Cédric Krier)
files
issue2550833 enhance the export csv action to include the keys for http://hg.code.sf.net:8000/p/roundup/code/#changeset-be99aa02c6167d88e1a9220a31abb78ae00d967b John Rouillard rouilj@ieee.org 2019-02-16T15:17:21-05:00 2019-02-16T15:17:21-05:00
changeset be99aa02c616
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2550833 enhance the export csv action to include the keys for
liked items rather than id's. So for nosy list display usernames and
not numbers.

The original code was renamed and also made available. See change
document.
files