http://hg.code.sf.net:8000/p/roundup/code/atom-log/tip/doc/upgrading.txt Mercurial Repository: p/roundup/code: doc/upgrading.txt history 2026-04-09T00:19:06-04:00 [default] doc: issue2551415 correct doc for change input->input_payload http://hg.code.sf.net:8000/p/roundup/code/#changeset-e4191aa7b402fd9aaf43be61c9a3eb7fd6514df8 John Rouillard rouilj@ieee.org 2026-04-09T00:19:06-04:00 2026-04-09T00:19:06-04:00
changeset e4191aa7b402
branch default
bookmark
tag tip
user John Rouillard <rouilj@ieee.org>
description doc: issue2551415 correct doc for change input->input_payload

in 2.5 the rest interface changed a variable name from input to
input_payload. An earlier commit changed the rest docs. This commit
adds an item for it to the upgrading 2.4.0->2.5.0 section. Also cross
reference added to the rest docs with the updated examples.
files
feature: add detector that prevents file content changes by Admin and other users. http://hg.code.sf.net:8000/p/roundup/code/#changeset-e738377b4ffee045c89cabbc722aa7b01e2f1ade John Rouillard rouilj@ieee.org 2026-03-24T22:11:27-04:00 2026-03-24T22:11:27-04:00
changeset e738377b4ffe
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feature: add detector that prevents file content changes by Admin and other users.

New detector and upgrading announcement. Also example of stripping
content editing from the User role.
files
refactor: rework mime type comparison and clean code http://hg.code.sf.net:8000/p/roundup/code/#changeset-1ffa1f42e1da553ec00de19d45536262bdf3577d John Rouillard rouilj@ieee.org 2026-03-24T21:30:47-04:00 2026-03-24T21:30:47-04:00
changeset 1ffa1f42e1da
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description refactor: rework mime type comparison and clean code

rest.py:

accept application/* as match for application/json in non
/binary_context rest path.

allow defining default mime type to return when file/message is
missing mime type. Make it a class variable to it can be changed from
text/plain to text/markdown or whatever.

extract code from determine_output_format() to create
create_valid_content_types() method which returns a list of matching
mime types for a given type/subtype.

Eliminate mostly duplicate return statements by introducing a variable
to specify valid mime types in error message.

rest_common.py:

Fix error messages that now return application/* as valid mime type.

CHANGES.txt upgrading.txt rest.txt:

top level notes and corrections.

Also correct rst syntax on earlier change.
files
bug: make web page follow login_empty_passwords setting. http://hg.code.sf.net:8000/p/roundup/code/#changeset-4184173d364f23d0e319e8c50ed587882416e600 John Rouillard rouilj@ieee.org 2026-03-18T17:49:16-04:00 2026-03-18T17:49:16-04:00
changeset 4184173d364f
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description bug: make web page follow login_empty_passwords setting.

remove the required attribute from password input in the the html
templates if login_empty_passwords is enabled in config.ini.

Also document in upgrading.txt.
files
bug: Issue2551393 - keep search name when paginating http://hg.code.sf.net:8000/p/roundup/code/#changeset-00aec15117c00105d298ad945616e4de9dd93c36 John Rouillard rouilj@ieee.org 2026-01-27T21:07:02-05:00 2026-01-27T21:07:02-05:00
changeset 00aec15117c0
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description bug: Issue2551393 - keep search name when paginating

Title: Named searches lose their name in title when next page is selected.

Include dispname in next/prev(ious) URL pagination links in index
pages if dispname is defined.
files
doc: initial attempt to document setup of pgp support for email. http://hg.code.sf.net:8000/p/roundup/code/#changeset-ed4ef394d5d6dac7dbd7a363d958912c64ce7ca4 John Rouillard rouilj@ieee.org 2025-11-15T16:59:24-05:00 2025-11-15T16:59:24-05:00
changeset ed4ef394d5d6
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: initial attempt to document setup of pgp support for email.

Used an AI assistant to help write this. Basic gpg commands seem to
work, but I have not tested this totally. Docs basically follow the
setup used for pgp testing in the test suite.

It looks like roundup accepts signed emails as well as encrypted
and signed emails. But it does not generate signed emails.

Also it looks like there is no PGP support for alternate email
addresses. Only primary addresses can do PGP emails.
files
fix: update updating.txt doc for user_src_input bug http://hg.code.sf.net:8000/p/roundup/code/#changeset-db435e272f262a282d8335c7890a4d49fbb5f669 John Rouillard rouilj@ieee.org 2025-09-26T16:08:30-04:00 2025-09-26T16:08:30-04:00
changeset db435e272f26
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: update updating.txt doc for user_src_input bug

I originally thought it was not worth documenting because the error
case had no impact. But deployed copies of the trackers can be updated
to reduce differences between deployed and distributed files.
files
feature: add thread local trace_id and trace_reason to logging. http://hg.code.sf.net:8000/p/roundup/code/#changeset-14c7c07b32d846e405260147a3112e8c8b15642a John Rouillard rouilj@ieee.org 2025-09-16T22:53:00-04:00 2025-09-16T22:53:00-04:00
changeset 14c7c07b32d8
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feature: add thread local trace_id and trace_reason to logging.

Added trace_id to default logging so that all logs for a given request
share the same trace_id.

This allows correlation of logs across a request.

admin_guide.txt, upgrading.txt:
add docs
update sample configs to include trace_id.
rewrite logging docs in admin_guide. Hopefully they are clearer now.
clean up some stuff in the logging config file docs.

admin.py:

add decorators to run_command to enable trace_id.
change calls to db.commit() to use run_command to get trace_id.

configuration.py:

clean up imports.
update docstrings, comments and inline docs.
add trace_id to default log format.
add function for testing decorated with trace_id.
add support for dumping stack trace in logging.
add check for pytest in sys.modules to enable log propagation when
pytest is running. Otherwise tests fail as the caplog logger doesn't
see the roundup logs.

logcontext.py:

new file to handle thread local contextvar mangement.

mailgw.py:

add decorators for trace_id etc.

scripts/roundup_xlmrpc_server.py:

add decorators for trace_id etc.
fix encoding bug turning bytes into a string.
fix command line issue where we can't set encoding. (not sure if
changing encoding via command line even works)

cgi/client.py

decorate two entry points for trace_id etc.

cgi/wsgi_handler.py:

decorate entry point for trace_id etc.

test/test_config.py:

add test for trace_id in new log format.
test various cases for sinfo and errors in formating msg.
files
doc: add disable saving roundup-admin history file for password changes http://hg.code.sf.net:8000/p/roundup/code/#changeset-7f7749d86da8bac05311974e6bb34cb8373f1be6 John Rouillard rouilj@ieee.org 2025-08-25T20:44:42-04:00 2025-08-25T20:44:42-04:00
changeset 7f7749d86da8
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: add disable saving roundup-admin history file for password changes
files
doc: reformat markdown-note footnote http://hg.code.sf.net:8000/p/roundup/code/#changeset-a6c41651f5530f25f7f6a81e1e0d7e100ce96fef John Rouillard rouilj@ieee.org 2025-08-25T20:32:14-04:00 2025-08-25T20:32:14-04:00
changeset a6c41651f553
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: reformat markdown-note footnote
files
feat: add support for using dictConfig to configure logging. http://hg.code.sf.net:8000/p/roundup/code/#changeset-94eed885e9582203be3a66fa5d0c0c122cbaaa84 John Rouillard rouilj@ieee.org 2025-08-19T22:32:46-04:00 2025-08-19T22:32:46-04:00
changeset 94eed885e958
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat: add support for using dictConfig to configure logging.

Basic logging config (one level and one output file non-rotating) was
always possible from config.ini. However the LOGGING_CONFIG setting
could be used to load an ini fileConfig style file to set various
channels (e.g. roundup.hyperdb) (also called qualname or tags) with
their own logging level, destination (rotating file, socket,
/dev/null) and log format.

This is now a deprecated method in newer logging modules. The
dictConfig format is preferred and allows disabiling other loggers as
well as invoking new loggers in local code. This commit adds support
for it reading the dict from a .json file. It also implements a
comment convention so you can document the dictConfig.

configuration.py:
new code

test_config.py:
test added for the new code.

admin_guide.txt, upgrading.txt CHANGES.txt:
docs added upgrading references the section in admin_guid.
files
[reauth-confirm_id] feat: finish reauth docs, enhance code. http://hg.code.sf.net:8000/p/roundup/code/#changeset-0663a7bcef6c1ac3ff2d11bdbcff893231151377 John Rouillard rouilj@ieee.org 2025-08-13T23:52:49-04:00 2025-08-13T23:52:49-04:00
changeset 0663a7bcef6c
branch reauth-confirm_id
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat: finish reauth docs, enhance code.

Decided to keep name Reauth for now.

admin_guide.txt:
add reference mark to roundup admin help. Used for template command
reference in upgrading.txt.

customizing.txt:
added worked example of adding a reauth auditor for address and password.
Also links to OWASP recommendations.

Added link to example code in design.doc on detectors.

glossary.txt:
reference using roundup-admin template command in def for tracker
templates.

pydoc.txt:
Added methods for Client class.

Added class and methods for (cgi) Action, LoginAction and ReauthAction.

reference.txt
Edited and restructured detector section.

Added section on registering a detector and priority use/execution order.
(reference to design doc was used before).

Added/enhanced description of exception an auditor can
raise (includes Reauth).

Added section on Reauth implementation and use (Confirming the User).
Also has paragraph on future ideas.

upgrading.txt
Stripped down the original section. Moved a lot to reference.txt.

Referenced customizing example, mention installation of
_generic.reauth.html and reference reference.txt.

cgi/actions.py:
fixed bad ReST that was breaking pydoc.txt processing

changed doc on limitations of Reauth code.

added docstring for Reauth::verifyPassword

cgi/client.py:
fix ReST for a method breaking pydoc.py processing

cgi/templating.py:
fix docstring on embed_form_fields

templates/*/html/_generic.reauth.html
disable spelling for password field

add timing info to the javascript function that processes file data.

reformat javascript IIFE

templates/jinja2/html/_generic.reauth.html
create a valid jinja2 template. Looks like my original jinja
template got overwritten and committed.

feature parity with the other reauth templates.

test/test_liveserver.py
add test case for Reauth workflow.

Makefile
add doc.
files
[reauth-confirm_id] feat(security): Add user confirmation/reauth for sensitive changes http://hg.code.sf.net:8000/p/roundup/code/#changeset-ef1ea918b07a9a4febcb511cb4b18690324d81cd John Rouillard rouilj@ieee.org 2025-08-11T14:01:12-04:00 2025-08-11T14:01:12-04:00
changeset ef1ea918b07a
branch reauth-confirm_id
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat(security): Add user confirmation/reauth for sensitive changes

Auditors can raise Reauth(reason) exception to require the user to
enter a token (e.g. account password) to verify the user is performing
the change.

Naming is subject to change.

actions.py: New ReauthAction class handler and verifyPassword() method
for overriding if needed.

client.py: Handle Reauth exception by calling Client:reauth() method.
Default client:reauth method. Add 'reauth' action declaration.

exceptions.py: Define and document Reauth exception as a subclass of
RoundupCGIException.

templating.py: Define method utils.embed_form_fields().

The original form making a change to the database has a lot of form
fields. These need to be resubmitted to Roundup as part of the form
submission that verifies the user's password.

This method turns all non file form fields into type=hidden inputs.
It escapes the names and values to prevent XSS.

For file form fields, it base64 encodes the contents and puts them
in hidden pre blocks. The pre blocks have data attributes for the
filename, filetype and the original field name. (Note the original
field name is not used.)

This stops the file content data (maybe binary e.g. jpegs) from
breaking the html page. The reauth template runs JavaScript that
turns the encoded data inside the pre tags back into a file. Then
it adds a multiple file input control to the page and attaches all
the files to it. This file input is submitted with the rest of the
fields.

_generic.reauth.html (multiple tracker templates): Generates a form
with id=reauth_form to:

display any message from the Reauth exception to the user (e.g. why
user is asked to auth).

get the user's password

submit the form

embed all the form data that triggered the reauth

recreate any file data that was submitted as part of the form and
generate a new file input to push the data to the back end

It has the JavaScript routine (as an IIFE) that regenerates a file
input without user intervention.

All the TAL based tracker templates use the same form. There is also
one for the jinja2 template. The JavaScript for both is the same.

reference.txt: document embed_form_fields utility method.

upgrading.txt: initial upgrading docs.

TODO:

Finalize naming. I am leaning toward ConfirmID rather than Reauth.
Still looking for a standard name for this workflow.

Externalize the javascript in _generic.reauth.html to a seperate file
and use utils.readfile() to embed it or change the script to load it
from a @@file url.

Clean up upgrading.txt with just steps to implement and less feature
detail/internals.

Document internals/troubleshooting in reference.txt.

Add tests using live server.
files
doc: link to security page for link to formal CVE report. http://hg.code.sf.net:8000/p/roundup/code/#changeset-7d1b50c02835bb196fb3d5cd888b0be631c60dbb John Rouillard rouilj@ieee.org 2025-07-11T22:27:18-04:00 2025-07-11T22:27:18-04:00
changeset 7d1b50c02835
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: link to security page for link to formal CVE report.
files
bug(security): CVE-2025-53865 - XSS bug http://hg.code.sf.net:8000/p/roundup/code/#changeset-4ac0bbb3e440678d1f591777ceb5b43e360d3d25 John Rouillard rouilj@ieee.org 2025-07-11T19:30:27-04:00 2025-07-11T19:30:27-04:00
changeset 4ac0bbb3e440
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description bug(security): CVE-2025-53865 - XSS bug

Extensive fixes in devel, responsive templates known to be
exploitable.

Similar constructs in classic and minimal templates not known
to be exploitable, but changed anyway.

doc/upgrading.txt:

Reformat to 66 characters.

Update with assigned CVE number.

Add section on fixing tal:replace with unsafe data.

Document analysis and assumptions in comment in file.

doc/security.txt:

Update with CVE number.
files
docs: fix example http://hg.code.sf.net:8000/p/roundup/code/#changeset-fee1b89ae6c3342bf46c9b14114d4ff63929c81e John Rouillard rouilj@ieee.org 2025-07-09T19:10:00-04:00 2025-07-09T19:10:00-04:00
changeset fee1b89ae6c3
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: fix example
files
doc: put example in callout using :: http://hg.code.sf.net:8000/p/roundup/code/#changeset-f6e58615a9987db6526a0e3386b94cbdb55fd40d John Rouillard rouilj@ieee.org 2025-07-09T09:44:59-04:00 2025-07-09T09:44:59-04:00
changeset f6e58615a998
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: put example in callout using ::
files
docs: relabel/label a couple of headers http://hg.code.sf.net:8000/p/roundup/code/#changeset-d98cb4730a4a64fa9c72a8bf8134c140526af269 John Rouillard rouilj@ieee.org 2025-07-09T09:43:37-04:00 2025-07-09T09:43:37-04:00
changeset d98cb4730a4a
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: relabel/label a couple of headers
files
bug(security): fix XSS exploit in devel and responsive templates http://hg.code.sf.net:8000/p/roundup/code/#changeset-abf1297e7a940f7bbe42fed4e319d9de96cdc740 John Rouillard rouilj@ieee.org 2025-07-08T13:38:08-04:00 2025-07-08T13:38:08-04:00
changeset abf1297e7a94
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description bug(security): fix XSS exploit in devel and responsive templates

Replace all occurances of:

tal:content="structure context/MUMBLE/plain"

with

tal:content="context/MUMBLE/plain"

This seems to have been an old way to handle display of a field when
the user did not have edit rights. It does not occur in current (later
than 2009) classic tracker templates. But probably was unsed in
earlier classic templates since devel, reponsive and the roundup issue
tracker templates were based on classic.

Add CVE placeholder to security.txt and link to fix directions added
to upgrading.txt. Add note in announcement.txt and CHANGES.txt

Add a details element around the table of contents in the upgrading
guide. It was getting long.

Updated a missed XSS issue in the roundup tracker template. Live site
is already fixed.

XSS bug reported by 4bug of ChaMd5 Security Team H1 Group
files
docs: fix typo http://hg.code.sf.net:8000/p/roundup/code/#changeset-226a4f391ae2085f012a383e8f75881523d1c0e1 John Rouillard rouilj@ieee.org 2025-07-03T13:33:59-04:00 2025-07-03T13:33:59-04:00
changeset 226a4f391ae2
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: fix typo
files
docs: fix registerutilMethod docs, format for highlights. http://hg.code.sf.net:8000/p/roundup/code/#changeset-6ea309c6d17c1371c374908214e0a3af135006fc John Rouillard rouilj@ieee.org 2025-07-03T12:30:21-04:00 2025-07-03T12:30:21-04:00
changeset 6ea309c6d17c
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: fix registerutilMethod docs, format for highlights.
files
docs: issue2551398 document enabling native browser number/integer types http://hg.code.sf.net:8000/p/roundup/code/#changeset-107761be1e75aac74e400f539db8e893c677aac2 John Rouillard rouilj@ieee.org 2025-06-25T23:10:56-04:00 2025-06-25T23:10:56-04:00
changeset 107761be1e75
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: issue2551398 document enabling native browser number/integer types

The original default was to have them enabled. Due to issues with
locale (e.g. decimal thousands separator vs comma seperator) this is
disabled by default. See issue for details.
files
docs(install): document how to install gpg 2.0 http://hg.code.sf.net:8000/p/roundup/code/#changeset-35fab0db52f5af5019609cd78db29a3e0ccba075 John Rouillard rouilj@ieee.org 2025-06-25T23:05:19-04:00 2025-06-25T23:05:19-04:00
changeset 35fab0db52f5
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs(install): document how to install gpg 2.0

Add link in install doc to upgrading doc and document pip command there.
files
docs: use bulleted list for deprecation; pydoc for shared dir http://hg.code.sf.net:8000/p/roundup/code/#changeset-3f43db05aa11221cf37ff13c70a3d2554eaf0df4 John Rouillard rouilj@ieee.org 2025-05-11T18:51:57-04:00 2025-05-11T18:51:57-04:00
changeset 3f43db05aa11
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: use bulleted list for deprecation; pydoc for shared dir

pydoc update for embedded/released docs isn't working quite right.
The html file is generated but no docs are extracted into it.
But the error emitted by the presence of pydoc.txt is suppressed.
files
Make native date and number elements configurable http://hg.code.sf.net:8000/p/roundup/code/#changeset-b99e76e764967599da7978a4d57365a155f8827e Ralf Schlatterbeck rsc@runtux.com 2025-02-19T12:38:06+01:00 2025-02-19T12:38:06+01:00
changeset b99e76e76496
branch
bookmark
tag
user Ralf Schlatterbeck <rsc@runtux.com>
description Make native date and number elements configurable

Now for Number() and Integer() properties the browser-native number
format can be configured with the use_browser_number_input config item
in seciont [web]. The default is 'yes'. For Date() properties the config
item is use_browser_date_input (also in section [web]) but the default
is 'no'.

In addition when defining Date() properties, these now have a parameter
'display_time' which defaults to 'yes' and a 'format' parameter which
defaults to None. These set defaults for the field() method of the
DateHTMLProperty which have the same parameters (but the display_time
parameter of field() takes a boolean, not 'yes'/'no').
files
feat(web) - Use native number type input for Number() and Integer(). http://hg.code.sf.net:8000/p/roundup/code/#changeset-6445e63bb423fe66f50ac30b2407d10eb2956eac John Rouillard rouilj@ieee.org 2025-01-18T14:54:31-05:00 2025-01-18T14:54:31-05:00
changeset 6445e63bb423
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat(web) - Use native number type input for Number() and Integer().

When editing hyperdb.Number() or hyperdb.Integer() properties, use a
native number input.

For Number you can enter digits, +/-, . and e/E for exponent (1E2 =
100).

For integer we have the same keys as number, but also add step=1 to
the input. This stops submitting 23.5 suggesting 23 or 24. It does
allow 2E4 to be submitted that is rejected with an error from the
backend. However if the spinner is used with 2E4 it is turned into
20000, a pure integer and incremented/decremented by the spinner.

The upgrade happens automatically. Directions on going back to text
input provided. User guide updated to describe addition of spinner.

Tests added.
files
fix: issue2551390 - Replace text input/calendar popup with native date input http://hg.code.sf.net:8000/p/roundup/code/#changeset-2bf0c4e7795e383a6722e70d071361bb3db45b2c John Rouillard rouilj@ieee.org 2025-01-18T12:23:23-05:00 2025-01-18T12:23:23-05:00
changeset 2bf0c4e7795e
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: issue2551390 - Replace text input/calendar popup with native date input

Docs, code and test changes for the changeover to a native date
element.

See issue for details.
files
doc: typo fix. http://hg.code.sf.net:8000/p/roundup/code/#changeset-b757cf5094802f11db6ee4587e3da8333f527e13 John Rouillard rouilj@ieee.org 2025-01-12T20:41:31-05:00 2025-01-12T20:41:31-05:00
changeset b757cf509480
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: typo fix.
files
fix(api): issue2551384. Verify REST authorization earlier http://hg.code.sf.net:8000/p/roundup/code/#changeset-35beff3168834a58b57bea4ac44ac499ba7942ab John Rouillard rouilj@ieee.org 2025-01-09T09:30:08-05:00 2025-01-09T09:30:08-05:00
changeset 35beff316883
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix(api): issue2551384. Verify REST authorization earlier

To reduce the ability of bad actors to spam (DOS) the REST endpoint
with bad data and generate logs meant for debugging, modify the flow
in client.py's REST handler to verify authorization earlier.

If the anonymous user is allowed to use REST, this won't make a
difference for a DOS attempt. The templates don't enable REST for the
anonymous user by default. Most admins don't change this.

The validation order for REST requests has been changed.

CORS identfied an handled
User authorization to use REST (return 403 on failure)
REST request validated (Origin header valid etc.) (return 400 for
bad request)

Incorrectly formatted CORS preflight requests (e.g. missing Origin
header) that are not recogized as a CORS request can now return HTTP
status 403 as well as status 400 (when anonymous is allowed
access). Note all CORS preflights are sent without authentication so
appear as anonymous requests.

The tests were updated to compensate, but it is not obvious to me from
specs what the proper evaulation order/return codes should be for this
case. Both 403/400 are failures and cause CORS to fail so there should
be no difference but...
files
feat: add templating utils method dynamically; method to set http code http://hg.code.sf.net:8000/p/roundup/code/#changeset-2a7c3eeaf167484beb05fecb7348f43d10cf8a7e John Rouillard rouilj@ieee.org 2025-01-07T20:22:33-05:00 2025-01-07T20:22:33-05:00
changeset 2a7c3eeaf167
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat: add templating utils method dynamically; method to set http code

Added new utils.set_http_response(integer) to set the HTML response
code from a template. Useful for error handling inside template.

Also noted that a real TemplatingUtils (like set_http_response) method
gets the TemplatingUtils object instance, but there is no way to do
this with registerUtil() from an extension file.

Added new instance.registerUtilMethod() method to register a function
in an extension as a method passing the client instance in as the first
parameter (aka self).
files
issue2551253. default hash is PBKDF2-SHA512. http://hg.code.sf.net:8000/p/roundup/code/#changeset-6bd11a73f2ede6bbbfc34d18df864e916ec4330e John Rouillard rouilj@ieee.org 2024-12-30T02:57:46-05:00 2024-12-30T02:57:46-05:00
changeset 6bd11a73f2ed
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551253. default hash is PBKDF2-SHA512.

The default password hashing algorithm has been upgraded to
PBKDF2-SHA512 from PBKDF2-SHA1. The default pbkdf2 rounds in the
config file has been changed to 250000.

Doc updated.
files
issue2551116 - Replace xmlrpclib (xmlrpc.client) with defusedxml. http://hg.code.sf.net:8000/p/roundup/code/#changeset-57325fea99826845b4b8f2e8baaaf4eaeea59e48 John Rouillard rouilj@ieee.org 2024-12-29T19:11:01-05:00 2024-12-29T19:11:01-05:00
changeset 57325fea9982
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551116 - Replace xmlrpclib (xmlrpc.client) with defusedxml.

defusedxml will be used to moneypatch the problematic client and
server modules.

Test added using an xml bomb.
files
doc: clarify adding ctx argument to check command in schema.py http://hg.code.sf.net:8000/p/roundup/code/#changeset-2d0bd038fc5e0c9271bf7f9664f783b21e958c76 John Rouillard rouilj@ieee.org 2024-12-23T21:10:54-05:00 2024-12-23T21:10:54-05:00
changeset 2d0bd038fc5e
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: clarify adding ctx argument to check command in schema.py

On IRC user was unsure what file was used for the 1.51 - 1.60 upgrade
section:

Enhancement to check command for Permissions

added file name desription and example. Also marked it as optional.
Clarified that if or when it becomes required there will be a new
required upgrade direction.
files
fix(REST): issue2551383; improve errors for bad json, fix PUT docs http://hg.code.sf.net:8000/p/roundup/code/#changeset-32aaf5dc562be2b244cc3ee62e26ff2e53a1ae6e John Rouillard rouilj@ieee.org 2024-12-17T19:42:46-05:00 2024-12-17T19:42:46-05:00
changeset 32aaf5dc562b
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix(REST): issue2551383; improve errors for bad json, fix PUT docs

While adding fuzz testing for email addresses via REST
/rest/data/user/1/address, I had an error when setting the address to
the same value it currently had. Traced this to a bug in
userauditor.py. Fixed the bug. Documented in upgrading.txt.

While trying to track down issue, I realized invalid json was being
accepted without error. So I fixed the code that parses the json and
have it return an error. Also modified some tests that broke (used
invalid json, or passed body (e.g. DELETE) but shouldn't have. Add
tests for bad json to verify new code.

Fixed test that wasn't initializing the body_file in each loop, so the
test wasn't actually supplying a body.

Also realised PUT documentation was not correct. Output format isn't
quite like GET.

Fuss tests for email address also added.
files
refactor: issue2551289. invalid REST Accept header stops request http://hg.code.sf.net:8000/p/roundup/code/#changeset-2967f37e73e43bfbb220aa6e3751010a04829057 John Rouillard rouilj@ieee.org 2024-12-08T01:09:34-05:00 2024-12-08T01:09:34-05:00
changeset 2967f37e73e4
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description refactor: issue2551289. invalid REST Accept header stops request

Sending a POST, PUT (maybe PATCH) with an accept header that is not
application/json or xml (if enabled) used to complete the request
before throwing a 406 error. This was wrong.

Now it reports an error without dispatching/processing the requested
transaction. This is the first of a series of refactors of the
dispatch method to make it faster and more readable by using return
early pattern and extracting methods from the code.

changes:

The following now return 406 errors not 400 errors
invalid version specified with @apiver in URL.
invalid version specified with @apiver in payload body
invalid version specified in accept headers as
application/vnd.roundup.test-vz+json or version property

Parsing the accept header returns a 400 when presented with a
parameter without an = sign or other parse error. They used to
return a 406 which is wrong since the header is malformed rather
than having a value I can't respond to.

Some error messages were made clearer.

Results in the case of an error are proper json error object rather
than text/plain strings.

New test added for testdetermine_output_formatBadAccept that test the
new method using the same test cases as for
testDispatchBadAccept. I intend to extend the test coverage for
determine_output_format to cover more cases. This should be a faster
unit test than for dispatch.

Removed .lower() calls for accept_mime_type as the input values are
taken from the values in the __accepted_content_type dict which
only has lower case values.
files
issue2551370 - prefix session cookie with __Secure- over https http://hg.code.sf.net:8000/p/roundup/code/#changeset-3f0f4746dc7ece6a24ccb5181efb9d2028019f25 John Rouillard rouilj@ieee.org 2024-11-26T17:11:13-05:00 2024-11-26T17:11:13-05:00
changeset 3f0f4746dc7e
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551370 - prefix session cookie with __Secure- over https

Limit use of roundup session cookie to HTTPS protocol by adding
__Secure- prefix. Automatic testing includes http behavior only.
Https behavious has been manually tested only. Need to be able to spin
up an https server using wsgiref to test https behavior in CI.
issue 2551373 opened to track automatic testing of https behavior.
files
- issue2551074 - In "responsive" template: click on hide comment leads http://hg.code.sf.net:8000/p/roundup/code/#changeset-800c8dd75051e2b294772c6d1201d1599a6a4cd7 John Rouillard rouilj@ieee.org 2024-10-20T18:42:21-04:00 2024-10-20T18:42:21-04:00
changeset 800c8dd75051
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description - issue2551074 - In "responsive" template: click on hide comment leads
to a red error msg.

to files in responsive template reference the 'icing' macro. This was
replaced by a 'frame' macro in the responsive template.
files
docs: replace redirecting url's with target http://hg.code.sf.net:8000/p/roundup/code/#changeset-394f72021dad0d3968066d769c7b29bce8c2d157 John Rouillard rouilj@ieee.org 2024-09-25T21:18:37-04:00 2024-09-25T21:18:37-04:00
changeset 394f72021dad
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: replace redirecting url's with target
files
issue2551343 - Remove support for PySQLite and sqlite v2. http://hg.code.sf.net:8000/p/roundup/code/#changeset-95f91b6f0386817e952300dafa4ed65418ffce7b John Rouillard rouilj@ieee.org 2024-07-13T18:52:55-04:00 2024-07-13T18:52:55-04:00
changeset 95f91b6f0386
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551343 - Remove support for PySQLite and sqlite v2.

It is unmaintained and sqlite3 is used which is the default for a
Python distribution.
files
final changes for 2.4.0 release http://hg.code.sf.net:8000/p/roundup/code/#changeset-a4cb4e75d4e923554399441497fb2b69acd3a375 John Rouillard rouilj@ieee.org 2024-07-12T20:32:20-04:00 2024-07-12T20:32:20-04:00
changeset a4cb4e75d4e9
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description final changes for 2.4.0 release
files
docs: set up for release documentation. http://hg.code.sf.net:8000/p/roundup/code/#changeset-d6b447de4f5909ddac6d22f6dd51e8e4ebd79c5d John Rouillard rouilj@ieee.org 2024-07-09T09:34:13-04:00 2024-07-09T09:34:13-04:00
changeset d6b447de4f59
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: set up for release documentation.

Make changes to publish security.html with CVE announcements referring
to the sections in upgrading.html rather than CVE.html.

Remove templates.zip as part of html build in Makefile.

Also update doc for using CVE.html.
files
fix(security): fix CVE-2024-39124, CVE-2024-39124, and CVE-2024-39125 http://hg.code.sf.net:8000/p/roundup/code/#changeset-28aa76443f58bc1605a9933bb4ea4d599c97af5a John Rouillard rouilj@ieee.org 2024-07-09T09:07:09-04:00 2024-07-09T09:07:09-04:00
changeset 28aa76443f58
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix(security): fix CVE-2024-39124, CVE-2024-39124, and CVE-2024-39125

Directions for fixing:

* `CVE-2024-39124`_ - :ref:`classhelpers (_generic.help.html) are
vulnerable to an XSS attack. <CVE-2024-39124>` Requires fixing
tracker homes.
* `CVE-2024-39125`_ - :ref:`if Referer header is set to a script
tag, it will be executed. <CVE-2024-39125>` Fixed in release 2.4.0,
directions available for fixing in prior versions.
* `CVE-2024-39126`_ - :ref:`PDF, XML and SVG files downloaded from
an issue can contain embedded JavaScript which is
executed. <CVE-2024-39126>` Fixed in release 2.4.0, directions
available for fixing in prior versions.

prior to 2.4.0 release this weekend that fixes the last two CVE's.
files
fix: disable spellchecking for password fields http://hg.code.sf.net:8000/p/roundup/code/#changeset-0e382e97f0e311c935908b01eb8b3b16ce0b099e John Rouillard rouilj@ieee.org 2024-07-07T10:49:40-04:00 2024-07-07T10:49:40-04:00
changeset 0e382e97f0e3
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: disable spellchecking for password fields

Some browser can send password to a server for spellchecking. This
gives the browser a strong hint that they should not spellcheck a
password. Since a Password is not supposed to be a real word in any
language, spellchecking is worthless.
files
doc: 2.3.0 is the last version to support xhtml http://hg.code.sf.net:8000/p/roundup/code/#changeset-3ddc6a7d41de5edbff85d931b852d42aa9f8632e John Rouillard rouilj@ieee.org 2024-06-26T23:13:33-04:00 2024-06-26T23:13:33-04:00
changeset 3ddc6a7d41de
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: 2.3.0 is the last version to support xhtml
files
doc: clarify basic upgrade instructions to target single instance http://hg.code.sf.net:8000/p/roundup/code/#changeset-a0876d16e299548491ee9fcf2b11a961f0a26738 John Rouillard rouilj@ieee.org 2024-06-26T23:11:25-04:00 2024-06-26T23:11:25-04:00
changeset a0876d16e299
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: clarify basic upgrade instructions to target single instance
files
doc: deprication sqlite2 http://hg.code.sf.net:8000/p/roundup/code/#changeset-c53117e6775f6a52c1da2a099caebffdf49e4510 John Rouillard rouilj@ieee.org 2024-06-26T23:10:01-04:00 2024-06-26T23:10:01-04:00
changeset c53117e6775f
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: deprication sqlite2
files
doc: bold status/severity keywords in key; discuss virtual env install http://hg.code.sf.net:8000/p/roundup/code/#changeset-ab96dcb1beb4c34af024b091ec4aa087d4f860ad John Rouillard rouilj@ieee.org 2024-06-26T23:09:34-04:00 2024-06-26T23:09:34-04:00
changeset ab96dcb1beb4
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: bold status/severity keywords in key; discuss virtual env install
files
docs: add references for utf8 -> utf8mb4 conversion/issues for mysql http://hg.code.sf.net:8000/p/roundup/code/#changeset-6d1b62ffbb5d34e3e5844ba88e3f042c3a1d807f John Rouillard rouilj@ieee.org 2024-06-06T20:23:31-04:00 2024-06-06T20:23:31-04:00
changeset 6d1b62ffbb5d
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: add references for utf8 -> utf8mb4 conversion/issues for mysql
files
commit to 2025 as sunset date for python2 support. http://hg.code.sf.net:8000/p/roundup/code/#changeset-8e790efaabcef7d6b9cef89d7711958c02dea846 John Rouillard rouilj@ieee.org 2024-05-25T23:45:30-04:00 2024-05-25T23:45:30-04:00
changeset 8e790efaabce
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description commit to 2025 as sunset date for python2 support.

It looks like I can make it go that far.
files
fix: issue2551341 - if @columns missing from an index url, the group headers colspan property = 0 http://hg.code.sf.net:8000/p/roundup/code/#changeset-1e9c16b079faaa1eaf7482860d99bef42bbb2e38 John Rouillard rouilj@ieee.org 2024-05-25T22:57:44-04:00 2024-05-25T22:57:44-04:00
changeset 1e9c16b079fa
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: issue2551341 - if @columns missing from an index url, the group headers colspan property = 0

Fix packaged templates with fix for group headers not spaning the
whole table if @columns parameter is missing from the url.
files
issue2551353 - Add roundup-classhelper for 2.4.0 release http://hg.code.sf.net:8000/p/roundup/code/#changeset-fe0348bbe45b6feaa1f35365449d334fa4686d55 John Rouillard rouilj@ieee.org 2024-05-21T01:17:28-04:00 2024-05-21T01:17:28-04:00
changeset fe0348bbe45b
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551353 - Add roundup-classhelper for 2.4.0 release

Changes to the classic template are not done yet. Still testing.

This commit has document updates and changes to rest.py.

rest.py:

add /rest/data/user/role endpoint to core so the user doesn't have
to add the /rest/roles endpoint via interfaces.py. It will only send
roles for a user with Admin role and there is no way to override
this currently.


acknowledgements.txt:

Added members of team3 to other contributors. Specified for all
other contributes what they worked on.

upgrading.txt:

added classhelper section and basic template change
directions. Linked to admin_guide for full directions.

admin_guide.txt:

documented install, translation, troubleshooting, config etc.

user_guide.txt:

added section on using the classhelper. Added reference to section
earlier in the doc. Added image for section.
files
issue2551323 - Remove XHTML support http://hg.code.sf.net:8000/p/roundup/code/#changeset-791b61ed11c989e2415016366537ac2004a3205d John Rouillard rouilj@ieee.org 2024-05-14T20:50:25-04:00 2024-05-14T20:50:25-04:00
changeset 791b61ed11c9
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551323 - Remove XHTML support

First step: raise error if xhtml used in config.ini. Back end code is
still present to allow an admin to re-enable xhtml as a temporary
measure.
files
docs: postgres user; wsgi default mode update; diff for task.index.html http://hg.code.sf.net:8000/p/roundup/code/#changeset-32ead43b829920d5779dd868292ac31051dbeb21 John Rouillard rouilj@ieee.org 2024-05-13T13:32:02-04:00 2024-05-13T13:32:02-04:00
changeset 32ead43b8299
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: postgres user; wsgi default mode update; diff for task.index.html

Update docs for creating postgresql roundup user for database and
schema cases.

Reword wsgi performance improvement disabling doc and add request for
email if it needs to be disabled.

Add link to diff for changes to task.index.html.
files
doc: spelling fix. http://hg.code.sf.net:8000/p/roundup/code/#changeset-88239d4ac4ab85a7c11d08b40863ccd7aec0a290 John Rouillard rouilj@ieee.org 2024-05-12T22:25:47-04:00 2024-05-12T22:25:47-04:00
changeset 88239d4ac4ab
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: spelling fix.
files
doc: issue1525113 - notation to filter by logged-in user http://hg.code.sf.net:8000/p/roundup/code/#changeset-ce5a554b2f881a14b9a92ac2cde6f2bf02c93911 John Rouillard rouilj@ieee.org 2024-05-06T01:33:49-04:00 2024-05-06T01:33:49-04:00
changeset ce5a554b2f88
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: issue1525113 - notation to filter by logged-in user

Little update.
files
feat: issue1525113 - notation to filter by logged-in user http://hg.code.sf.net:8000/p/roundup/code/#changeset-a9b1365658388afef20fbfdb6d1dd8cd59e7b424 John Rouillard rouilj@ieee.org 2024-05-06T00:49:43-04:00 2024-05-06T00:49:43-04:00
changeset a9b136565838
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat: issue1525113 - notation to filter by logged-in user

At long last (almost 18 years) this patch lands. It allows sharing of
queries that want to use the currently logged in user (i.e. I or me).
By replacing an id number for the user by '@current_user' in the query
you can share the query for "my issues" where 'my' is the logged in
user not the person who created the query.

Updated the templates to use this.
Updated upgrading.py for directions on using it.
RDBMS and anydbm both work. Also expressions using it (e.g. not
@current_user) work and are tested.
Test code done.


I am not sure what the change to templating.py does. I am following
the original patch and have built a test case to hit the if
clause. But the rest of the test doesn't actualy provide the props I
need. If I knew what that code was supposed to do there I would create
a real test.
files
fix: issue2551347 - make _generic.help.html work without property settings http://hg.code.sf.net:8000/p/roundup/code/#changeset-c05ea62b4c7a71d9c300592ada956f997e2a3095 John Rouillard rouilj@ieee.org 2024-05-02T22:11:37-04:00 2024-05-02T22:11:37-04:00
changeset c05ea62b4c7a
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: issue2551347 - make _generic.help.html work without property settings

This lets the classhelp() method run in read only mode to provide
information on a Link property without updating the property.

Updating the property can be done by selecting the proper value using
the drop-down.

Update classic and minimal templates to allow this to work.

Update upgrading and reference docs.
files
issue2551285 - Remove StructuredText support http://hg.code.sf.net:8000/p/roundup/code/#changeset-29a666d8a70d065186db60f9dff44ead46b299a6 John Rouillard rouilj@ieee.org 2024-04-30T22:27:57-04:00 2024-04-30T22:27:57-04:00
changeset 29a666d8a70d
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551285 - Remove StructuredText support

Asked on the users mailing list if anybody was using it. Got no responses.
I have never seen CI installing structuredtext packages so it's
untested as well.
files
announce deprecation for PySQLite in 2.5.0 http://hg.code.sf.net:8000/p/roundup/code/#changeset-ded9f1c3f11210ef4b35eaba01bedc7855b64959 John Rouillard rouilj@ieee.org 2024-04-30T22:17:50-04:00 2024-04-30T22:17:50-04:00
changeset ded9f1c3f112
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description announce deprecation for PySQLite in 2.5.0

sqlite3 is better and installed with Python.
files
issue2551302 - Remove support for sqlite version 1 from back_sqlite.py http://hg.code.sf.net:8000/p/roundup/code/#changeset-e3975f679bf17e1fdcf5df649bd0358494eaab2d John Rouillard rouilj@ieee.org 2024-04-30T22:16:22-04:00 2024-04-30T22:16:22-04:00
changeset e3975f679bf1
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551302 - Remove support for sqlite version 1 from back_sqlite.py

Remove sqlite v1 support and document.
files