Mercurial > p > roundup > code
changeset 8412:0663a7bcef6c reauth-confirm_id
feat: finish reauth docs, enhance code.
Decided to keep name Reauth for now.
admin_guide.txt:
add reference mark to roundup admin help. Used for template command
reference in upgrading.txt.
customizing.txt:
added worked example of adding a reauth auditor for address and password.
Also links to OWASP recommendations.
Added link to example code in design.doc on detectors.
glossary.txt:
reference using roundup-admin template command in def for tracker
templates.
pydoc.txt:
Added methods for Client class.
Added class and methods for (cgi) Action, LoginAction and ReauthAction.
reference.txt
Edited and restructured detector section.
Added section on registering a detector and priority use/execution order.
(reference to design doc was used before).
Added/enhanced description of exception an auditor can
raise (includes Reauth).
Added section on Reauth implementation and use (Confirming the User).
Also has paragraph on future ideas.
upgrading.txt
Stripped down the original section. Moved a lot to reference.txt.
Referenced customizing example, mention installation of
_generic.reauth.html and reference reference.txt.
cgi/actions.py:
fixed bad ReST that was breaking pydoc.txt processing
changed doc on limitations of Reauth code.
added docstring for Reauth::verifyPassword
cgi/client.py:
fix ReST for a method breaking pydoc.py processing
cgi/templating.py:
fix docstring on embed_form_fields
templates/*/html/_generic.reauth.html
disable spelling for password field
add timing info to the javascript function that processes file data.
reformat javascript IIFE
templates/jinja2/html/_generic.reauth.html
create a valid jinja2 template. Looks like my original jinja
template got overwritten and committed.
feature parity with the other reauth templates.
test/test_liveserver.py
add test case for Reauth workflow.
Makefile
add doc.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Wed, 13 Aug 2025 23:52:49 -0400 |
| parents | ef1ea918b07a |
| children | 8a90350cc78b |
| files | doc/admin_guide.txt doc/customizing.txt doc/glossary.txt doc/pydoc.txt doc/reference.txt doc/upgrading.txt roundup/cgi/actions.py roundup/cgi/client.py roundup/cgi/templating.py share/roundup/templates/classic/html/_generic.reauth.html share/roundup/templates/devel/html/_generic.reauth.html share/roundup/templates/jinja2/html/_generic.reauth.html share/roundup/templates/minimal/html/_generic.reauth.html share/roundup/templates/responsive/html/_generic.reauth.html test/test_liveserver.py website/www/Makefile |
| diffstat | 16 files changed, 865 insertions(+), 375 deletions(-) [+] |
line wrap: on
line diff
--- a/doc/admin_guide.txt Mon Aug 11 14:01:12 2025 -0400 +++ b/doc/admin_guide.txt Wed Aug 13 23:52:49 2025 -0400 @@ -1729,6 +1729,8 @@ single: roundup-admin; man page reference pair: roundup-admin; designator +.. _`roundup-admin templates`: + Using roundup-admin ===================
--- a/doc/customizing.txt Mon Aug 11 14:01:12 2025 -0400 +++ b/doc/customizing.txt Wed Aug 13 23:52:49 2025 -0400 @@ -1815,6 +1815,60 @@ you could search for all currently-Pending users and do a bulk edit of all their roles at once (again probably with some simple javascript help). +.. _sensitive_changes: + +Confirming Users Making Sensitive Account Changes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Some changes to account data: user passwords or email addresses are +particularly sensitive. The `OWASP Authentication`_ recommendations +include asking for a re-authentication or confirmation step when making +these changes. This can be easily implemented using an auditor. + +Create a file in your detectors directory with the following +contents:: + + from roundup.cgi.exceptions import Reauth + + def confirmid(db, cl, nodeid, newvalues): + + if hasattr(db, 'reauth_done'): + # the user has confirmed their identity + return + + # if the password or email are changing, require id confirmation + if 'password' in newvalues: + raise Reauth('Add an optional message to the user') + + if 'address' in newvalues: + raise Reauth('Add an optional message to the user') + + def init(db): + db.user.audit('set', confirmid, priority=110) + +If a change is made to any user's password or address fields, the user +making the change will be shown a page where they have to enter an +identity verifier (by default the invoking user's account password). +If the verifier is successfully verified it will set the +``reauth_done`` attribute on the db object and reprocess the change. + +The default auditor priority is 100. This auditor is set to run +**after** most other auditors. This allows the user to correct any +failing information on the form before being asked to confirm their +identity. Once they confirm their identity the change is expected to +be committed without issue. See :ref:`Confirming the User` for +details on customizing the verification operation. + +Also you could use an existing auditor and add:: + + if 'someproperty' in newvalues and not hasattr(db, 'reauth_done'): + raise Reauth('Need verification before changing someproperty') + +at the end of the auditor (after all checks are done) to force user +verification. Just make sure you import Reauth at the top of the file. + +.. _`OWASP Authentication`: + https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html#require-re-authentication-for-sensitive-features Changes to the Web User Interface --------------------------------- @@ -2436,6 +2490,10 @@ <reference.html#extending-the-configuration-file>`_. * `Adding a new Permission <reference.html#adding-a-new-permission>`_ +as does the design document: + +* `detector examples <design.html#detector-example>`_ + Examples on the Wiki ====================
--- a/doc/glossary.txt Mon Aug 11 14:01:12 2025 -0400 +++ b/doc/glossary.txt Wed Aug 13 23:52:49 2025 -0400 @@ -99,7 +99,9 @@ tracker with a particular look and feel, :term:`schema`, permissions model, and :term:`detectors`. Roundup ships with five templates and people on the net `have produced other - templates`_ + templates`_. You can find the installed location of the + standard Roundup templates using the :ref:`roundup-admin + templates <roundup-admin templates>` command. tracker
--- a/doc/pydoc.txt Mon Aug 11 14:01:12 2025 -0400 +++ b/doc/pydoc.txt Wed Aug 13 23:52:49 2025 -0400 @@ -9,7 +9,32 @@ ============ .. autoclass:: roundup.cgi.client::Client + :members: + +CGI Action class +================ +Action class and selected derived classes. + +Action +------ +.. autoclass:: roundup.cgi.actions::Action + :members: + +LoginAction +------------ + +.. autoclass:: roundup.cgi.actions::LoginAction + :members: + +.. _`ReauthAction_pydoc`: + +ReauthAction +------------ + +.. autoclass:: roundup.cgi.actions::ReauthAction + :members: + Templating Utils class ======================
--- a/doc/reference.txt Mon Aug 11 14:01:12 2025 -0400 +++ b/doc/reference.txt Wed Aug 13 23:52:49 2025 -0400 @@ -888,7 +888,7 @@ See :ref:`CustomExamples` for examples. -The Roundup wiki CategorySchema`_ provides a list of additional +The `Roundup wiki CategorySchema`_ provides a list of additional examples of how to customize schemas to add new functionality. .. _Roundup wiki CategorySchema: @@ -948,11 +948,36 @@ the database is initialised via the ``roundup-admin initialise`` command. -Detectors in your tracker are run *before* (**auditors**) and *after* -(**reactors**) changes to the contents of your database. You will have -some installed by default - have a look. You can write new detectors -or modify the existing ones. The existing detectors installed for you -are: +There are two types of detectors: + + 1. *auditors* are run before changes are made to the database + 2. *reactors* are run after the change has been committed to the database + +.. index:: auditors; rules for use + single: reactors; rules for use + +Auditor or Reactor? +------------------- + +Generally speaking, you should observe the following rules: + +**Auditors** + Are used for `vetoing creation of or changes to items`_. They might + also make automatic changes to item properties. They can raise the + ``Reject`` or ``CheckId`` exceptions to control database changes. +**Reactors** + Detect changes in the database and react accordingly. They should avoid + making changes to the database where possible, as this could create + detector loops. + + +Detectors Installed by Default +------------------------------ + +You will have some detectors installed by default - have a look in the +``detectors`` subdirectory of your tracker home. You can write new +detectors or modify the existing ones. The existing detectors +installed for you are: .. index:: detectors; installed @@ -962,6 +987,13 @@ to issues. The nosy auditor (``updatenosy``) fires when issues are changed, and figures out what changes need to be made to the nosy list (such as adding new authors, etc.) + + If you are running a tracker started with ``roundup-demo`` or the + ``demo.py`` script, this detector will be missing.This is + intentional to prevent email from being sent from a demo + tracker. You can find the nosyreaction.py detector in the + :term:`template directory (meaning 3) <template>` and copy it into + your tracker if you want email to be sent. **statusauditor.py** This provides the ``chatty`` auditor which changes the issue status from ``unread`` or ``closed`` to ``chatting`` if new messages appear. @@ -978,13 +1010,61 @@ If you don't want this default behaviour, you are completely free to change or remove these detectors. -See the detectors section in the `design document`__ for details of the -interface for detectors. - -__ design.html - +The rest of this section includes much of the information from the +`detectors section in the design document`_. But there are some +details of the detector interface that are only in the design +document. Also the design document `includes examples`_ of a project +that requires three approvals before it can be processed and rejecting +an email to create an issue if it doesn't have an attached patch. + +.. _`detectors section in the design document`: design.html#detector-interface + +.. _`includes examples`: design.html#detector-example + +Registering Detectors +--------------------- + +Detectors are registered using the ``audit`` or ``react`` methods of a +database schema class. Each detector file must define an ``init(db)`` +function. This function is run to register the detectors. For example +this registers two auditors for changes made to the user class in the +database:: + + def init(db): + # fire before changes are made + db.user.audit('set', audit_user_fields) + db.user.audit('create', audit_user_fields) + +while this registers two auditors and two reactors for the issue +class:: + + def init(db): + db.issue.react('create', nosyreaction) + db.issue.react('set', nosyreaction) + db.issue.audit('create', updatenosy) + db.issue.audit('set', updatenosy) + +The arguments for ``audit`` and ``react`` are the same: + + * operation - one of ``create``, ``set``, ``retire``, or ``restore`` + * function name - use the function name without ``()`` after + it. (You want the function name not the result of calling the + function.) + * priority - (optional default 100) the priority allows you to order + the application order of the detectors. + + A detector with a priority of 110 will run after a detector with + the default priority of 100. A detector with a priority of 90 will + run before a detector with the default priority of 100. + + Detectors with the same priority are run in an undefined + order. All the examples above use the default priority of 100. + +If no auditor raises an exception, the changes are committed to the +database. Then all the reactors registered for the operation are run. .. index:: detectors; writing api +.. _detector_api: Detector API ------------ @@ -996,7 +1076,7 @@ Auditors are called with the arguments:: - audit(db, cl, itemid, newdata) + an_auditor(db, cl, itemid, newdata) where ``db`` is the database, ``cl`` is an instance of Class or IssueClass within the database, and ``newdata`` is a dictionary mapping @@ -1018,7 +1098,7 @@ Reactors are called with the arguments:: - react(db, cl, itemid, olddata) + a_reactor(db, cl, itemid, olddata) where ``db`` is the database, ``cl`` is an instance of Class or IssueClass within the database, and ``olddata`` is a dictionary mapping @@ -1033,6 +1113,27 @@ For a ``retire()`` or ``restore()`` operation, ``itemid`` is the id of the retired or restored item and ``olddata`` is None. +Your auditor can raise one of two exceptions: + +``Reject('Reason for rejection')`` + indicates that the change is rejected. The user should see the same + page they used to make the change with a the 'Reason for rejection' + message displayed in the error feedback section of the page. See + :ref:`vetoing creation of or changes to items` for an example. +``Reauth('Reason for confirmation')`` + indicates that the user needs to enter their password or other + identity confirming information (e.g. a one time key) before the + change will be committed. This can be used when a user's password or + email address are changed. This can prevent an attacker from changing + the information by providing confirmation of the person making the + change. This addition confirmation step is recommended in the `OWASP + Authentication Cheat Sheet`_. An example can be found at + :ref:`sensitive_changes`. See :ref:`Confirming the User` for details + and warnings. + +.. _`OWASP Authentication Cheat Sheet`: + https://cheatsheetseries.owasp.org/cheatsheets/Authentication_Cheat_Sheet.html#require-re-authentication-for-sensitive-features + .. index:: detectors; additional Additional Detectors Ready For Use @@ -1069,22 +1170,7 @@ information, see the detector code - it has a lengthy explanation. -.. index:: auditors; rules for use - single: reactors; rules for use - -Auditor or Reactor? -------------------- - -Generally speaking, you should observe the following rules: - -**Auditors** - Are used for `vetoing creation of or changes to items`_. They might - also make automatic changes to item properties. -**Reactors** - Detect changes in the database and react accordingly. They should avoid - making changes to the database where possible, as this could create - detector loops. - +.. _`Vetoing creation of or changes to items`: Vetoing creation of or changes to items --------------------------------------- @@ -1109,6 +1195,141 @@ the ``RejectRaw`` should be used. All security implications should be carefully considering before using ``RejectRaw``. +.. _`Confirming the User`: + +Confirming the User +------------------- + +Being able to add a user confirmation step in a workflow is useful. +In an auditor adding:: + + from roundup.cgi.exceptions import Reauth + +at the top of the file and then adding:: + + if 'password' in newvalues and not hasattr(db, 'reauth_done'): + raise Reauth('Add an optional message to the user') + +will present the user with an authorization page. The optional message +will be shown at the top of the page. The user will enter their +password (or other verification token) and submit the page to commit +the change. This section describes the mechanism used. + +The ``Reauth`` exception is handled by the core code in the +``Client.reauth()`` method. It modifies the current request's context +cleaning up some fields and saving the current action and template for +later use. Then it sets the template to ``reauth`` which is used to +render the next page. + +The page is rendered using the standard template mechanism. You can +create a ``user.reauth.html`` template that is displayed when +requiring reauth for a user. However most users will probably just use +the default ``_generic.reauth.html`` template provided with Roundup. + +Look at the generic template to understand what has to be in a new +template. The basic idea is to get the password from the user into the +``@reauth_password`` input field. Also the form that is generated +embeds a bunch of hidden fields that record the information that +triggered the reauth request. + +If you are not using reauth in a workflow that can upload files, you +are all set. The embedding of the hidden fields just works. + +If you need to support reauth on a form that allows file uploads, the +generic template supports handling file uploads. It requires +JavaScript in the browser to support files. Since browsers don't allow +a server to assign values to a file input, the template uses a +workaround. The reauth template encodes the contents of each uploaded +file as a base64 encoded string. It then embeds this string inside a +hidden ``<pre>`` tag. This encoded string is about 1/3 larger than the +size of the original file(s). + +When the page is loaded, a Javascript function runs and turns the +tagged base64 strings back into file objects with their original +content. It then creates a file input element and assigns these files +to it. This allows the files to be submitted with the rest of the +form. If JavaScript is not enabled, these files will not be submitted +to the server. + +**Future ideas for handling disabled JavaScript** - The `server could +detect disabled JavaScript on the browser +<https://wiki.roundup-tracker.org/IsJavascriptAvailable>`_ and +generate a warning. Alternatively, allowing the browser to submit all +the file data by replacing the ``<pre>`` tag with a ``<textarea>`` tag +and a name attribute like ``@filecontents-1``, +``@filecontents-2``. Along with this, the current ``data-`` +attributes on the ``<pre>`` tag need to move to hidden inputs with +names like: ``@filecontents-1.mimetype``, +``@filecontents-1.filename``. Submitting the form will include all +this data that Roundup could use to pretend the files were submitted +as a proper file input. + +When the reauth page is submitted, it invokes the ``reauth`` action +(using the ``@action`` parameter). Verification is done by the +:ref:`ReauthAction <ReauthAction_pydoc>` class in +``roundup.cgi.actions``. By default +``ReauthAction.verifyPassword()`` calls:: + + roundup.cgi.actions:LoginAction::verifyPassword() + +to verify the user's password against the one stored in the database. + +You can change the verification command using `interfaces.py`_. Adding +the following code to ``interfaces.py``:: + + from roundup.cgi.actions import ReauthAction + + old_verify = ReauthAction.verifyPassword + + def new_verify_password(self): + if self.form['@reauth_password'].value == 'LetMeIn': + return True + + return old_verify(self) + + ReauthAction.verifyPassword = new_verify_password + +will accept the passphrase ``LetMeIn`` as well as the user's +password. This example (which you should not use as is) could be +adapted to verify a `Time-based One-Time Password (TOTP)`_. An example +of `implementing a TOTP for Roundup is available on the wiki`_. + +.. _`Time-based One-Time Password (TOTP)`: + https://en.wikipedia.org/wiki/Time-based_One-Time_Password + +.. _`implementing a TOTP for Roundup is available on the wiki`: + https://wiki.roundup-tracker.org/OneTimePasswords + +If the verification succeeds, the original action (e.g. edit) is +invoked on the data sent with the reauth request. To prevent the +auditor from triggering another Reauth, the attribute ``reauth_done`` +is added to the db object. As a result, the ``hasattr`` call shown +above will return True and the Reauth exception is not raised. (Note +that the value of the ``reauth_done`` attribute is True, so +``getattr(db, "reauth_done", False)`` will return True when reauth is +done and the defaul value of False if the attribute is missing. If the +default is not set, `getattr` raises an ``AttributeError`` which might +be useful for flow control.) + +There is only one reauth for a submitted change. You cannot Reauth +multiple properties separately. If you need to reauth multiple +properties separately, you need to reject the change and force the +user to submit each sensitive property separately. For example:: + + if 'password' in newvalues and 'realname' in newvalues: + raise Reject('Changing the username and the realname ' + 'at the same time is not allowed. Please ' + 'submit two changes.') + + if 'password' in newvalues and not hasattr(db, 'reauth_done'): + raise Reauth() + + if 'realname' in newvalues and not hasattr(db, 'reauth_done'): + raise Reauth() + +See also: client.py:Client:reauth(self, exception) which can be +changed using interfaces.py in your tracker if you have some special +handling that must be done. Generating email from Roundup -----------------------------
--- a/doc/upgrading.txt Mon Aug 11 14:01:12 2025 -0400 +++ b/doc/upgrading.txt Wed Aug 13 23:52:49 2025 -0400 @@ -117,57 +117,22 @@ useful to ask for a validated authorization. This makes sure that the user is present by typing their password. -In an auditor adding:: - - from roundup.cgi.exceptions import Reauth - - if 'password' in newvalues and not getattr(db, 'reauth_done', False): - raise Reauth('Add an optional message to the user') - -will present the user with a authorization page and optional message -when the password is changed. The page is generated from the -``_generic.reauth.html`` template by default. - -Once the user enters their password and submits the page, the -password will be verified using: - - roundup.cgi.actions:LoginAction::verifyPassword() - -If the password is correct the original change is done. - -To prevent the auditor from trigering another Reauth, the -attribute ``reauth_done`` is added to the db object. As a result, -the getattr call will return True and not raise Reauth. - -You get one reauth for a submitted change. Note you cannot Reauth -multiple properties separately. If you need to auth multiple -properties separately, you need to reject the change and force the -user to submit each sensitive property separately. For example:: - - if 'password' in newvalues and 'realname' in newvalues: - raise Reject('Changing the username and the realname ' - 'at the same time is not allowed. Please ' - 'submit two changes.') - - if 'password' in newvalues and not getattr(db, 'reauth_done', False): - raise Reauth() - - if 'realname' in newvalues and not getattr(db, 'reauth_done', False): - raise Reauth() - -See also: client.py:Client:reauth(self, exception) which can be changed -using interfaces.py in your tracker. - -You should copy ``_generic.reauth.html`` into your tracker's html -subdirectory. See the classic template directory for a copy. If you -are using jinja2, see the jinja2 template directory. Then you can -raise a Reauth exception and have the proper page displayed. +You can add this to your auditors using the example +:ref:`sensitive_changes`. + +To use this, you must copy ``_generic.reauth.html`` into your +tracker's html subdirectory. See the classic template directory for a +copy. If you are using jinja2, see the jinja2 template directory. +Then you can raise a Reauth exception and have the proper page +displayed. Also javascript *MUST* be turned on if this is used with a file input. If JavaScript is not turned on, attached files are lost during the reauth step. Information from other types of inputs (password, date, text etc.) do not need JavaScript to work. +See :ref:`Confirming the User` in the reference manual for details. + .. index:: Upgrading; 2.4.0 to 2.5.0 Migrating from 2.4.0 to 2.5.0
--- a/roundup/cgi/actions.py Mon Aug 11 14:01:12 2025 -0400 +++ b/roundup/cgi/actions.py Wed Aug 13 23:52:49 2025 -0400 @@ -75,18 +75,18 @@ validates: For each component, Appendix A of RFC 3986 says the following - are allowed: + are allowed:: - pchar = unreserved / pct-encoded / sub-delims / ":" / "@" - query = *( pchar / "/" / "?" ) - unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~" - pct-encoded = "%" HEXDIG HEXDIG - sub-delims = "!" / "$" / "&" / "'" / "(" / ")" - / "*" / "+" / "," / ";" / "=" + pchar = unreserved / pct-encoded / sub-delims / ":" / "@" + query = *( pchar / "/" / "?" ) + unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~" + pct-encoded = "%" HEXDIG HEXDIG + sub-delims = "!" / "$" / "&" / "'" / "(" / ")" + / "*" / "+" / "," / ";" / "=" Checks all parts with a regexp that matches any run of 0 or more allowed characters. If the component doesn't validate, - raise ValueError. Don't attempt to urllib_.quote it. Either + raise ValueError. Don't attempt to ``urllib_.quote`` it. Either it's correct as it comes in or it's a ValueError. Finally paste the whole thing together and return the new url. @@ -1886,7 +1886,7 @@ if 'realname' in newvalues and not getattr(db, 'reauth_done', False): raise Reauth() - Limitations: It can not handle file input fields. + Limitations: Handling file inputs requires JavaScript on the browser. See also: client.py:Client:reauth() which can be changed using interfaces.py in your tracker. @@ -1968,6 +1968,13 @@ self.add_error_message(str(err), escape=escape) def verifyPassword(self): + """Verify the reauth password/token + + This can be overridden using interfaces.py. + + The default implementation uses the + LoginAction::verifyPassword() method. + """ login = self.client.get_action_class('login')(self.client) return login.verifyPassword(self.userid, self.form['@reauth_password'].value)
--- a/roundup/cgi/client.py Mon Aug 11 14:01:12 2025 -0400 +++ b/roundup/cgi/client.py Wed Aug 13 23:52:49 2025 -0400 @@ -1406,10 +1406,12 @@ Header is ok (return True) if ORIGIN is missing and it is a GET. Header is ok if ORIGIN matches the base url. - If this is a API call: - Header is ok if ORIGIN matches an element of allowed_api_origins. - Header is ok if allowed_api_origins includes '*' as first - element and credentials is False. + If this is an API call: + + * Header is ok if ORIGIN matches an element of allowed_api_origins. + * Header is ok if allowed_api_origins includes '*' as first + element and credentials is False. + Otherwise header is not ok. In a credentials context, if we match * we will return
--- a/roundup/cgi/templating.py Mon Aug 11 14:01:12 2025 -0400 +++ b/roundup/cgi/templating.py Wed Aug 13 23:52:49 2025 -0400 @@ -3618,10 +3618,18 @@ def embed_form_fields(self, excluded_fields=None): """Used to create a hidden input field for each client.form element - :param: excluded_fields string or something with - __contains__ dunder method (tuple, list, set...). - - Limitations: It ignores file input fields. + :param excluded_fields: + these fields will not have a hidden field created for them. + Value can be a string or multiple strings contained in + something with a __contains__ dunder method: + tuple, list, set.... + + File input fields are represented by a <pre> tag with + base64 encoded contents and attributes to store the + filename and mimetype. It requires JavaScript on the + browser to turn these <pre> tags back into files that can + be submitted with the form. + """ if excluded_fields is None: excluded_fields = ()
--- a/share/roundup/templates/classic/html/_generic.reauth.html Mon Aug 11 14:01:12 2025 -0400 +++ b/share/roundup/templates/classic/html/_generic.reauth.html Wed Aug 13 23:52:49 2025 -0400 @@ -16,7 +16,7 @@ your change.</p> <form id="reauth_form" method="POST" enctype="multipart/form-data"> - <input name="@reauth_password" type="password" autofocus> + <input name="@reauth_password" type="password" spellcheck="false" autofocus> <input type="hidden" name="@action" value="reauth"> <input type="submit" name="submit" value=" Authorize Change " i18n:attributes="value"> @@ -64,61 +64,67 @@ 'use strict'; (function attach_file_data() { - - const pre_file_list = document.querySelectorAll('pre[data-mimetype]'); - /* if no files, skip all this. */ - if (! pre_file_list.length) return; - + console.time('reattach_files'); - function base64ToUint8Array(base64String) { - // source: google search AI: "turn atob into uint8array javascript" + /* skip file entries without a name (created by empty file input) */ + const pre_file_list = document.querySelectorAll( + 'pre[data-filename]:not([data-filename=""]'); + /* if no files, skip all this. */ + if (! pre_file_list.length) { + console.timeEnd('reattach_files'); + return; + } - // Decode the Base64 string - const binaryString = window.atob(base64String); + function base64ToUint8Array(base64String) { + // source: google search AI: "turn atob into uint8array javascript" - // Create a Uint8Array with the same length as the binary string - const uint8Array = new Uint8Array(binaryString.length); - - // Populate the Uint8Array with the character codes - for (let i = 0; i < binaryString.length; i++) { - uint8Array[i] = binaryString.charCodeAt(i); - } + // Decode the Base64 string + const binaryString = window.atob(base64String); + + // Create a Uint8Array with the same length as the binary string + const uint8Array = new Uint8Array(binaryString.length); - return uint8Array; + // Populate the Uint8Array with the character codes + for (let i = 0; i < binaryString.length; i++) { + uint8Array[i] = binaryString.charCodeAt(i); } + return uint8Array; + } - const transfer = new DataTransfer(); + const transfer = new DataTransfer(); - pre_file_list.forEach( file => - transfer.items.add( - new File([base64ToUint8Array(file.textContent)], - file.dataset.filename, - {"type": file.dataset.mimetype} - ) - ) + pre_file_list.forEach( file => + transfer.items.add( + new File([base64ToUint8Array(file.textContent)], + file.dataset.filename, + {"type": file.dataset.mimetype} + ) ) + ) - const form = document.querySelector("#reauth_form") - if (!form) alert("Unable to find form with id=reauth_form on page.\n\n" + - "Please notify your administrator") - let file_input = document.createElement('input') - /* make it hidden first so no flash on screen */ - file_input.setAttribute("hidden", "") - file_input = form.appendChild(file_input) - /* Set the rest of the attributes now that is in the DOM. - One report said some attributes only worked once it - was added to the DOM. - */ - file_input.setAttribute("type", "file") - file_input.setAttribute("name", "@file") - /* Put all the files on one file input rather than - separate inputs. AFAICT there is no benefit to - creating a bunch of single file inputs and assigning - the files one by one. - */ - file_input.setAttribute("multiple", "") - file_input.files = transfer.files -})() + const form = document.querySelector("#reauth_form") + if (!form) alert("Unable to find form with id=reauth_form on page.\n\n" + + "Please notify your administrator") + let file_input = document.createElement('input') + /* make it hidden first so no flash on screen */ + file_input.setAttribute("hidden", "") + file_input = form.appendChild(file_input) + /* Set the rest of the attributes now that is in the DOM. + One report said some attributes only worked once it + was added to the DOM. + */ + file_input.setAttribute("type", "file") + file_input.setAttribute("name", "@file") + /* Put all the files on one file input rather than + separate inputs. AFAICT there is no benefit to + creating a bunch of single file inputs and assigning + the files one by one. + */ + file_input.setAttribute("multiple", "") + file_input.files = transfer.files + + console.timeEnd('reattach_files'); +})() </script> </td> </tal:block>
--- a/share/roundup/templates/devel/html/_generic.reauth.html Mon Aug 11 14:01:12 2025 -0400 +++ b/share/roundup/templates/devel/html/_generic.reauth.html Wed Aug 13 23:52:49 2025 -0400 @@ -16,7 +16,7 @@ your change.</p> <form id="reauth_form" method="POST" enctype="multipart/form-data"> - <input name="@reauth_password" type="password" autofocus> + <input name="@reauth_password" type="password" spellcheck="false" autofocus> <input type="hidden" name="@action" value="reauth"> <input type="submit" name="submit" value=" Authorize Change " i18n:attributes="value"> @@ -64,61 +64,67 @@ 'use strict'; (function attach_file_data() { - - const pre_file_list = document.querySelectorAll('pre[data-mimetype]'); - /* if no files, skip all this. */ - if (! pre_file_list.length) return; - + console.time('reattach_files'); - function base64ToUint8Array(base64String) { - // source: google search AI: "turn atob into uint8array javascript" + /* skip file entries without a name (created by empty file input) */ + const pre_file_list = document.querySelectorAll( + 'pre[data-filename]:not([data-filename=""]'); + /* if no files, skip all this. */ + if (! pre_file_list.length) { + console.timeEnd('reattach_files'); + return; + } - // Decode the Base64 string - const binaryString = window.atob(base64String); + function base64ToUint8Array(base64String) { + // source: google search AI: "turn atob into uint8array javascript" - // Create a Uint8Array with the same length as the binary string - const uint8Array = new Uint8Array(binaryString.length); - - // Populate the Uint8Array with the character codes - for (let i = 0; i < binaryString.length; i++) { - uint8Array[i] = binaryString.charCodeAt(i); - } + // Decode the Base64 string + const binaryString = window.atob(base64String); + + // Create a Uint8Array with the same length as the binary string + const uint8Array = new Uint8Array(binaryString.length); - return uint8Array; + // Populate the Uint8Array with the character codes + for (let i = 0; i < binaryString.length; i++) { + uint8Array[i] = binaryString.charCodeAt(i); } + return uint8Array; + } - const transfer = new DataTransfer(); + const transfer = new DataTransfer(); - pre_file_list.forEach( file => - transfer.items.add( - new File([base64ToUint8Array(file.textContent)], - file.dataset.filename, - {"type": file.dataset.mimetype} - ) - ) + pre_file_list.forEach( file => + transfer.items.add( + new File([base64ToUint8Array(file.textContent)], + file.dataset.filename, + {"type": file.dataset.mimetype} + ) ) + ) - const form = document.querySelector("#reauth_form") - if (!form) alert("Unable to find form with id=reauth_form on page.\n\n" + - "Please notify your administrator") - let file_input = document.createElement('input') - /* make it hidden first so no flash on screen */ - file_input.setAttribute("hidden", "") - file_input = form.appendChild(file_input) - /* Set the rest of the attributes now that is in the DOM. - One report said some attributes only worked once it - was added to the DOM. - */ - file_input.setAttribute("type", "file") - file_input.setAttribute("name", "@file") - /* Put all the files on one file input rather than - separate inputs. AFAICT there is no benefit to - creating a bunch of single file inputs and assigning - the files one by one. - */ - file_input.setAttribute("multiple", "") - file_input.files = transfer.files -})() + const form = document.querySelector("#reauth_form") + if (!form) alert("Unable to find form with id=reauth_form on page.\n\n" + + "Please notify your administrator") + let file_input = document.createElement('input') + /* make it hidden first so no flash on screen */ + file_input.setAttribute("hidden", "") + file_input = form.appendChild(file_input) + /* Set the rest of the attributes now that is in the DOM. + One report said some attributes only worked once it + was added to the DOM. + */ + file_input.setAttribute("type", "file") + file_input.setAttribute("name", "@file") + /* Put all the files on one file input rather than + separate inputs. AFAICT there is no benefit to + creating a bunch of single file inputs and assigning + the files one by one. + */ + file_input.setAttribute("multiple", "") + file_input.files = transfer.files + + console.timeEnd('reattach_files'); +})() </script> </td> </tal:block>
--- a/share/roundup/templates/jinja2/html/_generic.reauth.html Mon Aug 11 14:01:12 2025 -0400 +++ b/share/roundup/templates/jinja2/html/_generic.reauth.html Wed Aug 13 23:52:49 2025 -0400 @@ -1,58 +1,39 @@ -<tal:block metal:use-macro="templates/page/macros/icing"> -<title metal:fill-slot="head_title" i18n:translate="">Authorize - <span - i18n:name="tracker" tal:replace="config/TRACKER_NAME" /></title> -<span metal:fill-slot="body_title" tal:omit-tag="python:1" - i18n:translate="">Authorize Change</span> -<td class="content" metal:fill-slot="content"> +{% extends 'layout/page.html' %} + +{% block head_title %} + {% trans %}Authorize{% endtrans %} - {{ config.TRACKER_NAME }} +{% endblock %} + +{% block page_header %} + {% trans %}Authorize Change{% endtrans %} +{% endblock %} - <h2>Authorization required</h2> +{% block content %} - <p tal:condition="python:'@reauth_message' in request.client.form" - tal:content="request/client/form/@reauth_message/value"></p> + {% include 'layout/permission.html' %} - <p i18n:translate="">The action you requested needs to be - authorized.</p> - <p i18n:translate="">Please enter your password to continue with - your change.</p> + {% if '@reauth_message' in request.client.form %} + <p> {{ request.client.form['@reauth_message'].value }} </p> + {% endif %} + <p> {% trans %}The action you requested needs to be + authorized.{% endtrans %}</p> + <p> {% trans %}Please enter your password to continue with your + change. {% endtrans %}</p> <form id="reauth_form" method="POST" enctype="multipart/form-data"> - <input name="@reauth_password" type="password" autofocus> - <input type="hidden" name="@action" value="reauth"> - <input type="submit" name="submit" value=" Authorize Change " - i18n:attributes="value"> - <input name="@csrf" type="hidden" - tal:attributes="value python:utils.anti_csrf_nonce()"> - - - <tal:comment tal:replace="nothing"> - Embed all fields from the original form as hidden - fields. Once the reauth is done, these fields will be - processed to make the change that was requested. + <input name="@reauth_password" type="password" + spellcheck="false" autofocus class="form-control"> + <input type="hidden" name="@action" value="reauth"> + <input type="submit" name="submit" + value="{% trans %} Authorize Change {% endtrans %}"> + <input name="@csrf" type="hidden" + value="{{ python:utils.anti_csrf_nonce() }}"> - Standard fields like: @action, @csrf and @template are stripped - by the code that handles Reauth requests. But there can be a few - fields that still need to be stripped based on the template that - generated the reauth process. - - Use the templating function to make this easier and safer. - - utils:embed_form_fields(excluded_fields=('fieldname1', 'fieldname2')) + {{ utils.embed_form_fields(('submit',)) }} - excluded_fields can be any object with a __contains__ dunder - method: lists, set, tuple... - - embed_form_fields encodes values and names safely even if a user - uses a name like '"><script>alert("hello")</script>'' - - It also base64 encodes the contents of file inputs into pre blocks. - The textContent of these blocks is then processed by javascript - to recreate a file input. - </tal:comment> - <tal:x tal:content="structure - python:utils.embed_form_fields(('submit',))" /> </form> -<script tal:attributes="nonce request/client/client_nonce"> +<script nonce="{{ request.client.client_nonce() }}"> /* This IIFE decodes the base64 file contents in the pre blocks, creates a new file blob for each one. Then adds a multiple file input and attaches all the files to it. @@ -64,61 +45,67 @@ 'use strict'; (function attach_file_data() { - - const pre_file_list = document.querySelectorAll('pre[data-mimetype]'); - /* if no files, skip all this. */ - if (! pre_file_list.length) return; - + console.time('reattach_files'); - function base64ToUint8Array(base64String) { - // source: google search AI: "turn atob into uint8array javascript" + /* skip file entries without a name (created by empty file input) */ + const pre_file_list = document.querySelectorAll( + 'pre[data-filename]:not([data-filename=""]'); + /* if no files, skip all this. */ + if (! pre_file_list.length) { + console.timeEnd('reattach_files'); + return; + } - // Decode the Base64 string - const binaryString = window.atob(base64String); + function base64ToUint8Array(base64String) { + // source: google search AI: "turn atob into uint8array javascript" - // Create a Uint8Array with the same length as the binary string - const uint8Array = new Uint8Array(binaryString.length); - - // Populate the Uint8Array with the character codes - for (let i = 0; i < binaryString.length; i++) { - uint8Array[i] = binaryString.charCodeAt(i); - } + // Decode the Base64 string + const binaryString = window.atob(base64String); + + // Create a Uint8Array with the same length as the binary string + const uint8Array = new Uint8Array(binaryString.length); - return uint8Array; + // Populate the Uint8Array with the character codes + for (let i = 0; i < binaryString.length; i++) { + uint8Array[i] = binaryString.charCodeAt(i); } + return uint8Array; + } - const transfer = new DataTransfer(); + const transfer = new DataTransfer(); - pre_file_list.forEach( file => - transfer.items.add( - new File([base64ToUint8Array(file.textContent)], - file.dataset.filename, - {"type": file.dataset.mimetype} - ) - ) + pre_file_list.forEach( file => + transfer.items.add( + new File([base64ToUint8Array(file.textContent)], + file.dataset.filename, + {"type": file.dataset.mimetype} + ) ) + ) - const form = document.querySelector("#reauth_form") - if (!form) alert("Unable to find form with id=reauth_form on page.\n\n" + - "Please notify your administrator") - let file_input = document.createElement('input') - /* make it hidden first so no flash on screen */ - file_input.setAttribute("hidden", "") - file_input = form.appendChild(file_input) - /* Set the rest of the attributes now that is in the DOM. - One report said some attributes only worked once it - was added to the DOM. - */ - file_input.setAttribute("type", "file") - file_input.setAttribute("name", "@file") - /* Put all the files on one file input rather than - separate inputs. AFAICT there is no benefit to - creating a bunch of single file inputs and assigning - the files one by one. - */ - file_input.setAttribute("multiple", "") - file_input.files = transfer.files -})() + const form = document.querySelector("#reauth_form") + if (!form) alert("Unable to find form with id=reauth_form on page.\n\n" + + "Please notify your administrator") + let file_input = document.createElement('input') + /* make it hidden first so no flash on screen */ + file_input.setAttribute("hidden", "") + file_input = form.appendChild(file_input) + /* Set the rest of the attributes now that is in the DOM. + One report said some attributes only worked once it + was added to the DOM. + */ + file_input.setAttribute("type", "file") + file_input.setAttribute("name", "@file") + /* Put all the files on one file input rather than + separate inputs. AFAICT there is no benefit to + creating a bunch of single file inputs and assigning + the files one by one. + */ + file_input.setAttribute("multiple", "") + file_input.files = transfer.files + + console.timeEnd('reattach_files'); +})() </script> -</td> -</tal:block> + +{% endblock %}
--- a/share/roundup/templates/minimal/html/_generic.reauth.html Mon Aug 11 14:01:12 2025 -0400 +++ b/share/roundup/templates/minimal/html/_generic.reauth.html Wed Aug 13 23:52:49 2025 -0400 @@ -16,7 +16,7 @@ your change.</p> <form id="reauth_form" method="POST" enctype="multipart/form-data"> - <input name="@reauth_password" type="password" autofocus> + <input name="@reauth_password" type="password" spellcheck="false" autofocus> <input type="hidden" name="@action" value="reauth"> <input type="submit" name="submit" value=" Authorize Change " i18n:attributes="value"> @@ -64,61 +64,67 @@ 'use strict'; (function attach_file_data() { - - const pre_file_list = document.querySelectorAll('pre[data-mimetype]'); - /* if no files, skip all this. */ - if (! pre_file_list.length) return; - + console.time('reattach_files'); - function base64ToUint8Array(base64String) { - // source: google search AI: "turn atob into uint8array javascript" + /* skip file entries without a name (created by empty file input) */ + const pre_file_list = document.querySelectorAll( + 'pre[data-filename]:not([data-filename=""]'); + /* if no files, skip all this. */ + if (! pre_file_list.length) { + console.timeEnd('reattach_files'); + return; + } - // Decode the Base64 string - const binaryString = window.atob(base64String); + function base64ToUint8Array(base64String) { + // source: google search AI: "turn atob into uint8array javascript" - // Create a Uint8Array with the same length as the binary string - const uint8Array = new Uint8Array(binaryString.length); - - // Populate the Uint8Array with the character codes - for (let i = 0; i < binaryString.length; i++) { - uint8Array[i] = binaryString.charCodeAt(i); - } + // Decode the Base64 string + const binaryString = window.atob(base64String); + + // Create a Uint8Array with the same length as the binary string + const uint8Array = new Uint8Array(binaryString.length); - return uint8Array; + // Populate the Uint8Array with the character codes + for (let i = 0; i < binaryString.length; i++) { + uint8Array[i] = binaryString.charCodeAt(i); } + return uint8Array; + } - const transfer = new DataTransfer(); + const transfer = new DataTransfer(); - pre_file_list.forEach( file => - transfer.items.add( - new File([base64ToUint8Array(file.textContent)], - file.dataset.filename, - {"type": file.dataset.mimetype} - ) - ) + pre_file_list.forEach( file => + transfer.items.add( + new File([base64ToUint8Array(file.textContent)], + file.dataset.filename, + {"type": file.dataset.mimetype} + ) ) + ) - const form = document.querySelector("#reauth_form") - if (!form) alert("Unable to find form with id=reauth_form on page.\n\n" + - "Please notify your administrator") - let file_input = document.createElement('input') - /* make it hidden first so no flash on screen */ - file_input.setAttribute("hidden", "") - file_input = form.appendChild(file_input) - /* Set the rest of the attributes now that is in the DOM. - One report said some attributes only worked once it - was added to the DOM. - */ - file_input.setAttribute("type", "file") - file_input.setAttribute("name", "@file") - /* Put all the files on one file input rather than - separate inputs. AFAICT there is no benefit to - creating a bunch of single file inputs and assigning - the files one by one. - */ - file_input.setAttribute("multiple", "") - file_input.files = transfer.files -})() + const form = document.querySelector("#reauth_form") + if (!form) alert("Unable to find form with id=reauth_form on page.\n\n" + + "Please notify your administrator") + let file_input = document.createElement('input') + /* make it hidden first so no flash on screen */ + file_input.setAttribute("hidden", "") + file_input = form.appendChild(file_input) + /* Set the rest of the attributes now that is in the DOM. + One report said some attributes only worked once it + was added to the DOM. + */ + file_input.setAttribute("type", "file") + file_input.setAttribute("name", "@file") + /* Put all the files on one file input rather than + separate inputs. AFAICT there is no benefit to + creating a bunch of single file inputs and assigning + the files one by one. + */ + file_input.setAttribute("multiple", "") + file_input.files = transfer.files + + console.timeEnd('reattach_files'); +})() </script> </td> </tal:block>
--- a/share/roundup/templates/responsive/html/_generic.reauth.html Mon Aug 11 14:01:12 2025 -0400 +++ b/share/roundup/templates/responsive/html/_generic.reauth.html Wed Aug 13 23:52:49 2025 -0400 @@ -16,7 +16,7 @@ your change.</p> <form id="reauth_form" method="POST" enctype="multipart/form-data"> - <input name="@reauth_password" type="password" autofocus> + <input name="@reauth_password" type="password" spellcheck="false" autofocus> <input type="hidden" name="@action" value="reauth"> <input type="submit" name="submit" value=" Authorize Change " i18n:attributes="value"> @@ -64,61 +64,67 @@ 'use strict'; (function attach_file_data() { - - const pre_file_list = document.querySelectorAll('pre[data-mimetype]'); - /* if no files, skip all this. */ - if (! pre_file_list.length) return; - + console.time('reattach_files'); - function base64ToUint8Array(base64String) { - // source: google search AI: "turn atob into uint8array javascript" + /* skip file entries without a name (created by empty file input) */ + const pre_file_list = document.querySelectorAll( + 'pre[data-filename]:not([data-filename=""]'); + /* if no files, skip all this. */ + if (! pre_file_list.length) { + console.timeEnd('reattach_files'); + return; + } - // Decode the Base64 string - const binaryString = window.atob(base64String); + function base64ToUint8Array(base64String) { + // source: google search AI: "turn atob into uint8array javascript" - // Create a Uint8Array with the same length as the binary string - const uint8Array = new Uint8Array(binaryString.length); - - // Populate the Uint8Array with the character codes - for (let i = 0; i < binaryString.length; i++) { - uint8Array[i] = binaryString.charCodeAt(i); - } + // Decode the Base64 string + const binaryString = window.atob(base64String); + + // Create a Uint8Array with the same length as the binary string + const uint8Array = new Uint8Array(binaryString.length); - return uint8Array; + // Populate the Uint8Array with the character codes + for (let i = 0; i < binaryString.length; i++) { + uint8Array[i] = binaryString.charCodeAt(i); } + return uint8Array; + } - const transfer = new DataTransfer(); + const transfer = new DataTransfer(); - pre_file_list.forEach( file => - transfer.items.add( - new File([base64ToUint8Array(file.textContent)], - file.dataset.filename, - {"type": file.dataset.mimetype} - ) - ) + pre_file_list.forEach( file => + transfer.items.add( + new File([base64ToUint8Array(file.textContent)], + file.dataset.filename, + {"type": file.dataset.mimetype} + ) ) + ) - const form = document.querySelector("#reauth_form") - if (!form) alert("Unable to find form with id=reauth_form on page.\n\n" + - "Please notify your administrator") - let file_input = document.createElement('input') - /* make it hidden first so no flash on screen */ - file_input.setAttribute("hidden", "") - file_input = form.appendChild(file_input) - /* Set the rest of the attributes now that is in the DOM. - One report said some attributes only worked once it - was added to the DOM. - */ - file_input.setAttribute("type", "file") - file_input.setAttribute("name", "@file") - /* Put all the files on one file input rather than - separate inputs. AFAICT there is no benefit to - creating a bunch of single file inputs and assigning - the files one by one. - */ - file_input.setAttribute("multiple", "") - file_input.files = transfer.files -})() + const form = document.querySelector("#reauth_form") + if (!form) alert("Unable to find form with id=reauth_form on page.\n\n" + + "Please notify your administrator") + let file_input = document.createElement('input') + /* make it hidden first so no flash on screen */ + file_input.setAttribute("hidden", "") + file_input = form.appendChild(file_input) + /* Set the rest of the attributes now that is in the DOM. + One report said some attributes only worked once it + was added to the DOM. + */ + file_input.setAttribute("type", "file") + file_input.setAttribute("name", "@file") + /* Put all the files on one file input rather than + separate inputs. AFAICT there is no benefit to + creating a bunch of single file inputs and assigning + the files one by one. + */ + file_input.setAttribute("multiple", "") + file_input.files = transfer.files + + console.timeEnd('reattach_files'); +})() </script> </td> </tal:block>
--- a/test/test_liveserver.py Mon Aug 11 14:01:12 2025 -0400 +++ b/test/test_liveserver.py Wed Aug 13 23:52:49 2025 -0400 @@ -9,6 +9,7 @@ from roundup.cgi.wsgi_handler import RequestDispatcher from .wsgi_liveserver import LiveServerTestCase from . import db_test_base +from textwrap import dedent from time import sleep from .test_postgresql import skip_postgresql @@ -109,6 +110,20 @@ # set up and open a tracker cls.instance = db_test_base.setupTracker(cls.dirname, cls.backend) + # add an auditor that triggers a Reauth + with open("%s/detectors/reauth.py" % cls.dirname, "w") as f: + auditor = dedent(""" + from roundup.cgi.exceptions import Reauth + + def trigger_reauth(db, cl, nodeid, newvalues): + if 'realname' in newvalues and not hasattr(db, 'reauth_done'): + raise Reauth('Add an optional message to the user') + + def init(db): + db.user.audit('set', trigger_reauth, priority=110) + """) + f.write(auditor) + # open the database cls.db = cls.instance.open('admin') @@ -116,6 +131,12 @@ cls.db.user.create(username="fred", roles='User', password=password.Password('sekrit'), address='fred@example.com') + # add a user for reauth tests + cls.db.user.create(username="reauth", + realname="reauth test user", + password=password.Password("reauth"), + address="reauth@example.com", roles="User") + # set the url the test instance will run at. cls.db.config['TRACKER_WEB'] = cls.tracker_web # set up mailhost so errors get reported to debuging capture file @@ -123,6 +144,9 @@ cls.db.config.MAIL_HOST = "localhost" cls.db.config.MAIL_DEBUG = "../_test_tracker_mail.log" + # also report it in the web. + cls.db.config.WEB_DEBUG = "yes" + # added to enable csrf forgeries/CORS to be tested cls.db.config.WEB_CSRF_ENFORCE_HEADER_ORIGIN = "required" cls.db.config.WEB_ALLOWED_API_ORIGINS = "https://client.com" @@ -336,6 +360,171 @@ wsgi server is started with various feature flags """ + def test_reauth_workflow(self): + """as admin user: + change reauth user realname include all fields on the form + also add a dummy file to the submitted request. + get back a reauth page/template (look for id="reauth_form") + verify hidden input for realname + verify hidden input for roles + verify the base 64 file content are on the page. + + submit form with bad password + verify error reported + verify hidden input for realname + (note the file contents will be gone because + preserving that requires javascript) + + enter good password + verify on user page (look for + "(the default is 0)" hint for timezone) + verify new name present + verify success banner + """ + from html.parser import HTMLParser + class HTMLExtractForm(HTMLParser): + """Custom parser to extract input fields from a form. + + Set the form_label to extract inputs only inside a form + with a name or id matching form_label. Default is + "reauth_form". + + Set names to a tuple/list/set with the names of the + inputs you are interested in. Defalt is None which + extracts all inputs on the page with a name property. + """ + def __init__(self, names=None, form_label="reauth_form"): + super().__init__() + self.fields = {} + self.names = names + self.form_label = form_label + self._inside_form = False + + def handle_starttag(self, tag, attrs): + if tag == 'form': + for attr, value in attrs: + if attr in ('id', 'name') and value == self.form_label: + self._inside_form = True + return + + if not self._inside_form: return + + if tag == 'input': + field_name = None + field_value = None + for attr, value in attrs: + if attr == 'name': + field_name = value + if attr == 'value': + field_value = value + + # skip input type="submit" without name + if not field_name: return + + if self.names is None: + self.fields[field_name] = field_value + elif field_name in self.names: + self.fields[field_name] = field_value + + def handle_endtag(self, tag): + if tag == "form": + self._inside_form = False + + def get_fields(self): + return self.fields + + + user_url = "%s/user%s" % (self.url_base(), + self.db.user.lookup('reauth')) + + session, _response = self.create_login_session() + + user_page = session.get(user_url) + + self.assertEqual(user_page.status_code, 200) + self.assertTrue(b'reauth' in user_page.content) + + parser = HTMLExtractForm(('@lastactivity', '@csrf'), 'itemSynopsis') + parser.feed(user_page.text) + + change = {"realname": "reauth1", + "username": "reauth", + "password": "", + "@confirm@password": "", + "phone": "", + "organisation": "", + "roles": "User", + "timezone": "", + "address": "reauth@example.com", + "alternate_addresses": "", + "@template": "item", + "@required": "username,address", + "@submit_button": "Submit Changes", + "@action": "edit", + **parser.get_fields() + } + lastactivity = parser.get_fields()['@lastactivity'] + + # make the simple name/value dict into a name/tuple dict + # setting tuple[0] to None to indicate pulre string + # value. Then we use change2 with file to trigger + # multipart/form-data form encoding which preserves fields + # with empty values. application/x-www-form-urlencoded forms + # have fields with empty values dropped by cgi by default. + userpage_change = {key: (None, value) for key, value in change.items()} + userpage_change.update({"@file": ("filename.txt", "this is some text")}) + + on_reauth = session.post(user_url, files=userpage_change) + + self.assertIn(b'id="reauth_form"', on_reauth.content) + self.assertIn(b'Please enter your password to continue with', + on_reauth.content) + # make sure the base64 encoded content for @file is present on + # the page. Because we are not running a javascript capable + # browser, it is not converted into an actual file input. + # But this check shows that a file generated by reauth is trying + # to maintain the file input. + self.assertIn(b'dGhpcyBpcyBzb21lIHRleHQ=', on_reauth.content) + + parser = HTMLExtractForm() + parser.feed(on_reauth.text) + fields = parser.get_fields() + self.assertEqual(fields["@lastactivity"], lastactivity) + self.assertEqual(fields["@next_action"], "edit") + self.assertEqual(fields["@action"], "reauth") + self.assertEqual(fields["address"], "reauth@example.com") + self.assertEqual(fields["phone"], "") + self.assertEqual(fields["roles"], "User") + self.assertEqual(fields["realname"], "reauth1") + + reauth_fields = { + "@reauth_password": (None, "sekret not right"), + "submit": (None, " Authorize Change "), + } + reauth_submit = {key: (None, value) for key, value in fields.items()} + reauth_submit.update(reauth_fields) + + fail_reauth = session.post(user_url, + files=reauth_submit) + self.assertIn(b'id="reauth_form"', fail_reauth.content) + self.assertIn(b'Please enter your password to continue with', + fail_reauth.content) + self.assertIn(b'Password incorrect', fail_reauth.content) + + parser = HTMLExtractForm(('@csrf',)) + parser.feed(fail_reauth.text) + # remeber we are logged in as admin - use admin pw. + reauth_submit.update({"@reauth_password": (None, "sekrit"), + "@csrf": + (None, parser.get_fields()['@csrf'])}) + pass_reauth = session.post(user_url, + files=reauth_submit) + self.assertNotIn(b'id="reauth_form"', pass_reauth.content) + self.assertNotIn(b'Please enter your password to continue with', + pass_reauth.content) + self.assertIn(b'user 4 realname edited ok', pass_reauth.content) + self.assertIn(b'(the default is 0)', pass_reauth.content) + def test_cookie_attributes(self): session, _response = self.create_login_session()
--- a/website/www/Makefile Mon Aug 11 14:01:12 2025 -0400 +++ b/website/www/Makefile Wed Aug 13 23:52:49 2025 -0400 @@ -12,7 +12,7 @@ # after upgrade to sphinx 1.8.5, search.html is missing load of searchtools. # fix that in postprocess # also sed index.html to properly format meta og:... entries. -html: doc_links ## make standalone HTML files +html: doc_links ## make standalone HTML files (KEEP_HTML=1 speeds build) if [ -z "${KEEP_HTML}" ]; then rm -rf html; fi rm -f html/robots.txt # otherwise sphinx errors mkdir -p $(TMP)/doctrees $(HTML)
