http://hg.code.sf.net:8000/p/roundup/code/atom-log/tip/doc/admin_guide.txt Mercurial Repository: p/roundup/code: doc/admin_guide.txt history 2026-03-25T18:00:19-04:00 doc: inplace db migration; replace [database] with [rdbms] http://hg.code.sf.net:8000/p/roundup/code/#changeset-05e21949657a03060fd3a0dbc5c46679538c39f2 John Rouillard rouilj@ieee.org 2026-03-25T18:00:19-04:00 2026-03-25T18:00:19-04:00
changeset 05e21949657a
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: inplace db migration; replace [database] with [rdbms]

Document inplace tracker migration using exporttable and
importtable. I just tested it with anydbm and sqlite.

Also directions included old name for [rdbms] section of config.ini
file. Fixed it and said old version of roundup may have a [database]
section.
files
doc: fix bad formatting arg in json logging config file. http://hg.code.sf.net:8000/p/roundup/code/#changeset-0ad1e7d563a6c0ebac1e82b6746f995b77ec663a John Rouillard rouilj@ieee.org 2025-12-22T12:21:33-05:00 2025-12-22T12:21:33-05:00
changeset 0ad1e7d563a6
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: fix bad formatting arg in json logging config file.
files
merge heads: CHANGES.txt also whitespace fixups. http://hg.code.sf.net:8000/p/roundup/code/#changeset-7142740e6547acc933baab05a985441a2bce2f45 John Rouillard rouilj@ieee.org 2025-12-08T23:05:26-05:00 2025-12-08T23:05:26-05:00
changeset 7142740e6547
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description merge heads: CHANGES.txt also whitespace fixups.
files
feat: add nanoid pkg trace_id gen and decorator for setting processName http://hg.code.sf.net:8000/p/roundup/code/#changeset-b09ef85f0da62646efe031ee76934ee9b158ae07 John Rouillard rouilj@ieee.org 2025-12-08T00:23:14-05:00 2025-12-08T00:23:14-05:00
changeset b09ef85f0da6
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat: add nanoid pkg trace_id gen and decorator for setting processName

nanoid is a shorter unique id generator and faster than uuid.
I truncate nanoid id's to 12 chars to make it more readable.

Also added decorator to allow setting the default processName
definition in the logging module. admin.py and wsgi_handler now set
processName. configuration.py knows how to overide the processName if
set to the default MainProcess.

Updated install docs to add nanoid as optional, how to switch to
different trace_id output. pydoc generated docs include logcontext
module and are referenced from admin.py.
files
docs: key from keyserver, check key before import to production http://hg.code.sf.net:8000/p/roundup/code/#changeset-0fda84bc7584a9f2033f6e6c2f04750d2fd3c14d John Rouillard rouilj@ieee.org 2025-12-07T17:30:41-05:00 2025-12-07T17:30:41-05:00
changeset 0fda84bc7584
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: key from keyserver, check key before import to production
files
doc: initial attempt to document setup of pgp support for email. http://hg.code.sf.net:8000/p/roundup/code/#changeset-ed4ef394d5d6dac7dbd7a363d958912c64ce7ca4 John Rouillard rouilj@ieee.org 2025-11-15T16:59:24-05:00 2025-11-15T16:59:24-05:00
changeset ed4ef394d5d6
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: initial attempt to document setup of pgp support for email.

Used an AI assistant to help write this. Basic gpg commands seem to
work, but I have not tested this totally. Docs basically follow the
setup used for pgp testing in the test suite.

It looks like roundup accepts signed emails as well as encrypted
and signed emails. But it does not generate signed emails.

Also it looks like there is no PGP support for alternate email
addresses. Only primary addresses can do PGP emails.
files
doc: add docs for producing jsonl formatted output http://hg.code.sf.net:8000/p/roundup/code/#changeset-d36bd4c521acb1c21a08810d09a180ef97395bfb John Rouillard rouilj@ieee.org 2025-11-12T22:28:16-05:00 2025-11-12T22:28:16-05:00
changeset d36bd4c521ac
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: add docs for producing jsonl formatted output
files
doc: remove uWSGI references as it is in maint mode and not being updated http://hg.code.sf.net:8000/p/roundup/code/#changeset-1522e0e1590351b9be9002024a6aaa13ab951a31 John Rouillard rouilj@ieee.org 2025-09-30T18:08:14-04:00 2025-09-30T18:08:14-04:00
changeset 1522e0e15903
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: remove uWSGI references as it is in maint mode and not being updated

Replaced with Waitress where appropriate.
files
feature: add thread local trace_id and trace_reason to logging. http://hg.code.sf.net:8000/p/roundup/code/#changeset-14c7c07b32d846e405260147a3112e8c8b15642a John Rouillard rouilj@ieee.org 2025-09-16T22:53:00-04:00 2025-09-16T22:53:00-04:00
changeset 14c7c07b32d8
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feature: add thread local trace_id and trace_reason to logging.

Added trace_id to default logging so that all logs for a given request
share the same trace_id.

This allows correlation of logs across a request.

admin_guide.txt, upgrading.txt:
add docs
update sample configs to include trace_id.
rewrite logging docs in admin_guide. Hopefully they are clearer now.
clean up some stuff in the logging config file docs.

admin.py:

add decorators to run_command to enable trace_id.
change calls to db.commit() to use run_command to get trace_id.

configuration.py:

clean up imports.
update docstrings, comments and inline docs.
add trace_id to default log format.
add function for testing decorated with trace_id.
add support for dumping stack trace in logging.
add check for pytest in sys.modules to enable log propagation when
pytest is running. Otherwise tests fail as the caplog logger doesn't
see the roundup logs.

logcontext.py:

new file to handle thread local contextvar mangement.

mailgw.py:

add decorators for trace_id etc.

scripts/roundup_xlmrpc_server.py:

add decorators for trace_id etc.
fix encoding bug turning bytes into a string.
fix command line issue where we can't set encoding. (not sure if
changing encoding via command line even works)

cgi/client.py

decorate two entry points for trace_id etc.

cgi/wsgi_handler.py:

decorate entry point for trace_id etc.

test/test_config.py:

add test for trace_id in new log format.
test various cases for sinfo and errors in formating msg.
files
feat: allow admin to set logging format from config.ini http://hg.code.sf.net:8000/p/roundup/code/#changeset-39a6825d10ca01b1291dc124cd848708932054ee John Rouillard rouilj@ieee.org 2025-09-01T21:54:48-04:00 2025-09-01T21:54:48-04:00
changeset 39a6825d10ca
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat: allow admin to set logging format from config.ini

This is prep work for adding a per thread logging variable that can be
used to tie all logs for a single request together.

This uses the same default logging format as before, just moves it to
config.ini.

Also because of configparser, the logging format has to have doubled %
signs. So use:

%%(asctime)s

not '%(asctime)s' as configparser tries to interpolate that string and
asctime is not defined in the configparser's scope. Using %%(asctime)s
is not interpolated by configparser and is passed into Roundup.
files
fix missing formatting http://hg.code.sf.net:8000/p/roundup/code/#changeset-0bb29d0509c9b99e2a366e9eacc6db69e51a33a2 John Rouillard rouilj@ieee.org 2025-09-01T20:35:54-04:00 2025-09-01T20:35:54-04:00
changeset 0bb29d0509c9
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix missing formatting
files
feat: add support for ! history and readline command in roundup-admin http://hg.code.sf.net:8000/p/roundup/code/#changeset-3bdae15252c65b87f524942af3e1d019896295e1 John Rouillard rouilj@ieee.org 2025-08-31T16:54:17-04:00 2025-08-31T16:54:17-04:00
changeset 3bdae15252c6
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat: add support for ! history and readline command in roundup-admin

Ad support to change input mode emacs/vi using new 'readline'
roundup-admin command. Also bind keys to command/input strings, List
numbered history and allow rerunning a command with !<number> or allow
user to edit it using !<number>:p.

admin_guide.txt:
Added docs.

admin.py:

add functionality. Reconcile import commands to standard. Replace
IOError with FileNotFoundError no that we have removed python 2.7
support. Add support for identifying backend used to supply line
editing/history functions. Add support for saving commands sent on
stdin to history to allow preloading of history.

test_admin.py:

Test code. Can't test mode changes as lack of pty when driving command
line turns off line editing in readline/pyreadline3. Similarly can't
test key bindings/settings.

Some refactoring of test conditions that had to change because of
additional output reporting backend library.
files
feat: add 'q' as alias to quit to exit interactive roundup-admin http://hg.code.sf.net:8000/p/roundup/code/#changeset-1a93dc58f97529a96a14e3973a1589870a82b726 John Rouillard rouilj@ieee.org 2025-08-26T23:37:42-04:00 2025-08-26T23:37:42-04:00
changeset 1a93dc58f975
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat: add 'q' as alias to quit to exit interactive roundup-admin

Also require no arguments to 'q', 'quit' or 'exit' before exiting.
Now typing 'quit a' will get an unknown command error.

Add to admin-guide how to get out of interactive mode.

Also test 'q' and 'exit' commands.

No upgrading docs added. Not that big a feature. Just noted in
CHANGES. Reporting error if argument provided is unlikely to be an
issue IMO, so no upgrading.txt entry.
files
feat: change comment in dictConfig json file to // from # http://hg.code.sf.net:8000/p/roundup/code/#changeset-de1dac9abcb64889eb06281fcf1474ae44cb7e96 John Rouillard rouilj@ieee.org 2025-08-26T22:24:00-04:00 2025-08-26T22:24:00-04:00
changeset de1dac9abcb6
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat: change comment in dictConfig json file to // from #

Emacs json mode at least will properly indent when using // as a
comment character and not #.
files
doc: add disable saving roundup-admin history file for password changes http://hg.code.sf.net:8000/p/roundup/code/#changeset-7f7749d86da8bac05311974e6bb34cb8373f1be6 John Rouillard rouilj@ieee.org 2025-08-25T20:44:42-04:00 2025-08-25T20:44:42-04:00
changeset 7f7749d86da8
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: add disable saving roundup-admin history file for password changes
files
feat: add support for using dictConfig to configure logging. http://hg.code.sf.net:8000/p/roundup/code/#changeset-94eed885e9582203be3a66fa5d0c0c122cbaaa84 John Rouillard rouilj@ieee.org 2025-08-19T22:32:46-04:00 2025-08-19T22:32:46-04:00
changeset 94eed885e958
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat: add support for using dictConfig to configure logging.

Basic logging config (one level and one output file non-rotating) was
always possible from config.ini. However the LOGGING_CONFIG setting
could be used to load an ini fileConfig style file to set various
channels (e.g. roundup.hyperdb) (also called qualname or tags) with
their own logging level, destination (rotating file, socket,
/dev/null) and log format.

This is now a deprecated method in newer logging modules. The
dictConfig format is preferred and allows disabiling other loggers as
well as invoking new loggers in local code. This commit adds support
for it reading the dict from a .json file. It also implements a
comment convention so you can document the dictConfig.

configuration.py:
new code

test_config.py:
test added for the new code.

admin_guide.txt, upgrading.txt CHANGES.txt:
docs added upgrading references the section in admin_guid.
files
[reauth-confirm_id] feat: finish reauth docs, enhance code. http://hg.code.sf.net:8000/p/roundup/code/#changeset-0663a7bcef6c1ac3ff2d11bdbcff893231151377 John Rouillard rouilj@ieee.org 2025-08-13T23:52:49-04:00 2025-08-13T23:52:49-04:00
changeset 0663a7bcef6c
branch reauth-confirm_id
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat: finish reauth docs, enhance code.

Decided to keep name Reauth for now.

admin_guide.txt:
add reference mark to roundup admin help. Used for template command
reference in upgrading.txt.

customizing.txt:
added worked example of adding a reauth auditor for address and password.
Also links to OWASP recommendations.

Added link to example code in design.doc on detectors.

glossary.txt:
reference using roundup-admin template command in def for tracker
templates.

pydoc.txt:
Added methods for Client class.

Added class and methods for (cgi) Action, LoginAction and ReauthAction.

reference.txt
Edited and restructured detector section.

Added section on registering a detector and priority use/execution order.
(reference to design doc was used before).

Added/enhanced description of exception an auditor can
raise (includes Reauth).

Added section on Reauth implementation and use (Confirming the User).
Also has paragraph on future ideas.

upgrading.txt
Stripped down the original section. Moved a lot to reference.txt.

Referenced customizing example, mention installation of
_generic.reauth.html and reference reference.txt.

cgi/actions.py:
fixed bad ReST that was breaking pydoc.txt processing

changed doc on limitations of Reauth code.

added docstring for Reauth::verifyPassword

cgi/client.py:
fix ReST for a method breaking pydoc.py processing

cgi/templating.py:
fix docstring on embed_form_fields

templates/*/html/_generic.reauth.html
disable spelling for password field

add timing info to the javascript function that processes file data.

reformat javascript IIFE

templates/jinja2/html/_generic.reauth.html
create a valid jinja2 template. Looks like my original jinja
template got overwritten and committed.

feature parity with the other reauth templates.

test/test_liveserver.py
add test case for Reauth workflow.

Makefile
add doc.
files
docs: fix registerutilMethod docs, format for highlights. http://hg.code.sf.net:8000/p/roundup/code/#changeset-6ea309c6d17c1371c374908214e0a3af135006fc John Rouillard rouilj@ieee.org 2025-07-03T12:30:21-04:00 2025-07-03T12:30:21-04:00
changeset 6ea309c6d17c
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: fix registerutilMethod docs, format for highlights.
files
fix: issue2551238 - roundup-server should exit with error ... http://hg.code.sf.net:8000/p/roundup/code/#changeset-03513f5066f3d0c1900adaf0eee050dd73796822 John Rouillard rouilj@ieee.org 2025-01-20T15:51:53-05:00 2025-01-20T15:51:53-05:00
changeset 03513f5066f3
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: issue2551238 - roundup-server should exit with error ...

if -d <pidfile> is used without -l <logfile>.

Now exits with error if -l or -L are not specified with -d.

This exposed a problem with relative paths when using a logging
config file. -d enables deamon mode by default. This changes the
current direcory to /. So relative paths in a logging config file
are opened in / and fail.

Added documentation to dmin guide noting the problem.
files
feat: add templating utils method dynamically; method to set http code http://hg.code.sf.net:8000/p/roundup/code/#changeset-2a7c3eeaf167484beb05fecb7348f43d10cf8a7e John Rouillard rouilj@ieee.org 2025-01-07T20:22:33-05:00 2025-01-07T20:22:33-05:00
changeset 2a7c3eeaf167
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description feat: add templating utils method dynamically; method to set http code

Added new utils.set_http_response(integer) to set the HTML response
code from a template. Useful for error handling inside template.

Also noted that a real TemplatingUtils (like set_http_response) method
gets the TemplatingUtils object instance, but there is no way to do
this with registerUtil() from an extension file.

Added new instance.registerUtilMethod() method to register a function
in an extension as a method passing the client instance in as the first
parameter (aka self).
files
docs: add roundup-admin interactive help output to admin_guide.txt http://hg.code.sf.net:8000/p/roundup/code/#changeset-ea1f377c87d68d382c06bc1a6dd93cc294b7193c John Rouillard rouilj@ieee.org 2024-07-16T22:02:39-04:00 2024-07-16T22:02:39-04:00
changeset ea1f377c87d6
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: add roundup-admin interactive help output to admin_guide.txt

This only works for html output. It is recreated if admin.py changes.
files
docs: fix some missing/broken links. http://hg.code.sf.net:8000/p/roundup/code/#changeset-57ef20b6c003673d05a572265bb80d8b9bf3e509 John Rouillard rouilj@ieee.org 2024-07-12T22:23:32-04:00 2024-07-12T22:23:32-04:00
changeset 57ef20b6c003
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: fix some missing/broken links.
files
docs: update link to redis.io security doc. http://hg.code.sf.net:8000/p/roundup/code/#changeset-335b8a40e35595604df70d4644511ddc8d33bbec John Rouillard rouilj@ieee.org 2024-07-03T21:34:42-04:00 2024-07-03T21:34:42-04:00
changeset 335b8a40e355
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: update link to redis.io security doc.
files
docs: use virtual env for upgrade; stop cron jobs. http://hg.code.sf.net:8000/p/roundup/code/#changeset-704757c71b844ab0b9354a48a470258b20da7515 John Rouillard rouilj@ieee.org 2024-06-19T00:23:51-04:00 2024-06-19T00:23:51-04:00
changeset 704757c71b84
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: use virtual env for upgrade; stop cron jobs.
files
refactor: make mime_type_allowlist class prop to configure from interfaces.py http://hg.code.sf.net:8000/p/roundup/code/#changeset-e1cff9745fb41a5de14ebac7d2f2c9658f10b15e John Rouillard rouilj@ieee.org 2024-06-17T23:35:03-04:00 2024-06-17T23:35:03-04:00
changeset e1cff9745fb4
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description refactor: make mime_type_allowlist class prop to configure from interfaces.py

The list of mime types that are served to the browser was located in a
list inside a function. The allow_html_file setting provided a limited
mechanism to add/remove the text/html mime type from the list.

Move the list from the function to the Client class level so that the
admin can add/remove from the list as required using interfaces.py.

Also remove application/pdf by default and provide docs in
admin_guide.txt on how to reenable viewing pdf's in the browser.
files
docs: escape + in table, prevent it from starting an unordered list http://hg.code.sf.net:8000/p/roundup/code/#changeset-098ceff3834966854f41c8ac07bfc7e13761b711 John Rouillard rouilj@ieee.org 2024-05-30T20:01:08-04:00 2024-05-30T20:01:08-04:00
changeset 098ceff38349
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: escape + in table, prevent it from starting an unordered list

The markup was being interpreted badly.
files
docs: fix markup http://hg.code.sf.net:8000/p/roundup/code/#changeset-72fb18809559a104b1383b47fc3f13c53409d6d4 John Rouillard rouilj@ieee.org 2024-05-30T19:58:24-04:00 2024-05-30T19:58:24-04:00
changeset 72fb18809559
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: fix markup
files
docs: add section on loading classhelper.js script. http://hg.code.sf.net:8000/p/roundup/code/#changeset-eae83a32f56472da0008c2e500292871a0b50ce4 John Rouillard rouilj@ieee.org 2024-05-21T23:49:19-04:00 2024-05-21T23:49:19-04:00
changeset eae83a32f564
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: add section on loading classhelper.js script.

Also provide some info about precompressing and minimizing the file.

Add a reference for the compression config info.
files
docs: fix formatting. http://hg.code.sf.net:8000/p/roundup/code/#changeset-ecb09e5d0bfada9ae2834f849256cbcf5ac3a1af John Rouillard rouilj@ieee.org 2024-05-21T20:17:36-04:00 2024-05-21T20:17:36-04:00
changeset ecb09e5d0bfa
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: fix formatting.
files
docs: fix formatting. http://hg.code.sf.net:8000/p/roundup/code/#changeset-749c9a58cfda342c4d47b1f8ae7bf08fb65408d3 John Rouillard rouilj@ieee.org 2024-05-21T20:15:29-04:00 2024-05-21T20:15:29-04:00
changeset 749c9a58cfda
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: fix formatting.
files
docs: update cache updating for css file. http://hg.code.sf.net:8000/p/roundup/code/#changeset-d3c0d3c9f5e7021b3a35f7fe118f5c6d65692d23 John Rouillard rouilj@ieee.org 2024-05-21T20:14:38-04:00 2024-05-21T20:14:38-04:00
changeset d3c0d3c9f5e7
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: update cache updating for css file.
files
docs: more cleanups http://hg.code.sf.net:8000/p/roundup/code/#changeset-adc3db39cb53300a3898cb8b7f0828454a175122 John Rouillard rouilj@ieee.org 2024-05-21T20:13:18-04:00 2024-05-21T20:13:18-04:00
changeset adc3db39cb53
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: more cleanups
files
doc: grammar fixup. http://hg.code.sf.net:8000/p/roundup/code/#changeset-16c2e2849fd6a736cb1b35056214b7ae3def9d0c John Rouillard rouilj@ieee.org 2024-05-21T20:10:28-04:00 2024-05-21T20:10:28-04:00
changeset 16c2e2849fd6
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: grammar fixup.
files
doc: classhelper translation, advanced config w/ roles endpoint http://hg.code.sf.net:8000/p/roundup/code/#changeset-4dcc64d0138ea8f2c4b7936cd0f0b895e113c2e6 John Rouillard rouilj@ieee.org 2024-05-21T20:03:56-04:00 2024-05-21T20:03:56-04:00
changeset 4dcc64d0138e
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: classhelper translation, advanced config w/ roles endpoint

Added info on how to translate default title and data-popup-title.

Also added advanced config section on deploying a new roles rest
endpoint and changing classhelper to use it.
files
issue2551353 - Add roundup-classhelper for 2.4.0 release http://hg.code.sf.net:8000/p/roundup/code/#changeset-fe0348bbe45b6feaa1f35365449d334fa4686d55 John Rouillard rouilj@ieee.org 2024-05-21T01:17:28-04:00 2024-05-21T01:17:28-04:00
changeset fe0348bbe45b
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551353 - Add roundup-classhelper for 2.4.0 release

Changes to the classic template are not done yet. Still testing.

This commit has document updates and changes to rest.py.

rest.py:

add /rest/data/user/role endpoint to core so the user doesn't have
to add the /rest/roles endpoint via interfaces.py. It will only send
roles for a user with Admin role and there is no way to override
this currently.


acknowledgements.txt:

Added members of team3 to other contributors. Specified for all
other contributes what they worked on.

upgrading.txt:

added classhelper section and basic template change
directions. Linked to admin_guide for full directions.

admin_guide.txt:

documented install, translation, troubleshooting, config etc.

user_guide.txt:

added section on using the classhelper. Added reference to section
earlier in the doc. Added image for section.
files
doc: update links and docs on checking links http://hg.code.sf.net:8000/p/roundup/code/#changeset-828f79ae965b96387ee9d1e4145f25b3a46942ce John Rouillard rouilj@ieee.org 2024-05-13T18:42:26-04:00 2024-05-13T18:42:26-04:00
changeset 828f79ae965b
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: update links and docs on checking links

Add timeout for link checking so make linkcheck completes.

Document grep command to use to filter broken links to external links.

Fix PostgreSQL links to point to current pages.
Fix pyreadline3 link.
Fix vim po file plugin link
Use wayback machine for zpi internationalization link.
files
doc: replace X by + to indicate compatible options. http://hg.code.sf.net:8000/p/roundup/code/#changeset-2946354d6ccdd157b6cad3da3b09613d2dca9acc John Rouillard rouilj@ieee.org 2024-04-18T20:52:15-04:00 2024-04-18T20:52:15-04:00
changeset 2946354d6ccd
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: replace X by + to indicate compatible options.

X looks like it indicates non-compatible options.
files
issue2551328/issue2551264 unneeded next link and total_count incorrect http://hg.code.sf.net:8000/p/roundup/code/#changeset-03c1b7ae3a6827724ecf71b9674ebb50541f0648 John Rouillard rouilj@ieee.org 2024-04-01T09:57:16-04:00 2024-04-01T09:57:16-04:00
changeset 03c1b7ae3a68
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description issue2551328/issue2551264 unneeded next link and total_count incorrect

Fix: issue2551328 - REST results show next link if number of
results is a multiple of page size. (Found by members of
team 3 in the UMass-Boston CS682 Spring 2024 class.)

issue2551264 - REST X-Total-Count header and @total_size
count incorrect when paginated

These issues arose because we retrieved the exact number of rows
from the database as requested by the user using the @page_size
parameter. With this changeset, we retrieve up to 10 million + 1
rows from the database. If the total number of rows exceeds 10
million, we set the total_count indicators to -1 as an invalid
size. (The max number of requested rows (default 10 million +1)
can be modified by the admin through interfaces.py.)

By retrieving more data than necessary, we can calculate the
total count by adding @page_index*@page_size to the number of
rows returned by the query.

Furthermore, since we return more than @page_size rows, we can
determine the existence of a row at @page_size+1 and use that
information to determine if a next link should be
provided. Previously, a next link was returned if @page_size rows
were retrieved.

This change does not guarantee that the user will get @page_size
rows returned. Access policy filtering occurs after the rows are
returned, and discards rows inaccessible by the user.

Using the current @page_index/@page_size it would be difficult to
have the roundup code refetch data and make sure that a full
@page_size set of rows is returned. E.G. @page_size=100 and 5 of
them are dropped due to access restrictions. We then fetch 10
items and add items 1-4 and 6 (5 is inaccessible). There is no
way to calculate the new database offset at:
@page_index*@page_size + 6 from the URL. We would need to add an
@page_offset=6 or something.

This could work since the client isn't adding 1 to @page_index to
get the next page. Thanks to HATEOAS, the client just uses the
'next' url. But I am not going to cross that bridge without a
concrete use case.

This can also be handled client side by merging a short response
with the next response and re-paginating client side.

Also added extra index markers to the docs to highlight use of
interfaces.py.
files
Summary: feat: roundup-admin history command has human interpretable output http://hg.code.sf.net:8000/p/roundup/code/#changeset-8bdf0484215cf7beebd02bc43e754856c6b0a685 John Rouillard rouilj@ieee.org 2024-03-10T21:59:20-04:00 2024-03-10T21:59:20-04:00
changeset 8bdf0484215c
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Summary: feat: roundup-admin history command has human interpretable output

Reformat journal entries and make try to make readable sentences out
of them.

Set up translation markers and added hints for the tanslators by
marking translator comments with;

# .Hint text for translator

on the line before _() markers.

Doc'ed changes in roundup-admin docs and added info to upgrading.txt.

If the user wants old format, they can call

history designator raw
files
save roundup-admin history between sessions. http://hg.code.sf.net:8000/p/roundup/code/#changeset-d7d68da9878f1e0ddd7f9bc746ec8cc1c56a2235 John Rouillard rouilj@ieee.org 2024-03-10T13:44:13-04:00 2024-03-10T13:44:13-04:00
changeset d7d68da9878f
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description save roundup-admin history between sessions.

Also load ~/.roundup_admin_rlrc (rlrc readline run commands) file to
set history-size persistently, allow user to use vi mode etc.
files
fix: out of memory error when importing under postgresql http://hg.code.sf.net:8000/p/roundup/code/#changeset-5b41018617f2984f188ec7f1516cfcd970a8b9ba John Rouillard rouilj@ieee.org 2023-10-19T16:11:25-04:00 2023-10-19T16:11:25-04:00
changeset 5b41018617f2
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix: out of memory error when importing under postgresql

If you try importing more than 20k items under postgresql you can run
out of memory:

psycopg2.errors.OutOfMemory: out of shared memory
HINT: You might need to increase max_locks_per_transaction.

Tuning memory may help, it's unknown at this point.

This checkin forces a commit to the postgres database after 10,000
rows have been added. This clears out the savepoints for each row and
starts a new transaction.

back_postgresql.py:

Implement commit mechanism in checkpoint_data(). Add two class level
attributes for tracking the number of savepoints and the limit when
the commit should happen.

roundup_admin.py:

implement pragma and dynamically create the config item
RDBMS_SAVEPOINT_LIMIT used by checkpoint_data.

Also fixed formatting of descriptions when using pragma list in
verbose mode.

admin_guide.txt, upgrading.txt:

Document change and use of pragma savepoint_limit in roundup-admin
for changing the default of 10,000.

test/db_test_base.py:

add some more asserts. In existing testAdminImportExport, set the
savepoint limit to 5 to test setting method and so that the commit
code will be run by existing tests. This provides coverage, but
does not actually test that the commit is done every 5 savepoints
8-(. The verification of every 5 savepoints was done manually
using a pdb breakpoint just before the commit.

acknowledgements.txt:

Added 2.4.0 section mentioning Norbert as he has done a ton of
testing with much larger datasets than I can test with.
files
doc: document the messages generated from import/export http://hg.code.sf.net:8000/p/roundup/code/#changeset-08e4399c3ae4dbf2e2c5d777da249ba45aadbde0 John Rouillard rouilj@ieee.org 2023-10-19T14:07:56-04:00 2023-10-19T14:07:56-04:00
changeset 08e4399c3ae4
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: document the messages generated from import/export

The numbers reported when exporting/importing don't make sense unless
you know how they are generated and used. Document that. Norbert
Schlemmer asked about it while debugging an import issue with
postgresql. So document it since I did the work.
files
doc: remove whitelist replace with allowing http://hg.code.sf.net:8000/p/roundup/code/#changeset-60fc16b6a1cc857c99456c883a81f1221fcbbfa2 John Rouillard rouilj@ieee.org 2023-09-03T14:03:48-04:00 2023-09-03T14:03:48-04:00
changeset 60fc16b6a1cc
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description doc: remove whitelist replace with allowing
files
docs: fix sentence. http://hg.code.sf.net:8000/p/roundup/code/#changeset-c9180009a286fe1c5f5800f3e17162c89a443c56 John Rouillard rouilj@ieee.org 2023-07-31T18:50:07-04:00 2023-07-31T18:50:07-04:00
changeset c9180009a286
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: fix sentence.
files
docs: clarify sentence. http://hg.code.sf.net:8000/p/roundup/code/#changeset-7c3aee35a5ac44330b7c9c034f0231f8df8b057f John Rouillard rouilj@ieee.org 2023-07-31T18:43:56-04:00 2023-07-31T18:43:56-04:00
changeset 7c3aee35a5ac
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description docs: clarify sentence.
files
Add -P pragma=value command line option to roundup-admin. http://hg.code.sf.net:8000/p/roundup/code/#changeset-534f8bdb8f943a61056d14d62e53c681b50f2204 John Rouillard rouilj@ieee.org 2023-07-14T00:30:44-04:00 2023-07-14T00:30:44-04:00
changeset 534f8bdb8f94
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Add -P pragma=value command line option to roundup-admin.

To set pragmas when using non-interactive mode, or set on command line
when going into interactive mode.

Also changed specification test to use command line pragma setting
rather than interactive. This tests the -P option without having to
run an extra test.

Docs updated as well.
files
Update roundup-mailgw install with other methods; formatting http://hg.code.sf.net:8000/p/roundup/code/#changeset-7b6278da3faa671acba60fe622e0f1c5d042d235 John Rouillard rouilj@ieee.org 2023-06-21T14:34:37-04:00 2023-06-21T14:34:37-04:00
changeset 7b6278da3faa
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Update roundup-mailgw install with other methods; formatting

reference admin_guide.html for alternate imap/pop methods (imap_oauth
or apop etc.)

Change example written intline into an exampe as line wrapping
distorts whitespace (space removed if line is wrapped) and needs to be
presented in a pre example section.
files
Change customizing to customising in all variants. http://hg.code.sf.net:8000/p/roundup/code/#changeset-a072331c843b55ed5458029b147f4dd5e16d7507 John Rouillard rouilj@ieee.org 2023-06-18T11:35:16-04:00 2023-06-18T11:35:16-04:00
changeset a072331c843b
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Change customizing to customising in all variants.

Make the spelling of customizing follow the British (also Australian
since Richard Jones is from there) spelling. Non-native English
speakers will see a consistent spelling of the word. The filename is
still customizing.txt/html (US spelling) though.
files
typo fix. http://hg.code.sf.net:8000/p/roundup/code/#changeset-f40626db85dd12d0d576f35e3a359508a2a140fc John Rouillard rouilj@ieee.org 2023-06-11T20:50:01-04:00 2023-06-11T20:50:01-04:00
changeset f40626db85dd
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description typo fix.
files
Add anchor for init section. http://hg.code.sf.net:8000/p/roundup/code/#changeset-76b2f3c95c034bdd0c3a918d7c1c558196ee3209 John Rouillard rouilj@ieee.org 2023-06-05T12:19:01-04:00 2023-06-05T12:19:01-04:00
changeset 76b2f3c95c03
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Add anchor for init section.
files
Add additional command that do not require a tracker home. http://hg.code.sf.net:8000/p/roundup/code/#changeset-1642a7b501dfdc38c774a213cb86772931675a93 John Rouillard rouilj@ieee.org 2023-05-25T13:56:03-04:00 2023-05-25T13:56:03-04:00
changeset 1642a7b501df
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Add additional command that do not require a tracker home.
files
- issue2551190 - Allow roundup-admin reindex to work in batches. http://hg.code.sf.net:8000/p/roundup/code/#changeset-312d52305583653240391f70b8ceeb72963cb95f John Rouillard rouilj@ieee.org 2023-05-24T12:13:05-04:00 2023-05-24T12:13:05-04:00
changeset 312d52305583
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description - issue2551190 - Allow roundup-admin reindex to work in batches.

Running:

roundup-admin -i ... reindex issue:1-1000

will reindex the first 1000 issues while reporting any missing issues
in the range.

Also completion progress is reported when indexing a specific class.

Note this require a chnge that makes an invalid command like:

reindex issue23f

to error with no such class issue23f. It used to reindex
issue23 which I consider a bug.

Updates to man page admin_guide.py.

Tests added.
files
- issue2551275 - Allow configuring max_children in roundup-server. http://hg.code.sf.net:8000/p/roundup/code/#changeset-46f92ac4e170835d7232e3f94726812d0013e782 John Rouillard rouilj@ieee.org 2023-05-23T23:34:06-04:00 2023-05-23T23:34:06-04:00
changeset 46f92ac4e170
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description - issue2551275 - Allow configuring max_children in roundup-server.

new -m and --max_children command line arguments and max_children
config file setting for roundup_server.py/roundup-server.

CHANGES.txt, admin_guide.txt, roundup-server.1 updated.
files
fix typo http://hg.code.sf.net:8000/p/roundup/code/#changeset-87e9d761ecc6fcfa28293c1ddcc3fa447497ddc1 John Rouillard rouilj@ieee.org 2023-05-23T11:37:40-04:00 2023-05-23T11:37:40-04:00
changeset 87e9d761ecc6
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description fix typo
files
whitespace changes. http://hg.code.sf.net:8000/p/roundup/code/#changeset-59ae15e0b8dd7e6f3e90e56bedd03ed9b1f3d1e6 John Rouillard rouilj@ieee.org 2023-05-22T21:49:46-04:00 2023-05-22T21:49:46-04:00
changeset 59ae15e0b8dd
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description whitespace changes.
files
Change primary/session db compatibility table from ascii to real table http://hg.code.sf.net:8000/p/roundup/code/#changeset-2884ff18c944dfa7dd977690008011e13508bd43 John Rouillard rouilj@ieee.org 2023-05-22T21:44:20-04:00 2023-05-22T21:44:20-04:00
changeset 2884ff18c944
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Change primary/session db compatibility table from ascii to real table
files
Unident lists. http://hg.code.sf.net:8000/p/roundup/code/#changeset-602cc42f9eddcfcbfef745e4c94d64d9a9481d45 John Rouillard rouilj@ieee.org 2023-05-16T21:08:29-04:00 2023-05-16T21:08:29-04:00
changeset 602cc42f9edd
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description Unident lists.

Indented lists are wrapped in blockquote and indented too far when
displayed. Get rid of the extra unneeded indentation.
files
initial pass to change references to reference doc where needed. http://hg.code.sf.net:8000/p/roundup/code/#changeset-692242b3effddaaefa5e22e0896910dc0d7fd760 John Rouillard rouilj@ieee.org 2023-05-16T01:19:16-04:00 2023-05-16T01:19:16-04:00
changeset 692242b3effd
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description initial pass to change references to reference doc where needed.

There are some references that don't seem to exist anymore. They were
left pointing to their original values.

These are in the 0.6 upgrade directions and whatsnew-0.7. They
reference macros that don't exist or config.py that is no longer
valid.
files
document result of running save-config in a tracker home directory http://hg.code.sf.net:8000/p/roundup/code/#changeset-b84718ea422881a7090232a6c5eb03a8a8f67113 John Rouillard rouilj@ieee.org 2023-05-14T01:03:09-04:00 2023-05-14T01:03:09-04:00
changeset b84718ea4228
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description document result of running save-config in a tracker home directory
files
reword doc update http://hg.code.sf.net:8000/p/roundup/code/#changeset-a83c932a00dcfb81fdf40f161dee8041f2d28dfb John Rouillard rouilj@ieee.org 2023-05-14T00:49:43-04:00 2023-05-14T00:49:43-04:00
changeset a83c932a00dc
branch
bookmark
tag
user John Rouillard <rouilj@ieee.org>
description reword doc update
files