| changeset | f7fadbac5856 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | take2: merge multiple fix: python < 3.12 returns ParsingError not RuntimeError; print exception |
| files |
| changeset | 9f7e863fd1cd |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | merge multiple fix: python < 3.12 returns ParsingError not RuntimeError; print exception |
| files |
| changeset | 6d6f689d3701 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | fix: python < 3.12 returns ParsingError not RuntimeError; print exception RuntimeError added in 3.12. Support older configparser.ParsingError. Supporting older version also requires different test string for the two versions. Also use str(exception) rather than exception.args[0] for producing message to user. Include exception name in output. |
| files |
| changeset | e91ff70e4563 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | bug: improve error reporting for errors for logging fileConfig. |
| files |
| changeset | 401c6f0be6c5 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | bug: fix json logging config file syntax exception/fix test for windows If the json logging config file has mimatched {} or [], it raises an IndexError. Handle that case and test it. Also handle embedded filenames in tests when testsare run on windows:(/ vs \ directory sep). |
| files |
| changeset | d06be9346c68 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | bug, test: fix tests for trace_id; readd import logging.config Made save_restore_logging a test level fixture. It was a class level which worked fine until I started using caplog for tests in the same class. Due to loading config from dict, the roundup channel was set to not propagate which broke the new formatting test used for trace_id. Forgot to update some tests due to change in default format adding %(trace_id). Also re-added logging.config import which broke loading logging config files in configuration.py. |
| files |
| changeset | 14c7c07b32d8 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | feature: add thread local trace_id and trace_reason to logging. Added trace_id to default logging so that all logs for a given request share the same trace_id. This allows correlation of logs across a request. admin_guide.txt, upgrading.txt: add docs update sample configs to include trace_id. rewrite logging docs in admin_guide. Hopefully they are clearer now. clean up some stuff in the logging config file docs. admin.py: add decorators to run_command to enable trace_id. change calls to db.commit() to use run_command to get trace_id. configuration.py: clean up imports. update docstrings, comments and inline docs. add trace_id to default log format. add function for testing decorated with trace_id. add support for dumping stack trace in logging. add check for pytest in sys.modules to enable log propagation when pytest is running. Otherwise tests fail as the caplog logger doesn't see the roundup logs. logcontext.py: new file to handle thread local contextvar mangement. mailgw.py: add decorators for trace_id etc. scripts/roundup_xlmrpc_server.py: add decorators for trace_id etc. fix encoding bug turning bytes into a string. fix command line issue where we can't set encoding. (not sure if changing encoding via command line even works) cgi/client.py decorate two entry points for trace_id etc. cgi/wsgi_handler.py: decorate entry point for trace_id etc. test/test_config.py: add test for trace_id in new log format. test various cases for sinfo and errors in formating msg. |
| files |
| changeset | 39a6825d10ca |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | feat: allow admin to set logging format from config.ini This is prep work for adding a per thread logging variable that can be used to tie all logs for a single request together. This uses the same default logging format as before, just moves it to config.ini. Also because of configparser, the logging format has to have doubled % signs. So use: %%(asctime)s not '%(asctime)s' as configparser tries to interpolate that string and asctime is not defined in the configparser's scope. Using %%(asctime)s is not interpolated by configparser and is passed into Roundup. |
| files |
| changeset | 66284037142e |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | refactor: also error on missing file or invalid extension Refactored the code to reuse check that logging config file is set and that the file exists. Now throws error and exits if file name does not end in .ini or .json. Now throws error if file doesn't exist. Before it would just configure default logging as though file wasn't specified. Added tests for these two cases. |
| files |
| changeset | de1dac9abcb6 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | feat: change comment in dictConfig json file to // from # Emacs json mode at least will properly indent when using // as a comment character and not #. |
| files |
| changeset | 3210729950b1 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | test: fix code that does not run a test on 3.7 |
| files |
| changeset | cdf876bcd370 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | test: test dictLoggerConfig - working logging reset and windows Finally figured out why things weren't being restored. Bug in the code. Created a class fixture that stores and restores the logging config. Also using os.path.join and other machinations to make the tests run under windows and linux correctly. |
| files |
| changeset | b34c3b8338f0 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | test: more fun with logger leakage. I have disabled all calls to dictConfig. I can't see how to stop this from leaking. I even tried storing the root and roundup logger state (copying lists) before running anything and restoring afterwards. the funny part is if I removae all dictConfig calls and keep the state saving and logger reset and restoring code it sill fails. It passes if I don't restore the handler state for the root logger. However the test will fail even when I comment out the root logger config in the dict if I apply the dict. ???? |
| files |
| changeset | cd0edc091b97 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | test: more fixes for variations among python versions. reset logging. The loggers I installed are bleeding through to tests in other modules. Then handle recogniton of: trailing comma in json reporting line with comma not following line incorrect type value in dictConfig int where string should be 3.7 fils to detect. 3.10+ add "got 'int'" to reason. |
| files |
| changeset | 4a948ad46579 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | test: fix testDictLoggerConfigViaJson The path to the log file in the config did not exist. Change to use the tracker home. Also add a test where the log file directory does not exist. This reports the full path, so have to edit the full path in the error message before comparison. |
| files |
| changeset | 94eed885e958 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | feat: add support for using dictConfig to configure logging. Basic logging config (one level and one output file non-rotating) was always possible from config.ini. However the LOGGING_CONFIG setting could be used to load an ini fileConfig style file to set various channels (e.g. roundup.hyperdb) (also called qualname or tags) with their own logging level, destination (rotating file, socket, /dev/null) and log format. This is now a deprecated method in newer logging modules. The dictConfig format is preferred and allows disabiling other loggers as well as invoking new loggers in local code. This commit adds support for it reading the dict from a .json file. It also implements a comment convention so you can document the dictConfig. configuration.py: new code test_config.py: test added for the new code. admin_guide.txt, upgrading.txt CHANGES.txt: docs added upgrading references the section in admin_guid. |
| files |
| changeset | 1e38ca6fb16e |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | test: clean up test for deprecation of xhtml. xhtml was used as a value in a test. Choose different value for testing. Also add test to verify that xhtml use raises an error. |
| files |
| changeset | 6102ae426390 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | issue2551334 - get test suite running under windows Support windows style filepaths. |
| files |
| changeset | 011941fcb598 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | test: skip test requiring postgresql backend if not present |
| files |
| changeset | be6cb2e0d471 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | feat: add support for rotating jwt keys This allows jwt_secret to have multiple ',' separated secrets. The first/leftmost should be used to sign new JWTs. All of them are used (starting from left/newest) to try to verify a JWT. If the first secret is < 32 chars in length JWTs are disabled. If any of the other secrets are < 32 chars, the configuration code causes the software to exit. This prevents insecure (too short) secrets from being used. Updated doc examples and tests. |
| files |
| changeset | a5629f6e7ec2 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | test: fix mising / in directory spec. |
| files |
| changeset | 273c8c2b5042 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | fix(api): - issue2551063 - Rest/Xmlrpc interfaces needs failed login protection. Failed API login rate limiting with expiring lockout added. |
| files |
| changeset | 89a59e46b3af |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | improve REST interface security When using REST, we reflect the client's origin. If the wildcard '*' is used in allowed_api_origins all origins are allowed. When this is done, it also added an 'Access-Control-Allow-Credentials: true' header. This Credentials header should not be added if the site is matched only by '*'. This header should be provided only for explicit origins (e.g. https://example.org) not for the wildcard. This is now fixed for CORS preflight OPTIONS request as well as normal GET, PUT, DELETE, POST, PATCH and OPTIONS requests. A missing Access-Control-Allow-Credentials will prevent the tracker from being accessed using credentials. This prevents an unauthorized third party web site from using a user's credentials to access information in the tracker that is not publicly available. Added test for this specific case. In addition, allowed_api_origins can include explicit origins in addition to '*'. '*' must be first in the list. Also adapted numerous tests to work with these changes. Doc updates. |
| files |
| changeset | e0f0aed72097 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Fix test if redis is not loaded handle the KeyError from trying to remove redis from sys.modules if it not loaded. |
| files |
| changeset | 3f60a71b0812 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Summary: Support selecion session/otk data store. Add redis as data store. Allow admin to select the backend data store. Compatibility matrix: main\/ session>| anydbm | sqlite | redis | mysql | postgresql | anydbm | D | | X | | | sqlite | X | D | X | | | mysql | | | | D | | postgresql | | | | | D | --------------------------------------------------------------+ D - default if unconfigured, X - compatible choice DETAILS roundup/configuration.py: add config.ini section sessiondb with settings: backend and redis_url. CHANGES.txt, doc/admin_guide.txt, doc/installation.txt, doc/upgrading.txt: doc on config of session db and redis. Plus some other fixes: admin - clarified why we do not drop __words and __testids table in native-fts conversion. TYpo fix. upgrading - doc how you can keep using anydbm for session data with sqlite. Fix dupe sentence in an upgrading config.ini section. roundup/backends/back_anydbm.py, roundup/backends/back_sqlite.py: code to support redis, redis/anydbm backends respectively. roundup/backends/sessions_redis.py new storage backend for redis. roundup/rest.py, roundup/cgi/actions.py, roundup/cgi/templating.py redis uses a different way of calculating lifetime/timestamp. Since expiration of an item occurred if its timestamp was more than 1 week old, code would calculate: now - 1 week + lifetime. But this results in faster expiration in redis if used for lifetime/timestamp. Convert code to use the lifetime() method in BasicDatabase that generates the right timestamp for each backend. test/session_common.py: added tests for more cases, get without default, getall non-existing key etc. timestamp test changed to use new self.get_ts which is overridden in other tests. Test that datatypes survive storage. test/test_redis_session.py: test redis session store with sqlite and anydbm primary databases test/test_anydbm.py, test/test_sqlite.py add test to make sure the databases are properly set up sqlite - add test cases where anydbm is used as datastore anydbm - remove updateTimestamp override add get_ts(). test/test_config.py tests on redis_url and compatibility on choice of sessiondb backend .travis.yml: add redis db and redis-py |
| files |
| changeset | ab2ed11c021e |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | issue2551205: Add support for specifying valid origins for api: xmlrpc/rest We now have an allow list to filter the hosts allowed to do api requests. An element of this allow list must match the http ORIGIN header exactly or the rest/xmlrpc CORS request will result in an error. The tracker host is always allowed to do a request. |
| files |
| changeset | 0d99ae7c8de6 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Allow Roundup to use PostgreSQL database native full text search back_postgreql.py - schema version changes for schema version 7. configuration.py - added indexer_language checks for postgresql. Hardcoded list for now. Docs admin_guide and upgrading Tests. This also restructures the version upgrade tests for the rdbms backends. They can run all of them now as the proper cascade is developed to roll back changes to version 6. |
| files |
| changeset | 770503bd211e |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Validate SecretOption and support validate method Needed to validate SecretOption and verify that file is readable and valid. validator() now calls the validate method for each Option subclass. To add post config load validation, just define a method validate(self, options) for the Option subclass. |
| files |
| changeset | b1f1539c6a31 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | issue2551182 - ... allow loading values from external file. flake8 cleanups Secrets (passwords, secrets) can specify a file using file:// or file:///. The first line of the file is used as the secret. This allows committing config.ini to a VCS. Following settings are changed: [tracker] secret_key [tracker] jwt_secret [rdbms] password [mail] password details: in roundup/configuration.py: Defined SecretMandatoryOptions and SecretNullableOptions. Converted all secret keys and password to one of the above. Also if [mail] username is defined but [mail] password is not it throws an error at load. Cleaned up a couple of methods whose call signature included: def ...(..., settings={}): settings=None and it is set to empty dict inside the method. Also replace exception.message with str(exception) for python3 compatibility. in test/test_config: changed munge_configini to support changing only within a section, replacing keyword text. |
| files |
| changeset | fdfe3b7387ea |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Add test for config parse error by running AdminTool. |
| files |
| changeset | 8687c096a945 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Handle configparser.InterpolationSyntaxError Under Python 3, an option value with a single % (e.g. this % is a test) throws configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%s))' Added code to capture this, raise a different exception. roundup-admin handles the error and exits cleanly. Other code shows the traceback. The new error message reports the file, section and option causing the problem to allow easier repair. Also updated roundup translations and added tests. |
| files |
| changeset | 5c1db5d4baed |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | Ralf Schlatterbeck <rsc@runtux.com> |
| description | Fix failing xapian test |
| files |
| changeset | e3ed3ad9e795 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | issue2551125 Add indexer_lang test that runs if xapian not installed Test case for xapian not installed. Same test as testInvalidIndexerLanguage_xapian_missing except without magling sys.modules. |
| files |
| changeset | 8a7de159d1a6 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | issue2551125 Make indexer_lang test work if xapian not installed. Reported by Ralf in https://issues.roundup-tracker.org/issue2551125. |
| files |
| changeset | 08e209a7f22b |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Use OptionValueError for indexer_language error, add configuration tests Added more tests for configuration.py. This is the last of the easy to do tests for the module. While doing that figured out how to get the proper arguments to generate OptionValueError if indexer_language is invalid. Changed implementation to generate same and updated test. |
| files |
| changeset | 2d42a308927b |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Test trying to load missing config.ini file. Noticed this case wasn't covered by tests. Low hanging fruit. |
| files |
| changeset | 42db48c30d31 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Validate indexer values Make sure that indexer values are one of the supported indexers. Invalid values would have raised an invalid indexer exception on access. |
| files |
| changeset | c985ed52ca2d |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Add testcases for invalid indexer_language Set invalid indexer_language and test: indexer = "" and xapian not able to be loaded -> no exception indexer = native -> no exception indexer = xapian -> ValueError exception Also test reset to verify that indexer_language is reset to english from NO_LANG. Also create helper method to munge config.ini for each test case. Changed all test to use it. Also moved doc from class to the test that it described. |
| files |
| changeset | c26b9ce33ae3 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | issue2551123 - validate indexer_language in configuration.py Was validated in backends/indexer_xapian.py which would throw an error on access rather than on start. Added validator function to CoreConfig class that runs after config.ini is read. At this time we have access to the indexer setting so can determine if xapian is actually going to be used. Moved test into test/test_config.py and pulled validation code from indexer_xapian.py and test/test_indexer.py. |
| files |
| changeset | c547e05d7a54 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Test case where backend is missing from config.ini. |
| files |
| changeset | 6cf9f2f49b89 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Fix UTC timezone test case if pytz not available; add tests UTC is a special known timezone so it doesn't raise an exception even if pytz is missing. Added test case with bogus timezone that raises exception. Added testcase for EST that is known with and without pytz. |
| files |
| changeset | 95a366d46065 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Replace deprecated assertEquals with assertEqual and failUnlessRaises with assertRaises. |
| files |
| changeset | 765f8c0e99ef |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Sanity checking improvements. All IntegerNumberOption really have to be positive integers or zero. Replace type with new IntegerNumberGeqZeroOption class definition to enforce this range. Use update_option to test now unused IntegerNumberOption and never used FloatNumberOption. Test save function, make sure it creates .bak file. Test a few more configuration option classes. |
| files |
| changeset | 8dbe307bdb57 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Finish up login rate limit code. Set config item to 0 disables, make sure config item can't be negative integer. |
| files |
| changeset | f91da208f26b |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | Validate that TRACKER_WEB url starts with https:// or http:// and ends with a /. |
| files |
| changeset | dd642afb4440 |
|---|---|
| branch | |
| bookmark | |
| tag | |
| user | John Rouillard <rouilj@ieee.org> |
| description | starter tests for roundup/configuration.py |
| files |
| changeset | 13719594278b |
|---|---|
| branch | config-0-4-0-branch |
| bookmark | |
| tag | |
| user | Richard Jones <richard@users.sourceforge.net> |
| description | I've re-worked the config structure a little so it's simpler (one less file) and added a unit test so we can be sure it's working. |
| files |