Mercurial Repository: p/roundup/code: roundup/hyperdb.py history

Generate savepoint only if necessary

2025-03-01T18:55:54+01:00

changeset	a81a3cd067fa
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Generate savepoint only if necessary Now some methods got an additional 'allow_abort' parameter. By default this is True. When False the postgres backend generates a savepoint. The methods are called with allow_abort=False from some of the cgi methods which can produce a traceback when called with data from the web-interface.
files

Factor common code to hyperdb/roundupdb

2025-03-01T13:08:09+01:00

changeset	24549122f9b1
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Factor common code to hyperdb/roundupdb There was some common copied code in backends/back_anydbm.py and backends/rdbms_common.py. This is now moved to hyperdb.py and roundupdb.py, respectively (the FileClass lives in hyperdb.py while the IssueClass is in roundupdb.py)
files

Make native date and number elements configurable

2025-02-19T12:38:06+01:00

changeset	b99e76e76496
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Make native date and number elements configurable Now for Number() and Integer() properties the browser-native number format can be configured with the use_browser_number_input config item in seciont [web]. The default is 'yes'. For Date() properties the config item is use_browser_date_input (also in section [web]) but the default is 'no'. In addition when defining Date() properties, these now have a parameter 'display_time' which defaults to 'yes' and a 'format' parameter which defaults to None. These set defaults for the field() method of the DateHTMLProperty which have the same parameters (but the display_time parameter of field() takes a boolean, not 'yes'/'no').
files

[permission-performance] Rename parameter of hasPermission

2024-10-24T09:19:25+02:00

changeset	de58ff07890e
branch	permission-performance
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Rename parameter of hasPermission Rename only_no_check to skip_permissions_with_check. Revert explicit no-properties check in Permission.searchable, this check is already taken care of by the _properties_dict check. Add a comment on what _properties_dict does.
files

[permission-performance] Make permission filter functions configurable

2024-10-23T17:46:05+02:00

changeset	5a2b9435a04d
branch	permission-performance
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Make permission filter functions configurable For debugging and performance measurements it makes sense to allow turning permission filter functions off.
files

[permission-performance] Test new filter function in security checks

2024-10-23T16:29:43+02:00

changeset	aa5ae3f84889
branch	permission-performance
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Test new filter function in security checks And add bug-fix after moving filter_with_permissions to hyperdb.
files

[permission-performance] Filter exact_match_spec in filter_with_permissions

2024-10-22T12:04:50+02:00

changeset	627871650f4f
branch	permission-performance
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Filter exact_match_spec in filter_with_permissions
files

[permission-performance] Move permission check code to hyperdb

2024-10-21T18:12:03+02:00

changeset	f7bd22bdef9d
branch	permission-performance
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Move permission check code to hyperdb Now the hyperdb has a method filter_with_permissions that performs the permission checks before (for filtering on sort/group/filterspec arguments) and after a call to hyperdb.filter. This also fixes possible problems on the unfiltered sort/group/filterspec arguments in roundup/rest.py and roundup/cgi/templating.py
files

[issue2550923_computed_property] merge default into branch

2023-07-10T17:31:34-04:00

changeset	14a8e11f3a87
branch	issue2550923_computed_property
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	merge default into branch
files

Reformat docstring for _Pointer::init

2023-07-10T00:39:32-04:00

changeset	b8dc16d9624a
branch
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	Reformat docstring for _Pointer::__init__
files

[issue2550923_computed_property] merge from tip into issue2550923_computed_property

2023-06-08T00:10:32-04:00

changeset	82bbb95e5690
branch	issue2550923_computed_property
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	merge from tip into issue2550923_computed_property
files

Add config argument for one more call

2023-03-04T11:51:45-05:00

changeset	76a21cf791b9
branch
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	Add config argument for one more call Also added Mock so it gets tested.
files

issue2551250: Fix sorting of detectors

2022-12-21T20:13:39+01:00

changeset	33eb82ad26ba
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	issue2551250: Fix sorting of detectors .. even if there are two with the same name and priority (can happen if they are created in two different files).
files

Fix internationalized strings with multiple unlabeled % replacements.

2022-11-21T18:02:19-05:00

changeset	9b4bd9bc9bdc
branch
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	Fix internationalized strings with multiple unlabeled % replacements. Get rid of warnings from gettext about untranslatable strings.
files

[issue2550923_computed_property] merge from main tip. This should fix test failure in Markdown2TestCase.test_string_markdown_code_block_attribute by merging html diff method used in tests.

2022-11-07T22:58:38-05:00

changeset	ca90f7270cd4
branch	issue2550923_computed_property
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	merge from main tip. This should fix test failure in Markdown2TestCase.test_string_markdown_code_block_attribute by merging html diff method used in tests.
files

inital flake8 cleanups

2022-09-13T17:42:32-04:00

changeset	8473039648da
branch
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	inital flake8 cleanups Replaced some calls to re.compile with module level variable. Renamed some variables.
files

issue2551159 - cl.filter fails if filterspec is None ...

2022-05-15T12:02:57-04:00

changeset	8ab98de22df0
branch
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	issue2551159 - cl.filter fails if filterspec is None ... Fix case where passing None into a filter() call as the filterprop causes a traceback. Also fix cases where sort or group are None causing tracebacks. Also fix typo in hyperdb filter() docstring.
files

[issue2550923_computed_property] merge from default branch. Fix travis.ci so CI builds don't error out

2022-04-21T16:54:17-04:00

changeset	e1588ae185dc
branch	issue2550923_computed_property
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	merge from default branch. Fix travis.ci so CI builds don't error out
files

Database class close method was not raising NotImplementedError

2022-03-05T19:04:50-05:00

changeset	6f35be77324c
branch
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	Database class close method was not raising NotImplementedError All other stubbed methods raise NotImplementedErrors. The close() method did not raise the error. Now it does.
files

Fix dict being changed during iteration

2021-10-21T13:07:33+02:00

changeset	3e6c2039d0a8
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Fix dict being changed during iteration .. for python3: simply using .keys() is not enough.
files

[issue2550923_computed_property] merge trunk changes into this branch

2021-09-16T14:30:56-04:00

changeset	30358e334232
branch	issue2550923_computed_property
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	merge trunk changes into this branch
files

fix isset() for StringHTMLProperty

2021-08-02T23:26:20-04:00

changeset	8e06194ff0b0
branch
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	fix isset() for StringHTMLProperty isset() is used in templates on a StringHTMLProperty, it returns True. 2.1.0 made default_value work properly. Hyperdb.py's String(_Type) class sets the default value to the empty string and not None. Change __init__ so default_value is None and not "". roundup-user mailing list thread: https://sourceforge.net/p/roundup/mailman/roundup-users/thread/20210801020640.73ac1729%40Dell/#msg37328813
files

Fix searching+sorting for Link properties

2021-05-17T20:52:27+02:00

changeset	7b1b6dffc7ed
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Fix searching+sorting for Link properties Note that this turns off recursive pre-populating the cache in filter_iter, we only pre-populate the top-level element. Otherwise this interacts with searching producing duplicates.
files

Tests for Link expressions

2021-05-17T17:35:50+02:00

changeset	a0c0ee3ed8b1
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Tests for Link expressions .. and fixes for anydbm backend, this now passes all the new tests.
files

Bug-fix in expression parser

2021-05-12T13:32:18+02:00

changeset	66ccddb034f2
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Bug-fix in expression parser Tried to get minimum of an empty list. Add a test for this.
files

Add documentation for multilink expression syntax

2021-05-11T11:32:26+02:00

changeset	ce99e0d39262
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Add documentation for multilink expression syntax
files

Tests and bug-fix for issue2551119

2021-05-10T16:19:37+02:00

changeset	9957d8d10783
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Tests and bug-fix for issue2551119 .. and some other failing tests I came up with when trying to reproduce the problem of the issue.
files

Make rev multilink for Link work

2021-05-08T18:13:20+02:00

changeset	619807d9a2df
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Make rev multilink for Link work Fix searching of retired items. Fix implementation for anydbm.
files

Make rev multilink expressions work for anydbm

2021-05-08T12:52:35+02:00

changeset	8bc5faeb7677
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Make rev multilink expressions work for anydbm
files

[issue2550923_computed_property] Initial implementation of Computed property

2020-11-27T18:09:00-05:00

changeset	1e5ed659e8ca
branch	issue2550923_computed_property
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	Initial implementation of Computed property It supports query/display in html, rest and xml interfaces. You can specify a cache parameter, but using it raises NotImplementedError. It does not support: search, sort or grouping by the computed field. Checking in on a branch to get more eyeballs on it and maybe some people to help.
files

issue2550522 - add transitive searching to filter in roundup-admin

2020-08-05T02:05:10-04:00

changeset	95183d73ac64
branch
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	issue2550522 - add transitive searching to filter in roundup-admin issue2550522 - Add 'filter' command to command-line interface. Filter command was actually added in 2.0.0, but this issue requested transitive searching. So that: roundup-admin -i . filter issue assignedto.username=Admin will work. This also fixes two bugs. If assignedto.username had no matches, all issues would be returned. admin.py had a find() call where the should have been a filter() call. Was tripped when -S, -c or -s were used.
files

Summary: Constrain format of classname and document

2020-07-28T06:24:39-04:00

changeset	6834bb5473da
branch
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	Summary: Constrain format of classname and document Define the format of a valid classname. Updated design doc, function doc and glossary to document format. Error message for case where we have a redefinition of a classname now says why ValueError is raised since we are raising ValueError for incorrect classname format as well. Tests for all cases including verification of cause for ValueError exceptions.
files

Fix reverse multilink iteration

2020-07-15T13:28:58+02:00

changeset	f40c6b5de370
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Fix reverse multilink iteration Adding a reverse multilink to the same class would trigger a traceback about a modified dictionary on iteration
files

Fix repr of hyperdb.Class

2020-05-20T10:21:12+02:00

changeset	bb198596f85c
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Fix __repr__ of hyperdb.Class If a error occurs during schema parsing, classname isn't known yet, account for this case in __repr__.
files

Fix rev_multilink properties search/retrieval

2020-05-19T09:25:48+02:00

changeset	a701c9c81597
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Fix rev_multilink properties search/retrieval The code now only returns live (non-retired) items. Since for reverse multilinks the Link/Multilink property in the retired item cannot be changed, we now only return non-retired items in search (filter) and retrieve (get).
files

Allow to define reverse Multilinks

2020-04-29T16:30:27+02:00

changeset	8497bf3f23a1
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Allow to define reverse Multilinks Now it's possible to specify a rev_multilink parameter when creating Link or Multilink properties. The parameter takes a property name to be inserted into the linked-to class. It allows to navigate from the other side of the link as if it where a forward Multilink using the existing data structures.
files

Flake8 fixes; use isinstance rather than type equality.

2019-12-28T14:41:55-05:00

changeset	cf800f1febe6
branch
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	Flake8 fixes; use isinstance rather than type equality. Also remove assignment for unused variable. Replace x with _x in for loop variables that are not used but required to unpack tuple.
files

Backwards-compatible signature of _proptree method

2019-08-26T22:45:42+02:00

changeset	acc4a128ab9b
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Backwards-compatible signature of _proptree method
files

Implement limit and offset for filter

2019-08-26T20:23:53+02:00

changeset	16e1255b16cf
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Implement limit and offset for filter These map to the corresponding SQL parameters for the SQL backends.
files

Implement exact string search

2019-08-26T18:18:02+02:00

changeset	ee2e8f8d6648
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Implement exact string search .. in the 'filter' method of hyperdb.Class (and the corresponding backend implementations).
files

Try to fix:

2019-06-12T20:34:47-04:00

changeset	936275dfe1fa
branch
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	Try to fix: DeprecationWarning: invalid escape sequence \d DeprecationWarning: invalid escape sequence \s DeprecationWarning: invalid escape sequence \) Strings under python 3 are unicode strings rather then "regular" strings as under python 2. So all regexps need to be raw strings. We will see how many I fixed and if I broke any.
files

Try to deal with this warning/error: DeprecationWarning: generator

2019-06-12T19:38:33-04:00

changeset	0467d80eaeec
branch
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	Try to deal with this warning/error: DeprecationWarning: generator 'iter_roles' raised StopIteration PEP479 specifies how to rework it by using return rather than raise StopIteration. I am not sure if this is right, but test/test_cgi.py::FormTestCase::testRoles is still passing.
files

Properly handle mapping of KeyError to AttributeError in getattr

2019-04-08T19:38:06-04:00

changeset	5a9159ad773f
branch
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	Properly handle mapping of KeyError to AttributeError in __getattr__ Update doc indicating same mapping in __setattr__.
files

Python3: fix crash bug when importing binary file (like jpeg). The

2019-03-15T17:27:31-04:00

changeset	a4514df51ded
branch
bookmark
tag
user	John Rouillard <rouilj@ieee.org>
description	Python3: fix crash bug when importing binary file (like jpeg). The hyperdb content property inteprets the file as text and raises an error if the UTF-8 decode fails. To fix, read file as binary/byte and do conversion (ignoring errors) at the import level not the hyperdb level.
files

Avoid errors from selecting "no selection" on multilink (issue2550722).

2018-09-27T11:33:01+00:00

changeset	fea11d05110e
branch
bookmark
tag
user	Joseph Myers <jsm@polyomino.org.uk>
description	Avoid errors from selecting "no selection" on multilink (issue2550722). As discussed in issue 2550722 there are various cases where selecting "no selection" on a multilink can result in inappropriate errors from Roundup: * If selecting "no selection" produces a null edit (a value was set in the multilink in an edit with an error, then removed again, along with all other changes, in the next form submission), so the page is rendered from the form contents including the "-" value for "no selection" for the multilink. * If creating an item with a nonempty value for a multilink has an error, and the resubmission changes that multilink to "no selection" (and this in turn has subcases, according to whether the creation then succeeds or fails on the resubmission, which need fixes in different places in the Roundup code). All of these cases have in common that it is expected and OK to have a "-" value for a submission for a multilink when is not set in that multilink in the database (because the original attempt to set in that multilink had an error), so the hyperdb.py logic to give an error in that case is thus removed. In the subcase of the second case where the resubmission with "no selection" has an error, the templating code tries to produce a menu entry for the "-" multilink value, which also results in an error, hence the templating.py change to ignore such values in the list for a multilink.
files

No traceback for non-existent items in history

2018-09-03T12:00:46+02:00

changeset	a7c3cd2edf51
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	No traceback for non-existent items in history
files

Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).

2018-09-02T23:48:04+00:00

changeset	bb7865241f8a
branch
bookmark
tag
user	Joseph Myers <jsm@polyomino.org.uk>
description	Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975). The roundup-admin export and import commands are used for migrating between different database backends. It is desirable that they should be usable also for migrations between Python 2 and Python 3, and in some cases (e.g. with the anydbm backend) this may be required. To be usable for such migrations, the format of the generated CSV files needs to be stable, meaning the same as currently used with Python 2. The export process uses repr() to produce the fields in the CSV files and eval() to convert them back to Python data structures. repr() of strings with non-ASCII characters produces different results for Python 2 and Python 3. This patch adds repr_export and eval_import functions to roundup/anypy/strings.py which provide the required operations that are just repr() and eval() in Python 2, but are more complicated in Python 3 to use data representations compatible with Python 2. These functions are then used in the required places for export and import. repr() and eval() are also used in storing the dict of changed values in the journal for the RDBMS backends. It is similarly desirable that the database be compatible between Python 2 and Python 3, so that export and import do not need to be used for a migration between Python versions for non-anydbm back ends. Thus, this patch changes rdbms_common.py in the places involved in storing journals in the database, not just in those involved in import/export. Given this patch, import/export with non-ASCII characters appear based on some limited testing to work across Python versions, and an instance using the sqlite backend appears to be compatible between Python versions without needing import/export, if the sessions/otks databases (which use anydbm) are deleted when changing Python version.
files

helper to allow comparing dicts and None values in Python 3

2018-08-01T22:03:50+01:00

changeset	9a09719b0d8e
branch
bookmark
tag
user	Christof Meerwald <cmeerw@cmeerw.org>
description	helper to allow comparing dicts and None values in Python 3
files

Python 3 built-in types can't be compared against None

2018-07-28T20:34:31+01:00

changeset	09966de4d3f6
branch
bookmark
tag
user	Christof Meerwald <cmeerw@cmeerw.org>
description	Python 3 built-in types can't be compared against None
files

container modification while iterating over it

2018-07-24T21:25:38+01:00

changeset	87f22a5d65ca
branch
bookmark
tag
user	Christof Meerwald <cmeerw@cmeerw.org>
description	container modification while iterating over it
files

Python 3 preparation: use list() around zip() as needed.

2018-07-24T23:13:17+00:00

changeset	f2c5e0f6506e
branch
bookmark
tag
user	Joseph Myers <jsm@polyomino.org.uk>
description	Python 3 preparation: use list() around zip() as needed. Tool-assisted patch.
files

Python 3 preparation: update calls to dict methods.

2018-07-24T23:04:42+00:00

changeset	23b8e6067f7c
branch
bookmark
tag
user	Joseph Myers <jsm@polyomino.org.uk>
description	Python 3 preparation: update calls to dict methods. Tool-assisted patch. Changes of iterkeys / itervalues / iteritems to keys / values / items are fully automated, but may make things less efficient in Python 2. Automated tools want to add list() around many calls to keys / values / items, but I thought most such list() additions were unnecessary because it seemed the result of keys / values / items was just iterated over while the set of dict keys remained unchanged, rather than used in a way requiring an actual list, or used while the set of keys in the dict could change. It's quite possible I missed some cases where list() was really needed, or left in some unnecessary list() calls. In cases where list() was only needed because the resulting list was then sorted in-place, I changed the code to use calls to sorted().
files

Python 3 preparation: make relative imports explicit.

2018-07-24T22:22:08+00:00

changeset	d26921b851c3
branch
bookmark
tag
user	Joseph Myers <jsm@polyomino.org.uk>
description	Python 3 preparation: make relative imports explicit. Tool-generated patch.
files

Python 3 preparation: change "x.has_key(y)" to "y in x".

2018-07-24T22:08:17+00:00

changeset	0942fe89e82e
branch
bookmark
tag
user	Joseph Myers <jsm@polyomino.org.uk>
description	Python 3 preparation: change "x.has_key(y)" to "y in x". (Also likewise "not in" where appropriate.) Tool-generated patch.
files

Python 3 preparation: "raise" syntax.

2018-07-24T21:39:58+00:00

changeset	35ea9b1efc14
branch
bookmark
tag
user	Joseph Myers <jsm@polyomino.org.uk>
description	Python 3 preparation: "raise" syntax. Changing "raise Exception, value" to "raise Exception(value)". Tool-assisted patch. Particular cases to check carefully are the one place in frontends/ZRoundup/ZRoundup.py where a string exception needed to be fixed, and the one in roundup/cgi/client.py involving raising an exception with a traceback (requires three-argument form of raise in Python 2, which as I understand it requires exec() to avoid a Python 3 syntax error).
files

Merge old commit

2018-05-29T13:41:52+02:00

changeset	8cd518058007
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Merge old commit
files

Remove duplicate super call

2016-12-12T16:51:01+01:00

changeset	882dfa723df5
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Remove duplicate super call
files

Fix commits although a Reject exception is raised

2018-04-20T18:46:28+02:00

changeset	62de601bdf6f
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Fix commits although a Reject exception is raised Fix the problem that changes are committed to the database (due to commits to otk handling) even when a Reject exception occurs. The fix implements separate database connections for otk/session handling and normal database operation.
files

Add a 'retired' parameter to Class.filter

2018-04-19T20:01:43+02:00

changeset	506c7ee9a385
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Add a 'retired' parameter to Class.filter .. to allow searching for retired, non-retired or all (retired and non-retired) items similar to the argument of the same name to Class.getnodeids. This is 'False' by default (finding only non-retired items for backwards compatibility) and can be set to None (for finding retired and non-retired items) or True (for finding only retired items).
files

Fix issue2550954: History display breaks

2018-04-19T12:59:23+02:00

changeset	5a014410f254
branch
bookmark
tag
user	Ralf Schlatterbeck <rsc@runtux.com>
description	Fix issue2550954: History display breaks .. on removed properties: Now changes to removed properties, and link/unlink events from non-existing properties or classes no longer trigger a traceback. Concerning the visibility: We have a new config-item obsolete_history_roles in the main section that defines which roles may see removed properties. By default only role Admin is allowed to see these.
files