Mercurial > p > roundup > code
comparison roundup/cgi/actions.py @ 6593:e70e2789bc2c
issue2551189 - increase text search maxlength
This removes I think all the magic references to 25 and 30 (varchar
size) and replaces them with references to maxlength or maxlength+5.
I am not sure why the db column is 5 characters larger than the size
of what should be the max size of a word, but I'll keep the buffer
of 5 as making it 1/5 the size of maxlength makes less sense.
Also added tests for fts search in templating which were missing.
Added postgres, mysql and sqlite native indexing backends in which to
test fts. Added fts test to native-fts as well to make sure it's
working.
I want to commit this now for CI.
Todo:
add test cases for the use of FTS in the csv output in
actions.py. There is no test coverage of the match case there.
change maxlength to a higher value (50) as requested in the ticket.
Modify existing extremewords test cases to allow words > 25 and < 51
write code to migrate column sizes for mysql and postgresql to match
maxlength I will roll this into the version 7 schema update that
supports use of database fts support.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Tue, 25 Jan 2022 13:22:00 -0500 |
| parents | 91ab3e0ffcd0 |
| children | b336cc98d9d2 |
comparison
equal
deleted
inserted
replaced
| 6592:828e2eaee7cd | 6593:e70e2789bc2c |
|---|---|
| 1431 % {'column': html_escape(cname), | 1431 % {'column': html_escape(cname), |
| 1432 'class': request.classname}) | 1432 'class': request.classname}) |
| 1433 | 1433 |
| 1434 # full-text search | 1434 # full-text search |
| 1435 if request.search_text: | 1435 if request.search_text: |
| 1436 indexer = self.db.indexer | |
| 1436 if self.db.indexer.query_language: | 1437 if self.db.indexer.query_language: |
| 1437 try: | 1438 try: |
| 1438 matches = self.db.indexer.search( | 1439 matches = indexer.search( |
| 1439 [request.search_text], klass) | 1440 [request.search_text], klass) |
| 1440 except Exception as e: | 1441 except Exception as e: |
| 1441 error = " ".join(e.args) | 1442 error = " ".join(e.args) |
| 1442 self.client.add_error_message(error) | 1443 self.client.add_error_message(error) |
| 1443 self.client.response_code = 400 | 1444 self.client.response_code = 400 |
| 1444 # trigger error reporting. NotFound isn't right but... | 1445 # trigger error reporting. NotFound isn't right but... |
| 1445 raise exceptions.NotFound(error) | 1446 raise exceptions.NotFound(error) |
| 1446 else: | 1447 else: |
| 1447 matches = self.db.indexer.search( | 1448 matches = indexer.search( |
| 1448 re.findall(r'\b\w{2,25}\b', request.search_text), klass) | 1449 re.findall(r'\b\w{%s,%s}\b' % (indexer.minlength, |
| 1450 indexer.maxlength), | |
| 1451 request.search_text), klass) | |
| 1449 else: | 1452 else: |
| 1450 matches = None | 1453 matches = None |
| 1451 | 1454 |
| 1452 header = self.client.additional_headers | 1455 header = self.client.additional_headers |
| 1453 header['Content-Type'] = 'text/csv; charset=%s' % self.client.charset | 1456 header['Content-Type'] = 'text/csv; charset=%s' % self.client.charset |
| 1607 % {'column': html_escape(cname), | 1610 % {'column': html_escape(cname), |
| 1608 'class': request.classname}) | 1611 'class': request.classname}) |
| 1609 | 1612 |
| 1610 # full-text search | 1613 # full-text search |
| 1611 if request.search_text: | 1614 if request.search_text: |
| 1612 if self.db.indexer.query_language: | 1615 indexer = self.db.indexer |
| 1616 if indexer.query_language: | |
| 1613 try: | 1617 try: |
| 1614 matches = self.db.indexer.search( | 1618 matches = indexer.search( |
| 1615 [request.search_text], klass) | 1619 [request.search_text], klass) |
| 1616 except Exception as e: | 1620 except Exception as e: |
| 1617 error = " ".join(e.args) | 1621 error = " ".join(e.args) |
| 1618 self.client.add_error_message(error) | 1622 self.client.add_error_message(error) |
| 1619 self.client.response_code = 400 | 1623 self.client.response_code = 400 |
| 1620 # trigger error reporting. NotFound isn't right but... | 1624 # trigger error reporting. NotFound isn't right but... |
| 1621 raise exceptions.NotFound(error) | 1625 raise exceptions.NotFound(error) |
| 1622 else: | 1626 else: |
| 1623 matches = self.db.indexer.search( | 1627 matches = indexer.search( |
| 1624 re.findall(r'\b\w{2,25}\b', request.search_text), | 1628 re.findall(r'\b\w{%s,%s}\b' % (indexer.minlength, |
| 1629 indexer.maxlength), | |
| 1630 request.search_text), | |
| 1625 klass) | 1631 klass) |
| 1626 else: | 1632 else: |
| 1627 matches = None | 1633 matches = None |
| 1628 | 1634 |
| 1629 h = self.client.additional_headers | 1635 h = self.client.additional_headers |