annotate locale/GNUmakefile @ 6593:e70e2789bc2c

issue2551189 - increase text search maxlength This removes I think all the magic references to 25 and 30 (varchar size) and replaces them with references to maxlength or maxlength+5. I am not sure why the db column is 5 characters larger than the size of what should be the max size of a word, but I'll keep the buffer of 5 as making it 1/5 the size of maxlength makes less sense. Also added tests for fts search in templating which were missing. Added postgres, mysql and sqlite native indexing backends in which to test fts. Added fts test to native-fts as well to make sure it's working. I want to commit this now for CI. Todo: add test cases for the use of FTS in the csv output in actions.py. There is no test coverage of the match case there. change maxlength to a higher value (50) as requested in the ticket. Modify existing extremewords test cases to allow words > 25 and < 51 write code to migrate column sizes for mysql and postgresql to match maxlength I will roll this into the version 7 schema update that supports use of database fts support.
author John Rouillard <rouilj@ieee.org>
date Tue, 25 Jan 2022 13:22:00 -0500
parents 9730eff21de3
children b6452b669720
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2330
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
1 # Extract translatable strings from Roundup sources,
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
2 # update and compile all existing translations
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
3
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
4 # tool locations
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
5 XPOT ?= xpot
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
6 MSGFMT ?= msgfmt
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
7 MSGMERGE ?= msgmerge
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
8 XGETTEXT ?= xgettext
2384
33469189bf86 extract messages from web templates
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2383
diff changeset
9 PYTHON ?= python
2330
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
10
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
11 TEMPLATE=roundup.pot
2306
1421b19cd61b build message template file
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
12
4094
5a9631828e33 update message template file building:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3761
diff changeset
13 PACKAGES=$(shell find ../roundup ../share/roundup/templates -name '*.py' \
3761
800bc51a9c8e add template directories to the list of Python packages for message extraction
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3593
diff changeset
14 | sed -e 's,/[^/]*$$,,' | sort | uniq)
2330
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
15 SOURCES=$(PACKAGES:=/*.py)
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
16 PO_FILES=$(wildcard *.po)
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
17 MO_FILES=$(PO_FILES:.po=.mo)
6222
2c0f7f16ce04 Check that build link is present otherwise abort.
John Rouillard <rouilj@ieee.org>
parents: 4570
diff changeset
18 PYTHON_BUILD=../build/lib
2c0f7f16ce04 Check that build link is present otherwise abort.
John Rouillard <rouilj@ieee.org>
parents: 4570
diff changeset
19 RUN_PYTHON=PYTHONPATH=$(PYTHON_BUILD) $(PYTHON) -O
2330
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
20
2455
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
21 all: dist
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
22
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
23 help:
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
24 @echo "$(MAKE) - build MO files. Run this before sdist"
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
25 @echo "$(MAKE) template - update message template from sources"
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
26 @echo "$(MAKE) locale.po - update message file from template"
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
27 @echo "$(MAKE) locale.mo - compile individual message file"
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
28 @echo "$(MAKE) help - this text"\
2330
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
29
2455
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
30 # This will rebuild all MO files without updating their corresponding PO
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
31 # files first. Run before creating Roundup distribution (hence the name).
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
32 # PO files should be updated by their translators only, automatic update
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
33 # adds unwanted fuzzy labels.
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
34 dist:
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
35 for file in $(PO_FILES); do \
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
36 ${MSGFMT} -o `basename $$file .po`.mo $$file; \
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
37 done
2306
1421b19cd61b build message template file
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
diff changeset
38
2455
ece6ed48dd3d by default, build binary message catalogs only...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2384
diff changeset
39 template:
6222
2c0f7f16ce04 Check that build link is present otherwise abort.
John Rouillard <rouilj@ieee.org>
parents: 4570
diff changeset
40 test -d $(PYTHON_BUILD) || (echo "Missing build directory. ln -s lib to build library"; exit 1)
2459
fd04f41ab988 fix: 'template' target was meant to make 'roundup.pot' file
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2455
diff changeset
41 ${XPOT} -n -o $(TEMPLATE) $(SOURCES)
6222
2c0f7f16ce04 Check that build link is present otherwise abort.
John Rouillard <rouilj@ieee.org>
parents: 4570
diff changeset
42
2459
fd04f41ab988 fix: 'template' target was meant to make 'roundup.pot' file
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2455
diff changeset
43 ${RUN_PYTHON} ../roundup/cgi/TAL/talgettext.py -u $(TEMPLATE) \
4094
5a9631828e33 update message template file building:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3761
diff changeset
44 ../share/roundup/templates/classic/html/*.html \
6405
a21b574554c3 extract translatable strings from devel and responsive templates
John Rouillard <rouilj@ieee.org>
parents: 6222
diff changeset
45 ../share/roundup/templates/devel/html/*.html \
a21b574554c3 extract translatable strings from devel and responsive templates
John Rouillard <rouilj@ieee.org>
parents: 6222
diff changeset
46 ../share/roundup/templates/minimal/html/*.html \
a21b574554c3 extract translatable strings from devel and responsive templates
John Rouillard <rouilj@ieee.org>
parents: 6222
diff changeset
47 ../share/roundup/templates/responsive/html/*.html
2330
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
48 ${XGETTEXT} -j -w 80 -F \
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
49 --msgid-bugs-address=roundup-devel@lists.sourceforge.net \
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
50 --copyright-holder="See Roundup README.txt" \
2459
fd04f41ab988 fix: 'template' target was meant to make 'roundup.pot' file
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2455
diff changeset
51 -o $(TEMPLATE) $(SOURCES)
2330
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
52
6480
0f0cee081990 Force test of a locale
John Rouillard <rouilj@ieee.org>
parents: 6405
diff changeset
53 local_install: dist
0f0cee081990 Force test of a locale
John Rouillard <rouilj@ieee.org>
parents: 6405
diff changeset
54 for file in $(MO_FILES); do \
6483
9730eff21de3 Remove verbose from local_install. Cleanup log.
John Rouillard <rouilj@ieee.org>
parents: 6480
diff changeset
55 lang=`basename $$file .mo`; \
6480
0f0cee081990 Force test of a locale
John Rouillard <rouilj@ieee.org>
parents: 6405
diff changeset
56 mkdir -p locale/$$lang/LC_MESSAGES; \
0f0cee081990 Force test of a locale
John Rouillard <rouilj@ieee.org>
parents: 6405
diff changeset
57 cp $$file locale/$$lang/LC_MESSAGES/roundup.mo; \
0f0cee081990 Force test of a locale
John Rouillard <rouilj@ieee.org>
parents: 6405
diff changeset
58 done
0f0cee081990 Force test of a locale
John Rouillard <rouilj@ieee.org>
parents: 6405
diff changeset
59
3578
782287e01b99 added template diff target: view changes in message strings
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2459
diff changeset
60 # helps to check template file before check in
782287e01b99 added template diff target: view changes in message strings
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2459
diff changeset
61 diff:
4094
5a9631828e33 update message template file building:
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3761
diff changeset
62 svn diff roundup.pot|grep -v '^[-+]#'|vim -Rv -
3578
782287e01b99 added template diff target: view changes in message strings
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2459
diff changeset
63
2330
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
64 %.po: $(TEMPLATE)
3593
3d9e1906ba58 don't wrap po files at position 72
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3578
diff changeset
65 ${MSGMERGE} -U --suffix=.bak $@ $<
2330
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
66
8ae58eda9331 in addition to xpot...
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2306
diff changeset
67 %.mo: %.po
2342
106737dbe689 show statistics for message catalogs
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2330
diff changeset
68 ${MSGFMT} --statistics -o $@ $<

Roundup Issue Tracker: http://roundup-tracker.org/