Mercurial > p > roundup > code
annotate test/test_token.py @ 8180:d02ce1d14acd
feat: issue2551068 - Provide way to retrieve file/msg data via rest endpoint.
Use Allow header to change format of /binary_content endpoint. If
Allow header for endpoint is not application/json, it will be matched
against the mime type for the file. */*, text/* are supported and will
return the native mime type if present.
Changes:
move */* mime type from static dict of supported types. It was
hardcoded to return json only. Now it can return a matching
non-json mime type for the /binary_content endpoint.
Edited some errors to explicitly add */* mime type.
Cleanups to use ', ' separation in lists of valid mime types rather
than just space separated.
Remove ETag header when sending raw content. See issue 2551375 for
background.
Doc added to rest.txt.
Small format fix up (add dash) in CHANGES.txt.
Make passing an unset/None/False accept_mime_type to
format_dispatch_output a 500 error. This used to be the fallback
to produce a 406 error after all processing had happened. It
should no longer be possible to take that code path as all 406
errors (with valid accept_mime_types) are generated before
processing takes place.
Make format_dispatch_output handle output other than json/xml so it
can send back binary_content data.
Removed a spurious client.response_code = 400 that seems to not be
used.
Tests added for all code paths.
Database setup for tests msg and file entry. This required a file
upload test to change so it doesn't look for file1 as the link
returned by the upload. Download the link and verify the data
rather than verifying the link.
Multiple formatting changes to error messages to make all lists of
valid mime types ', ' an not just space separated.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Sun, 08 Dec 2024 17:22:33 -0500 |
| parents | 9a74dfeb8620 |
| children |
| rev | line source |
|---|---|
|
470
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
1 # |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
2 # Copyright (c) 2001 Richard Jones |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
3 # This module is free software, and you may redistribute it and/or modify |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
4 # under the same terms as Python, so long as this copyright message and |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
5 # disclaimer are retained in their original form. |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
6 # |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
7 # This module is distributed in the hope that it will be useful, |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
8 # but WITHOUT ANY WARRANTY; without even the implied warranty of |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
9 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
10 |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
11 import unittest, time |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
12 |
|
7181
6971c9249c6d
Fix missed roundup.token import.
John Rouillard <rouilj@ieee.org>
parents:
5037
diff
changeset
|
13 from roundup.token_r import token_split |
|
470
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
14 |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
15 class TokenTestCase(unittest.TestCase): |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
16 def testValid(self): |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
17 l = token_split('hello world') |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
18 self.assertEqual(l, ['hello', 'world']) |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
19 |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
20 def testIgnoreExtraSpace(self): |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
21 l = token_split('hello world ') |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
22 self.assertEqual(l, ['hello', 'world']) |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
23 |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
24 def testQuoting(self): |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
25 l = token_split('"hello world"') |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
26 self.assertEqual(l, ['hello world']) |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
27 l = token_split("'hello world'") |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
28 self.assertEqual(l, ['hello world']) |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
29 |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
30 def testEmbedQuote(self): |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
31 l = token_split(r'Roch\'e Compaan') |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
32 self.assertEqual(l, ["Roch'e", "Compaan"]) |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
33 l = token_split('address="1 2 3"') |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
34 self.assertEqual(l, ['address=1 2 3']) |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
35 |
|
7859
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
36 def testEmbedEscapeQuote(self): |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
37 l = token_split(r'"Roch\'e Compaan"') |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
38 self.assertEqual(l, ["Roch'e Compaan"]) |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
39 |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
40 l = token_split(r'"Roch\"e Compaan"') |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
41 self.assertEqual(l, ['Roch"e Compaan']) |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
42 |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
43 l = token_split(r'sql "COLLATE = \"utf8mb4_unicode_ci\";"') |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
44 self.assertEqual(l, ["sql", 'COLLATE = "utf8mb4_unicode_ci";']) |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
45 |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
46 l = token_split(r'''sql 'COLLATE = "utf8mb4_unicode_ci";' ''') |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
47 self.assertEqual(l, ["sql", 'COLLATE = "utf8mb4_unicode_ci";']) |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
48 |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
49 l = token_split(r'''sql 'COLLATE = \"utf8mb4_unicode_ci\";' ''') |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
50 self.assertEqual(l, ["sql", 'COLLATE = "utf8mb4_unicode_ci";']) |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
51 |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
52 l = token_split(r'''sql 'COLLATE = \'utf8mb4_unicode_ci\';' ''') |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
53 self.assertEqual(l, ["sql", "COLLATE = 'utf8mb4_unicode_ci';"]) |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
54 |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
55 l = token_split(r'''sql 'new\nline\rneed \ttab' ''') |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
56 self.assertEqual(l, ["sql", "new\nline\rneed \ttab"]) |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
57 |
|
470
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
58 def testEscaping(self): |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
59 l = token_split('"Roch\'e" Compaan') |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
60 self.assertEqual(l, ["Roch'e", "Compaan"]) |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
61 l = token_split(r'hello\ world') |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
62 self.assertEqual(l, ['hello world']) |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
63 l = token_split(r'\\') |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
64 self.assertEqual(l, ['\\']) |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
65 l = token_split(r'\n') |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
66 self.assertEqual(l, ['\n']) |
|
7859
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
67 l = token_split(r'\r') |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
68 self.assertEqual(l, ['\r']) |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
69 l = token_split(r'\t') |
|
9a74dfeb8620
feat: can use escaped tokens inside quotes including quotes.
John Rouillard <rouilj@ieee.org>
parents:
7181
diff
changeset
|
70 self.assertEqual(l, ['\t']) |
|
470
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
71 |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
72 def testBadQuote(self): |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
73 self.assertRaises(ValueError, token_split, '"hello world') |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
74 self.assertRaises(ValueError, token_split, "Roch'e Compaan") |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
75 |
|
9f7320624bc2
Added better tokenising to roundup-admin - handles spaces and stuff.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
76 # vim: set filetype=python ts=4 sw=4 et si |
