comparison doc/rest.txt @ 7045:ca90f7270cd4 issue2550923_computed_property

merge from main tip. This should fix test failure in Markdown2TestCase.test_string_markdown_code_block_attribute by merging html diff method used in tests.
author John Rouillard <rouilj@ieee.org>
date Mon, 07 Nov 2022 22:58:38 -0500
parents e7b4ad2c57ac
children 42e68162279b
comparison
equal deleted inserted replaced
6638:e1588ae185dc 7045:ca90f7270cd4
1 .. meta:: 1 .. meta::
2 :description language=en: 2 :description:
3 Documentation on the RESTful interface to the Roundup Issue 3 Documentation on the RESTful interface to the Roundup Issue
4 Tracker. 4 Tracker.
5 5
6 6
7 .. index:: pair: api; Representational state transfer 7 .. index:: pair: api; Representational state transfer
69 ======================= 69 =======================
70 70
71 Clients should set the header X-REQUESTED-WITH to any value and the 71 Clients should set the header X-REQUESTED-WITH to any value and the
72 tracker's config.ini should have ``csrf_enforce_header_x-requested-with 72 tracker's config.ini should have ``csrf_enforce_header_x-requested-with
73 = yes`` or ``required``. 73 = yes`` or ``required``.
74
75 If you want to allow Roundup's api to be accessed by an application
76 that is not hosted at the same origin as Roundup, you must permit
77 the origin using the ``allowed_api_origins`` setting in
78 ``config.ini``.
74 79
75 Rate Limiting the API 80 Rate Limiting the API
76 ===================== 81 =====================
77 82
78 This is a work in progress. This version of roundup includes Rate 83 This is a work in progress. This version of roundup includes Rate
201 206
202 Otherwise Content-Type is allowed to be ``application/json`` or 207 Otherwise Content-Type is allowed to be ``application/json`` or
203 ``application/x-www-form-urlencoded``. Any other value returns error 208 ``application/x-www-form-urlencoded``. Any other value returns error
204 code 415. 209 code 415.
205 210
211 CORS preflight requests
212 ^^^^^^^^^^^^^^^^^^^^^^^
213
214 CORS preflight requests are done using the OPTIONS method. They
215 require that REST be enabled. These requests do not make any changes
216 or get any information from the database. As a result they are
217 available to the anonymous user and any authenticated user. The user
218 does not need to have `Rest Access` permissions. Also these requests
219 bypass CSRF checks except for the Origin header check which is always
220 run for preflight requests.
221
222 You can permit only allowed ORIGINS by setting ``allowed_api_origins``
223 in ``config.ini`` to the list of origins permitted to access your
224 api. By default only your tracker's origin is allowed. If a preflight
225 request fails, the api request will be stopped by the browser.
226
227 The following CORS preflight headers are usually added automatically by
228 the browser and must all be present:
229
230 * `Access-Control-Request-Headers`
231 * `Access-Control-Request-Method`
232 * `Origin`
233
234 The 204 response will include the headers:
235
236 * `Access-Control-Allow-Origin`
237 * `Access-Control-Allow-Headers`
238 * `Access-Control-Allow-Methods`
239 * `Access-Control-Allow-Credentials: true`
240 * `Access-Control-Max-Age: 86400`
241
242 If the endpoint accepts the PATCH verb the header `Accept-Patch` with
243 valid mime types (usually `application/x-www-form-urlencoded,
244 multipart/form-data`) will be included.
245
246 It will also include rate limit headers since the request is included
247 in the rate limit for the URL. The results from the CORS preflight
248 should be cached for a day so preflight requests are not expected to
249 cause a problem. If it is an issue, you can see
250 `Creating Custom Rate Limits`_
251 and craft a rate limiter that ignores anonymous OPTIONS requests.
252
206 Response Formats 253 Response Formats
207 ================ 254 ================
208 255
209 The default response format is json. 256 The default response format is json.
210 257
211 If you add the ``dicttoxml.py`` module you can request XML formatted 258 If you add the ``dicttoxml.py`` module you can request XML formatted
212 data using the header ``Accept: application/xml`` in your 259 data using the header ``Accept: application/xml`` in your
213 request. Both output formats are similar in structure. 260 request. Both output formats are similar in structure.
261
262 ``dicttoxml.py`` should be installed in the Python install directory,
263 or the file can be added to the Roundup installation directory long
264 ide ``rest.py``. It can also be enabled on a per tracker basis by
265 adding ``dicttoxml.py`` to the lib directory in your tracker home (you
266 may need to create the directory). Then this can be added to
267 `interfaces.py`_ to enable xml::
268
269 from roundup import rest
270 from dicttoxml import dicttoxml as dtox # from tracker_root/lib directory
271
272 rest.dicttoxml = dtox
273
274 .. _interfaces.py: customizing.html#interfaces-py-hooking-into-the-core-of-roundup
214 275
215 The rest interface accepts the http accept header and can include 276 The rest interface accepts the http accept header and can include
216 ``q`` values to specify the preferred mechanism. This is the preferred 277 ``q`` values to specify the preferred mechanism. This is the preferred
217 way to specify alternate acceptable response formats. 278 way to specify alternate acceptable response formats.
218 279
421 - Link 482 - Link
422 - find any issue whose status link is set to the id 2. 483 - find any issue whose status link is set to the id 2.
423 * - ``status=open`` 484 * - ``status=open``
424 - Link 485 - Link
425 - find any issue where the name of the status is open. 486 - find any issue where the name of the status is open.
426 Note this is not a string match so using nosy=ope will fail. 487 Note this is not a string match so using status=ope will fail.
427 * - ``nosy=1`` 488 * - ``nosy=1``
428 - MultiLink 489 - MultiLink
429 - find any issue where the multilink nosy includes the id 1. 490 - find any issue where the multilink nosy includes the id 1.
430 * - ``nosy=admin`` 491 * - ``nosy=admin``
431 - MultiLink 492 - MultiLink
641 702
642 { 703 {
643 "data": { 704 "data": {
644 "collection": [ 705 "collection": [
645 { 706 {
646 "link": 707 "link": "https://.../rest/data/issue/1",
647 "https://.../rest/data/issue/1",
648 "id": "1", 708 "id": "1",
649 "status": { 709 "status": {
650 "link": 710 "link": "https://.../rest/data/status/1",
651 "https://.../rest/data/status/1",
652 "id": "1" 711 "id": "1"
653 } 712 }
654 }, 713 },
655 ... 714 ...
656 } 715 }
1100 "1", 1159 "1",
1101 "5" 1160 "5"
1102 ] 1161 ]
1103 }, 1162 },
1104 "type": "issue", 1163 "type": "issue",
1105 "link": 1164 "link": "https://example.com/demo/rest/data/issue/23",
1106 "https://example.com/demo/rest/data/issue/23",
1107 "id": "23" 1165 "id": "23"
1108 } 1166 }
1109 } 1167 }
1110 1168
1111 If the above command is repeated with the data attribute:: 1169 If the above command is repeated with the data attribute::
1264 { 1322 {
1265 "data": { 1323 "data": {
1266 "id": "11", 1324 "id": "11",
1267 "type": "<class 'str'>", 1325 "type": "<class 'str'>",
1268 "link": "https://.../demo/rest/data/msg/11/content", 1326 "link": "https://.../demo/rest/data/msg/11/content",
1269 "data": "of has to who pleasure. or of account give because the reprehenderit\neu to quisquam velit, passage, was or...", 1327 "data": "of has to who pleasure. or of account give because the
1328 reprehenderit\neu to quisquam velit, passage, was or...",
1270 "@etag": "\"584f82231079e349031bbb853747df1c\"" 1329 "@etag": "\"584f82231079e349031bbb853747df1c\""
1271 } 1330 }
1272 } 1331 }
1273 1332
1274 (the content property is wrapped for display, it is one long line.) 1333 (the content property is wrapped for display, it is one long line.)
1405 >>> print(r.json()) 1464 >>> print(r.json())
1406 1465
1407 Note the addition of headers for: x-requested-with and referer. This 1466 Note the addition of headers for: x-requested-with and referer. This
1408 allows the request to pass the CSRF protection mechanism. You may need 1467 allows the request to pass the CSRF protection mechanism. You may need
1409 to add an Origin header if this check is enabled in your tracker's 1468 to add an Origin header if this check is enabled in your tracker's
1410 config.ini (look for csrf_enforce_header_origin). 1469 config.ini (look for csrf_enforce_header_origin). (Note the Origin
1470 header check may have to be disabled if an application is making a
1471 CORS request to the Roundup server. If you have this issue, please
1472 contact the Roundup team using the mailing lists as this is a bug.)
1411 1473
1412 A similar curl based retire example is to use:: 1474 A similar curl based retire example is to use::
1413 1475
1414 curl -s -u admin:admin \ 1476 curl -s -u admin:admin \
1415 -H "Referer: https://tracker.example.com/demo/" \ 1477 -H "Referer: https://tracker.example.com/demo/" \
1544 structure and implementation. 1606 structure and implementation.
1545 1607
1546 Adding new rest endpoints 1608 Adding new rest endpoints
1547 ========================= 1609 =========================
1548 1610
1549 Add or edit the file interfaces.py at the root of the tracker 1611 Add or edit the file `interfaces.py`_ at the root of the tracker
1550 directory. 1612 directory.
1551 1613
1552 In that file add:: 1614 In that file add::
1553 1615
1554 from roundup.rest import Routing, RestfulInstance, _data_decorator 1616 from roundup.rest import Routing, RestfulInstance, _data_decorator
1799 1861
1800 1. install pyjwt library using pip or pip3. If roundup can't find the 1862 1. install pyjwt library using pip or pip3. If roundup can't find the
1801 jwt module you will see the error ``Support for jwt disabled.`` 1863 jwt module you will see the error ``Support for jwt disabled.``
1802 2. create a new role that allows Create access to timelog and edit/view 1864 2. create a new role that allows Create access to timelog and edit/view
1803 access to an issues' ``times`` property. 1865 access to an issues' ``times`` property.
1804 3. add support for issuing (and validating) jwts to the rest interface. 1866 3. add support for issuing (and validating) JWTs to the rest interface.
1805 This uses the `Adding new rest endpoints`_ mechanism. 1867 This uses the `Adding new rest endpoints`_ mechanism.
1806 4. configure roundup's config.ini [web] jwt_secret with at least 32 1868 4. configure roundup's config.ini [web] jwt_secret with at least 32
1807 random characters of data. (You will get a message 1869 random characters of data. (You will get a message
1808 ``Support for jwt disabled by admin.`` if it's not long enough.) 1870 ``Support for jwt disabled by admin.`` if it's not long enough.)
1809 5. add an auditor to make sure that users with this role are appending 1871 5. add an auditor to make sure that users with this role are appending
1973 return 401, str(err) 2035 return 401, str(err)
1974 2036
1975 return 200, result 2037 return 200, result
1976 2038
1977 **Note this is sample code. Use at your own risk.** It breaks a few 2039 **Note this is sample code. Use at your own risk.** It breaks a few
1978 rules about jwts (e.g. it allows you to make unlimited lifetime 2040 rules about JWTs (e.g. it allows you to make unlimited lifetime
1979 jwts). If you subscribe to the concept of jwt refresh tokens, this code 2041 JWTs). If you subscribe to the concept of JWT refresh tokens, this code
1980 will have to be changed as it will only generate jwts with 2042 will have to be changed as it will only generate JWTs with
1981 username/password authentication. 2043 username/password authentication.
1982 2044
1983 Currently use of jwts an experiment. If this appeals to you consider 2045 Currently use of JWTs an experiment. If this appeals to you consider
1984 providing patches to existing code to: 2046 providing patches to existing code to:
1985 2047
1986 1. record all jwts created by a user 2048 1. record all JWTs created by a user
1987 2. using the record to allow jwts to be revoked and ignored by the 2049 2. using the record to allow JWTs to be revoked and ignored by the
1988 roundup core 2050 roundup core
1989 3. provide a UI page for managing/revoking jwts 2051 3. provide a UI page for managing/revoking JWTs
1990 4. provide a rest api for revoking jwts 2052 4. provide a rest api for revoking JWTs
1991 2053
1992 These end points can be used like:: 2054 These end points can be used like::
1993 2055
1994 curl -u demo -s -X POST -H "Referer: https://.../demo/" \ 2056 curl -u demo -s -X POST -H "Referer: https://.../demo/" \
1995 -H "X-requested-with: rest" \ 2057 -H "X-requested-with: rest" \
1996 -H "Content-Type: application/json" \ 2058 -H "Content-Type: application/json" \
1997 --data '{"lifetime": "3600", "roles": [ "user:timelog" ] }' \ 2059 --data '{"lifetime": "3600", "roles": [ "user:timelog" ] }' \
1998 https://.../demo/rest/jwt/issue 2060 https://.../demo/rest/JWT/issue
1999 2061
2000 (note roles is a json array/list of strings not a string) to get:: 2062 (note roles is a json array/list of strings not a string) to get::
2001 2063
2002 { 2064 {
2003 "data": { 2065 "data": {
2004 "jwt": "eyJ0eXAiOiJK......XxMDb-Q3oCnMpyhxPXMAk" 2066 "JWT": "eyJ0eXAiOiJK......XxMDb-Q3oCnMpyhxPXMAk"
2005 } 2067 }
2006 } 2068 }
2007 2069
2008 The jwt is shortened in the example since it's large. You can validate 2070 The JWT is shortened in the example since it's large. You can validate
2009 a jwt to see if it's still valid using:: 2071 a JWT to see if it's still valid using::
2010 2072
2011 2073
2012 curl -s -H "Referer: https://.../demo/" \ 2074 curl -s -H "Referer: https://.../demo/" \
2013 -H "X-requested-with: rest" \ 2075 -H "X-requested-with: rest" \
2014 https://.../demo/rest/jwt/validate?jwt=eyJ0eXAiOiJK...XxMDb-Q3oCnMpyhxPXMAk 2076 https://.../demo/rest/JWT/validate?JWT=eyJ0eXAiOiJK...XxMDb-Q3oCnMpyhxPXMAk
2015 2077
2016 (note no login is required) which returns:: 2078 (note no login is required) which returns::
2017 2079
2018 { 2080 {
2019 "data": { 2081 "data": {
2026 "iat": 1569542404, 2088 "iat": 1569542404,
2027 "exp": 1569546004 2089 "exp": 1569546004
2028 } 2090 }
2029 } 2091 }
2030 2092
2093
2094 There is an issue for `thoughts on JWT credentials`_ that you can view
2095 for ideas or add your own.
2096
2097 .. _thoughts on JWT credentials: https://issues.roundup-tracker.org/issue2551064
2098
2031 Final steps 2099 Final steps
2032 ^^^^^^^^^^^ 2100 ^^^^^^^^^^^
2033 2101
2034 See the `upgrading directions`_ on how to use the ``updateconfig`` 2102 See the `upgrading directions`_ on how to use the ``updateconfig``
2035 command to generate an updated copy of config.ini using 2103 command to generate an updated copy of config.ini using
2036 roundup-admin. Then set the ``jwt_secret`` to at least 32 characters 2104 roundup-admin. Then set the ``JWT_secret`` to at least 32 characters
2037 (more is better up to 512 bits). 2105 (more is better up to 512 bits).
2038 2106
2039 Writing an auditor that uses "db.user.get_roles" to see if the user 2107 Writing an auditor that uses "db.user.get_roles" to see if the user
2040 making the change has the ``user:timelog`` role, and then comparing 2108 making the change has the ``user:timelog`` role, and then comparing
2041 the original ``times`` list to the new list to verify that it is being 2109 the original ``times`` list to the new list to verify that it is being
2043 reader. (If you develop one, please contribute via the tracker: 2111 reader. (If you develop one, please contribute via the tracker:
2044 https://issues.roundup-tracker.org/.) 2112 https://issues.roundup-tracker.org/.)
2045 2113
2046 Lastly you can create a JWT using the end point above and make a rest 2114 Lastly you can create a JWT using the end point above and make a rest
2047 call to create a new timelog entry and another call to update the 2115 call to create a new timelog entry and another call to update the
2048 issues times property. If you have other ideas on how jwts can be 2116 issues times property. If you have other ideas on how JWTs can be
2049 used, please share on the roundup mailing lists. See: 2117 used, please share on the roundup mailing lists. See:
2050 https://sourceforge.net/p/roundup/mailman/ for directions on 2118 https://sourceforge.net/p/roundup/mailman/ for directions on
2051 subscribing and for archives of the lists. 2119 subscribing and for archives of the lists.
2052 2120
2053 2121
2054 Creating Custom Rate Limits 2122 Creating Custom Rate Limits
2055 =========================== 2123 ===========================
2056 2124
2057 You can replace the default rate limiter that is configured using 2125 You can replace the default rate limiter that is configured using
2058 the tracker's ``config.ini``. You can return different rate 2126 the tracker's ``config.ini``. You can return different rate
2059 limits based on the user, time of day, phase of moon etc. 2127 limits based on the user, time of day, phase of moon, request
2128 method (via self.client.request.command) etc.
2060 2129
2061 Assume you add two integer valued properties to the user 2130 Assume you add two integer valued properties to the user
2062 object. Let's call them ``rate_limit_interval`` and 2131 object. Let's call them ``rate_limit_interval`` and
2063 ``rate_limit_calls``. Add code similar to this to interfaces.py 2132 ``rate_limit_calls``. Add code similar to this to interfaces.py
2064 to override the default rate limiter code:: 2133 to override the default rate limiter code::
2097 Test Examples 2166 Test Examples
2098 ^^^^^^^^^^^^^ 2167 ^^^^^^^^^^^^^
2099 2168
2100 Rate limit tests:: 2169 Rate limit tests::
2101 2170
2102 seq 1 300 | xargs -P 20 -n 1 curl --head -si \ 2171 seq 1 300 | xargs -P 20 -n 1 curl --head -u user:password -si \
2103 https://.../rest/data/status/new \# | grep Remaining 2172 https://.../rest/data/status/new | grep Remaining
2173
2174 will show you the number of remaining requests to the REST interface
2175 for the user identified by password.

Roundup Issue Tracker: http://roundup-tracker.org/