Mercurial > p > roundup > code
comparison doc/rest.txt @ 7045:ca90f7270cd4 issue2550923_computed_property
merge from main tip. This should fix test failure in Markdown2TestCase.test_string_markdown_code_block_attribute by merging html diff method used in tests.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Mon, 07 Nov 2022 22:58:38 -0500 |
| parents | e7b4ad2c57ac |
| children | 42e68162279b |
comparison
equal
deleted
inserted
replaced
| 6638:e1588ae185dc | 7045:ca90f7270cd4 |
|---|---|
| 1 .. meta:: | 1 .. meta:: |
| 2 :description language=en: | 2 :description: |
| 3 Documentation on the RESTful interface to the Roundup Issue | 3 Documentation on the RESTful interface to the Roundup Issue |
| 4 Tracker. | 4 Tracker. |
| 5 | 5 |
| 6 | 6 |
| 7 .. index:: pair: api; Representational state transfer | 7 .. index:: pair: api; Representational state transfer |
| 69 ======================= | 69 ======================= |
| 70 | 70 |
| 71 Clients should set the header X-REQUESTED-WITH to any value and the | 71 Clients should set the header X-REQUESTED-WITH to any value and the |
| 72 tracker's config.ini should have ``csrf_enforce_header_x-requested-with | 72 tracker's config.ini should have ``csrf_enforce_header_x-requested-with |
| 73 = yes`` or ``required``. | 73 = yes`` or ``required``. |
| 74 | |
| 75 If you want to allow Roundup's api to be accessed by an application | |
| 76 that is not hosted at the same origin as Roundup, you must permit | |
| 77 the origin using the ``allowed_api_origins`` setting in | |
| 78 ``config.ini``. | |
| 74 | 79 |
| 75 Rate Limiting the API | 80 Rate Limiting the API |
| 76 ===================== | 81 ===================== |
| 77 | 82 |
| 78 This is a work in progress. This version of roundup includes Rate | 83 This is a work in progress. This version of roundup includes Rate |
| 201 | 206 |
| 202 Otherwise Content-Type is allowed to be ``application/json`` or | 207 Otherwise Content-Type is allowed to be ``application/json`` or |
| 203 ``application/x-www-form-urlencoded``. Any other value returns error | 208 ``application/x-www-form-urlencoded``. Any other value returns error |
| 204 code 415. | 209 code 415. |
| 205 | 210 |
| 211 CORS preflight requests | |
| 212 ^^^^^^^^^^^^^^^^^^^^^^^ | |
| 213 | |
| 214 CORS preflight requests are done using the OPTIONS method. They | |
| 215 require that REST be enabled. These requests do not make any changes | |
| 216 or get any information from the database. As a result they are | |
| 217 available to the anonymous user and any authenticated user. The user | |
| 218 does not need to have `Rest Access` permissions. Also these requests | |
| 219 bypass CSRF checks except for the Origin header check which is always | |
| 220 run for preflight requests. | |
| 221 | |
| 222 You can permit only allowed ORIGINS by setting ``allowed_api_origins`` | |
| 223 in ``config.ini`` to the list of origins permitted to access your | |
| 224 api. By default only your tracker's origin is allowed. If a preflight | |
| 225 request fails, the api request will be stopped by the browser. | |
| 226 | |
| 227 The following CORS preflight headers are usually added automatically by | |
| 228 the browser and must all be present: | |
| 229 | |
| 230 * `Access-Control-Request-Headers` | |
| 231 * `Access-Control-Request-Method` | |
| 232 * `Origin` | |
| 233 | |
| 234 The 204 response will include the headers: | |
| 235 | |
| 236 * `Access-Control-Allow-Origin` | |
| 237 * `Access-Control-Allow-Headers` | |
| 238 * `Access-Control-Allow-Methods` | |
| 239 * `Access-Control-Allow-Credentials: true` | |
| 240 * `Access-Control-Max-Age: 86400` | |
| 241 | |
| 242 If the endpoint accepts the PATCH verb the header `Accept-Patch` with | |
| 243 valid mime types (usually `application/x-www-form-urlencoded, | |
| 244 multipart/form-data`) will be included. | |
| 245 | |
| 246 It will also include rate limit headers since the request is included | |
| 247 in the rate limit for the URL. The results from the CORS preflight | |
| 248 should be cached for a day so preflight requests are not expected to | |
| 249 cause a problem. If it is an issue, you can see | |
| 250 `Creating Custom Rate Limits`_ | |
| 251 and craft a rate limiter that ignores anonymous OPTIONS requests. | |
| 252 | |
| 206 Response Formats | 253 Response Formats |
| 207 ================ | 254 ================ |
| 208 | 255 |
| 209 The default response format is json. | 256 The default response format is json. |
| 210 | 257 |
| 211 If you add the ``dicttoxml.py`` module you can request XML formatted | 258 If you add the ``dicttoxml.py`` module you can request XML formatted |
| 212 data using the header ``Accept: application/xml`` in your | 259 data using the header ``Accept: application/xml`` in your |
| 213 request. Both output formats are similar in structure. | 260 request. Both output formats are similar in structure. |
| 261 | |
| 262 ``dicttoxml.py`` should be installed in the Python install directory, | |
| 263 or the file can be added to the Roundup installation directory long | |
| 264 ide ``rest.py``. It can also be enabled on a per tracker basis by | |
| 265 adding ``dicttoxml.py`` to the lib directory in your tracker home (you | |
| 266 may need to create the directory). Then this can be added to | |
| 267 `interfaces.py`_ to enable xml:: | |
| 268 | |
| 269 from roundup import rest | |
| 270 from dicttoxml import dicttoxml as dtox # from tracker_root/lib directory | |
| 271 | |
| 272 rest.dicttoxml = dtox | |
| 273 | |
| 274 .. _interfaces.py: customizing.html#interfaces-py-hooking-into-the-core-of-roundup | |
| 214 | 275 |
| 215 The rest interface accepts the http accept header and can include | 276 The rest interface accepts the http accept header and can include |
| 216 ``q`` values to specify the preferred mechanism. This is the preferred | 277 ``q`` values to specify the preferred mechanism. This is the preferred |
| 217 way to specify alternate acceptable response formats. | 278 way to specify alternate acceptable response formats. |
| 218 | 279 |
| 421 - Link | 482 - Link |
| 422 - find any issue whose status link is set to the id 2. | 483 - find any issue whose status link is set to the id 2. |
| 423 * - ``status=open`` | 484 * - ``status=open`` |
| 424 - Link | 485 - Link |
| 425 - find any issue where the name of the status is open. | 486 - find any issue where the name of the status is open. |
| 426 Note this is not a string match so using nosy=ope will fail. | 487 Note this is not a string match so using status=ope will fail. |
| 427 * - ``nosy=1`` | 488 * - ``nosy=1`` |
| 428 - MultiLink | 489 - MultiLink |
| 429 - find any issue where the multilink nosy includes the id 1. | 490 - find any issue where the multilink nosy includes the id 1. |
| 430 * - ``nosy=admin`` | 491 * - ``nosy=admin`` |
| 431 - MultiLink | 492 - MultiLink |
| 641 | 702 |
| 642 { | 703 { |
| 643 "data": { | 704 "data": { |
| 644 "collection": [ | 705 "collection": [ |
| 645 { | 706 { |
| 646 "link": | 707 "link": "https://.../rest/data/issue/1", |
| 647 "https://.../rest/data/issue/1", | |
| 648 "id": "1", | 708 "id": "1", |
| 649 "status": { | 709 "status": { |
| 650 "link": | 710 "link": "https://.../rest/data/status/1", |
| 651 "https://.../rest/data/status/1", | |
| 652 "id": "1" | 711 "id": "1" |
| 653 } | 712 } |
| 654 }, | 713 }, |
| 655 ... | 714 ... |
| 656 } | 715 } |
| 1100 "1", | 1159 "1", |
| 1101 "5" | 1160 "5" |
| 1102 ] | 1161 ] |
| 1103 }, | 1162 }, |
| 1104 "type": "issue", | 1163 "type": "issue", |
| 1105 "link": | 1164 "link": "https://example.com/demo/rest/data/issue/23", |
| 1106 "https://example.com/demo/rest/data/issue/23", | |
| 1107 "id": "23" | 1165 "id": "23" |
| 1108 } | 1166 } |
| 1109 } | 1167 } |
| 1110 | 1168 |
| 1111 If the above command is repeated with the data attribute:: | 1169 If the above command is repeated with the data attribute:: |
| 1264 { | 1322 { |
| 1265 "data": { | 1323 "data": { |
| 1266 "id": "11", | 1324 "id": "11", |
| 1267 "type": "<class 'str'>", | 1325 "type": "<class 'str'>", |
| 1268 "link": "https://.../demo/rest/data/msg/11/content", | 1326 "link": "https://.../demo/rest/data/msg/11/content", |
| 1269 "data": "of has to who pleasure. or of account give because the reprehenderit\neu to quisquam velit, passage, was or...", | 1327 "data": "of has to who pleasure. or of account give because the |
| 1328 reprehenderit\neu to quisquam velit, passage, was or...", | |
| 1270 "@etag": "\"584f82231079e349031bbb853747df1c\"" | 1329 "@etag": "\"584f82231079e349031bbb853747df1c\"" |
| 1271 } | 1330 } |
| 1272 } | 1331 } |
| 1273 | 1332 |
| 1274 (the content property is wrapped for display, it is one long line.) | 1333 (the content property is wrapped for display, it is one long line.) |
| 1405 >>> print(r.json()) | 1464 >>> print(r.json()) |
| 1406 | 1465 |
| 1407 Note the addition of headers for: x-requested-with and referer. This | 1466 Note the addition of headers for: x-requested-with and referer. This |
| 1408 allows the request to pass the CSRF protection mechanism. You may need | 1467 allows the request to pass the CSRF protection mechanism. You may need |
| 1409 to add an Origin header if this check is enabled in your tracker's | 1468 to add an Origin header if this check is enabled in your tracker's |
| 1410 config.ini (look for csrf_enforce_header_origin). | 1469 config.ini (look for csrf_enforce_header_origin). (Note the Origin |
| 1470 header check may have to be disabled if an application is making a | |
| 1471 CORS request to the Roundup server. If you have this issue, please | |
| 1472 contact the Roundup team using the mailing lists as this is a bug.) | |
| 1411 | 1473 |
| 1412 A similar curl based retire example is to use:: | 1474 A similar curl based retire example is to use:: |
| 1413 | 1475 |
| 1414 curl -s -u admin:admin \ | 1476 curl -s -u admin:admin \ |
| 1415 -H "Referer: https://tracker.example.com/demo/" \ | 1477 -H "Referer: https://tracker.example.com/demo/" \ |
| 1544 structure and implementation. | 1606 structure and implementation. |
| 1545 | 1607 |
| 1546 Adding new rest endpoints | 1608 Adding new rest endpoints |
| 1547 ========================= | 1609 ========================= |
| 1548 | 1610 |
| 1549 Add or edit the file interfaces.py at the root of the tracker | 1611 Add or edit the file `interfaces.py`_ at the root of the tracker |
| 1550 directory. | 1612 directory. |
| 1551 | 1613 |
| 1552 In that file add:: | 1614 In that file add:: |
| 1553 | 1615 |
| 1554 from roundup.rest import Routing, RestfulInstance, _data_decorator | 1616 from roundup.rest import Routing, RestfulInstance, _data_decorator |
| 1799 | 1861 |
| 1800 1. install pyjwt library using pip or pip3. If roundup can't find the | 1862 1. install pyjwt library using pip or pip3. If roundup can't find the |
| 1801 jwt module you will see the error ``Support for jwt disabled.`` | 1863 jwt module you will see the error ``Support for jwt disabled.`` |
| 1802 2. create a new role that allows Create access to timelog and edit/view | 1864 2. create a new role that allows Create access to timelog and edit/view |
| 1803 access to an issues' ``times`` property. | 1865 access to an issues' ``times`` property. |
| 1804 3. add support for issuing (and validating) jwts to the rest interface. | 1866 3. add support for issuing (and validating) JWTs to the rest interface. |
| 1805 This uses the `Adding new rest endpoints`_ mechanism. | 1867 This uses the `Adding new rest endpoints`_ mechanism. |
| 1806 4. configure roundup's config.ini [web] jwt_secret with at least 32 | 1868 4. configure roundup's config.ini [web] jwt_secret with at least 32 |
| 1807 random characters of data. (You will get a message | 1869 random characters of data. (You will get a message |
| 1808 ``Support for jwt disabled by admin.`` if it's not long enough.) | 1870 ``Support for jwt disabled by admin.`` if it's not long enough.) |
| 1809 5. add an auditor to make sure that users with this role are appending | 1871 5. add an auditor to make sure that users with this role are appending |
| 1973 return 401, str(err) | 2035 return 401, str(err) |
| 1974 | 2036 |
| 1975 return 200, result | 2037 return 200, result |
| 1976 | 2038 |
| 1977 **Note this is sample code. Use at your own risk.** It breaks a few | 2039 **Note this is sample code. Use at your own risk.** It breaks a few |
| 1978 rules about jwts (e.g. it allows you to make unlimited lifetime | 2040 rules about JWTs (e.g. it allows you to make unlimited lifetime |
| 1979 jwts). If you subscribe to the concept of jwt refresh tokens, this code | 2041 JWTs). If you subscribe to the concept of JWT refresh tokens, this code |
| 1980 will have to be changed as it will only generate jwts with | 2042 will have to be changed as it will only generate JWTs with |
| 1981 username/password authentication. | 2043 username/password authentication. |
| 1982 | 2044 |
| 1983 Currently use of jwts an experiment. If this appeals to you consider | 2045 Currently use of JWTs an experiment. If this appeals to you consider |
| 1984 providing patches to existing code to: | 2046 providing patches to existing code to: |
| 1985 | 2047 |
| 1986 1. record all jwts created by a user | 2048 1. record all JWTs created by a user |
| 1987 2. using the record to allow jwts to be revoked and ignored by the | 2049 2. using the record to allow JWTs to be revoked and ignored by the |
| 1988 roundup core | 2050 roundup core |
| 1989 3. provide a UI page for managing/revoking jwts | 2051 3. provide a UI page for managing/revoking JWTs |
| 1990 4. provide a rest api for revoking jwts | 2052 4. provide a rest api for revoking JWTs |
| 1991 | 2053 |
| 1992 These end points can be used like:: | 2054 These end points can be used like:: |
| 1993 | 2055 |
| 1994 curl -u demo -s -X POST -H "Referer: https://.../demo/" \ | 2056 curl -u demo -s -X POST -H "Referer: https://.../demo/" \ |
| 1995 -H "X-requested-with: rest" \ | 2057 -H "X-requested-with: rest" \ |
| 1996 -H "Content-Type: application/json" \ | 2058 -H "Content-Type: application/json" \ |
| 1997 --data '{"lifetime": "3600", "roles": [ "user:timelog" ] }' \ | 2059 --data '{"lifetime": "3600", "roles": [ "user:timelog" ] }' \ |
| 1998 https://.../demo/rest/jwt/issue | 2060 https://.../demo/rest/JWT/issue |
| 1999 | 2061 |
| 2000 (note roles is a json array/list of strings not a string) to get:: | 2062 (note roles is a json array/list of strings not a string) to get:: |
| 2001 | 2063 |
| 2002 { | 2064 { |
| 2003 "data": { | 2065 "data": { |
| 2004 "jwt": "eyJ0eXAiOiJK......XxMDb-Q3oCnMpyhxPXMAk" | 2066 "JWT": "eyJ0eXAiOiJK......XxMDb-Q3oCnMpyhxPXMAk" |
| 2005 } | 2067 } |
| 2006 } | 2068 } |
| 2007 | 2069 |
| 2008 The jwt is shortened in the example since it's large. You can validate | 2070 The JWT is shortened in the example since it's large. You can validate |
| 2009 a jwt to see if it's still valid using:: | 2071 a JWT to see if it's still valid using:: |
| 2010 | 2072 |
| 2011 | 2073 |
| 2012 curl -s -H "Referer: https://.../demo/" \ | 2074 curl -s -H "Referer: https://.../demo/" \ |
| 2013 -H "X-requested-with: rest" \ | 2075 -H "X-requested-with: rest" \ |
| 2014 https://.../demo/rest/jwt/validate?jwt=eyJ0eXAiOiJK...XxMDb-Q3oCnMpyhxPXMAk | 2076 https://.../demo/rest/JWT/validate?JWT=eyJ0eXAiOiJK...XxMDb-Q3oCnMpyhxPXMAk |
| 2015 | 2077 |
| 2016 (note no login is required) which returns:: | 2078 (note no login is required) which returns:: |
| 2017 | 2079 |
| 2018 { | 2080 { |
| 2019 "data": { | 2081 "data": { |
| 2026 "iat": 1569542404, | 2088 "iat": 1569542404, |
| 2027 "exp": 1569546004 | 2089 "exp": 1569546004 |
| 2028 } | 2090 } |
| 2029 } | 2091 } |
| 2030 | 2092 |
| 2093 | |
| 2094 There is an issue for `thoughts on JWT credentials`_ that you can view | |
| 2095 for ideas or add your own. | |
| 2096 | |
| 2097 .. _thoughts on JWT credentials: https://issues.roundup-tracker.org/issue2551064 | |
| 2098 | |
| 2031 Final steps | 2099 Final steps |
| 2032 ^^^^^^^^^^^ | 2100 ^^^^^^^^^^^ |
| 2033 | 2101 |
| 2034 See the `upgrading directions`_ on how to use the ``updateconfig`` | 2102 See the `upgrading directions`_ on how to use the ``updateconfig`` |
| 2035 command to generate an updated copy of config.ini using | 2103 command to generate an updated copy of config.ini using |
| 2036 roundup-admin. Then set the ``jwt_secret`` to at least 32 characters | 2104 roundup-admin. Then set the ``JWT_secret`` to at least 32 characters |
| 2037 (more is better up to 512 bits). | 2105 (more is better up to 512 bits). |
| 2038 | 2106 |
| 2039 Writing an auditor that uses "db.user.get_roles" to see if the user | 2107 Writing an auditor that uses "db.user.get_roles" to see if the user |
| 2040 making the change has the ``user:timelog`` role, and then comparing | 2108 making the change has the ``user:timelog`` role, and then comparing |
| 2041 the original ``times`` list to the new list to verify that it is being | 2109 the original ``times`` list to the new list to verify that it is being |
| 2043 reader. (If you develop one, please contribute via the tracker: | 2111 reader. (If you develop one, please contribute via the tracker: |
| 2044 https://issues.roundup-tracker.org/.) | 2112 https://issues.roundup-tracker.org/.) |
| 2045 | 2113 |
| 2046 Lastly you can create a JWT using the end point above and make a rest | 2114 Lastly you can create a JWT using the end point above and make a rest |
| 2047 call to create a new timelog entry and another call to update the | 2115 call to create a new timelog entry and another call to update the |
| 2048 issues times property. If you have other ideas on how jwts can be | 2116 issues times property. If you have other ideas on how JWTs can be |
| 2049 used, please share on the roundup mailing lists. See: | 2117 used, please share on the roundup mailing lists. See: |
| 2050 https://sourceforge.net/p/roundup/mailman/ for directions on | 2118 https://sourceforge.net/p/roundup/mailman/ for directions on |
| 2051 subscribing and for archives of the lists. | 2119 subscribing and for archives of the lists. |
| 2052 | 2120 |
| 2053 | 2121 |
| 2054 Creating Custom Rate Limits | 2122 Creating Custom Rate Limits |
| 2055 =========================== | 2123 =========================== |
| 2056 | 2124 |
| 2057 You can replace the default rate limiter that is configured using | 2125 You can replace the default rate limiter that is configured using |
| 2058 the tracker's ``config.ini``. You can return different rate | 2126 the tracker's ``config.ini``. You can return different rate |
| 2059 limits based on the user, time of day, phase of moon etc. | 2127 limits based on the user, time of day, phase of moon, request |
| 2128 method (via self.client.request.command) etc. | |
| 2060 | 2129 |
| 2061 Assume you add two integer valued properties to the user | 2130 Assume you add two integer valued properties to the user |
| 2062 object. Let's call them ``rate_limit_interval`` and | 2131 object. Let's call them ``rate_limit_interval`` and |
| 2063 ``rate_limit_calls``. Add code similar to this to interfaces.py | 2132 ``rate_limit_calls``. Add code similar to this to interfaces.py |
| 2064 to override the default rate limiter code:: | 2133 to override the default rate limiter code:: |
| 2097 Test Examples | 2166 Test Examples |
| 2098 ^^^^^^^^^^^^^ | 2167 ^^^^^^^^^^^^^ |
| 2099 | 2168 |
| 2100 Rate limit tests:: | 2169 Rate limit tests:: |
| 2101 | 2170 |
| 2102 seq 1 300 | xargs -P 20 -n 1 curl --head -si \ | 2171 seq 1 300 | xargs -P 20 -n 1 curl --head -u user:password -si \ |
| 2103 https://.../rest/data/status/new \# | grep Remaining | 2172 https://.../rest/data/status/new | grep Remaining |
| 2173 | |
| 2174 will show you the number of remaining requests to the REST interface | |
| 2175 for the user identified by password. |
