Mercurial > p > roundup > code

diff test/rest_common.py @ 5710:0b79bfcb3312

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add support for making an idempotent POST. This allows retrying a POST that was interrupted. It involves creating a post once only (poe) url /rest/data/<class>/@poe/<random_token>. This url acts the same as a post to /rest/data/<class>. However once the @poe url is used, it can't be used for a second POST. To make these changes: 1) Take the body of post_collection into a new post_collection_inner function. Have post_collection call post_collection_inner. 2) Add a handler for POST to rest/data/class/@poe. This will return a unique POE url. By default the url expires after 30 minutes. The POE random token is only good for a specific user and is stored in the session db. 3) Add a handler for POST to rest/data/<class>/@poe/<random token>. The random token generated in 2 is validated for proper class (if token is not generic) and proper user and must not have expired. If everything is valid, call post_collection_inner to process the input and generate the new entry. To make recognition of 2 stable (so it's not confused with rest/data/<:class_name>/<:item_id>), removed @ from Routing::url_to_regex. The current Routing.execute method stops on the first regular expression to match the URL. Since item_id doesn't accept a POST, I was getting 405 bad method sometimes. My guess is the order of the regular expressions is not stable, so sometime I would get the right regexp for /data/<class>/@poe and sometime I would get the one for /data/<class>/<item_id>. By removing the @ from the url_to_regexp, there was no way for the item_id case to match @poe. There are alternate fixes we may need to look at. If a regexp matches but the method does not, return to the regexp matching loop in execute() looking for another match. Only once every possible match has failed should the code return a 405 method failure. Another fix is to implement a more sophisticated mechanism so that @Routing.route("/data/<:class_name>/<:item_id>/<:attr_name>", 'PATCH') has different regexps for matching <:class_name> <:item_id> and <:attr_name>. Currently the regexp specified by url_to_regex is used for every component. Other fixes: Made failure to find any props in props_from_args return an empty dict rather than throwing an unhandled error. Make __init__ for SimulateFieldStorageFromJson handle an empty json doc. Useful for POSTing to rest/data/class/@poe with an empty document. Testing: added testPostPOE to test/rest_common.py that I think covers all the code that was added. Documentation: Add doc to rest.txt in the "Client API" section titled: Safely Re-sending POST". Move existing section "Adding new rest endpoints" in "Client API" to a new second level section called "Programming the REST API". Also a minor change to the simple rest client moving the header setting to continuation lines rather than showing one long line.
author John Rouillard <rouilj@ieee.org>
date Sun, 14 Apr 2019 21:07:11 -0400
parents ad786c394788
children aea2cc142c1b
line wrap: on
line diff
--- a/test/rest_common.py	Sat Apr 13 13:53:24 2019 -0400
+++ b/test/rest_common.py	Sun Apr 14 21:07:11 2019 -0400
@@ -85,6 +85,8 @@
 
         self.server = RestfulInstance(self.dummy_client, self.db)
 
+        self.db.Otk = self.db.getOTKManager()
+
     def tearDown(self):
         self.db.close()
         try:
@@ -986,6 +988,238 @@
                          'foo bar')
         del(self.headers)
 
+    def testPostPOE(self):
+        ''' test post once exactly: get POE url, create issue
+            using POE url. Use dispatch entry point.
+        '''
+        import time
+        # setup environment
+        etag = "not needed"
+        body=b'{ "title": "foo bar", "priority": "critical" }'
+        env = { "CONTENT_TYPE": "application/json",
+                "CONTENT_LENGTH": len(body),
+                "REQUEST_METHOD": "POST"
+        }
+        headers={"accept": "application/json",
+                 "content-type": env['CONTENT_TYPE'],
+                 "content-length": len(body)
+        }
+        self.headers=headers
+        body_file=BytesIO(body)  # FieldStorage needs a file
+        form = client.BinaryFieldStorage(body_file,
+                                headers=headers,
+                                environ=env)
+
+        ## Obtain the POE url.
+        self.server.client.request.headers.get=self.get_header
+        results = self.server.dispatch('POST',
+                            "/rest/data/issue/@poe",
+                            form)
+
+        self.assertEqual(self.server.client.response_code, 200)
+        json_dict = json.loads(b2s(results))
+        url=json_dict['data']['link']
+
+        # strip tracker web prefix leaving leading /.
+        url = url[len(self.db.config['TRACKER_WEB'])-1:]
+
+        ## create an issue using poe url.
+        self.server.client.request.headers.get=self.get_header
+        results = self.server.dispatch('POST',
+                            url,
+                            form)
+
+        self.assertEqual(self.server.client.response_code, 201)
+        json_dict = json.loads(b2s(results))
+        issue_id=json_dict['data']['id']
+        results = self.server.get_element('issue',
+                            str(issue_id), # must be a string not unicode
+                            self.empty_form)
+        self.assertEqual(self.dummy_client.response_code, 200)
+        self.assertEqual(results['data']['attributes']['title'],
+                         'foo bar')
+
+        ## Reuse POE url. It will fail.
+        self.server.client.request.headers.get=self.get_header
+        results = self.server.dispatch('POST',
+                            url,
+                            form)
+        # get the last component stripping the trailing /
+        poe=url[url.rindex('/')+1:]
+        self.assertEqual(self.server.client.response_code, 400)
+        results = json.loads(b2s(results))
+        self.assertEqual(results['error']['status'], 400)
+        self.assertEqual(results['error']['msg'],
+                         "POE token \'%s\' not valid."%poe)
+
+        ## Try using GET on POE url. Should fail with method not
+        ## allowed (405)
+        self.server.client.request.headers.get=self.get_header
+        results = self.server.dispatch('GET',
+                            "/rest/data/issue/@poe",
+                            form)
+        self.assertEqual(self.server.client.response_code, 405)
+
+
+        ## Try creating generic POE url.
+        body_poe=b'{"generic": "null", "lifetime": "100" }'
+        body_file=BytesIO(body_poe)  # FieldStorage needs a file
+        form = client.BinaryFieldStorage(body_file,
+                                headers=headers,
+                                environ=env)
+        self.server.client.request.headers.get=self.get_header
+        results = self.server.dispatch('POST',
+                            "/rest/data/issue/@poe",
+                            form)
+        json_dict = json.loads(b2s(results))
+        url=json_dict['data']['link']
+
+        # strip tracker web prefix leaving leading /.
+        url = url[len(self.db.config['TRACKER_WEB'])-1:]
+        url = url.replace('/issue/', '/keyword/')
+
+        body_keyword=b'{"name": "keyword"}'
+        body_file=BytesIO(body_keyword)  # FieldStorage needs a file
+        form = client.BinaryFieldStorage(body_file,
+                                headers=headers,
+                                environ=env)
+        results = self.server.dispatch('POST',
+                            url,
+                            form)
+        self.assertEqual(self.server.client.response_code, 201)
+        json_dict = json.loads(b2s(results))
+        url=json_dict['data']['link']
+        id=json_dict['data']['id']
+        self.assertEqual(id, "1")
+        self.assertEqual(url, "http://tracker.example/cgi-bin/roundup.cgi/bugs/rest/data/keyword/1")
+
+        ## Create issue POE url and try to use for keyword.
+        ## This should fail.
+        body_poe=b'{"lifetime": "100" }'
+        body_file=BytesIO(body_poe)  # FieldStorage needs a file
+        form = client.BinaryFieldStorage(body_file,
+                                headers=headers,
+                                environ=env)
+        self.server.client.request.headers.get=self.get_header
+        results = self.server.dispatch('POST',
+                            "/rest/data/issue/@poe",
+                            form)
+        json_dict = json.loads(b2s(results))
+        url=json_dict['data']['link']
+
+        # strip tracker web prefix leaving leading /.
+        url = url[len(self.db.config['TRACKER_WEB'])-1:]
+        url = url.replace('/issue/', '/keyword/')
+
+        body_keyword=b'{"name": "keyword"}'
+        body_file=BytesIO(body_keyword)  # FieldStorage needs a file
+        form = client.BinaryFieldStorage(body_file,
+                                headers=headers,
+                                environ=env)
+        results = self.server.dispatch('POST',
+                            url,
+                            form)
+        poe=url[url.rindex('/')+1:]
+        self.assertEqual(self.server.client.response_code, 400)
+        json_dict = json.loads(b2s(results))
+        stat=json_dict['error']['status']
+        msg=json_dict['error']['msg']
+        self.assertEqual(stat, 400)
+        self.assertEqual(msg, "POE token '%s' not valid for keyword, was generated for class issue"%poe)
+
+
+        ## Create POE with 10 minute lifetime and verify
+        ## expires is within 10 minutes.
+        body_poe=b'{"lifetime": "30" }'
+        body_file=BytesIO(body_poe)  # FieldStorage needs a file
+        form = client.BinaryFieldStorage(body_file,
+                                headers=headers,
+                                environ=env)
+        self.server.client.request.headers.get=self.get_header
+        results = self.server.dispatch('POST',
+                            "/rest/data/issue/@poe",
+                            form)
+        json_dict = json.loads(b2s(results))
+        expires=int(json_dict['data']['expires'])
+        # allow up to 3 seconds between time stamp creation
+        # done under dispatch and this point.
+        expected=int(time.time() + 30)
+        print("expected=%d, expires=%d"%(expected,expires))
+        self.assertTrue((expected - expires) < 3 and (expected - expires) >= 0)
+
+
+        ## Use a token created above as joe by a different user.
+        self.db.setCurrentUser('admin')
+        url=json_dict['data']['link']
+        # strip tracker web prefix leaving leading /.
+        url = url[len(self.db.config['TRACKER_WEB'])-1:]
+        body_file=BytesIO(body_keyword)  # FieldStorage needs a file
+        form = client.BinaryFieldStorage(body_file,
+                                         headers=headers,
+                                         environ=env)
+        results = self.server.dispatch('POST',
+                                       url,
+                                       form)
+        print(results)
+        self.assertEqual(self.server.client.response_code, 400)
+        json_dict = json.loads(b2s(results))
+        # get the last component stripping the trailing /
+        poe=url[url.rindex('/')+1:]
+        self.assertEqual(json_dict['error']['msg'],
+                         "POE token '%s' not valid."%poe)
+
+        ## Create POE with bogus lifetime
+        body_poe=b'{"lifetime": "10.2" }'
+        body_file=BytesIO(body_poe)  # FieldStorage needs a file
+        form = client.BinaryFieldStorage(body_file,
+                                headers=headers,
+                                environ=env)
+        self.server.client.request.headers.get=self.get_header
+        results = self.server.dispatch('POST',
+                            "/rest/data/issue/@poe",
+                            form)
+        self.assertEqual(self.server.client.response_code, 400)
+        print(results)
+        json_dict = json.loads(b2s(results))
+        self.assertEqual(json_dict['error']['msg'],
+                         "Value \'lifetime\' must be an integer specify "
+                         "lifetime in seconds. Got 10.2.")
+
+        ## Create POE with lifetime > 1 hour
+        body_poe=b'{"lifetime": "3700" }'
+        body_file=BytesIO(body_poe)  # FieldStorage needs a file
+        form = client.BinaryFieldStorage(body_file,
+                                headers=headers,
+                                environ=env)
+        self.server.client.request.headers.get=self.get_header
+        results = self.server.dispatch('POST',
+                            "/rest/data/issue/@poe",
+                            form)
+        self.assertEqual(self.server.client.response_code, 400)
+        print(results)
+        json_dict = json.loads(b2s(results))
+        self.assertEqual(json_dict['error']['msg'],
+                         "Value 'lifetime' must be between 1 second and 1 "
+                         "hour (3600 seconds). Got 3700.")
+
+        ## Create POE with lifetime < 1 second
+        body_poe=b'{"lifetime": "-1" }'
+        body_file=BytesIO(body_poe)  # FieldStorage needs a file
+        form = client.BinaryFieldStorage(body_file,
+                                headers=headers,
+                                environ=env)
+        self.server.client.request.headers.get=self.get_header
+        results = self.server.dispatch('POST',
+                            "/rest/data/issue/@poe",
+                            form)
+        self.assertEqual(self.server.client.response_code, 400)
+        print(results)
+        json_dict = json.loads(b2s(results))
+        self.assertEqual(json_dict['error']['msg'],
+                         "Value 'lifetime' must be between 1 second and 1 "
+                         "hour (3600 seconds). Got -1.")
+        del(self.headers)
+
     def testPutElement(self):
         """
         Change joe's 'realname'
Roundup Issue Tracker: http://roundup-tracker.org/