Investigate novel datastore system test failures #1089
Description
-
502 on RPC
====================================================================== ERROR: test_allocate_ids (datastore.TestDatastoreAllocateIDs) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/travis/build/GoogleCloudPlatform/gcloud-python/system_tests/datastore.py", line 47, in test_allocate_ids allocated_keys = CLIENT.allocate_ids(CLIENT.key('Kind'), num_ids) File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests/lib/python2.7/site-packages/gcloud/datastore/client.py", line 418, in allocate_ids incomplete_key_pbs) File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 372, in allocate_ids datastore_pb.AllocateIdsResponse) File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 115, in _rpc data=request_pb.SerializeToString()) File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 92, in _request raise make_exception(headers, content, use_json=False) gcloud.exceptions.GCloudError: 502 <!DOCTYPE html> <html lang=en> <meta charset=utf-8> <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width"> <title>Error 502 (Server Error)!!1</title> <style> *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/errors/logo_sm_2.png) no-repeat}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/errors/logo_sm_2_hr.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/errors/logo_sm_2_hr.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/errors/logo_sm_2_hr.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:55px;width:150px} </style> <a href=//www.google.com/><span id=logo aria-label=Google></span></a> <p><b>502.</b> <ins>That’s an error.</ins> <p>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds. <ins>That’s all we know.</ins>
From
-
Query results eventually consistent (this shouldn't occur because we use entity groups)
From: https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/77772358
3. 503 Backend Error
======================================================================
ERROR: test_ancestor_query (datastore.TestDatastoreQuery)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/travis/build/GoogleCloudPlatform/gcloud-python/system_tests/datastore.py", line 40, in tearDown
CLIENT.delete_multi(keys)
File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests3/lib/python3.4/site-packages/gcloud/datastore/batch.py", line 228, in __exit__
self.commit()
File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests3/lib/python3.4/site-packages/gcloud/datastore/transaction.py", line 170, in commit
super(Transaction, self).commit()
File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests3/lib/python3.4/site-packages/gcloud/datastore/batch.py", line 203, in commit
self.dataset_id, self.mutation, self._id)
File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests3/lib/python3.4/site-packages/gcloud/datastore/connection.py", line 329, in commit
datastore_pb.CommitResponse)
File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests3/lib/python3.4/site-packages/gcloud/datastore/connection.py", line 113, in _rpc
data=request_pb.SerializeToString())
File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests3/lib/python3.4/site-packages/gcloud/datastore/connection.py", line 90, in _request
raise make_exception(headers, content, use_json=False)
gcloud.exceptions.ServiceUnavailable: 503 Backend ErrorFrom: https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/78109143
4. 500 Internal Error
ERROR: test_query_group_by (datastore.TestDatastoreQuery)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/travis/build/GoogleCloudPlatform/gcloud-python/system_tests/datastore.py", line 322, in test_query_group_by
entities = list(query.fetch(limit=expected_matches + 1))
File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests/lib/python2.7/site-packages/gcloud/datastore/query.py", line 444, in __iter__
self.next_page()
File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests/lib/python2.7/site-packages/gcloud/datastore/query.py", line 408, in next_page
transaction_id=transaction and transaction.id,
File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 266, in run_query
_datastore_pb2.RunQueryResponse)
File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 117, in _rpc
data=request_pb.SerializeToString())
File "/home/travis/build/GoogleCloudPlatform/gcloud-python/.tox/system-tests/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 94, in _request
raise make_exception(headers, content, use_json=False)
gcloud.exceptions.InternalServerError: 500 Internal ErrorFrom: https://travis-ci.org/GoogleCloudPlatform/gcloud-python/builds/98468887
This could be solved by reviving #535