Attempt to un-break Appveyor: #1965

tseaver · 2016-07-08T18:22:55Z

Put _testing on PYTHONPATH only for Py3k (debug grpico-on-Py3k issues later).
Use setup.py develop rather than setup.py build to get dependencies installed.
Run pip list afterwards to show what got installed.

Toward #1863.

- Put '_testing' on PYTHONPATH only for Py3k (debug grpico-on-Py3k issues later). - Use 'setup.py develop' rather than 'setup.py build' to get dependencies installed. - Run 'pip list' afterwards to show what got installed. Toward #1863.

tseaver · 2016-07-08T18:31:46Z

Now we see if it works.

daspecster · 2016-07-08T18:46:45Z

@tseaver, I get those timestamp_pb2 errors locally on my machine with a fresh venv and doing a python setup.py install.

I have to import them first before I make a client...really weird. I thought it was just something with my machine.

>>> from gcloud import storage
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "gcloud/storage/__init__.py", line 43, in <module>
    from gcloud.storage.blob import Blob
  File "gcloud/storage/blob.py", line 30, in <module>
    from gcloud._helpers import _rfc3339_to_datetime
  File "gcloud/_helpers.py", line 28, in <module>
    from google.protobuf import timestamp_pb2
ImportError: cannot import name timestamp_pb2

If I import timestamp_pb2 before I import gcloud.storage it works...

pip install protobuf==3.0.0b2

>>> from google.protobuf import timestamp_pb2
>>> from gcloud import storage
>>>

tseaver · 2016-07-08T18:54:49Z

Hmmm. Installing from PyPI works:

$ /tmp/fscked/bin/pip install gcloud
Collecting gcloud
  Downloading gcloud-0.17.0.tar.gz (458kB)
    100% |████████████████████████████████| 460kB 1.5MB/s 
Collecting httplib2>=0.9.1 (from gcloud)
Collecting googleapis-common-protos (from gcloud)
Collecting oauth2client>=2.0.1 (from gcloud)
Collecting protobuf!=3.0.0.b2.post1,>=3.0.0b2 (from gcloud)
  Using cached protobuf-3.0.0b3-py2-none-any.whl
Collecting six (from gcloud)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting grpcio>=0.14.0 (from gcloud)
  Using cached grpcio-0.15.0-cp27-cp27m-manylinux1_x86_64.whl
Collecting google-gax>=0.12.1 (from gcloud)
Collecting gax-google-pubsub-v1>=0.7.10 (from gcloud)
Collecting gax-google-logging-v2>=0.7.10 (from gcloud)
Collecting rsa>=3.1.4 (from oauth2client>=2.0.1->gcloud)
  Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client>=2.0.1->gcloud)
  Using cached pyasn1_modules-0.0.8-py2.py3-none-any.whl
Collecting pyasn1>=0.1.7 (from oauth2client>=2.0.1->gcloud)
  Using cached pyasn1-0.1.9-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools in /tmp/fscked/lib/python2.7/site-packages (from protobuf!=3.0.0.b2.post1,>=3.0.0b2->gcloud)
Collecting enum34>=1.0.4 (from grpcio>=0.14.0->gcloud)
  Using cached enum34-1.1.6-py2-none-any.whl
Collecting futures>=2.2.0 (from grpcio>=0.14.0->gcloud)
  Using cached futures-3.0.5-py2-none-any.whl
Collecting ply==3.8 (from google-gax>=0.12.1->gcloud)
Collecting grpc-google-pubsub-v1>=0.7.11 (from gax-google-pubsub-v1>=0.7.10->gcloud)
Collecting grpc-google-logging-v2>=0.7.11 (from gax-google-logging-v2>=0.7.10->gcloud)
Building wheels for collected packages: gcloud
  Running setup.py bdist_wheel for gcloud ... done
  Stored in directory: /.../.cache/pip/wheels/d3/00/37/a3194dbb53f305c998b78c6d6b74af80dff545207a7909f45a
Successfully built gcloud
Installing collected packages: httplib2, six, protobuf, googleapis-common-protos, pyasn1, rsa, pyasn1-modules, oauth2client, enum34, futures, grpcio, ply, google-gax, grpc-google-pubsub-v1, gax-google-pubsub-v1, grpc-google-logging-v2, gax-google-logging-v2, gcloud
Successfully installed enum34-1.1.6 futures-3.0.5 gax-google-logging-v2-0.7.11 gax-google-pubsub-v1-0.7.11 gcloud-0.17.0 google-gax-0.12.2 googleapis-common-protos-1.2.0 grpc-google-logging-v2-0.7.11 grpc-google-pubsub-v1-0.7.11 grpcio-0.15.0 httplib2-0.9.2 oauth2client-2.2.0 ply-3.8 protobuf-3.0.0b3 pyasn1-0.1.9 pyasn1-modules-0.0.8 rsa-3.4.2 six-1.10.0

As does installing from Github with pip:

$ /opt/Python-2.7.11/bin/virtualenv /tmp/borked
New python executable in /tmp/borked/bin/python
Installing setuptools, pip, wheel...done.
$ /tmp/borked/bin/pip install git+https://github.com/GoogleCloudPlatform/gcloud-python.git
Collecting git+https://github.com/GoogleCloudPlatform/gcloud-python.git
  Cloning https://github.com/GoogleCloudPlatform/gcloud-python.git to /tmp/pip-rhbBzC-build
Collecting httplib2>=0.9.1 (from gcloud==0.17.0)
Collecting googleapis-common-protos (from gcloud==0.17.0)
Collecting oauth2client>=2.0.1 (from gcloud==0.17.0)
Collecting protobuf!=3.0.0.b2.post1,>=3.0.0b2 (from gcloud==0.17.0)
  Using cached protobuf-3.0.0b3-py2-none-any.whl
Collecting six (from gcloud==0.17.0)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting grpcio>=0.14.0 (from gcloud==0.17.0)
  Using cached grpcio-0.15.0-cp27-cp27m-manylinux1_x86_64.whl
Collecting google-gax>=0.12.1 (from gcloud==0.17.0)
Collecting gax-google-pubsub-v1>=0.7.10 (from gcloud==0.17.0)
Collecting gax-google-logging-v2>=0.7.10 (from gcloud==0.17.0)
Collecting rsa>=3.1.4 (from oauth2client>=2.0.1->gcloud==0.17.0)
  Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client>=2.0.1->gcloud==0.17.0)
  Using cached pyasn1_modules-0.0.8-py2.py3-none-any.whl
Collecting pyasn1>=0.1.7 (from oauth2client>=2.0.1->gcloud==0.17.0)
  Using cached pyasn1-0.1.9-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools in /tmp/borked/lib/python2.7/site-packages (from protobuf!=3.0.0.b2.post1,>=3.0.0b2->gcloud==0.17.0)
Collecting enum34>=1.0.4 (from grpcio>=0.14.0->gcloud==0.17.0)
  Using cached enum34-1.1.6-py2-none-any.whl
Collecting futures>=2.2.0 (from grpcio>=0.14.0->gcloud==0.17.0)
  Using cached futures-3.0.5-py2-none-any.whl
Collecting ply==3.8 (from google-gax>=0.12.1->gcloud==0.17.0)
Collecting grpc-google-pubsub-v1>=0.7.11 (from gax-google-pubsub-v1>=0.7.10->gcloud==0.17.0)
Collecting grpc-google-logging-v2>=0.7.11 (from gax-google-logging-v2>=0.7.10->gcloud==0.17.0)
Installing collected packages: httplib2, six, protobuf, googleapis-common-protos, pyasn1, rsa, pyasn1-modules, oauth2client, enum34, futures, grpcio, ply, google-gax, grpc-google-pubsub-v1, gax-google-pubsub-v1, grpc-google-logging-v2, gax-google-logging-v2, gcloud
  Running setup.py install for gcloud ... done
Successfully installed enum34-1.1.6 futures-3.0.5 gax-google-logging-v2-0.7.11 gax-google-pubsub-v1-0.7.11 gcloud-0.17.0 google-gax-0.12.2 googleapis-common-protos-1.2.0 grpc-google-logging-v2-0.7.11 grpc-google-pubsub-v1-0.7.11 grpcio-0.15.0 httplib2-0.9.2 oauth2client-2.2.0 ply-3.8 protobuf-3.0.0b3 pyasn1-0.1.9 pyasn1-modules-0.0.8 rsa-3.4.2 six-1.10.0
$ /tmp/borked/bin/python
Python 2.7.11 (default, Feb 22 2016, 07:44:44) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from gcloud import storage
>>>

daspecster · 2016-07-08T18:59:53Z

Yeah I had the same results with pip. Not sure why python setup.py install does that off master.

tseaver · 2016-07-08T19:20:19Z

setup.py develop doesn't get protobuf installed, either:

$ /opt/Python-2.7.11/bin/virtualenv /tmp/snorged
New python executable in /tmp/snorged/bin/python
Installing setuptools, pip, wheel...done.
$ cd /tmp/snorged/
$ git clone --depth 1 https://github.com/GoogleCloudPlatform/gcloud-python.git
Cloning into 'gcloud-python'...
remote: Counting objects: 409, done.
remote: Compressing objects: 100% (384/384), done.
remote: Total 409 (delta 78), reused 101 (delta 20), pack-reused 0
Receiving objects: 100% (409/409), 5.77 MiB | 779.00 KiB/s, done.
Resolving deltas: 100% (78/78), done.
Checking connectivity... done.
$ cd gcloud-python/
$ ../bin/python setup.py -q develop
...
$ ../bin/python 
Python 2.7.11 (default, Feb 22 2016, 07:44:44) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from gcloud import storage
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "gcloud/storage/__init__.py", line 43, in <module>
    from gcloud.storage.blob import Blob
  File "gcloud/storage/blob.py", line 30, in <module>
    from gcloud._helpers import _rfc3339_to_datetime
  File "gcloud/_helpers.py", line 28, in <module>
    from google.protobuf import timestamp_pb2
ImportError: cannot import name timestamp_pb2

Which leads me to a smoking gun:

$ /tmp/snorged/bin/python 
Python 2.7.11 (default, Feb 22 2016, 07:44:44) 
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import google
>>> from google import protobuf
>>> protobuf.__path__
['/tmp/snorged/lib/python2.7/site-packages/gax_google_logging_v2-0.7.11-py2.7.egg/google/protobuf']

Which is Just Wrong (TM).

@bjwatson both gax-google-logging-v2-0.7.11 and gax-google-pubsub-v1-0.7.11 include empty google/protobuf packages (just and empty __init__.py). ISTM that the protobuf package shouldn't be here: it isn't declared a namespace package in its real home, and those packages are interfering with a non-coalesced install.

daspecster · 2016-07-08T19:42:15Z

Should we make an issue for this over on gax?

bjwatson · 2016-07-15T20:40:08Z

IIUC, this is describing the same issue as #1972, right?

daspecster · 2016-07-15T20:41:04Z

@bjwatson I believe so.

bjwatson · 2016-07-15T20:47:14Z

Thanks @daspecster. I shared my opinion in that issue.

dhermes · 2016-07-18T17:29:34Z

@tseaver This didn't seem to do the trick: https://ci.appveyor.com/project/dhermes/gcloud-python/build/job/1n38cps6rh8075g8

tseaver · 2016-07-18T17:54:36Z

@dhermes There are no Py3k Windows wheels published for grpcio 0.15.0, which means it falls back to BFS, and thus needs protoc et al :(

dhermes · 2016-07-18T18:10:52Z

Yup. But the unit tests don't need grpcio to work.

tseaver · 2016-07-18T18:35:16Z

@dhermes If you can figure out a better way to install grpcio and the gaxlibraries on Appveyor only for py27-* builds, please do: I tried (e.g. #1877) but lack the Powershell-fu.

Attempt to un-break Appveyor:

ff737ac

- Put '_testing' on PYTHONPATH only for Py3k (debug grpico-on-Py3k issues later). - Use 'setup.py develop' rather than 'setup.py build' to get dependencies installed. - Run 'pip list' afterwards to show what got installed. Toward #1863.

tseaver added the testing label Jul 8, 2016

tseaver assigned daspecster and dhermes Jul 8, 2016

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Jul 8, 2016

tseaver merged commit 804ec9f into googleapis:master Jul 8, 2016

tseaver deleted the 1863-debug-missing-protobuf-on-appveyor branch July 8, 2016 18:31

This was referenced Aug 3, 2016

Prepare 0.18.0 release #2044

Closed

Prepare 0.18.0 release. #2048

Merged

theacodes unassigned daspecster and dhermes Sep 28, 2018

Attempt to un-break Appveyor: #1965

Attempt to un-break Appveyor: #1965

Uh oh!

Conversation

tseaver commented Jul 8, 2016

Uh oh!

tseaver commented Jul 8, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daspecster commented Jul 8, 2016

Uh oh!

tseaver commented Jul 8, 2016

Uh oh!

daspecster commented Jul 8, 2016

Uh oh!

tseaver commented Jul 8, 2016

Uh oh!

daspecster commented Jul 8, 2016

Uh oh!

bjwatson commented Jul 15, 2016

Uh oh!

daspecster commented Jul 15, 2016

Uh oh!

bjwatson commented Jul 15, 2016

Uh oh!

dhermes commented Jul 18, 2016

Uh oh!

tseaver commented Jul 18, 2016

Uh oh!

dhermes commented Jul 18, 2016

Uh oh!

tseaver commented Jul 18, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

tseaver commented Jul 8, 2016 •

edited

Loading