Wrong output from UMAP and T-SNE when running projector locally. #4687
Description
I am running the standalone projector plugin using:
bazel run tensorboard/plugins/projector/vz_projector:standaloneI thought it was working fine, but the output of T-SNE and UMAP (perhaps PCA, too) seem off compared to what I get when using https://projector.tensorflow.org. I am using the default Word2Vec 10K dataset for comparison. I'm assuming this is an installation issue but I don't know what is causing it or how I can fix it. In particular, UMAP is running but the output I am getting is wrong. There are no errors reported on the console.
Environment information
Diagnostics
Diagnostics output
--- check: autoidentify
INFO: diagnose_tensorboard.py version e43767ef2b648d0d5d57c00f38ccbd38390e38da
--- check: general
INFO: sys.version_info: sys.version_info(major=3, minor=7, micro=9, releaselevel='final', serial=0)
INFO: os.name: posix
INFO: os.uname(): posix.uname_result(sysname='Darwin', nodename='redacted', release='19.6.0', version='Darwin Kernel Version 19.6.0: Tue Nov 10 00:10:30 PST 2020; root:xnu-6153.141.10~1/RELEASE_X86_64', machine='x86_64')
INFO: sys.getwindowsversion(): N/A
--- check: package_management
INFO: has conda-meta: True
INFO: $VIRTUAL_ENV: None
--- check: installed_packages
INFO: installed: tensorboard==2.0.0
INFO: installed: tensorflow==2.0.0
INFO: installed: tensorflow-estimator==2.0.0
--- check: tensorboard_python_version
INFO: tensorboard.version.VERSION: '2.5.0a0'
--- check: tensorflow_python_version
WARNING: Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING: Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING: Limited tf.summary API due to missing TensorBoard installation.
INFO: tensorflow.__version__: '2.0.0'
INFO: tensorflow.__git_version__: 'unknown'
--- check: tensorboard_data_server_version
INFO: no data server installed
--- check: tensorboard_binary_path
INFO: which tensorboard: b'/Users/bsejdiu/Programs/anaconda3/envs/projector/bin/tensorboard\n'
--- check: addrinfos
socket.has_ipv6 = True
socket.AF_UNSPEC = <AddressFamily.AF_UNSPEC: 0>
socket.SOCK_STREAM = <SocketKind.SOCK_STREAM: 1>
socket.AI_ADDRCONFIG = <AddressInfo.AI_ADDRCONFIG: 1024>
socket.AI_PASSIVE = <AddressInfo.AI_PASSIVE: 1>
Loopback flags: <AddressInfo.AI_ADDRCONFIG: 1024>
Loopback infos: [(<AddressFamily.AF_INET6: 30>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::1', 0, 0, 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 0))]
Wildcard flags: <AddressInfo.AI_PASSIVE: 1>
Wildcard infos: [(<AddressFamily.AF_INET6: 30>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::', 0, 0, 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('0.0.0.0', 0))]
--- check: readable_fqdn
INFO: socket.getfqdn(): 'redacted'
--- check: stat_tensorboardinfo
INFO: directory: /var/folders/4c/n4b2c_fd4z383dljxs5gtjv8jkmjc9/T/.tensorboard-info
INFO: os.stat(...): os.stat_result(st_mode=16895, st_ino=11405864, st_dev=16777220, st_nlink=2, st_uid=589972873, st_gid=633320910, st_size=64, st_atime=1613552950, st_mtime=1613553034, st_ctime=1613553034)
INFO: mode: 0o40777
--- check: source_trees_without_genfiles
INFO: tensorboard_roots (3): ['', '/Users/bsejdiu/projects/work/projector/tensorboard', '/Users/bsejdiu/Programs/anaconda3/envs/projector/lib/python3.7/site-packages']; bad_roots (2): ['', '/Users/bsejdiu/projects/work/projector/tensorboard']
--- check: full_pip_freeze
INFO: pip freeze --all:
absl-py @ file:///tmp/build/80754af9/absl-py_1607439979954/work
aiohttp @ file:///Users/runner/miniforge3/conda-bld/aiohttp_1610358591733/work
astor==0.8.1
async-timeout==3.0.1
attrs @ file:///home/conda/feedstock_root/build_artifacts/attrs_1605083924122/work
blinker==1.4
brotlipy==0.7.0
cachetools @ file:///home/conda/feedstock_root/build_artifacts/cachetools_1611555765219/work
certifi==2020.12.5
cffi @ file:///Users/runner/miniforge3/conda-bld/cffi_1613413897768/work
chardet @ file:///Users/runner/miniforge3/conda-bld/chardet_1602255311078/work
click==7.1.2
coverage @ file:///opt/concourse/worker/volumes/live/c4ae5873-1840-4da1-74c5-19e7db8e684f/volume/coverage_1611690852652/work
cryptography @ file:///Users/runner/miniforge3/conda-bld/cryptography_1612993767354/work
gast==0.2.2
google-auth @ file:///home/conda/feedstock_root/build_artifacts/google-auth_1608136875028/work
google-auth-oauthlib @ file:///home/conda/feedstock_root/build_artifacts/google-auth-oauthlib_1603996258953/work
google-pasta==0.2.0
grpcio @ file:///opt/concourse/worker/volumes/live/b2f90c2d-e2fb-4c06-6bed-1cffafec49d6/volume/grpcio_1613130423367/work
h5py @ file:///opt/concourse/worker/volumes/live/98bee3fd-2687-4017-5421-d742d1f18c35/volume/h5py_1593454156895/work
idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1593328102638/work
importlib-metadata @ file:///tmp/build/80754af9/importlib-metadata_1602276842396/work
Keras-Applications @ file:///tmp/build/80754af9/keras-applications_1594366238411/work
Keras-Preprocessing @ file:///tmp/build/80754af9/keras-preprocessing_1612283640596/work
Markdown @ file:///opt/concourse/worker/volumes/live/ab9d027b-aa96-4f3e-4a67-57db8504e6b6/volume/markdown_1605111065503/work
mkl-fft==1.2.0
mkl-random==1.1.1
mkl-service==2.3.0
multidict @ file:///Users/runner/miniforge3/conda-bld/multidict_1610319029348/work
numpy @ file:///opt/concourse/worker/volumes/live/a1b5ea96-b05c-40ba-5dfb-4542ece1c186/volume/numpy_and_numpy_base_1603491228242/work
oauthlib==3.0.1
opt-einsum==3.1.0
pip==20.3.3
protobuf==3.14.0
pyasn1==0.4.8
pyasn1-modules==0.2.7
pycparser @ file:///home/conda/feedstock_root/build_artifacts/pycparser_1593275161868/work
PyJWT @ file:///home/conda/feedstock_root/build_artifacts/pyjwt_1610910308735/work
pyOpenSSL @ file:///home/conda/feedstock_root/build_artifacts/pyopenssl_1608055815057/work
PySocks @ file:///Users/runner/miniforge3/conda-bld/pysocks_1610291468418/work
requests @ file:///home/conda/feedstock_root/build_artifacts/requests_1608156231189/work
requests-oauthlib @ file:///home/conda/feedstock_root/build_artifacts/requests-oauthlib_1595492159598/work
rsa @ file:///home/conda/feedstock_root/build_artifacts/rsa_1613425487541/work
scipy @ file:///opt/concourse/worker/volumes/live/53520250-e526-4734-60b4-b4658592fe75/volume/scipy_1612469556481/work
setuptools==52.0.0.post20210125
six @ file:///opt/concourse/worker/volumes/live/f983ba11-c9fe-4dff-7ce7-d89b95b09771/volume/six_1605205318156/work
tensorboard==2.0.0
tensorflow==2.0.0
tensorflow-estimator==2.0.0
termcolor==1.1.0
typing-extensions @ file:///home/conda/feedstock_root/build_artifacts/typing_extensions_1602702424206/work
urllib3 @ file:///home/conda/feedstock_root/build_artifacts/urllib3_1611695416663/work
Werkzeug @ file:///home/ktietz/src/ci/werkzeug_1611932622770/work
wheel==0.36.2
wrapt==1.12.1
yarl @ file:///Users/runner/miniforge3/conda-bld/yarl_1610354318691/work
zipp @ file:///tmp/build/80754af9/zipp_1604001098328/work
Suggestion: Avoid tensorboard packages without genfiles
Your Python path contains a tensorboard package that does not
include generated files. This can happen if your current directory
includes the TensorBoard source tree (e.g., you are in the TensorBoard
Git repository). The following directories from your Python path may
be problematic:
- current directory
- '/Users/bsejdiu/projects/work/projector/tensorboard' (duplicate underlying directory)
Next steps
Please try each suggestion enumerated above to determine whether it
solves your problem. If none of these suggestions works, please copy
ALL of the above output, including the lines containing only
backticks, into your GitHub issue or comment. Be sure to redact any
sensitive information.
Steps to reproduce
conda create --name projector python=3.7
conda install tensorflow
bazel --version
# bazel 4.0.0-homebrew
bazel run tensorboard/plugins/projector/vz_projector:standaloneI have tested MacOS 10.15.7 and WSL 1 in Windows 10. I observe the same issue.
Here are two screenshots to compare the output of UMAP by projector installed locally (left) vs the webserver (right picture). The parameters are left to their default values.
And the following is the T-SNE output after ~80 iterations (left is the local installation):
The output of T-SNE changes very little and simply remains a blob. It also runs much more slowly compared to the webserver.