Add docs to a pytorch.github.io/doc/tag directory when repo is tagged #45204

mattip · 2020-09-23T15:47:01Z

In coordination with @jlin27.

This PR is meant to build documentation when the repo is tagged. For instance, tagging the repo with 1.7.0rc1 will push that commit's documentation to pytorch/pytorch.github.io/docs/1.7.

Subsequently tagging 1.7.0rc2 will override the 1.7 docs, as will 1.7.0, and 1.7.1. I think this is as it should be: there should be one, latest, version for the 1.7 docs. This can be tweaked differently if desired.

There is probably work that needs to be done to adjust the versions.html to add the new tag?

Is there a way to test the tagging side of this without breaking the production documentation?

As an aside, the documentation is being built via the pytorch_linux_xenial_py3_6_gcc5_4_build image. Some projects are starting to move on from python3.6 since it is in security-only support mode, no new binaries are being released.

dr-ci · 2020-09-23T16:26:31Z

💊 CI failures summary and remediations

As of commit e6cf93d (more details on the Dr. CI page):

1/1 failures possibly* introduced in this PR
- 1/1 non-CircleCI failure(s)

ci.pytorch.org: 1 failed

Failed: pr/pytorch-linux-bionic-rocm3.7-py3.6

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 43 times.

mattip · 2020-09-25T10:07:37Z

@seemethere the build is successful when triggered on master. Is there a way to do any kind of test for a new tag?

zou3519 · 2020-09-25T15:26:04Z

.circleci/cimodel/data/pytorch_build_definitions.py

I think it's reasonable to move this to a more modern build. We could do that in a follow-up PR

zou3519 · 2020-09-25T15:34:25Z

Subsequently tagging 1.7.0rc2 will override the 1.7 docs, as will 1.7.0, and 1.7.1. I think this is as it should be: there should be one, latest, version for the 1.7 docs. This can be tweaked differently if desired.

In the past, we've kept around the documentation for dot releases separately.

There is probably work that needs to be done to adjust the versions.html to add the new tag?

Probably. Some work also needs to be done in moving the stable/ symlink to point to the right thing when we do releases. I think it's ok if we don't automate those two things for now (it's very easy to just toss up a PR to pytorch.github.io to adjust versions.html and the stable symlink)

mattip · 2020-09-26T17:31:32Z

In the past, we've kept around the documentation for dot releases separately.

Changed in 9ed16dc to keep this behaviour. It would be nice to merge this before the upcoming release cycle.

…gged

codecov · 2020-09-27T11:01:58Z

Codecov Report

Merging #45204 into master will decrease coverage by 0.00%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #45204      +/-   ##
==========================================
- Coverage   68.16%   68.16%   -0.01%     
==========================================
  Files         395      395              
  Lines       51092    51092              
==========================================
- Hits        34828    34827       -1     
- Misses      16264    16265       +1

Impacted Files	Coverage Δ
torch/testing/_internal/expecttest.py	`77.55% <0.00%> (-1.03%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 958c208...e6cf93d. Read the comment docs.

zou3519 · 2020-09-28T14:02:38Z

This looks fine to me but I don't know much about the circleci parts. @seemethere could you take a look please?

seemethere

overall LGTM, there is one gotcha though that might limit the effectiveness on release day.

seemethere · 2020-09-28T15:06:10Z

.circleci/cimodel/data/pytorch_build_definitions.py

+                "filters": gen_filter_dict(branches_list=["nightly"],
+                                           tags_list=RC_PATTERN)


On gotcha that I can foresee potentially happening is that CircleCI has a weird requirement that not all of the jobs can run on a tag, so we'd not only have to add this gen_filter_dict call here but we'd need to add it for all of the parent jobs as well. (i.e. all of the jobs that the docs builds requires)

Ahh, I thought that would resolve automatically. The parent jobs are pytorch_cpp_doc_build, pytorch_python_doc_build, and the pytorch-linux-xenial-py3.6-gcc5.4 composite job.

mattip · 2020-09-29T09:24:49Z

I added tag filters to all the parents of the doc push jobs. The one in docker_definitions.py is particularly ugly, but I think it works.

facebook-github-bot

@seemethere has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

seemethere · 2020-09-29T19:15:42Z

Thanks for the contribution @mattip! 😄

facebook-github-bot · 2020-09-29T20:16:26Z

@seemethere merged this pull request in 147c88e.

facebook-github-bot · 2020-09-29T20:16:36Z

@seemethere merged this pull request in 147c88e.

mattip · 2020-09-30T16:47:26Z

@seemethere it seems something is not quite right with this pr. 1.7.0rc1 was tagged, and the build triggered, but bothe the pytorch_cpp_doc_push and the pytorch_python_doc_push failed. They are missing the $GITHUB_PYTORCHBOT_TOKEN ?

Edit: that was a false alarm, the build must have been canceled or otherwise stopped.

mattip · 2020-09-30T21:36:01Z

Now the doc push is running, but it fails.

Details

#!/bin/bash -eo pipefail
pushd /tmp/workspace
git push -u origin "site"

/tmp/workspace ~/project
To https://github.com/pytorch/pytorch.github.io
 ! [rejected]                site -> site (fetch first)
error: failed to push some refs to 'https://github.com/pytorch/pytorch.github.io'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

Exited with code exit status 1

CircleCI received exit code 1

mattip · 2020-09-30T21:46:25Z

Not sure how, but something created the docs https://pytorch.org/docs/1.7.0/. Was this PR not needed, or did it succeed in spite of the run I pointed to failing?

seemethere · 2020-10-01T21:34:53Z

Oh the first one was an abort, we had to re-run due to an issue with CircleCI

pytorchbot added the open source label Sep 23, 2020

ezyang requested a review from jlin27 September 23, 2020 19:17

ezyang added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Sep 23, 2020

mattip force-pushed the push-docs-on-tag branch 6 times, most recently from 6823111 to 208519b Compare September 25, 2020 07:51

zou3519 reviewed Sep 25, 2020

View reviewed changes

mattip force-pushed the push-docs-on-tag branch from 81fba5e to 9ed16dc Compare September 26, 2020 17:30

mattip added 2 commits September 27, 2020 09:59

add docs to a pytorch.github.io/doc/tag directory when the repo is ta…

0db6175

…gged

change tagged doc version to major.minor.patch

a6a609a

mattip force-pushed the push-docs-on-tag branch from 9ed16dc to a6a609a Compare September 27, 2020 06:59

zou3519 requested a review from seemethere September 28, 2020 14:01

seemethere reviewed Sep 28, 2020

View reviewed changes

mattip force-pushed the push-docs-on-tag branch from 48d70a2 to 1628dc0 Compare September 29, 2020 09:22

add tag filters to the parent dependencies of the doc push jobs

e6cf93d

mattip force-pushed the push-docs-on-tag branch from 1628dc0 to e6cf93d Compare September 29, 2020 09:32

seemethere approved these changes Sep 29, 2020

View reviewed changes

facebook-github-bot reviewed Sep 29, 2020

View reviewed changes

facebook-github-bot closed this in 147c88e Sep 29, 2020

facebook-github-bot added the merged label Sep 29, 2020

mruberry added the Merged label Oct 28, 2020

		"filters": gen_filter_dict(branches_list=["nightly"],
		tags_list=RC_PATTERN)

Add docs to a pytorch.github.io/doc/tag directory when repo is tagged #45204

Add docs to a pytorch.github.io/doc/tag directory when repo is tagged #45204

Uh oh!

Conversation

mattip commented Sep 23, 2020

Uh oh!

dr-ci bot commented Sep 23, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

ci.pytorch.org: 1 failed

Uh oh!

mattip commented Sep 25, 2020

Uh oh!

zou3519 Sep 25, 2020

Choose a reason for hiding this comment

Uh oh!

zou3519 commented Sep 25, 2020

Uh oh!

mattip commented Sep 26, 2020

Uh oh!

codecov bot commented Sep 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

zou3519 commented Sep 28, 2020

Uh oh!

seemethere left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seemethere Sep 28, 2020

Choose a reason for hiding this comment

Uh oh!

mattip Sep 28, 2020

Choose a reason for hiding this comment

Uh oh!

mattip commented Sep 29, 2020

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

seemethere commented Sep 29, 2020

Uh oh!

facebook-github-bot commented Sep 29, 2020

Uh oh!

facebook-github-bot commented Sep 29, 2020

Uh oh!

mattip commented Sep 30, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mattip commented Sep 30, 2020

Uh oh!

mattip commented Sep 30, 2020

Uh oh!

seemethere commented Oct 1, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

dr-ci bot commented Sep 23, 2020 •

edited

Loading

codecov bot commented Sep 27, 2020 •

edited

Loading

seemethere left a comment •

edited

Loading

mattip commented Sep 30, 2020 •

edited

Loading