Based on #10891, we are not currently catching documenation build issues in our CI. As in Jupyter notebook jupyter/notebook#1749 , we should be both building the documentation and documenting jupyter/notebook#1749 how to build the docs.

@Carreau should we also get Appveyor to build?