🐛 Describe the bug

# main.py
import torch

@torch.compile()
def fn():
    pass

❯ python -VV
Python 3.9.18 | packaged by conda-forge | (main, Aug 30 2023, 03:49:32) 
[GCC 12.3.0]
❯ python -Werror main.py
[...]
ModuleNotFoundError: No module named 'packaging.version'
[...]
DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html

As seen above, I don't have packaging installed in my env and pkg_resources is deprecated. Not only that, but starting with Python 3.12, setuptools (which supplies pkg_resources) will no longer be installed as part of a venv. Meaning, users of that will see errors when trying to use torch.compile.

The underlying issue, i.e. the non-declared dependency, was already reported in #71280 and highlighted again in #71902 (comment) but never addressed.

torch.compile hits this because it crawls torch modules at startup:

And with that it for example hits

That by itself would not cause the dependency issue to show up, because we only lazy load either of the packages (#71345). However, during the crawling we also check each obj we find whether it is part of a special case list

and that in turn materializes the version

ultimately triggering the traceback above.

I think we should do one of two things:

1. Add a runtime dependency on packaging, i.e. have another go at Replaced deprecated pkg_resources.packaging with packaging module #113023.
2. Vendor the relevant parts of packaging.version as was proposed in Import packaging.version in torch_version, if available #71902 (comment)

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @zou3519 @malfet @rgommers @vfdev-5 @hauntsaninja

Versions

2.2.0.dev20231116+cpu