Should we just load UCC when CUDA/GPU is not available?

UCC will be used to handle CPU tensors for ProcessGroupNCCL, so no, even if CUDA/GPU is available, UCC will still be useful.

One question about UCC dependence. Does OSS UCC depend on NCCL? If so, when UCC is loaded through torch_ucc, is it possible that NCCL will also be loaded? Will this end up with two different versions of NCCL being loaded into PyTorch?

Yes, UCC supports using NCCL (as well as UCX and other libraries) as its transport. But in our case, we are not using this functionality. When we use UCC, we should be using the UCX backend and when we want to use NCCL, we will use it directly.

For the question of having multiple versions of NCCL, my understanding is:

If both PyTorch and UCC are dynamically linked to NCCL

The documentation of dlopen[1] says:

Symbol references in the shared object are resolved using (in order): symbols in the link map of objects loaded for the main program and its dependencies; symbols in shared objects (and their dependencies) that were previously opened with dlopen() using the RTLD_GLOBAL flag; and definitions in the shared object itself (and any dependencies that were loaded for that object).

Because PyTorch is always loaded earlier than torch_ucc, by the time torch_ucc is loaded, symbols of NCCL should have already been available in the main program. So dlopen will just use these symbols for UCC.

If both PyTorch and UCC are statically linked to NCCL

For static linking, symbol resolution should happen at link time. So PyTorch will have its copy of NCCL code, and UCC will also have its copy. Each will use its own copy. The NCCL symbols in libucc.so will not be added to the main program for later symbol resolution because we dlopen it with RTLD_LOCAL, and according to the documentation[1]:

This is the converse of RTLD_GLOBAL, and the default if neither flag is specified. Symbols defined in this shared object are not made available to resolve references in subsequently loaded shared objects.

If PyTorch is statically linked to NCCL and UCC is dynamically linked to NCCL

For this case, PyTorch will use the NCCL that it is compiled with because symbol resolution happened at link time. I don't know which NCCL UCC will use, because I don't know if NCCL's symbol will be copied to PyTorch and added to the link map. If yes, then UCC will use that one from PyTorch. If not, then UCC will loads its own NCCL and use the symbol there.

If PyTorch is dynamically linked to NCCL and UCC is statically linked to NCCL

PyTorch is always loaded earlier than torch_ucc, so PyTorch will load NCCL from the system and use that NCCL. But UCC will still use its own NCCL because all the NCCL function calls are already resolved when libucc.so is generated.

Reference

• [1] https://man7.org/linux/man-pages/man3/dlopen.3.html

Do we really need this cmake flag if we are anyways only conditionally loading the ucc lib at runtime? What is the downside if we don't have this flag at all and default to true? Would there be some issues in the case where ucc lib is not found?

I added this based on #69552 (comment). Right now, it might seem a bit redundant, but in the future, when ProcessGroupNCCL starts to automatically search and load torch_ucc.so from the user's system, having a cmake flag will be helpful. In my opinion, if we are sure we will need one in the future, it won't be bad to have it now, but I can remove this flag if you want.

Why do we need to pass & into the lambda? uccLib_ is static and we shouldn't need & to access it in the lambda?

This is for rank_, otherwise we will have

/home/gaoxiang/pytorch-ucc2/torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp: In lambda function:
/home/gaoxiang/pytorch-ucc2/torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:599:32: error: ‘this’ was not captured for this lambda function
  599 |       LOG(INFO) << "[Rank " << rank_  << "] torch_ucc.so loaded";
      |                                ^~~~~
/home/gaoxiang/pytorch-ucc2/torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:599:32: error: invalid use of non-static data member ‘c10d::ProcessGroup::rank_’
In file included from /home/gaoxiang/pytorch-ucc2/torch/csrc/distributed/c10d/ProcessGroupNCCL.hpp:13,
                 from /home/gaoxiang/pytorch-ucc2/torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:1:
/home/gaoxiang/pytorch-ucc2/torch/csrc/distributed/c10d/ProcessGroup.hpp:431:13: note: declared here
  431 |   const int rank_;
      |             ^~~~~

nit: In an else clause should we log that torch_ucc.so was not loaded.

This is based on #69552 (comment)

Let's make this interface as inline? Otherwise it causes conflicting symbol in building time. Thanks!

done

Thanks! Imported and doing another round of tests