mostly just wondering - were the cpp tests no running in CI before this? And any reason to only enable them for cuda builds on CI? (once the XLA plugin is ready, I'm guessing we want it to run on XLA too?)

correct, it was an oversight, but not a big deal since we mostly looked at the lazy_tensor_staging CI, and also this test was running in sandcastle. But it did confuse me for a while when I was trying to find the output of the expanded test suite in my later diff!

also note, this change is supposed to be landed first in #74449

I'm not sure why the rebase didn't work as intended, but this diff should only be adding the per-operator-headers stuff, and i'll land the other one first and make sure this one is clean.

hmm I might just be misunderstanding. But why does the actual codegen need to know if we're building the per-operator flag or not?

I think the way it works today is that the codegen will always emit the per-operator headers. The codegen templates files themselves include an IFDEF like this:

#ifndef AT_PER_OPERATOR_HEADERS
#include <ATen/Operators.h>
#else
${operator_headers}
#endif

That way the conditional code just lives directly in the C++ template file, instead of needing to clutter up the codegen with it. Example template:

I think I didn't realize this logic existed, but it is still not enough since I also had to fix this (see below)

'ops_headers': '#include <ATen/Functions.h>' if not per_operator_headers else '',

Perhaps we could move ATen/Functions.h into the template ifdef and then things would have been OK.

I think the way it works today is that the codegen will always emit the per-operator headers. The codegen templates files themselves include an IFDEF like this:

I think this isn't quite correct, the RegisterFunctionalization template is a nice thing to emulate but it is not how RegisterDispatchKey works currently. RegisterDispatchKey currently has its per_operator_headers thing hardcoded to False in gen_backend_stubs.py (probably since that is presumed only used out of tree), and RegisterDispatchKey.cpp doesn't have the nice #ifndef that you pointed out in RegisterFunctionalization.

I was starting to mess with this, and I am kind of wondering if it's ok to land as-is and then update how it works. I mainly don't want to add any more delay to landing the TS backend since that is blocking other stuff downstream.

ok yep, I definitely don't want to hold up the PR on this if landing this PR is actively blocking other things.

you're also totally right - it looks like we do something similar for RegisterDispatchKey.cpp (plumbing this flag into the codegen), e.g. here: