torch.fx: add debug-level logging to Interpreter.run_node (#117351) by mdbarnesUCSD · Pull Request #166622 · pytorch/pytorch

mdbarnesUCSD · 2025-10-30T07:47:30Z

Summary

Adds a debug-level logging statement to torch.fx.Interpreter.run_node, as proposed in #117351, to make FX graph execution traceable when debugging or instrumenting model transformations.

When debug logging is enabled, each executed node emits a single structured log line formatted via LazyString(lambda: n.format_node()), deferring string construction unless logging is active.

Example Output

With logging.DEBUG enabled:

run_node x = x()
run_node add = _operator.add(x, 1)
run_node clamp = torch.clamp(add, min=0.0, max=5.0)
run_node output = output(clamp)

With logging.DEBUG disabled no additional output is produced (unchanged default behavior).

Test Plan

Verified locally with Python 3.11 on macOS using a PyTorch build from source.

With logging.DEBUG enabled: each node emits a debug log via LazyString.
With logging.DEBUG disabled: no additional output.
Confirmed all Interpreter tests pass locally:
pytest test/test_fx.py -k "Interpreter"

Updated the example output to reflect the new _format_fx_node helper and inclusion of kwargs.

cc @ezyang @EikanWang @jgong5 @wenzhe-nrv

…7351)

pytorch-bot · 2025-10-30T07:47:35Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/166622

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (4 Unrelated Failures)

As of commit 2ce2726 with merge base a533526 ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

pull / linux-jammy-py3.10-clang12 / test (dynamo_wrapped, 2, 3, lf.linux.2xlarge) (gh) (similar failure)
test/test_extension_utils.py::TestExtensionUtils::test_external_module_register_with_renamed_backend
pull / linux-jammy-py3.10-clang12 / test (dynamo_wrapped, 3, 3, lf.linux.2xlarge) (gh) (similar failure)
test/test_rename_privateuse1_to_existing_device.py::TestRenamePrivateuseoneToExistingBackend::test_external_module_register_with_existing_backend
pull / linux-jammy-py3.13-clang12 / test (dynamo_wrapped, 2, 3, lf.linux.2xlarge) (gh) (similar failure)
test/test_extension_utils.py::TestExtensionUtils::test_external_module_register_with_renamed_backend

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / linux-jammy-py3.13-clang12 / test (dynamo_wrapped, 3, 3, lf.linux.2xlarge) (gh) (trunk failure)
test/test_rename_privateuse1_to_existing_device.py::TestRenamePrivateuseoneToExistingBackend::test_external_module_register_with_existing_backend

This comment was automatically generated by Dr. CI and updates every 15 minutes.

linux-foundation-easycla · 2025-10-30T07:47:38Z

The committers listed above are authorized under a signed CLA.

✅ login: mdbarnesUCSD / name: Mark Barnes (2ce2726, 99c9262, ece30aa)

ezyang · 2025-10-30T19:11:23Z

can you do the python printing instead

mdbarnesUCSD · 2025-10-31T04:18:09Z

Thanks @ezyang, I’ve updated the PR to use python printing instead.

Each node is now logged in a format that mirrors the original Python code, including module prefixes like torch.relu and _operator.add.

Example output:

run_node x = x()
run_node add = _operator.add(x, 1)
run_node relu = torch.relu(add)
run_node output = output(relu)

Let me know if any further changes would be helpful.

aorenste · 2025-10-31T04:23:06Z

torch/fx/interpreter.py

+                f"{n.name} = "
+                f"{getattr(n.target, '__module__', '') + '.' if hasattr(n.target, '__module__') else ''}"
+                f"{getattr(n.target, '__name__', n.target)}"
+                f"({', '.join(map(str, n.args))})"


Should we print kwargs too?

aorenste · 2025-10-31T04:23:54Z

torch/fx/interpreter.py

        """
+        log.debug(
+            "run_node %s",
+            LazyString(lambda:


nit: This might be easier to read if the LazyString called a top-level helper function - especially if we want to eventually do more complex formatting (such as printing "call_method" as obj.method() instead of class.method(obj))
(but I don't feel strongly - so only do it if you agree)

Thanks for the review.

I added kwargs to the debug output and moved the string formatting into a top-level _format_fx_node helper for readability and easier future extensions.

aorenste · 2025-10-31T16:10:54Z

Need to update the example output in the PR summary

aorenste · 2025-10-31T16:52:00Z

This LGTM but the lintrunner error needs to be addressed (I can't imagine it will cause anything else to fail).

… formatting

mdbarnesUCSD · 2025-11-01T07:03:19Z

I ran lintrunner -a torch/fx/interpreter.py and applied the fixes for this file, and I also updated the example output in the PR summary.

I noticed that several of the remaining CI jobs are failing under the Linux Jammy configurations (e.g., the dynamo_wrapped test jobs). Are these known flakes, or is this something I should take a closer look at?

aorenste · 2025-11-01T15:03:09Z

I ran lintrunner -a torch/fx/interpreter.py and applied the fixes for this file, and I also updated the example output in the PR summary.

I noticed that several of the remaining CI jobs are failing under the Linux Jammy configurations (e.g., the dynamo_wrapped test jobs). Are these known flakes, or is this something I should take a closer look at?

You should be good. If you look above (in the first "pytorch-bot" comment) it gives a status message. It currently says "You can merge normally" so once I've approved you can use torchbot to merge.

mdbarnesUCSD · 2025-11-06T00:10:11Z

Thanks @aorenste, I’ll go ahead and comment @pytorchbot merge per your note above.

mdbarnesUCSD · 2025-11-06T00:24:06Z

@pytorchbot merge

pytorchmergebot · 2025-11-06T00:26:17Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

torch.fx: add debug-level logging to Interpreter.run_node (pytorch#11…

99c9262

…7351)

pytorch-bot bot added the release notes: fx release notes category label Oct 30, 2025

facebook-github-bot added the fx label Oct 30, 2025

pytorchbot added the open source label Oct 30, 2025

jerryzh168 requested review from aorenste and wconstab October 30, 2025 19:46

jerryzh168 added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Oct 30, 2025

torch.fx: add Python-like debug logging to Interpreter.run_node

ece30aa

aorenste reviewed Oct 31, 2025

View reviewed changes

torch.fx: include kwargs in Interpreter debug logs and add helper for…

2ce2726

… formatting

mdbarnesUCSD force-pushed the fx-interpreter-logging branch from 9611467 to 2ce2726 Compare October 31, 2025 17:33

aorenste approved these changes Nov 1, 2025

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Nov 6, 2025

pytorchmergebot added the merging label Nov 6, 2025

pytorchmergebot added the Merged label Nov 6, 2025

pytorchmergebot closed this in 3feea29 Nov 6, 2025

pytorchmergebot removed the merging label Nov 6, 2025

Conversation

mdbarnesUCSD commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Example Output

Test Plan

Uh oh!

pytorch-bot bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/166622

✅ You can merge normally! (4 Unrelated Failures)

Uh oh!

linux-foundation-easycla bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ezyang commented Oct 30, 2025

Uh oh!

mdbarnesUCSD commented Oct 31, 2025

Uh oh!

aorenste Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

aorenste Oct 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mdbarnesUCSD Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

aorenste commented Oct 31, 2025

Uh oh!

aorenste commented Oct 31, 2025

Uh oh!

mdbarnesUCSD commented Nov 1, 2025

Uh oh!

aorenste commented Nov 1, 2025

Uh oh!

mdbarnesUCSD commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mdbarnesUCSD commented Nov 6, 2025

Uh oh!

pytorchmergebot commented Nov 6, 2025

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

mdbarnesUCSD commented Oct 30, 2025 •

edited

Loading

pytorch-bot bot commented Oct 30, 2025 •

edited

Loading

linux-foundation-easycla bot commented Oct 30, 2025 •

edited

Loading

aorenste Oct 31, 2025 •

edited

Loading

mdbarnesUCSD commented Nov 6, 2025 •

edited

Loading