Skip to content

New APIs for cuda graph inspection and manipulation #87163

@ngimel

Description

@ngimel

A stalled #85519 adds capabilities to dump cuda graphs for inspection. We would like to extend it to

  1. provide a way to get all nodes from the graph
  2. modify nodes to patch in addresses for the inputs to avoid copying inputs. Note, this is more complicated than just comparing old and new pointers and swapping one for another because kernels could operate on input slices, so one would need to inspect if kernel argument falls within the range of the original input tensor and, if so, replace it with the corresponding new address + offset.

cc @mcarilli @ezyang @eqy

Metadata

Metadata

Assignees

No one assigned

    Labels

    module: cuda graphsAbility to capture and then replay streams of CUDA kernelstriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions