[ghstack-poisoned]

Previously we were re-running CSE every time we recursed into a new block, which in turn created a new Alias Db for the whole graph. This was O(# Nodes * # Blocks). 

For graphs which don't have any autodiff opportunities, such as Densenet,  create_autodiff_subgraphs is now linear in number of nodes. For Densenet this pass was measured at ~.1 seconds. 

This pass is still non-linear for models which actually do create autodiff subgraphs, because in the 
```
      bool any_changed = true;
      while (any_changed) {
        AliasDb aliasDb(graph_);
        any_changed = false;
        for (auto it = workblock.end()->reverseIterator();
             it != workblock.begin()->reverseIterator();) {
          bool changed;
          std::tie(it, changed) = scanNode(*it, aliasDb);
          any_changed |= changed;
        }
      }
``` 
loop we recreate the AliasDb (which is O(N)) every time we merge something and scan node returns. I will make that linear in next PR in the stack.


[ghstack-poisoned]

Previously we were re-running CSE every time we recursed into a new block, which in turn created a new Alias Db for the whole graph. This was O(# Nodes * # Blocks). 

For graphs which don't have any autodiff opportunities, such as Densenet,  create_autodiff_subgraphs is now linear in number of nodes. For Densenet this pass was measured at ~.1 seconds. 

This pass is still non-linear for models which actually do create autodiff subgraphs, because in the 
```
      bool any_changed = true;
      while (any_changed) {
        AliasDb aliasDb(graph_);
        any_changed = false;
        for (auto it = workblock.end()->reverseIterator();
             it != workblock.begin()->reverseIterator();) {
          bool changed;
          std::tie(it, changed) = scanNode(*it, aliasDb);
          any_changed |= changed;
        }
      }
``` 
loop we recreate the AliasDb (which is O(N)) every time we merge something and scan node returns. I will make that linear in next PR in the stack.


[ghstack-poisoned]

Previously we were re-running CSE every time we recursed into a new block, which in turn created a new Alias Db for the whole graph. This was O(# Nodes * # Blocks). 

For graphs which don't have any autodiff opportunities, such as Densenet,  create_autodiff_subgraphs is now linear in number of nodes. For Densenet this pass was measured at ~.1 seconds. 

This pass is still non-linear for models which actually do create autodiff subgraphs, because in the 
```
      bool any_changed = true;
      while (any_changed) {
        AliasDb aliasDb(graph_);
        any_changed = false;
        for (auto it = workblock.end()->reverseIterator();
             it != workblock.begin()->reverseIterator();) {
          bool changed;
          std::tie(it, changed) = scanNode(*it, aliasDb);
          any_changed |= changed;
        }
      }
``` 
loop we recreate the AliasDb (which is O(N)) every time we merge something and scan node returns. I will make that linear in next PR in the stack.

Differential Revision: [D22600606](https://our.internmc.facebook.com/intern/diff/D22600606)

[ghstack-poisoned]

Previously we were re-running CSE every time we recursed into a new block, which in turn created a new Alias Db for the whole graph. This was O(# Nodes * # Blocks). 

For graphs which don't have any autodiff opportunities, such as Densenet,  create_autodiff_subgraphs is now linear in number of nodes. For Densenet this pass was measured at ~.1 seconds. 

This pass is still non-linear for models which actually do create autodiff subgraphs, because in the 
```
      bool any_changed = true;
      while (any_changed) {
        AliasDb aliasDb(graph_);
        any_changed = false;
        for (auto it = workblock.end()->reverseIterator();
             it != workblock.begin()->reverseIterator();) {
          bool changed;
          std::tie(it, changed) = scanNode(*it, aliasDb);
          any_changed |= changed;
        }
      }
``` 
loop we recreate the AliasDb (which is O(N)) every time we merge something and scan node returns. I will make that linear in next PR in the stack.

Differential Revision: [D22600606](https://our.internmc.facebook.com/intern/diff/D22600606)

[ghstack-poisoned]