Add docs talking about how to adding symbolic for unsupported ops (#3741)

houseroad · soumith · commit dde10e1d4b20 · 2017-12-15T09:37:09.000-05:00
diff --git a/docs/source/onnx.rst b/docs/source/onnx.rst
@@ -148,6 +148,16 @@ The following operators are supported:
 * log_softmax
 * unfold (experimental support with ATen-Caffe2 integration)
 * elu
+* concat
+* abs
+* index_select
+* pow
+* clamp
+* max
+* min
+* eq
+* exp
+* permute
 * Conv
 * BatchNorm
 * MaxPool1d (ceil_mode not supported)
@@ -159,7 +169,6 @@ The following operators are supported:
 * Dropout
 * FeatureDropout (training mode not supported)
 * Index (constant integer and tuple indices supported)
-* Negate
 
 The operator set above is sufficient to export the following models:
 
@@ -173,8 +182,132 @@ The operator set above is sufficient to export the following models:
 * VGG
 * `word_language_model <https://github.com/pytorch/examples/tree/master/word_language_model>`_
 
-The interface for specifying operator definitions is highly experimental
-and undocumented; adventurous users should note that the APIs will probably
+Adding export support for operators is an *advance usage*.
+To achieve this, developers need to touch the source code of PyTorch.
+Please follow the `instructions <https://github.com/pytorch/pytorch#from-source>`_
+for installing PyTorch from source.
+If the wanted operator is standardized in ONNX, it should be easy to add
+support for exporting such operator (adding a symbolic function for the operator).
+To confirm whether the operator is standardized or not, please check the
+`ONNX operator list <http://https://github.com/onnx/onnx/blob/master/docs/Operators.md>`_.
+
+If the operator is an ATen operator, which means you can find the declaration
+of the function in ``torch/csrc/autograd/generated/VariableType.h``
+(available in generated code in PyTorch install dir), you should add the symbolic
+function in ``torch/onnx/symbolic.py`` and follow the instructions listed as below:
+
+* Define the symbolic function in
+  `torch/onnx/symbolic.py <https://github.com/pytorch/pytorch/blob/master/torch/onnx/symbolic.py>`_.
+  Make sure the function has the same name as the ATen operator/function
+  defined in ``VariableType.h``.
+* The first parameter is always the exported ONNX graph.
+  Parameter names must EXACTLY match the names in ``VariableType.h``,
+  because dispatch is done with keyword arguments.
+* Parameter ordering does NOT necessarily match what is in ``VariableType.h``,
+  tensors (inputs) are always first, then non-tensor arguments.
+* In the symbolic function, if the operator is already standardized in ONNX,
+  we only need to create a node to represent the ONNX operator in the graph.
+* If the input argument is a tensor, but ONNX asks for a scalar, we have to
+  explicitly do the conversion. The helper function ``_scalar`` can convert a
+  scalar tensor into a python scalar, and ``_if_scalar_type_as`` can turn a
+  Python scalar into a PyTorch tensor.
+
+If the operator is a non-ATen operator, the symbolic function has to be
+added in the corresponding PyTorch Function class. Please read the following
+instructions:
+
+* Create a symbolic function named ``symbolic`` in the corresponding Function class.
+* The first parameter is always the exported ONNX graph.
+* Parameter names except the first must EXACTLY match the names in ``forward``.
+* The output tuple size must match the outputs of ``forward``.
+* In the symbolic function, if the operator is already standardized in ONNX,
+  we just need to create a node to represent the ONNX operator in the graph.
+
+Symbolic functions should be implemented in Python. All of these functions interact
+with Python methods which are implemented via C++-Python bindings,
+but intuitively the interface they provide looks like this::
+
+
+    def operator/symbolic(g, *inputs):
+      """
+      Modifies Graph (e.g., using "op"), adding the ONNX operations representing
+      this PyTorch function, and returning a Value or tuple of Values specifying the
+      ONNX outputs whose values correspond to the original PyTorch return values
+      of the autograd Function (or None if an output is not supported by ONNX).
+
+      Arguments:
+        g (Graph): graph to write the ONNX representation into
+        inputs (Value...): list of values representing the variables which contain
+            the inputs for this function
+      """
+
+    class Value(object):
+      """Represents an intermediate tensor value computed in ONNX."""
+      def type(self):
+        """Returns the Type of the value."""
+
+    class Type(object):
+      def sizes(self):
+        """Returns a tuple of ints representing the shape of a tensor this describes."""
+
+    class Graph(object):
+      def op(self, opname, *inputs, **attrs):
+        """
+        Create an ONNX operator 'opname', taking 'args' as inputs
+        and attributes 'kwargs' and add it as a node to the current graph,
+        returning the value representing the single output of this
+        operator (see the `outputs` keyword argument for multi-return
+        nodes).
+
+        The set of operators and the inputs/attributes they take
+        is documented at https://github.com/onnx/onnx/blob/master/docs/Operators.md
+
+        Arguments:
+            opname (string): The ONNX operator name, e.g., `Abs` or `Add`.
+            args (Value...): The inputs to the operator; usually provided
+                as arguments to the `symbolic` definition.
+            kwargs: The attributes of the ONNX operator, with keys named
+                according to the following convention: `alpha_f` indicates
+                the `alpha` attribute with type `f`.  The valid type specifiers are
+                `f` (float), `i` (int), `s` (string) or `t` (Tensor).  An attribute
+                specified with type float accepts either a single float, or a
+                list of floats (e.g., you would say `dims_i` for a `dims` attribute
+                that takes a list of integers).
+            outputs (int, optional):  The number of outputs this operator returns;
+                by default an operator is assumed to return a single output.
+                If `outputs` is greater than one, this functions returns a tuple
+                of output `Value`, representing each output of the ONNX operator
+                in positional.
+        """
+
+The ONNX graph C++ definition is in ``torch/csrc/jit/ir.h``.
+
+Here is an example of handling missing symbolic function for ``elu`` operator.
+We try to export the model and see the error message as below::
+
+    UserWarning: ONNX export failed on elu because torch.onnx.symbolic.elu does not exist
+    RuntimeError: ONNX export failed: Couldn't export operator elu
+
+The export fails because PyTorch does not support exporting ``elu`` operator.
+We find ``virtual Tensor elu(const Tensor & input, Scalar alpha, bool inplace) const override;``
+in ``VariableType.h``. This means ``elu`` is an ATen operator.
+We check the `ONNX operator list <http://https://github.com/onnx/onnx/blob/master/docs/Operators.md>`_,
+and confirm that ``Elu`` is standardized in ONNX.
+We add the following lines to ``symbolic.py``::
+
+    def elu(g, input, alpha, inplace=False):
+        return g.op("Elu", input, alpha_f=_scalar(alpha))
+
+Now PyTorch is able to export ``elu`` operator.
+
+There are more examples in
+`symbolic.py <https://github.com/pytorch/pytorch/blob/master/torch/onnx/symbolic.py>`_,
+`tensor.py <https://github.com/pytorch/pytorch/blob/99037d627da68cdf53d3d0315deceddfadf03bba/torch/autograd/_functions/tensor.py#L24>`_,
+`padding.py <https://github.com/pytorch/pytorch/blob/99037d627da68cdf53d3d0315deceddfadf03bba/torch/nn/_functions/padding.py#L8>`_.
+
+
+The interface for specifying operator definitions is experimental;
+adventurous users should note that the APIs will probably
 change in a future interface.
 
 Functions
