Skip to content

training.rst

Latest commit

 

History

History
33 lines (24 loc) · 826 Bytes

training.rst

File metadata and controls

33 lines (24 loc) · 826 Bytes

Training API

:func:`deepspeed.initialize` returns a training engine in its first argument of type :class:`DeepSpeedEngine`. This engine is used to progress training:

for step, batch in enumerate(data_loader):
    #forward() method
    loss = model_engine(batch)

    #runs backpropagation
    model_engine.backward(loss)

    #weight update
    model_engine.step()

Forward Propagation

.. autofunction:: deepspeed.DeepSpeedEngine.forward

Backward Propagation

.. autofunction:: deepspeed.DeepSpeedEngine.backward

Optimizer Step

.. autofunction:: deepspeed.DeepSpeedEngine.step

Gradient Accumulation

.. autofunction:: deepspeed.DeepSpeedEngine.is_gradient_accumulation_boundary