Optimizers DeepSpeed offers high-performance implementations of Adam and Lamb optimizers on CPU and GPU, respectively. DeepSpeed CPU Adam .. autoclass:: deepspeed.ops.adam.DeepSpeedCPUAdam DeepSpeed Fused Lamb .. autoclass:: deepspeed.ops.adam.DeepSpeedCPUAdam