Support for one RL model and related pipelines is included in the experimental source of diffusers.

To try some of this in colab, please look at the following example:

• Model-based reinforcement learning on Colab