Inference  #1

Activity

1. Daniel-0222 commented on May 28, 2024

   

   Daniel-0222

   on May 28, 2024

   Contributor

   More actions

   Hello! If the code for inference with the open-source models available?

   We use the vLLM framework for inference, if the open-source model support it. For other models, we use FastChat framework.