This repository was archived by the owner on Dec 6, 2024. It is now read-only.

Repository navigation

Name		Name	Last commit message	Last commit date
parent directory ..
kernels		kernels
README.md		README.md
benchmark_latency.py		benchmark_latency.py
benchmark_serving.py		benchmark_serving.py
benchmark_throughput.py		benchmark_throughput.py
launch_tgi_server.sh		launch_tgi_server.sh

README.md

Benchmarking vLLM

Downloading the ShareGPT dataset

You can download the dataset by running:

wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json