11# 🦙 Python Bindings for ` llama.cpp `
22
3+ [ ![ PyPI] ( https://img.shields.io/pypi/v/llama-cpp-python )] ( https://pypi.org/project/llama-cpp-python/ )
4+ [ ![ PyPI - Python Version] ( https://img.shields.io/pypi/pyversions/llama-cpp-python )] ( https://pypi.org/project/llama-cpp-python/ )
5+ [ ![ PyPI - License] ( https://img.shields.io/pypi/l/llama-cpp-python )] ( https://pypi.org/project/llama-cpp-python/ )
6+ [ ![ PyPI - Downloads] ( https://img.shields.io/pypi/dm/llama-cpp-python )] ( https://pypi.org/project/llama-cpp-python/ )
7+
38Simple Python bindings for ** @ggerganov 's** [ ` llama.cpp ` ] ( https://github.com/ggerganov/llama.cpp ) library.
49This package provides:
510
@@ -8,6 +13,42 @@ This package provides:
813 - OpenAI-like API
914 - LangChain compatibility
1015
16+ ## Installation
17+
18+ Install from PyPI:
19+
20+ ``` bash
21+ pip install llama-cpp-python
22+ ```
23+
24+ ## Usage
25+
26+ ``` python
27+ >> > from llama_cpp import Llama
28+ >> > llm = Llama(model_path = " models/7B/..." )
29+ >> > output = llm(" Q: Name the planets in the solar system? A: " , max_tokens = 32 , stop = [" Q:" , " \n " ], echo = True )
30+ >> > print (output)
31+ {
32+ " id" : " cmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" ,
33+ " object" : " text_completion" ,
34+ " created" : 1679561337 ,
35+ " model" : " models/7B/..." ,
36+ " choices" : [
37+ {
38+ " text" : " Q: Name the planets in the solar system? A: Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus, Neptune and Pluto." ,
39+ " index" : 0 ,
40+ " logprobs" : None ,
41+ " finish_reason" : " stop"
42+ }
43+ ],
44+ " usage" : {
45+ " prompt_tokens" : 14 ,
46+ " completion_tokens" : 28 ,
47+ " total_tokens" : 42
48+ }
49+ }
50+ ```
51+
1152
1253## API Reference
1354
@@ -20,4 +61,8 @@ This package provides:
2061
2162::: llama_cpp.llama_cpp
2263 options:
23- show_if_no_docstring: true
64+ show_if_no_docstring: true
65+
66+ ## License
67+
68+ This project is licensed under the terms of the MIT license.
0 commit comments