feat(low-level-api): Improve API static type-safety and performance by abetlen · Pull Request #1205 · abetlen/llama-cpp-python

abetlen · 2024-02-21T18:06:39Z

Low level api calls now call directly into the loaded dll, not wrapped by a python function first. Reduces overhead however this now requires all low level calls to be positional argument only (this is type checked also).
Use NewType to wrap opaque pointers like ggml_context_p and ggml_model_p, improves static type safety

…que pointers.

abetlen added 2 commits February 21, 2024 13:03

Remove low-level api call indirection and use NewType wrapper for opa…

0611aba

…que pointers.

Merge branch 'main' into improve-low-level-perf-and-type-safety

c7db763

abetlen merged commit 7f51b60 into main Feb 21, 2024

abetlen deleted the improve-low-level-perf-and-type-safety branch February 26, 2024 19:42

Provide feedback