sci-ml/llama-cpp-python
Python bindings for the llama.cpp library
Screenshots
-
llama-cpp-python-0.3.36~amd64native openblas +openmp vulkan video_cards_amdgpu cpu_flags_x86_amx_bf16 cpu_flags_x86_amx_int8 cpu_flags_x86_amx_tile cpu_flags_x86_avx cpu_flags_x86_avx_vnni cpu_flags_x86_avx2 cpu_flags_x86_avx512_bf16 cpu_flags_x86_avx512bw cpu_flags_x86_avx512f cpu_flags_x86_avx512vbmi cpu_flags_x86_avx512_vnni cpu_flags_x86_bmi2 cpu_flags_x86_fma3 cpu_flags_x86_f16c cpu_flags_x86_sse4_2 python_single_target_python3_12 python_single_target_python3_13 python_single_target_python3_14
View 
Download 
Browse License: MIT Overlay: local-ai
ChangeLog
commit 4f6579913f95ce6322c6340c45ef98a9f836c175
Author: Plamen K. Kosseff <plamen@ipnmod.org>
Date: Wed Oct 28 10:00:26 2026 +0000
localai-backend/faster-whisper: new backend, venv-free, with the shared python layer under it
New packages: sci-ml/torchmetrics, vector-quantize-pytorch, modelscope, llama-cpp-python, torchao, faster-whisper, pytorch-lightning, lightning. sci-libs/ctranslate2 and dev-python/ctranslate2.
ctranslate2: empty submodule dirs filled from the tree (spdlog, cxxopts) and from pinned tarballs (cpu_features, ruy with dev-libs/cpuinfo). MKL and the Intel OpenMP runtime off, openblas/dnnl/ruy as USE flags, rocm wired like local-ai-ggml.eclass.
llama-cpp-python: builds its vendored llama.cpp, same CPU gates and vulkan surface as the ggml eclass, libllama under llama_cpp/lib.
torchao: built from the GitHub tag with USE_CPP=0, the pure-python package upstream ships as its py3-none-any wheel.
The faster-whisper backend carries only what backend.py imports. The upstream requirement files list unrelated packages.
metadata/docs/todo-kokoro-kitten-tts.md: findings for the two backends set aside.
Assisted-by: Claude Code:claude-fable-5
Signed-off-by: Plamen K. Kosseff <plamen@ipnmod.org>
Author: Plamen K. Kosseff <plamen@ipnmod.org>
Date: Wed Oct 28 10:00:26 2026 +0000
localai-backend/faster-whisper: new backend, venv-free, with the shared python layer under it
New packages: sci-ml/torchmetrics, vector-quantize-pytorch, modelscope, llama-cpp-python, torchao, faster-whisper, pytorch-lightning, lightning. sci-libs/ctranslate2 and dev-python/ctranslate2.
ctranslate2: empty submodule dirs filled from the tree (spdlog, cxxopts) and from pinned tarballs (cpu_features, ruy with dev-libs/cpuinfo). MKL and the Intel OpenMP runtime off, openblas/dnnl/ruy as USE flags, rocm wired like local-ai-ggml.eclass.
llama-cpp-python: builds its vendored llama.cpp, same CPU gates and vulkan surface as the ggml eclass, libllama under llama_cpp/lib.
torchao: built from the GitHub tag with USE_CPP=0, the pure-python package upstream ships as its py3-none-any wheel.
The faster-whisper backend carries only what backend.py imports. The upstream requirement files list unrelated packages.
metadata/docs/todo-kokoro-kitten-tts.md: findings for the two backends set aside.
Assisted-by: Claude Code:claude-fable-5
Signed-off-by: Plamen K. Kosseff <plamen@ipnmod.org>