sci-misc/kvmem-llama-cpp
CUDA llama.cpp runtime with tiered KV memory for long contexts
Screenshots
-
kvmem-llama-cpp-0.17.0~amd64cpu_flags_x86_avx cpu_flags_x86_avx2 cpu_flags_x86_bmi2 cpu_flags_x86_f16c cpu_flags_x86_fma3 cpu_flags_x86_sse4_2 +openmp test
View 
Download 
Browse License: Apache-2.0 MIT Overlay: ruizhi-overlay
Runtime Dependencies
kvmem-llama-cpp-0.17.0
>=dev-util/nvidia-cuda-toolkit-13.2.2:=
openmp?
( llvm-runtimes/openmp:= )
x11-drivers/nvidia-drivers
openmp?
( llvm-runtimes/openmp:= )
x11-drivers/nvidia-drivers