sci-ml/llama-cpp
LLM inference in C/C++
Screenshots
-
llama-cpp-9999openblas +openmp blis rocm cuda opencl vulkan flexiblas wmma examples rpc +server cpu_flags_x86_sse4_2 cpu_flags_x86_avx cpu_flags_x86_avx_vnni cpu_flags_x86_avx2 cpu_flags_x86_bmi2 cpu_flags_x86_avx512f cpu_flags_x86_avx512cd cpu_flags_x86_avx512vl cpu_flags_x86_avx512dq cpu_flags_x86_avx512bw cpu_flags_x86_avx512vbmi cpu_flags_x86_avx512_vnni cpu_flags_x86_avx512_bf16 cpu_flags_x86_fma3 cpu_flags_x86_f16c cpu_flags_x86_amx_tile cpu_flags_x86_amx_int8 cpu_flags_x86_amx_bf16 cpu_flags_riscv_v cpu_flags_riscv_zfh cpu_flags_riscv_zvfh cpu_flags_riscv_zicbop cpu_flags_riscv_zihintpause cpu_flags_riscv_xtheadvector +amdgpu_targets_gfx908 +amdgpu_targets_gfx90a +amdgpu_targets_gfx942 +amdgpu_targets_gfx1030 +amdgpu_targets_gfx1100 +amdgpu_targets_gfx1101 +amdgpu_targets_gfx1200 +amdgpu_targets_gfx1201 amdgpu_targets_gfx803 amdgpu_targets_gfx900 amdgpu_targets_gfx906 amdgpu_targets_gfx940 amdgpu_targets_gfx941 amdgpu_targets_gfx1010 amdgpu_targets_gfx1011 amdgpu_targets_gfx1012 amdgpu_targets_gfx1031 amdgpu_targets_gfx1102 amdgpu_targets_gfx1103 amdgpu_targets_gfx1150 amdgpu_targets_gfx1151
View 
Download 
Browse License: MIT Overlay: gentoo-zh -
llama-cpp-9999systemd utils static lto test examples disable-arm-neon dynamic-backends curl hbm android msvc +accelerate blas blis +llamafile cann musa cuda cuda-force-mmq cuda-force-cublas +cuda-unified-memory cuda-f16 cuda-no-peer-copy cuda-no-vmm cuda-fa-all-quants +cuda-graphs hip hip-graphs +hip-no-vmm hip-uma vulkan vulkan-check-results vulkan-debug vulkan-memory-debug vulkan-shader-debug-info vulkan-perf vulkan-validate vulkan-run-tests kompute +openmp rpc opencl opencl-profiling +opencl-embed-kernels +opencl-use-adreno-kernels metal metal-use-bf16 metal-ndebug metal-shader-debug +metal-embed-library +cpu webgpu cpu-native cpu-all-variants cpu_flags_x86_avx cpu_flags_x86_avx_vnni cpu_flags_x86_avx2 cpu_flags_x86_avx512 cpu_flags_x86_avx512_vbmi cpu_flags_x86_avx512_vnni cpu_flags_x86_avx512_bf16 cpu_flags_x86_fma cpu_flags_x86_f16c cpu_flags_x86_amx_tile cpu_flags_x86_amx_int8 cpu_flags_x86_amx_bf16 cpu_flags_x86_sse cpu_flags_x86_sse2 cpu_flags_x86_sse3 cpu_flags_x86_sse4 cpu_flags_x86_sse4a cpu_flags_x86_sse41 cpu_flags_x86_sse42 cpu_flags_x86_ssse3 kleidiai cpu_flags_loong_lasx cpu_flags_loong_lsx cpu_flags_riscv_rvv abi_x86_32 abi_x86_64 abi_x86_x32 abi_mips_n32 abi_mips_n64 abi_mips_o32 abi_s390_32 abi_s390_64 +amdgpu_targets_gfx908 +amdgpu_targets_gfx90a +amdgpu_targets_gfx942 +amdgpu_targets_gfx1030 +amdgpu_targets_gfx1100 +amdgpu_targets_gfx1101 +amdgpu_targets_gfx1200 +amdgpu_targets_gfx1201 amdgpu_targets_gfx803 amdgpu_targets_gfx900 amdgpu_targets_gfx906 amdgpu_targets_gfx940 amdgpu_targets_gfx941 amdgpu_targets_gfx1010 amdgpu_targets_gfx1011 amdgpu_targets_gfx1012 amdgpu_targets_gfx1031 amdgpu_targets_gfx1102 amdgpu_targets_gfx1103 amdgpu_targets_gfx1150 amdgpu_targets_gfx1151 python_single_target_python3_11 python_single_target_python3_12
View Download Browse License: MIT Overlay: gentooplusplus -
llama-cpp-0_pre8583~amd64 ~riscvopenblas +openmp blis rocm cuda opencl vulkan flexiblas wmma examples rpc +server cpu_flags_x86_sse4_2 cpu_flags_x86_avx cpu_flags_x86_avx_vnni cpu_flags_x86_avx2 cpu_flags_x86_bmi2 cpu_flags_x86_avx512f cpu_flags_x86_avx512cd cpu_flags_x86_avx512vl cpu_flags_x86_avx512dq cpu_flags_x86_avx512bw cpu_flags_x86_avx512vbmi cpu_flags_x86_avx512_vnni cpu_flags_x86_avx512_bf16 cpu_flags_x86_fma3 cpu_flags_x86_f16c cpu_flags_x86_amx_tile cpu_flags_x86_amx_int8 cpu_flags_x86_amx_bf16 cpu_flags_riscv_v cpu_flags_riscv_zfh cpu_flags_riscv_zvfh cpu_flags_riscv_zicbop cpu_flags_riscv_zihintpause cpu_flags_riscv_xtheadvector +amdgpu_targets_gfx908 +amdgpu_targets_gfx90a +amdgpu_targets_gfx942 +amdgpu_targets_gfx1030 +amdgpu_targets_gfx1100 +amdgpu_targets_gfx1101 +amdgpu_targets_gfx1200 +amdgpu_targets_gfx1201 amdgpu_targets_gfx803 amdgpu_targets_gfx900 amdgpu_targets_gfx906 amdgpu_targets_gfx940 amdgpu_targets_gfx941 amdgpu_targets_gfx1010 amdgpu_targets_gfx1011 amdgpu_targets_gfx1012 amdgpu_targets_gfx1031 amdgpu_targets_gfx1102 amdgpu_targets_gfx1103 amdgpu_targets_gfx1150 amdgpu_targets_gfx1151
View Download Browse License: MIT Overlay: gentoo-zh -
llama-cpp-0.6644~amd64+curl +server +amdgpu_targets_gfx908 +amdgpu_targets_gfx90a +amdgpu_targets_gfx942 +amdgpu_targets_gfx1030 +amdgpu_targets_gfx1100 +amdgpu_targets_gfx1101 +amdgpu_targets_gfx1200 +amdgpu_targets_gfx1201 amdgpu_targets_gfx803 amdgpu_targets_gfx900 amdgpu_targets_gfx906 amdgpu_targets_gfx940 amdgpu_targets_gfx941 amdgpu_targets_gfx1010 amdgpu_targets_gfx1011 amdgpu_targets_gfx1012 amdgpu_targets_gfx1031 amdgpu_targets_gfx1102 amdgpu_targets_gfx1103 amdgpu_targets_gfx1150 amdgpu_targets_gfx1151
View Download Browse License: MIT Overlay: phackerlay -
llama-cpp-0.6335~amd64 ~arm ~arm64 ~arm64-macos ~loong ~riscv ~x64-macos ~x86 ~x86-macossystemd utils static lto test examples disable-arm-neon dynamic-backends curl hbm android msvc +accelerate blas blis +llamafile cann musa cuda cuda-force-mmq cuda-force-cublas +cuda-unified-memory cuda-f16 cuda-no-peer-copy cuda-no-vmm cuda-fa-all-quants +cuda-graphs hip hip-graphs +hip-no-vmm hip-uma vulkan vulkan-check-results vulkan-debug vulkan-memory-debug vulkan-shader-debug-info vulkan-perf vulkan-validate vulkan-run-tests kompute +openmp rpc opencl opencl-profiling +opencl-embed-kernels +opencl-use-adreno-kernels metal metal-use-bf16 metal-ndebug metal-shader-debug +metal-embed-library +cpu webgpu cpu-native cpu-all-variants cpu_flags_x86_avx cpu_flags_x86_avx_vnni cpu_flags_x86_avx2 cpu_flags_x86_avx512 cpu_flags_x86_avx512_vbmi cpu_flags_x86_avx512_vnni cpu_flags_x86_avx512_bf16 cpu_flags_x86_fma cpu_flags_x86_f16c cpu_flags_x86_amx_tile cpu_flags_x86_amx_int8 cpu_flags_x86_amx_bf16 cpu_flags_x86_sse cpu_flags_x86_sse2 cpu_flags_x86_sse3 cpu_flags_x86_sse4 cpu_flags_x86_sse4a cpu_flags_x86_sse41 cpu_flags_x86_sse42 cpu_flags_x86_ssse3 kleidiai cpu_flags_loong_lasx cpu_flags_loong_lsx cpu_flags_riscv_rvv abi_x86_32 abi_x86_64 abi_x86_x32 abi_mips_n32 abi_mips_n64 abi_mips_o32 abi_s390_32 abi_s390_64 +amdgpu_targets_gfx908 +amdgpu_targets_gfx90a +amdgpu_targets_gfx942 +amdgpu_targets_gfx1030 +amdgpu_targets_gfx1100 +amdgpu_targets_gfx1101 +amdgpu_targets_gfx1200 +amdgpu_targets_gfx1201 amdgpu_targets_gfx803 amdgpu_targets_gfx900 amdgpu_targets_gfx906 amdgpu_targets_gfx940 amdgpu_targets_gfx941 amdgpu_targets_gfx1010 amdgpu_targets_gfx1011 amdgpu_targets_gfx1012 amdgpu_targets_gfx1031 amdgpu_targets_gfx1102 amdgpu_targets_gfx1103 amdgpu_targets_gfx1150 amdgpu_targets_gfx1151 python_single_target_python3_11 python_single_target_python3_12
View Download Browse License: MIT Overlay: gentooplusplus
USE Flags
openblas
* This flag is undocumented *
+openmp
* This flag is undocumented *
blis
* This flag is undocumented *
rocm
* This flag is undocumented *
cuda
Global: Build CUDA binaries.
opencl
Global: Build OpenCL binaries.
vulkan
* This flag is undocumented *
flexiblas
* This flag is undocumented *
wmma
* This flag is undocumented *
examples
Global: Install examples, usually source code
rpc
Global: Enable quota interaction via RPC
+server
* This flag is undocumented *
cpu_flags_x86_sse4_2
* This flag is undocumented *
cpu_flags_x86_avx
* This flag is undocumented *
cpu_flags_x86_avx_vnni
* This flag is undocumented *
cpu_flags_x86_avx2
* This flag is undocumented *
cpu_flags_x86_bmi2
* This flag is undocumented *
cpu_flags_x86_avx512f
* This flag is undocumented *
cpu_flags_x86_avx512cd
* This flag is undocumented *
cpu_flags_x86_avx512vl
* This flag is undocumented *
cpu_flags_x86_avx512dq
* This flag is undocumented *
cpu_flags_x86_avx512bw
* This flag is undocumented *
cpu_flags_x86_avx512vbmi
* This flag is undocumented *
cpu_flags_x86_avx512_vnni
* This flag is undocumented *
cpu_flags_x86_avx512_bf16
* This flag is undocumented *
cpu_flags_x86_fma3
* This flag is undocumented *
cpu_flags_x86_f16c
* This flag is undocumented *
cpu_flags_x86_amx_tile
* This flag is undocumented *
cpu_flags_x86_amx_int8
* This flag is undocumented *
cpu_flags_x86_amx_bf16
* This flag is undocumented *
cpu_flags_riscv_v
* This flag is undocumented *
cpu_flags_riscv_zfh
* This flag is undocumented *
cpu_flags_riscv_zvfh
* This flag is undocumented *
cpu_flags_riscv_zicbop
* This flag is undocumented *
cpu_flags_riscv_zihintpause
* This flag is undocumented *
cpu_flags_riscv_xtheadvector
* This flag is undocumented *
+amdgpu_targets_gfx908
* This flag is undocumented *
+amdgpu_targets_gfx90a
* This flag is undocumented *
+amdgpu_targets_gfx942
* This flag is undocumented *
+amdgpu_targets_gfx1030
* This flag is undocumented *
+amdgpu_targets_gfx1100
* This flag is undocumented *
+amdgpu_targets_gfx1101
* This flag is undocumented *
+amdgpu_targets_gfx1200
* This flag is undocumented *
+amdgpu_targets_gfx1201
* This flag is undocumented *
amdgpu_targets_gfx803
* This flag is undocumented *
amdgpu_targets_gfx900
* This flag is undocumented *
amdgpu_targets_gfx906
* This flag is undocumented *
amdgpu_targets_gfx940
* This flag is undocumented *
amdgpu_targets_gfx941
* This flag is undocumented *
amdgpu_targets_gfx1010
* This flag is undocumented *
amdgpu_targets_gfx1011
* This flag is undocumented *
amdgpu_targets_gfx1012
* This flag is undocumented *
amdgpu_targets_gfx1031
* This flag is undocumented *
amdgpu_targets_gfx1102
* This flag is undocumented *
amdgpu_targets_gfx1103
* This flag is undocumented *
amdgpu_targets_gfx1150
* This flag is undocumented *
amdgpu_targets_gfx1151
* This flag is undocumented *
systemd
* This flag is undocumented *
utils
Global: Install command-line PDF converters and various utilities.
static
Global: !!do not set this during bootstrap!! Causes binaries to be statically linked instead of dynamically
lto
Global: Add support for link-time optimizations (unsupported, use at your own risk).
test
Global: Workaround to pull in packages needed to run with FEATURES=test. Portage-2.1.2 handles this internally, so don't set it in make.conf/package.use anymore
disable-arm-neon
* This flag is undocumented *
dynamic-backends
* This flag is undocumented *
curl
Global: Adds support for client-side URL transfer library
hbm
* This flag is undocumented *
android
* This flag is undocumented *
msvc
* This flag is undocumented *
+accelerate
* This flag is undocumented *
blas
Global: Adds support for the virtual/blas numerical library
+llamafile
* This flag is undocumented *
cann
* This flag is undocumented *
musa
* This flag is undocumented *
cuda-force-mmq
* This flag is undocumented *
cuda-force-cublas
* This flag is undocumented *
+cuda-unified-memory
* This flag is undocumented *
cuda-f16
* This flag is undocumented *
cuda-no-peer-copy
* This flag is undocumented *
cuda-no-vmm
* This flag is undocumented *
cuda-fa-all-quants
* This flag is undocumented *
+cuda-graphs
* This flag is undocumented *
hip
* This flag is undocumented *
hip-graphs
* This flag is undocumented *
+hip-no-vmm
* This flag is undocumented *
hip-uma
* This flag is undocumented *
vulkan-check-results
* This flag is undocumented *
vulkan-debug
* This flag is undocumented *
vulkan-memory-debug
* This flag is undocumented *
vulkan-shader-debug-info
* This flag is undocumented *
vulkan-perf
* This flag is undocumented *
vulkan-validate
* This flag is undocumented *
vulkan-run-tests
* This flag is undocumented *
kompute
* This flag is undocumented *
opencl-profiling
* This flag is undocumented *
+opencl-embed-kernels
* This flag is undocumented *
+opencl-use-adreno-kernels
* This flag is undocumented *
metal
* This flag is undocumented *
metal-use-bf16
* This flag is undocumented *
metal-ndebug
* This flag is undocumented *
metal-shader-debug
* This flag is undocumented *
+metal-embed-library
* This flag is undocumented *
+cpu
* This flag is undocumented *
webgpu
* This flag is undocumented *
cpu-native
* This flag is undocumented *
cpu-all-variants
* This flag is undocumented *
cpu_flags_x86_avx512
* This flag is undocumented *
cpu_flags_x86_avx512_vbmi
* This flag is undocumented *
cpu_flags_x86_fma
* This flag is undocumented *
cpu_flags_x86_sse
* This flag is undocumented *
cpu_flags_x86_sse2
* This flag is undocumented *
cpu_flags_x86_sse3
* This flag is undocumented *
cpu_flags_x86_sse4
* This flag is undocumented *
cpu_flags_x86_sse4a
* This flag is undocumented *
cpu_flags_x86_sse41
* This flag is undocumented *
cpu_flags_x86_sse42
* This flag is undocumented *
cpu_flags_x86_ssse3
* This flag is undocumented *
kleidiai
* This flag is undocumented *
cpu_flags_loong_lasx
* This flag is undocumented *
cpu_flags_loong_lsx
* This flag is undocumented *
cpu_flags_riscv_rvv
* This flag is undocumented *
abi_x86_32
* This flag is undocumented *
abi_x86_64
* This flag is undocumented *
abi_x86_x32
* This flag is undocumented *
abi_mips_n32
* This flag is undocumented *
abi_mips_n64
* This flag is undocumented *
abi_mips_o32
* This flag is undocumented *
abi_s390_32
* This flag is undocumented *
abi_s390_64
* This flag is undocumented *
python_single_target_python3_11
* This flag is undocumented *
python_single_target_python3_12
* This flag is undocumented *
+curl
* This flag is undocumented *