localai-backend/llama-cpp
LocalAI text-generation backend (llama.cpp gRPC server)
Screenshots
-
llama-cpp-4.12.1~amd64test +amdgpu_targets_gfx908 +amdgpu_targets_gfx90a +amdgpu_targets_gfx942 +amdgpu_targets_gfx950 +amdgpu_targets_gfx1030 +amdgpu_targets_gfx1100 +amdgpu_targets_gfx1101 +amdgpu_targets_gfx1102 +amdgpu_targets_gfx1103 +amdgpu_targets_gfx1150 +amdgpu_targets_gfx1151 +amdgpu_targets_gfx1152 +amdgpu_targets_gfx1153 +amdgpu_targets_gfx1200 +amdgpu_targets_gfx1201 amdgpu_targets_gfx803 amdgpu_targets_gfx900 amdgpu_targets_gfx906 amdgpu_targets_gfx90c amdgpu_targets_gfx1010 amdgpu_targets_gfx1011 amdgpu_targets_gfx1012 amdgpu_targets_gfx1031 amdgpu_targets_gfx1032 amdgpu_targets_gfx1033 amdgpu_targets_gfx1034 amdgpu_targets_gfx1035 amdgpu_targets_gfx1036 native openblas +openmp vulkan video_cards_amdgpu cuda rocm cpu_flags_x86_amx_bf16 cpu_flags_x86_amx_int8 cpu_flags_x86_amx_tile cpu_flags_x86_avx cpu_flags_x86_avx_vnni cpu_flags_x86_avx2 cpu_flags_x86_avx512_bf16 cpu_flags_x86_avx512bw cpu_flags_x86_avx512f cpu_flags_x86_avx512vbmi cpu_flags_x86_avx512_vnni cpu_flags_x86_bmi2 cpu_flags_x86_fma3 cpu_flags_x86_f16c cpu_flags_x86_sse4_2
View 
Download 
Browse License: MIT Overlay: local-ai
Runtime Dependencies
llama-cpp-4.12.1
net-misc/curl
dev-cpp/abseil-cpp:=
dev-libs/protobuf:=
net-libs/grpc:=
openblas?
( sci-libs/openblas )
vulkan?
( media-libs/vulkan-loader video_cards_amdgpu?
( media-libs/mesa[vulkan,video_cards_radeonsi] )
)
cuda?
( dev-util/nvidia-cuda-toolkit:= )
rocm?
( >=dev-util/hip-10.1:= >=sci-libs/hipBLAS-10.1:= >=sci-libs/rocBLAS-10.1:= )
dev-cpp/abseil-cpp:=
dev-libs/protobuf:=
net-libs/grpc:=
openblas?
( sci-libs/openblas )
vulkan?
( media-libs/vulkan-loader video_cards_amdgpu?
( media-libs/mesa[vulkan,video_cards_radeonsi] )
)
cuda?
( dev-util/nvidia-cuda-toolkit:= )
rocm?
( >=dev-util/hip-10.1:= >=sci-libs/hipBLAS-10.1:= >=sci-libs/rocBLAS-10.1:= )