sci-ml/koboldcpp
All-in-one local AI server (LLM, image, speech, TTS) on ggml/llama.cpp
Screenshots
-
koboldcpp-1.120~amd64cuda rocm +vulkan cpu_flags_x86_avx cpu_flags_x86_avx2 cpu_flags_x86_f16c cpu_flags_x86_fma3 python_single_target_python3_12 python_single_target_python3_13 python_single_target_python3_14 +amdgpu_targets_gfx908 +amdgpu_targets_gfx90a +amdgpu_targets_gfx942 +amdgpu_targets_gfx1030 +amdgpu_targets_gfx1100 +amdgpu_targets_gfx1101 +amdgpu_targets_gfx1200 +amdgpu_targets_gfx1201 amdgpu_targets_gfx803 amdgpu_targets_gfx900 amdgpu_targets_gfx906 amdgpu_targets_gfx940 amdgpu_targets_gfx941 amdgpu_targets_gfx1010 amdgpu_targets_gfx1011 amdgpu_targets_gfx1012 amdgpu_targets_gfx1031 amdgpu_targets_gfx1102 amdgpu_targets_gfx1103 amdgpu_targets_gfx1150 amdgpu_targets_gfx1151
View 
Download 
Browse License: AGPL-3+ MIT Overlay: bentoo
ChangeLog
commit c7319188f83ee52953b4baa103535304648cda88
Author: lucascouts <lucascs@protonmail.com>
Date: Sat Aug 29 13:37:25 2026 -0300
add(metadata/{md5-cache/app-editors/cursor-3.18.9, md5-cache/app-editors/zed-1.19.0_pre20260829, md5-cache/dev-util/codex-0.151.0, md5-cache/dev-util/codex-bin-0.151.0, md5-cache/dev-util/codex-desktop-bin-26.825.41651, md5-cache/dev-util/mesa_clc-26.3.0_pre20260829, md5-cache/media-libs/mesa-26.3.0_pre20260829, md5-cache/media-libs/vulkan-loader-1.4.361_p20260829, md5-cache/sci-ml/koboldcpp-1.120, md5-cache/sys-kernel/linux-firmware-20260829}), mod(.autoupdate/packages.toml), up(app-editors/{cursor-3.17.21 -> 3.18.9, zed-1.19.0_pre20260828 -> 1.19.0_pre20260829}, dev-util/codex{,-bin}-0.150.1 -> 0.151.0, media-libs/{mesa-26.3.0_pre20260828 -> 26.3.0_pre20260829, vulkan-loader-1.4.361_p20260828 -> 1.4.361_p20260829}, sci-ml/koboldcpp-1.119 -> 1.120, sys-kernel/linux-firmware-20260828 -> 20260829)
commit 736e2d100620b0b387d9601f36967377cb02b0d5
Author: lucascouts <lucascs@protonmail.com>
Date: Sun Aug 16 23:32:07 2026 -0300
add(metadata/{md5-cache/app-editors/vim-9.2.0958, md5-cache/app-editors/vim-core-9.2.0958, md5-cache/app-editors/zed-1.17.0_pre20260817, md5-cache/app-office/libreoffice-26.2.6.1, md5-cache/app-office/libreoffice-l10n-26.2.6.1, md5-cache/dev-util/mesa_clc-26.3.0_pre20260817, md5-cache/media-libs/mesa-26.3.0_pre20260817, md5-cache/media-libs/vulkan-layers-1.4.360_p20260816, md5-cache/net-libs/libqmi-1.39.1_pre20260816, md5-cache/sci-ml/koboldcpp-1.119, md5-cache/sci-ml/llama-cpp-0_pre10453, md5-cache/sci-ml/ollama-0.32.14, md5-cache/sci-ml/ollama-bin-0.32.14, md5-cache/sys-kernel/linux-firmware-20260814}), mod(.autoupdate/packages.toml), up(app-editors/{vim-9.2.0957 -> 9.2.0958, vim-core-9.2.0957 -> 9.2.0958, zed-1.17.0_pre20260816 -> 1.17.0_pre20260817}, dev-util/mesa_clc-26.3.0_pre20260815 -> 26.3.0_pre20260817, media-libs/{mesa-26.3.0_pre20260815 -> 26.3.0_pre20260817, vulkan-layers-1.4.360_p20260815 -> 1.4.360_p20260816}, net-libs/libqmi-1.39.1_pre20260814 -> 1.39.1_pre20260816, sci-ml/{koboldcpp-1.118.1 -> 1.119, llama-cpp-0_pre10452 -> 0_pre10453, ollama-0.32.13-r1 -> 0.32.14, ollama-bin-0.32.13 -> 0.32.14}, sys-kernel/linux-firmware-20260706 -> 20260814)
commit 81ab2102426e32af91802191d0f866f0dd2383f3
Author: lucascouts <lucascs@protonmail.com>
Date: Sun Aug 16 08:46:09 2026 -0300
fix(sci-ml/{sherpa-onnx-1.13.5, koboldcpp-1.118.1}): guard cuda.eclass src_prepare
sci-ml/sherpa-onnx failed the merge gate in the PREPARE phase, for every user
without the CUDA toolkit installed -- including USE=-cuda:
Could not execute cuda-config
ERROR: failed (prepare phase): cuda-config not found
Called cuda_src_prepare -> cuda_sanitize -> cuda_gccdir -> die
cuda.eclass does `EXPORT_FUNCTIONS src_prepare`, and its cuda_src_prepare calls
cuda_sanitize unconditionally. sherpa-onnx defined no src_prepare of its own,
so it inherited the eclass one with nothing gating it. The package was
uninstallable without CUDA -- the whole default configuration.
Introduced by this branch: the adopted ebuild hardcoded
CUDAHOSTCXX=/usr/bin/g++-15, and replacing that with cuda_gccdir (correct on
its own) meant inheriting cuda.eclass, which brought the exported phase along
unnoticed. pkgcheck reports nothing: it does not run phases.
sherpa-onnx now defines src_prepare calling cmake_src_prepare (not `default` --
the cmake eclass tracks state that cmake_src_configure depends on) and
cuda_src_prepare only under `use cuda`.
koboldcpp is fixed for the mirror-image defect. Its own src_prepare overrode
the exported one, which is why it passed the gate, but it therefore never ran
cuda_sanitize at all -- so a USE=cuda build passed unsanitized NVCCFLAGS
straight to nvcc. Now calls cuda_src_prepare under the same guard. Its
cuda_gccdir/cuda_add_sandbox uses were already correctly inside `use cuda`.
Verified without root via `ebuild ... clean prepare`: both packages report
"Source prepared." with USE=-cuda. koboldcpp's two gate legs must be re-run,
since this touches a phase they exercised.
Gate results so far: llama-swap 2/2 legs merged, koboldcpp 2/2 merged,
sherpa-onnx 0/2 -- this is the defect it found.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 494447b3a9e9c2b9a276dfcfc7f8346b92fa186d
Author: lucascouts <lucascs@protonmail.com>
Date: Sun Aug 16 07:14:48 2026 -0300
add(sci-ml/{koboldcpp-1.118.1, sherpa-onnx-1.13.5, llama-swap-250}), mod(.autoupdate/packages.toml)
Story 003 Tasks 1, 3 and 4. NOT MERGED ANYWHERE: authoring and pkgcheck only.
No emerge has been run against any of the three, which is why this sits on a
branch instead of master. Story 002 recorded what skipping that gate costs --
six defects shipped, two of them invisible to pkgcheck until resolution was
attempted. The gate commands are in each task in .epic/stories/003-*/tasks.md.
koboldcpp -- adapted from istitov/stuff sci-misc/koboldcpp.
LLAMA_PORTABLE=1 keeps the Makefile off -march=native; cpu_flags_x86_* wired
to its three tiers via LLAMA_NOAVX1/LLAMA_NOAVX2. cuda and rocm each get
their own emake run: a single invocation defines ggml-cuda.o twice and the
HIP recipes silently override the CUDA ones, so USE="cuda rocm" would have
produced a "cuBLAS" library built by hipcc. GPU_TARGETS is set explicitly
because upstream's default appends $(shell amdgpu-arch), sizing the package
by the build host's GPU. Five prebuilt blobs beyond the documented glslc
pair are removed. pkgcheck: zero findings.
sherpa-onnx -- adapted from istitov/stuff, links sci-libs/onnxruntime rather
than bundling. 17 distfiles pre-staged so CMake FetchContent resolves
locally; the .zip pins stay .zip because the cmake modules match those
filenames literally and fall through to the network otherwise (verified in
the 1.13.5 sources; pkgcheck's TarballAvailable is a false positive here and
says so inline). Dropped QA_PREBUILT, which claimed to cover prebuilt
libraries that this ebuild in fact compiles. Fixed a doubled Prefix in the
site-packages relocation: python_get_sitedir already carries EPREFIX and was
being passed to dodir and concatenated with ED. No -bin, deliberately.
llama-swap -- SRC_URI is now solely the upstream tarball; the adopted ebuild
took even the main source from a third party's personal account. Adds
-tags embed_ui, without which USE=ui ran the whole npm build and shipped a
binary with no UI. The systemd unit could never have worked: it used %i and
User=%i but installed as llama-swap.service, and systemd only expands %i for
a name containing '@' -- now installed via systemd_newunit as
llama-swap@.service, with the EnvironmentFile it expects shipped as an
example. Keeps ~arm64, unlike the ggml family: this is pure Go with no cgo
and no arch-specific path, and upstream builds linux-arm64 itself.
Uses ego mod download under RESTRICT="network-sandbox", matching
dev-util/trivy and dev-util/act. This is a KNOWING divergence from story 003
R4.1, which asked for a sandbox-clean default: upstream ships no vendor/
directory, so the alternative commits the overlay to hosting a generated
tarball on distfiles.obentoo.org and to hold = true forever after. The
alternative implementation is preserved and documented; the choice between
them should be made once, for this package and sci-ml/localai together.
All three records download-validate against upstream and pass the bentoo
checker. KEYWORDS is ~amd64 for the two C++ packages, consistent with the
decision taken across the ggml family: an unvalidated keyword is worse than
none.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Author: lucascouts <lucascs@protonmail.com>
Date: Sat Aug 29 13:37:25 2026 -0300
add(metadata/{md5-cache/app-editors/cursor-3.18.9, md5-cache/app-editors/zed-1.19.0_pre20260829, md5-cache/dev-util/codex-0.151.0, md5-cache/dev-util/codex-bin-0.151.0, md5-cache/dev-util/codex-desktop-bin-26.825.41651, md5-cache/dev-util/mesa_clc-26.3.0_pre20260829, md5-cache/media-libs/mesa-26.3.0_pre20260829, md5-cache/media-libs/vulkan-loader-1.4.361_p20260829, md5-cache/sci-ml/koboldcpp-1.120, md5-cache/sys-kernel/linux-firmware-20260829}), mod(.autoupdate/packages.toml), up(app-editors/{cursor-3.17.21 -> 3.18.9, zed-1.19.0_pre20260828 -> 1.19.0_pre20260829}, dev-util/codex{,-bin}-0.150.1 -> 0.151.0, media-libs/{mesa-26.3.0_pre20260828 -> 26.3.0_pre20260829, vulkan-loader-1.4.361_p20260828 -> 1.4.361_p20260829}, sci-ml/koboldcpp-1.119 -> 1.120, sys-kernel/linux-firmware-20260828 -> 20260829)
commit 736e2d100620b0b387d9601f36967377cb02b0d5
Author: lucascouts <lucascs@protonmail.com>
Date: Sun Aug 16 23:32:07 2026 -0300
add(metadata/{md5-cache/app-editors/vim-9.2.0958, md5-cache/app-editors/vim-core-9.2.0958, md5-cache/app-editors/zed-1.17.0_pre20260817, md5-cache/app-office/libreoffice-26.2.6.1, md5-cache/app-office/libreoffice-l10n-26.2.6.1, md5-cache/dev-util/mesa_clc-26.3.0_pre20260817, md5-cache/media-libs/mesa-26.3.0_pre20260817, md5-cache/media-libs/vulkan-layers-1.4.360_p20260816, md5-cache/net-libs/libqmi-1.39.1_pre20260816, md5-cache/sci-ml/koboldcpp-1.119, md5-cache/sci-ml/llama-cpp-0_pre10453, md5-cache/sci-ml/ollama-0.32.14, md5-cache/sci-ml/ollama-bin-0.32.14, md5-cache/sys-kernel/linux-firmware-20260814}), mod(.autoupdate/packages.toml), up(app-editors/{vim-9.2.0957 -> 9.2.0958, vim-core-9.2.0957 -> 9.2.0958, zed-1.17.0_pre20260816 -> 1.17.0_pre20260817}, dev-util/mesa_clc-26.3.0_pre20260815 -> 26.3.0_pre20260817, media-libs/{mesa-26.3.0_pre20260815 -> 26.3.0_pre20260817, vulkan-layers-1.4.360_p20260815 -> 1.4.360_p20260816}, net-libs/libqmi-1.39.1_pre20260814 -> 1.39.1_pre20260816, sci-ml/{koboldcpp-1.118.1 -> 1.119, llama-cpp-0_pre10452 -> 0_pre10453, ollama-0.32.13-r1 -> 0.32.14, ollama-bin-0.32.13 -> 0.32.14}, sys-kernel/linux-firmware-20260706 -> 20260814)
commit 81ab2102426e32af91802191d0f866f0dd2383f3
Author: lucascouts <lucascs@protonmail.com>
Date: Sun Aug 16 08:46:09 2026 -0300
fix(sci-ml/{sherpa-onnx-1.13.5, koboldcpp-1.118.1}): guard cuda.eclass src_prepare
sci-ml/sherpa-onnx failed the merge gate in the PREPARE phase, for every user
without the CUDA toolkit installed -- including USE=-cuda:
Could not execute cuda-config
ERROR: failed (prepare phase): cuda-config not found
Called cuda_src_prepare -> cuda_sanitize -> cuda_gccdir -> die
cuda.eclass does `EXPORT_FUNCTIONS src_prepare`, and its cuda_src_prepare calls
cuda_sanitize unconditionally. sherpa-onnx defined no src_prepare of its own,
so it inherited the eclass one with nothing gating it. The package was
uninstallable without CUDA -- the whole default configuration.
Introduced by this branch: the adopted ebuild hardcoded
CUDAHOSTCXX=/usr/bin/g++-15, and replacing that with cuda_gccdir (correct on
its own) meant inheriting cuda.eclass, which brought the exported phase along
unnoticed. pkgcheck reports nothing: it does not run phases.
sherpa-onnx now defines src_prepare calling cmake_src_prepare (not `default` --
the cmake eclass tracks state that cmake_src_configure depends on) and
cuda_src_prepare only under `use cuda`.
koboldcpp is fixed for the mirror-image defect. Its own src_prepare overrode
the exported one, which is why it passed the gate, but it therefore never ran
cuda_sanitize at all -- so a USE=cuda build passed unsanitized NVCCFLAGS
straight to nvcc. Now calls cuda_src_prepare under the same guard. Its
cuda_gccdir/cuda_add_sandbox uses were already correctly inside `use cuda`.
Verified without root via `ebuild ... clean prepare`: both packages report
"Source prepared." with USE=-cuda. koboldcpp's two gate legs must be re-run,
since this touches a phase they exercised.
Gate results so far: llama-swap 2/2 legs merged, koboldcpp 2/2 merged,
sherpa-onnx 0/2 -- this is the defect it found.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
commit 494447b3a9e9c2b9a276dfcfc7f8346b92fa186d
Author: lucascouts <lucascs@protonmail.com>
Date: Sun Aug 16 07:14:48 2026 -0300
add(sci-ml/{koboldcpp-1.118.1, sherpa-onnx-1.13.5, llama-swap-250}), mod(.autoupdate/packages.toml)
Story 003 Tasks 1, 3 and 4. NOT MERGED ANYWHERE: authoring and pkgcheck only.
No emerge has been run against any of the three, which is why this sits on a
branch instead of master. Story 002 recorded what skipping that gate costs --
six defects shipped, two of them invisible to pkgcheck until resolution was
attempted. The gate commands are in each task in .epic/stories/003-*/tasks.md.
koboldcpp -- adapted from istitov/stuff sci-misc/koboldcpp.
LLAMA_PORTABLE=1 keeps the Makefile off -march=native; cpu_flags_x86_* wired
to its three tiers via LLAMA_NOAVX1/LLAMA_NOAVX2. cuda and rocm each get
their own emake run: a single invocation defines ggml-cuda.o twice and the
HIP recipes silently override the CUDA ones, so USE="cuda rocm" would have
produced a "cuBLAS" library built by hipcc. GPU_TARGETS is set explicitly
because upstream's default appends $(shell amdgpu-arch), sizing the package
by the build host's GPU. Five prebuilt blobs beyond the documented glslc
pair are removed. pkgcheck: zero findings.
sherpa-onnx -- adapted from istitov/stuff, links sci-libs/onnxruntime rather
than bundling. 17 distfiles pre-staged so CMake FetchContent resolves
locally; the .zip pins stay .zip because the cmake modules match those
filenames literally and fall through to the network otherwise (verified in
the 1.13.5 sources; pkgcheck's TarballAvailable is a false positive here and
says so inline). Dropped QA_PREBUILT, which claimed to cover prebuilt
libraries that this ebuild in fact compiles. Fixed a doubled Prefix in the
site-packages relocation: python_get_sitedir already carries EPREFIX and was
being passed to dodir and concatenated with ED. No -bin, deliberately.
llama-swap -- SRC_URI is now solely the upstream tarball; the adopted ebuild
took even the main source from a third party's personal account. Adds
-tags embed_ui, without which USE=ui ran the whole npm build and shipped a
binary with no UI. The systemd unit could never have worked: it used %i and
User=%i but installed as llama-swap.service, and systemd only expands %i for
a name containing '@' -- now installed via systemd_newunit as
llama-swap@.service, with the EnvironmentFile it expects shipped as an
example. Keeps ~arm64, unlike the ggml family: this is pure Go with no cgo
and no arch-specific path, and upstream builds linux-arm64 itself.
Uses ego mod download under RESTRICT="network-sandbox", matching
dev-util/trivy and dev-util/act. This is a KNOWING divergence from story 003
R4.1, which asked for a sandbox-clean default: upstream ships no vendor/
directory, so the alternative commits the overlay to hosting a generated
tarball on distfiles.obentoo.org and to hold = true forever after. The
alternative implementation is preserved and documented; the choice between
them should be made once, for this package and sci-ml/localai together.
All three records download-validate against upstream and pass the bentoo
checker. KEYWORDS is ~amd64 for the two C++ packages, consistent with the
decision taken across the ggml family: an unvalidated keyword is worse than
none.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>