sci-ml/fastflowlm
NPU-first LLM runtime for AMD Ryzen AI (XDNA2) processors
ChangeLog
commit 7c384c8cac70105c06770e0ccb7af0fc295f2471
Author: Raukaan Cogbrother <cogbrother@raukaan.local>
Date: Sun May 31 03:29:35 2026 +0200
metadata: normalize tab → 2-space indent in 42 imported files
42 metadata.xml files inherited tab indentation from their origin
overlays (mostly the ROCm cluster, plus a handful of dev-python and
AMD-tooling imports). The overlay convention is 2-space; this brings
them in line with the other 498 files. Maintainer attributions
preserved verbatim.
commit dc43643ab26b51b7d29843f8879f257ff10b76e3
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Wed May 27 18:20:35 2026 +0200
sci-ml/fastflowlm: set CMAKE_XCLBIN_PREFIX so the runtime finds xclbins
Upstream's linux-default cmake preset sets it to match the install
rule's share/flm DESTINATION; src_configure doesn't drive the build
via the preset, so without an explicit -D the macro was baked as ""
and the runtime fell back to dirname(/proc/self/exe), looking under
bin/xclbins/ instead of share/flm/xclbins/. #269.
commit 97b222aeced2669a0c84267c1de0b8a6574292e2
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Wed May 27 13:14:09 2026 +0200
sci-misc/llama-swap, sci-ml/fastflowlm: add openrc + systemd services
Both get supervise-daemon-driven openrc init.d/conf.d and a systemd
per-instance template (pkg@<user>.service). Runners are USE-gated
and default off; the existing manual-start flow stays the unchanged
baseline.
llama-swap openrc refuses to start until LLAMA_SWAP_USER is set;
auto-derives LLAMA_SWAP_CONFIG from that user's $HOME via getent if
unset. Listener defaults to 127.0.0.1:8080 so a fresh install doesn't
expose the LLM API to the LAN. systemd unit has no default
LLAMA_SWAP_CONFIG -- systemd's %h resolves to /root for system-
manager units and /home/%i would bake in a passwd layout we can't
promise -- so EnvironmentFile=/etc/default/llama-swap@%i is required.
fastflowlm openrc refuses to start until FLM_USER is set; rc_ulimit
'-l unlimited' (and LimitMEMLOCK=infinity in systemd) is needed
because flm mlocks NPU buffers. systemd ExecStart goes through
/bin/sh -c with $$ escapes so ${FLM_PORT:+--port "$FLM_PORT"}
parameter expansion runs in the shell -- systemd's variable parser
has no :+ semantics.
Hardening:
- llama-swap: ProtectSystem=full rather than =strict so backends
spawned by it (llama.cpp et al.) can still write to ~/.cache/.
- fastflowlm: deliberately omits ProtectKernelTunables (NPU power-
mode may touch /sys/) and MemoryDenyWriteExecute (XDNA path may
use JIT); revisit once empirically verified safe.
Service files live at files/<pkg>.service (no @) because pkgcheck
BannedCharacter rejects @ in files/* filenames; systemd_newunit's
target arg adds the @ at install.
commit 73ec11612adb4a09558b153fa200ad19050f992b
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Wed May 27 11:57:40 2026 +0200
sci-ml/fastflowlm: add 0.9.43
Only ebuild-level change: TOKENIZERS_CPP_COMMIT reverted from 0.9.42's
34885cf back to acbdc5a (the 0.9.41-and-earlier pin). nested
sentencepiece + msgpack pins at acbdc5a still match the existing
values, so MSGPACK_COMMIT and SENTENCEPIECE_COMMIT stay put.
src/CMakeLists.txt is byte-identical to 0.9.42, so the sed pattern
stripping the symlink-into-/usr/local/bin block and the
NPU_VERSION/FLM_VERSION cmake args still apply unchanged. Verification
stamp re-dated accordingly.
Upstream substance is mostly in the model code: a new
common/whisper/ subsystem (which links a libwhisper_npu .so into the
binary) and a Whisper_Config : LM_Config subclass land in 0.9.43.
src/include/utils/json_check.hpp -- the validator that triggered
the #545 Whisper crash -- was deleted, which may mean the workaround
in flm-patch-whisper / the ewarn is no longer necessary. Conservative
call: keep both for one release cycle; if no user reports needing
the patch we can drop the ewarn at the next bump.
Build-checked locally: 91/91 ninja targets clean against
=fastflowlm-0.9.43 (default USE; no IUSE flags).
commit b17a9c755ac32614184a0cf85eb1eb0f8c6a13d3
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Mon May 25 16:07:27 2026 +0200
sci-ml/fastflowlm: ship flm-patch-whisper helper (upstream #545)
flm crashes at startup on any invocation that loads whisper-v3:turbo
because LM_Config::from_pretrained asserts on decoder-only LM-shape
fields (hidden_size, intermediate_size, num_attention_heads,
num_hidden_layers, num_key_value_heads) that HuggingFace's Whisper
config.json doesn't carry — Whisper uses the encoder-decoder
equivalents (d_model, encoder_ffn_dim, encoder_attention_heads,
encoder_layers) instead.
Ship a small Python helper at /usr/bin/flm-patch-whisper that walks
~/.config/flm/models/* (honoring $FLM_MODEL_PATH), finds Whisper
configs missing the decoder-only fields, and adds them with values
mapped from the encoder-* keys; idempotent on repeat runs. A
pkg_postinst ewarn points users at it as the required follow-up
after `flm pull whisper-v3:turbo`. Drop both once upstream merges
a real fix for issue #545.
commit d7891d49aca438b2da5ea2a5aceaf7f7dcfe40fd
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Thu May 14 11:49:41 2026 +0200
sci-ml/fastflowlm: add 0.9.42
commit c79366016e1891d1f4f488e0357c8b134de7b160
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Wed May 13 14:15:10 2026 +0200
sci-ml/fastflowlm: drop 0.9.38, 0.9.39
commit 27ae6103d77b7d627bf17bd5517801eada91dcd7
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Mon May 11 21:11:45 2026 +0200
sci-ml/fastflowlm: add 0.9.38
Older release retained alongside the newer ones. Third-party submodule
pins (tokenizers-cpp, sentencepiece, msgpack) are identical to 0.9.39's,
so the corresponding distfile entries already exist in the Manifest;
only fastflowlm-0.9.38.tar.gz is newly tracked.
commit e2f3b66d00008de38b637d8d7da84335eb1061db
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Mon May 11 17:34:26 2026 +0200
sci-ml/fastflowlm: add missing direct RDEPENDs
The flm binary links directly against libav{format,codec,util},
libsw{resample,scale}, libcurl, libboost_program_options, libfftw3f,
and libreadline (per readelf -d on /opt/fastflowlm/bin/flm), none of
which were declared. Build and merge succeed on systems where these
libraries are commonly installed; a minimal box could break after
depclean with no dependency trail.
Adds ffmpeg, curl, boost, fftw:3.0, and readline (subslot operators
for ABI tracking) to all four ebuild versions. Matches the Arch
official package's depends list and the build-deps documented in
upstream docs/linux-getting-started.md. Metadata-only — same binary
contents, same versions, just honest dependency declaration.
commit 95cd21b4cb4c970cd0bd2f8361bcf4f55aef2c51
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Sun May 10 00:29:00 2026 +0200
sci-ml/fastflowlm: drop www. prefix from HOMEPAGE
fastflowlm.com 308-redirects www.fastflowlm.com to the bare apex. Strip
the www. prefix across all four ebuilds (0.9.39/40/41 + 9999) so
pkgcheck stops reporting RedirectedUrl.
commit 262574fdcdf7dc7ee8c0257a476a8003bdd42e27
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Wed May 6 19:28:35 2026 +0200
sci-ml/fastflowlm: fix BDEPEND, sync -9999 with tagged ebuilds
- Replace nonexistent virtual/rust with || ( dev-lang/rust
dev-lang/rust-bin ) on 0.9.39, 0.9.40, 0.9.41.
- Add the same rust BDEPEND to -9999 (cargo via tokenizers-cpp/rust
is fetched and built on the live ebuild too).
- Add PROPERTIES=live + RESTRICT=network-sandbox to -9999 — same
cargo-fetch-online behavior that motivates them on the tagged
ebuilds; without RESTRICT, the build fails under sandbox.
- Carry the memlock/limits.d hint into -9999's pkg_postinst so the
NPU stack guidance is consistent across release and live.
commit 78b5135ded7e76a3cb7cf92b257b3334dc618063
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Wed May 6 19:15:28 2026 +0200
sci-ml/fastflowlm: add 0.9.40
Same submodule pin set as 0.9.39 (tokenizers-cpp acbdc5a, plus
unchanged sentencepiece + msgpack); ebuild byte-identical except
for filename.
commit d518de1486ae9354e400597aba4f888bf9c8ef6b
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Wed May 6 19:13:36 2026 +0200
sci-ml/fastflowlm: new package, add 9999 + 0.9.39 + 0.9.41
NPU-first LLM runtime for AMD Ryzen AI XDNA2 processors.
Three ebuilds: live -9999 tracks upstream main (EGIT_SUBMODULES='*'),
0.9.39 + 0.9.41 are tagged releases with submodule commit pins
(tokenizers-cpp + nested sentencepiece + msgpack-c) baked into SRC_URI.
0.9.41 bumps tokenizers-cpp from acbdc5a to 34885cf; the nested pins
are unchanged.
Build keeps PROPERTIES=live + RESTRICT=network-sandbox because cargo
inside tokenizers-cpp/rust fetches crates online. src_prepare strips
upstream's symlink-into-/usr/local/bin block; we wrap flm via newbin
and env.d to set LD_LIBRARY_PATH for XRT + FLM_CONFIG_PATH defaults.
LICENSE='MIT FastFlowLM-Binary': orchestration MIT (LICENSE_RUNTIME),
NPU compute kernels proprietary (LICENSE_BINARY, free for commercial
use under USD 10M annual revenue).
RDEPEND on dev-util/xrt + dev-libs/{xdna-driver,xrt-xdna} not yet
present in this overlay or ::gentoo; ride along ::guru's xrt-2.21.75
stack for now (cascade in a follow-up commit).
Author: Raukaan Cogbrother <cogbrother@raukaan.local>
Date: Sun May 31 03:29:35 2026 +0200
metadata: normalize tab → 2-space indent in 42 imported files
42 metadata.xml files inherited tab indentation from their origin
overlays (mostly the ROCm cluster, plus a handful of dev-python and
AMD-tooling imports). The overlay convention is 2-space; this brings
them in line with the other 498 files. Maintainer attributions
preserved verbatim.
commit dc43643ab26b51b7d29843f8879f257ff10b76e3
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Wed May 27 18:20:35 2026 +0200
sci-ml/fastflowlm: set CMAKE_XCLBIN_PREFIX so the runtime finds xclbins
Upstream's linux-default cmake preset sets it to match the install
rule's share/flm DESTINATION; src_configure doesn't drive the build
via the preset, so without an explicit -D the macro was baked as ""
and the runtime fell back to dirname(/proc/self/exe), looking under
bin/xclbins/ instead of share/flm/xclbins/. #269.
commit 97b222aeced2669a0c84267c1de0b8a6574292e2
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Wed May 27 13:14:09 2026 +0200
sci-misc/llama-swap, sci-ml/fastflowlm: add openrc + systemd services
Both get supervise-daemon-driven openrc init.d/conf.d and a systemd
per-instance template (pkg@<user>.service). Runners are USE-gated
and default off; the existing manual-start flow stays the unchanged
baseline.
llama-swap openrc refuses to start until LLAMA_SWAP_USER is set;
auto-derives LLAMA_SWAP_CONFIG from that user's $HOME via getent if
unset. Listener defaults to 127.0.0.1:8080 so a fresh install doesn't
expose the LLM API to the LAN. systemd unit has no default
LLAMA_SWAP_CONFIG -- systemd's %h resolves to /root for system-
manager units and /home/%i would bake in a passwd layout we can't
promise -- so EnvironmentFile=/etc/default/llama-swap@%i is required.
fastflowlm openrc refuses to start until FLM_USER is set; rc_ulimit
'-l unlimited' (and LimitMEMLOCK=infinity in systemd) is needed
because flm mlocks NPU buffers. systemd ExecStart goes through
/bin/sh -c with $$ escapes so ${FLM_PORT:+--port "$FLM_PORT"}
parameter expansion runs in the shell -- systemd's variable parser
has no :+ semantics.
Hardening:
- llama-swap: ProtectSystem=full rather than =strict so backends
spawned by it (llama.cpp et al.) can still write to ~/.cache/.
- fastflowlm: deliberately omits ProtectKernelTunables (NPU power-
mode may touch /sys/) and MemoryDenyWriteExecute (XDNA path may
use JIT); revisit once empirically verified safe.
Service files live at files/<pkg>.service (no @) because pkgcheck
BannedCharacter rejects @ in files/* filenames; systemd_newunit's
target arg adds the @ at install.
commit 73ec11612adb4a09558b153fa200ad19050f992b
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Wed May 27 11:57:40 2026 +0200
sci-ml/fastflowlm: add 0.9.43
Only ebuild-level change: TOKENIZERS_CPP_COMMIT reverted from 0.9.42's
34885cf back to acbdc5a (the 0.9.41-and-earlier pin). nested
sentencepiece + msgpack pins at acbdc5a still match the existing
values, so MSGPACK_COMMIT and SENTENCEPIECE_COMMIT stay put.
src/CMakeLists.txt is byte-identical to 0.9.42, so the sed pattern
stripping the symlink-into-/usr/local/bin block and the
NPU_VERSION/FLM_VERSION cmake args still apply unchanged. Verification
stamp re-dated accordingly.
Upstream substance is mostly in the model code: a new
common/whisper/ subsystem (which links a libwhisper_npu .so into the
binary) and a Whisper_Config : LM_Config subclass land in 0.9.43.
src/include/utils/json_check.hpp -- the validator that triggered
the #545 Whisper crash -- was deleted, which may mean the workaround
in flm-patch-whisper / the ewarn is no longer necessary. Conservative
call: keep both for one release cycle; if no user reports needing
the patch we can drop the ewarn at the next bump.
Build-checked locally: 91/91 ninja targets clean against
=fastflowlm-0.9.43 (default USE; no IUSE flags).
commit b17a9c755ac32614184a0cf85eb1eb0f8c6a13d3
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Mon May 25 16:07:27 2026 +0200
sci-ml/fastflowlm: ship flm-patch-whisper helper (upstream #545)
flm crashes at startup on any invocation that loads whisper-v3:turbo
because LM_Config::from_pretrained asserts on decoder-only LM-shape
fields (hidden_size, intermediate_size, num_attention_heads,
num_hidden_layers, num_key_value_heads) that HuggingFace's Whisper
config.json doesn't carry — Whisper uses the encoder-decoder
equivalents (d_model, encoder_ffn_dim, encoder_attention_heads,
encoder_layers) instead.
Ship a small Python helper at /usr/bin/flm-patch-whisper that walks
~/.config/flm/models/* (honoring $FLM_MODEL_PATH), finds Whisper
configs missing the decoder-only fields, and adds them with values
mapped from the encoder-* keys; idempotent on repeat runs. A
pkg_postinst ewarn points users at it as the required follow-up
after `flm pull whisper-v3:turbo`. Drop both once upstream merges
a real fix for issue #545.
commit d7891d49aca438b2da5ea2a5aceaf7f7dcfe40fd
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Thu May 14 11:49:41 2026 +0200
sci-ml/fastflowlm: add 0.9.42
commit c79366016e1891d1f4f488e0357c8b134de7b160
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Wed May 13 14:15:10 2026 +0200
sci-ml/fastflowlm: drop 0.9.38, 0.9.39
commit 27ae6103d77b7d627bf17bd5517801eada91dcd7
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Mon May 11 21:11:45 2026 +0200
sci-ml/fastflowlm: add 0.9.38
Older release retained alongside the newer ones. Third-party submodule
pins (tokenizers-cpp, sentencepiece, msgpack) are identical to 0.9.39's,
so the corresponding distfile entries already exist in the Manifest;
only fastflowlm-0.9.38.tar.gz is newly tracked.
commit e2f3b66d00008de38b637d8d7da84335eb1061db
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Mon May 11 17:34:26 2026 +0200
sci-ml/fastflowlm: add missing direct RDEPENDs
The flm binary links directly against libav{format,codec,util},
libsw{resample,scale}, libcurl, libboost_program_options, libfftw3f,
and libreadline (per readelf -d on /opt/fastflowlm/bin/flm), none of
which were declared. Build and merge succeed on systems where these
libraries are commonly installed; a minimal box could break after
depclean with no dependency trail.
Adds ffmpeg, curl, boost, fftw:3.0, and readline (subslot operators
for ABI tracking) to all four ebuild versions. Matches the Arch
official package's depends list and the build-deps documented in
upstream docs/linux-getting-started.md. Metadata-only — same binary
contents, same versions, just honest dependency declaration.
commit 95cd21b4cb4c970cd0bd2f8361bcf4f55aef2c51
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Sun May 10 00:29:00 2026 +0200
sci-ml/fastflowlm: drop www. prefix from HOMEPAGE
fastflowlm.com 308-redirects www.fastflowlm.com to the bare apex. Strip
the www. prefix across all four ebuilds (0.9.39/40/41 + 9999) so
pkgcheck stops reporting RedirectedUrl.
commit 262574fdcdf7dc7ee8c0257a476a8003bdd42e27
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Wed May 6 19:28:35 2026 +0200
sci-ml/fastflowlm: fix BDEPEND, sync -9999 with tagged ebuilds
- Replace nonexistent virtual/rust with || ( dev-lang/rust
dev-lang/rust-bin ) on 0.9.39, 0.9.40, 0.9.41.
- Add the same rust BDEPEND to -9999 (cargo via tokenizers-cpp/rust
is fetched and built on the live ebuild too).
- Add PROPERTIES=live + RESTRICT=network-sandbox to -9999 — same
cargo-fetch-online behavior that motivates them on the tagged
ebuilds; without RESTRICT, the build fails under sandbox.
- Carry the memlock/limits.d hint into -9999's pkg_postinst so the
NPU stack guidance is consistent across release and live.
commit 78b5135ded7e76a3cb7cf92b257b3334dc618063
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Wed May 6 19:15:28 2026 +0200
sci-ml/fastflowlm: add 0.9.40
Same submodule pin set as 0.9.39 (tokenizers-cpp acbdc5a, plus
unchanged sentencepiece + msgpack); ebuild byte-identical except
for filename.
commit d518de1486ae9354e400597aba4f888bf9c8ef6b
Author: Ivan S. Titov <iohann.s.titov@gmail.com>
Date: Wed May 6 19:13:36 2026 +0200
sci-ml/fastflowlm: new package, add 9999 + 0.9.39 + 0.9.41
NPU-first LLM runtime for AMD Ryzen AI XDNA2 processors.
Three ebuilds: live -9999 tracks upstream main (EGIT_SUBMODULES='*'),
0.9.39 + 0.9.41 are tagged releases with submodule commit pins
(tokenizers-cpp + nested sentencepiece + msgpack-c) baked into SRC_URI.
0.9.41 bumps tokenizers-cpp from acbdc5a to 34885cf; the nested pins
are unchanged.
Build keeps PROPERTIES=live + RESTRICT=network-sandbox because cargo
inside tokenizers-cpp/rust fetches crates online. src_prepare strips
upstream's symlink-into-/usr/local/bin block; we wrap flm via newbin
and env.d to set LD_LIBRARY_PATH for XRT + FLM_CONFIG_PATH defaults.
LICENSE='MIT FastFlowLM-Binary': orchestration MIT (LICENSE_RUNTIME),
NPU compute kernels proprietary (LICENSE_BINARY, free for commercial
use under USD 10M annual revenue).
RDEPEND on dev-util/xrt + dev-libs/{xdna-driver,xrt-xdna} not yet
present in this overlay or ::gentoo; ride along ::guru's xrt-2.21.75
stack for now (cascade in a follow-up commit).


View
Download
Browse