Gentoo Portage Overlays - dev-python/tokenizers

dev-python/tokenizers

Fast, efficient, Rust-backed tokenizers for Python (HuggingFace)

Screenshots

https://github.com/huggingface/tokenizers
https://pypi.org/project/tokenizers/

tokenizers-0.22.1

~amd64 ~arm64 ~x86

onig hf-hub indicatif fancy-regex test debug python_targets_python3_12 python_targets_python3_13 python_targets_python3_14

View

Download

Browse License: Apache-2.0

Overlay: ha-bleending-edge

ChangeLog USE Flags Dependencies Reverse Deps Related Bugs

Bugs

Bug #	Severity	Platform	Status	Description

These bugs were grabbed from http://bugs.gentoo.org and have only passed a preliminary search using the package title, to do a more through search please visit: http://bugs.gentoo.org.