dev-python/curated-tokenizers
Lightweight piece tokenization library.
-
curated-tokenizers-0.0.9~amd64python_targets_python3_11 python_targets_python3_12 python_targets_python3_13
View
Download
Browse License: MIT Overlay: tatsh-overlay
Runtime Dependencies
curated-tokenizers-0.0.9
>=dev-python/regex-2022[python_targets_python3_11
(-)
?
,python_targets_python3_12
(-)
?
,python_targets_python3_13
(-)
?
]
dev-cpp/abseil-cpp
dev-libs/protobuf
sci-ml/sentencepiece
python_targets_python3_11?
( dev-lang/python:3.11 )
python_targets_python3_12?
( dev-lang/python:3.12 )
python_targets_python3_13?
( dev-lang/python:3.13 )

