dev-python/tiktoken
Fast BPE tokeniser for use with OpenAI's models.
-
tiktoken-0.12.0~amd64debug python_targets_python3_11 python_targets_python3_12 python_targets_python3_13 debug
View
Download
Browse License: MIT Apache-2.0 Unicode-3.0 Overlay: tatsh-overlay
Runtime Dependencies
tiktoken-0.12.0
>=dev-python/regex-2022.1.18[python_targets_python3_11
(-)
?
,python_targets_python3_12
(-)
?
,python_targets_python3_13
(-)
?
]
>=dev-python/requests-2.26.0[python_targets_python3_11
(-)
?
,python_targets_python3_12
(-)
?
,python_targets_python3_13
(-)
?
]
python_targets_python3_11?
( dev-lang/python:3.11 )
python_targets_python3_12?
( dev-lang/python:3.12 )
python_targets_python3_13?
( dev-lang/python:3.13 )

