gpo.zugaina.org

Search Portage & Overlays:

dev-python/tiktoken

Fast BPE tokeniser for use with OpenAI's models.

Screenshots

  • tiktoken-0.12.0
    ~amd64
    debug python_targets_python3_11 python_targets_python3_12 python_targets_python3_13 python_targets_python3_14 debug

    View      Download      Browse     License: MIT Apache-2.0 Unicode-3.0   
  • tiktoken-0.12.0
    ~amd64 ~arm64
    test debug python_targets_python3_11 python_targets_python3_12 python_targets_python3_13 python_targets_python3_14 debug

    View      Download      Browse     License: MIT Apache-2.0-with-LLVM-exceptions MIT Unicode-3.0   
    Overlay: guru
  • tiktoken-0.12.0
    ~amd64
    debug python_targets_python3_11 python_targets_python3_12 python_targets_python3_13 python_targets_python3_14 debug

    View      Download      Browse     License: MIT Apache-2.0 Unicode-3.0   
    Overlay: tatsh-overlay

USE Flags

debug
Global: Enable extra debug codepaths, like asserts and extra output. If you want to get meaningful backtraces see http://www.gentoo.org/proj/en/qa/backtraces.xml
python_targets_python3_11
* This flag is undocumented *
python_targets_python3_12
* This flag is undocumented *
python_targets_python3_13
* This flag is undocumented *
python_targets_python3_14
* This flag is undocumented *
test
Global: Workaround to pull in packages needed to run with FEATURES=test. Portage-2.1.2 handles this internally, so don't set it in make.conf/package.use anymore