gpo.zugaina.org

Search Portage & Overlays:

dev-python/curated-tokenizers

Lightweight piece tokenization library

Screenshots

  • curated-tokenizers-2.0.0
    ~amd64 ~x86
    python_targets_python3_11 python_targets_python3_12 python_targets_python3_13 python_targets_python3_14

    View      Download      Browse     License: MIT   
    Overlay: pypi
  • curated-tokenizers-0.0.10
    ~amd64
    test python_targets_python3_12 python_targets_python3_13 python_targets_python3_14 python_targets_python3_15 debug

    View      Download      Browse     License: MIT   
    Overlay: tatsh-overlay
  • curated-tokenizers-0.0.9-r1
    ~amd64 ~arm ~arm64
    python_targets_python3_12 python_targets_python3_13 python_targets_python3_14 debug

    View      Download      Browse     License: MIT   
    Overlay: stuff

Runtime Dependencies

curated-tokenizers-2.0.0





>=dev-python/regex-2022.0[python_targets_python3_11
      (-)
?
,python_targets_python3_12
      (-)
?
,python_targets_python3_13
      (-)
?
,python_targets_python3_14
      (-)
?
]

python_targets_python3_11?
      ( dev-lang/python:3.11 )

python_targets_python3_12?
      ( dev-lang/python:3.12 )

python_targets_python3_13?
      ( dev-lang/python:3.13 )

python_targets_python3_14?
      ( dev-lang/python:3.14 )

curated-tokenizers-0.0.10

dev-cpp/abseil-cpp:=
dev-libs/protobuf:=
sci-ml/sentencepiece




>=dev-python/regex-2022[python_targets_python3_12
      (-)
?
,python_targets_python3_13
      (-)
?
,python_targets_python3_14
      (-)
?
,python_targets_python3_15
      (-)
?
]

python_targets_python3_12?
      ( dev-lang/python:3.12 )

python_targets_python3_13?
      ( dev-lang/python:3.13 )

python_targets_python3_14?
      ( dev-lang/python:3.14 )

python_targets_python3_15?
      ( dev-lang/python:3.15 )

curated-tokenizers-0.0.9-r1


python_targets_python3_12?
      ( dev-lang/python:3.12 )

python_targets_python3_13?
      ( dev-lang/python:3.13 )

python_targets_python3_14?
      ( dev-lang/python:3.14 )




>=dev-python/regex-2022.0.0[python_targets_python3_12
      (-)
?
,python_targets_python3_13
      (-)
?
,python_targets_python3_14
      (-)
?
]

python_targets_python3_12?
      ( dev-lang/python:3.12 )

python_targets_python3_13?
      ( dev-lang/python:3.13 )

python_targets_python3_14?
      ( dev-lang/python:3.14 )