gpo.zugaina.org

Search Portage & Overlays:

sci-ml/tokenizers

Implementation of today's most used tokenizers

Screenshots

  • tokenizers-0.23.2
    ~amd64 ~arm64
    test debug python_single_target_python3_12 python_single_target_python3_13 python_single_target_python3_14 debug

    View      Download      Browse     License: Apache-2.0 Apache-2.0 Apache-2.0-with-LLVM-exceptions BSD BSD-2 CDLA-Permissive-2.0 ISC MIT MPL-2.0 Unicode-3.0 ZLIB   
    Overlay: stuff
  • tokenizers-0.23.1
    ~amd64 ~arm64
    test debug python_single_target_python3_12 python_single_target_python3_13 python_single_target_python3_14 debug

    View      Download      Browse     License: Apache-2.0 Apache-2.0 Apache-2.0-with-LLVM-exceptions BSD-2 BSD ISC MIT MPL-2.0 Unicode-DFS-2016   
    Overlay: stuff
  • tokenizers-0.22.2
    ~amd64 ~arm64
    test debug python_single_target_python3_12 python_single_target_python3_13 python_single_target_python3_14 debug

    View      Download      Browse     License: Apache-2.0 Apache-2.0 Apache-2.0-with-LLVM-exceptions BSD-2 BSD ISC MIT MPL-2.0 Unicode-DFS-2016   
    Overlay: stuff
  • tokenizers-0.22.1
    ~amd64 ~arm64
    test debug python_single_target_python3_12 python_single_target_python3_13 python_single_target_python3_14 debug

    View      Download      Browse     License: Apache-2.0 Apache-2.0 Apache-2.0-with-LLVM-exceptions BSD-2 BSD ISC MIT MPL-2.0 Unicode-DFS-2016   
    Overlay: stuff