nixpkgs/pkgs/development/python-modules/tokenizers
Daniël de Kok d2e918cc12 python3Packages.tokenizers: 0.8.1 -> 0.9.2
Changelog:

https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.0
https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.1
https://github.com/huggingface/tokenizers/releases/tag/python-v0.9.2

Changes in the derivation:

* Tokenizers uses a newer version of pyo3 that does not require Rust
  nightly anymore. So, we do not have to use any cheat codes anymore.
* Tokenizers is now a mixed Rust/Python project. The way it is set
  up does not work with Maturin, so switch to setuptools-rust instead.
* Add additional data files needed for tests.
* Use `pytestCheckHook`.
2020-10-28 10:41:27 -07:00
..
default.nix python3Packages.tokenizers: 0.8.1 -> 0.9.2 2020-10-28 10:41:27 -07:00