Commit graph

40 commits

Author SHA1 Message Date
Jonathan Ringer 3973a3c79c python: add pythonRemoveTestsDirHook
This will automatically remove a top-level "tests"
directory from being installed.
2020-03-03 20:01:00 +01:00
adisbladis 2d6f1ff4dd
python: Add support for installing Python eggs 2020-01-08 13:59:04 +00:00
Frederik Rietdijk f7e28bf5d8 Split buildPythonPackage into setup hooks
This commit splits the `buildPythonPackage` into multiple setup hooks.

Generally, Python packages are built from source to wheels using `setuptools`.
The wheels are then installed with `pip`. Tests were often called with
`python setup.py test` but this is less common nowadays. Most projects
now use a different entry point for running tests, typically `pytest`
or `nosetests`.

Since the wheel format was introduced more tools were built to generate these,
e.g. `flit`. Since PEP 517 is provisionally accepted, defining a build-system
independent format (`pyproject.toml`), `pip` can now use that format to
execute the correct build-system.

In the past I've added support for PEP 517 (`pyproject`) to the Python
builder, resulting in a now rather large builder. Furthermore, it was not possible
to reuse components elsewhere. Therefore, the builder is now split into multiple
setup hooks.

The `setuptoolsCheckHook` is included now by default but in time it should
be removed from `buildPythonPackage` to make it easier to use another hook
(curently one has to pass in `dontUseSetuptoolsCheck`).
2019-09-06 15:18:45 +02:00
volth f3282c8d1e treewide: remove unused variables (#63177)
* treewide: remove unused variables

* making ofborg happy
2019-06-16 19:59:05 +00:00
Frederik Rietdijk d801204fdd
Merge pull request #51641 from codedownio/nousersite-master
Allow skipping the use of the PYTHONNOUSERSITE variable
2019-05-13 12:39:25 +02:00
Tom McLaughlin a3f24daa7b Add flag to disable PYTHONNOUSERSITE for wrapped binaries in python environments 2019-05-13 02:56:56 -07:00
Tom McLaughlin 0045164b15 Add flag to disable PYTHONNOUSERSITE for wrapped python binaries 2019-05-13 02:56:32 -07:00
Jan Tojnar 83914f2c56
buildPython*: allow overriding strictDeps 2019-04-24 22:27:06 +02:00
Frederik Rietdijk 1fccd25595 buildPythonPackage: always export LANG=C.UTF-8 2019-02-23 20:08:26 +01:00
Frederik Rietdijk b4acd97729 buildPython*: enable strictDeps 2019-02-17 14:40:48 +01:00
Frederik Rietdijk c5d99308de buildPython*: fix nativeBuildInputs 2019-01-15 10:16:11 +01:00
Frederik Rietdijk f665828fa3 Python: improve cross-compilation
This changeset allows for cross-compilation of Python packages. Packages
built with buildPythonPackage are not allowed to refer to the build
machine. Executables that have shebangs will refer to the host.
2019-01-04 10:45:22 +01:00
Frederik Rietdijk 3d5025ba79 buildPython*: don't override custom updateScript 2018-12-23 09:59:56 +01:00
Jan Tojnar e69ee9b53e
buildPython*: simplify updateScript
updateScript accepts a list, in which case, it will execute the head with the
tail as arguments. Switching to this style will allow us to get rid of the need
to create extra script doing just that.
2018-12-20 00:28:49 +01:00
Frederik Rietdijk 74d0b82f29 buildPython*: add updateScript to passthru
All Python packages now have an updateScript. The script calls
`update-python-libraries` and passes it the position of the derivation
expression obtained using `meta.position`. This works fine in case a Nix
expression represents only a single derivation. If there are more in it,
`update-python-libraries` will fail.
2018-12-07 15:48:54 +01:00
Jan Malakhovski 87651b32fe stdenv: steal checkInputs from buildPythonPackage
Note that a bunch of non-python packages use this attribute already.
Some of those are clearly unaware of the fact that this attribute does
not exists in stdenv because they define it but don't to add it to
their `bulidInputs` :)

Also note that I use `buildInputs` here and only handle regular
builds because python and haskell builders do it this way and I'm not
sure how to properly handle the cross-compilation case.
2018-04-26 20:22:51 +00:00
Jan Malakhovski 912cfb8aaa buildPythonPackage: use config.doCheckByDefault 2018-04-26 20:15:52 +00:00
Tuomas Tynkkynen 68052b5619 python: Pull ensureNewerSourcesHook call to all-packages.nix
Documents the reason why it's needed and also prevents the
ensureNewerSourcesHook call being evaluated again and again for every
single Python package.
2018-02-13 16:32:16 +02:00
John Ericson dc906952be buildPythonPackage: Make setup hook part of nativeBuildInputs
This is better organization in general, but also needed to make python
work with #26805.
2017-12-21 15:54:52 -05:00
Frederik Rietdijk 35ccdb8632
Merge pull request #32544 from FRidh/pythonmodule
Python: rewrite requiredPythonModules to prevent stack overflows
2017-12-11 09:46:15 +01:00
Frederik Rietdijk a334930490 Python: rewrite requiredPythonModules. Add requiredPythonModules attribute to derivation 2017-12-10 20:40:34 +01:00
Frederik Rietdijk d945b3e53b buildPythonPackage: remove bytecode from bin folder
When a Python script has the extension `.py`, bytecode is generated.
Typically, executables in bin have no extension, so no bytecode is
generated. However, some packages do provide executables with
extensions, and thus bytecode is generated.
2017-12-10 16:01:09 +01:00
Frederik Rietdijk 80329e7123 python.pkgs.buildPythonPackage: some comments 2017-11-23 15:11:03 +01:00
Frederik Rietdijk 209ee2f20f python.pkgs.buildPythonPackage: do not pass attributes that stdenv.mkDerivation does not use 2017-11-23 15:11:03 +01:00
Frederik Rietdijk 42eece04e4 python.pkgs.buildPythonPackage: meta.maintainers: don't add chaoflow 2017-11-23 15:11:03 +01:00
Frederik Rietdijk 40851a4d26 Python: the pythonModule attribute
Python libraries or modules now have an attribute `pythonModule = interpreter;` to indicate
they provide Python modules for the specified `interpreter`.

The package set provides the following helper functions:

- hasPythonModule: Check whether a derivation provides a Python module.
- requiredPythonModules: Recurse into a list of Python modules, returning all Python modules that are required.
- makePythonPath: Create a PYTHONPATH from a list of Python modules.

Also included in this commit is:
- disabledIf: Helper function for disabling non-buildPythonPackage functions.
2017-11-23 15:11:02 +01:00
Orivej Desh 78ed9da5e4 mkPythonDerivation: let name default to ${pname}-${version} 2017-11-03 04:15:36 +00:00
Frederik Rietdijk c3a0c3c9f1 Revert "python.pkgs.buildPythonPackage: use distutils-cfg to block downloads, fixes #25428"
This reverts commit b73e3bfafd.

See https://github.com/NixOS/nixpkgs/issues/29103
2017-09-08 08:55:05 +02:00
Frederik Rietdijk b73e3bfafd python.pkgs.buildPythonPackage: use distutils-cfg to block downloads, fixes #25428
While we tell pip not to fetch (with the `--no-index` option),
`setuptools` can do so itself. In the past we used a `distutils.cfg`
with `allow-hosts = None` to prevent setuptools from fetching itself.
This was removed when we started building wheels in
2562f94de4e4fd2ddc677187fa2e2848L69.

The `dist-utils.cfg` code was still there, and adding it to
`buildInputs` is sufficient.

Tested with python.pkgs.passlib by removing the `checkInputs` / `nose`.
2017-09-07 19:22:06 +02:00
Robert Scott 45ac7700fa mkPythonDerivation: obey a dontWrapPythonPrograms attribute
This should allow a python derivation to disable possibly overzealous
python program wrapping without having to override the whole fixupPhase
2017-05-27 14:25:08 +02:00
Frederik Rietdijk 97f67d1b5a buildPythonPackage: do not keep checkInputs
When tests are disabled, we do not want to pass checkInputs to
stdenv.mkDerivation. This reduces the build requirements and, more
importantly, helps cutting cycles.
2017-05-27 14:25:08 +02:00
Frederik Rietdijk acd32a4caf Python: set DETERMINISTIC_BUILD and PYTHONHASHSEED in setupHook
The Python interpreters are patched so they can build .pyc bytecode free
of certain indeterminism.

When building Python packages we currently set

```
compiling python files.
in nix store.
DETERMINISTIC_BUILD=1;
PYTHONHASHSEED = 0;
```

Instead if setting these environment variables in the function that
builds the package, this commit sets the variables instead in the Python
setup hook. That way, whenever Python is included in a derivation, these
variables are set.

See also the issue https://github.com/NixOS/nixpkgs/issues/25707.
2017-05-19 16:28:11 +02:00
Johannes Bornhold 249b34aadc
python: Move catch_conflicts.py into subdirectory (#23600)
Python does add the script's directory into "sys.path". For the case of
"catch_conflicts.py" this means "/nix/store" is added to "sys.path". This can
result in very long delays if the store contains a lot of entries.

(moved from master commit 76213d102c)
2017-04-01 12:22:36 +02:00
Domen Kožar fb40889ddb
mk-python-derivation: remove me as maintainer
The time has come ...
2017-03-03 14:37:28 +01:00
Frederik Rietdijk dd3a501a4b Python: mkPythonDerivation: use PYTHONHASHSEED=0 2017-02-26 14:50:09 +01:00
Frederik Rietdijk 42bad32e13 buildPythonPackage: refactor 2016-12-05 10:29:22 +01:00
Frederik Rietdijk 85a87f5155 mkPythonDerivation: append postFixup, fixes #9204 2016-10-26 10:46:12 +02:00
Frederik Rietdijk e1d1a573e5 Python: fix mkPythonDerivation doCheck 2016-09-01 17:11:32 +02:00
Frederik Rietdijk 4f6b6f1472 pythonPackages: use mkPythonDerivation 2016-09-01 16:16:31 +02:00
Frederik Rietdijk 3e05cce97c Python: separate buildPythonPackage into two functions
1. mkDerivation which is used when the source is without setup.py and
not a wheel
2. buildPythonPackage which is used as before and calls mkDerivation
2016-09-01 16:16:31 +02:00