Commit graph

142 commits

Author SHA1 Message Date
Martin Weinelt ebc4dae8cb python310: 3.10.0b3 -> 3.10.0rc1 2021-08-09 07:49:03 +02:00
Frederik Rietdijk 8dcfe523f7
Merge pull request #131719 from FRidh/python2-to-3
Python: further migrate packages from 2 to 3
2021-07-29 10:49:00 +02:00
Frederik Rietdijk b3b31074b2 python interpreters: use correct python when
calling pythonPackages.
2021-07-27 20:42:01 +02:00
github-actions[bot] 732bd85cf5
Merge staging-next into staging 2021-07-17 12:01:55 +00:00
github-actions[bot] 4ee4d805d2
Merge master into staging-next 2021-07-17 12:01:22 +00:00
Dmitry Kalinkin 3d68137c6e
pypy: 7.3.3 -> 7.3.5 2021-07-16 23:31:15 -04:00
Dmitry Kalinkin c168edeeb9
pypy3: change pythonVersion to 3.7 2021-07-16 10:43:55 -04:00
Martin Weinelt b322c5ecaa
python37: 3.7.10 -> 3.7.11 2021-07-10 00:02:07 +02:00
Martin Weinelt e1e245f203
python36: 3.6.13 -> 3.6.14 2021-07-10 00:01:56 +02:00
Martin Weinelt 360c50f11d
python39: 3.9.5 -> 3.9.6 2021-07-10 00:01:41 +02:00
Martin Weinelt 250f0514f2
python38: 3.8.10 -> 3.8.11 2021-07-10 00:01:20 +02:00
Andreas Rammhold 1e47dd9cb8 cpython: Enable LTO on all builds as it doesn't break reproducibility
LTO allows us to optimise the binaries we are shipping a bit further
than just with the regular -Ox and other compiler flags. It also is
deterministic and doesn't harm our reproducibility efforts while
providing us with up to 10% performance gain (and sometimes slightly
slower). See the table below for a comparsion of this version with the
Python 3.9 build that sets -fno-semantic-interposition.

+-------------------------+--------------------+------------------------+--------------+------------------------+
| Benchmark               | py39-nsip.nix.json | py39-nsip-lto.nix.json | Change       | Significance           |
+=========================+====================+========================+==============+========================+
| 2to3                    | 642 ms             | 620 ms                 | 1.03x faster | Significant (t=12.04)  |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| chameleon               | 14.6 ms            | 14.4 ms                | 1.02x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| chaos                   | 182 ms             | 182 ms                 | 1.00x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| crypto_pyaes            | 175 ms             | 172 ms                 | 1.02x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| deltablue               | 11.2 ms            | 11.2 ms                | 1.00x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| django_template         | 82.0 ms            | 81.4 ms                | 1.01x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| dulwich_log             | 101 ms             | 99.8 ms                | 1.01x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| fannkuch                | 634 ms             | 638 ms                 | 1.01x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| float                   | 176 ms             | 189 ms                 | 1.08x slower | Significant (t=-4.30)  |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| go                      | 366 ms             | 365 ms                 | 1.00x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| hexiom                  | 14.9 ms            | 15.1 ms                | 1.01x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| json_dumps              | 18.7 ms            | 18.5 ms                | 1.01x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| json_loads              | 37.9 us            | 37.3 us                | 1.02x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| logging_format          | 14.2 us            | 14.3 us                | 1.01x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| logging_silent          | 305 ns             | 313 ns                 | 1.02x slower | Significant (t=-3.91)  |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| logging_simple          | 13.0 us            | 13.2 us                | 1.01x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| mako                    | 24.7 ms            | 23.3 ms                | 1.06x faster | Significant (t=10.74)  |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| meteor_contest          | 130 ms             | 128 ms                 | 1.01x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| nbody                   | 201 ms             | 201 ms                 | 1.00x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| nqueens                 | 152 ms             | 154 ms                 | 1.01x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| pathlib                 | 26.3 ms            | 26.2 ms                | 1.01x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| pickle                  | 13.1 us            | 13.7 us                | 1.05x slower | Significant (t=-10.36) |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| pickle_dict             | 26.6 us            | 27.1 us                | 1.02x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| pickle_list             | 4.34 us            | 4.31 us                | 1.01x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| pickle_pure_python      | 738 us             | 759 us                 | 1.03x slower | Significant (t=-5.26)  |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| pidigits                | 181 ms             | 181 ms                 | 1.00x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| pyflate                 | 959 ms             | 974 ms                 | 1.02x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| python_startup          | 34.0 ms            | 31.4 ms                | 1.08x faster | Significant (t=29.75)  |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| python_startup_no_site  | 15.3 ms            | 14.5 ms                | 1.06x faster | Significant (t=17.58)  |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| raytrace                | 849 ms             | 846 ms                 | 1.00x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| regex_compile           | 261 ms             | 261 ms                 | 1.00x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| regex_dna               | 187 ms             | 221 ms                 | 1.18x slower | Significant (t=-46.94) |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| regex_effbot            | 3.66 ms            | 3.98 ms                | 1.09x slower | Significant (t=-18.55) |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| regex_v8                | 29.7 ms            | 29.3 ms                | 1.01x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| richards                | 113 ms             | 110 ms                 | 1.03x faster | Significant (t=3.47)   |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| scimark_fft             | 592 ms             | 590 ms                 | 1.00x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| scimark_lu              | 253 ms             | 265 ms                 | 1.05x slower | Significant (t=-8.97)  |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| scimark_monte_carlo     | 170 ms             | 177 ms                 | 1.04x slower | Significant (t=-4.96)  |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| scimark_sor             | 310 ms             | 315 ms                 | 1.02x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| scimark_sparse_mat_mult | 8.36 ms            | 8.33 ms                | 1.00x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| spectral_norm           | 232 ms             | 229 ms                 | 1.02x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| sqlalchemy_declarative  | 185 ms             | 183 ms                 | 1.01x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| sqlalchemy_imperative   | 27.4 ms            | 27.6 ms                | 1.01x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| sqlite_synth            | 4.73 us            | 4.73 us                | 1.00x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| sympy_expand            | 813 ms             | 819 ms                 | 1.01x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| sympy_integrate         | 31.6 ms            | 31.8 ms                | 1.01x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| sympy_str               | 477 ms             | 479 ms                 | 1.00x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| sympy_sum               | 245 ms             | 247 ms                 | 1.01x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| telco                   | 11.3 ms            | 11.4 ms                | 1.00x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| tornado_http            | 172 ms             | 172 ms                 | 1.00x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| unpack_sequence         | 56.2 ns            | 51.2 ns                | 1.10x faster | Significant (t=2.50)   |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| unpickle                | 19.8 us            | 19.5 us                | 1.02x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| unpickle_list           | 5.75 us            | 5.75 us                | 1.00x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| unpickle_pure_python    | 524 us             | 522 us                 | 1.01x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| xml_etree_generate      | 148 ms             | 148 ms                 | 1.00x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| xml_etree_iterparse     | 129 ms             | 131 ms                 | 1.02x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| xml_etree_parse         | 179 ms             | 177 ms                 | 1.01x faster | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
| xml_etree_process       | 118 ms             | 119 ms                 | 1.00x slower | Not significant        |
+-------------------------+--------------------+------------------------+--------------+------------------------+
2021-07-05 12:34:02 -07:00
Frederik Rietdijk 94cd70bae2 python3: 3.8 -> 3.9 2021-07-05 12:34:02 -07:00
Martin Weinelt bb696403b0 python310: 3.10.0a5 -> 3.10.0b3 2021-06-29 07:27:22 -07:00
Martin Weinelt 7ca18ab93f python39: 3.9.4 -> 3.9.5 2021-06-29 07:27:22 -07:00
Martin Weinelt 3898eb1897 python38: 3.8.9 -> 3.8.10 2021-06-29 07:27:22 -07:00
DavHau 6c0b85cf3a buildPythonPackage: add support for conda 2021-05-28 14:31:40 +02:00
Sandro Jäckel 0197f1dc41 pythonPackages: Add aliases 🎉 2021-05-22 15:02:50 -07:00
John Ericson 470640e7fe treewide: Do a number of no-op cleanups for cross and darwin
I am taking the non-invasive parts of #110914 to hopefully help out with #111988.

In particular:

 - Use `lib.makeScopeWithSplicing` to make the `darwin` package set have
   a proper `callPackage`.

 - Adjust Darwin `stdenv`'s overlays keeping things from the previous
   stage to not stick around too much.

 - Expose `binutilsNoLibc` / `darwin.binutilsNoLibc` to hopefully get us
   closer to a unified LLVM and GCC bootstrap.
2021-05-06 11:17:26 -04:00
Frederik Rietdijk f310134dec python39: 3.9.3 -> 3.9.4 2021-04-05 10:54:26 +02:00
Frederik Rietdijk 4f02e12f8a python2Packages: separate extension 2021-04-03 17:49:42 +02:00
Frederik Rietdijk 9e83f91bb1 python39: 3.9.2 -> 3.9.3 2021-04-03 10:29:11 +02:00
Frederik Rietdijk d754c20366 python38: 3.8.8 -> 3.8.9 2021-04-03 10:29:10 +02:00
Frederik Rietdijk 0c1aa67215 python3Minimal and python3: don't override python3
This reduces the amount of packages that are required to use
builtins.fetchurl to fetch. Without this change, mime-types
would not be able to use fetchzip when mime-types support is
added to python3.
2021-03-14 17:01:44 +01:00
Frederik Rietdijk 5b1abe5dd2 python3: mimetypesSupport is optional 2021-03-14 15:45:15 +01:00
Frederik Rietdijk 7617b03403 python39: hardcode path to tzdata
Python 3.9 has a new module `zoneinfo` which requires tzdata. By default
it searches TZPATH for folders containing `zoneinfo`.

This commit makes the dependency on tzdata pure.
2021-03-13 14:05:41 +01:00
Lassulus 4f85b3667b
Merge pull request #113142 from andersk/pypy-7.3.3
pypy, pypy3: 7.3.2 → 7.3.3
2021-03-07 16:52:42 +01:00
Frederik Rietdijk 0dc64d5d71 python: fix full builds by referring to the correct interpreter
The package set is an attribute of the interpreter. The function to
build an environment (`buildEnv`/`withPackages`) is part of the
interpreter. The interpreter is passed to itself, and needs to be
updated when overridden.

For cross-compilation we splice the package set, and for that the
various `build/host` interpreters and sets need to be available. We
select these currently through `pkgs.${pythonAttr}`. The `pythonAttr`
attribute was not fixed for `pythonFull`.

https://github.com/NixOS/rfcs/pull/83
https://github.com/NixOS/nixpkgs/pull/104201

We need a better solution for this because this is very brittle.
2021-02-27 11:12:05 +01:00
Frederik Rietdijk 04f436940c python310: 3.10.0a4 -> 3.10.0a5 2021-02-20 10:03:11 +01:00
Frederik Rietdijk 8ab9914a8c python36: 3.6.12 -> 3.6.13 2021-02-20 10:03:11 +01:00
Frederik Rietdijk 2cbc212983 python37: 3.7.9 -> 3.7.10 2021-02-20 10:03:11 +01:00
Frederik Rietdijk 84edcadb4a python38: 3.8.7 -> 3.8.8 2021-02-20 10:03:11 +01:00
Frederik Rietdijk a39b5b5c94 python39: 3.9.1 -> 3.9.2 2021-02-20 10:03:11 +01:00
Anders Kaseorg 4a8b6c0317 pypy, pypy3: 7.3.2 → 7.3.3
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2021-02-14 10:11:54 -08:00
taku0 28adb5b0c1
Merge pull request #104150 from andersk/pypy-7.3.2
pypy, pypy3: 7.3.1 → 7.3.2
2021-02-13 21:48:05 +09:00
Jonathan Ringer 0389d5b148 python310: 3.10.0a3 -> 3.10.0a4 2021-01-22 07:27:21 -08:00
github-actions[bot] bd9e17e21f
Merge staging-next into staging 2021-01-10 18:41:46 +00:00
Profpatsch 2497553b23 toplevel: stdenv.lib -> lib
The library does not depend on stdenv, that `stdenv` exposes `lib` is
an artifact of the ancient origins of nixpkgs.
2021-01-10 17:44:59 +01:00
Frederik Rietdijk 7c61ecbefd Revert "Revert "python: 3.8.6 -> 3.8.7.""
This was originally reverted because of an invalid patch.
This has been fixed and now we want this update again.

This reverts commit e1ed9b3b08.
2021-01-09 13:12:44 +01:00
Frederik Rietdijk e1ed9b3b08 Revert "python: 3.8.6 -> 3.8.7."
Need to fix a patch first. Next iteration.

This reverts commit 507efdb11f.
2021-01-04 20:58:21 +01:00
Drew Hess 507efdb11f python: 3.8.6 -> 3.8.7. 2021-01-04 20:48:55 +01:00
github-actions[bot] abb757ba28
Merge master into staging-next 2020-12-18 00:42:45 +00:00
Jonathan Ringer 222fafa02b python310: 3.10.0a2 -> 3.10.0a3 2020-12-17 11:18:47 -08:00
Greg Roodt 42d28c2a77 python: 3.9.0 -> 3.9.1 2020-12-12 19:32:08 +01:00
adisbladis 49b8738a5f
python: Propagate packageOverrides to pythonForBuild
When overriding versions of build tools injected via hooks
`packageOverrides` was not taken into account and 2 incompatible
versions of the same package (`wheel` in this case) ended up in the
closure, causing the builds to fail.
2020-11-30 11:12:59 +01:00
Frederik Rietdijk 455774e546 Python: add todo note regarding pythonForBuild 2020-11-28 17:36:23 +01:00
Frederik Rietdijk 5804c8a7f5 Python splicing: splice package sets, not Python derivation
In the original commit the various builds of Python were added to
`otherSplices`, instead of the intended Python package sets.
2020-11-28 17:36:23 +01:00
Frederik Rietdijk 6cf25f9dbd Python: rename parameters and arguments passed to passthru
As part of the splicing the build/host/target combinations of the interpreter
need to be passed around internally. The chosen names were not very clear,
implying they were package sets whereas actually there were derivations.
2020-11-28 17:36:23 +01:00
Frederik Rietdijk cce2fd547b Python: use pythonPackagesBuildHost instead of pythonForBuild
Follow-up to #104201, related to #105113.
2020-11-28 16:36:03 +01:00
John Ericson b57c5d4456 python: Use makeScopeWithSplicing
Now non-`buildInputs` that are python packages should be resolved
correctly.
2020-11-19 11:58:07 -05:00