Commit graph

19 commits

Author SHA1 Message Date
Martin Weinelt 0f692b229a python3Packages.pyspark: 3.1.1 -> 3.1.2 2021-06-22 13:42:38 +02:00
Frederik Rietdijk 45c400f86f python3Packages.pyspark: 3.0.2 -> 3.1.1 2021-04-03 17:49:15 +02:00
Frederik Rietdijk 5324655b9d python3Packages.pyspark: 3.0.1 -> 3.0.2 2021-02-26 11:01:53 +01:00
Pavol Rusnak a6ce00c50c
treewide: remove stdenv where not needed 2021-01-25 18:31:47 +01:00
Profpatsch 4a7f99d55d treewide: with stdenv.lib; in meta -> with lib;
Part of: https://github.com/NixOS/nixpkgs/issues/108938

meta = with stdenv.lib;

is a widely used pattern. We want to slowly remove
the `stdenv.lib` indirection and encourage people
to use `lib` directly. Thus let’s start with the meta
field.

This used a rewriting script to mostly automatically
replace all occurances of this pattern, and add the
`lib` argument to the package header if it doesn’t
exist yet.

The script in its current form is available at
https://cs.tvl.fyi/depot@2f807d7f141068d2d60676a89213eaa5353ca6e0/-/blob/users/Profpatsch/nixpkgs-rewriter/default.nix
2021-01-11 10:38:22 +01:00
Joshua Skrzypek cde89e3ecf python3Packages.pyspark: Fix out of date postPatch
Pyspark switched to pinning py4j==0.10.9 with v3.0.0 - see this commit:
https://github.com/apache/spark/\
   commit/fc4e56a54c15e20baf085e6061d3d83f5ce1185d

This meant that since the bump to pyspark v3.0.0 - in this commit:
https://github.com/NixOS/nixpkgs/\
   commit/5181547ae6624b462919a806c4d0888e6e4630f4 -
the patch was no longer matching on the 'py4j==0.10.7' string that was
working previously.

The failing patch went unnoticed previously because the version of py4j
pinned by pyspark>=3.0.0 was the same as the py4j provided by nixpkgs.

However, a recent PR (#101636) bumped the version of py4j to 0.10.9.1 in
this commit:
https://github.com/NixOS/nixpkgs/\
   commit/43a91282d66223c5cb978d53fbe1033f56dd7f2b
which caused the version pinned by pyspark to no longer match the
version provided by nixpkgs. FWIW, @jonringer flagged this issue on
another PR that tried to bump py4j: #100623.

My solution here was to upgrade the patch's target string to match the
version found in pyspark's current setup.py.
2020-11-23 16:19:14 -08:00
R. RyanTM 937a2f9cbb python37Packages.pyspark: 3.0.0 -> 3.0.1 2020-10-17 11:04:50 -07:00
Frederik Rietdijk 5181547ae6 python: pyspark: 2.4.6 -> 3.0.0 2020-08-16 19:31:12 +02:00
Frederik Rietdijk fd82b84d5e python: pyspark: 2.4.5 -> 2.4.6 2020-06-11 10:00:19 +02:00
Frederik Rietdijk 237e6a0052 python3Packages.pyspark: 2.4.4 -> 2.4.5 2020-05-11 22:13:12 +02:00
Michael Reilly 84cf00f980
treewide: Per RFC45, remove all unquoted URLs 2020-04-10 17:54:53 +01:00
Frederik Rietdijk 1856412da0 python: pyspark: 2.4.3 -> 2.4.4 2019-10-18 09:54:00 +02:00
Frederik Rietdijk 038664dcc3 python: pyspark: 2.4.2 -> 2.4.3 2019-06-15 08:44:05 +02:00
Frederik Rietdijk a4643e9e6f python: pyspark: 2.4.1 -> 2.4.2 2019-04-24 18:43:26 +02:00
Frederik Rietdijk bdb8f9fe81 python: pyspark: 2.4.0 -> 2.4.1 2019-04-18 12:42:42 +02:00
Matthew Bauer c0f82700ee pyspark: 2.3.2 -> 2.4.0, fix version bounds
py4j seems to work fine with this version
2019-01-30 13:29:29 -05:00
Frederik Rietdijk 5a3963fd1c python: pyspark: 2.3.1 -> 2.3.2 2018-10-13 09:50:16 +02:00
Frederik Rietdijk 2c279a2f92 python: pyspark: 2.3.0 -> 2.3.1 2018-06-22 13:09:27 +02:00
Shea Levy 2963d76ca3
pythonPackages.pyspark: Init at 2.3.0 2018-03-17 09:54:12 -04:00