Commit graph

4036 commits

Author SHA1 Message Date
Frederik Rietdijk 1c68570ab2 Merge staging-next into staging 2020-06-05 19:42:16 +02:00
Frederik Rietdijk 43f71029cc Merge master into staging-next 2020-06-05 19:40:53 +02:00
Jörg Thalheim f0396574ab
buildRustPackage: add documentation on how to create cargo.lock patches 2020-06-05 10:41:46 +01:00
Vladimír Čunát 677e3960b5
Merge #82342: rustPlatform: increase build-speed of checkPhase
...for rust-packages (into staging)
2020-06-05 09:12:30 +02:00
Frederik Rietdijk 08900c0554 Merge master into staging-next 2020-06-04 15:25:54 +02:00
Benjamin Hipple 4af635feff
Merge pull request #87866 from danieldk/validate-pkgconfig-hook
validatePkgConfig: init
2020-06-02 23:27:27 -04:00
DwarfMaster a9eab6d2b9 Fix building of emacsWithPackages not findind subdirs.el 2020-06-02 11:52:39 +02:00
Florian Klink 8a388c8296
Merge pull request #82258 from erikarvstedt/fix-xchg-caching
fix inconsistent caching of VM xchg dirs
2020-06-01 22:34:48 +02:00
Erik Arvstedt d85dc4f690
qemu-vm: fix inconsistent caching of xchg dirs
xchg is advertised as a bidirectional exchange dir, but file content
transfer from host to VM fails due to caching:
If a file is read in the VM and then modified on the host, subsequent
re-reads in the VM can yield old, cached data.
This is caused by the use of 9p's cache=loose mode that is explicitly
meant for read-only mounts.

9p doesn't provide any suitable cache modes, so fix this by disabling
caching.

Also, remove a now unnecessary sync in the test driver.
2020-06-01 21:55:33 +02:00
Alex Rice e215c3bcac
agda: install literate files 2020-06-01 13:59:20 +01:00
Daniël de Kok 13083b1a01 validatePkgConfig: init
This adds the `validatePkgConfig` hook, which can be used to validate
pkg-config files in the output(s). Currently, this will just run
`pkg-config --validate` on all `.pc` files, capturing errors such as
the issue that was fixed in #87789.

The hook could be extended in the future with more fine-grained
checks.
2020-05-31 20:45:03 +02:00
Pavol Rusnak 9444756222
appimage: replace radare2+jq with readelf+awk+sha256sum
radare2 does not play nicely with filenames containing spaces
https://github.com/radareorg/radare2/issues/16958
2020-05-29 11:14:16 +02:00
Frederik Rietdijk 362d88c2b1 Merge staging-next into staging 2020-05-27 15:27:28 +02:00
Michael Howell c21cbf22d0
buildRustCrate: Replace hyphen with underscore in env variables (#88054)
* Add test case for include dir
* buildRustCrate: replace hyphen with underscore in env

This fixes a bug that prevents encoding_c from building.
2020-05-26 20:52:18 +02:00
adisbladis d8b6e82126
Merge pull request #88856 from zowoq/prefetch-docker
dockerTools: fix skopeo commands
2020-05-25 09:52:09 +02:00
rnhmjoj aee614c996
treewide: replace bazaar with breezy 2020-05-25 09:22:54 +02:00
zowoq 365d07cea0 dockerTools: fix skopeo commands 2020-05-25 14:29:54 +10:00
Florian Klink 9b7a4694a0 fetchgitlab: escape rev to support "+" in it 2020-05-24 22:24:50 +02:00
Shea Levy fffa6e8c18
Merge pull request #88762 from alexbiehl/alex/docker-tools
dockerTools: Properly add /nix/ and /nix/store/ first to layer.tar
2020-05-24 08:34:42 -04:00
Alex Biehl 886c92332c dockerTools: Properly add /nix/ and /nix/store/ first to layer.tar
In https://github.com/NixOS/nixpkgs/pull/58431 the authors ensured that
the resulting layer.tar would always list

  /nix/
  /nix/store/

first to fully comply to the tar spec. Various refactorings later it is only
ensured to create /nix/ but NOT /nix/store anymore. Instead tar transformed
them to /nix/nix and /nix/nix/store.
2020-05-24 14:28:35 +02:00
Frederik Rietdijk d578248611 Merge staging-next into staging 2020-05-24 10:10:06 +02:00
Matthew Bauer c74a9cd3e2
Merge pull request #69144 from SFrijters/gdc
gdc: init at 9.3.0
2020-05-23 16:30:47 -05:00
Orivej Desh 16d7f7edae Merge branch 'master' into staging 2020-05-22 09:13:23 +00:00
John Ericson eae4f22176
Merge pull request #88518 from Ericson2314/fix-pkg-config
pkg-config, pkgconf: Misc fixes
2020-05-21 16:05:42 -04:00
Profpatsch 8b270a96fa skawarePackages.cleanPackaging: copy docFiles before rm noise 2020-05-21 19:46:38 +02:00
John Ericson 7e9d807f2c pkg-config-wrapper: Move comments outside of strings
This is much better because then we can freely keep the comments up to
date without causing mass rebuilds.

Someday, somebody should make the same change with `cc-wrapper` and
`bintools-wrapper`.
2020-05-21 11:35:46 -04:00
John Ericson 95080ea90b pkgconf: Wrap with pkg-config-wrapper 2020-05-21 11:35:46 -04:00
John Ericson 4b2ab665fb pkg-config-wrapper: Provide autoconf macros
This fixes libarchive, and probably numerous other builds.
2020-05-21 09:16:34 -04:00
John Ericson f29919da61 pkg-config: Multiple outputs
This is good practice, and will help with tests. Wrapper adjust to match
bintools-wrapper re these new outputs.
2020-05-21 08:58:34 -04:00
Stefan Frijters fc9b93d2fc gdc: init at 9.3.0 2020-05-19 23:06:52 +02:00
lewo a498da343a
Merge pull request #87154 from utdemir/buildimage-optimizations
Some performance optimizations to dockerTools.build{,Layered}Image
2020-05-19 15:39:25 +02:00
Jan Tojnar f8a9c6efac
Merge branch 'staging-next' into staging 2020-05-18 21:09:48 +02:00
John Ericson c71ab32a67 pkg-config-wrapper: Init
This fixes longstanding build issues
2020-05-16 00:21:21 +00:00
John Ericson ee8cfe4bf3 treewide: pkg-config has targetPrefix 2020-05-16 00:21:21 +00:00
zowoq 42232493a3 dockerTools: pass insecure-policy and tmpdir to skopeo 2020-05-16 08:46:13 +10:00
lewo df928fafd4
Merge pull request #87725 from nlewo/fix-dockertools
dockerTools.examples: fix the nginx image
2020-05-15 16:54:09 +02:00
lewo 98a723e6b3
Merge pull request #87253 from utdemir/dockertools-preserve-env
Preserve environment variables from the parent image on dockerTools.buildImage
2020-05-15 09:59:00 +02:00
Dmitry Kalinkin 9943fd1a1d
Merge pull request #76653 from alexarice/agda-rework
Agda rework
2020-05-14 18:02:50 -04:00
Alex Rice d30e2468e0
agda: rework builder 2020-05-14 20:54:11 +01:00
Frederik Rietdijk 404fe35d65 Merge staging-next into staging 2020-05-14 09:37:03 +02:00
Matthew Bauer d38bad590e
Merge pull request #83180 from matthewbauer/only-add-ldflags-for-macos
bintools: only add macos flags when targeting macOS
2020-05-13 10:34:40 -05:00
Antoine Eiche 9ac444d80e dockerTools.examples: fix the nginx image
Nginx now requires directories `/var/log/nginx` and `/var/cache/nginx`
to be existing.

The objective is to fix the test `nixosTests.docker-tools`.
2020-05-13 10:29:53 +02:00
Maximilian Bosch 6b23cfe689
rustPlatform: add buildAndTestSubdir-argument
There are several tarballs (such as the `rust-lang/rust`-source) with a
`Cargo.toml` at root and several sub-packages (with their own Cargo.toml)
without using workspaces[1].

In such a case it's needed to move into a subdir to only build the
specified sub-package (e.g. `rustfmt` or `rsl`), however the artifacts
are at `/target` in the root-dir of the build environment. This breaks
the build since `buildRustPackage` searches for executables in `target`
(which is at the build-env's root) at the end of the `buildPhase`.

With the optional `buildAndTestSubdir`-argument, the builder moves into
the specified subdir using `pushd`/`popd` during `buildPhase` and
`checkPhase`.

Also moved the logic to find executables and libs to the end of the `buildPhase`
from a custom `postBuild`-hook to fix packages with custom `build`/`install`-procedures
such as `uutils-coreutils`.

[1] https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html
2020-05-13 01:47:17 +02:00
Maximilian Bosch 736462d995
rustPlatform: make it possible to override the profile for cargo test 2020-05-13 01:39:44 +02:00
John Ericson 1ac5398589 *-wrapper; Switch from infixSalt to suffixSalt
I hate the thing too even though I made it, and rather just get rid of
it. But we can't do that yet. In the meantime, this brings us more
inline with autoconf and will make it slightly easier for me to write a
pkg-config wrapper, which we need.
2020-05-12 00:44:44 -04:00
Matthew Bauer 43545032af
Merge pull request #87314 from matthewbauer/bazel-flat
build-bazel-package: switch hash mode to “flat”
2020-05-11 15:27:48 -05:00
Frederik Rietdijk d20d734bcf Merge master into staging-next 2020-05-11 09:19:26 +02:00
Vladimír Čunát 5eaabaf089
Merge branch 'staging-next' into PR 82342
Hydra nixpkgs: ?compare=1586582
2020-05-11 08:22:59 +02:00
talyz 9f09253e52
php.buildPecl: Allow PECLs to depend on other PECLs
Some PECLs depend on other PECLs and, like internal PHP extension
dependencies, need to be loaded in the correct order. This makes this
possible by adding the argument "peclDeps" to buildPecl, which adds
the extension to buildInputs and is treated the same way as
internalDeps when the extension config is generated.
2020-05-09 23:38:21 +02:00
Matthew Bauer 1e19cc37bd build-bazel-package: pass proxy env vars 2020-05-09 13:35:47 -05:00
Matthew Bauer 2a8d05627b build-bazel-package: switch hash mode to “flat”
flat hashes can be substituted through hashed-mirrors, while recursive
hashes can’t. This is especially important for Bazel since the bazel
fetch dependencies can come from multiple different methods (git,
http, ftp, etc.). To do this, we create tar archives from the
output/external directory, which is then extracted to build. All of
the Bazel hashes are all updated.
2020-05-09 13:33:26 -05:00
Maximilian Bosch f236714d65
rustPlatform: fix log 2020-05-08 22:59:50 +02:00
Maximilian Bosch e32c005772
rustPlatform: don't install test executables
This is done by gathering all binaries to install before running the
checkPhase.
2020-05-08 22:59:50 +02:00
Utku Demir f5a90a7aab
dockerTools.buildImage: Preserve environment variables from the parent image 2020-05-08 21:49:16 +12:00
Frederik Rietdijk bc48203b98 Merge staging-next into staging 2020-05-08 06:46:14 +02:00
Benjamin Hipple 7f845578af rustPlatform: fix bug with ambiguous diff tool
If a user provides `nativeBuildInputs = [ llvmPackages.bintools ]` or any other
package containing a `${prefix}/bin/diff`, the builder could use it instead
of the standard unix `diff`, causing a build failure.

This updates the call to specify an abspath to `diff` and avoid reliance on `PATH`.

Resolves #87081
2020-05-06 21:03:41 -04:00
Utku Demir f12346d493
dockerTools: Calculate tarsum's on the fly
Calculating the tarsum after creating a layer is inefficient, since
we have to read the tarball we've just written from the disk.

This commit simultaneously calculates the tarsum while creating the
tarball.
2020-05-07 11:50:07 +12:00
Utku Demir 69f6294724
dockerTools.buildLayeredImage: Avoid appending to tarballs when building layers
Appending to an existing tar archive repeatedly seems to be a quadratic
operation, since tar seems to traverse the existing archive even using
the `-r, --append` flag. This commit avoids that by passing the list of
files to a single tar invocation.
2020-05-07 11:50:00 +12:00
Maximilian Bosch 04248f606f
rustPlatform: increase build-speed of checkPhase for rust-packages
When running `cargo test --release`, the artifacts from `buildPhase`
will be reused here. Previously, most of the stuff had to be recompiled
without optimizations.
2020-05-06 23:47:31 +02:00
Frederik Rietdijk 9875bbae75 Merge master into staging-next 2020-05-05 19:51:09 +02:00
Benjamin Hipple ba160ab741
Merge pull request #85410 from matthewbauer/skip-post-fetch-in-hashed-mirrors
fetchurl: don’t run ‘postFetch’ on hashed-mirrors
2020-05-04 22:45:19 -04:00
Jörg Thalheim 2fb5dac372 nix-prefetch-git: also add store path to json output 2020-05-04 17:43:29 +05:30
Pavol Rusnak 7b0167204d treewide: use https for nixos.org and hydra.nixos.org
tarballs.nixos.org is omitted from the change because urls from there
are always hashed and checked
2020-05-03 22:14:21 -07:00
adisbladis fafb127947
dockertools: Add a buildLayeredImageWithNixDb function
This is analogous to buildImageWithNixDb but instead uses
buildLayeredImage under the hood.
2020-05-02 15:59:39 +01:00
Frederik Rietdijk afb1041148 Merge master into staging-next 2020-05-02 09:39:00 +02:00
Jörg Thalheim d67098ec2e
appimage: fix r2 commands with newer radare2 version 2020-05-01 12:36:18 +01:00
Frederik Rietdijk 484ee79050 Merge staging-next into staging 2020-05-01 08:57:10 +02:00
Frederik Rietdijk 2da19f9483
Merge pull request #85653 from veprbl/pr/darwin_binutils_add_man
darwin.binutils: propagate man pages from darwin.cctools
2020-05-01 08:49:56 +02:00
Mario Rodas f7ac956a2b buildDunePackage: add support for parallel build
dune allows to specify the number of jobs to build the package.
2020-04-29 16:08:52 +02:00
Simonas Kazlauskas dde3e8edf4 nix-prefetch-docker: Add jq into path
Otheriwse running the tool fails with

    line 41: jq: command not found
2020-04-27 22:21:55 +03:00
zimbatm 1bf7f5f240
Merge pull request #85750 from zimbatm/srcOnly-docs
srcOnly: fix invocation and document
2020-04-27 14:49:55 +02:00
zimbatm 31ead7d030
srcOnly: rename buildInputs to nativeBuildInputs
The only reason to pass build inputs is to extend the unpackPhase with
custom unpack commands. Eg: add "unrar" to unpack rar sources. And those
should really be passed as native build inputs. Why? Because
nativeBuildInputs is for dependencies that are used at build time but
will not propagate as runtime dependencies. And also, cross-compilation.
2020-04-27 14:31:40 +02:00
Dmitry Kalinkin f0da3b5e94
Merge pull request #85952 from LnL7/darwin-lapack
lapack: fix darwin library identifiers
2020-04-26 00:39:26 -04:00
Silvan Mosberger eeb4e522b5
Merge pull request #85255 from Infinisil/sanitize-run-command-name 2020-04-25 00:11:41 +02:00
Daiderd Jordan 5ae928e18f
lapack: fix darwin library identifiers
The build system already sets these properly to the absolute path so no
need to patch the libraries on darwin.

    $ otool -D result/lib/liblapacke.dylib
    result/lib/liblapacke.dylib:
    /nix/store/k88gy5s765yn3dc5ws3jbykyvklm7z96-openblas-0.3.8/lib/libopenblasp-r0.3.8.dylib

Fixes #85713
2020-04-24 20:00:56 +02:00
Frederik Rietdijk cff0669a48 Merge master into staging-next 2020-04-23 08:11:16 +02:00
Benjamin Hipple 21be830522
Merge pull request #85799 from matthewbauer/blas-lapack-expand-docs
BLAS/LAPACK fixup and expand docs
2020-04-22 21:56:20 -04:00
Matthew Bauer 5a500ff0ba blas,lapack: use correct name for library
To match the soname, we need to use libblas.so.3, liblapack.so.3.
2020-04-22 12:37:04 -05:00
Frederik Rietdijk 8374a2a0ee Merge master into staging-next 2020-04-22 17:20:20 +02:00
zimbatm bbbc419186
srcOnly: fix invocation and document
Previously, callPackage would try and fill the arguments such as `name`
and `src` which would cause problems if those existed as top-level
attributes. This also makes it clearer what part is the function
signature.

Then document the derivation inline in the code to explain the ellipsis
and various use-cases.
2020-04-22 14:06:28 +02:00
edef e71990711f
Merge pull request #85680 from NixOS/fix-srconly
srcOnly: ignore additional arguments
2020-04-21 22:45:34 +00:00
Frederik Rietdijk 23be4a8b4d Merge master into staging-next 2020-04-21 19:59:56 +02:00
edef 11e13acc46 srcOnly: ignore additional arguments
This reverts commit b32a057425,
which breaks even the most straightforward uses of srcOnly:

    nix-repl> srcOnly guile
    error: anonymous function at /home/src/nixpkgs/pkgs/build-support/src-only/default.nix:1:1 called with unexpected argument 'drvPath', at /home/src/nixpkgs/lib/customisation.nix:69:16

    nix-repl> srcOnly hello
    error: anonymous function at /home/src/nixpkgs/pkgs/build-support/src-only/default.nix:1:1 called with unexpected argument 'drvPath', at /home/src/nixpkgs/lib/customisation.nix:69:16

Link: https://github.com/NixOS/nixpkgs/pull/80903#issuecomment-617172927
2020-04-21 13:20:17 +00:00
Frederik Rietdijk 803b3d296c Merge staging-next into staging 2020-04-21 08:29:51 +02:00
Matthew Bauer 8f58fa044d blas,lapack: symlink headers when using mkl
Some of these are necessary for things like python.pkgs.numexpr
2020-04-21 00:30:44 -05:00
Matthew Bauer 0caa375b4c blas,lapack: provide symlink for libmkl_rt.so
This is needed for numpy to detect mkl correctly.
2020-04-20 23:47:10 -05:00
Dmitry Kalinkin c450617f99
bintools-wrapper: only propagate .info output if it exists in the original 2020-04-20 23:49:02 -04:00
Matthew Bauer ff2f2644f8 blas,lapack: use isILP64 instead of is64bit
This is a better name since we have multiple 64-bit things that could
be referred to.

LP64  : integer=32, long=64, pointer=64
ILP64 : integer=64, long=64, pointer=64
2020-04-20 16:02:43 -05:00
Matthew Bauer 3c9c894f83 blas,lapack: don’t patchelf
We have some unused RPATHs we don’t want shrunk. These are used in MKL
to dlopen based on architecture.
2020-04-20 15:50:28 -05:00
Lars Jellema 8e79583510 gnat: init at 9.3.0 2020-04-20 14:41:22 +02:00
Sébastien Maret 712bf551d8 blas: fix build on darwin 2020-04-19 23:07:10 +02:00
John Ericson 1ea80c2cc3 Merge remote-tracking branch 'upstream/master' into staging 2020-04-18 15:40:49 -04:00
John Ericson eb4acb6ca9
Merge pull request #85468 from Ericson2314/no-dumpmachine
treewide: Get rid of -dumpmachine in favor of static info
2020-04-18 00:35:19 -04:00
John Ericson e3d50e5cb0 Merge branch 'master' of github.com:NixOS/nixpkgs into staging 2020-04-18 00:10:08 -04:00
Milan Pässler fc5090b315 blas: fix build 2020-04-18 03:06:47 +02:00
John Ericson 33c2a76c5e Merge remote-tracking branch 'upstream/master' into staging 2020-04-17 18:40:51 -04:00
Matthew Bauer d83e9c9573 setup-hooks/audit-blas.sh: init
Add a simple hook that makes sure blas and lapack are linked to the
right place.
2020-04-17 16:24:25 -05:00
Matthew Bauer 43873351ff blas/lapack: add wrapper for “alternative”s of BLAS/LAPACK provider
This is based on previous work for switching between BLAS and LAPACK
implementation in Debian[1] and Gentoo[2]. The goal is to have one way
to depend on the BLAS/LAPACK libraries that all packages must use. The
attrs “blas” and “lapack” are used to represent a wrapped BLAS/LAPACK
provider. Derivations that don’t care how BLAS and LAPACK are
implemented can just use blas and lapack directly. If you do care what
you get (perhaps for some CPP), you should verify that blas and lapack
match what you expect with an assertion.

The “blas” package collides with the old “blas” reference
implementation. This has been renamed to “blas-reference”. In
addition, “lapack-reference” is also included, corresponding to
“liblapack” from Netlib.org.

Currently, there are 3 providers of the BLAS and LAPACK interfaces:

- lapack-reference: the BLAS/LAPACK implementation maintained by netlib.org
- OpenBLAS: an optimized version of BLAS and LAPACK
- MKL: Intel’s unfree but highly optimized BLAS/LAPACK implementation

By default, the above implementations all use the “LP64” BLAS and
LAPACK ABI. This corresponds to “openblasCompat” and is the safest way
to use BLAS/LAPACK. You may received some benefits from “ILP64” or
8-byte integer BLAS at the expense of breaking compatibility with some
packages.

This can be switched at build time with an override like:

    import <nixpkgs> {
        config.allowUnfree = true;
        overlays = [(self: super: {
          lapack = super.lapack.override {
            lapackProvider = super.lapack-reference;
          };
          blas = super.blas.override {
            blasProvider = super.lapack-reference;
          };
        })];
      }

or, switched at runtime via LD_LIBRARY_PATH like:

    $ LD_LIBRARY_PATH=$(nix-build -E '(with import <nixpkgs> {}).lapack.override { lapackProvider = pkgs.mkl; is64bit = true; })')/lib:$(nix-build -E '(with import <nixpkgs> {}).blas.override { blasProvider = pkgs.mkl; is64bit = true; })')/lib ./your-blas-linked-binary

By default, we use OpenBLAS LP64 also known in Nixpkgs as
openblasCompat.

[1]: https://wiki.debian.org/DebianScience/LinearAlgebraLibraries
[2]: https://wiki.gentoo.org/wiki/Blas-lapack-switch
2020-04-17 16:23:55 -05:00
Matthew Bauer 6367a55312 fetchurl: don’t run ‘postFetch’ on hashed-mirrors
hashed-mirrors are content addressed. So if $outputHash is in the
hashed-mirror, changes from ‘postFetch’ would already be made. So,
running postFetch will end up applying the change /again/, which we
don’t want.
2020-04-16 16:36:56 -05:00
Silvan Mosberger c60e559e13
trivial-builders: Sanitize derivation name
This then supports using functions like writeShellScriptBin with script names
that would be invalid as derivation names
2020-04-15 00:31:57 +02:00
John Ericson c8a6ea5161 Merge remote-tracking branch 'upstream/master' into staging 2020-04-13 22:17:15 -04:00
John Ericson 204d710405 treewide: Get rid of -dumpmachine in favor of static info 2020-04-13 19:26:19 -04:00
Andreas Rammhold 84b91899c3
Merge pull request #85172 from andir/buildRustCrate-proc-macro
buildRustCrate: support proc-macro in default prelude
2020-04-13 23:35:19 +02:00
Matthew Bauer e520d6af29
Merge pull request #84415 from matthewbauer/mb-cross-fixes-april2020
Cross compilation fixes [april 2020]
2020-04-13 16:48:38 -04:00
Andreas Rammhold 310b5f6c34
Merge pull request #84179 from kolloch/build-rust-crate-multi-version-rename
buildRustCrate: Allow version specific dependency renaming + parameter doc
2020-04-13 16:25:16 +02:00
Andreas Rammhold a9fdfebc6b
buildRustCrate: support proc-macro in default prelude 2020-04-13 16:01:21 +02:00
Matthew Bauer 9ef6f1aa99
Merge pull request #84933 from matthewbauer/fix-bazel-copts
build-bazel-package: fix linkopt flags
2020-04-11 15:31:35 -04:00
Matthew Bauer 184cd9f6ff build-bazel-package: fix linkopt flags
These values were incorrect. We need to use NIX_LDFLAGS, not
NIX_LD_FLAGS. Also need to prefix all flags with -Wl, for GCC to
accept it.
2020-04-10 15:41:39 -04:00
Michael Reilly 84cf00f980
treewide: Per RFC45, remove all unquoted URLs 2020-04-10 17:54:53 +01:00
Peter Kolloch bb660fe228 buildRustCrate: Support versioned crate renames 2020-04-10 00:55:44 +02:00
Peter Kolloch 5f9af254a5 buildRustCrate: Document parameters
I know, heretic, but...

I also know that this is not perfect but it is a good start, I think. It
would be nice if this were part of the automatic "nixdoc" function
reference. I'd like guidance if this should be part of the rust section
or something else.
2020-04-10 00:55:05 +02:00
Matthew Bauer 433d5178ca setup-hooks/auto-patchelf.sh: get prefixed readelf 2020-04-06 16:36:29 -04:00
Elis Hirwing 3b6539896b
Merge pull request #83896 from etu/slim-down-default-php-v3
PHP: Make the default package more sane [v3]
2020-04-05 20:00:03 +02:00
talyz 9ac0d9ad61
php.packages.buildPecl: Introduce built-in extension dependencies
Introduce the internalDeps attribute used by mkExtension, to refer to
internal php extensions built by mkExtension.
2020-04-05 16:44:13 +02:00
Frederik Rietdijk 866c5aa090 Merge master into staging-next 2020-04-05 08:33:39 +02:00
Matthew Bauer eab992b4ac
Merge pull request #83112 from bhipple/dev/reltools-cleanup
releaseTools: no-op expression cleanup
2020-04-04 17:18:28 -05:00
Frederik Rietdijk 92124ed660 Merge master into staging-next 2020-04-03 21:54:40 +02:00
Robert Helgesson bf486f784d
emacs: fix setup-hook
This change fixes byte compilation of, e.g., Helm without breaking
builds using, e.g., `trivialBuild`.

See https://github.com/NixOS/nixpkgs/pull/82604#issuecomment-607201755
2020-04-02 23:04:08 +02:00
Léo Gaspard a3ee24b2ff
Merge pull request #83894 from symphorien/unbreak-os-prober-test
Unbreak os prober test
2020-03-31 22:17:27 +02:00
Jan Tojnar dbb4a47de0
Merge branch 'staging-next' into staging 2020-03-31 21:33:00 +02:00
Jan Tojnar 3e0f4e202f
Merge branch 'master' into staging-next 2020-03-31 21:32:15 +02:00
Orivej Desh 7df81361e4 makeDesktopItem: make categories optional
"Application" is deprecated, "Other" is invalid, there are no generic
categories, and the Categories fields is optional per the spec.

Fixes the defaults after #75729.
2020-03-31 04:10:58 +00:00
worldofpeace dcf6e71d73
Merge pull request #75729 from worldofpeace/validate-makeDesktopItem
makeDesktopItem: add desktop file validation
2020-03-30 14:27:40 -04:00
Profpatsch bf3bd5fee1 skawarePackages: support static builds via pkgsStatic
Most of the skaware packages already build just fine with pkgsStatic,
however the wrapper scripts for execline and stdnotify-wrapper needed
the `-lskarlib` argument to go at the end.

`utmps` and `nsss` still fail with this error:

```
exec ./tools/install.sh -D -m 600 utmps-utmpd /bin/utmps-utmpd
/build/utmps-0.0.3.1/tools/install.sh: line 48: can't create /bin/utmps-utmpd.tmp.479: Permission denied
make: *** [Makefile:121: /bin/utmps-utmpd] Error 1
```
2020-03-30 01:35:39 +02:00
Profpatsch 7329dfcc63 buildSkawarePackage: use pname and version directly 2020-03-30 01:23:46 +02:00
Profpatsch 894c74da1f Revert "buildSkawarePackage: pass through extra args"
This reverts commit a50653295d.

The reasons cited were “debugging”, in which case
you can just add the attribute to `buildSkawarePackage`
and “customizing”, which is still possible with
a normal `overrideDerivation`.

The patch removed `outputs` for some reason
(possible oversight), so building nsss failed.
Plus lots of complexity (e.g. don’t forget to add new
arguments to `removeAttrs` otherwise there’s a bug now).
2020-03-30 01:23:46 +02:00
Symphorien Gibol 671dc2a5ca vmTools.diskImageFuns.debian9i386: update source url
referenced packets were removed from the mirrors

Fixes #77396 #80041
2020-03-29 15:30:15 +02:00
Jörg Thalheim e8323a0bf9
buildRustPackage: enable strictDeps
This will improve cross-compiling support by forcing
users to specify buildInputs/nativeBuildInputs correctly.
2020-03-29 14:09:32 +01:00
Jörg Thalheim 10059e4b71
Merge remote-tracking branch 'upstream/master' into HEAD 2020-03-29 14:08:10 +01:00
Peter Kolloch 782b304dba buildRustCrate: Add tests for checking files in outputs.
...and remove superfluous dependency files (*.d).
...and copy dSYM directories on Mac OS when in release=false mode.
2020-03-29 13:00:21 +02:00
Frederik Rietdijk a36be028f5 Merge staging-next into staging 2020-03-28 21:15:15 +01:00
Andreas Rammhold c8de31baa6 buildRustCrateTests: Fix link order test on darwin
As it turns out Darwin does most of the things differently then "normal"
systems. They are using a different shared library extension and require
an obscure commandline parameter that has to be added to every build
system out there. That issue seems to be with clang on Darwin as on
Linux that flag isn't required to build the very same tests (when using
clang).

After adjusting these two details the tests are running fine on the
darwin box that I was able to obtain.
2020-03-28 21:13:16 +01:00
Andreas Rammhold d86bfec309
Merge pull request #83379 from symphorien/rust-link
buildRustCrate: don't sort link flags
2020-03-28 16:02:51 +01:00
Symphorien Gibol 2f7fb1c497 buildRustCrateTests: add regression test for link order 2020-03-28 12:00:00 +00:00
Alyssa Ross 7533876312
buildRustCrate: fewer backslashes
This is a slight readability boost, I think.
2020-03-27 09:56:19 +00:00
Benjamin Hipple 259f3b3b68
Revert "fetchzip, fetchgit: cleanup handling of optional features and whitespace" 2020-03-26 00:35:40 -04:00
Benjamin Hipple 368ea2eb5e
Merge pull request #79581 from bhipple/feature/fetcher-whitespace-clean
fetchzip, fetchgit: cleanup handling of optional features and whitespace
2020-03-26 00:15:24 -04:00
Symphorien Gibol d8b853799d buildRustCrate: don't sort link flags
Linkage order is significant and sorting can result in link errors.
2020-03-25 12:00:00 +00:00
Jan Tojnar 986fbf4799
Merge branch 'staging-next' into staging 2020-03-24 01:51:55 +01:00
Vincent Laporte 0229936bbe buildDunePackage: add a “useDune2” option 2020-03-23 09:12:50 +01:00
Matthew Bauer b0b0a06788 bintools: only add macos flags when targeting macOS
We can’t set this for cross-compiling since we use the GNU linker.
Instead, set these flags only when targetPlatform is macOS.

Fixes #80754

Fixes #83141
2020-03-23 00:32:45 -04:00
Orivej Desh 1b89aa3f7a Merge branch 'master' into staging 2020-03-23 00:53:16 +00:00
Benjamin Hipple 79d875ae77 releaseTools: no-op expression cleanup
No functional change, was just reading through these and cleaning/sanitizing
them a bit while I'm here.
2020-03-21 21:15:33 -04:00
Bryan Gardiner bc21c247fc
patch-shebangs.sh: Fix error messages to go to stderr, not '0' file. 2020-03-20 17:08:10 -07:00
Andreas Rammhold 4e8d0993a9
Merge pull request #82404 from danieldk/build-script-link-fix
buildRustCrate: only link build deps into build script
2020-03-20 17:56:21 +01:00
John Ericson 19a0b38cbe
Merge pull request #82882 from obsidiansystems/armv6-embedded
Misc fixes for armv6 bare metal cross
2020-03-19 10:38:22 -04:00
Benjamin Hipple 05343f6ff1 rust: remove legacy cargo fetcher
We have now migrated every single Rust package in NixPkgs! This deletes the
legacy fetcher, which is now unused.

Resolves #79975
2020-03-18 20:12:32 -07:00
John Ericson bc054004ac cc-wrapper, clang: libstdcxxHook should a propagated build input
Lumping it in with the target platform libraries was incorrect, and
caused eval failures when gcc couldn't be built for the target platform.
2020-03-18 11:28:52 -04:00
Robert Helgesson 2d2de743d0
emacs: improve setup hook
- Add packages installed in a sub-directory of site-lisp, such as
  mu4e, to EMACSLOADPATH.

- Add ELPA packages to EMACSLOADPATH.

- Add each package only once to EMACSLOADPATH. Before, each package
  would typically be added twice for each transitive dependency
  leading to a huge variable for a package having many dependencies.

Fixed #78680
2020-03-14 19:46:28 +01:00
Daniël de Kok 412c72d20f buildRustCrate: sort linker options in-place 2020-03-13 11:21:07 +01:00
Daniël de Kok ea6e048c37 buildRustCrate: only link build deps into build script
According to the Cargo documentation:

> The build script does not have access to the dependencies listed in
> the dependencies or dev-dependencies section (they’re not built
> yet!). Also, build dependencies are not available to the package
> itself unless also explicitly added in the [dependencies] table.

https://doc.rust-lang.org/cargo/reference/build-scripts.html

This change separates linkage of regular dependencies and build
dependencies.
2020-03-13 11:13:27 +01:00
Vladimír Čunát f8d4120c52
Merge branch 'master' into staging-next
Hydra nixpkgs: ?compare=1574844
2020-03-10 17:18:44 +01:00
John Ericson 320334aee7
Merge pull request #82130 from Ericson2314/fetchsvn-cross
fetchsvn: Fix for cross
2020-03-10 09:53:17 -04:00
Andreas Rammhold e968961d14
Merge pull request #82155 from kolloch/buildRustCrate_findMatchingDir
buildRustCrate: Search for matching Cargo.toml in sub directories
2020-03-10 13:27:26 +01:00
Mario Rodas cad87836ac
Merge pull request #81974 from bhipple/feature/rust-doc
buildRustPackage: update docstring comment
2020-03-10 06:38:42 -05:00
Vladimír Čunát 1d9c10c8de
stdenv cc-wrapper: deal with edge-case regressions
Regression introduced in PR #81191 80729b6787.  The file does not exist
somewhere during bootstrap of pkgsStatic.busybox which is used in nix
(by default).

I tested the builds.
2020-03-10 11:09:26 +01:00
Peter Kolloch 8a6638daa9 build-support/rust/buildRustCrate: Search for matching Cargo.toml in sub directories
This is what cargo does for git repositories.

See related issues:

* https://github.com/kolloch/crate2nix/issues/53
* https://github.com/kolloch/crate2nix/issues/33
2020-03-09 15:11:50 +01:00
Peter Kolloch 04e7462ee6 buildRustCrate: refactor colored logging
* Make errors include the crate name and make them much more prominent.
* Move more code into lib.sh
* Already source generated logging code and lib.sh in configure
2020-03-09 14:26:28 +01:00
Jörg Thalheim 115a8e8d14
Merge pull request #81833 from bignaux/appimage-run
appimage-run: unify appimageTools and appimage-run
2020-03-09 11:18:21 +00:00
John Ericson 2607aae93f fetchsvn: Fix for cross
Just use `nativeBuildInputs` at build time.
2020-03-09 02:11:45 -04:00
Domen Kožar 508a2c6849
Merge pull request #80921 from hercules-ci/buildLayeredImage-allow-empty-store
buildLayeredImage: Allow empty store, no paths to add
2020-03-08 23:05:27 +01:00
Bignaux Ronan a3df64c8a7 appimage-exec.sh: simplify and improve getopts options 2020-03-08 20:34:23 +01:00
Bignaux Ronan 49e7871196 appimageTools: wrapType1 = wrapType2; 2020-03-08 18:07:04 +01:00
Bignaux Ronan 95dc1ef1f0 appimageTools: fix appimage-exec.sh passing arguments 2020-03-08 13:53:48 +01:00
Bignaux Ronan 2a6cd0e6c0 appimageTools: fix appimage-exec.sh according to shellcheck 2020-03-08 13:38:32 +01:00
John Ericson 5c4dd08a03
Merge pull request #81191 from clkamp/stdenv-multi-line-libc-cflags
stdenv: Fix gcc multi line libc-cflags
2020-03-07 21:28:33 -05:00
Bignaux Ronan b36d3f2c4a appimage: replace writeScript to a standalone appimage-exec.sh 2020-03-07 20:24:26 +01:00
Benjamin Hipple 37fb7a5568 buildRustPackage: update docstring comment
The inlined readme that we were iterating on has been moved to GitHub
issue #79975, and the default is now the new cargo fetcher, so this
doc comment is out of date.
2020-03-07 09:34:29 -05:00
Christian Lütke-Stetzkamp 80729b6787 stdenv: Fix gcc multi line libc-cflags
which broke at least libstdc++5.

Closes #79761
2020-03-06 10:34:57 +01:00
Frederik Rietdijk 188c3a4d03 Merge staging-next into staging 2020-03-06 07:09:30 +01:00
Matthew Bauer b8102aa32e
Merge pull request #81844 from lopsided98/gcc-separate-output-cross
gcc: place cross-compiled target libraries in lib output
2020-03-05 20:30:35 -05:00
Jan Tojnar ef34994d9a
Merge pull request #81564 from NixOS/fetchpatch-base64
fetchpatch: Allow using base64-encoded hash
2020-03-04 23:32:29 +01:00
Ben Wolsieffer e1831ebea3 gcc: move target-specific runtime libraries to separate output 2020-03-04 16:12:36 -05:00
Jan Tojnar 1a44e325d2
Merge branch 'staging-next' into staging 2020-03-04 21:24:05 +01:00
Jan Tojnar bf83a7cc66
Merge pull request #81572 from bignaux/appimage
appimage: refactoring extractType2 to avoid use of appimage packer
2020-03-04 16:06:09 +01:00
Bignaux Ronan 3443b693cc appimage: fix suspicious ownership 2020-03-04 15:10:11 +01:00
Bignaux Ronan 92b36343a8 appimage: split checks between appimageSignature and appimageType 2020-03-04 13:57:58 +01:00
worldofpeace 29fc27b4ac wrapGAppsHook: Fix #78803
Add to gappsWrapperArgs in preFixupPhases.
2020-03-03 19:32:36 -05:00
Bignaux Ronan 04037d1e03 appimage: unify extract functions 2020-03-04 00:46:00 +01:00
Bignaux Ronan ebaedb5377 appimage: refactoring extractType2 to avoid use of appimage packer 2020-03-03 01:40:57 +01:00
Andreas Rammhold fa6c5c3dc7
buildBazelPackage: prime fixed output derivations
All bazel fixed output derivations should be specific to the bazel
version that was used to generate them. There is not guarantee that the
build will still succeed or reproduces (without the cached fixed output)
if the fetch phase wasn't rerun with a different bazel version.

In the past bazel had been bumped but not all those packages that have
fixed outputs from bazel builds. This lead to compiling and somewhat
working TF versions that couldn't be reproduced without the cached fixed
outputs.
2020-03-03 01:12:26 +01:00
Jan Tojnar 01e352ac44
fetchpatch: Allow using base64-encoded hash
Nix now returns base64-encoded SRI hashes on hash mismatch. Usually,
people copy the returned hashes in TOFU fashion but since base64-encoded
strings can contain slashes, they often broke our use of them for temporary file name.

Escaping them should prevent the failures.
2020-03-02 23:57:40 +01:00
Michael Lingelbach 34296980d1 tensorflow: 1.15.0 -> 1.15.1
* Apply glibc 2.3 patch
* build tensorflow with bazel_1
* Bump openssl version to 1.1
2020-03-02 10:07:37 +01:00
Frederik Rietdijk 21b808cd76 Merge staging-next into staging 2020-02-29 15:29:45 +01:00
Benjamin Hipple ad30a30488 rustPlatform.fetchCargo: handle custom Cargo.lock patchfiles with validation
Previously, we would asssert that the lockfiles are consistent during the
unpackPhase, but if the pkg has a patch for the lockfile itself then we must
wait until the patchPhase is complete to check.

This also removes an implicity dependency on the src attribute coming from
`fetchzip` / `fetchFromGitHub`, which happens to name the source directory
"source". Now we glob for it, so different fetchers will work consistently.
2020-02-28 18:54:23 -08:00
Robert Hensing 6dab1b50a6 buildLayeredImage: Allow empty store, no paths to add
This is useful when buildLayeredImage is called in a generic way
that should allow simple (base) images to be built, which may not
reference any store paths.
2020-02-28 14:59:04 +01:00
Eelco Dolstra 4c0821461e
enable-coverage-instrumentation.sh: Fix unbound variable error
https://hydra.nixos.org/build/113299582
2020-02-28 12:31:50 +01:00
zimbatm b32a057425
srcOnly: remove the ellipsis (#80903)
The function doesn't make use of the extra arguments.
2020-02-24 23:12:37 +00:00
Samuel Dionne-Riel 972678ec74
Merge pull request #80680 from samueldr/fix/71157
runInLinuxVM: Ensure tools requiring /etc/passwd work
2020-02-23 16:05:21 -05:00
Samuel Dionne-Riel 687e2195d8 runInLinuxVM: Ensure tools requiring /etc/passwd work
This includes, but is not limited to:

 * whoami
 * nix >= 2.3.1

See

 * https://github.com/NixOS/nixpkgs/issues/71157
 * https://github.com/NixOS/nixops/issues/1216
 * https://github.com/nix-community/nixops-libvirtd/issues/5
2020-02-20 18:51:17 -05:00
Jan Tojnar 477ea066ca
Merge pull request #80091 from NixOS/npregit-dumpall
nix-prefetch-git: report deepClone & leaveDotGit
2020-02-19 07:37:11 +01:00
Profpatsch 493c357720 pkgs/build-support/writers: use runCommandLocal for scripts
If we just want to write a non-compiled script (e.g. writeDash), it’s
usually a lot faster just doing it locally. That’s what
`runCommandLocal` was introduced for, so let’s use it in `writers`.
2020-02-18 19:53:18 +01:00
(cdep)illabout 55e1529335 symlinkJoin: add more documentation 2020-02-18 16:35:28 +01:00
(cdep)illabout ae607de23d linkFarmFromDrvs: Add a trivial builder to create a linkFarm from a list of derivations 2020-02-18 16:35:28 +01:00
Andreas Rammhold 453589696b
Merge pull request #79816 from andir/buildRustCrate-no-override-dep
buildRustCrate: remove superfluous dependency overrides
2020-02-18 15:18:44 +01:00
Bas van Dijk 4e0a6bc4be symlinkJoin: fix "bash: Argument list too long" error
When the `paths` argument is too big `symlinkJoin` will fail with:

```
while setting up the build environment: executing '/nix/store/rm1hz1lybxangc8sdl7xvzs5dcvigvf7-bash-4.4-p23/bin/bash': Argument list too long
```

This is fixed by passing `paths` as a file instead of as an
environment variable.
2020-02-17 11:45:44 +01:00
Benjamin Hipple 6d881472ef rust: Fix for legacy fetch cargo
See inline comment and #79975 for details.
2020-02-15 17:41:35 -08:00
Jan Tojnar 28faf5bc86
nix-prefetch-git: Fix output mangling
`git repack` and `git gc` sometimes print “Nothing new to pack.”
to stdout, which breaks redirecting output to JSON file.

Let’s move the stdout of all git calls where it is not used to stderr
so that we still receive the info but it does not pollute our output.
2020-02-14 22:15:36 +01:00
Bjørn Forsman d394c82cde
build-fhs-userenv: runCommand -> runCommandLocal (#77253)
It's a trivial derivation, no need to build remotely (it's slower that
way).
2020-02-14 20:16:49 +02:00
lewo d49804cf45
Merge pull request #80102 from nlewo/fix-78744
dockerTools.buildLayeredImage: store all paths passed in final layer (fix 78744)
2020-02-14 09:48:52 +01:00
Mario Rodas 7696369bec
docker-slim: move expression to outside of build-support (#80078) 2020-02-14 09:42:18 +01:00
Antoine Eiche baa78de594 nixosTests.docker-tools: add bulk-layer test
A regression test for https://github.com/NixOS/nixpkgs/issues/78744.
2020-02-14 09:26:26 +01:00
Richard Wallace 3b65b3f6d6 dockerTools.buildLayeredImage: store all paths passed in final layer
Fixes #78744

My previous change broke when there are more packages than the maximum
number of layers. I had assumed that the `store-path-to-layer.sh` was
only ever passed a single store path, but that is not the case if
there are multiple packages going into the final layer. To fix this, we
loop through the paths going into the final layer, appending them to the
tar file and making sure they end up at the right path.
2020-02-14 09:26:26 +01:00
Benjamin Hipple eb11feaa0b treewide: change fetchCargoTarball default to opt-out
Changes the default fetcher in the Rust Platform to be the newer
`fetchCargoTarball`, and changes every application using the current default to
instead opt out.

This commit does not change any hashes or cause any rebuilds. Once integrated,
we will start deleting the opt-outs and recomputing hashes.

See #79975 for details.
2020-02-13 22:41:37 -08:00
Jan Tojnar f19a101a73
nix-prefetch-git: report deepClone & leaveDotGit
Just like in be30ba8e0e
we want `fetchgit (builtins.fromJSON (builtins.readFile ./src.json))` to work.
2020-02-14 06:05:43 +01:00
Benjamin Hipple c1542fc4cd rust: Replace migration README with GitHub issue
The readme was nice to discuss in the implementation PR, but now that this is
merged it's better to have an issue that can be linked against in PRs and
doesn't require further merges to update status.

Ported with a status update in #79975
2020-02-12 22:28:53 -05:00
Frederik Rietdijk 424697d512 Merge master into staging-next 2020-02-12 09:55:31 +01:00
Andreas Rammhold be5597fc9d
buildRustCrate: remove superfluous dependency overrides
By overriding each dependency on every level of the dependency tree we
are creating a lot of unnecessary instances of the same derivation

Looking at the output size of `nix-instantiate --trace-function-calls
-vvvv …` and the execution time I got about a 10x improvement after
applying this change.

It was probably good intentions that lead to these overrides but in
practice no tooling (that I know of) really needs this. `carnix` and
`crate2nix` are fine without those overrides. Furthermore I believe that
it is the job of the tooling around `buildRustCrate` to provide a
coherent set of overrides. By not enforcing all of the overrides, debug
flags, verbosity, … to be the same throughout the closure we also allow
consumers to override specific aspects of the crates. Some (older?)
crates might need different `crateOverrides` then newer crates with the
same name. Currently such situations can not (easily) be implemented
with the override in-place.
2020-02-11 11:48:45 +01:00
Benjamin Hipple 2115a2037c fetchcargo: use flat tar.gz file for vendored src instead of recursive hash dir
This has several advantages:

1. It takes up less space on disk in-between builds in the nix store.
2. It uses less space in the binary cache for vendor derivation packages.
3. It uses less network traffic downloading from the binary cache.
4. It plays nicely with hashed mirrors like tarballs.nixos.org, which only
   substitute --flat hashes on single files (not recursive directory hashes).
5. It's consistent with how simple `fetchurl` src derivations work.
6. It provides a stronger abstraction between input src-package and output
   package, e.g., it's harder to accidentally depend on the src derivation at
   runtime by referencing something like `${src}/etc/index.html`. Likewise, in
   the store it's harder to get confused with something that is just there as a
   build-time dependency vs. a runtime dependency, since the build-time
   src dependencies are tarred up.

Disadvantages are:
1. It takes slightly longer to untar at the start of a build.

As currently implemented, this attaches the compacted vendor.tar.gz feature as a
rider on `verifyCargoDeps`, since both of them are relatively newly implemented
behavior that change the `cargoSha256`.

If this PR is accepted, I will push forward the remaining rust packages with a
series of treewide PRs to update the `cargoSha256`s.
2020-02-10 10:17:29 -05:00
Benjamin Hipple caff183402 fetchzip, fetchgit: cleanup handling of optional features and whitespace
No intended functional difference, just trying to polish to make this easier to
extend and more aligned with other nix conventions.
2020-02-08 21:51:46 -05:00
Andreas Rammhold 56e11bc8df
buildRustCrate: remap the current build dir to / for (more) reproducible builds 2020-02-06 01:18:59 +01:00
Uri Baghin 10d1a04b02 bazel-deps: remove 2020-02-04 12:35:47 +11:00
Uri Baghin 3dc9c1b885 bazel-remote: 2019-01-12 -> 2020-01-29 2020-02-04 02:26:15 +01:00
worldofpeace 8ecbde9def
Merge pull request #78945 from jbedo/singularity-img
singularity-tools: fix buildImage
2020-02-03 15:19:47 -05:00
Justin Bedo 56c22350c0
singularity-tools: fix buildImage 2020-02-02 09:18:39 +11:00
Silvan Mosberger 0a351c3f65
dockerTools.*: Assertion against building for Darwin (#77952)
dockerTools.*: Assertion against building for Darwin
2020-01-31 21:17:40 +01:00
lewo 86f8732194
Merge pull request #78834 from nlewo/fix-two-layers-image
Fix dockerTools.buildLayerImage with 2 layers
2020-01-30 21:10:52 +01:00
Antoine Eiche 01a68479cc dockerTools.buildLayeredImage: assert maxLayers > 1
Since a layer is reserved for "customization", the image can not
contains less than 2 layers.

The user gets the following message at evaluation:

    nix-instantiate nixos/tests/docker-tools.nix
    trace: the maxLayers argument of dockerTools.buildLayeredImage function must be greather than 1 (current value: 1)
2020-01-30 21:09:52 +01:00
Antoine Eiche 283bcc1003 dockerTools.buildLayeredImage: fix image with only 2 layers
A test is also added to ensure an image with 2 layers can be built.
2020-01-30 21:09:44 +01:00
Andreas Rammhold a57d0fe0bb
buildRustCrate: fix #78412
`build.rs` files might create files. Those files are supposed to go into
`OUT_DIR` (envirionment variable) and not be overlayed onto the source
tree.
2020-01-28 14:07:58 +01:00
Andreas Rammhold 19698d15ce
buildRustCrateTests: add regression test for #74071 2020-01-28 14:07:58 +01:00
Frederik Rietdijk 7562c2112d
Merge pull request #77739 from NixOS/staging-next
Staging next
2020-01-28 14:03:29 +01:00
Andreas Rammhold cd27940df0
Merge pull request #78188 from andir/buildRustCrate-support-rlib
buildRustCrate: support `rlib` dependency type
2020-01-28 13:47:57 +01:00
Frederik Rietdijk dce0ca29d9 Merge master into staging-next 2020-01-28 10:46:13 +01:00
Mario Rodas c3ccac15eb
libredirect: fix build on darwin
Fix build failure on darwin due to absence of `O_TMPFILE`.
2020-01-23 20:20:20 -05:00
Andreas Rammhold 78faab1be0 buildRustCrateTests: add test case for rlib linking 2020-01-21 17:46:32 +01:00
Andreas Rammhold 406e0c9d51
buildRustCrateTests: fix some formatting issues 2020-01-21 17:32:48 +01:00
Andreas Rammhold d6a8b55fb0
buildRustCrate: treat rlib crates just like lib crates
Both version provide `rlib` files to link against. Previously we would
try to find a matching shared library in the `lib` output.
2020-01-21 17:22:59 +01:00
Andreas Rammhold 60fea726de
Merge pull request #77822 from andir/buildRustCrate-remove-custom-lib-path
buildRustCrate: remove custom lib path heuristic
2020-01-21 17:21:55 +01:00
Silvan Mosberger b6a9211bf4
dockerTools.*: Assertion against building for darwin
Building a docker image with darwin binaries just yields a confusing
error when ran:

  standard_init_linux.go:211: exec user process caused "exec format error"

This change prevents people from building such images in the first place
2020-01-20 19:14:12 +01:00
Jonas Schürmann 61870727de mirrors.nix: Use HTTPS in maven mirrors
Since 2020-01-15, the maven central repository does no longer support HTTP.
See https://blog.sonatype.com/central-repository-moving-to-https for details.
2020-01-20 09:54:14 +01:00
Mario Rodas 21d0b2067f
Merge pull request #76831 from mbrgm/docker-slim
docker-slim: init at 1.26.1
2020-01-18 07:51:03 -05:00
Andreas Rammhold 69c96adc53
buildRustCrateTests: use releaseTools.aggregate
Previously I did use `runCommand` to do the same. Using
releaseTools.aggregate seems a lot saner and we might get nicer hydra
output of the tests that are failing.
2020-01-16 13:24:15 +01:00
Andreas Rammhold 29a8575e3d
buildRustCrate: remove one of the odd library filename cases
It used to be the case (ref missing) that cargo did treat
`src/$libName.rs` as an alternative to `src/lib.rs` when the latter
wasn't present. Recently I failed to reproduce that with vanilla cargo
and it started to cause pain with some crates of the form:

some_crate/
 `- src
   `- main.rs
   `- some_crate.rs

We would build `src/some_crate.rs` and thing it is a library while that
might not be the actual case. This crate is a valid `bin` crate not a
`lib` crate as far as I can tell from the samples I took.

I removed support for the previously required heuristic and commented
out the test cases in case we will need them again. We could crawl in
the Git history but chances are that the next person looking into this
doesn't know about the history.
2020-01-16 13:24:13 +01:00
Anders Kaseorg 3cd8ce3bce treewide: Fix unsafe concatenation of $LD_LIBRARY_PATH
Naive concatenation of $LD_LIBRARY_PATH can result in an empty
colon-delimited segment; this tells glibc to load libraries from the
current directory, which is definitely wrong, and may be a security
vulnerability if the current directory is untrusted.  (See #67234, for
example.)  Fix this throughout the tree.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2020-01-15 09:47:03 +01:00
Andreas Rammhold cb007e69a1 stdenv: make symlinks that refer to the same output relative
While looking at the graph of all the outputs in my personal binary
cache it became obvious that we have a lot of self references within the
package set. That isn't an isuse by itself. However it increases the
size of the binary cache for every (reproducible) build of a package
that carries references to itself. You can no longer deduplicate the
outputs since they are all unique. One of the ways to get rid of (a few)
references is to rewrite all the symlinks that are currently used to be
relative symlinks. Two build of something that didn't really change but
carries a self-reference can the be store as the same NAR file again.

I quickly hacked together this change to see if that would yield and
success. My bash scripting skills are probably not great but so far it
seem to somewhat work.
2020-01-15 09:26:40 +01:00
Benjamin Hipple 6e8c377562 rustPlatform.buildRustPackage: cleaner output on verifyCargoDeps (#77567)
When this fails, the user may want to copy-paste the path to the "bad"
Cargo.lock file to inspect. The trailing `.` on `$cargoDeps.` gets caught in
most terminal copy-pastes. Since half the lines already don't have it, this
removes it from all of them for consistent output.
2020-01-12 17:19:17 +00:00
Antoine Eiche da261e3631 dockerTools.buildLayeredImage: fix typo in comments 2020-01-11 09:02:30 +01:00
lewo 0d983f9f99
Merge pull request #75911 from Simspace/parallel-docker-buildlayeredimage
when building a layered docker image, ignore it if tar encounters cha…
2020-01-11 08:55:34 +01:00
Andreas Rammhold 3e61906e1c
buildRustCrate: slight "rewording" and reformatting
There is no point in reinventinb builtins through `filterAttrs` or the
like. Lets just stick to what we already have in our toolbelt.
2020-01-07 11:57:34 +01:00
Andreas Rammhold a3a51763f9
buildRustCrate: add buildTests flag to tell rustc to build tests instead of binaries
This helps us instruct rustc to build tests instead of binaries. The
actual build will then ONLY produce test executables. This is a first
step towards having rust crate tests within nixpkgs.

We default back to only a single output in test cases since that is the
only reasonable thing to do here.

Producing libraries or binaries in addition to tests would theoretically
be feasible but usually generates different dependency trees. It is very
common to have some libraries in `[dev-depdendencies]` within Cargo.toml
just for your tests. To not start mixing things up going with a
dedicated derivation for the test build sounds like the best choice for
now.

To use this you must provide a proper test dependency chain to
`buildRustCrate` (as you would usually do with your non-test inputs).
And then set the `buildTests` attribute to `true`. The derivation will
then contain all tests that were built in `$out/tests`. All common test
patterns and directories should be supported and tested by this change.

Below is an example how you would run a single test from the derivation.
This commit contains some more examples in the `buildRustCrateTests`
attribute set that might be helpful.

```
let
  drv = buildRustCrate {
     …
     buildTests true;
  };
in runCommand "test-my-crate" {} ''
  touch $out
  exec ${drv}/tests/my-test
''
```
2020-01-07 11:57:34 +01:00
Andreas Rammhold 6383b42dae
buildRustCrate: fixup usage of builtins.filterSource
While unifying most of the lib function calls I accidentially changed
the filterSource functions as well. Since there were no tests I ended
up forgetting about this case (even thought I ran into it…).
2020-01-07 00:49:48 +01:00
Marius Bergmann d2394a4789 docker-slim: init at 1.26.1
Co-authored-by: Mario Rodas <marsam@users.noreply.github.com>
Co-authored-by: Oleksii Filonenko <brightone@protonmail.com>
2020-01-03 11:18:00 +01:00
Andreas Rammhold 9f03cb8562
Merge pull request #75563 from andir/cleanup-buildRustCrate
Cleanup buildRustCrate expression
2020-01-02 13:42:33 +01:00
John Ericson cfd013813e
Merge pull request #74090 from obsidiansystems/ghcjs-cross-without-cc
stdenv, haskell: bonafied GHCJS cross compilation without stdenv.cc
2019-12-30 16:40:43 -08:00
Richard Wallace 3be767593b dockerTools.buildLayeredImage: fix building layered images in parallel
when tar'ing store paths into layered archives when building layered
images, don't use the absolute nix store path so that tar won't complain
if something new is added to the nix store

when building the final docker image, ignore any file changes tar
detects in the layers. they are all immutable and the only thing that
might change is the number of hard links due to store optimization
2019-12-30 14:47:11 -07:00
Robin Gloster ac8eaa8507
treewide: fix *Flags 2019-12-30 04:50:37 +01:00
Alyssa Ross b9d274b89d rustPlatform.fetchcargo: expose 2019-12-23 18:27:56 +00:00
Alyssa Ross fdfbb4671e rustPlatform: forward unpackPhase to fetchcargo
If a custom unpackPhase is used for the package, it needs to also be
used for fetchcargo so the same source is available for vendoring.
2019-12-23 18:27:56 +00:00
Alyssa Ross 839c9e9344 rustPlatform: forward fetchcargo args to stdenv
Most stdenv wrappers already work like this -- it allows greater
customisation.  We just have to be careful to remove arguments we're
using that shouldn't be passed to stdenv.  I've been conservative
here, because fetchcargo checksums shouldn't change lightly.
2019-12-23 18:27:56 +00:00