Commit graph

3940 commits

Author SHA1 Message Date
CrystalGamma b155b1dafb makeModulesClosure: handle builtin modules better
The previous code discarded entire dependency trees if the first entry in the dependency list compiled by `modprobe --show-depends` is a builtin and otherwise handled its output in a rather hackish way.
2020-07-16 20:44:07 +02:00
Robert Hensing c1908bf802
Merge pull request #92934 from hercules-ci/dockerTools-set-imageTag
dockerTools: Always set imageTag attribute
2020-07-16 17:05:58 +02:00
Maximilian Bosch d2694d936e
rustPlatform: don't install artifacts modified by checkPhase
While the artifacts from `buildPhase` should be used for testing as
well, it should be avoided that those are modified during `checkPhase`.

This can happen if a package is built e.g. with special
`cargoBuildFlags` that don't apply to the `checkPhase`. In that case, a
binary would be installed into `$out` without those flags since
`checkPhase` overrides the binary in the `target`-directory.

This patch copies the state of `target/release` into a temporary
location at the end of the `buildPhase` and installs the results from
that temporary directory into `$out` while `checkPhase` can continue
using the configured build-dir.

cc #91689
Closes #93119
Closes #91191
2020-07-15 20:08:30 +02:00
Maximilian Bosch 7713fba8f8
Revert "buildRustPackage: fix cargoBuildFlags"
This reverts commit deb78151a9.

Mixing up two distinct phases of a derivation's build is not a good idea. See
also https://github.com/NixOS/nixpkgs/pull/91689#issuecomment-657813954.
2020-07-14 17:39:09 +02:00
Robert Hensing 8c0459f611 dockerTools: Always set imageTag attribute
The image tag can be specified or generated from the output hash.
Previously, a generated tag could be recovered from the evaluated
image with some string operations.

However, with the introduction of streamLayeredImage, it's not
feasible to compute the generated tag yourself.

With this change, the imageTag attribute is set unconditionally,
for the buildImage, buildLayeredImage, streamLayeredImage functions.
2020-07-11 16:58:25 +02:00
Frederik Rietdijk ae8eee9c3f Merge master into staging-next 2020-07-11 08:59:54 +02:00
Jos van Bakel 66737168a4
dockerTools.buildLayeredImage: fix created=now 2020-07-09 09:34:18 +02:00
Matthew Bauer 669eb09bd5
Merge branch 'master' into fetchurl-no-hash 2020-07-06 14:40:49 -04:00
Utku Demir 06db331922
dockerTools: Verify nix-store contents on buildLayeredImage test 2020-07-06 16:59:58 +12:00
Utku Demir 2ae2186dfd
dockerTools.streamLayeredImage: Correctly copy hardlinks 2020-07-06 16:42:03 +12:00
Matthew Bauer e7ea3de835 blas: use $NM instead of nm
This is the right one on cross compilation.
2020-07-06 00:25:32 -04:00
Matthew Bauer f42aa7e1d7 cc-wrapper: set FC when langFortran is on
We need to set FC so that CMake and other tools can find the fortran
compiler. Also we need to limit the hardening flags since fortify and
format don’t work with fortran.

Fixes #88449
2020-07-06 00:22:19 -04:00
Utku Demir cc46362929
dockerTools: Support files directly under /nix/store
Also makes sure that the files inside a layer added in a sorted order
to make the results more deterministic.
2020-07-04 22:00:57 +12:00
Flakebi deb78151a9 buildRustPackage: fix cargoBuildFlags
When features were supplied in cargoBuildFlags, the binaries were built
with these features enabled. Unless checking was disabled, `cargo test`
was executed without these build flags, meaning the binaries were
rebuilt and overwritten without the specified features.

Fix this bug by running tests after the installation phase.
2020-07-03 08:53:57 +10:00
Vladimír Čunát 9dcb508f2d
Merge #92089: {cc,bintools}-wrapper: Some fixes 2020-07-02 23:23:04 +02:00
Frederik Rietdijk c33ca7c5ce Merge staging-next into staging 2020-07-02 17:14:53 +02:00
Vincent Laporte 885fb1406d ocamlPackages.buildOasis: fix handling of installation prefix 2020-07-02 07:13:34 +02:00
John Ericson 704daf7a6e {bintools,cc}-wrapper: Ensure nix-support/*-flags files exist
This will unbreak firefox and a few other packages which try to grab
some of the libcxx flags.
2020-06-30 18:26:37 +00:00
John Ericson 1085403acd {cc,bintools}-wrapper: Comments no longer spliced
This means we can freely keep the comments up to date without the
penalty of a mass rebuild.
2020-06-30 15:04:10 +00:00
Alyssa Ross 1421404d42 stdenv: fix backward multiple outputs conditional
This is supposed to shareDocName to a fallback value if it can't be
determined from looking at the configure script.  But the conditional
checked whether shareDocName was set, rather than if it wasn't.  This
meant that if shareDocName had been detected from a configure script,
it would be immediately overridden by the package name, and if it
couldn't be detected, shareDocName would remain unset.

This resulted in QEMU installing files like $out/share/doc/index.html,
which should of course have been in $out/share/doc/qemu/index.html.

An interesting side effect of this is that, since
9f8751528c when this code was added, the
detected package name has never actually been used for installing
documentation, because it would always be overridden.  So this patch
will actually enable that for the first time, four years later.

Fixes: https://github.com/NixOS/nixpkgs/issues/90486
2020-06-29 13:56:27 +00:00
John Ericson fa54dd346f
Merge pull request #85189 from Ericson2314/cxx-wrapper-debt
C++ compilers: Be sane with standard libraries
2020-06-22 10:38:34 -04:00
DavHau c8c09b7dda add autoPatchelfIgnoreMissingDeps to auto-patchelf.sh 2020-06-22 10:41:35 +02:00
John Ericson f3f7612a40 C++ Compilers: Systematize handling of standard libraries 2020-06-22 04:24:44 +00:00
Utku Demir fa8f2bf34f
stream_layered_image: Use pathlib module for directory traversal 2020-06-22 15:11:04 +12:00
Utku Demir 16199521ea
stream_layered_image: Always set uid and gid to root 2020-06-21 12:35:39 +12:00
Utku Demir fe71abfc12
stream_layered_image: Clarify assertion failure 2020-06-21 12:35:39 +12:00
Utku Demir 87a538e149
stream_layered_image: Use more descriptive variable name 2020-06-21 12:35:39 +12:00
Utku Demir 4bff5b7f3d
dockerTools: Properly quote a shell variable 2020-06-21 12:35:39 +12:00
Utku Demir f1d7a09146
stream_layered_image: Add docstrings to functions 2020-06-21 12:35:38 +12:00
Utku Demir 26402290bf
stream_layered_image: Add main method 2020-06-21 12:35:38 +12:00
Utku Demir 307804d97d
dockerTools.streamLayeredImage: comments 2020-06-21 12:35:38 +12:00
Utku Demir 4ab7baf6f6
stream_layered_image.py: comments 2020-06-21 12:35:38 +12:00
Utku Demir a5a611cacb
Set mtime in even more places 2020-06-21 12:35:38 +12:00
Utku Demir 4e0109f873
Set mtime to get more deterministic builds 2020-06-21 12:35:37 +12:00
Utku Demir 560201da66
Implement dockerTools.streamLayeredImage 2020-06-21 12:35:32 +12:00
Robert Hensing 06469493de
Merge pull request #91085 from hercules-ci/fix-dockerTools-nix-symlinks
Fix docker tools nix symlinks
2020-06-19 12:01:47 +02:00
Robert Hensing 2dd94af186 dockerTools: keep symlinks to nix intact 2020-06-19 10:08:34 +02:00
Robert Hensing caf47063b4 dockerTools: test that tar keeps nix binary symlinks intact 2020-06-19 10:08:34 +02:00
Frederik Rietdijk 4af4a49251 Merge master into staging-next 2020-06-16 18:05:38 +02:00
Andreas Rammhold d282a7e07f
Merge pull request #90193 from danieldk/build-rs-feature-env
buildRustCrate: set CARGO_FEATURE_* when running the build script
2020-06-16 14:19:56 +02:00
Lassulus 453014b456
Merge pull request #89245 from alexarice/agda-literate
agda: install literate files
2020-06-16 14:11:01 +02:00
Daniël de Kok fe50bab788 buildRustCrate: set CARGO_FEATURE_* when running the build script
Cargo sets `CARGO_FEATURE_*` for all features when running a build
script:

https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-build-scripts

Some crates have build scripts (e.g. openblas-src) that rely on the
feature variables being properly set.

Since we now need several representations of features, this change
also updates `createFeatures` to be a list of features, rather than
`rustc` feature arguments. `configureCrate` and `buildCrate` then
build the required representations as-needed.

Fixes #68978
2020-06-13 14:09:06 +02:00
Frederik Rietdijk 2e4b4e3300 Merge staging-next into staging 2020-06-13 11:03:26 +02:00
Frederik Rietdijk 1523382160 Merge master into staging-next 2020-06-13 11:02:21 +02:00
volth 54c4223e55 treewide: central.maven.org -> repo1.maven.org 2020-06-12 14:26:08 +00:00
Frederik Rietdijk febc27b59a Merge master into staging-next 2020-06-12 08:57:26 +02:00
Matthew Bauer f5c1ab06af
Merge pull request #89351 from dwarfmaster/fix-emacs-build
Fix building of emacsWithPackages not findind subdirs.el
2020-06-11 10:52:25 -05:00
José Romildo Malaquias 87eb2f4342
Merge pull request #90004 from romildo/buildenv
buildenv: paths to link resolving to the same absolute path is not a conflict
2020-06-11 12:50:31 -03:00
José Romildo Malaquias 4a1ef63c55 buildenv: paths to link resolving to the same absolute path is not a conflict
When building an environment if two paths conflict but one or both are
symbolic links and they resolve to the same real path, the conflict is
discarded because the contents of both paths are the same. One of them
is chosen and there is no need to recur into them in order to build
deeper symbolic links.
2020-06-10 06:42:48 -03:00
Matthew Bauer 0046802ab6 fetchurl: only allow empty hash when cacert is available
We can use cacert to validate that the data passes SSL certificates.
Normally, this doesn’t happen because we already have the hash, but in
the hash = "" case we don’t.
2020-06-09 12:56:27 -05:00
Matthew Bauer f2e9046de5 fetchurl: allow empty hash
Meant as a companion to https://github.com/NixOS/nix/pull/3674

This just resets outputHash if nothing is passed in.
2020-06-09 01:37:10 -05:00
Frederik Rietdijk 8576d24b2a Merge staging-next into staging 2020-06-08 12:08:51 +02:00
Frederik Rietdijk dc33419285 Merge master into staging-next 2020-06-08 12:06:12 +02:00
hyperfekt 0a8fb89fca rustPlatform: fix cross-compiling by using native diff 2020-06-08 10:05:00 +05:30
Frederik Rietdijk 1c68570ab2 Merge staging-next into staging 2020-06-05 19:42:16 +02:00
Frederik Rietdijk 43f71029cc Merge master into staging-next 2020-06-05 19:40:53 +02:00
Jörg Thalheim f0396574ab
buildRustPackage: add documentation on how to create cargo.lock patches 2020-06-05 10:41:46 +01:00
Vladimír Čunát 677e3960b5
Merge #82342: rustPlatform: increase build-speed of checkPhase
...for rust-packages (into staging)
2020-06-05 09:12:30 +02:00
Frederik Rietdijk 08900c0554 Merge master into staging-next 2020-06-04 15:25:54 +02:00
Benjamin Hipple 4af635feff
Merge pull request #87866 from danieldk/validate-pkgconfig-hook
validatePkgConfig: init
2020-06-02 23:27:27 -04:00
DwarfMaster a9eab6d2b9 Fix building of emacsWithPackages not findind subdirs.el 2020-06-02 11:52:39 +02:00
Florian Klink 8a388c8296
Merge pull request #82258 from erikarvstedt/fix-xchg-caching
fix inconsistent caching of VM xchg dirs
2020-06-01 22:34:48 +02:00
Erik Arvstedt d85dc4f690
qemu-vm: fix inconsistent caching of xchg dirs
xchg is advertised as a bidirectional exchange dir, but file content
transfer from host to VM fails due to caching:
If a file is read in the VM and then modified on the host, subsequent
re-reads in the VM can yield old, cached data.
This is caused by the use of 9p's cache=loose mode that is explicitly
meant for read-only mounts.

9p doesn't provide any suitable cache modes, so fix this by disabling
caching.

Also, remove a now unnecessary sync in the test driver.
2020-06-01 21:55:33 +02:00
Alex Rice e215c3bcac
agda: install literate files 2020-06-01 13:59:20 +01:00
Daniël de Kok 13083b1a01 validatePkgConfig: init
This adds the `validatePkgConfig` hook, which can be used to validate
pkg-config files in the output(s). Currently, this will just run
`pkg-config --validate` on all `.pc` files, capturing errors such as
the issue that was fixed in #87789.

The hook could be extended in the future with more fine-grained
checks.
2020-05-31 20:45:03 +02:00
Pavol Rusnak 9444756222
appimage: replace radare2+jq with readelf+awk+sha256sum
radare2 does not play nicely with filenames containing spaces
https://github.com/radareorg/radare2/issues/16958
2020-05-29 11:14:16 +02:00
Frederik Rietdijk 362d88c2b1 Merge staging-next into staging 2020-05-27 15:27:28 +02:00
Michael Howell c21cbf22d0
buildRustCrate: Replace hyphen with underscore in env variables (#88054)
* Add test case for include dir
* buildRustCrate: replace hyphen with underscore in env

This fixes a bug that prevents encoding_c from building.
2020-05-26 20:52:18 +02:00
adisbladis d8b6e82126
Merge pull request #88856 from zowoq/prefetch-docker
dockerTools: fix skopeo commands
2020-05-25 09:52:09 +02:00
rnhmjoj aee614c996
treewide: replace bazaar with breezy 2020-05-25 09:22:54 +02:00
zowoq 365d07cea0 dockerTools: fix skopeo commands 2020-05-25 14:29:54 +10:00
Florian Klink 9b7a4694a0 fetchgitlab: escape rev to support "+" in it 2020-05-24 22:24:50 +02:00
Shea Levy fffa6e8c18
Merge pull request #88762 from alexbiehl/alex/docker-tools
dockerTools: Properly add /nix/ and /nix/store/ first to layer.tar
2020-05-24 08:34:42 -04:00
Alex Biehl 886c92332c dockerTools: Properly add /nix/ and /nix/store/ first to layer.tar
In https://github.com/NixOS/nixpkgs/pull/58431 the authors ensured that
the resulting layer.tar would always list

  /nix/
  /nix/store/

first to fully comply to the tar spec. Various refactorings later it is only
ensured to create /nix/ but NOT /nix/store anymore. Instead tar transformed
them to /nix/nix and /nix/nix/store.
2020-05-24 14:28:35 +02:00
Frederik Rietdijk d578248611 Merge staging-next into staging 2020-05-24 10:10:06 +02:00
Matthew Bauer c74a9cd3e2
Merge pull request #69144 from SFrijters/gdc
gdc: init at 9.3.0
2020-05-23 16:30:47 -05:00
Orivej Desh 16d7f7edae Merge branch 'master' into staging 2020-05-22 09:13:23 +00:00
John Ericson eae4f22176
Merge pull request #88518 from Ericson2314/fix-pkg-config
pkg-config, pkgconf: Misc fixes
2020-05-21 16:05:42 -04:00
Profpatsch 8b270a96fa skawarePackages.cleanPackaging: copy docFiles before rm noise 2020-05-21 19:46:38 +02:00
John Ericson 7e9d807f2c pkg-config-wrapper: Move comments outside of strings
This is much better because then we can freely keep the comments up to
date without causing mass rebuilds.

Someday, somebody should make the same change with `cc-wrapper` and
`bintools-wrapper`.
2020-05-21 11:35:46 -04:00
John Ericson 95080ea90b pkgconf: Wrap with pkg-config-wrapper 2020-05-21 11:35:46 -04:00
John Ericson 4b2ab665fb pkg-config-wrapper: Provide autoconf macros
This fixes libarchive, and probably numerous other builds.
2020-05-21 09:16:34 -04:00
John Ericson f29919da61 pkg-config: Multiple outputs
This is good practice, and will help with tests. Wrapper adjust to match
bintools-wrapper re these new outputs.
2020-05-21 08:58:34 -04:00
Stefan Frijters fc9b93d2fc gdc: init at 9.3.0 2020-05-19 23:06:52 +02:00
lewo a498da343a
Merge pull request #87154 from utdemir/buildimage-optimizations
Some performance optimizations to dockerTools.build{,Layered}Image
2020-05-19 15:39:25 +02:00
Jan Tojnar f8a9c6efac
Merge branch 'staging-next' into staging 2020-05-18 21:09:48 +02:00
John Ericson c71ab32a67 pkg-config-wrapper: Init
This fixes longstanding build issues
2020-05-16 00:21:21 +00:00
John Ericson ee8cfe4bf3 treewide: pkg-config has targetPrefix 2020-05-16 00:21:21 +00:00
zowoq 42232493a3 dockerTools: pass insecure-policy and tmpdir to skopeo 2020-05-16 08:46:13 +10:00
lewo df928fafd4
Merge pull request #87725 from nlewo/fix-dockertools
dockerTools.examples: fix the nginx image
2020-05-15 16:54:09 +02:00
lewo 98a723e6b3
Merge pull request #87253 from utdemir/dockertools-preserve-env
Preserve environment variables from the parent image on dockerTools.buildImage
2020-05-15 09:59:00 +02:00
Dmitry Kalinkin 9943fd1a1d
Merge pull request #76653 from alexarice/agda-rework
Agda rework
2020-05-14 18:02:50 -04:00
Alex Rice d30e2468e0
agda: rework builder 2020-05-14 20:54:11 +01:00
Frederik Rietdijk 404fe35d65 Merge staging-next into staging 2020-05-14 09:37:03 +02:00
Matthew Bauer d38bad590e
Merge pull request #83180 from matthewbauer/only-add-ldflags-for-macos
bintools: only add macos flags when targeting macOS
2020-05-13 10:34:40 -05:00
Antoine Eiche 9ac444d80e dockerTools.examples: fix the nginx image
Nginx now requires directories `/var/log/nginx` and `/var/cache/nginx`
to be existing.

The objective is to fix the test `nixosTests.docker-tools`.
2020-05-13 10:29:53 +02:00
Maximilian Bosch 6b23cfe689
rustPlatform: add buildAndTestSubdir-argument
There are several tarballs (such as the `rust-lang/rust`-source) with a
`Cargo.toml` at root and several sub-packages (with their own Cargo.toml)
without using workspaces[1].

In such a case it's needed to move into a subdir to only build the
specified sub-package (e.g. `rustfmt` or `rsl`), however the artifacts
are at `/target` in the root-dir of the build environment. This breaks
the build since `buildRustPackage` searches for executables in `target`
(which is at the build-env's root) at the end of the `buildPhase`.

With the optional `buildAndTestSubdir`-argument, the builder moves into
the specified subdir using `pushd`/`popd` during `buildPhase` and
`checkPhase`.

Also moved the logic to find executables and libs to the end of the `buildPhase`
from a custom `postBuild`-hook to fix packages with custom `build`/`install`-procedures
such as `uutils-coreutils`.

[1] https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html
2020-05-13 01:47:17 +02:00
Maximilian Bosch 736462d995
rustPlatform: make it possible to override the profile for cargo test 2020-05-13 01:39:44 +02:00
John Ericson 1ac5398589 *-wrapper; Switch from infixSalt to suffixSalt
I hate the thing too even though I made it, and rather just get rid of
it. But we can't do that yet. In the meantime, this brings us more
inline with autoconf and will make it slightly easier for me to write a
pkg-config wrapper, which we need.
2020-05-12 00:44:44 -04:00
Matthew Bauer 43545032af
Merge pull request #87314 from matthewbauer/bazel-flat
build-bazel-package: switch hash mode to “flat”
2020-05-11 15:27:48 -05:00