Commit graph

1861 commits

Author SHA1 Message Date
Jan Malakhovski 67a7b42167 pkgs/stdenv/booter.nix: fix a typo 2019-03-14 14:01:20 +00:00
Matthew Bauer 9edb851308
Merge pull request #51628 from ju1m/arm
stdenv: add armv7a-linux system
2019-03-01 15:36:07 -05:00
Frederik Rietdijk 2fcb11a244 Merge staging-next into master 2019-03-01 09:06:20 +01:00
Matthew Bauer b86e62d30d llvm: support cross compilation with useLLVM flag
You can build (partially) with LLVM toolchain using the useLLVM flag.
This works like so:

  nix-build -A hello --arg crossSystem '{ system =
    "aarch64-unknown-linux-musl"; useLLVM = true }'

also don’t separate debug info in lldClang

It doesn’t work currently with that setup hook. Missing build-id?
2019-02-26 19:45:35 -05:00
Daiderd Jordan 7ec53a932f
stdenv: only set __darwinAllowLocalNetworking on darwin
This is a darwin only nix attribute for sandbox builds, it can be
ignored on other platforms to avoid unnecessary rebuilds.
2019-02-14 21:38:08 +01:00
Matthew Bauer 5c09d977c7 Merge remote-tracking branch 'origin/master' into staging 2019-02-09 12:14:06 -05:00
Matthew Bauer c6f8f8d98d make-derivation: only modify name when name is given
This preserves Nix’s native error handling of missing name:

  error: derivation name missing
2019-02-04 09:15:03 -05:00
Vincent Laporte 0d17ecce2c
mkDerivation: cleaner handling of the name argument 2019-02-01 16:02:42 +00:00
Vladimír Čunát 8ba516664b
Merge branch 'staging-next' into staging 2019-02-01 09:42:53 +01:00
Vladimír Čunát 5effa4e0f9
Merge branch 'master' into staging-next
Comments on conflicts:
- llvm: d6f401e1 vs. 469ecc70 - docs for 6 and 7 say the default is
  to build all targets, so we should be fine
- some pypi hashes: they were equivalent, just base16 vs. base32
2019-02-01 09:22:29 +01:00
Matthew Bauer 8020bd6869
Merge pull request #35884 from dtzWill/fix/man-in-outputsToInstall
default to including "man" in outputsToInstall
2019-01-31 14:55:12 -05:00
Matthew Bauer e2fe4c2d49 make-derivation: fix ordering of conditionals
cross should have higher precedence
2019-01-28 11:38:30 -05:00
Matthew Bauer 9fd1c170cc make-derivation: try to fix stdenv.cc == null conditional
sometimes this gets an infinite recursion error
2019-01-28 10:41:19 -05:00
Matthew Bauer 7e589e5594 make-derivation: fix position in trace
For a long time now, tracing has been broken in Nixpkgs. So when you
have an eval error you would get something like this:

  error: while evaluating the attribute 'buildInputs' of the derivation 'hello-2.10' at /home/mbauer/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:185:11:
  while evaluating 'chooseDevOutputs' at /home/mbauer/nixpkgs/lib/attrsets.nix:474:22, called from undefined position:
  while evaluating 'optionals' at /home/mbauer/nixpkgs/lib/lists.nix:257:5, called from /home/mbauer/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:132:17:

This is coming from how Nix handles string context and how
make-derivation messes with the "name" attribute. This commit should
restore the old behavior so you get a nice line number like:

  error: while evaluating the attribute 'buildInputs' of the derivation 'hello-2.10' at /home/mbauer/nixpkgs/pkgs/applications/misc/hello/default.nix:4:3:
  while evaluating 'chooseDevOutputs' at /home/mbauer/nixpkgs/lib/attrsets.nix:474:22, called from undefined position:
  while evaluating 'optionals' at /home/mbauer/nixpkgs/lib/lists.nix:257:5, called from /home/mbauer/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:132:17:

NOTE: This will still be broken for cross compilation due to the
prefixes we are adding to name.
2019-01-28 09:25:47 -05:00
Matthew Bauer d54fbbebbb
Merge pull request #49552 from matthewbauer/setup-dedupe
setup.sh: avoid running the same hook twice
2019-01-27 18:22:41 -05:00
Matthew Bauer 9172c1eee2 setup.sh: avoid running the same hook twice
In strictDeps=false, we don’t want the same package hook to be run
twice, otherwise we get duplicates in some flags.

Fixes #41340
2019-01-27 17:49:13 -05:00
Matthew Bauer ae16dd1a15
stdenv/make-derivation: don't hide broken packages
This behavior ended up breaking the handleEvalIssue functionality by hiding those packages. So something like this:

$ nix-env -iA nixpkgs.zoom-us

would silently fail, without telling the user how to fix it! Regardless, this "bug" should be handled in Nix - not Nixpkgs.

Fixes #38952.
2019-01-27 17:02:17 -05:00
Matthew Bauer bd62ac6b88
Merge pull request #54652 from matthewbauer/shell-in-flags-array
setup.sh: put SHELL in flagsArray
2019-01-27 13:55:54 -05:00
Matthew Bauer 329913f733 setup.sh: put SHELL in flagsArray
We don’t want to modify makeFlags, that is given to us by our
environment. Adding to it could lead to duplicates after repeated use.

Fixes #27533
2019-01-26 21:14:36 -05:00
Matthew Bauer 8babcc3d44
Merge branch 'master' into fix/man-in-outputsToInstall 2019-01-26 10:51:12 -05:00
Frederik Rietdijk a1a5ea5943 stdenv: make checkInputs native
We can't run the checkPhase when build != host, so we may as well make
the checkInputs native.

This signicantly improves the situation of Python packages when enabling
strictDeps.
2019-01-13 14:43:18 +01:00
Orivej Desh (NixOS) 9a21967f0a
stdenv: prune libtool files by default (#51767)
See the motivation in fd97db43bc (#41819).
2019-01-11 13:20:46 +00:00
Franz Pletz 237deba4e7
Merge remote-tracking branch 'origin/master' into staging 2019-01-10 16:00:34 +01:00
Dylan Simon 04beae0979 stdenv: allow custom stdenv to take crossOverlays 2019-01-07 20:53:10 -06:00
Matthew Bauer d02bb3c197 darwin.make-bootstrap-tools: remove dsymutil
This hopefully is not needed for the bootstrap tools. Needs more testing.
2019-01-06 22:15:43 -06:00
Daniel Goertzen 1c10efc912 add generic x86_32 support (#52634)
* add generic x86_32 support

- Add support for i386-i586.
- Add `isx86_32` predicate that can replace most uses of `isi686`.
- `isi686` is reinterpreted to mean "exactly i686 arch, and not say i585 or i386".
- This branch was used to build working i586 kernel running on i586 hardware.

* revert `isi[345]86`, remove dead code

- Remove changes to dead code in `doubles.nix` and `for-meta.nix`.
- Remove `isi[345]86` predicates since other cpu families don't have specific model predicates.

* remove i386-linux since linux not supported on that cpu
2019-01-06 12:57:36 -06:00
Frederik Rietdijk 092e3b50a8 Merge master into staging-next 2019-01-02 21:08:27 +01:00
Andrew Dunham 8efe3df795 stdenv/native: add missing argument 2019-01-02 12:19:25 -05:00
Will Dietz 1bb4e5c8dc
Merge pull request #36384 from dtzWill/musl-native-bootstrap
x86_64-musl bootstrap: refresh bootstrap tools package
2018-12-31 14:59:56 -06:00
Jan Tojnar c45e9d0fac
Merge branch 'master' into staging 2018-12-25 17:03:57 +01:00
Jörg Thalheim 1b146a8c6f
treewide: remove paxutils from stdenv
More then one year ago we removed grsecurity kernels from nixpkgs:
https://github.com/NixOS/nixpkgs/pull/25277

This removes now also paxutils from stdenv.
2018-12-22 12:55:05 +01:00
Frederik Rietdijk 9ab61ab8e2 Merge staging-next into staging 2018-12-19 09:00:36 +01:00
Sander van der Burg 2757ddb182 Separate androidndkpkgs from androidenv 2018-12-18 21:14:43 +01:00
Graham Christensen a375d4bfc3
stdenv: shorten evaluation errors when in Hydra
Hydra's page showing evaluation errors is about a mile long, showing
buckets of user-friendly errors, like this:

    in job ‘seyren.aarch64-linux’:
    Package ‘oraclejre-8u191’ in /nix/store/fa9zzkbljkvdavwzirkrr5irg25ymbjl-source/pkgs/development/compilers/oraclejdk/jdk-linux-base.nix:71 has an unfree license (‘unfree’), refusing to evaluate.

    a) For `nixos-rebuild` you can set
      { nixpkgs.config.allowUnfree = true; }
    in configuration.nix to override this.

    b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
      { allowUnfree = true; }
    to ~/.config/nixpkgs/config.nix.

    in job ‘jetbrains.webstorm.x86_64-linux’:
    Package ‘webstorm-2018.3.1’ in /nix/store/fa9zzkbljkvdavwzirkrr5irg25ymbjl-source/pkgs/applications/editors/jetbrains/default.nix:230 has an unfree license (‘unfree’), refusing to evaluate.

    a) For `nixos-rebuild` you can set
      { nixpkgs.config.allowUnfree = true; }
    in configuration.nix to override this.

    b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
      { allowUnfree = true; }
    to ~/.config/nixpkgs/config.nix.

This makes it extremely hard to find actual issues in the output. This
patch set makes the output much more condensed in Hydra:

    Failed to evaluate nifticlib-2.0.0: «unsupported»: is not supported on ‘x86_64-apple-darwin’
    Failed to evaluate dmd-2.081.2: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
    Failed to evaluate dmdBuild-2.081.2: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
    Failed to evaluate ldc-1.11.0: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
    Failed to evaluate ldcBuild-1.11.0: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
    Failed to evaluate ldc-0.17.5: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
    Failed to evaluate ldcBuild-0.17.5: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
2018-12-18 10:50:53 -05:00
Jan Tojnar aead6e12f9
Merge remote-tracking branch 'upstream/master' into staging 2018-12-16 22:55:06 +01:00
James Kay eb76dd37a7
makeStaticBinaries: add a static glibc when performing a mkDerivation 2018-12-14 11:02:18 +00:00
Matthew Bauer 9564b8ed9e cctools: don’t depend on clang at runtime
--disable-clang-as flag disables the use of clang by cctools-port
2018-12-10 17:31:47 -06:00
Piotr Bogdan 44c9c27d54 stdenv: prune libtool files by default 2018-12-09 22:45:15 +00:00
Frederik Rietdijk 5f554279ec Merge master into staging-next 2018-12-07 15:22:35 +01:00
Julien Moutinho 3168e85833 stdenv: add armv7a-linux system 2018-12-06 17:55:25 +01:00
Matthew Bauer 0b8574540b stdenv/darwin: fix portable libsystem hook
Some packages don’t have /bin directories. We should only run
install_name_tool if that directory exists.
2018-12-05 12:56:12 -06:00
Matthew Bauer 6d90a8b894 top-level/stage.nix: add static overlay
Adds the static overlay that can be used to build Nixpkgs statically.
Can be used like:

  nix build pkgsStatic.hello

Not all packages build, as some rely on dynamic linking.
2018-12-04 21:56:03 -06:00
Matthew Bauer 8726f6a558 stdenv/adapters.nix: fixup makeStaticBinaries
- makeStaticBinaries don’t work on Darwin (no stable ABI!)
- Need to make sure NIX_CFLAGS_LINK appends
- isStatic is not used anymore
2018-12-04 21:12:17 -06:00
Matthew Bauer a3a6ad7a01 stdenv: implement crossOverlays
crossOverlays only apply to the packages being built, not the build
packages. It is useful when you don’t care what is used to build your
packages, just what is being built. The idea relies heavily on the
cross compiling infrastructure. Using this implies that we need to
create a cross stdenv.
2018-12-04 21:06:46 -06:00
Jan Tojnar c5881ec2c9
Merge remote-tracking branch 'upstream/master' into staging 2018-11-30 20:09:45 +01:00
Matthew Bauer 28e2277305 make-derivation: remove selfConsistent check
version is set in lots of places but might not need to be in a name.

Alternative to #50364.
2018-11-30 17:58:33 +01:00
Frederik Rietdijk 1828a5c5ba Merge master into staging-next 2018-11-30 17:46:21 +01:00
Matthew Bauer e6834171b9
Merge pull request #51183 from matthewbauer/make-derivation-no-assertions
Fix breaking changes to make-derivation
2018-11-29 19:35:55 -06:00
Vaibhav Sagar b5504e26b0 pkgs/stdenv/generic/make-derivation.nix: s/targetPlatform/hostPlatform/ 2018-11-29 17:54:59 -05:00
Frederik Rietdijk 9b81c7e455 Merge staging-next into staging 2018-11-29 09:18:35 +01:00
Matthew Bauer a56fe056ec make-derivation: don’t add host suffix if there is no c compiler
Some trivial builders use the name attr to choose the exec name
produced. For example nixos-install,

  nixos-install = makeProg {
    name = "nixos-install";
    src = ./nixos-install.sh;
    nix = config.nix.package.out;
    path = makeBinPath [ nixos-enter ];
  };

When cross compiling, this puts the prog in,

  /bin/nixos-install-powerpc64le-unknown-linux-gnu
2018-11-28 12:44:10 -06:00
Matthew Bauer d64f288154 make-derivation: don’t assert on separateDebugInfo
When separateDebugInfo = true & !stdenv.hostPlatform.isLinux, we
always gave an error. There is no reason to do this. Instead, just
don’t add separate debug info when we aren’t on Linux.
2018-11-28 12:29:00 -06:00
Matthew Bauer 13e45aeed3 make-derivation: remove selfConsistent check
version is set in lots of places but might not need to be in a name.

Alternative to #50364.
2018-11-28 12:26:21 -06:00
Léo Gaspard 8ecd555ded
Merge branch 'pr-46056' into staging
* pr-46056:
  binutils: use shared libs
  binutils: fix #44936 the huge size regression
2018-11-28 09:55:05 +09:00
Vladimír Čunát a5de78b7d7
Merge branch 'master' into staging-next 2018-11-26 10:28:00 +01:00
Matthew Bauer 7eeb02d47b Revert "make-derivation: add disallowedReferences in strictDeps"
This reverts commit 8dbfb61e46.
Also reverts commit fc99c337ed.

Fixes #50915
2018-11-25 15:53:28 -06:00
Frederik Rietdijk c31cb577ae Merge master into staging-next 2018-11-22 09:57:08 +01:00
Matthew Bauer fc99c337ed make-derivation: don’t disallow propagated native inputs
propagateNativeBuildInputs will end up going in the output derivation.
This case is allowed to end up in references because of that. Sorry
for the disruption!

Fixes #50865
2018-11-21 09:39:47 -06:00
Daiderd Jordan c9223a17bc
Revert "patch-shebangs: use --build for auto patch shebangs"
Completely breaks darwin. Every package in the stdenv that has shebangs
in the output will end up with references to bootstrap-tools.

This reverts commit eb7c50a993.
2018-11-14 23:37:31 +01:00
Daiderd Jordan 25fafd2eb5
Revert "darwin/stdenv: bash is a build input"
Completely breaks darwin. Every package in the stdenv that has shebangs
in the output will end up with references to bootstrap-tools.

This reverts commit 2f2e635dd5.
2018-11-14 23:37:28 +01:00
Matthew Bauer f19bb8321a
Merge pull request #35304 from volth/patch-97
[staging] substitute() print warning if does nothing
2018-11-12 12:52:51 -06:00
Frederik Rietdijk 3b052406ea Merge staging-next into staging 2018-11-12 19:01:36 +01:00
Frederik Rietdijk daf3297cb4 Merge master into staging-next 2018-11-12 18:59:08 +01:00
Léo Gaspard 2986ce16a8
meta.tests: rename into passthru.tests
Nix currently rejects derivations in `meta` values. This works around
that limitation by using `passthru` instead.

Closes https://github.com/NixOS/nixpkgs/issues/50230
2018-11-11 23:11:46 +09:00
Léo Gaspard cae5598611
meta.tests: drop meta.needsVMSupport
Its job is already handled by `requiredSystemFeatures`
2018-11-11 23:11:46 +09:00
Léo Gaspard 83b27f60ce
tests: split into a separate all-tests.nix file
This will make the list much easier to re-use, eg. for `nixosTests`

The drawback is that this approaches makes the
```
nix-build release.nix -A tests.opensmtpd.x86_64-linux
```
command about twice as slow (3s to 6s): it now has to evaluate `nixpkgs`
once for each architecture, instead of just having the hardcoded list of
tests that allowed to say “ok just evaluate for x86_64-linux”.

On the other hand, complete evaluation of `release.nix` should be much
faster because we no longer import `nixpkgs` for each test: testing with
the following command went from 30s to 18s, and that's just for a few
tests.
```
time nix-instantiate --eval --strict nixos/release.nix -A tests.nat
```
I initially wanted to test on the whole `release.nix`, but there are too
many broken tests and it takes too long to eval them all, especially
compared to the fact that the current implementation breaks some setup.

Given developers can just `nix-build nixos/tests/my-test.nix`, it sounds
like an overall win.
2018-11-11 23:11:46 +09:00
Frederik Rietdijk 1d3bff25db Merge staging-next into staging 2018-11-11 14:28:08 +01:00
Frederik Rietdijk 53d00c3351 Merge master into staging-next 2018-11-10 11:08:54 +01:00
Matthew Bauer c8aff96110
Merge pull request #49608 from matthewbauer/cross-patch-shebangs-2
Restore cross-patch-shebangs branch
2018-11-07 13:37:02 -06:00
Matthew Bauer 6d531f3541 make-derivation: enable pie hardening with musl
Fixes #49071

On ld.gold, we produce broken executables when linking with the Musl
libc. This appears to be a known bug when using ld.gold and Musl. This
thread describes the workaround as enabling PIE when using ld.gold and
Musl:

https://www.openwall.com/lists/musl/2015/05/01/5

By default we don’t enable PIE to avoid breaking things. But in the
Musl case we are breaking things by not enabling PIE. So this adds a
special case for defaultHardeningFlags which keeps the pie hardening
for everything. Any packages that break with PIE can add the pie flag
to disableHardeningFlags array (a no-op for now on anything but Musl).
2018-11-07 15:24:51 +01:00
Timo Kaufmann 6141939d6e
Merge pull request #44439 from Ekleog/meta-tests
[RFC] Use `meta.tests` to link from packages to the tests that test them
2018-11-07 00:05:22 +01:00
Patrick Hilhorst 320c9c10de
make-derivation: use pname-version as default name if both are present 2018-11-06 00:04:21 +01:00
Frederik Rietdijk 322f87137c Merge master into staging-next 2018-11-04 11:33:17 +01:00
Frederik Rietdijk abea6f461a Revert "Merge pull request #49398 from Synthetica9/implement-rfc0035" to fix eval
This reverts commit 3fc7d5eb83, reversing
changes made to 1fddf2b689.

The idea is good, however, before enforcing, make sure all occurences
are fixed.
2018-11-04 11:02:43 +01:00
Frederik Rietdijk cb4ff927a1 Merge master into staging-next 2018-11-04 08:49:24 +01:00
Jörg Thalheim 3fc7d5eb83
Merge pull request #49398 from Synthetica9/implement-rfc0035
Implement rfc0035: default `name` from `pname`
2018-11-03 21:33:36 +00:00
Matthew Bauer 8dbfb61e46 make-derivation: add disallowedReferences in strictDeps
When strictDeps = true, we don’t want native build inputs to end up in
the output. For instance gcc is a builtin native build input and
should only show up in an output if it is also listed in buildInputs.

/cc @ericson2314
2018-11-02 19:31:51 -05:00
Matthew Bauer 2f2e635dd5 darwin/stdenv: bash is a build input
patch shebangs needs to be in build inputs for it to get into
HOST_PATH.
2018-11-02 00:27:14 -05:00
Matthew Bauer eb7c50a993 patch-shebangs: use --build for auto patch shebangs
In strictDeps=false, autoPatchshebangs should use
--build (corresponding to PATH) to lookup commands. This restores the
previous behavior of patchshebangs so that we don’t break stuff that
isn’t careful in the buildInputs vs. nativeBuildInputs distinction.
Unfortunately this won’t work under cross compilation.
2018-11-02 00:27:14 -05:00
John Ericson e3082c313b Merge remote-tracking branch 'upstream/master' into release-lib-cleanup 2018-11-01 16:47:42 -04:00
John Ericson f2ed7c7af9 linux bootstrap tools: Use right system for some raw derivations
This allows cross builds to work. Evidentallyy this has been done wrong
since I combined the bootstrap tool creation files in
ab651d2c9b. Oops!
2018-11-01 16:22:00 -04:00
John Ericson ffaffb36d1 linux bootstrap-tools: use stdenv.*Platform to avoid deprecation warning 2018-11-01 16:18:51 -04:00
Patrick Hilhorst 1f7fc09176
make-derivation: use a more descriptive assert message
As suggested by @Profpatsch
2018-10-30 14:33:14 +01:00
Léo Gaspard 02e1f00ffd
dovecot, opensmtpd: add link to test in meta.tests
Rationale
---------

Currently, tests are hard to discover. For instance, someone updating
`dovecot` might not notice that the interaction of `dovecot` with
`opensmtpd` is handled in the `opensmtpd.nix` test.

And even for someone updating `opensmtpd`, it requires manual work to go
check in `nixos/tests` whether there is actually a test, especially
given not so many packages in `nixpkgs` have tests and this is thus most
of the time useless.

Finally, for the reviewer, it is much easier to check that the “Tested
via one or more NixOS test(s)” has been checked if the file modified
already includes the list of relevant tests.

Implementation
--------------

Currently, this commit only adds the metadata in the package. Each
element of the `meta.tests` attribute is a derivation that, when it
builds successfully, means the test has passed (ie. following the same
convention as NixOS tests).

Future Work
-----------

In the future, the tools could be made aware of this `meta.tests`
attribute, and for instance a `--with-tests` could be added to
`nix-build` so that it also builds all the tests. Or a `--without-tests`
to build without all the tests. @Profpatsch described in his NixCon talk
such systems.

Another thing that would help in the future would be the possibility to
reasonably easily have cross-derivation nix tests without the whole
NixOS VM stack. @7c6f434c already proposed such a system.

This RFC currently handles none of these concerns. Only the addition of
`meta.tests` as metadata to be used by maintainers to remember to run
relevant tests.
2018-10-30 21:31:39 +09:00
Patrick Hilhorst c7e026bec4
make-derivation: use lib.assertMsg
As suggested by @Profpatsch
2018-10-29 18:25:59 +01:00
Patrick Hilhorst 5be927db14
make-derivation: use ? instead of builtins.hasAttr
As suggested by @edolstra
2018-10-29 15:17:13 +01:00
Patrick Hilhorst 2962f94fec
make-derivation: add check that the name is consistent with pname and version 2018-10-29 14:58:12 +01:00
Patrick Hilhorst 149a55eca7
make-derivation: get position info from version 2018-10-29 14:51:22 +01:00
Patrick Hilhorst efca8b4b97
make-derivation: use pname-version as default name if both are present 2018-10-29 14:51:08 +01:00
Matthew Bauer b3041b4455 make-derivation: set CMAKE_SYSTEM_* when cross compiling
Uses uname data to find what to set these variables:

- CMAKE_SYSTEM_NAME
- CMAKE_SYSTEM_PROCESSOR
- CMAKE_SYSTEM_VERSION
- CMAKE_HOST_SYSTEM_NAME
- CMAKE_HOST_SYSTEM_PROCESSOR
- CMAKE_HOST_SYSTEM_VERSION
2018-10-16 21:50:37 -05:00
Yegor Timoshenko cd0c8739d7
Merge pull request #37600 from abbradar/impureusenative
impureUseNativeOptimizations: add stdenv adapter
2018-10-13 14:09:55 +00:00
Matthew Bauer 93834fe194
Merge pull request #47230 from bhipple/fix/licenses
Remove dead code from stdenv check-meta license logic
2018-10-05 22:58:27 -05:00
John Ericson f49ca01c50 Revert "stdenv: partial revert of f2bb59e"
This reverts commit 607063f61b.
2018-09-26 14:47:16 -04:00
Matthew Bauer 607063f61b stdenv: partial revert of f2bb59e
/cc @Ericson2314

PR was https://github.com/NixOS/nixpkgs/pull/46857

This line broke MacOS cross compilation. paxctl cannot be built on
macOS. Maybe it can be fixed, but no reason to break things
unnecessarily.

Regardless, you definitely need to be more careful about backporting.
I think it’s fine to move fast and break things on master but
with release-18.09 we should be more careful. Something like more
automated testing for cross compilation would also be
helpful (hopefully even making it block).

(cherry picked from commit f9c4075873cb56464126f993d22a1a72f7cfac45)
2018-09-26 11:13:22 -04:00
xeji 6aa5f2db8f
Merge pull request #47245 from dtzWill/fix/coreutils-8.30-bootstrap
coreutils: try 8.30 again, fix bootstrap tools expression motivating revert before
2018-09-25 12:10:27 +02:00
aszlig b25b6e0c75
stdenv: Improve ELF detection for isELF
The isELF function only checks whether ELF is contained within the first
4 bytes of the file, which is a bit fuzzy and will also return
successful if it's a text file starting with ELF, for example:

  ELF headers
  -----------

  Some text here about ELF headers...

So instead, we're now doing a precise match on \x7fELF.

Signed-off-by: aszlig <aszlig@nix.build>
Acked-by: @Ericson2314
Closes: https://github.com/NixOS/nixpkgs/pull/47244
2018-09-25 06:55:18 +02:00
John Ericson 2b4b7d4ef3
Merge pull request #47233 from oxij/tree/mass-rebuild-noop-cleanups
treewide: mass rebuild noop cleanups
2018-09-25 00:04:52 -04:00
Will Dietz 4d9f9f171b make-bootstrap-tools: fix with latest coreutils
Since gcc.lib/lib64 is a symlink to 'lib', the use of
"lib*/libgcc_s.so*" triggered a warning (error) with
the latest coreutils.  Essentially we were doing:

$ cp a/x b/x y/

And latest coreutils rejects such invocations.

Just copy from 'lib', lib64 is a link to it anyway.

* Nothing else in this file bothers looking at lib*
* AFAICT lib* only ever possibly matched lib64 anyway
2018-09-23 14:54:09 -05:00
Jan Malakhovski b2c7a5a271 bintools-wrapper, cc-wrapper, stdenv: infer propagateDoc automatically
02c09e0171 (NixOS/nixpkgs#44558) was reverted in
c981787db9 but, as it turns out, it fixed an issue
I didn't know about at the time: the values of `propagateDoc` options were
(and now again are) inconsistent with the underlying things those wrappers wrap
(see NixOS/nixpkgs#46119), which was (and now is) likely to produce more instances
of NixOS/nixpkgs#43547, if not now, then eventually as stdenv changes.

This patch (which is a simplified version of the original reverted patch) is the
simplest solution to this whole thing: it forces wrappers to directly inspect the
outputs of the things they are wrapping instead of making stdenv guess the correct
values.
2018-09-23 17:29:56 +00:00
Benjamin Hipple 0b9d9ab256 Remove dead code from stdenv check-meta license logic
The `unfree` and `unfreeRedistributable` licenses both have `free = false`,
which will trigger the first portion of logic. This removes dead code to
simplify the logic.

As a follow-up, I plan to add an attribute `redistributable = [true|false]`,
which can be used by Hydra to determine whether a given package with a given
license can be included in the channel.
2018-09-23 12:48:02 -04:00
Frederik Rietdijk 56853dc6d8
Merge pull request #45941 from NixOS/staging-next
Staging next
2018-09-23 09:31:28 +02:00
Dan Peebles eeeeacc9a6 Revert "stdenv/darwin: bump bootstrap tools"
This accidentally added some unwanted dependencies on the bootstrap
tools, and I don't have time to fix before I go on vacation, so I'm
backing it out until I have time to address it properly.

This reverts commit dc5c68a7bb.
2018-09-20 23:43:53 -04:00
John Ericson 7319013ea1 Merge remote-tracking branch 'upstream/master' into staging 2018-09-18 16:55:42 -04:00
John Ericson 35378f0141
Merge pull request #46857 from obsidiansystems/darwin-to-linux-prep
misc pkgs: various cross fixes in preparation for darwin->linux
2018-09-18 16:52:30 -04:00
John Ericson f2bb59e710 stdenv linux, stdenv cross: Harmonize extraNativeBuildInputs
Want to make sure these are the same per host platform, without duplication.
2018-09-18 16:27:04 -04:00
John Ericson 2111e7b742 mkDerivation: Make separateDebugInfo assertion lazier to match other assertions
This is needed to access attributes of derivations on platforms where
they cannot be built.
2018-09-18 16:25:19 -04:00
Frederik Rietdijk de419917a3 Merge master into staging-next 2018-09-18 18:44:48 +02:00
Graham Christensen b80c9ce4a9
stdenv: Validate meta.outputsToInstall
If meta.outputsToInstall is set to include absent outputs, various
tools break including channel updates and nix-env.

    grahamc@Morbo> nix-env -i -f . -A elf-header-real
    installing 'elf-header'
    error: this derivation has bad 'meta.outputsToInstall'

This patch verifies each value in meta.outputsToInstall is a valid
output. It validates this condition only if checkMeta is true.

    grahamc@Morbo> nix-build . -A elf-header-real
    error: Package ‘elf-header’ in /home/grahamc/projects/nixpkgs/pkgs/development/libraries/elf-header/default.nix:36 has invalid meta.outputsToInstall, refusing to evaluate.

    The package elf-header has set meta.outputsToInstall to: bin

    however elf-header only has the outputs: out

    and is missing the following ouputs:

      - bin

    (use '--show-trace' to show detailed location information)

Note, now the nix-env experience is decidedly worse for users who have
checkMeta set to true:

    grahamc@Morbo> nix-env -i -f . -A elf-header-real; echo $?
    0

though since this is already an issue for unfree, broken, unsupported,
and insecure validity problems I'm not sure we should do something
different here.
2018-09-18 10:38:44 -04:00
Dan Peebles dc5c68a7bb stdenv/darwin: bump bootstrap tools
You can verify the provenance of these yourself by checking Hydra here:
https://hydra.nixos.org/build/81511173
2018-09-17 18:46:26 -04:00
Matthew Bauer ba5717a6f5 stdenv: fix HOST_PATH change
a4630c65ca was incorrect in assuming $SHELL would be a path to the
bash derivation. In fact $SHELL will be a path to the bash executable.

Unfortunately this did not fix the original issue. So instead, we just
have to reuse initialPath can be added like PATH is.

Sorry for the inconvenience! I hadn’t thought through the effects of
the last commit.

/cc @copumpkin @ericson2314
2018-09-17 14:18:06 -05:00
Daniel Peebles 481dd45e61
Merge pull request #46730 from copumpkin/add-darwin-lto
cctools: support LTO on Darwin
2018-09-17 00:59:22 -04:00
Matthew Bauer a4630c65ca stdenv: add shell to HOST_PATH for backwards compatibility
To avoid breaking things, we need to make sure SHELL goes into
HOST_PATH. This reflects my changes to patch-shebangs to make it cross
compilation ready. When a script is patched from the Nix store it now
looks to HOST_PATH to get the targeted machine’s executables.
Unfortunately, this only works in native builds.
2018-09-16 15:58:21 -05:00
Dan Peebles 110c252870 cctools: support LTO on Darwin
LTO is disabled during bootstrap to keep the bootstrap tools small and
avoid unnecessary LLVM rebuilds, but is enabled in the final stdenv
stage and should be usable by normal packages.
2018-09-16 02:12:11 -04:00
Dan Peebles 4efd4053ed stdenv/darwin: integrate a new CoreFoundation
This also updates the bootstrap tool builder to LLVM 5, but not the ones
we actually use for bootstrap. I'll make that change in a subsequent commit
so as to provide traceable provenance of the bootstrap tools.
2018-09-15 16:05:46 -04:00
Uli Baum 1df2560dde Merge branch 'master' into staging-next 2018-09-13 10:08:53 +02:00
Jan Malakhovski b7bd0561be Merge branch 'master' into staging 2018-09-08 22:08:32 +00:00
John Ericson 24209d29f0
Merge pull request #46148 from obsidiansystems/plain-system-host
top-level, stdenv: Make `system` and `stdenv.system` describe the hostPlatform
2018-09-06 09:37:20 -04:00
John Ericson 773233ca77 top-level, stdenv: Make system and stdenv.system describe the hostPlatform.
Intuitively, one cares mainly about the host platform: Platforms differ
in meaningful ways but compilation is morally a pure process and
probably doesn't care, or those difference are already abstracted away.
@Dezgeg also empirically confirmed that > 95% of checks are indeed of
the host platform.

Yet these attributes in the old cross infrastructure were defined to be
the build platform, for expediency. And this was never before changed.
(For native builds build and host coincide, so it isn't clear what the
intention was.)

Fixing this doesn't affect native builds, since again they coincide. It
also doesn't affect cross builds of anything in Nixpkgs, as these are no
longer used. It could affect external cross builds, but I deem that
unlikely as anyone thinking about cross would use more explicit
attributes for clarity, all the more so because the rarity of inspecting
the build platform.
2018-09-06 08:33:51 -04:00
John Ericson 32df8909e5
Merge pull request #46076 from oxij/tree/cleanups
trivial: treewide: random noop cleanups
2018-09-05 15:14:12 -04:00
John Ericson 51907d257c stdenv, neovim: Use lib.warn for deprecation warnings 2018-09-05 11:40:29 -04:00
Jan Malakhovski c63ca0a431 stdenv: implement enableParallelChecking option
Works similarly to `enableParallelBuilding`, but is set by default when
`enableParallelBuilding` is set. In my experience most packages that build
fine in parallel also check fine in parallel.
2018-09-05 01:14:39 +00:00
Jan Malakhovski 7931d455a4 stdenv: linux: cleanup a bit 2018-09-04 22:04:56 +00:00
Vladimír Čunát beb063a103
binutils: use shared libs 2018-09-04 21:36:07 +02:00
Vladimír Čunát 1428d00aa4
Merge branch 'master' into staging-next
Hydra: ?compare=1477053
2018-09-04 13:06:45 +02:00
John Ericson 06cd7c15a3 mkDerivation: Fix cross compilation
Derivations where drawing their `system` attribute from `hostPlatform`
instead of `buildPlatform`. Fix that, and add an explanatory commment.

Fixes #45993
2018-09-03 23:18:24 -04:00
Vladimír Čunát 2d6179d1e8
Merge branch 'master' into staging
A few trivial conflicts due to *Platforms mass replace.
2018-09-01 17:38:18 +02:00
John Ericson 82110ae656 stdenv: Better message for deprecated isArm
The message should say what to do instead.
2018-08-31 16:01:58 -04:00
John Ericson 0828e2d8c3 treewide: Remove usage of remaining redundant platform compatability stuff
Want to get this out of here for 18.09, so it can be deprecated
thereafter.
2018-08-30 17:20:32 -04:00
John Ericson 2c2f1e37d4 reewide: Purge all uses stdenv.system and top-level system
It is deprecated and will be removed after 18.09.
2018-08-30 17:20:32 -04:00
Vladimír Čunát b71b222b4c
Merge #44910: cmake: 3.11.2 -> 3.12.1 2018-08-30 16:04:52 +02:00
Jörg Thalheim 9efffe0135 hurd: cleanup unmaintained target
This has been not touched in 6 years. Let's remove it to cause less
problems when adding new cross-compiling infrastructure.
This also simplify gcc significantly.
2018-08-28 22:18:02 +01:00
CrystalGamma 72d161f548 [RFC] ppc64le enablement (#45340)
* ppc64le enablement

* gcc, glibc: properly handle __float128

* lib/systems, stdenv: syntax cleanup

* gcc7: remove ugly hack

* gcc: add/update __float128 flags

* stdenv: add another pair of quotes for consistency

* gcc: move __float128 flag for ppc64le-glibc into common/platform-flags.nix
2018-08-21 15:31:34 -04:00
xeji 55e6e1b05e
Merge pull request #44932 from oxij/stdenv/no-texinfo-rebuild-try-2
stdenv: linux: inherit texinfo in stage4 instead of the final stdenv
2018-08-19 23:54:17 +02:00
Daiderd Jordan ea1542e9ee
Merge pull request #43140 from LnL7/macos-10.14
darwin: fix Libsystem compatibility for macOS 10.14
2018-08-14 23:02:04 +02:00
Andrew Childs e841ffec18 darwin stdenv stage1: use default cmake 2018-08-13 11:05:54 +09:00
Jan Malakhovski 1acbc939c5 stdenv: linux: inherit texinfo in stage4 instead of the final stdenv
This reverts commit a809fdc8e1 and then
achieves the same result (not rebuilding texinfo three times)
but without dragging bootstrap tools into the closure.
2018-08-12 11:47:18 +00:00
Eelco Dolstra c981787db9
Revert "cc-wrapper, bintools-wrapper: simply symlink man and info outputs"
This reverts commit 02c09e0171.
2018-08-09 12:57:38 +02:00
John Ericson fe68c9d450
Merge pull request #44517 from oxij/stdenv/shadows
stdenv: simplify a bit using shadowing
2018-08-08 15:20:39 -04:00
Jan Malakhovski 02c09e0171 cc-wrapper, bintools-wrapper: simply symlink man and info outputs
See discussion in #44516.
2018-08-06 20:50:16 +00:00
Jan Malakhovski eb5b313b16 stdenv: shadow outputs 2018-08-05 19:24:46 +00:00
Jan Malakhovski a53504f4a4 stdenv: shadow doCheck and doInstallCheck 2018-08-05 19:24:32 +00:00
John Ericson 7a337cde31
Merge pull request #44423 from obsidiansystems/configureFlags-cleanup
treewide: All configureFlags are lists
2018-08-03 17:49:01 -04:00
John Ericson 3e034bac1a stdenv adapter: Keep configureFlags a list 2018-08-03 17:06:03 -04:00
John Ericson df5d1673ea
Merge pull request #44367 from obsidiansystems/no-stdenv-is-cross
treewide: Remove stdenv.isCross
2018-08-02 16:36:01 -04:00
John Ericson 34da7e2ce2 treewide: Remove stdenv.isCross
I *want* cross-specific overrides to be verbose, so I rather not have
this shorthand. This makes the syntactic overhead more proportional to
the maintainence cost. Hopefully this pushes people towards fewer
conditionals and more abstractions.
2018-08-02 15:01:58 -04:00
James Deikun bd63de114e stdenv/build-support: support .tbz and .txz tarballs 2018-07-30 15:30:16 -04:00
Frederik Rietdijk 8db716d9f3 Merge master into staging-next 2018-07-28 12:31:35 +02:00
Daiderd Jordan 3ec81ae27e
llvm-packages: fix manpages with python-boot and cleanup
On darwin llvmPackages is built using python-boot to avoid dependencies
in the stdenv, but we can't and shouldn't use that when building the
manpages since it depends on python packages.
2018-07-28 00:08:26 +02:00
Frederik Rietdijk f59e6f4037 Merge master into staging 2018-07-26 16:18:53 +02:00
John Ericson 87357fc39c cross stdenv: Forget allowedRequisites = nulll; on inline 2018-07-25 03:58:49 -04:00
John Ericson b81ee9df9f makeStdenvCross: Remove
It is inlined into the cross stdenv, which is its last use-case after
the previous commit.
2018-07-24 20:01:43 -04:00
John Ericson c12b50b267 stdenv cross adapter: Use extraNativeBuildInputs
This means we don't need to hackily avoid overwriting `mkDerivation`.
2018-07-24 20:01:43 -04:00
John Ericson 87b4a5d6a7 stdenv: Remove crossAttrs, now that it is no longer used
Fix #33302
2018-07-24 20:01:43 -04:00
Frederik Rietdijk 099c13da1b Merge staging-next into master (#44009)
* substitute(): --subst-var was silently coercing to "" if the variable does not exist.

* libffi: simplify using `checkInputs`

* pythonPackges.hypothesis, pythonPackages.pytest: simpify dependency cycle fix

* utillinux: 2.32 -> 2.32.1

https://lkml.org/lkml/2018/7/16/532

* busybox: 1.29.0 -> 1.29.1

* bind: 9.12.1-P2 -> 9.12.2

https://ftp.isc.org/isc/bind9/9.12.2/RELEASE-NOTES-bind-9.12.2.html

* curl: 7.60.0 -> 7.61.0

* gvfs: make tests run, but disable

* ilmbase: disable tests on i686. Spooky!

* mdds: fix tests

* git: disable checks as tests are run in installcheck

* ruby: disable tests

* libcommuni: disable checks as tests are run in installcheck

* librdf: make tests run, but disable

* neon, neon_0_29: make tests run, but disable

* pciutils: 3.6.0 -> 3.6.1

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/pciutils/versions.

* mesa: more include fixes

mostly from void-linux (thanks!)

* npth: 1.5 -> 1.6

minor bump

* boost167: Add lockfree next_prior patch

* stdenv: cleanup darwin bootstrapping

Also gets rid of the full python and some of it's dependencies in the
stdenv build closure.

* Revert "pciutils: use standardized equivalent for canonicalize_file_name"

This reverts commit f8db20fb3a.
Patching should no longer be needed with 3.6.1.

* binutils-wrapper: Try to avoid adding unnecessary -L flags

(cherry picked from commit f3758258b8895508475caf83e92bfb236a27ceb9)
Signed-off-by: Domen Kožar <domen@dev.si>

* libffi: don't check on darwin

libffi usages in stdenv broken darwin. We need to disable doCheck for that case.

* "rm $out/share/icons/hicolor/icon-theme.cache" -> hicolor-icon-theme setup-hook

* python.pkgs.pytest: setupHook to prevent creation of .pytest-cache folder, fixes #40273

When `py.test` was run with a folder as argument, it would not only
search for tests in that folder, but also create a .pytest-cache folder.
Not only is this state we don't want, but it was also causing
collisions.

* parity-ui: fix after merge

* python.pkgs.pytest-flake8: disable test, fix build

* Revert "meson: 0.46.1 -> 0.47.0"

With meson 0.47.0 (or 0.47.1, or git)
things are very wrong re:rpath handling
resulting in at best missing libs but
even corrupt binaries :(.

When we run patchelf it masks the problem
by removing obviously busted paths.
Which is probably why this wasn't noticed immediately.

Unfortunately the binary already
has a long series of paths scribbled
in a space intended for a much smaller string;
in my testing it was something like
lengths were 67 with 300+ written to it.

I think we've reported the relevant issues upstream,
but unfortunately it appears our patches
are what introduces the overwrite/corruption
(by no longer being correct in what they assume)

This doesn't look so bad to fix but it's
not something I can spend more time on
at the moment.

--

Interestingly the overwritten string data
(because it is scribbled past the bounds)
remains in the binary and is why we're suddenly
seeing unexpected references in various builds
-- notably this is is the reason we're
seeing the "extra-utils" breakage
that entirely crippled NixOS on master
(and probably on staging before?).

Fixes #43650.

This reverts commit 305ac4dade.

(cherry picked from commit 273d68eff8f7b6cd4ebed3718e5078a0f43cb55d)
Signed-off-by: Domen Kožar <domen@dev.si>
2018-07-24 15:04:48 +01:00
John Ericson 611eefb067
Merge pull request #44025 from obsidiansystems/stdenv-deps-typo
mkDerivation: depsHostBuild -> depsHostHost
2018-07-23 16:09:21 -04:00
John Ericson ce2637c97a mkDerivation: depsHostBuild -> depsHostHost
`depsHostBuild` is not a thing, would never be a thing per the rules,
and isn't used anywhere. This is just my typo, hitherto unnoticed
because "host -> host" dependencies are by far the most obscure form.
2018-07-23 15:22:48 -04:00
Matthew Bauer cd91d1ad9f setup.sh: add HOST_PATH
HOST_PATH contains the path of the host package. This will include the
packages listed in buildInputs & depsHostHost. Use this to find
runtime commands that the host needs.

For instance to find the runtime version of perl,

$ PATH="$HOST_PATH" command -v perl
/nix/store/...-perl-5.28.0-aarch64-unknown-linux-android/bin/perl

This path should not be executed directly (it will break for cross
compilation). Only use it to find the location of executables that
will be run by your host system. Your build tools will, as always, be
available on the default PATH.
2018-07-22 23:14:49 -04:00
Frederik Rietdijk 3209775495 Merge master into staging 2018-07-22 11:14:41 +02:00
Daiderd Jordan d3cc05e607
Merge pull request #43561 from LnL7/darwin-llvm-boot
stdenv: cleanup darwin bootstrapping
2018-07-21 22:37:17 +02:00
Matthew Bauer 37273afd27
Merge pull request #43224 from volth/patch-179
[staging] substitute(): --subst-var was silently coercing to "" if the variable does not exist.
2018-07-21 15:48:14 -04:00
Frederik Rietdijk ad47d658d1 Merge master into staging-next 2018-07-21 19:43:53 +02:00
Daiderd Jordan 93c34c7616
stdenv: cleanup darwin bootstrapping
Also gets rid of the full python and some of it's dependencies in the
stdenv build closure.
2018-07-21 11:42:37 +02:00
volth 6d2857a311 [bot] treewide: remove unused 'inherit' in let blocks 2018-07-20 19:38:19 +00:00
volth 5ff872aa24 substituteStream(): print warning if nothing done 2018-07-10 22:47:34 +00:00
Frederik Rietdijk f8701caafc Merge master into staging 2018-07-10 15:51:20 +02:00
aszlig 739c835515
stdenv-setup: Remove superfluous check for /bin/sh
The line was essentially checking whether /bin/sh exists and is
executable and if that's the case, the isScript function returns
successfully.

When asking the author of this line on IRC it seems that even they can't
remember or imagine what this was supposed to be.

In summary: Whenever /bin/sh doesn't exist during a build, *any* file
given to isScript is reported as being a script even if it isn't.

This is kinda counter-intuitive and not something what somebody would
expect from a function called "isScript".

Signed-off-by: aszlig <aszlig@nix.build>
Cc: @edolstra
2018-07-09 20:43:34 +02:00
Will Dietz 218d4dc154 make-derivation: Don't add host-suffix to fixed-output derivations names
Not only does the suffix unnecessarily reduce sharing, but it also breaks
unpacker setup hooks (e.g. that of `unzip`) which identify interesting tarballs
using the file extension.

This also means we can get rid of the splicing hacks for fetchers.
2018-07-09 11:07:10 -04:00
volth 0de0ce5893 substitute(): --subst-var was silently coercing to "" if the variable does not exist. 2018-07-08 16:32:03 +00:00
Daiderd Jordan 5024e4aa39
stdenv-bootstrap-tools: update unpack to use $reexportedLibrariesFile 2018-07-07 11:43:27 +02:00
Will Dietz f3693b0647 make-bootstrap-tools-cross: remove broken i686-musl variant
Not terribly difficult to get this working, but until it does
remove it so the cross jobset doesn't have the failures this introduces.
2018-06-25 11:16:10 -05:00
Frederik Rietdijk 95eb2fa40e Merge staging into master 2018-06-23 08:23:24 +02:00
Matthew Bauer c8fd285c8d android: add ndkVer to resolve ndk ambiguity
It wasn’t exactly clear which NDK you were using previously. This adds
an attribute to system that handles what version of the NDK we should
use when building things.

/cc @Ericson2314
2018-06-22 11:06:17 -04:00
Matthew Bauer cf09ffe9aa android: Use NDK 17 for aarch32 2018-06-22 09:33:25 -04:00
John Ericson b3d957649f Merge remote-tracking branch 'upstream/master' into staging 2018-06-18 15:30:39 -04:00
John Ericson 627aae5884 darwin stdenv: Put back "man" attribute on clang and llvm
We take care to make it use the final stdenv to avoid mass rebuilds and
bootstrap python.
2018-06-18 14:47:29 -04:00
John Ericson 6e7e22da70 llvm 5: split out compiler-rt and remove libcxxabi dep
We already did them on non-mass-rebuild llvm 6. Also, this allows
simplifying the stdenv booting.

We were missing the libcxxabi dep in compile-rt in llvm 6, so fixed that
too.
2018-06-14 19:22:15 -04:00
John Ericson 1eacf21bd4 darwin bootstrapping: Avoid overriding aliases for LLVM
Respect the fix points and aliases by overriding originals.
2018-06-14 15:05:18 -04:00
John Ericson d7895c2810 darwin stdenv: fix llvmPackage overrides
It may seem nice and abstract to just override the default version, but
that breaks the alias relationship where the original llvmPackages_* is
no longer in sync. Put another away, modifying the referee rather
instead of breaking the reference "copy-on-write" is impossible.
2018-06-13 16:25:12 -04:00
John Ericson ee9dc37e04
Merge pull request #40933 from obsidiansystems/linux-to-darwin
stdenv, binutils: Build cctools targeting macOS on Linux without pointless rebuilds
2018-05-23 11:37:43 -04:00
John Ericson aac8be76f5
Merge pull request #40992 from obsidiansystems/stdenv-darwin-persist
darwin stdenv: Properly compose overrides
2018-05-23 10:56:11 -04:00
John Ericson 983e74ae4e stdenv: Avoid targetPlatform.isDarwin causing a mass rebuild
We want `buildPackages` to be almost the same as
`buildPackages.buildPackges`, but that is only true if most packages
don't care about the target platform. The commented code however made
them all care about whether the target platform was Darwin.
2018-05-23 10:06:08 -04:00
John Ericson 17316643fd Merge remote-tracking branch 'upstream/master' into staging 2018-05-23 09:59:57 -04:00
John Ericson f567a851a1 darwin stdenv: Properly compose overrides
`super` usage was very suspect.
2018-05-23 09:59:29 -04:00
John Ericson 64c90e53e7 darwin stdenv: Persist darwin.binutils-unwrapped
binutils-unwrapped was added in ef3db7d14c
and needs a corresponding `persisted` attr in the Darwin stdenv.
2018-05-23 09:51:16 -04:00
Tuomas Tynkkynen 003473613a Merge remote-tracking branch 'upstream/master' into staging
Conflicts:
	pkgs/top-level/all-packages.nix
2018-05-18 03:54:38 +03:00
Bastian Köcher 832a8ca087 androidndk: Fix usage as crossSystem 2018-05-17 17:22:27 +02:00
John Ericson 5a860c20f3 stdenv cross adapter: Don't define crossConfig
All its uses have been removed.
2018-05-14 23:33:12 -04:00
John Ericson 5e17335bd7 Merge remote-tracking branch 'upstream/staging' into strictDeps 2018-05-14 23:33:03 -04:00
John Ericson 330ca731e8 treewide: Get rid of all uses of crossConfig
The hack of using `crossConfig` to enforce stricter handling of
dependencies is replaced with a dedicated `strictDeps` for that purpose.
(Experience has shown that my punning was a terrible idea that made more
difficult and embarrising to teach teach.)

Now that is is clear, a few packages now use `strictDeps`, to fix
various bugs:

 - bintools-wrapper and cc-wrapper
2018-05-14 23:30:37 -04:00
John Ericson d00cc1242f
Merge pull request #40040 from obsidiansystems/gnu-config-arm
gnu-config: Update, allowing hacks to be removed
2018-05-14 11:20:09 -04:00
John Ericson f2b575bd7b Merge remote-tracking branch 'upstream/master' into gnu-config-arm 2018-05-14 10:58:15 -04:00
John Ericson eebd455dc0 linux stdenv: Update gnu-config on all non-x86
Not just Aarch64. Other non-x86 platforms might be old enough, but I am
about to update gnu-config to handle things better across the board.
2018-05-14 10:55:32 -04:00
Daiderd Jordan f14841f3ce
Revert "darwin.libSystem: multiple outputs"
Broke the stdenv.

This reverts commit 69d1b7ab79.
2018-05-13 10:01:52 +02:00
Frederik Rietdijk a18b493e02 Merge master into staging 2018-05-09 10:48:01 +02:00
John Ericson 5bdfe37daf
Merge pull request #40154 from obsidiansystems/stdenv-arm-alias
stdenv: Put back isArm, with deprecation notice.
2018-05-08 14:24:58 -04:00
John Ericson 302c4c5f2d stdenv: Put back isArm, with deprecation notice.
This was always meant to be deprecated rather than removed.
2018-05-07 20:14:52 -04:00
John Ericson 4f7cdd35d5
Merge pull request #40139 from obsidiansystems/modular-setup-hooks
treewide: Modular setup hooks
2018-05-07 15:32:10 -04:00
John Ericson 34a3233a2e stdenv: Support concatenating setup hooks from multiple parts. 2018-05-07 14:43:22 -04:00
Matthew Justin Bauer 5206574be3
stdenv: add libSystem.dev to allowedRequisites 2018-05-03 18:02:57 -05:00
Matthew Justin Bauer eeb016e8f0
Merge branch 'staging' into fix-ncurses-darwin-extensions 2018-05-02 15:40:38 -05:00
Michael Raskin a591d28c17
Merge pull request #39464 from oxij/stdenv/docheck-infra
stdenv: implement most of #33599
2018-04-30 17:17:24 +00:00
Michael Raskin ce929e6a92
Merge pull request #39457 from oxij/stdenv/texinfo-bash
stdenv: change texinfo bootstrap handling; texinfo, bash: simplify expressions
2018-04-27 19:03:26 +00:00
Michael Raskin 6ca343103b check-meta.nix: specify meta.timeout 2018-04-27 00:25:37 +02:00
Jan Malakhovski 87651b32fe stdenv: steal checkInputs from buildPythonPackage
Note that a bunch of non-python packages use this attribute already.
Some of those are clearly unaware of the fact that this attribute does
not exists in stdenv because they define it but don't to add it to
their `bulidInputs` :)

Also note that I use `buildInputs` here and only handle regular
builds because python and haskell builders do it this way and I'm not
sure how to properly handle the cross-compilation case.
2018-04-26 20:22:51 +00:00
Jan Malakhovski 845fa56921 stdenv: cleanup things a little bit 2018-04-26 20:15:51 +00:00
Jan Malakhovski d834ba6654 stdenv: introduce and use config.doCheckByDefault option 2018-04-26 20:15:47 +00:00
Will Dietz f31b147f63 linux bootstrap: remove assertion failure from dev, fixes 'nix eval'
As in:

$ nix eval -f . bash

Also remove the glibc propagation inherit that made these necessary,
stages handle propagating libc themselves (apparently) and
AFAICT no hashes are changed as a result of this.
2018-04-26 10:19:52 -05:00
Jan Malakhovski 50af975d85 stdenv: implement checkTarget and installCheckTarget autodetection 2018-04-25 19:53:25 +00:00
Jan Malakhovski e9e06888ed stdenv: generic/setup.sh: cleanup installPhase 2018-04-25 19:53:25 +00:00
Jan Malakhovski ad98c36f1b stdenv: generic/setup.sh: simplify buildPhase Makefile check 2018-04-25 19:53:24 +00:00
Jan Malakhovski a809fdc8e1 stdenv: linux: stop recompiling texinfo
This makes stdenv bootstraping a bit more efficient.
We don't recompile most of other stuff used in bootstrapping stdenv too.
2018-04-25 19:50:54 +00:00
John Ericson b9acfb4ecf treewide: isArm -> isAarch32
Following legacy packing conventions, `isArm` was defined just for
32-bit ARM instruction set. This is confusing to non packagers though,
because Aarch64 is an ARM instruction set.

The official ARM overview for ARMv8[1] is surprisingly not confusing,
given the overall state of affairs for ARM naming conventions, and
offers us a solution. It divides the nomenclature into three levels:

```
ISA:             ARMv8   {-A, -R, -M}
                 /    \
Mode:     Aarch32     Aarch64
             |         /   \
Encoding:   A64      A32   T32
```

At the top is the overall v8 instruction set archicture. Second are the
two modes, defined by bitwidth but differing in other semantics too, and
buttom are the encodings, (hopefully?) isomorphic if they encode the
same mode.

The 32 bit encodings are mostly backwards compatible with previous
non-Thumb and Thumb encodings, and if so we can pun the mode names to
instead mean "sets of compatable or isomorphic encodings", and then
voilà we have nice names for 32-bit and 64-bit arm instruction sets
which do not use the word ARM so as to not confused either laymen or
experienced ARM packages.

[1]: https://developer.arm.com/products/architecture/a-profile

(cherry picked from commit ba52ae5048)
2018-04-25 15:50:41 -04:00
John Ericson ba52ae5048 treewide: isArm -> isAarch32
Following legacy packing conventions, `isArm` was defined just for
32-bit ARM instruction set. This is confusing to non packagers though,
because Aarch64 is an ARM instruction set.

The official ARM overview for ARMv8[1] is surprisingly not confusing,
given the overall state of affairs for ARM naming conventions, and
offers us a solution. It divides the nomenclature into three levels:

```
ISA:             ARMv8   {-A, -R, -M}
                 /    \
Mode:     Aarch32     Aarch64
             |         /   \
Encoding:   A64      A32   T32
```

At the top is the overall v8 instruction set archicture. Second are the
two modes, defined by bitwidth but differing in other semantics too, and
buttom are the encodings, (hopefully?) isomorphic if they encode the
same mode.

The 32 bit encodings are mostly backwards compatible with previous
non-Thumb and Thumb encodings, and if so we can pun the mode names to
instead mean "sets of compatable or isomorphic encodings", and then
voilà we have nice names for 32-bit and 64-bit arm instruction sets
which do not use the word ARM so as to not confused either laymen or
experienced ARM packages.

[1]: https://developer.arm.com/products/architecture/a-profile
2018-04-25 15:28:55 -04:00
Will Dietz 97698b0d6d musl-stdenv: don't add special allowance for libiconv 2018-04-25 08:08:47 -05:00
Will Dietz 791b5bbbf4 musl bootstrap: remove libiconv 2018-04-24 21:27:28 -05:00
Tuomas Tynkkynen 3c6e077301 Merge remote-tracking branch 'upstream/master' into HEAD
Conflicts:
	pkgs/development/tools/misc/binutils/default.nix
2018-04-22 22:31:30 +03:00
John Ericson cf0dd1dbd2
Merge pull request #39172 from obsidiansystems/ios-cross
ios-sdk-pkgs: Init from iOS SDK from XCode
2018-04-19 17:00:05 -04:00
Ken Micklas ef3db7d14c ios-sdk-pkgs: Init from iOS SDK from XCode 2018-04-19 16:09:30 -04:00
John Ericson 53686e8995
Merge pull request #38485 from obsidiansystems/nixos-nixpkgs-options
nixpkgs module: Clean up platform options
2018-04-19 14:59:58 -04:00
John Ericson 204b73ffba
Merge pull request #38932 from svanderburg/cygwin-fixes
Fix stdenv-native and fix rebasing on cygwin
2018-04-18 11:05:19 -04:00
John Ericson 9edda8aef1
Merge pull request #38639 from shlevy/platforms-not-broken
meta: Don't bypass unsupported platforms with allowBroken.
2018-04-17 17:23:50 -04:00
John Ericson cb212cf549 meta: Add NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM for consistency 2018-04-17 16:02:50 -04:00
Shea Levy 3955b84698 meta: Don't bypass unsupported platforms with allowBroken.
Our platforms are open-world oriented these days, and anyway there's allowUnsupportedSystem.
2018-04-17 16:02:15 -04:00
Nikolay Amiantov 15bfee85f3 impureUseNativeOptimizations: add stdenv adapter
This allows one to force a compiler to use native machine optimizations. This
goes contrary to all the usual guarantees of Nix and so should be used only by
end-user and only in specific cases when they know what are they doing.

In my case this is needed to get a noticeable FPS boost in RPCS3 which is very
CPU-hungry PlayStation 3 emulator.
2018-04-17 13:12:07 +03:00
Sander van der Burg a7adffdd47 Fix invalid Baseaddress must be > 0x200000000 on cygwin 2018-04-14 14:06:09 +02:00
Sander van der Burg 5a545ca9bd Fix stdenv-native by creating a bintools wrapper with similar settings 2018-04-14 14:05:09 +02:00
John Ericson 0884027ef5 Revert "Revert "Merge pull request #28029 from cstrahan/hardening-fix""
This reverts commit 6c064e6b1f, reapplying
PR #28029 which was supposed to have gone to staging all along.
2018-04-11 14:00:13 -04:00
John Ericson c6f7d43678 nixpkgs module: Clean up platform options
- `localSystem` is added, it strictly supercedes system

 - `crossSystem`'s description mentions `localSystem` (and vice versa).

 - No more weird special casing I don't even understand

TEMP
2018-04-06 12:41:44 -04:00
John Ericson adaa110a72 binutils: No more darwin conditionals
Since at least d7bddc27b2, we've had a
situation where one should depend on:

 - `stdenv.cc.bintools`: for executables at build time
 - `libbfd` or `libiberty`: for those libraries
 - `targetPackages.cc.bintools`: for exectuables at *run* time
 - `binutils`: only for specifically GNU Binutils's executables,
   regardless of the host platform, at run time.

and that commit cleaned up this usage to reflect that. This PR flips the
switch so that:

 - `binutils` is indeed unconditionally GNU Binutils
 - `binutils-raw`, which previously served that role, is gone.

so that the correct usage will be enforced going forward and everything
is simple.

N.B. In a few cases `binutils-unwrapped` (which before and now was
unconditionally actual GNU binutils), rather than `binutils` was used to
replace old `binutils-raw` as it is friendly towards some cross
compilation usage by avoiding a reference to the next bootstrapping
change.
2018-04-03 13:34:52 -04:00
John Ericson f8ed783f4f meta: Simplify platform check logic
Code golf or readability, you decide
2018-03-27 11:59:59 -04:00
Shea Levy b0482248fe
meta: Add badPlatforms attribute for platform blacklisting. 2018-03-27 08:12:45 -04:00
John Ericson 45e253cacb cross stdenv: Make depsBuildBuild overrideable by config too. 2018-03-20 15:58:09 -04:00
John Ericson 3c331bff5b
Merge pull request #37395 from obsidiansystems/lib-meta-platform
lib: Factor in tiny bit of `meta.platform` checking
2018-03-19 20:12:50 -04:00
John Ericson e547bd0dc4 lib: Factor in tiny bit of meta.platform checking
I need it in stdenv and release-lib, so that seems motivation enough.
2018-03-19 19:29:16 -04:00
John Ericson 2fa2197a96
Merge pull request #34444 from obsidiansystems/meta-check
lib: Fix #30902
2018-03-18 13:51:03 -04:00
John Ericson c26252af3e lib, stdenv: Check meta.platforms against host platform and be open world
First, we need check against the host platform, not the build platform.
That's simple enough.

Second, we move away from exahustive finite case analysis (i.e.
exhaustively listing all platforms the package builds on). That only
work in a closed-world setting, where we know all platforms we might
build one. But with cross compilation, we may be building for arbitrary
platforms, So we need fancier filters. This is the closed world to open
world change.

The solution is instead of having a list of systems (strings in the form
"foo-bar"), we have a list of of systems or "patterns", i.e. attributes
that partially match the output of the parsers in `lib.systems.parse`.
The "check meta" logic treats the systems strings as an exact whitelist
just as before, but treats the patterns as a fuzzy whitelist,
intersecting the actual `hostPlatform` with the pattern and then
checking for equality. (This is done using `matchAttrs`).

The default convenience lists for `meta.platforms` are now changed to be
lists of patterns (usually a single pattern) in
`lib/systems/for-meta.nix` for maximum flexibility under this new
system.

Fixes #30902
2018-03-15 00:44:34 -04:00
John Ericson 4c52e34ca6 stdenv: Clean up check meta args 2018-03-14 18:58:07 -04:00
Jan Malakhovski 7079e744d4 Merge branch 'master' into staging
Resolved the following conflicts (by carefully applying patches from the both
branches since the fork point):

   pkgs/development/libraries/epoxy/default.nix
   pkgs/development/libraries/gtk+/3.x.nix
   pkgs/development/python-modules/asgiref/default.nix
   pkgs/development/python-modules/daphne/default.nix
   pkgs/os-specific/linux/systemd/default.nix
2018-03-10 20:38:13 +00:00
John Ericson 8f54d931cd
Merge pull request #36401 from obsidiansystems/no-stdenv-cross-adapter
stdenv cross adapter: Get rid if `selfNativeBuildInput`
2018-03-06 20:17:13 -05:00
John Ericson 8d58008039 stdenv cross adapter: Get rid if selfNativeBuildInput
It is finally no longer used anywhere

Fixes #30587
2018-03-06 20:05:13 -05:00
Will Dietz a6e7c85ab8 x86_64-musl bootstrap regen, much smaller
compressed: 28M -> 20M
uncompressed: 201M -> 119M

Built using:
94f3dc4fdf

cc https://github.com/NixOS/nixpkgs/pull/36383
2018-03-06 11:40:23 -06:00
Will Dietz d7c2cffd86 x86_64-musl: regen again w/symlinks preserved 2018-03-06 11:40:08 -06:00
Will Dietz 856335396f x86_64-musl: use native-built bootstrap package on x86_64
Built using:
46f83fa9d1
2018-03-06 11:36:52 -06:00
Jan Malakhovski 1d4f3fa371 stdenv: check-meta: fix fallout from #36119 2018-03-06 14:14:17 +00:00
Will Dietz ece4c62d4b make-bootstrap-tools: preserve coreutils symlinks
We go out of our way (see top of file) to build a single binary
with symlinks for all of the tools, but were losing them
when preparing the bootstrap tools.
2018-03-06 15:13:56 +02:00
Tuomas Tynkkynen 34f95d92a2 Merge remote-tracking branch 'upstream/master' into staging
Conflicts:
	pkgs/applications/misc/pytrainer/default.nix
	pkgs/development/tools/pew/default.nix
	pkgs/tools/misc/you-get/default.nix
2018-02-28 20:52:49 +02:00
John Ericson 2482e2858e prebuilt android tools: Init using SDK
Expose as an option for the cross stdenv.
2018-02-27 14:15:39 -05:00
John Ericson dfc5d7835d
Merge pull request #35247 from telent/mips32
lib, treewide: Add missing MIPS arches, and fix existing usage
2018-02-27 14:01:15 -05:00
Will Dietz 3a21d5bce2 default to including "man" in outputsToInstall 2018-02-27 10:31:07 -06:00
John Ericson 4a29081a94
Merge pull request #35071 from oxij/stdenv/infopages
stdenv, bash: fixing info pages and stuff
2018-02-26 18:06:11 -05:00