Commit graph

1991 commits

Author SHA1 Message Date
Matthew Bauer d38bad590e
Merge pull request #83180 from matthewbauer/only-add-ldflags-for-macos
bintools: only add macos flags when targeting macOS
2020-05-13 10:34:40 -05:00
John Ericson 1ac5398589 *-wrapper; Switch from infixSalt to suffixSalt
I hate the thing too even though I made it, and rather just get rid of
it. But we can't do that yet. In the meantime, this brings us more
inline with autoconf and will make it slightly easier for me to write a
pkg-config wrapper, which we need.
2020-05-12 00:44:44 -04:00
Gaelan Steele 23ce1eed34 stdenv: only build LLVM once on darwin 2020-05-11 10:28:31 -07:00
John Ericson 05d26adb0a mkDerivation mesonFlags: Fix arm cpu families
In my haste to unbreak eval, I screwed up and got the bit-widths,
backwards.
2020-04-28 22:56:20 -04:00
John Ericson cf858e6d57 mkDerivation mesonFlags: Fix cross file logic to handle more cases
Otherwise eval breaks
2020-04-28 16:51:44 -04:00
Jude Taylor 0de6275003 pkgsStatic: fix eval on Darwin 2020-04-28 08:41:48 -07:00
John Ericson 8245230753 meson: Make target-agnostic
The cross file is added in the `mkDerivation`. It isn't nice putting
build tool-specific stuff here, but our current architecture gives us
little alternative.
2020-04-28 10:55:33 -04:00
Daiderd Jordan 950ac2bc8f
meta: expose availability flags in derivation metadata
Currently it's not possible to determine the reason why a package is
unavailable without evaluating nixpkgs multiple times with different
settings. eg.

    nix-repl> :p android-studio.meta
    { available = false; broken = false; unfree = true; unsupported = true; ... }

The following snippet is an example that uses this information to query
the availability information of all packages in nixpkgs, giving an
overview of all the packages currently marked as broken, etc.

    { pkgs }:

    with import <nixpkgs/lib>;

    let

      mapPkgs =
        let mapPkgs' = path: f: mapAttrs (n: v:
          let result = builtins.tryEval (v ? meta); in
          if !result.success then {} else
          if isDerivation v then f (path ++ [n]) v else
          if isAttrs v && v.recurseForDerivations or false then mapPkgs' (path ++ [n]) f v else
          {}
        );
        in mapPkgs' [];

      getMeta = path: drv:
        if drv.meta ? available then
          let meta = {
            pkg = concatStringsSep "." path;
            inherit (drv.meta) broken unfree unsupported insecure;
          };
          in builtins.trace meta.pkg meta
          else {};

      metaToList = attrs: flatten (map (v: if v ? pkg then v else metaToList v) (attrValues attrs));

    in metaToList (mapPkgs getMeta pkgs)
2020-04-19 12:53:04 +02:00
Jan Tojnar b4a6714571
Merge branch 'staging-next' into staging 2020-04-13 18:54:59 +02:00
worldofpeace 5384d72885 setup.sh: add dontPatch
Fixes #85038
2020-04-12 07:04:35 -04:00
Michael Reilly 84cf00f980
treewide: Per RFC45, remove all unquoted URLs 2020-04-10 17:54:53 +01:00
Matthew Bauer e0fb0df64f generic/setup.sh: allow clobbering env-vars file
If the option ‘noclobber’ is set in Bash, we get an error when we
clobber an already existing env-vars. This is an okay error to ignore,
so just >| instead. Note that >| is NOT a Bashism[[1]].

Fixes #79651

[1]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_02
2020-04-09 22:39:16 -04:00
Matthew Bauer e45f16b9f5 stdenv/linux: remove powerpc64le bootstrap tools
These files never existed, so best to not leave the reference. If
someone want to step up to maintain this, that would be fine. I don’t
have the hardware to test these out. In addition, someone tried to use
the bootstrap-tools currently built by Hydra and found that they were
broken in some unclear way.
2020-03-31 15:03:26 -04:00
Matthew Bauer b0b0a06788 bintools: only add macos flags when targeting macOS
We can’t set this for cross-compiling since we use the GNU linker.
Instead, set these flags only when targetPlatform is macOS.

Fixes #80754

Fixes #83141
2020-03-23 00:32:45 -04:00
Graham Christensen 6b361c3a31
armv{5tel,6l,7l}: bootstrap-files: quote URLs for rfc-45
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2020-03-16 19:04:39 -04:00
Graham Christensen 11b54adeac
armv{5tel,6l,7l}: update bootstrap tools after #80497 2020-03-16 18:11:59 -04:00
Frederik Rietdijk 0eb0ddc4db Merge staging-next into master 2020-03-08 08:11:01 +01:00
Ben Wolsieffer 05fc2a3524 unpack-bootstrap-tools: remove obsolete linker script patching
The linker scripts no longer contain store paths, so this does nothing. More
importantly, libpthread.so is not longer a linker script on ARM, so the patching
would corrupt it.
2020-03-06 06:58:20 +01:00
Ben Wolsieffer ee39e49b68 stdenv: update ARM bootstrap tarballs
Fixes building glibc 2.30, which requires at least GCC 6.

Picked from the following cross-trunk evaluation:
https://hydra.nixos.org/eval/1568864 based on nixpkgs
commit ac241fb7a5.

armv5tel job: https://hydra.nixos.org/build/111556418
armv6l job: https://hydra.nixos.org/build/111556521
armv7l job: https://hydra.nixos.org/build/111556551
2020-03-06 06:58:20 +01:00
Ben Wolsieffer 4478766b64 make-bootstrap-tools: add libssp to allow stack protector to work 2020-02-18 22:40:30 -05:00
Vladimír Čunát cc1ae9f21b
make-bootstrap-tools: fixup after glibc update
There's a generated header that got comment about the source header
from glibc.dev, which added unwanted runtime dependency.  Tested:
nix build -f pkgs/top-level/release.nix stdenvBootstrapTools.{aarch64,i686,x86_64}-linux.test
2020-02-10 17:31:35 +01:00
Frederik Rietdijk 03755ed59a Merge master into staging-next 2020-02-09 09:17:51 +01:00
Chuck 5d4821141b stdenv: Remove isArm (use isAarch32 instead)
isArm has been deprecated for three releases.  All references have been
removed.  Tree-wide substitution was performed in #37401 21 months ago.
2020-02-05 10:56:14 -08:00
Vladimír Čunát cf242659fe
stdenv bootstrap tools: switch one file to official URL 2020-02-05 13:59:18 +01:00
Vladimír Čunát 48a997cd76
Merge #66528: glibc: 2.27 -> 2.30 (into staging)
Includes update of stdenv bootstap tools (for three main platforms)
and many package fixes with new glibc.
2020-02-05 13:41:09 +01:00
Vladimír Čunát 53349b9c27
stdenv bootstrap tools: regenerate "officialy"
From Hydra's binaries for c5aabb0 (19.09).
This time I re-tested full stdenv bootstrap on these three platforms.
https://github.com/NixOS/nixpkgs/pull/66528#issuecomment-582351637
2020-02-05 13:04:13 +01:00
Frederik Rietdijk e4c28f78e7 Merge staging-next into staging 2020-02-02 15:33:13 +01:00
Maximilian Bosch ea8ae88f04
Merge branch 'staging' into glibc230 2020-02-01 17:42:03 +01:00
Daiderd Jordan 68513e4071
cmake: move CMAKE_OSX_ARCHITECTURES out of the stdenv
This was initially introduced in 92188d9d17,
not clear how relevant this still is but i686 isn't supported anymore so
disable it explicitly.
2020-01-31 21:52:13 +01:00
Daiderd Jordan 6567823996
stdenv: introduce appleSdkVersion and macosVersionMin 2020-01-31 21:52:11 +01:00
Daiderd Jordan a826b49c97
stdenv: make darwin builds reproducable
Fixes #21629

Passing these extra linker flags removes both the semi-random uuid
included in most binaries as well as making the sdk version consistent
instead of based on the current os version.

    Load command 8
         cmd LC_UUID
     cmdsize 24
        uuid 70FAF921-5DC8-371C-B814-4F121FADFDF4

    Load command 9
          cmd LC_VERSION_MIN_MACOSX
      cmdsize 16
      version 10.12
          sdk 10.13

The -macosx_version_min flag isn't strictly necessary since that's
already handled by MACOSX_DEPLOYMENT_TARGET.
2020-01-31 21:52:07 +01:00
Luka Blaskovic c206bf12ab stdenv, darwin: fix setupHook location 2020-01-30 18:35:30 -05:00
Maximilian Bosch eddfcc32b4
Merge branch 'staging' into glibc230 2020-01-23 11:31:13 +01:00
Andreas Rammhold cb007e69a1 stdenv: make symlinks that refer to the same output relative
While looking at the graph of all the outputs in my personal binary
cache it became obvious that we have a lot of self references within the
package set. That isn't an isuse by itself. However it increases the
size of the binary cache for every (reproducible) build of a package
that carries references to itself. You can no longer deduplicate the
outputs since they are all unique. One of the ways to get rid of (a few)
references is to rewrite all the symlinks that are currently used to be
relative symlinks. Two build of something that didn't really change but
carries a self-reference can the be store as the same NAR file again.

I quickly hacked together this change to see if that would yield and
success. My bash scripting skills are probably not great but so far it
seem to somewhat work.
2020-01-15 09:26:40 +01:00
Frederik Rietdijk bc18cc72dd
Merge pull request #77610 from LnL7/darwin-stdenv-python3
darwin-stdenv: bootstrap with python3
2020-01-15 09:24:57 +01:00
Maximilian Bosch 2d36eac780 stdenv: rebuild aarch64 bootstrap files with gcc8
The old bootstrapping files would break with glibc 2.30 as at least gcc6
is needed.
2020-01-14 08:52:25 +00:00
Vladimír Čunát 2aea16c4d6 glibc: depend on libidn2 (and libunistring, transitively)
It's a bit hacky, but ATM I can't see any better way for glibc >= 2.28.

Signed-off-by: Luka Blaskovic <lblasc@tvbeat.com>
2020-01-14 08:27:00 +00:00
Vladimír Čunát 0c75f51304 stdenv bootstrap: disable dejagnu tests
There's a failure - I don't know why.  The checks from "normal" dejagnu
still succeed, so I assume this commit is quite a safe work-around.
2020-01-14 08:27:00 +00:00
Luka Blaskovic e80a85a057 bootstrap-files: bump WIP 2020-01-14 08:26:58 +00:00
Daiderd Jordan 6328518e98
stdenv: bootstrap darwin with python3
- Replaced python override from the final stdenv, instead we
  propagate our bootstrap python to stage4 and override both
  CF and xnu to use it.

- Removed CF argument from python interpreters, this is redundant
  since it's not overidden anymore.

- Inherit CF from stage4, making it the same as the stdenv.
2020-01-13 11:34:36 +01:00
Dmitry Kalinkin d05ee9c8ff
pkgsStatic: set BUILD_SHARED_LIBS=OFF for cmake 2020-01-03 11:31:17 -05:00
John Ericson cfd013813e
Merge pull request #74090 from obsidiansystems/ghcjs-cross-without-cc
stdenv, haskell: bonafied GHCJS cross compilation without stdenv.cc
2019-12-30 16:40:43 -08:00
John Ericson c1fb358a78 Merge remote-tracking branch 'obsidian/ghcjs-cross-without-cc-common' into ghcjs-cross-without-cc 2019-12-30 18:16:22 -05:00
John Ericson 6078f094c6 pkgs/stdenv/booter.nix: Add comment explaining hasCC trickery 2019-12-30 18:09:45 -05:00
Robert Scott e1e3df423a allowInsecureDefaultPredicate: fix to use getName
this allows correct operation with packages only having pname and version
specified, resolving issue #73737
2019-12-15 18:01:19 +00:00
John Ericson c4508df25d Merge remote-tracking branch 'obsidian/ghcjs-cross-without-cc-19.09' into ghcjs-cross-without-cc 2019-11-26 12:58:13 -05:00
John Ericson 8737963735 Fix lib tests
js-ghcjs didn't fit in an existing categor.
2019-11-26 12:57:16 -05:00
John Ericson 6bc456c91c Merge remote-tracking branch 'upstream/master' into ghcjs-cross-without-cc 2019-11-25 00:23:07 +00:00
John Ericson c739c420db Add support for cross compiling to js-ghcjs
This platform doesn't have a C compiler, and so relies and the changes
in the previous commit to work.
2019-11-25 00:12:38 +00:00
John Ericson 63bd851e95 stdenv: Introduce hasCC attribute
Before, we'd always use `cc = null`, and check for that. The problem is
this breaks for cross compilation to platforms that don't support a C
compiler.

It's a very subtle issue. One might think there is no problem because we
have `stdenvNoCC`, and presumably one would only build derivations that
use that. The problem is that one still wants to use tools at build-time
that are themselves built with a C compiler, and those are gotten via
"splicing". The runtime version of those deps will explode, but the
build time / `buildPackages` versions of those deps will be fine, and
splicing attempts to work this by using `builtins.tryEval` to filter out
any broken "higher priority" packages (runtime is the default and
highest priority) so that both `foo` and `foo.nativeDrv` works.

However, `tryEval` only catches certain evaluation failures (e.g.
exceptions), and not arbitrary failures (such as `cc.attr` when `cc` is
null). This means `tryEval` fails to let us use our build time deps, and
everything comes apart.

The right solution is, as usually, to get rid of splicing. Or, baring
that, to make it so `foo` never works and one has to explicitly do
`foo.*`. But that is a much larger change, and certaily one unsuitable
to be backported to stable.

Given that, we instead make an exception-throwing `cc` attribute, and
create a `hasCC` attribute for those derivations which wish to
condtionally use a C compiler: instead of doing `stdenv.cc or null ==
null` or something similar, one does `stdenv.hasCC`. This allows quering
without "tripping" the exception, while also allowing `tryEval` to work.

No platform without a C compiler is yet wired up by default. That will
be done in a following commit.
2019-11-25 00:12:38 +00:00
Lily Ballard d45d6205de setup.sh: rewrite stripHash
Rewrite the `stripHash` helper function with 2 differences:

* Paths starting with `--` will no longer produce an error.
* Use Bash string manipulation instead of shelling out to `grep` and
  `cut`. This should be faster.
2019-11-12 14:38:41 +01:00
John Ericson 5b6da99ea0 stdenv: Don't unset propagated*DepFiles before main phases
A bunch of stdenv-internal variables were deleted in
1601a7fcce, but these are needed in the
fixup phase, whereas the rest are just needed for the initial work
(findInputs, etc) before the user phases.

CC @matthewbauer
2019-11-05 16:32:22 -05:00
Matthew Bauer 9ffedfef81
Merge pull request #69603 from matthewbauer/unset-unused-setup-var
Unset unused variables in setup.sh
2019-11-05 14:43:11 -05:00
Frederik Rietdijk c4e30cf98c Merge staging-next into staging 2019-11-05 14:18:08 +01:00
Dmitry Kalinkin cf8a2d0225
Revert "stdenv/check-meta: getEnv if the attribute is unset (#72376)" (#72752)
This reverts commit 71184f8e15.
2019-11-03 20:38:35 -05:00
zimbatm 71184f8e15
stdenv/check-meta: getEnv if the attribute is unset (#72376)
There were two issues:

* builtins.getEnv was called deep into the nixpkgs tree making it hard
  to discover. This is solved by moving the call into
  pkgs/top-level/impure.nix
* when the config was explicitly set by the user to false, it would
  still try and load the environment variable. This meant that it was
  not possible to guarantee the same outcome on two different systems.
2019-11-03 17:40:43 +00:00
Matthew Bauer a6a3958c26
Merge pull request #72070 from graham33/fix/license_list
Fix handling of lists in whitelistedLicenses and blacklistedLicenses
2019-11-03 12:08:41 -05:00
Will Dietz 43c721df4a make-bootstrap-tools: use 'extreme' option to reduce size but not cost
Apparently this option trades compression time for size,
and explicitly does so without increasing resources needed in decomp.

Doesn't make tarball creation unbearable, so add it to options!
2019-11-03 14:52:44 +01:00
Will Dietz 78d3cb1d16 binutils: offer "for bootstrap" variant that's less.. feature-complete 2019-11-03 14:52:44 +01:00
Will Dietz 5b46a7c240 gcc7: make building w/LTO support optional, don't use in bootstrap tools 2019-11-03 14:52:44 +01:00
John Ericson 9df7efe0c6 stdenv: Don't stop set -u-ing
Before, we very carefully unapplied and reapplied `set -u` so the rest
of Nixpkgs could continue to not fail on undefined variables. Let's rip
off the band-aid.
2019-11-01 22:03:47 +00:00
John Ericson 2811b032d6 treewide: Make still dont* Variables are optional in most cases
Go beyond the obvious setup hooks now, with a bit of sed, with a skipped case:

 - cc-wrapper's `dontlink`, because it already is handled.

Also, in nix files escaping was manually added.

EMP
2019-11-01 14:44:44 -04:00
Graham Bennett b12605563f Fix handling of lists in whitelistedLicenses and blacklistedLicenses
A package's meta.license can either be a single license or a list.  The
code to check config.whitelistedLicenses and config.blackListedLicenses
wasn't handling this, nor was the showLicense function.
2019-10-27 10:23:53 +00:00
Matthew Bauer 1601a7fcce generic/setup.sh: Unset locally defined variables
setup.sh adds a bunch of variables that only it needs. To avoid
polluting environments, we should unset these as soon as we are done
with them.
2019-09-26 18:47:38 -04:00
Matthew Bauer 8e9b98a183
Merge pull request #69028 from matthewbauer/remove-iselfexec-iselfdyn
Revert "setup.sh introduce isELFExec, isELFDyn"
2019-09-20 23:26:48 -04:00
Matthew Bauer d8b7b95ac6 Merge remote-tracking branch 'origin/master' into staging 2019-09-20 23:25:24 -04:00
Albert Safin 42482a1d60 setup.sh: avoid subshells: iterating a file 2019-09-20 02:45:53 +00:00
Albert Safin cf4e4820f6 setup.sh: avoid subshells: type -t in _callImplicitHook 2019-09-20 02:45:52 +00:00
Albert Safin d53920a5be setup.sh: avoid subshells: mapOffset 2019-09-20 02:45:52 +00:00
Albert Safin 6f024f6e65 setup.sh: avoid subshells: type -t 2019-09-20 02:45:52 +00:00
Matthew Bauer 07d78691da stdenv/adapters.nix: remove static from makeStaticLibraries
This logic should be in the pkgs/top-level/static.nix. We don’t want
to pollute Nixpkgs with =if stdenv.static=. Also, "static" is not
descriptive. We have two types of static stdenvs, ‘makeStaticLibaries’
and ‘makeStaticBinaries’. We shouldn’t rely on a static boolean like
this.
2019-09-19 12:52:28 -04:00
Albert Safin 463463b395 setup.sh: avoid subshells: shopt -po nounset 2019-09-19 09:54:29 +00:00
Matthew Bauer 24c6aef75a Revert "setup.sh introduce isELFExec, isELFDyn"
This reverts commit e1b80a5a99.

This is broken in PIE (#68513). Best to not keep it in until something
else starts using it.
2019-09-18 11:28:27 -04:00
Vladimír Čunát f21211ebfe
Merge branch 'master' into staging 2019-09-02 23:25:24 +02:00
John Ericson c33d80c071 Merge remote-tracking branch 'upstream/master' into feature/js-unknown-ghcjs 2019-09-02 01:31:31 -04:00
Moritz Angermann 446f8c851d Add support for js-unknown-ghcjs
This adds enough logic to nixpkgs to support the `js-unknown-ghcjs` triple.
2019-09-02 01:27:05 -04:00
volth 08f68313a4 treewide: remove redundant rec 2019-08-28 11:07:32 +00:00
Frederik Rietdijk 5061fe0c2c Merge staging-next into staging 2019-08-28 08:26:42 +02:00
volth 35d68ef143 treewide: remove redundant quotes 2019-08-26 21:40:19 +00:00
Danylo Hlynskyi 2ca09a94be
Merge pull request #66657 from danbst/pgpackages-fixes
Postgresql plugins fixes
2019-08-18 17:30:56 +03:00
Florian Klink dbd7ea5f29
Merge pull request #66725 from flokli/wrapqtappshook-exec
stdenv: add isELFExec, isELFDyn, fix wrappers
2019-08-18 13:58:10 +02:00
Florian Klink e1b80a5a99 setup.sh introduce isELFExec, isELFDyn
These can be used to determine whether a ELF file with ELF header is an
executable or shared library.

We can't implement it in pure bash, as bash has problems with null
bytes.
2019-08-17 16:45:52 +02:00
danbst cd5b8620bb stdenv/check-meta: construct name from pname and version if name unavailable 2019-08-15 02:25:56 +03:00
Matthew Bauer 166905cb31
Merge pull request #65484 from arcnmx/pr-stdenv-cmake-cross
stdenv: correct cross CMAKE_SYSTEM_NAME
2019-08-14 10:17:12 -04:00
Frederik Rietdijk 55e4555b77 Merge master into staging-next 2019-08-01 09:42:54 +02:00
arcnmx 8cedc7fc6c stdenv: correct cross CMAKE_SYSTEM_NAME
As described in cmake cross instructions, CMAKE_SYSTEM_NAME should be
set to "Generic" if there is no applicable OS:

https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/CrossCompiling#setting-up-the-system-and-toolchain
2019-07-27 15:23:31 -07:00
Tobias Mayer bb71bb15ce pkgsStatic: propagate all buildInputs 2019-07-24 16:04:51 +02:00
Tobias Mayer 3c12ae1d35 Signal static build option in makeStaticLibraries
This flag can replace the extra static parameter to packages that
don't work automatically with this adapter.
2019-07-21 06:44:50 +02:00
Daiderd Jordan f1070eec27
stdenv: MACOSX_DEPLOYMENT_TARGET 10.10 -> 10.12 2019-07-08 21:56:40 +02:00
worldofpeace 4a88f4ebfc setup.sh: add dontUnpack 2019-07-01 04:23:51 -04:00
worldofpeace 0197c05786 setup.sh: add dontConfigure
There's already 21 occurences of this and I've
expected this to exist without knowing it had no affect for a while.
2019-07-01 01:52:54 -04:00
volth f3282c8d1e treewide: remove unused variables (#63177)
* treewide: remove unused variables

* making ofborg happy
2019-06-16 19:59:05 +00:00
Vladimír Čunát 576af17187
Merge branch 'master' into staging-next
Hydra nixpkgs: ?compare=1523575
2019-06-05 11:06:44 +02:00
Matthew Bauer 760c9995b0
Merge pull request #60349 from matthewbauer/fix-60345
check-meta: use system tuple in platforms
2019-06-04 11:29:48 -04:00
Vladimír Čunát b4ae841b23
Merge branch 'staging-next' into staging 2019-05-26 09:48:55 +02:00
Vladimír Čunát 96a604320a
Merge #61179: stdenv, cacert: $NIX_SSL_CERT_FILE changes
... into staging
2019-05-19 10:56:11 +02:00
Matthew Bauer 5ba633970b
Merge pull request #61169 from matthewbauer/meson-static
static: set default_library flag for meson in makeStaticLibrary
2019-05-18 15:02:33 -05:00
Robin Gloster 6cf583cf2f
Merge pull request #60406 from JohnAZoidberg/remove-isnull
treewide: Remove usage of isNull
2019-05-18 09:36:24 +00:00
Michael Raskin 76e2a96475
Changelog meta entry (#60371)
meta.changelog: enable, document, add for GNU Hello
2019-05-10 16:55:29 +00:00
Vladimír Čunát b27cc37671
stdenv: also override cert files in pure nix-shell
That's very much consistent with the spirit of nix-shell --pure

BTW, nix 1.x shells will be always treated as pure;
in that version detection isn't possible.
https://github.com/NixOS/nix/commit/1bffd83e1a9c
2019-05-09 09:49:42 +02:00
Vladimír Čunát 79bd4ad579
stdenv, cacert: consider $NIX_SSL_CERT_FILE in hooks
Some SSL libs don't react to $SSL_CERT_FILE.
That actually makes sense to me, as we add this behavior
as nixpkgs-specific, so it seems "safer" to use $NIX_*.
2019-05-09 08:46:22 +02:00
Matthew Bauer fea2422398 static: set default_library flag for meson in makeStaticLibrary
Adds support for building static libraries with meson.
2019-05-08 21:52:59 -04:00
Matthew Bauer 424f467e06
Merge branch 'master' into update-ios-gnu-config 2019-05-07 14:25:33 -04:00
Matthew Bauer 5a69a4ecc1 cross: use newer gnu-config on iOS 2019-05-07 13:36:25 -04:00
Matthew Bauer a52e317200 check-meta: use system tuple in platforms
Fixes #60345
2019-04-30 12:59:03 -04:00
Daniel Schaefer 786f02f7a4 treewide: Remove usage of isNull
isNull "is deprecated; just write e == null instead" says the Nix manual
2019-04-29 14:05:50 +02:00
Matthew Bauer 87944c3125
Merge pull request #56744 from matthewbauer/macos-10-12
Update macOS to 10.12
2019-04-26 22:20:03 -04:00
Matthew Bauer f1461d8c3d darwin-bootstrap: copy libLLVM dylib
This is needed for the bootstrap tools to work:

https://hydra.nixos.org/build/92534667
2019-04-26 21:55:09 -04:00
Matthew Bauer c1cb58e81e darwin/make-bootstrap-tools: fix build 2019-04-26 21:55:02 -04:00
Matthew Bauer 2b0b63c229 stdenv/darwin: use libxml2-nopython for building llvm 2019-04-26 21:54:50 -04:00
Matthew Bauer 037f62bbad stdenv/darwin: put bash in extraBuildInputs
patch-shebangs should pick these up as they are runtime dependencies.
This avoids bootstrap-tools leaking into the outputs.
2019-04-26 21:54:49 -04:00
Matthew Bauer ec7d72a57d setup.sh: make sure initialPath goes at end of HOST_PATH
We want initialPath to have lowest precedence.

In addition, unset _PATH and _HOST_PATH as they shouldn’t be needed
after final PATH and HOST_PATH are set.
2019-04-26 21:54:49 -04:00
Matthew Bauer 2ecc1d1599 stdenv/darwin: move secure-format patch to gnum4
This is the only derivation that needs it anymore.
2019-04-26 21:54:47 -04:00
Matthew Bauer 36450d8d0b cctools: enable tapi everywhere
Gets us closer to #19906.
2019-04-26 21:54:46 -04:00
Matthew Bauer 7488a367af
Merge pull request #56555 from matthewbauer/wasm
Initial WebAssembly/WASI cross-compilation support
2019-04-23 22:44:33 -04:00
John Ericson 1a7a96a093 stdenv, compiler-rt: Compress WASI conditionals 2019-04-23 21:48:58 -04:00
Matthew Bauer 4048acb5cf gnu-config: bump to latest version for wasi 2019-04-23 21:48:58 -04:00
Matthew Bauer 9abff4af4f wasm: init cross target
Adds pkgsCross.wasm32 and pkgsCross.wasm64. Use it to build Nixpkgs
with a WebAssembly toolchain.

stdenv/cross: use static overlay on isWasm

isWasm doesn’t make sense dynamically linked.
2019-04-23 21:48:57 -04:00
Matthew Bauer 7f23d9fc5f
Merge pull request #59755 from matthewbauer/patches-in-make-derivation
make-derivation: put patches in all derivations
2019-04-23 18:33:04 -04:00
volth b0193379e5
make-derivation.nix: minor
it seems to change nothing (.platform is always there), just to be consisted with the style of other checks
2019-04-21 20:05:13 +00:00
volth b1c3581b46
make-derivation.nix: @matthewbauer's review 2019-04-21 19:16:28 +00:00
volth 5d87bc2650 fix bootstrap when platform.gcc.arch=="skylake" 2019-04-21 16:37:49 +00:00
Matthew Bauer ae1e940cc0 darwin: move to llvm7 2019-04-20 23:17:04 -04:00
Matthew Bauer dd584d8eeb stdenv/linux: use isCompatible to find bootstrap tools
This avoids part of the issue where things like armv7a don’t work
because the system doesn’t realize it can use the armv7l bootstrap
tools.
2019-04-19 12:00:44 -04:00
Matthew Bauer c7ccb9f197 make-derivation: put patches in all derivations
This puts patches in all derivations even if it unspecified by the
derivation. By default it will be an empty list. This simplifies
overrides, as we can now assume that patches is a valid name so that
this works:

self: super: {
  mypkg = super.pkg.overrideAttrs (o: {
    patches = o.patches ++ [ ./my-very-own.patch ];
  });
}

That is, you don’t need to provide a default "or []", make-derivation
provides one for you.

Unfortunately, this is a mass rebuild.
2019-04-16 22:51:26 -04:00
Matthew Bauer 7edf2db0fd
Merge pull request #59338 from matthewbauer/llvm-libc++
llvm8: support c++ in cross case
2019-04-12 14:09:00 -04:00
Matthew Bauer d453273fbf llvm8: support c++ in cross case
this adds libc++ to the LLVM cross, giving us access to the full
Nixpkgs set. This requires 4 stages of wrapped compilers:

- Clang with no libraries
- Clang with just compiler-rt
- Clang with Libc, and compiler-rt
- Clang with Libc++, Libc, and compiler-rt
2019-04-11 21:28:51 -04:00
Graham Christensen 817c933878
check-env: don't execute check-meta.nix 15,000 times
Generated from https://github.com/NixOS/nix/pull/2761:

```
                                                                                          ns     calls ns/call
- /home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:22:5 591200 15026 39.3451
+ /home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:22:5 8744   308   28.3896
```

more, generated by:

```
$ NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1 nix-instantiate ./pkgs/top-level/release.nix -A unstable > before 2>&1
$ jq -r '.functions | map((.name + ":" + .file + ":" + (.line|tostring) + ":" + (.column|tostring) + " " + (.count|tostring))) | .[]' before | sort  > before.list
```

applying this patch, then:

```
$ NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1 nix-instantiate ./pkgs/top-level/release.nix -A unstable > after 2>&1
$ jq -r '.functions | map((.name + ":" + .file + ":" + (.line|tostring) + ":" + (.column|tostring) + " " + (.count|tostring))) | .[]' after | sort  > after.list
```

and then diffing before.list and after.list to get:

```
                                                                                                        calls
- :/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:4:1               7513
+ :/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:4:1               154

- mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:23                7513
+ mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:23                154

- mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:26                7513
+ mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:26                154

- onlyLicenses:/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:21:18 15026
+ onlyLicenses:/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:21:18 308
```

The following information is from `NIX_SHOW_STATS=1 GC_INITIAL_HEAP_SIZE=4g nix-env -f ./outpaths.nix -qaP --no-name --out-path --arg checkMeta true`:

| stat                       | before         | after          | Δ               | Δ%      |
|:---------------------------|---------------:|---------------:|:----------------|--------:|
| **cpuTime**                |        179.915 |        145.543 | 🡖 34.372        | -19.10% |
| **envs-bytes**             |  3,900,878,824 |  3,599,483,208 | 🡖 301,395,616   |  -7.73% |
| **envs-elements**          |    214,426,071 |    185,881,709 | 🡖 28,544,362    | -13.31% |
| **envs-number**            |    136,591,891 |    132,026,846 | 🡖 4,565,045     |  -3.34% |
| **gc-heapSize**            | 11,400,048,640 | 12,314,890,240 | 🡕 914,841,600   |   8.02% |
| **gc-totalBytes**          | 25,976,902,560 | 24,510,740,176 | 🡖 1,466,162,384 |  -5.64% |
| **list-bytes**             |  1,665,290,080 |  1,665,290,080 | 0               |         |
| **list-concats**           |      7,264,417 |      7,264,417 | 0               |         |
| **list-elements**          |    208,161,260 |    208,161,260 | 0               |         |
| **nrAvoided**              |    191,359,386 |    179,693,661 | 🡖 11,665,725    |  -6.10% |
| **nrFunctionCalls**        |    119,665,062 |    116,348,547 | 🡖 3,316,515     |  -2.77% |
| **nrLookups**              |     80,996,257 |     76,069,825 | 🡖 4,926,432     |  -6.08% |
| **nrOpUpdateValuesCopied** |    213,930,649 |    213,930,649 | 0               |         |
| **nrOpUpdates**            |     12,025,937 |     12,025,937 | 0               |         |
| **nrPrimOpCalls**          |     88,105,604 |     86,451,598 | 🡖 1,654,006     |  -1.88% |
| **nrThunks**               |    196,842,044 |    175,126,701 | 🡖 21,715,343    | -11.03% |
| **sets-bytes**             |  7,678,425,776 |  7,285,767,928 | 🡖 392,657,848   |  -5.11% |
| **sets-elements**          |    310,241,340 |    294,373,227 | 🡖 15,868,113    |  -5.11% |
| **sets-number**            |     29,079,202 |     27,601,310 | 🡖 1,477,892     |  -5.08% |
| **sizes-Attr**             |             24 |             24 | 0               |         |
| **sizes-Bindings**         |              8 |              8 | 0               |         |
| **sizes-Env**              |             16 |             16 | 0               |         |
| **sizes-Value**            |             24 |             24 | 0               |         |
| **symbols-bytes**          |     16,474,666 |     16,474,676 | 🡕 10            |   0.00% |
| **symbols-number**         |        376,426 |        376,427 | 🡕 1             |   0.00% |
| **values-bytes**           |  6,856,506,288 |  6,316,585,560 | 🡖 539,920,728   |  -7.87% |
| **values-number**          |    285,687,762 |    263,191,065 | 🡖 22,496,697    |  -7.87% |

The following information is from `NIX_SHOW_STATS=1 GC_INITIAL_HEAP_SIZE=4g nix-instantiate ./nixos/release-combined.nix -A tested`:

| stat                       | before         | after          | Δ               | Δ%     |
|:---------------------------|---------------:|---------------:|:----------------|-------:|
| **cpuTime**                |        256.071 |        237.531 | 🡖 18.54         | -7.24% |
| **envs-bytes**             |  7,111,004,192 |  7,041,478,520 | 🡖 69,525,672    | -0.98% |
| **envs-elements**          |    346,236,940 |    339,588,487 | 🡖 6,648,453     | -1.92% |
| **envs-number**            |    271,319,292 |    270,298,164 | 🡖 1,021,128     | -0.38% |
| **gc-heapSize**            |  8,995,291,136 | 10,110,009,344 | 🡕 1,114,718,208 | 12.39% |
| **gc-totalBytes**          | 37,172,737,408 | 36,878,391,888 | 🡖 294,345,520   | -0.79% |
| **list-bytes**             |  1,886,162,656 |  1,886,163,472 | 🡕 816           |  0.00% |
| **list-concats**           |      6,898,114 |      6,898,114 | 0               |        |
| **list-elements**          |    235,770,332 |    235,770,434 | 🡕 102           |  0.00% |
| **nrAvoided**              |    328,829,821 |    326,618,157 | 🡖 2,211,664     | -0.67% |
| **nrFunctionCalls**        |    240,850,845 |    239,998,495 | 🡖 852,350       | -0.35% |
| **nrLookups**              |    144,849,632 |    142,126,339 | 🡖 2,723,293     | -1.88% |
| **nrOpUpdateValuesCopied** |    251,032,504 |    251,032,504 | 0               |        |
| **nrOpUpdates**            |     17,903,110 |     17,903,110 | 0               |        |
| **nrPrimOpCalls**          |    140,674,913 |    139,485,975 | 🡖 1,188,938     | -0.85% |
| **nrThunks**               |    294,643,131 |    288,678,022 | 🡖 5,965,109     | -2.02% |
| **sets-bytes**             |  9,464,322,192 |  9,456,172,048 | 🡖 8,150,144     | -0.09% |
| **sets-elements**          |    377,474,889 |    377,134,877 | 🡖 340,012       | -0.09% |
| **sets-number**            |     50,615,607 |     50,616,875 | 🡕 1,268         |  0.00% |
| **sizes-Attr**             |             24 |             24 | 0               |        |
| **sizes-Bindings**         |              8 |              8 | 0               |        |
| **sizes-Env**              |             16 |             16 | 0               |        |
| **sizes-Value**            |             24 |             24 | 0               |        |
| **symbols-bytes**          |      3,147,102 |      3,147,064 | 🡖 38            | -0.00% |
| **symbols-number**         |         82,819 |         82,819 | 0               |        |
| **values-bytes**           | 11,147,448,768 | 10,996,111,512 | 🡖 151,337,256   | -1.36% |
| **values-number**          |    464,477,032 |    458,171,313 | 🡖 6,305,719     | -1.36% |
2019-04-11 19:21:38 -04:00
Matthew Bauer 589c2c2870 androidndk: fixup mess
New android ndk (18) now uses clang. We were going through the wrapper
that are provided. This lead to surprising errors when building.
Ideally we could use the llvm linker as well, but this leads to errors
as many packages don’t support the llvm linker.
2019-04-10 01:30:34 -04:00
John Ericson 70d71bbbe4 top-level: Create pkgs{Build,Host,Target}{Build,Host,Target}
This is needed to avoid confusing and repeated boilerplate for
`fooForTarget`.  The vast majority of use-cases can still use
`buildPackages or `targetPackages`, which are now defined in terms of
these.
2019-03-24 22:12:15 -04:00
Jan Malakhovski 67a7b42167 pkgs/stdenv/booter.nix: fix a typo 2019-03-14 14:01:20 +00:00
Matthew Bauer 9edb851308
Merge pull request #51628 from ju1m/arm
stdenv: add armv7a-linux system
2019-03-01 15:36:07 -05:00
Frederik Rietdijk 2fcb11a244 Merge staging-next into master 2019-03-01 09:06:20 +01:00
Matthew Bauer b86e62d30d llvm: support cross compilation with useLLVM flag
You can build (partially) with LLVM toolchain using the useLLVM flag.
This works like so:

  nix-build -A hello --arg crossSystem '{ system =
    "aarch64-unknown-linux-musl"; useLLVM = true }'

also don’t separate debug info in lldClang

It doesn’t work currently with that setup hook. Missing build-id?
2019-02-26 19:45:35 -05:00
Daiderd Jordan 7ec53a932f
stdenv: only set __darwinAllowLocalNetworking on darwin
This is a darwin only nix attribute for sandbox builds, it can be
ignored on other platforms to avoid unnecessary rebuilds.
2019-02-14 21:38:08 +01:00
Matthew Bauer 5c09d977c7 Merge remote-tracking branch 'origin/master' into staging 2019-02-09 12:14:06 -05:00
Matthew Bauer c6f8f8d98d make-derivation: only modify name when name is given
This preserves Nix’s native error handling of missing name:

  error: derivation name missing
2019-02-04 09:15:03 -05:00
Vincent Laporte 0d17ecce2c
mkDerivation: cleaner handling of the name argument 2019-02-01 16:02:42 +00:00
Vladimír Čunát 8ba516664b
Merge branch 'staging-next' into staging 2019-02-01 09:42:53 +01:00
Vladimír Čunát 5effa4e0f9
Merge branch 'master' into staging-next
Comments on conflicts:
- llvm: d6f401e1 vs. 469ecc70 - docs for 6 and 7 say the default is
  to build all targets, so we should be fine
- some pypi hashes: they were equivalent, just base16 vs. base32
2019-02-01 09:22:29 +01:00
Matthew Bauer 8020bd6869
Merge pull request #35884 from dtzWill/fix/man-in-outputsToInstall
default to including "man" in outputsToInstall
2019-01-31 14:55:12 -05:00
Matthew Bauer e2fe4c2d49 make-derivation: fix ordering of conditionals
cross should have higher precedence
2019-01-28 11:38:30 -05:00
Matthew Bauer 9fd1c170cc make-derivation: try to fix stdenv.cc == null conditional
sometimes this gets an infinite recursion error
2019-01-28 10:41:19 -05:00
Matthew Bauer 7e589e5594 make-derivation: fix position in trace
For a long time now, tracing has been broken in Nixpkgs. So when you
have an eval error you would get something like this:

  error: while evaluating the attribute 'buildInputs' of the derivation 'hello-2.10' at /home/mbauer/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:185:11:
  while evaluating 'chooseDevOutputs' at /home/mbauer/nixpkgs/lib/attrsets.nix:474:22, called from undefined position:
  while evaluating 'optionals' at /home/mbauer/nixpkgs/lib/lists.nix:257:5, called from /home/mbauer/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:132:17:

This is coming from how Nix handles string context and how
make-derivation messes with the "name" attribute. This commit should
restore the old behavior so you get a nice line number like:

  error: while evaluating the attribute 'buildInputs' of the derivation 'hello-2.10' at /home/mbauer/nixpkgs/pkgs/applications/misc/hello/default.nix:4:3:
  while evaluating 'chooseDevOutputs' at /home/mbauer/nixpkgs/lib/attrsets.nix:474:22, called from undefined position:
  while evaluating 'optionals' at /home/mbauer/nixpkgs/lib/lists.nix:257:5, called from /home/mbauer/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:132:17:

NOTE: This will still be broken for cross compilation due to the
prefixes we are adding to name.
2019-01-28 09:25:47 -05:00
Matthew Bauer d54fbbebbb
Merge pull request #49552 from matthewbauer/setup-dedupe
setup.sh: avoid running the same hook twice
2019-01-27 18:22:41 -05:00
Matthew Bauer 9172c1eee2 setup.sh: avoid running the same hook twice
In strictDeps=false, we don’t want the same package hook to be run
twice, otherwise we get duplicates in some flags.

Fixes #41340
2019-01-27 17:49:13 -05:00
Matthew Bauer ae16dd1a15
stdenv/make-derivation: don't hide broken packages
This behavior ended up breaking the handleEvalIssue functionality by hiding those packages. So something like this:

$ nix-env -iA nixpkgs.zoom-us

would silently fail, without telling the user how to fix it! Regardless, this "bug" should be handled in Nix - not Nixpkgs.

Fixes #38952.
2019-01-27 17:02:17 -05:00
Matthew Bauer bd62ac6b88
Merge pull request #54652 from matthewbauer/shell-in-flags-array
setup.sh: put SHELL in flagsArray
2019-01-27 13:55:54 -05:00
Matthew Bauer 329913f733 setup.sh: put SHELL in flagsArray
We don’t want to modify makeFlags, that is given to us by our
environment. Adding to it could lead to duplicates after repeated use.

Fixes #27533
2019-01-26 21:14:36 -05:00
Matthew Bauer 8babcc3d44
Merge branch 'master' into fix/man-in-outputsToInstall 2019-01-26 10:51:12 -05:00
Frederik Rietdijk a1a5ea5943 stdenv: make checkInputs native
We can't run the checkPhase when build != host, so we may as well make
the checkInputs native.

This signicantly improves the situation of Python packages when enabling
strictDeps.
2019-01-13 14:43:18 +01:00
Orivej Desh (NixOS) 9a21967f0a
stdenv: prune libtool files by default (#51767)
See the motivation in fd97db43bc (#41819).
2019-01-11 13:20:46 +00:00
Franz Pletz 237deba4e7
Merge remote-tracking branch 'origin/master' into staging 2019-01-10 16:00:34 +01:00
Dylan Simon 04beae0979 stdenv: allow custom stdenv to take crossOverlays 2019-01-07 20:53:10 -06:00
Matthew Bauer d02bb3c197 darwin.make-bootstrap-tools: remove dsymutil
This hopefully is not needed for the bootstrap tools. Needs more testing.
2019-01-06 22:15:43 -06:00
Daniel Goertzen 1c10efc912 add generic x86_32 support (#52634)
* add generic x86_32 support

- Add support for i386-i586.
- Add `isx86_32` predicate that can replace most uses of `isi686`.
- `isi686` is reinterpreted to mean "exactly i686 arch, and not say i585 or i386".
- This branch was used to build working i586 kernel running on i586 hardware.

* revert `isi[345]86`, remove dead code

- Remove changes to dead code in `doubles.nix` and `for-meta.nix`.
- Remove `isi[345]86` predicates since other cpu families don't have specific model predicates.

* remove i386-linux since linux not supported on that cpu
2019-01-06 12:57:36 -06:00
Frederik Rietdijk 092e3b50a8 Merge master into staging-next 2019-01-02 21:08:27 +01:00
Andrew Dunham 8efe3df795 stdenv/native: add missing argument 2019-01-02 12:19:25 -05:00
Will Dietz 1bb4e5c8dc
Merge pull request #36384 from dtzWill/musl-native-bootstrap
x86_64-musl bootstrap: refresh bootstrap tools package
2018-12-31 14:59:56 -06:00
Jan Tojnar c45e9d0fac
Merge branch 'master' into staging 2018-12-25 17:03:57 +01:00
Jörg Thalheim 1b146a8c6f
treewide: remove paxutils from stdenv
More then one year ago we removed grsecurity kernels from nixpkgs:
https://github.com/NixOS/nixpkgs/pull/25277

This removes now also paxutils from stdenv.
2018-12-22 12:55:05 +01:00
Frederik Rietdijk 9ab61ab8e2 Merge staging-next into staging 2018-12-19 09:00:36 +01:00
Sander van der Burg 2757ddb182 Separate androidndkpkgs from androidenv 2018-12-18 21:14:43 +01:00
Graham Christensen a375d4bfc3
stdenv: shorten evaluation errors when in Hydra
Hydra's page showing evaluation errors is about a mile long, showing
buckets of user-friendly errors, like this:

    in job ‘seyren.aarch64-linux’:
    Package ‘oraclejre-8u191’ in /nix/store/fa9zzkbljkvdavwzirkrr5irg25ymbjl-source/pkgs/development/compilers/oraclejdk/jdk-linux-base.nix:71 has an unfree license (‘unfree’), refusing to evaluate.

    a) For `nixos-rebuild` you can set
      { nixpkgs.config.allowUnfree = true; }
    in configuration.nix to override this.

    b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
      { allowUnfree = true; }
    to ~/.config/nixpkgs/config.nix.

    in job ‘jetbrains.webstorm.x86_64-linux’:
    Package ‘webstorm-2018.3.1’ in /nix/store/fa9zzkbljkvdavwzirkrr5irg25ymbjl-source/pkgs/applications/editors/jetbrains/default.nix:230 has an unfree license (‘unfree’), refusing to evaluate.

    a) For `nixos-rebuild` you can set
      { nixpkgs.config.allowUnfree = true; }
    in configuration.nix to override this.

    b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
      { allowUnfree = true; }
    to ~/.config/nixpkgs/config.nix.

This makes it extremely hard to find actual issues in the output. This
patch set makes the output much more condensed in Hydra:

    Failed to evaluate nifticlib-2.0.0: «unsupported»: is not supported on ‘x86_64-apple-darwin’
    Failed to evaluate dmd-2.081.2: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
    Failed to evaluate dmdBuild-2.081.2: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
    Failed to evaluate ldc-1.11.0: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
    Failed to evaluate ldcBuild-1.11.0: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
    Failed to evaluate ldc-0.17.5: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
    Failed to evaluate ldcBuild-0.17.5: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
2018-12-18 10:50:53 -05:00
Jan Tojnar aead6e12f9
Merge remote-tracking branch 'upstream/master' into staging 2018-12-16 22:55:06 +01:00
James Kay eb76dd37a7
makeStaticBinaries: add a static glibc when performing a mkDerivation 2018-12-14 11:02:18 +00:00
Matthew Bauer 9564b8ed9e cctools: don’t depend on clang at runtime
--disable-clang-as flag disables the use of clang by cctools-port
2018-12-10 17:31:47 -06:00
Piotr Bogdan 44c9c27d54 stdenv: prune libtool files by default 2018-12-09 22:45:15 +00:00
Frederik Rietdijk 5f554279ec Merge master into staging-next 2018-12-07 15:22:35 +01:00
Julien Moutinho 3168e85833 stdenv: add armv7a-linux system 2018-12-06 17:55:25 +01:00
Matthew Bauer 0b8574540b stdenv/darwin: fix portable libsystem hook
Some packages don’t have /bin directories. We should only run
install_name_tool if that directory exists.
2018-12-05 12:56:12 -06:00
Matthew Bauer 6d90a8b894 top-level/stage.nix: add static overlay
Adds the static overlay that can be used to build Nixpkgs statically.
Can be used like:

  nix build pkgsStatic.hello

Not all packages build, as some rely on dynamic linking.
2018-12-04 21:56:03 -06:00
Matthew Bauer 8726f6a558 stdenv/adapters.nix: fixup makeStaticBinaries
- makeStaticBinaries don’t work on Darwin (no stable ABI!)
- Need to make sure NIX_CFLAGS_LINK appends
- isStatic is not used anymore
2018-12-04 21:12:17 -06:00
Matthew Bauer a3a6ad7a01 stdenv: implement crossOverlays
crossOverlays only apply to the packages being built, not the build
packages. It is useful when you don’t care what is used to build your
packages, just what is being built. The idea relies heavily on the
cross compiling infrastructure. Using this implies that we need to
create a cross stdenv.
2018-12-04 21:06:46 -06:00
Jan Tojnar c5881ec2c9
Merge remote-tracking branch 'upstream/master' into staging 2018-11-30 20:09:45 +01:00
Matthew Bauer 28e2277305 make-derivation: remove selfConsistent check
version is set in lots of places but might not need to be in a name.

Alternative to #50364.
2018-11-30 17:58:33 +01:00
Frederik Rietdijk 1828a5c5ba Merge master into staging-next 2018-11-30 17:46:21 +01:00
Matthew Bauer e6834171b9
Merge pull request #51183 from matthewbauer/make-derivation-no-assertions
Fix breaking changes to make-derivation
2018-11-29 19:35:55 -06:00
Vaibhav Sagar b5504e26b0 pkgs/stdenv/generic/make-derivation.nix: s/targetPlatform/hostPlatform/ 2018-11-29 17:54:59 -05:00
Frederik Rietdijk 9b81c7e455 Merge staging-next into staging 2018-11-29 09:18:35 +01:00
Matthew Bauer a56fe056ec make-derivation: don’t add host suffix if there is no c compiler
Some trivial builders use the name attr to choose the exec name
produced. For example nixos-install,

  nixos-install = makeProg {
    name = "nixos-install";
    src = ./nixos-install.sh;
    nix = config.nix.package.out;
    path = makeBinPath [ nixos-enter ];
  };

When cross compiling, this puts the prog in,

  /bin/nixos-install-powerpc64le-unknown-linux-gnu
2018-11-28 12:44:10 -06:00
Matthew Bauer d64f288154 make-derivation: don’t assert on separateDebugInfo
When separateDebugInfo = true & !stdenv.hostPlatform.isLinux, we
always gave an error. There is no reason to do this. Instead, just
don’t add separate debug info when we aren’t on Linux.
2018-11-28 12:29:00 -06:00
Matthew Bauer 13e45aeed3 make-derivation: remove selfConsistent check
version is set in lots of places but might not need to be in a name.

Alternative to #50364.
2018-11-28 12:26:21 -06:00
Léo Gaspard 8ecd555ded
Merge branch 'pr-46056' into staging
* pr-46056:
  binutils: use shared libs
  binutils: fix #44936 the huge size regression
2018-11-28 09:55:05 +09:00
Vladimír Čunát a5de78b7d7
Merge branch 'master' into staging-next 2018-11-26 10:28:00 +01:00
Matthew Bauer 7eeb02d47b Revert "make-derivation: add disallowedReferences in strictDeps"
This reverts commit 8dbfb61e46.
Also reverts commit fc99c337ed.

Fixes #50915
2018-11-25 15:53:28 -06:00
Frederik Rietdijk c31cb577ae Merge master into staging-next 2018-11-22 09:57:08 +01:00
Matthew Bauer fc99c337ed make-derivation: don’t disallow propagated native inputs
propagateNativeBuildInputs will end up going in the output derivation.
This case is allowed to end up in references because of that. Sorry
for the disruption!

Fixes #50865
2018-11-21 09:39:47 -06:00
Daiderd Jordan c9223a17bc
Revert "patch-shebangs: use --build for auto patch shebangs"
Completely breaks darwin. Every package in the stdenv that has shebangs
in the output will end up with references to bootstrap-tools.

This reverts commit eb7c50a993.
2018-11-14 23:37:31 +01:00
Daiderd Jordan 25fafd2eb5
Revert "darwin/stdenv: bash is a build input"
Completely breaks darwin. Every package in the stdenv that has shebangs
in the output will end up with references to bootstrap-tools.

This reverts commit 2f2e635dd5.
2018-11-14 23:37:28 +01:00
Matthew Bauer f19bb8321a
Merge pull request #35304 from volth/patch-97
[staging] substitute() print warning if does nothing
2018-11-12 12:52:51 -06:00
Frederik Rietdijk 3b052406ea Merge staging-next into staging 2018-11-12 19:01:36 +01:00
Frederik Rietdijk daf3297cb4 Merge master into staging-next 2018-11-12 18:59:08 +01:00
Léo Gaspard 2986ce16a8
meta.tests: rename into passthru.tests
Nix currently rejects derivations in `meta` values. This works around
that limitation by using `passthru` instead.

Closes https://github.com/NixOS/nixpkgs/issues/50230
2018-11-11 23:11:46 +09:00
Léo Gaspard cae5598611
meta.tests: drop meta.needsVMSupport
Its job is already handled by `requiredSystemFeatures`
2018-11-11 23:11:46 +09:00
Léo Gaspard 83b27f60ce
tests: split into a separate all-tests.nix file
This will make the list much easier to re-use, eg. for `nixosTests`

The drawback is that this approaches makes the
```
nix-build release.nix -A tests.opensmtpd.x86_64-linux
```
command about twice as slow (3s to 6s): it now has to evaluate `nixpkgs`
once for each architecture, instead of just having the hardcoded list of
tests that allowed to say “ok just evaluate for x86_64-linux”.

On the other hand, complete evaluation of `release.nix` should be much
faster because we no longer import `nixpkgs` for each test: testing with
the following command went from 30s to 18s, and that's just for a few
tests.
```
time nix-instantiate --eval --strict nixos/release.nix -A tests.nat
```
I initially wanted to test on the whole `release.nix`, but there are too
many broken tests and it takes too long to eval them all, especially
compared to the fact that the current implementation breaks some setup.

Given developers can just `nix-build nixos/tests/my-test.nix`, it sounds
like an overall win.
2018-11-11 23:11:46 +09:00
Frederik Rietdijk 1d3bff25db Merge staging-next into staging 2018-11-11 14:28:08 +01:00
Frederik Rietdijk 53d00c3351 Merge master into staging-next 2018-11-10 11:08:54 +01:00
Matthew Bauer c8aff96110
Merge pull request #49608 from matthewbauer/cross-patch-shebangs-2
Restore cross-patch-shebangs branch
2018-11-07 13:37:02 -06:00
Matthew Bauer 6d531f3541 make-derivation: enable pie hardening with musl
Fixes #49071

On ld.gold, we produce broken executables when linking with the Musl
libc. This appears to be a known bug when using ld.gold and Musl. This
thread describes the workaround as enabling PIE when using ld.gold and
Musl:

https://www.openwall.com/lists/musl/2015/05/01/5

By default we don’t enable PIE to avoid breaking things. But in the
Musl case we are breaking things by not enabling PIE. So this adds a
special case for defaultHardeningFlags which keeps the pie hardening
for everything. Any packages that break with PIE can add the pie flag
to disableHardeningFlags array (a no-op for now on anything but Musl).
2018-11-07 15:24:51 +01:00
Timo Kaufmann 6141939d6e
Merge pull request #44439 from Ekleog/meta-tests
[RFC] Use `meta.tests` to link from packages to the tests that test them
2018-11-07 00:05:22 +01:00
Patrick Hilhorst 320c9c10de
make-derivation: use pname-version as default name if both are present 2018-11-06 00:04:21 +01:00
Frederik Rietdijk 322f87137c Merge master into staging-next 2018-11-04 11:33:17 +01:00
Frederik Rietdijk abea6f461a Revert "Merge pull request #49398 from Synthetica9/implement-rfc0035" to fix eval
This reverts commit 3fc7d5eb83, reversing
changes made to 1fddf2b689.

The idea is good, however, before enforcing, make sure all occurences
are fixed.
2018-11-04 11:02:43 +01:00
Frederik Rietdijk cb4ff927a1 Merge master into staging-next 2018-11-04 08:49:24 +01:00
Jörg Thalheim 3fc7d5eb83
Merge pull request #49398 from Synthetica9/implement-rfc0035
Implement rfc0035: default `name` from `pname`
2018-11-03 21:33:36 +00:00
Matthew Bauer 8dbfb61e46 make-derivation: add disallowedReferences in strictDeps
When strictDeps = true, we don’t want native build inputs to end up in
the output. For instance gcc is a builtin native build input and
should only show up in an output if it is also listed in buildInputs.

/cc @ericson2314
2018-11-02 19:31:51 -05:00
Matthew Bauer 2f2e635dd5 darwin/stdenv: bash is a build input
patch shebangs needs to be in build inputs for it to get into
HOST_PATH.
2018-11-02 00:27:14 -05:00
Matthew Bauer eb7c50a993 patch-shebangs: use --build for auto patch shebangs
In strictDeps=false, autoPatchshebangs should use
--build (corresponding to PATH) to lookup commands. This restores the
previous behavior of patchshebangs so that we don’t break stuff that
isn’t careful in the buildInputs vs. nativeBuildInputs distinction.
Unfortunately this won’t work under cross compilation.
2018-11-02 00:27:14 -05:00
John Ericson e3082c313b Merge remote-tracking branch 'upstream/master' into release-lib-cleanup 2018-11-01 16:47:42 -04:00
John Ericson f2ed7c7af9 linux bootstrap tools: Use right system for some raw derivations
This allows cross builds to work. Evidentallyy this has been done wrong
since I combined the bootstrap tool creation files in
ab651d2c9b. Oops!
2018-11-01 16:22:00 -04:00
John Ericson ffaffb36d1 linux bootstrap-tools: use stdenv.*Platform to avoid deprecation warning 2018-11-01 16:18:51 -04:00
Patrick Hilhorst 1f7fc09176
make-derivation: use a more descriptive assert message
As suggested by @Profpatsch
2018-10-30 14:33:14 +01:00
Léo Gaspard 02e1f00ffd
dovecot, opensmtpd: add link to test in meta.tests
Rationale
---------

Currently, tests are hard to discover. For instance, someone updating
`dovecot` might not notice that the interaction of `dovecot` with
`opensmtpd` is handled in the `opensmtpd.nix` test.

And even for someone updating `opensmtpd`, it requires manual work to go
check in `nixos/tests` whether there is actually a test, especially
given not so many packages in `nixpkgs` have tests and this is thus most
of the time useless.

Finally, for the reviewer, it is much easier to check that the “Tested
via one or more NixOS test(s)” has been checked if the file modified
already includes the list of relevant tests.

Implementation
--------------

Currently, this commit only adds the metadata in the package. Each
element of the `meta.tests` attribute is a derivation that, when it
builds successfully, means the test has passed (ie. following the same
convention as NixOS tests).

Future Work
-----------

In the future, the tools could be made aware of this `meta.tests`
attribute, and for instance a `--with-tests` could be added to
`nix-build` so that it also builds all the tests. Or a `--without-tests`
to build without all the tests. @Profpatsch described in his NixCon talk
such systems.

Another thing that would help in the future would be the possibility to
reasonably easily have cross-derivation nix tests without the whole
NixOS VM stack. @7c6f434c already proposed such a system.

This RFC currently handles none of these concerns. Only the addition of
`meta.tests` as metadata to be used by maintainers to remember to run
relevant tests.
2018-10-30 21:31:39 +09:00
Patrick Hilhorst c7e026bec4
make-derivation: use lib.assertMsg
As suggested by @Profpatsch
2018-10-29 18:25:59 +01:00
Patrick Hilhorst 5be927db14
make-derivation: use ? instead of builtins.hasAttr
As suggested by @edolstra
2018-10-29 15:17:13 +01:00
Patrick Hilhorst 2962f94fec
make-derivation: add check that the name is consistent with pname and version 2018-10-29 14:58:12 +01:00
Patrick Hilhorst 149a55eca7
make-derivation: get position info from version 2018-10-29 14:51:22 +01:00
Patrick Hilhorst efca8b4b97
make-derivation: use pname-version as default name if both are present 2018-10-29 14:51:08 +01:00
Matthew Bauer b3041b4455 make-derivation: set CMAKE_SYSTEM_* when cross compiling
Uses uname data to find what to set these variables:

- CMAKE_SYSTEM_NAME
- CMAKE_SYSTEM_PROCESSOR
- CMAKE_SYSTEM_VERSION
- CMAKE_HOST_SYSTEM_NAME
- CMAKE_HOST_SYSTEM_PROCESSOR
- CMAKE_HOST_SYSTEM_VERSION
2018-10-16 21:50:37 -05:00
Yegor Timoshenko cd0c8739d7
Merge pull request #37600 from abbradar/impureusenative
impureUseNativeOptimizations: add stdenv adapter
2018-10-13 14:09:55 +00:00
Matthew Bauer 93834fe194
Merge pull request #47230 from bhipple/fix/licenses
Remove dead code from stdenv check-meta license logic
2018-10-05 22:58:27 -05:00
John Ericson f49ca01c50 Revert "stdenv: partial revert of f2bb59e"
This reverts commit 607063f61b.
2018-09-26 14:47:16 -04:00
Matthew Bauer 607063f61b stdenv: partial revert of f2bb59e
/cc @Ericson2314

PR was https://github.com/NixOS/nixpkgs/pull/46857

This line broke MacOS cross compilation. paxctl cannot be built on
macOS. Maybe it can be fixed, but no reason to break things
unnecessarily.

Regardless, you definitely need to be more careful about backporting.
I think it’s fine to move fast and break things on master but
with release-18.09 we should be more careful. Something like more
automated testing for cross compilation would also be
helpful (hopefully even making it block).

(cherry picked from commit f9c4075873cb56464126f993d22a1a72f7cfac45)
2018-09-26 11:13:22 -04:00
xeji 6aa5f2db8f
Merge pull request #47245 from dtzWill/fix/coreutils-8.30-bootstrap
coreutils: try 8.30 again, fix bootstrap tools expression motivating revert before
2018-09-25 12:10:27 +02:00
aszlig b25b6e0c75
stdenv: Improve ELF detection for isELF
The isELF function only checks whether ELF is contained within the first
4 bytes of the file, which is a bit fuzzy and will also return
successful if it's a text file starting with ELF, for example:

  ELF headers
  -----------

  Some text here about ELF headers...

So instead, we're now doing a precise match on \x7fELF.

Signed-off-by: aszlig <aszlig@nix.build>
Acked-by: @Ericson2314
Closes: https://github.com/NixOS/nixpkgs/pull/47244
2018-09-25 06:55:18 +02:00
John Ericson 2b4b7d4ef3
Merge pull request #47233 from oxij/tree/mass-rebuild-noop-cleanups
treewide: mass rebuild noop cleanups
2018-09-25 00:04:52 -04:00
Will Dietz 4d9f9f171b make-bootstrap-tools: fix with latest coreutils
Since gcc.lib/lib64 is a symlink to 'lib', the use of
"lib*/libgcc_s.so*" triggered a warning (error) with
the latest coreutils.  Essentially we were doing:

$ cp a/x b/x y/

And latest coreutils rejects such invocations.

Just copy from 'lib', lib64 is a link to it anyway.

* Nothing else in this file bothers looking at lib*
* AFAICT lib* only ever possibly matched lib64 anyway
2018-09-23 14:54:09 -05:00
Jan Malakhovski b2c7a5a271 bintools-wrapper, cc-wrapper, stdenv: infer propagateDoc automatically
02c09e0171 (NixOS/nixpkgs#44558) was reverted in
c981787db9 but, as it turns out, it fixed an issue
I didn't know about at the time: the values of `propagateDoc` options were
(and now again are) inconsistent with the underlying things those wrappers wrap
(see NixOS/nixpkgs#46119), which was (and now is) likely to produce more instances
of NixOS/nixpkgs#43547, if not now, then eventually as stdenv changes.

This patch (which is a simplified version of the original reverted patch) is the
simplest solution to this whole thing: it forces wrappers to directly inspect the
outputs of the things they are wrapping instead of making stdenv guess the correct
values.
2018-09-23 17:29:56 +00:00
Benjamin Hipple 0b9d9ab256 Remove dead code from stdenv check-meta license logic
The `unfree` and `unfreeRedistributable` licenses both have `free = false`,
which will trigger the first portion of logic. This removes dead code to
simplify the logic.

As a follow-up, I plan to add an attribute `redistributable = [true|false]`,
which can be used by Hydra to determine whether a given package with a given
license can be included in the channel.
2018-09-23 12:48:02 -04:00
Frederik Rietdijk 56853dc6d8
Merge pull request #45941 from NixOS/staging-next
Staging next
2018-09-23 09:31:28 +02:00
Dan Peebles eeeeacc9a6 Revert "stdenv/darwin: bump bootstrap tools"
This accidentally added some unwanted dependencies on the bootstrap
tools, and I don't have time to fix before I go on vacation, so I'm
backing it out until I have time to address it properly.

This reverts commit dc5c68a7bb.
2018-09-20 23:43:53 -04:00
John Ericson 7319013ea1 Merge remote-tracking branch 'upstream/master' into staging 2018-09-18 16:55:42 -04:00
John Ericson 35378f0141
Merge pull request #46857 from obsidiansystems/darwin-to-linux-prep
misc pkgs: various cross fixes in preparation for darwin->linux
2018-09-18 16:52:30 -04:00
John Ericson f2bb59e710 stdenv linux, stdenv cross: Harmonize extraNativeBuildInputs
Want to make sure these are the same per host platform, without duplication.
2018-09-18 16:27:04 -04:00
John Ericson 2111e7b742 mkDerivation: Make separateDebugInfo assertion lazier to match other assertions
This is needed to access attributes of derivations on platforms where
they cannot be built.
2018-09-18 16:25:19 -04:00
Frederik Rietdijk de419917a3 Merge master into staging-next 2018-09-18 18:44:48 +02:00
Graham Christensen b80c9ce4a9
stdenv: Validate meta.outputsToInstall
If meta.outputsToInstall is set to include absent outputs, various
tools break including channel updates and nix-env.

    grahamc@Morbo> nix-env -i -f . -A elf-header-real
    installing 'elf-header'
    error: this derivation has bad 'meta.outputsToInstall'

This patch verifies each value in meta.outputsToInstall is a valid
output. It validates this condition only if checkMeta is true.

    grahamc@Morbo> nix-build . -A elf-header-real
    error: Package ‘elf-header’ in /home/grahamc/projects/nixpkgs/pkgs/development/libraries/elf-header/default.nix:36 has invalid meta.outputsToInstall, refusing to evaluate.

    The package elf-header has set meta.outputsToInstall to: bin

    however elf-header only has the outputs: out

    and is missing the following ouputs:

      - bin

    (use '--show-trace' to show detailed location information)

Note, now the nix-env experience is decidedly worse for users who have
checkMeta set to true:

    grahamc@Morbo> nix-env -i -f . -A elf-header-real; echo $?
    0

though since this is already an issue for unfree, broken, unsupported,
and insecure validity problems I'm not sure we should do something
different here.
2018-09-18 10:38:44 -04:00
Dan Peebles dc5c68a7bb stdenv/darwin: bump bootstrap tools
You can verify the provenance of these yourself by checking Hydra here:
https://hydra.nixos.org/build/81511173
2018-09-17 18:46:26 -04:00
Matthew Bauer ba5717a6f5 stdenv: fix HOST_PATH change
a4630c65ca was incorrect in assuming $SHELL would be a path to the
bash derivation. In fact $SHELL will be a path to the bash executable.

Unfortunately this did not fix the original issue. So instead, we just
have to reuse initialPath can be added like PATH is.

Sorry for the inconvenience! I hadn’t thought through the effects of
the last commit.

/cc @copumpkin @ericson2314
2018-09-17 14:18:06 -05:00
Daniel Peebles 481dd45e61
Merge pull request #46730 from copumpkin/add-darwin-lto
cctools: support LTO on Darwin
2018-09-17 00:59:22 -04:00
Matthew Bauer a4630c65ca stdenv: add shell to HOST_PATH for backwards compatibility
To avoid breaking things, we need to make sure SHELL goes into
HOST_PATH. This reflects my changes to patch-shebangs to make it cross
compilation ready. When a script is patched from the Nix store it now
looks to HOST_PATH to get the targeted machine’s executables.
Unfortunately, this only works in native builds.
2018-09-16 15:58:21 -05:00
Dan Peebles 110c252870 cctools: support LTO on Darwin
LTO is disabled during bootstrap to keep the bootstrap tools small and
avoid unnecessary LLVM rebuilds, but is enabled in the final stdenv
stage and should be usable by normal packages.
2018-09-16 02:12:11 -04:00
Dan Peebles 4efd4053ed stdenv/darwin: integrate a new CoreFoundation
This also updates the bootstrap tool builder to LLVM 5, but not the ones
we actually use for bootstrap. I'll make that change in a subsequent commit
so as to provide traceable provenance of the bootstrap tools.
2018-09-15 16:05:46 -04:00
Uli Baum 1df2560dde Merge branch 'master' into staging-next 2018-09-13 10:08:53 +02:00
Jan Malakhovski b7bd0561be Merge branch 'master' into staging 2018-09-08 22:08:32 +00:00
John Ericson 24209d29f0
Merge pull request #46148 from obsidiansystems/plain-system-host
top-level, stdenv: Make `system` and `stdenv.system` describe the hostPlatform
2018-09-06 09:37:20 -04:00
John Ericson 773233ca77 top-level, stdenv: Make system and stdenv.system describe the hostPlatform.
Intuitively, one cares mainly about the host platform: Platforms differ
in meaningful ways but compilation is morally a pure process and
probably doesn't care, or those difference are already abstracted away.
@Dezgeg also empirically confirmed that > 95% of checks are indeed of
the host platform.

Yet these attributes in the old cross infrastructure were defined to be
the build platform, for expediency. And this was never before changed.
(For native builds build and host coincide, so it isn't clear what the
intention was.)

Fixing this doesn't affect native builds, since again they coincide. It
also doesn't affect cross builds of anything in Nixpkgs, as these are no
longer used. It could affect external cross builds, but I deem that
unlikely as anyone thinking about cross would use more explicit
attributes for clarity, all the more so because the rarity of inspecting
the build platform.
2018-09-06 08:33:51 -04:00
John Ericson 32df8909e5
Merge pull request #46076 from oxij/tree/cleanups
trivial: treewide: random noop cleanups
2018-09-05 15:14:12 -04:00
John Ericson 51907d257c stdenv, neovim: Use lib.warn for deprecation warnings 2018-09-05 11:40:29 -04:00