Commit graph

1779 commits

Author SHA1 Message Date
John Ericson f567a851a1 darwin stdenv: Properly compose overrides
`super` usage was very suspect.
2018-05-23 09:59:29 -04:00
John Ericson 64c90e53e7 darwin stdenv: Persist darwin.binutils-unwrapped
binutils-unwrapped was added in ef3db7d14c
and needs a corresponding `persisted` attr in the Darwin stdenv.
2018-05-23 09:51:16 -04:00
Tuomas Tynkkynen 003473613a Merge remote-tracking branch 'upstream/master' into staging
Conflicts:
	pkgs/top-level/all-packages.nix
2018-05-18 03:54:38 +03:00
Bastian Köcher 832a8ca087 androidndk: Fix usage as crossSystem 2018-05-17 17:22:27 +02:00
John Ericson 5a860c20f3 stdenv cross adapter: Don't define crossConfig
All its uses have been removed.
2018-05-14 23:33:12 -04:00
John Ericson 5e17335bd7 Merge remote-tracking branch 'upstream/staging' into strictDeps 2018-05-14 23:33:03 -04:00
John Ericson 330ca731e8 treewide: Get rid of all uses of crossConfig
The hack of using `crossConfig` to enforce stricter handling of
dependencies is replaced with a dedicated `strictDeps` for that purpose.
(Experience has shown that my punning was a terrible idea that made more
difficult and embarrising to teach teach.)

Now that is is clear, a few packages now use `strictDeps`, to fix
various bugs:

 - bintools-wrapper and cc-wrapper
2018-05-14 23:30:37 -04:00
John Ericson d00cc1242f
Merge pull request #40040 from obsidiansystems/gnu-config-arm
gnu-config: Update, allowing hacks to be removed
2018-05-14 11:20:09 -04:00
John Ericson f2b575bd7b Merge remote-tracking branch 'upstream/master' into gnu-config-arm 2018-05-14 10:58:15 -04:00
John Ericson eebd455dc0 linux stdenv: Update gnu-config on all non-x86
Not just Aarch64. Other non-x86 platforms might be old enough, but I am
about to update gnu-config to handle things better across the board.
2018-05-14 10:55:32 -04:00
Daiderd Jordan f14841f3ce
Revert "darwin.libSystem: multiple outputs"
Broke the stdenv.

This reverts commit 69d1b7ab79.
2018-05-13 10:01:52 +02:00
Frederik Rietdijk a18b493e02 Merge master into staging 2018-05-09 10:48:01 +02:00
John Ericson 5bdfe37daf
Merge pull request #40154 from obsidiansystems/stdenv-arm-alias
stdenv: Put back isArm, with deprecation notice.
2018-05-08 14:24:58 -04:00
John Ericson 302c4c5f2d stdenv: Put back isArm, with deprecation notice.
This was always meant to be deprecated rather than removed.
2018-05-07 20:14:52 -04:00
John Ericson 4f7cdd35d5
Merge pull request #40139 from obsidiansystems/modular-setup-hooks
treewide: Modular setup hooks
2018-05-07 15:32:10 -04:00
John Ericson 34a3233a2e stdenv: Support concatenating setup hooks from multiple parts. 2018-05-07 14:43:22 -04:00
Matthew Justin Bauer 5206574be3
stdenv: add libSystem.dev to allowedRequisites 2018-05-03 18:02:57 -05:00
Matthew Justin Bauer eeb016e8f0
Merge branch 'staging' into fix-ncurses-darwin-extensions 2018-05-02 15:40:38 -05:00
Michael Raskin a591d28c17
Merge pull request #39464 from oxij/stdenv/docheck-infra
stdenv: implement most of #33599
2018-04-30 17:17:24 +00:00
Michael Raskin ce929e6a92
Merge pull request #39457 from oxij/stdenv/texinfo-bash
stdenv: change texinfo bootstrap handling; texinfo, bash: simplify expressions
2018-04-27 19:03:26 +00:00
Michael Raskin 6ca343103b check-meta.nix: specify meta.timeout 2018-04-27 00:25:37 +02:00
Jan Malakhovski 87651b32fe stdenv: steal checkInputs from buildPythonPackage
Note that a bunch of non-python packages use this attribute already.
Some of those are clearly unaware of the fact that this attribute does
not exists in stdenv because they define it but don't to add it to
their `bulidInputs` :)

Also note that I use `buildInputs` here and only handle regular
builds because python and haskell builders do it this way and I'm not
sure how to properly handle the cross-compilation case.
2018-04-26 20:22:51 +00:00
Jan Malakhovski 845fa56921 stdenv: cleanup things a little bit 2018-04-26 20:15:51 +00:00
Jan Malakhovski d834ba6654 stdenv: introduce and use config.doCheckByDefault option 2018-04-26 20:15:47 +00:00
Will Dietz f31b147f63 linux bootstrap: remove assertion failure from dev, fixes 'nix eval'
As in:

$ nix eval -f . bash

Also remove the glibc propagation inherit that made these necessary,
stages handle propagating libc themselves (apparently) and
AFAICT no hashes are changed as a result of this.
2018-04-26 10:19:52 -05:00
Jan Malakhovski 50af975d85 stdenv: implement checkTarget and installCheckTarget autodetection 2018-04-25 19:53:25 +00:00
Jan Malakhovski e9e06888ed stdenv: generic/setup.sh: cleanup installPhase 2018-04-25 19:53:25 +00:00
Jan Malakhovski ad98c36f1b stdenv: generic/setup.sh: simplify buildPhase Makefile check 2018-04-25 19:53:24 +00:00
Jan Malakhovski a809fdc8e1 stdenv: linux: stop recompiling texinfo
This makes stdenv bootstraping a bit more efficient.
We don't recompile most of other stuff used in bootstrapping stdenv too.
2018-04-25 19:50:54 +00:00
John Ericson b9acfb4ecf treewide: isArm -> isAarch32
Following legacy packing conventions, `isArm` was defined just for
32-bit ARM instruction set. This is confusing to non packagers though,
because Aarch64 is an ARM instruction set.

The official ARM overview for ARMv8[1] is surprisingly not confusing,
given the overall state of affairs for ARM naming conventions, and
offers us a solution. It divides the nomenclature into three levels:

```
ISA:             ARMv8   {-A, -R, -M}
                 /    \
Mode:     Aarch32     Aarch64
             |         /   \
Encoding:   A64      A32   T32
```

At the top is the overall v8 instruction set archicture. Second are the
two modes, defined by bitwidth but differing in other semantics too, and
buttom are the encodings, (hopefully?) isomorphic if they encode the
same mode.

The 32 bit encodings are mostly backwards compatible with previous
non-Thumb and Thumb encodings, and if so we can pun the mode names to
instead mean "sets of compatable or isomorphic encodings", and then
voilà we have nice names for 32-bit and 64-bit arm instruction sets
which do not use the word ARM so as to not confused either laymen or
experienced ARM packages.

[1]: https://developer.arm.com/products/architecture/a-profile

(cherry picked from commit ba52ae5048)
2018-04-25 15:50:41 -04:00
John Ericson ba52ae5048 treewide: isArm -> isAarch32
Following legacy packing conventions, `isArm` was defined just for
32-bit ARM instruction set. This is confusing to non packagers though,
because Aarch64 is an ARM instruction set.

The official ARM overview for ARMv8[1] is surprisingly not confusing,
given the overall state of affairs for ARM naming conventions, and
offers us a solution. It divides the nomenclature into three levels:

```
ISA:             ARMv8   {-A, -R, -M}
                 /    \
Mode:     Aarch32     Aarch64
             |         /   \
Encoding:   A64      A32   T32
```

At the top is the overall v8 instruction set archicture. Second are the
two modes, defined by bitwidth but differing in other semantics too, and
buttom are the encodings, (hopefully?) isomorphic if they encode the
same mode.

The 32 bit encodings are mostly backwards compatible with previous
non-Thumb and Thumb encodings, and if so we can pun the mode names to
instead mean "sets of compatable or isomorphic encodings", and then
voilà we have nice names for 32-bit and 64-bit arm instruction sets
which do not use the word ARM so as to not confused either laymen or
experienced ARM packages.

[1]: https://developer.arm.com/products/architecture/a-profile
2018-04-25 15:28:55 -04:00
Will Dietz 97698b0d6d musl-stdenv: don't add special allowance for libiconv 2018-04-25 08:08:47 -05:00
Will Dietz 791b5bbbf4 musl bootstrap: remove libiconv 2018-04-24 21:27:28 -05:00
Tuomas Tynkkynen 3c6e077301 Merge remote-tracking branch 'upstream/master' into HEAD
Conflicts:
	pkgs/development/tools/misc/binutils/default.nix
2018-04-22 22:31:30 +03:00
John Ericson cf0dd1dbd2
Merge pull request #39172 from obsidiansystems/ios-cross
ios-sdk-pkgs: Init from iOS SDK from XCode
2018-04-19 17:00:05 -04:00
Ken Micklas ef3db7d14c ios-sdk-pkgs: Init from iOS SDK from XCode 2018-04-19 16:09:30 -04:00
John Ericson 53686e8995
Merge pull request #38485 from obsidiansystems/nixos-nixpkgs-options
nixpkgs module: Clean up platform options
2018-04-19 14:59:58 -04:00
John Ericson 204b73ffba
Merge pull request #38932 from svanderburg/cygwin-fixes
Fix stdenv-native and fix rebasing on cygwin
2018-04-18 11:05:19 -04:00
John Ericson 9edda8aef1
Merge pull request #38639 from shlevy/platforms-not-broken
meta: Don't bypass unsupported platforms with allowBroken.
2018-04-17 17:23:50 -04:00
John Ericson cb212cf549 meta: Add NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM for consistency 2018-04-17 16:02:50 -04:00
Shea Levy 3955b84698 meta: Don't bypass unsupported platforms with allowBroken.
Our platforms are open-world oriented these days, and anyway there's allowUnsupportedSystem.
2018-04-17 16:02:15 -04:00
Nikolay Amiantov 15bfee85f3 impureUseNativeOptimizations: add stdenv adapter
This allows one to force a compiler to use native machine optimizations. This
goes contrary to all the usual guarantees of Nix and so should be used only by
end-user and only in specific cases when they know what are they doing.

In my case this is needed to get a noticeable FPS boost in RPCS3 which is very
CPU-hungry PlayStation 3 emulator.
2018-04-17 13:12:07 +03:00
Sander van der Burg a7adffdd47 Fix invalid Baseaddress must be > 0x200000000 on cygwin 2018-04-14 14:06:09 +02:00
Sander van der Burg 5a545ca9bd Fix stdenv-native by creating a bintools wrapper with similar settings 2018-04-14 14:05:09 +02:00
John Ericson 0884027ef5 Revert "Revert "Merge pull request #28029 from cstrahan/hardening-fix""
This reverts commit 6c064e6b1f, reapplying
PR #28029 which was supposed to have gone to staging all along.
2018-04-11 14:00:13 -04:00
John Ericson c6f7d43678 nixpkgs module: Clean up platform options
- `localSystem` is added, it strictly supercedes system

 - `crossSystem`'s description mentions `localSystem` (and vice versa).

 - No more weird special casing I don't even understand

TEMP
2018-04-06 12:41:44 -04:00
John Ericson adaa110a72 binutils: No more darwin conditionals
Since at least d7bddc27b2, we've had a
situation where one should depend on:

 - `stdenv.cc.bintools`: for executables at build time
 - `libbfd` or `libiberty`: for those libraries
 - `targetPackages.cc.bintools`: for exectuables at *run* time
 - `binutils`: only for specifically GNU Binutils's executables,
   regardless of the host platform, at run time.

and that commit cleaned up this usage to reflect that. This PR flips the
switch so that:

 - `binutils` is indeed unconditionally GNU Binutils
 - `binutils-raw`, which previously served that role, is gone.

so that the correct usage will be enforced going forward and everything
is simple.

N.B. In a few cases `binutils-unwrapped` (which before and now was
unconditionally actual GNU binutils), rather than `binutils` was used to
replace old `binutils-raw` as it is friendly towards some cross
compilation usage by avoiding a reference to the next bootstrapping
change.
2018-04-03 13:34:52 -04:00
John Ericson f8ed783f4f meta: Simplify platform check logic
Code golf or readability, you decide
2018-03-27 11:59:59 -04:00
Shea Levy b0482248fe
meta: Add badPlatforms attribute for platform blacklisting. 2018-03-27 08:12:45 -04:00
John Ericson 45e253cacb cross stdenv: Make depsBuildBuild overrideable by config too. 2018-03-20 15:58:09 -04:00
John Ericson 3c331bff5b
Merge pull request #37395 from obsidiansystems/lib-meta-platform
lib: Factor in tiny bit of `meta.platform` checking
2018-03-19 20:12:50 -04:00
John Ericson e547bd0dc4 lib: Factor in tiny bit of meta.platform checking
I need it in stdenv and release-lib, so that seems motivation enough.
2018-03-19 19:29:16 -04:00
John Ericson 2fa2197a96
Merge pull request #34444 from obsidiansystems/meta-check
lib: Fix #30902
2018-03-18 13:51:03 -04:00
John Ericson c26252af3e lib, stdenv: Check meta.platforms against host platform and be open world
First, we need check against the host platform, not the build platform.
That's simple enough.

Second, we move away from exahustive finite case analysis (i.e.
exhaustively listing all platforms the package builds on). That only
work in a closed-world setting, where we know all platforms we might
build one. But with cross compilation, we may be building for arbitrary
platforms, So we need fancier filters. This is the closed world to open
world change.

The solution is instead of having a list of systems (strings in the form
"foo-bar"), we have a list of of systems or "patterns", i.e. attributes
that partially match the output of the parsers in `lib.systems.parse`.
The "check meta" logic treats the systems strings as an exact whitelist
just as before, but treats the patterns as a fuzzy whitelist,
intersecting the actual `hostPlatform` with the pattern and then
checking for equality. (This is done using `matchAttrs`).

The default convenience lists for `meta.platforms` are now changed to be
lists of patterns (usually a single pattern) in
`lib/systems/for-meta.nix` for maximum flexibility under this new
system.

Fixes #30902
2018-03-15 00:44:34 -04:00
John Ericson 4c52e34ca6 stdenv: Clean up check meta args 2018-03-14 18:58:07 -04:00
Jan Malakhovski 7079e744d4 Merge branch 'master' into staging
Resolved the following conflicts (by carefully applying patches from the both
branches since the fork point):

   pkgs/development/libraries/epoxy/default.nix
   pkgs/development/libraries/gtk+/3.x.nix
   pkgs/development/python-modules/asgiref/default.nix
   pkgs/development/python-modules/daphne/default.nix
   pkgs/os-specific/linux/systemd/default.nix
2018-03-10 20:38:13 +00:00
John Ericson 8f54d931cd
Merge pull request #36401 from obsidiansystems/no-stdenv-cross-adapter
stdenv cross adapter: Get rid if `selfNativeBuildInput`
2018-03-06 20:17:13 -05:00
John Ericson 8d58008039 stdenv cross adapter: Get rid if selfNativeBuildInput
It is finally no longer used anywhere

Fixes #30587
2018-03-06 20:05:13 -05:00
Will Dietz a6e7c85ab8 x86_64-musl bootstrap regen, much smaller
compressed: 28M -> 20M
uncompressed: 201M -> 119M

Built using:
94f3dc4fdf

cc https://github.com/NixOS/nixpkgs/pull/36383
2018-03-06 11:40:23 -06:00
Will Dietz d7c2cffd86 x86_64-musl: regen again w/symlinks preserved 2018-03-06 11:40:08 -06:00
Will Dietz 856335396f x86_64-musl: use native-built bootstrap package on x86_64
Built using:
46f83fa9d1
2018-03-06 11:36:52 -06:00
Jan Malakhovski 1d4f3fa371 stdenv: check-meta: fix fallout from #36119 2018-03-06 14:14:17 +00:00
Will Dietz ece4c62d4b make-bootstrap-tools: preserve coreutils symlinks
We go out of our way (see top of file) to build a single binary
with symlinks for all of the tools, but were losing them
when preparing the bootstrap tools.
2018-03-06 15:13:56 +02:00
Tuomas Tynkkynen 34f95d92a2 Merge remote-tracking branch 'upstream/master' into staging
Conflicts:
	pkgs/applications/misc/pytrainer/default.nix
	pkgs/development/tools/pew/default.nix
	pkgs/tools/misc/you-get/default.nix
2018-02-28 20:52:49 +02:00
John Ericson 2482e2858e prebuilt android tools: Init using SDK
Expose as an option for the cross stdenv.
2018-02-27 14:15:39 -05:00
John Ericson dfc5d7835d
Merge pull request #35247 from telent/mips32
lib, treewide: Add missing MIPS arches, and fix existing usage
2018-02-27 14:01:15 -05:00
Will Dietz 3a21d5bce2 default to including "man" in outputsToInstall 2018-02-27 10:31:07 -06:00
John Ericson 4a29081a94
Merge pull request #35071 from oxij/stdenv/infopages
stdenv, bash: fixing info pages and stuff
2018-02-26 18:06:11 -05:00
Jan Malakhovski 49afe1d490 stdenv: darwin: give distinct names to all the stages 2018-02-26 22:46:34 +00:00
Jan Malakhovski e654d9a9ca stdenv: linux: give distinct names to all the stages
Mainly for debugging.
2018-02-26 22:46:34 +00:00
Jan Malakhovski 522437362a stdenv: linux, darwin: don't build documentation for the the intermediate stages
For the cc of the intermediate stages, to be precise. Doing the same for
bintools requires lots of refactoring.

This is mainly for the future extensibility as now you can change
documentation generation with impunity without rebuilding the
whole of stdenv.
2018-02-26 22:45:10 +00:00
Shea Levy 1c1a6dfd23
libgcrypt: Fix cross-compilation 2018-02-24 22:51:22 -05:00
Daniel Barlow 9c50ae6898 lib, treewide: Add missing MIPS arches, and fix existing usage
Existing "mips64el" should be "mipsel".

This is just the barest minimum so that nixpkgs can recognize them as
systems - although required for building individual derivations onto
MIPS boards, it is not sufficient if you want to actually build nixos on
those targets
2018-02-23 20:43:42 -05:00
Vladimír Čunát 1d15dadbec
Merge branch 'master' into staging
Larger rebuilds from master.
2018-02-20 20:33:40 +01:00
Vladimír Čunát 882062004b
Merge branch 'staging' into gcc-7
I'm betting Hydra resources on gcc-7 getting to master before current staging.
2018-02-19 15:24:32 +01:00
Michael Raskin 7d2642724f
Merge pull request #35111 from oxij/stdenv/available-evaluates
stdenv.mkDerivation: rename `meta.evaluates` -> `meta.available`
2018-02-19 14:15:06 +00:00
Michael Raskin e37b756b91
Merge pull request #35110 from oxij/pkgs/pretty-fetchurl
fetchurl: cleanup, better errors
2018-02-19 14:05:12 +00:00
Shea Levy bbfc9ce376
Merge remote-tracking branch 'origin/master' into staging 2018-02-18 16:56:39 -05:00
Shea Levy 32ce7012f0
Merge branch 'master' into gcc-7 2018-02-18 16:27:22 -05:00
Shea Levy 0a0ecc4c2c
Enable building riscv64 cross bootstrap tools
Fixes #35089
2018-02-18 16:07:13 -05:00
Jan Malakhovski a89899ce4e fetchurl: cleanup, better errors
Also fix what seems like bugs in uncommon `stdenv`s.
2018-02-18 14:24:53 +00:00
Jan Malakhovski 810c4702cf stdenv.mkDerivation: rename meta.evaluates -> meta.available
A much better name.
2018-02-18 13:33:25 +00:00
Shea Levy b24ce2ae63
Handle sourceRoots with leading dashes 2018-02-15 11:50:24 -05:00
Tuomas Tynkkynen b1916b45a3 Merge remote-tracking branch 'upstream/staging' into gcc-7
Conflicts:
	pkgs/development/libraries/libidn/default.nix
	pkgs/top-level/all-packages.nix
2018-02-15 15:45:37 +02:00
Will Dietz 84a527ea5e x86_64-musl bootstrap: use separate 64bit busybox
Avoid issues like #24954.
2018-02-13 09:45:05 -06:00
Will Dietz 4619f8e06d new musl bootstrap URL's, much smaller. rebuild all the things. 2018-02-13 09:45:04 -06:00
Will Dietz 268bff6185 make-bootstrap-tools: set XZ to maximum level, big wins
x86_64 bootstrap tarball goes from 37M -> 21M (!)
2018-02-13 09:45:04 -06:00
Will Dietz a4c69744dc aarch64-musl, armv6l-musl: fetch from wdtz.org instead of storePath 2018-02-13 09:45:04 -06:00
Will Dietz 8bedb690a2 add musl bootstrap paths for aarch64, armv6l
Aarch64 tools tested briefly with qemu-aarch64,
but neither have been actually used yet :).

For now only "host" indirectly via binary cache
at cache.allvm.org.
2018-02-13 09:45:04 -06:00
Will Dietz d00f200f4e bootstrap-files: nicer naming 2018-02-13 09:45:03 -06:00
Will Dietz 119920faa6 restore stdenv.glibc, will remove separately. 2018-02-13 09:45:03 -06:00
Will Dietz c8d03e6298 make-bootstrap-tools: grab libc from stdenv.cc.libc
This is the same in current cases AFAICT,
other than uses musl instead of glibc when musl-native.
2018-02-13 09:44:57 -06:00
Will Dietz 9bc8127dc3 bootstrap-tools-cross, release-cross: add various musl entries 2018-02-13 09:44:56 -06:00
Will Dietz 4b6b83f159 linux stdenv: find bootstrap files by libc, then arch 2018-02-13 09:44:42 -06:00
Will Dietz 5a8002873e fixup linux stdenv bootstrap 2018-02-13 09:44:40 -06:00
Will Dietz 9dbbd75e54 musl bootstrap: allow libiconv 2018-02-13 09:44:40 -06:00
Will Dietz fd00d37b4f musl64 bootstrap: grab from gravity, requires netrc magic 2018-02-13 09:44:39 -06:00
Will Dietz 3ee5094934 linux bootstrap for musl: kludgery 2018-02-13 09:44:39 -06:00
Will Dietz 0e16989d39 bootstrap-tools-musl: WIP 2018-02-13 09:44:37 -06:00
Will Dietz a0af2aadb6 for now, copy bootstrap-tools instead of modify in-place 2018-02-13 09:44:37 -06:00
Will Dietz 0ac504227d make-bootstrap-tools: initial musl support, less glibc-specific 2018-02-13 09:44:37 -06:00
Will Dietz 4831495995 make-bootstrap-tools: glibc -> libcCross 2018-02-13 09:44:37 -06:00
Will Dietz 5dcf3cdf37 stdenv: automatically update config.sub for musl 2018-02-13 09:44:36 -06:00
Will Dietz 5ac5a3ebb5 make-bootstrap-tools-cross.nix: add musl -> musl64 2018-02-13 09:44:35 -06:00
Tuomas Tynkkynen a6fd03876e check-meta: Use concatStrings 2018-02-11 00:17:47 +02:00
Vladimír Čunát 03eef81727
darwin bootstrap tools: fix after #34339 (http2 in curl) 2018-02-10 19:20:44 +01:00
Franz Pletz c135ecd7ef
Merge remote-tracking branch 'origin/master' into gcc-7 2018-02-05 13:11:33 +01:00
Tuomas Tynkkynen 6a11ebfae9 Merge remote-tracking branch 'upstream/master' into gcc-7
Conflicts:
	pkgs/development/compilers/gcc/7/default.nix
	pkgs/development/libraries/SDL2/default.nix
	pkgs/top-level/all-packages.nix
2018-01-26 16:46:43 +02:00
Shea Levy f83b6e1130
unpackPhase: Handle sources starting with a hyphen 2018-01-24 21:58:57 -08:00
John Ericson 5a754e75b4 Merge branch 'ericson2314-cross-master' into staging 2018-01-16 13:05:39 -05:00
Vladimír Čunát 67e8392383
Merge #33057: stdenv meta checks: make them lazy
Closes #22277 - it's superseded;  I have some WIP on evaluation
performance, but best do that in a separate PR/thread.
2018-01-14 21:41:31 +01:00
Tuomas Tynkkynen 6ed0fe7e45 Merge remote-tracking branch 'upstream/master' into staging
Conflicts:
	pkgs/build-support/fetchbower/default.nix
	pkgs/build-support/fetchdarcs/default.nix
	pkgs/build-support/fetchgx/default.nix
	pkgs/development/python-modules/botocore/default.nix
	pkgs/os-specific/linux/firmware/firmware-linux-nonfree/default.nix
	pkgs/tools/admin/awscli/default.nix
2018-01-14 21:18:27 +02:00
Jan Malakhovski fac3d49e48 stdenv: provide meta.name 2018-01-14 13:24:30 +00:00
Jan Malakhovski 50148f0630 stdenv: hide name under check-meta assert
This is a temporary workaround to make `nix-env -qa` and `nix search` ignore
broken packages as they they did before this patchset.

This patch should be reverted after `nix` gets a proper fix for this.
See NixOS/nix#1771.
2018-01-14 13:24:30 +00:00
Jan Malakhovski eaee2a1199 stdenv: implement config.checkMetaRecursively
This option makes `meta.evaluate` into a close approximation of the result of
evaluating `.outPath` by checking all the dependencies recursively at a cost of
2x slowdown. Note that actually evaluating `.outPath` costs some
5x-7x more because `.outPath` also computes all the hashes.
2018-01-14 13:24:30 +00:00
Jan Malakhovski ecd3990cd4 stdenv: provide meta.evaluates
This gives a way to see the result of `check-meta` without triggering any assertions.
2018-01-14 13:24:29 +00:00
Jan Malakhovski 9956687151 stdenv: change some indent 2018-01-14 13:24:22 +00:00
John Ericson e017a027d5
Merge pull request #33681 from obsidiansystems/fixed-output-deps
Fixed output deps
2018-01-10 14:28:10 -05:00
John Ericson 940c4fa3f5 treewide: Fetchers should use stdenvNoCC. 2018-01-10 11:18:44 -05:00
John Ericson c836910e0e
Merge pull request #33670 from obsidiansystems/cross-check
stdenv: Fix doCheck and doInstallCheck logic
2018-01-09 16:37:11 -05:00
John Ericson 567feb3a54 stdenv: Fix doCheck and doInstallCheck logic
It's host != build, not host != target
2018-01-09 16:22:58 -05:00
John Ericson 06a8d66528
Merge pull request #33603 from obsidiansystems/cross-check
stdenv: Force `doCheck` to be false when we are cross compiling
2018-01-09 15:09:54 -05:00
John Ericson 4e907dbca1 stdenv: Force doCheck and doInstallCheck to be false when we are cross compiling
I hope this will be a temporary measure. If there is consensus around
issue #33599, then we can follow an explicit `dontCheck`, but default to
not checking during cross builds when none is given.
2018-01-09 12:37:12 -05:00
Will Dietz 9721ed22e8 schedulingPriority should be an int, fix check-meta type and in-tree use 2018-01-09 07:25:24 -06:00
adisbladis c2316114bc
stdenv: Kill off ensureDir 2018-01-09 11:14:48 +08:00
Daiderd Jordan e66a5f8862
Merge pull request #33342 from LnL7/clang-outputs
clang: add lib output
2018-01-07 09:43:38 +01:00
Jan Malakhovski d1d5ecb3bf stdenv: perform checks only when evaluating .drv and .out
This pushes check-meta evaluation to derivation evaluation step, leaving all other
attributes accessible.

Before this commit:

> $ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate --eval --strict ./default.nix -A xen --argstr system aarch64-linux
> Package ‘xen-4.5.5’ in pkgs/applications/virtualization/xen/generic.nix:226 is not supported on ‘aarch64-linux’, refusing to evaluate.

as expected

> $ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate --eval --strict ./default.nix -A xen.name --argstr system aarch64-linux
> Package ‘xen-4.5.5’ in pkgs/applications/virtualization/xen/generic.nix:226 is not supported on ‘aarch64-linux’, refusing to evaluate.

> $ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate --eval --strict ./default.nix -A xen.meta.description --argstr system aarch64-linux
> Package ‘xen-4.5.5’ in pkgs/applications/virtualization/xen/generic.nix:226 is not supported on ‘aarch64-linux’, refusing to evaluate.

which is unfortunate since its impossible to use packages in autogenerated
documentation on all platforms.

After this commit:

> $ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate --eval --strict ./default.nix -A xen --argstr system aarch64-linux

still fails

> $ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate --eval --strict ./default.nix -A xen.name --argstr system aarch64-linux
> "xen-4.5.5"

> $ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate --eval --strict ./default.nix -A xen.meta.description --argstr system aarch64-linux
> "Xen hypervisor and related components (vanilla)"
2018-01-03 12:20:25 +00:00
Daiderd Jordan b0c043d26c
stdenv: allow lib output of clang on darwin 2018-01-02 20:00:31 +01:00
John Ericson ab651d2c9b linux bootstrap tools: Use same derivation whether cross compiling or not 2018-01-02 13:52:41 -05:00
John Ericson 4d2b763817
Merge pull request #26805 from obsidiansystems/cross-elegant
Make cross compilation elegant
2017-12-30 22:58:02 -05:00
John Ericson 469fd89832 stdenv-setup: Ease the transition with native builds
- All deps go on the PATH

 - CC and Bintools wrappers with their host != depender's host still get their
   setup hooks run.

 - Environment hooks get applied to all packages

This isn't so elegent, but eases the transition on a very significant
PR.
2017-12-30 22:04:23 -05:00
John Ericson f083248290 linux bootstrap tools cross: Nuke more refs
libgcc.a and similar
2017-12-30 22:04:23 -05:00
John Ericson a036473a0a {bintools,cc}-wrapper: Fix setup hook to respect the role of the cc-compiler
We now have the information to properly determine the role the
cc-wrapper dependency has, by taking advantage of `offset`. No longer
use the soon-to-be-deprecated crossConfig environment variable, the
temp hack used before this change.
2017-12-30 22:04:21 -05:00
John Ericson 7f3ca3e21a stdenv: Fix handling of dependencies and hooks
4 far-reaching changes: Smaller PATH, New vars, different propagation
logic, and different hook logic

Smaller PATH
------------

`buildInputs` no longer go on the PATH at build time, as they cannot be
run when cross compiling and we don't want to special case. Simply make
a `nativeBuildInput` too if one needs them on the PATH. Fixes #21191.

Many new depedendency variables
-------------------------------

See the stdenv chapter of the nixpkgs manual. I pulled out the existing
documentation of dependency specification into a new section, and added
language for these two (and their propagated equivalents) along side
the others'.

More complex propagation logic
------------------------------

Before a propagated*XXX*Input always acted as if it was specified
directly as a *XXX*Input downstream. That's simple enough, but violates
the intended roles of each sort of dep, which has functional and not
just stylistic consequences.

The new algorithm is detailed in the manual, and ensures everything
ends up in the right place. I tried to give both an informal and formal
description, but I suspect in practice it will not make much sense
until one tries cross compiling, after which it will immediately make
sense as the only sane option.

Simplified hook logic
---------------------

Rather than `envHook` and `crossEnvHook`, whose behavior differs
depending on whether we are cross compiling or not, there is now one
hook per sort (or rather non-propagated and propagated pair of sorts)
of dependency. These new hooks have the same meaning regardless of
cross compilation. See the setup hook section of stdenv chapter of the
Nixpkgs manual for more details.
2017-12-30 22:04:21 -05:00
John Ericson eb27be0731
Merge pull request #33186 from obsidiansystems/cross-binutils
binutils: Fix cross, again
2017-12-29 17:42:38 -05:00
John Ericson 1cde06acf6 stdenv cross adapter: Get rid of extra propagatation of buildInputs
Binutils is patched so we don't rely on `--rpath-link`.
2017-12-29 17:32:28 -05:00
Tuomas Tynkkynen 47c782f7f5 Merge remote-tracking branch 'upstream/master' into staging 2017-12-26 21:10:12 +02:00
John Ericson 4651407654 darwin stdenv: Make stdenv.cc, not stdenv, bring in libcxx
stdenvNoCC should not inject any C++ standard library, just as it
doesn't inject any C standard library. stdenv still does, but only
indirectly through stdenv.cc. Wrapped clangs can be simplified now that
they don't need to worry about clobbering CoreFoundation when replacing
the C++ standard library implementation.

This generally-good cleanup should assist with debugging some C++
failures in #26805.
2017-12-25 19:32:07 -05:00
Vladimír Čunát ced4e5a683
darwin stdenv boostrap tools: use curl without kerberos
/cc #29785.  Otherwise we would have to put the lib in, etc.
2017-12-24 11:10:10 +01:00
Vladimír Čunát 24d81d6332
Merge branch 'master' into staging 2017-12-15 21:40:23 +01:00
Graham Christensen ae9a7c4969
stdenv: make knownVulnerabilities a known meta-type 2017-12-14 07:36:31 -05:00
John Ericson a0b1ebeee9 Merge remote-tracking branch 'upstream/staging' into binutils-wrapper 2017-12-13 16:14:47 -05:00
John Ericson 2bba929062 bintools-wrapper: Import separately from cc-wrapper 2017-12-13 16:08:18 -05:00
Graham Christensen f33a513d2b
stdenv: allow specifying a eval issuee handler 2017-12-12 18:08:11 -05:00
Vladimír Čunát 76bf375a16
stdenv checkMeta: throw -> trace
- tracing seems annoying enough
- we get errors for all packages instead of aborting on the first one
- easier to differentiate from unwanted packages (broken, unfree, etc.)
2017-12-12 18:07:07 -05:00
Will Dietz db7bee240e stdenv: fix use of config-specified allowInsecurePredicate
for some reason we were checking allowUnfreePredicate instead
2017-12-09 20:43:30 +00:00
Jan Malakhovski 1858e8909e tree-wide: fix either check-meta or meta attrs of all the packages I evaluate 2017-12-05 13:46:52 +01:00
Jan Malakhovski 8ae51ff9c1 stdenv/generic/check-meta: fix error message evaluation 2017-12-05 13:46:52 +01:00
John Ericson 405412dfd9
Merge pull request #31775 from obsidiansystems/stdenv-both-propagated-files
stdenv setup: Always use both propagated files
2017-11-22 15:23:37 -05:00
John Ericson da19c34d0f stdenv setup: Always use both propagated files
This continues #23374, which always kept around both attributes, by
always including both propagated files: `propgated-native-build-inputs`
and `propagated-build-inputs`. `nativePkgs` and `crossPkgs` are still
defined as before, however, so this change should only barely
observable.

This is an incremental step to fully keeping the dependencies separate
in all cases.
2017-11-21 10:44:44 -05:00
Orivej Desh d99a2fc093 Merge branch 'master' into staging
* master: (293 commits)
  go_1_9: skip flaky TestServerCancelsReadTimeoutWhenIdle
  qsyncthingtray: fix build
  qt56.qtwebengine: fix build
  stdman: d860212 -> 2017.04.02
  jackett: use mono50
  hg-git: disable with python3
  hg-git: 0.8.5 -> 0.8.10
  xfce4-settings: enable parallel building
  gcc-snapshot: mark as broken
  heaptrack: 2017-02-14 -> 2017-10-30
  nixos-container: Modify existing test to cover show-ip command
  nixos-container: Make show-ip work together with ipv4 + netmask
  linux-copperhead: 4.13.12.a -> 4.13.13.a
  matterbridge: 1.1.0 -> 1.4.1
  nixos/nghttpx: add module for the nghttpx proxy server (#31680)
  mattermost: 4.3.0 -> 4.4.0
  breakpad: delete
  simp_le: 0.2.0 -> 0.6.1
  certbot: 0.11.1 -> 0.19.0
  afl: 2.51b -> 2.52b
  ...
2017-11-17 05:35:09 +00:00
John Ericson b26038fa54
Merge pull request #31723 from obsidiansystems/stdenv-accum-them-setup
stdenv setup: Run setup hooks and other processing after accumulating deps
2017-11-16 10:42:39 -05:00
John Ericson 6a5cda5131 stdenv setup: Run setup hooks and other processing after accumulating deps
I find the separation of concerns, accumulating, then processing, easier
to follow. Also, with my yet-to-be-merged cross work, the accumulation
part will become more complex.
2017-11-15 18:51:06 -05:00
Vladimír Čunát 72b98b8b60
darwin.stdenv: fix a typo in comment ;-) 2017-11-14 21:13:53 +01:00
John Ericson 728446f755
Merge pull request #30484 from obsidiansystems/libbfd
bfd, opcodes: Init separate derivations for binutils libraries
2017-11-14 12:10:44 -05:00
Dan Peebles 49a5791bc4 darwin.stdenv: fix up to allow sandboxing 2017-11-14 10:36:04 -05:00
Dan Peebles f82063ed20 darwin.stdenv: use new, purer, bootstrap tools
This is the first step to getting a sandboxable nixpkgs
2017-11-13 21:03:45 -05:00
Dan Peebles d5bdfcbfe6 darwin.make-bootstrap-tools: fix ICU reference to improve purity 2017-11-13 20:27:11 -05:00
Michael Raskin bd8cd3eaed Revert switching hello to fetchipfs, re: #18296 2017-11-13 17:45:16 +01:00
John Ericson ff023c9bc6 treewide: Remove references to removed binutils outputs 2017-11-13 08:46:15 -05:00
Vladimír Čunát f6655ba128
bootstrap tools test: fixup after d71833ee36 2017-11-11 21:41:42 +01:00
Vladimír Čunát 1862b77b5d
Merge branch 'staging' into gcc-7 2017-11-11 20:33:50 +01:00
John Ericson 4d4f94cde4 treewide: Depend on targetPackages.stdenv.cc.bintools instead of binutils directly
One should do this when needed executables at run time. It is more
honest and cross-friendly than refering to binutils directly, if one
neeeds the default binary tools for the target platform, rather than
binutils in particular.
2017-11-05 17:10:53 -05:00
John Ericson 5ae8f18f4d Rename __targetPackages to targetPackages 2017-11-05 17:10:53 -05:00
Dan Peebles b426c85ce2 Get rid of most @rpath nonsense on Darwin
This requires some small changes in the stdenv, then working around the
weird choice LLVM made to hardcode @rpath in its install name, and then
lets us remove a ton of annoying workaround hacks in many of our Go
packages. With any luck this will mean less hackery going forward.
2017-10-08 16:13:46 -04:00
John Ericson f037625f87 Merge remote-tracking branch 'upstream/staging' into deps-reorg 2017-09-28 12:32:57 -04:00
John Ericson d349f9a340 cc-wrapper: Use stdenvNoCC to build
cc-wrapper may wrap a cc-compiler, but it doesn't need one to build
itself. (c.f. expand-response-params is a separate derivation.) This
helps avoid cycles on the cross stuff, in addition to removing a
useless dependency edge.

I could have been super careful with overrides in the stdenv to avoid
the mass rebuild, but I don't think it's worth it.
2017-09-26 14:08:21 -04:00
John Ericson f6fcb9bc0d stdenv: Turn on set -x if NIX_DEBUG >= 6
Why 6? It seems a decently high number, giving us room for more degrees
of debugging before the `set -x` sledgehammer without incurring a
mass-rebuild.
2017-09-26 11:24:19 -04:00
John Ericson 127a5f3357 treewide: Use (( "${NIX_DEBUG:-0}" >= 1) )) consistently 2017-09-26 11:24:19 -04:00
Vladimír Čunát e8bd4102c6
Merge branch 'master' into gcc-7 2017-09-25 12:37:31 +02:00
John Ericson 7a733ceda4 cross stdenv: No more native-fishing hack.
The comment says it was put there for pkgconfig, but now pkgconfig has
been dealt with.
2017-09-21 15:49:54 -04:00
John Ericson b9bf90ca6c all-packages: Remove gccCrossStageFinal; any gcc will not work 2017-09-21 15:49:18 -04:00
John Ericson 3af3d6efc2 stdenv cross adapater: Remove old extraBuildInputs
They, unlike their native counterparts, run on the wrong platform and
are therefore invalid.
2017-09-21 14:52:42 -04:00
John Ericson 05ef1034c7 stdenv cross adaptor: Add optional overrides parameter
By default, all previous overrides are discarded as before, as they
would only apply to the old host platform. But sometimes it is useful to
add some new ones, and this optional parameter allows that.
2017-09-21 14:52:07 -04:00
John Ericson 738bb4777c stdenv: Update autotools scripts on all Aarch64, not just Linux
This is needed when cross-compiling for iOS (Aarch64 + Darwin). I also
changed the syntax of the Linux stdenv for visual consistency, though
that has no effect on semantics as the os is already guaranteed to be
Linux.
2017-09-21 14:40:40 -04:00
John Ericson 51179c6e88 stdenv: Add back leading '-' in cross derivation name suffix
I messed this up when I moved the logic from `makeStdenvCross` in
e826a6a247.
2017-09-18 18:59:48 -04:00
John Ericson 77bd6313bb darwin stdenv: cctools override needs to go away when targetPlatform changes 2017-09-18 14:39:38 -04:00
John Ericson bf7b521500 treewide: Fix some references to binutils
- Compiler's shouldn't use `binutils.dev` as that doesn't yet exist
   with a cross binutils.

 - Last two `binutils.binutils` which weren't reverted
2017-09-14 18:12:54 -04:00
Vladimír Čunát c86eb1da5f
stdenv bootstrap: fix evaluation
This is probably a fallout from #28557 merge and revert.
I can't see why exactly this happened, but it seems a safe fix.
2017-09-10 11:16:47 +02:00
Eelco Dolstra 0061fae2e6 genericBuild: Communicate the current build phase to Nix
This allows the progress bar to show e.g.

  [1/9/59 built] building bison-3.0.4 (configurePhase): checking for strdup... yes
2017-09-07 22:24:27 +02:00
Eelco Dolstra 6b3cef2246 Remove tracePhases
This has not been used in a long time.
2017-09-07 22:15:37 +02:00
Eelco Dolstra ec8d41f08c
Revert "Merge pull request #28557 from obsidiansystems/binutils-wrapper"
This reverts commit 0a944b345e, reversing
changes made to 61733ed6cc.

I dislike these massive stdenv changes with unclear motivation,
especially when they involve gratuitous mass renames like NIX_CC ->
NIX_BINUTILS. The previous such rename (NIX_GCC -> NIX_CC) caused
months of pain, so let's not do that again.
2017-09-07 12:51:21 +02:00
John Ericson 9a1b7cf2ae top-level, linux stdenv make boootstrap tools: Fix eval
Now the NixOS tarball job succeeds again
2017-09-04 11:26:41 -04:00
John Ericson 0a944b345e Merge pull request #28557 from obsidiansystems/binutils-wrapper
Binutils-wrapper: Init by refactoring out of cc-wrapper
2017-09-03 10:37:27 -04:00
Orivej Desh f4044c1ccc stdenv-setup: list environment variables with awk 2017-09-03 12:57:08 +00:00
Orivej Desh a09d9e7cd4 stdenv-setup: fix substituteAll with set -eu
Environment variable filter in substituteAll was not precise and produced
undefined and invalid variable names.  Vladimír Čunát tried to fix that in [1],
but `env -0` did not work during Darwin bootstrap, so [2] reverted this change
and replaced an error due to invalid variables with a warning.  Recently in #28057
John Ericson added `set -u` to `setup.sh` and undefined variables made the setup
fail during e.g. `nix-build -A gnat` with `setup: line 519: !varName: unbound
variable`.

[1] 62fc8859c1
[2] 81df035429
2017-09-03 12:57:08 +00:00
Orivej Desh 447240b19f mkDerivation: explain "all" in hardeningDisable
https://github.com/NixOS/nixpkgs/pull/28806#discussion_r136516276
2017-09-03 12:57:08 +00:00
John Ericson dbf6d20d64 binutils-wrapper: Import separately from cc-wrapper 2017-09-01 11:44:56 -04:00
Orivej Desh d70006c6d9 mkDerivation: fix hardening flags check
- allow "all" in hardeningDisable
- fix busybox flags
- print detailed error message

Discussed at https://github.com/NixOS/nixpkgs/pull/28555#issuecomment-326413032
2017-09-01 01:01:24 +00:00
John Ericson 8dbdc5dd51 darwin-stdenv: Hack around impurity with --disable configure flag 2017-08-31 18:08:05 -04:00
John Ericson 3b6e7fe123 darwin-stdenv: Don't use nativeTools
Now, we'll actually use the wrapped ld to link
2017-08-31 12:37:48 -04:00
John Ericson 97a48835b7 mkDerivation, cc-wrapper: Check hardening flag validity in Nix
This becomes necessary if more wrappers besides cc-wrapper start
supporting hardening flags. Also good to make the warning into an
error.

Also ensure interface is being used right: Not as a string, not just in
bash.
2017-08-30 17:53:42 +02:00
Daiderd Jordan 92652b4d79
darwin-CF: use @rpath for library id and add an rpath entry for CF based on NIX_COREFOUNDATION_RPATH 2017-08-28 23:24:58 +02:00
John Ericson a470be5a16 expand-response-params: Build more normally 2017-08-25 15:10:02 -04:00
John Ericson 2e7a390212 Merge pull request #28057 from obsidiansystems/stdenv-set-u
stdenv-setup: use `set -u`
2017-08-25 11:19:58 -04:00
Vladimír Čunát 8137a8cb73
gawk: refactor
- Don't build with libsigsegv by default.  The build apparently attempted
  to link against it, but it never retained the reference anyway...
- Side effect: stdenv bootstrapping needs no libsigsegv anymore.
- Run checks, but only in the interactive gawk by default on Linux,
  so that stdenv bootstrap isn't slowed down (by glibc locales, etc.).
- xz should be no longer needed in inputs, as we have it in stdenvs now.

The whole change was triggered by some used kernel versions still
breaking libsigsegv tests #28464.
2017-08-24 11:06:53 +02:00
Tuomas Tynkkynen 0c0fad6141 treewide: Consistently call ARM 'arm'
No need for silly differences.
2017-08-24 01:17:01 +03:00
John Ericson 81194eef45 stdenv-setup: Use set -u as much as possible
Older bash version, like those in the bootstrap tools and on macOS,
currently confuse variables defined as an empty array with undefined
variables. `${foo+"${foo[@]}"}` will prevent `set -u` problems with
empty arrays and older without making a single '' in the empty case.

Care is taken to `set +u` when running hooks so as to not break existing
packages.
2017-08-23 15:57:56 -04:00
John Ericson 0135e61b4c Merge remote-tracking branch 'upstream/master' into staging
That way the tarball job succeeds
2017-08-21 18:42:57 -04:00
John Ericson c035711072 cc-wrapper: Remove unused params
Ensured hashes unchanged and eval succeeds in tarball job
2017-08-21 18:40:41 -04:00
Frederik Rietdijk 6bbc3a0b24 Merge commit '3b29468313bc8604fe8f85c8d9316fd276d3985c' into HEAD 2017-08-21 04:44:40 +02:00
Frederik Rietdijk 04cd1db2b7 Merge remote-tracking branch 'upstream/master' into HEAD 2017-08-21 01:28:27 +02:00
John Ericson fbab1d485b stdenvs: Distinguish between extraBuildInputs and extraNativeBuildInputs
This version continues to use bash + stdenv/setup for the default
inputs.
2017-08-18 12:02:13 -04:00
Tuomas Tynkkynen 7320fa9d45 Revert "stdenvs: Distinguish between extraBuildInputs and extraNativeBuildInputs"
This reverts commit eeabf85780.

This change suddenly makes tons of stdenv internals visible in
nativeBuildInputs of every derivation, which doesn't seem desirable.
E.g:

````
nix-repl> hello.nativeBuildInputs
[ «derivation /nix/store/bcfkyf6bhssxd2vzwgzmsbn7b5b9rpxc-patchelf-0.9.drv»
  «derivation /nix/store/4wnshnz9wwanpfzcrdd76rri7pyqn9sk-paxctl-0.9.drv»
  << snip 10+ lines >>
  «derivation /nix/store/d35pgh1lcg5nm0x28d899pxj30b8c9b2-gcc-wrapper-6.4.0.drv»
]
````
2017-08-18 13:21:56 +03:00
Daiderd Jordan 6a870a59f8 Merge pull request #28174 from matthewbauer/darwin-in-release
enable hydra jobs for packages x86_64-linux does not support
2017-08-17 23:29:00 +02:00
Frederik Rietdijk 8f2ea38f8f Merge remote-tracking branch 'upstream/master' into HEAD 2017-08-17 18:34:17 +02:00
John Ericson 61d241f405 darwin stdenv: Float persistentN bindings into per-stage lets 2017-08-16 16:36:21 -04:00
John Ericson eeabf85780 stdenvs: Distinguish between extraBuildInputs and extraNativeBuildInputs
Additionally, instead of pulling them from `setup.sh`, route them via
Nix. This gets us one step closer to making stdenv be a plain attribute
set instead of a derivation.
2017-08-15 18:24:54 -04:00
John Ericson a71cf06b16 mkDerivation: Simply Nix
No hashes were changed by this
2017-08-15 16:13:30 -04:00
Vladimír Čunát a1bb61312c
stdenv: fixup allowedRequisites on aarch64-linux 2017-08-15 08:09:26 +02:00
Will Dietz 3e8d68e514 make-bootstrap-tools: Fix config opts for ash builtins after upgrade 2017-08-14 22:40:11 +03:00
Vladimír Čunát 505e94256e
stdenv: resurrect the allowedRequisites check
Discovered in #28091.  I'm sorry I forgot to re-check my TODOs, long ago.
2017-08-13 11:44:36 +02:00
Tuomas Tynkkynen 3e9f76774a
nixpkgs release: Fix Darwin-only jobs
Currently the logic of generating nixpkgs Hydra jobs is to walk through
the pkgs evaluated for system = "x86_64-linux", collect any derivations
and their meta.platforms values. However, that doesn't work for
packages whose meta.platforms doesn't include x86_64-linux, as just
evaluating their meta attribute raises an error so they get skipped
completely.

As a less-intrusive fix (i.e. anything than rewriting the current package
enumeration logic), allow passing `config.allowUnsupportedSystem = true`
to permit evaluating packages regardless of their platform and use that
in the package listing phase.

Fixes #25200
2017-08-12 20:38:27 -07:00
Linus Heckemann 17753fa005 stdenv: fix typo in setup.sh 2017-08-09 17:33:02 +01:00
John Ericson 42f35503b5 cc-wrapper: Make hygienic
See the added comments for what exactly has been done.
2017-08-07 03:05:50 -04:00
John Ericson 8ba6012dfd Merge PR #27536 2017-08-03 17:25:30 -04:00
John Ericson 0c37778c2c cc-wrapper: WIP linking hack for mac OS
Probably best to override Haskell packages set, or anything else
linking a lot of libraries, with this.
2017-07-31 17:02:56 -04:00
John Ericson 9be40841ea Merge remote-tracking branch 'upstream/master' into staging-base
Conflicts:
	pkgs/build-support/cc-wrapper/default.nix
	pkgs/build-support/gcc-wrapper-old/builder.sh
	pkgs/build-support/trivial-builders.nix
	pkgs/desktops/kde-4.14/kde-package/default.nix
	pkgs/development/compilers/openjdk-darwin/8.nix
	pkgs/development/compilers/openjdk-darwin/default.nix
	pkgs/development/compilers/openjdk/7.nix
	pkgs/development/compilers/openjdk/8.nix
	pkgs/development/compilers/oraclejdk/jdk-linux-base.nix
	pkgs/development/compilers/zulu/default.nix
	pkgs/development/haskell-modules/generic-builder.nix
	pkgs/misc/misc.nix
	pkgs/stdenv/generic/builder.sh
	pkgs/stdenv/generic/setup.sh
2017-07-26 13:46:04 -04:00
John Ericson 820e4021d3 stdenv-setup: Remove any declare -g
This is invalid before bash-4.2, affecting bash used impurely in
nix-shell on MacOS.
2017-07-26 09:11:18 -04:00
John Ericson ea7d13cf1a stdenv-setup and misc hooks: Work with bash-3.4 for MacOS nix-shell
This is a temporary measure until this impurity is removed from Nix.
2017-07-26 09:08:01 -04:00
John Ericson f6f40e3fe5 stdenv-setup and misc pkgs: Revert to space-deliminated propagated-* files
We cannot switch to line-delimited yet, because certain Nix commands do
not read in the entire file, but just the first line.
2017-07-26 09:07:55 -04:00
John Ericson 34c0ba498c stdenv-setup: Add quotes that don't do anything for consistency.
@vcunat and others rightly point out that it's easier to quote always,
than learn Bash's idiosyncrasies enough to know when it doesn't make a
difference.

This reverts commit 2743078f66, which
removes quotes that don't do anything, and then goes further adding
even more quotes.
2017-07-25 14:36:00 -04:00
John Ericson 98cff3f446 darwin stdenv: Ensure libSystem reexports the right libraries
The logic was made pure for the normal libSystem, but this change never
made it to the bootstrap tools. Deduplication the logic as the comment
suggests would have prevented this, but here's a stop-gap until we do
so.
2017-07-25 14:35:43 -04:00
Eelco Dolstra ca8aa5dc87
Use GCC 7 by default 2017-07-25 13:56:57 +02:00
Dan Peebles 2829ea57cb stdenv/setup.sh: undo local -n change
It's better than the eval solution this is adding back, but until we can
rely on a particular version of bash in nix-shell, this just breaks too
much stuff.

See c94f3d5575
and https://github.com/NixOS/nix/pull/1483 for the better long-term
solution.
2017-07-24 13:09:32 -04:00
Eelco Dolstra 6669a3b477
stdenv: Remove log nesting
Nix/Hydra no longer support pretty printing of logs, so this is no
longer useful.
2017-07-24 14:30:46 +02:00
John Ericson b087618ac0 Revert "stdenv: Store one package per line in nix-support/propagated-*"
As @oxij points out in [1], this breakage is especially serious because
it changes the contents of built environments without a corresonding
change in their hashes. Also, the revert is easier than I thought.

This reverts commit 3cb745d5a6.

[1]: https://github.com/NixOS/nixpkgs/pull/27427#issuecomment-317293040
2017-07-24 01:05:30 -04:00
Daniel Peebles bd2e91e3a2 Merge pull request #27318 from copumpkin/darwin-high-sierra
Support High Sierra on Darwin
2017-07-18 17:06:06 -04:00
John Ericson aaaa470ff8 mkDerivation: Fix errors from #27365
`nix-build pkgs/top-level/release.nix -A tarball` now succeeds.

`configureFlags = null` lead to a type error, and one overrideDrv
needed to be converted to to append a configureFlags list instead of
string due to the normalization.

Thanks @vcunat for alerting me to the issues---sorry I did not catch
them before merging my own PR.
2017-07-15 13:47:21 -04:00
John Ericson e826a6a247 stdenv: Move some logic from cross adapter to stdenv proper
Eventually the adapter will be removed.  Moved is

 - Name suffix from hostPlatform

 - configurePlatforms

   To not cause more breakage, the default is currently [], but
   eventually it will be [ "build" "host" ], as the cross adapter makes
   it today.
2017-07-13 19:05:36 -04:00
John Ericson 30a1420414 stdenv-setup: Pull out and explain 3-part printing of commands
@Dezgeg made the good point that the reasons for doing this were not at
all intuitive.
2017-07-13 16:31:39 -04:00
John Ericson 2743078f66 stdenv-setup: Remove useless quotes
foo=$1 surprisingly doesn't need quotes in Bash. Word splits are only
syntactic in string variable (not array var!) assignments.
2017-07-13 14:59:53 -04:00
John Ericson 273a4c1c78 stdenv-setup: Combine [[ .. ]] && [[ .. ]] into one [[ .. && .. ]]
Also remove useless quotes on same line
2017-07-13 14:57:16 -04:00
John Ericson 5d693c84d2 stdenv-setup: Clean up 'substitute()' for style and error handling
It now blows up on null byte in file (rather than silently truncating),
and invalid arguments (rather than silently skipping).
2017-07-12 17:47:20 -04:00
John Ericson 5d4efb2c81 stdenv-setup: Misc improvements as directed by ShellCheck
I took some liberties with the flags-echoing code to make it more
concise and correct. Also, a few warnings in findInputs and friends I
skipped because I am going to rewrite those anyways.

Thanks @grahamc for telling me about this great linter!
2017-07-12 15:31:10 -04:00
John Ericson 8d76effc17 stdenv-setup: Make the package accumulators associative arrays instead of strings
This is generally cleaner: less eval, less worrying about separators,
and probably also faster. I got the idea from that python wrapper
script.
2017-07-12 15:30:56 -04:00
Dan Peebles 0419452113 Fix Darwin stdenv to work on 10.13
The main changes are in libSystem, which lost the coretls component in 10.13
and some hardening changes that quietly crash any program that uses %n in
a non-constant format string, so we've needed to patch a lot of programs that
use gnulib.
2017-07-11 21:56:38 -04:00
John Ericson 3cb745d5a6 stdenv: Store one package per line in nix-support/propagated-*
This makes those files a bit easier to read. Also, for what it's worth,
it brings us one baby step closer to handling spaces in store paths.

Also, I optimized handling of many transitive deps with read. Probably,
not very beneficial, but nice to enforce the pkg-per-line structure.
Doing so let me find much dubious code and fix it.

Two misc notes:

 - `propagated-user-env-packages` also needed to be adjusted as
   sometimes it is copied to/from the propagated input files.

 - `local fd` should ensure that file descriptors aren't clobbered
   during recursion.
2017-07-10 13:32:13 -04:00
John Ericson 5896d84dbb stdenv: Stop reversing the list of sandbox stuff
We're breaking hashes anyways
2017-07-10 11:25:51 -04:00
John Ericson 2f198956c7 stdenv: Make separate-debug-info.sh a nativeBuildInput 2017-07-10 11:25:51 -04:00
Vladimír Čunát bfb7ef86f3
Merge branch 'master' into staging
Mass rebuilds incoming.  The mass-rebuild situation got really messy
this weekend.
2017-07-09 18:07:52 +02:00
Daiderd Jordan 980346592c
Merge branch 'staging' into master 2017-07-08 22:22:17 +02:00
John Ericson a302d7360f top-level: {build,host,target}Platform are defined in the stdenv instead
See #27069 for a discussion of this
2017-07-07 12:55:02 -04:00
John Ericson f5364122b5 Merge pull request #27215 from obsidiansystems/stdenv-harden
Harden stdenv in two misc ways
2017-07-07 12:48:51 -04:00
John Ericson afc2023993 stdenv: Have mkDerivation pull the "extra" arguments from stdenv instead
Something more elaborate is needed for the "*Platform" arguments.
2017-07-07 12:16:51 -04:00
John Ericson 4cf4d7180d stdenv: Conservatively move mkDerivation into it's own file 2017-07-07 12:16:51 -04:00
Vladimír Čunát e8e57452f4 stdenv: separate all meta-checking code (~200 lines)
Only cosmetic changes are done otherwise.
Real refactoring is left for later.

There's a small slow-down on my machine:
$ time nix-env -qa -P >/dev/null
gets from ~2.8 to ~3.5 seconds (negligible change in RAM).
That's most likely caused by sharing less computation between different
mkDerivation calls, and I plan to improve that soon.
2017-07-07 12:16:26 -04:00
Vladimír Čunát dfc004e69c lib.lists.mutuallyExclusive: add function 2017-07-07 12:02:29 -04:00
Vladimír Čunát 5afcdc88fa stdenv: simple refactor to get rid of pos'
Suggested by Ericson2314.
2017-07-07 12:02:29 -04:00
Vladimír Čunát 7fdf18e892 stdenv: refactor (no change in semantics)
This just moves some expressions around in preparation to further changes.
2017-07-07 12:02:29 -04:00
John Ericson e57a220f81 stdenv, swift: Use local fd in is* bash functions for hygiene 2017-07-07 11:40:07 -04:00
John Ericson a14cf06182 stdenv: Harden hook runners
Instead of eval, use a "nameref" to get the name of the array and
iterate with that. Also, make the for-loop parameter a local variable,
too.
2017-07-07 11:35:09 -04:00
Ryan Trinkle 7004641566 Merge pull request #26974 from obsidiansystems/response-file-parsing-speed
cc-wrapper: improve response file parsing speed
2017-07-05 16:18:22 -04:00
Vladimír Čunát 5328aac7be
Merge branch 'staging'
Comparison looks OK; I'll try some fixes on master directly.
http://hydra.nixos.org/eval/1372577?compare=1372497
2017-07-05 08:55:26 +02:00
Tim Steinbach d788244423
stdenv / booter: imap -> imap1
Fix the usage of imap as a follow-up to #25543
2017-07-04 21:04:13 -04:00
Orivej Desh 2bc7b4e134 cc-wrapper: simplify expandResponseParams parser
Import from b2446902fe
2017-07-03 21:51:23 +00:00
Ryan Trinkle d07f30f628 cc-wrapper: improve response file parsing speed 2017-06-30 15:20:53 -04:00
John Ericson d61c22341b Merge accepted cross compilation PRs into staging 2017-06-29 18:07:13 -04:00
John Ericson ad8d8fb2f5 stdenv: Simplify dependency code
This is a bit simpler now, but more importantly it scales better when I
double the number of sorts of dependencies as part of my cross
compilation work.
2017-06-29 17:45:08 -04:00
Daiderd Jordan d76bc8ed15 Merge pull request #26818 from LnL7/llvm-manpages
llvm-packages: get rid of extra build depedencies for manpages
2017-06-26 21:40:55 +02:00
Daiderd Jordan f65b5365bb
darwin-stdenv: remove llvm/clang manpage overrides from 2017-06-26 20:24:17 +02:00
David McFarland f81117e324 stdenv: remove leftover 'system' reference
the rest were removed in 1dc6f15de9
2017-06-26 09:33:28 -03:00
David McFarland cdc5cf52c1 cygwin: rebase fixes
- use fixupOutputsHook to find libs in all outputs
- don't rebase symlinks
2017-06-26 09:26:10 -03:00
John Ericson e0492d58a6 cross stdenv adaptor: Disable checkPhase by default
Before gcc-cross-wrapper did this unconditionally
2017-06-22 17:52:28 -04:00
John Ericson 594d264205 cross stdenv adaptor: Support --host --build --target across the board
Packages get --host and --target by default, but can explicitly request
any subset to be passed as needed. See docs for more info.

rustc: Avoid hash breakage by using the old (ignored)
dontSetConfigureCross when not cross building
2017-06-22 17:52:28 -04:00
John Ericson 9bfd03eff7 cross stdenv adaptor: Remove ccCross binutils attrs and binutils extra buildDepends
It now has the correct wrapped tools and nothing else is needed.
2017-06-22 17:52:28 -04:00
John Ericson 459f1c60f5 cc-wrapper: Learn about target prefixes
This is first step towards getting rid of gcc-wrapper-cross
2017-06-22 17:52:27 -04:00
John Ericson c4ba2e3ef6 cc-wrapper: Remove stdenv.is* for targetPlatform.is*
Modify bootstrapping stdenvs to make sure `targetPlatform` also passed.
2017-06-22 17:52:27 -04:00
Daiderd Jordan 9e2aba3572
darwin-stdenv: remove python-sphinx and a bunch of other dependencies from the stdenv 2017-06-01 23:23:56 +02:00
Vladimír Čunát 538aa0f808
Merge branch 'staging' (early part)
The comparison looks nice on Hydra.
2017-05-30 16:08:21 +02:00
John Ericson 20e756a093 lib: Consolidate platform configurations (used for crossSystem)
This is good for maintenance and education.
2017-05-29 18:56:03 -04:00
Daiderd Jordan db4ff1d305 Merge pull request #26091 from LnL7/darwin-cf-10.10
CoreFounation: 10.9 -> 10.10
2017-05-28 20:38:16 +02:00
Daiderd Jordan 779ec14402 Merge pull request #25921 from dtzWill/feature/llvm-manpages
llvm-4/clang-4: Build and install man pages
2017-05-28 20:29:21 +02:00
Daiderd Jordan 077840e528
darwin-stdenv: use darwin.ICU instread of icu 2017-05-27 23:19:49 +02:00
Vladimír Čunát 8004e79415
Merge branch 'master' into staging 2017-05-24 03:24:06 +02:00
John Ericson eaa509f33a stdenv: Rename isGNU to isHurd as GNU is a userland
Elsewhere, things called GNU indeed includes GNU/Linux or GNU/Hurd, but this
predicate was defined excluding Linux regardless of userland.
2017-05-22 13:55:26 -04:00
Frederik Rietdijk 99cfab07b9 Merge remote-tracking branch 'upstream/master' into HEAD 2017-05-22 09:21:40 +02:00
John Ericson 1dc6f15de9 stdenv: define is* predicates with hostPlatform.is*
This is a saner default until stdenv's are removed altogether
2017-05-22 00:25:02 -04:00
John Ericson c5c6606048 lib: Infer libc field of platform if not specified
This is especially useful when not cross compiling. It means we can
remove the `stdenv.isGlibc` predicate too.

Additionally, use this to simplify the logic to choose the
appropriate libiconv derivation.
2017-05-22 00:25:02 -04:00
Daiderd Jordan 1e23b49f46
darwin-stdenv: allow clang-unwrapped.man 2017-05-21 12:40:07 +02:00
John Ericson 7e096024d7 glibc: Fix for cross 2017-05-19 18:44:24 -04:00
Vladimír Čunát e9aeb55f3b
Merge branch 'master' into staging 2017-05-18 11:24:18 +02:00
John Ericson bfa8f30499 linux cross stdenv: Use the cross stdenv and nativeBuildInputs
This is a cross derivation---it's built on one platform to run on
another---so let's structure it like all the other cross derivations.
2017-05-17 15:33:05 -04:00
John Ericson 5870795cc6 linux cross stdenv: Fix eval errors
`pkgsNoParams` was removed by me, but then #25035 was merged using it,
leading to an unbound identifier.

It would be nice to get travis to do build release-cross.nix or
something to catch these things.
2017-05-17 13:42:51 -04:00
Vladimír Čunát fa5d543770
Merge branch 'master' into staging 2017-05-06 13:24:28 +02:00
Vladimír Čunát ef5844be6c
stdenv: disable audit-tmpdir on non-Linux for now
Without changing any hashes.
2017-05-06 13:19:07 +02:00
Dan Peebles c217f59344 darwin.make-bootstrap-tools.test: fix build breakage
In the extremely unlikely case that our store hash path ends in several
digits (as is the case right now), the Darwin ld will try to interpret
those digits as a version number and barf. To avoid that, we pass in the
SDK version explicitly to stop it from trying to figure it out from iffy
context.
2017-05-05 21:34:28 -04:00
Eelco Dolstra 94d164dd7f
Add a setup hook for detecting $TMPDIR references in RPATHs and wrapper scripts 2017-05-04 20:23:57 +02:00
Domen Kožar e057e5927e Merge pull request #25427 from aneeshusa/fix-meta-priority-types
Fix meta priority types
2017-05-02 09:38:32 +02:00
Aneesh Agrawal d3acf9891c stdenv: More useful error message on bad meta attrs
This helps in debugging meta attribute type errors,
which are now enforced as of commit
90b9719f4f.
2017-05-02 01:45:30 -04:00
Dan Peebles f3a05a0fb3 stdenv: disable checkMeta by default until issues resolved
https://github.com/NixOS/nixpkgs/pull/25304#issuecomment-298385426
2017-05-01 13:51:12 -04:00
Michael Raskin bcbafdefc4 Merge pull request #25035 from elitak/cross-staging
Add some ARM platforms
2017-05-01 13:30:32 +02:00
Eric Sagnes 7004919243 stdenv-generic: add meta attributes checks 2017-04-29 17:07:01 +09:00
Dan Peebles 90b9719f4f treewide: fix the remaining issues with meta attributes 2017-04-29 04:24:34 +00:00
Dan Peebles 1a4ca220e1 treewide: fix assorted issues revealed by the meta checker
Turns out a couple of the licenses were wrong, as well as being strings.
2017-04-28 23:07:42 -04:00
Dan Peebles 32ae4bfc20 stdenv-generic: add meta attribute checking
This is turned off by default but I think we should fix all packages to
respect it and then turn it on by default
2017-04-28 18:12:18 -04:00
John Ericson 761af14778 Merge pull request #25227 from obsidiansystems/cross-purge-binutilsCross
Purge binutilsCross
2017-04-26 09:09:06 -04:00
John Ericson 295315cc87 binutilsCross: Remove and use binutils instead always
See previous commit for what was done to `binutils` to make this
possible.

There were some uses of `forcedNativePackages` added. The
combination of overrides with that attribute is highly spooky: it's
often important that if an overridden package comes from it, the
replaced arguments for that package come from it. Long term this
package set and all the spookiness should be gone and irrelevant:

  "Move along, nothing to see here!"

No hashes should be changed with this commit
2017-04-25 21:36:19 -04:00
John Ericson 85b4d30c0b binutils: Respect the targetPlatform
Use `buildPackages.binutils` to get build = host != target binutils,
i.e. the old `binutilsCross`, and use
`buildPackages.buildPackages.binutils` to get build = host = target
binutils, i.e. the old `binutils`.

`buildPackages` chains like this are supposed to remove the need for
all such `*Cross` derivations. We start with binutils because it's
comparatively easy.

No hashes of cross-tests should be changed
2017-04-25 21:31:50 -04:00
John Ericson 6cb0f0bcd9 Merge pull request #25225 from Ericson2314/linux-cross-stdenv-eval
linux cross stdenv: Pull platforms from lib to cut eval time
2017-04-25 20:01:46 -04:00
John Ericson 78bb5f5f37 linux cross stdenv: Pull platforms from lib to cut eval time 2017-04-25 19:57:05 -04:00
John Ericson 75441dd64a Merge pull request #25194 from obsidiansystems/host-target-unconfuse
stdenv.cross is a silly attribute that needs to go leaving the well-defined hostPlatform and targetPlatform. This PR doesn't remove it, but changes its definition: before it tracked the target platform which is sometimes more useful for compilers, and now it tracks the host platform which is more useful for everything else. Most usages are libraries, falling in the "everything else" category, so changing the definition makes sense to appease the majority. The few compiler (gcc in particular) uses that exist I remove to use targetPlatform --- preserving correctness and becoming more explicit in the process.

I would also update the documentation aside mentioning stdenv.cross as deprecated, but the definition given actually erroneously assumes this PR is already merged!
2017-04-24 19:07:48 -04:00
John Ericson af6e4c5b0d Merge pull request #25190 from obsidiansystems/ios-stdenv-tiny-refactor
crossStdenv on iphone: Just get info from `targetPlatform`
2017-04-24 16:57:54 -04:00
John Ericson 49c99b70cf cross-stdenv: Only prune most overrides in the final stage
Before all overrides were also pruned in the previous stage, now
only gcc and binutils are, because they alone care about about the
target platform. The rest of the overrides don't, so it's better to
preserve them in order to avoid spurious rebuilds.
2017-04-24 16:31:53 -04:00
John Ericson db5a921945 crossStdenv on iphone: Just get info from targetPlatform 2017-04-24 16:12:26 -04:00
Tuomas Tynkkynen 84982c28de Merge remote-tracking branch 'upstream/master' into staging 2017-04-24 15:04:43 +03:00
John Ericson 863d79b364 top-level: Introduce targetPackages and a "double link fold"
Each bootstrapping stage ought to just depend on the previous stage, but
poorly-written compilers break this elegence. This provides an easy-enough
way to depend on the next stage: targetPackages. PLEASE DO NOT USE IT
UNLESS YOU MUST!

I'm hoping someday in a pleasant future I can revert this commit :)
2017-04-23 14:01:12 -04:00
Eric Litak c3eca1f8dc platforms: add pogoplug4 (armv5tel softfloat) 2017-04-19 14:09:56 -07:00
Eric Litak 3b7395683c platforms: add scaleway-c1 (armv7 sans NEON) 2017-04-19 14:09:56 -07:00
Frederik Rietdijk e0abe74baf Merge remote-tracking branch 'upstream/master' into HEAD 2017-04-18 11:25:43 +02:00
Tuomas Tynkkynen 28f87e4141 stdenv: ARM bootstrap: Update bootstrap tarballs to latest hydra-built ones
Fixes sandboxed build of glibc. Discussion about a similar failure on aarch64 at:
8bfa9f528c.

Picked from the following cross-trunk evaluation:
http://hydra.nixos.org/eval/1349278 based on nixpkgs
commit 1f32d4b4eb.

armv5tel job: http://hydra.nixos.org/build/51569718
armv6l job: http://hydra.nixos.org/build/51569717
armv7l job: http://hydra.nixos.org/build/51569713
2017-04-15 01:52:33 +03:00
Vladimír Čunát f7a4f146c9
Merge branch 'master' into staging
This includes a fix for a bad merge.
2017-04-14 19:22:02 +02:00
Vladimír Čunát f3ceb764e4
Merge #23374: mkDerivation: simplify non-cross builds
Don't pass buildInputs to stdenv builder in nativeBuildInputs.
2017-04-14 11:01:10 +02:00
Tuomas Tynkkynen 1f32d4b4eb make-bootstrap-tools.nix: Fix bzip2
Apparently our native bzip2 builds switched to using dynamic libraries at some point.
2017-04-13 17:22:55 +03:00
Dan Peebles 72d9016b8b darwin.make-bootstrap-tools: fix to use LLVM 4
This should now roughly match the bootstrap tools we're using on Darwin
2017-04-08 16:38:48 -04:00
Dan Peebles b9e558597d stdenv-darwin: bump to use LLVM 4.0 & new bootstrap tools 2017-04-07 14:36:21 -04:00
Tuomas Tynkkynen 20d9edff17 stdenv: aarch64: Update bootstrap tarballs
Hopefully fixes sandboxed build of glibc on aarch64, as discussed on
8bfa9f528c.

Picked from the following cross-trunk evaluation:
http://hydra.nixos.org/eval/1341395 based on nixpkgs
commit bb3ef8a95c.

build job: http://hydra.nixos.org/build/50125932

(busybox's hash not changing is not a bug!)
2017-03-15 19:17:52 +02:00
Tuomas Tynkkynen ce56c99edc mkDerivation: Don't pass buildInputs to stdenv builder in nativeBuildInputs
When not cross compiling, nativeBuildInputs and buildInputs have
identical behaviour. Currently that is implemented by having
mkDerivation do a concatenation of those variables in Nix code and pass
that to the builder via the nativeBuildInputs attribute.

However, that has some annoying side effects, like `foo.buildInputs`
evaluating to `[ ]` even if buildInputs were specified in the nix
expression for foo.

Instead, pass buildInputs and nativeBuildInputs in separate variables as
usual, and move the logic of cross compilation vs. native compilation to
the stdenv builder script. This is probably a tiny bit uglier but
fixes the previous problem.

Issue #4855.
2017-03-02 03:26:48 +02:00
Graham Christensen a9c875fc2e
nixpkgs: allow packages to be marked insecure
If a package's meta has `knownVulnerabilities`, like so:

    stdenv.mkDerivation {
      name = "foobar-1.2.3";

      ...

      meta.knownVulnerabilities = [
        "CVE-0000-00000: remote code execution"
        "CVE-0000-00001: local privilege escalation"
      ];
    }

and a user attempts to install the package, they will be greeted with
a warning indicating that maybe they don't want to install it:

    error: Package ‘foobar-1.2.3’ in ‘...default.nix:20’ is marked as insecure, refusing to evaluate.

    Known issues:

     - CVE-0000-00000: remote code execution
     - CVE-0000-00001: local privilege escalation

    You can install it anyway by whitelisting this package, using the
    following methods:

    a) for `nixos-rebuild` you can add ‘foobar-1.2.3’ to
       `nixpkgs.config.permittedInsecurePackages` in the configuration.nix,
       like so:

         {
           nixpkgs.config.permittedInsecurePackages = [
             "foobar-1.2.3"
           ];
         }

    b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
    ‘foobar-1.2.3’ to `permittedInsecurePackages` in
    ~/.config/nixpkgs/config.nix, like so:

         {
           permittedInsecurePackages = [
             "foobar-1.2.3"
           ];
         }

Adding either of these configurations will permit this specific
version to be installed. A third option also exists:

  NIXPKGS_ALLOW_INSECURE=1 nix-build ...

though I specifically avoided having a global file-based toggle to
disable this check. This way, users don't disable it once in order to
get a single package, and then don't realize future packages are
insecure.
2017-02-24 07:41:05 -05:00
Graham Christensen 59d61ef34a Revert "nixpkgs: allow packages to be marked insecure" 2017-02-23 09:41:42 -05:00
Graham Christensen 38771badd3
nixpkgs: allow packages to be marked insecure
If a package's meta has `knownVulnerabilities`, like so:

    stdenv.mkDerivation {
      name = "foobar-1.2.3";

      ...

      meta.knownVulnerabilities = [
        "CVE-0000-00000: remote code execution"
        "CVE-0000-00001: local privilege escalation"
      ];
    }

and a user attempts to install the package, they will be greeted with
a warning indicating that maybe they don't want to install it:

    error: Package ‘foobar-1.2.3’ in ‘...default.nix:20’ is marked as insecure, refusing to evaluate.

    Known issues:

     - CVE-0000-00000: remote code execution
     - CVE-0000-00001: local privilege escalation

    You can install it anyway by whitelisting this package, using the
    following methods:

    a) for `nixos-rebuild` you can add ‘foobar-1.2.3’ to
       `nixpkgs.config.permittedInsecurePackages` in the configuration.nix,
       like so:

         {
           nixpkgs.config.permittedInsecurePackages = [
             "foobar-1.2.3"
           ];
         }

    b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
    ‘foobar-1.2.3’ to `permittedInsecurePackages` in
    ~/.config/nixpkgs/config.nix, like so:

         {
           permittedInsecurePackages = [
             "foobar-1.2.3"
           ];
         }

Adding either of these configurations will permit this specific
version to be installed. A third option also exists:

  NIXPKGS_ALLOW_INSECURE=1 nix-build ...

though I specifically avoided having a global file-based toggle to
disable this check. This way, users don't disable it once in order to
get a single package, and then don't realize future packages are
insecure.
2017-02-17 20:49:49 -05:00
John Ericson f6ef6b56fe Merge pull request #22387 from Ericson2314/cross-3-platforms
cross stdenv: let build package's build deps resolve to native packages
2017-02-05 17:41:31 -05:00
John Ericson 5eaea6cee0 cross stdenv: let build package's build deps resolve to native packages
This fixes the "sliding window" principle:
  0. Run packages:       build = native;  host = foreign; target = foreign;
  1. Build packages:     build = native;  host = native;  target = foreign;
  2. Vanilla packages:   build = native;  host = native;  target = native;
  3. Vanilla packages:   build = native;  host = native;  target = native;
  n+3. ...

Each stage's build dependencies are resolved against the previous stage,
and the "foreigns" are shifted accordingly. Vanilla packages alone are
built against themsevles, since there are no more "foreign"s to shift away.

Before, build packages' build dependencies were resolved against
themselves:
  0. Run packages:       build = native;  host = foreign; target = foreign;
  1. Build packages:     build = native;  host = native;  target = foreign;
  2. Build packages:     build = native;  host = native;  target = foreign;
  n+2. ...

This is wrong because that principle is violated by the target
platform staying foreign.

This will change the hashes of many build packages and run packages, but
that is OK. This is an unavoidable cost of fixing cross compiling.

The cross compilation docs have been updated to reflect this fix.
2017-02-05 12:01:53 -05:00
Tuomas Tynkkynen b2343099ab make-bootstrap-tools{,-cross}.nix: Fix build after binutils changes
Broken after commit 17a344a ("binutils: Add lib output").
2017-02-04 23:39:25 +02:00
Eelco Dolstra 9d6a55aefd
~/.nixpkgs -> ~/.config/nixpkgs
The former is still respected as a fallback for config.nix for
backwards compatibility (but not for overlays because they're a new
feature).
2017-02-01 16:07:55 +01:00
Tuomas Tynkkynen 18599495c4 stdenv: make is64bit evaluate true on aarch64
This should fix the NSS build.
2017-01-29 20:28:14 +02:00
Tuomas Tynkkynen d1d8ed21b6 stdenv: Aarch64 bootstrap: Update bootstrap tarballs to hydra-built ones
Picked from the following cross-trunk evaluation:
http://hydra.nixos.org/eval/1326772#tabs-still-succeed based on nixpkgs
commit 264d42b9cf.

dist job: http://hydra.nixos.org/build/47094514
2017-01-29 01:34:02 +02:00
Tuomas Tynkkynen e2a2f6d595 Merge pull request #22117 from dezgeg/aarch64-for-merge
Aarch64 (ARM64) support
2017-01-26 17:52:28 +02:00
John Ericson c869fe022e top-level: no more need to expose splicedPackages
This was just done temporarily on the last cross-overhauling PR for
testing purposes.
2017-01-25 09:24:55 -05:00
Tuomas Tynkkynen 5c0a385e1c stdenv: Add aarch64 bootstrap files
These are temporary and will be switched to Hydra-build ones once all
the aarch4 changs are merged.
2017-01-25 00:01:53 +02:00
Tuomas Tynkkynen de3cac0ece make-bootstrap-tools.nix test: Use busybox from store
Our bootstrap tools are actually broken right now due to busybox not
working when invoked directly from a store path. (It says e.g.
"0qqqw19y4gmknajw8vg4fvhx9gxdqlhz-busybox: applet not found").
Make this test actually fail in such case, the next commit will fix the
problem with busybox.
2017-01-25 00:01:52 +02:00
Tuomas Tynkkynen c909f1b18e stdenv: Add updateAutoconfGnuConfigScriptsHook for aarch64
This is required for Aarch64 since a lot of source tarballs ship with
outdated configure scripts that don't recognize aarch64. Simply
replacing the config.guess and config.sub with new versions from
upstream makes them build again.

This same approach is used by at least Buildroot and Fedora. In
principle this could be enabled for all architectures but
conditionalizing this on aarch64 avoids a mass rebuild on x86.
2017-01-25 00:01:52 +02:00
Tuomas Tynkkynen 7c8a060c09 stdenv: Bringup aarch64 architecture support 2017-01-25 00:01:51 +02:00
John Ericson bfb147b6a8 top-level: Only splice as needed for performance 2017-01-24 11:37:56 -05:00
John Ericson a1a798f017 top-level: crossSystem is no longer exposed to packages. Use *Platform. 2017-01-24 11:37:56 -05:00
John Ericson 1c0365bd88 cross-stdenv: Inline useless bindings and reindent
Semantics should be unchanged
2017-01-24 11:37:56 -05:00
John Ericson 92edcb7ebb top-level: Lay the groundwork for {build,host,target}Platform
The long term goal is a big replace:
  { inherit system platform; } => buildPlatform
  crossSystem => hostPlatform
  stdenv.cross => targetPlatform
And additionally making sure each is defined even when not cross compiling.

This commit refactors the bootstrapping code along that vision, but leaves
the old identifiers with their null semantics in place so packages can be
modernized incrementally.
2017-01-24 11:37:56 -05:00
John Ericson bf17d6dacf top-level: Introduce buildPackages for resolving build-time deps
[N.B., this package also applies to the commits that follow it in the same
PR.]

In most cases, buildPackages = pkgs so things work just as before. For
cross compiling, however, buildPackages is resolved as the previous
bootstrapping stage. This allows us to avoid the mkDerivation hacks cross
compiling currently uses today.

To avoid a massive refactor, callPackage will splice together both package
sets. Again to avoid churn, it uses the old `nativeDrv` vs `crossDrv` to do
so. So now, whether cross compiling or not, packages with get a `nativeDrv`
and `crossDrv`---in the non-cross-compiling case they are simply the same
derivation. This is good because it reduces the divergence between the
cross and non-cross dataflow. See `pkgs/top-level/splice.nix` for a comment
along the lines of the preceding paragraph, and the code that does this
splicing.

Also, `forceNativeDrv` is replaced with `forceNativePackages`. The latter
resolves `pkgs` unless the host platform is different from the build
platform, in which case it resolves to `buildPackages`. Note that the
target platform is not important here---it will not prevent
`forcedNativePackages` from resolving to `pkgs`.

--------

Temporarily, we make preserve some dubious decisions in the name of preserving
hashes:

Most importantly, we don't distinguish between "host" and "target" in the
autoconf sense. This leads to the proliferation of *Cross derivations
currently used. What we ought to is resolve native deps of the cross "build
packages" (build = host != target) package set against the "vanilla
packages" (build = host = target) package set. Instead, "build packages"
uses itself, with (informally) target != build in all cases.

This is wrong because it violates the "sliding window" principle of
bootstrapping stages that shifting the platform triple of one stage to the
left coincides with the next stage's platform triple. Only because we don't
explicitly distinguish between "host" and "target" does it appear that the
"sliding window" principle is preserved--indeed it is over the reductionary
"platform double" of just "build" and "host/target".

Additionally, we build libc, libgcc, etc in the same stage as the compilers
themselves, which is wrong because they are used at runtime, not build
time. Fixing this is somewhat subtle, and the solution and problem will be
better explained in the commit that does fix it.

Commits after this will solve both these issues, at the expense of breaking
cross hashes. Native hashes won't be broken, thankfully.

--------

Did the temporary ugliness pan out? Of the packages that currently build in
`release-cross.nix`, the only ones that have their hash changed are
`*.gcc.crossDrv` and `bootstrapTools.*.coreutilsMinimal`. In both cases I
think it doesn't matter.

 1. GCC when doing a `build = host = target = foreign` build (maximally
    cross), still defines environment variables like `CPATH`[1] with
    packages.  This seems assuredly wrong because whether gcc dynamically
    links those, or the programs built by gcc dynamically link those---I
    have no idea which case is reality---they should be foreign. Therefore,
    in all likelihood, I just made the gcc less broken.

 2. Coreutils (ab)used the old cross-compiling infrastructure to depend on
    a native version of itself. When coreutils was overwritten to be built
    with fewer features, the native version it used would also be
    overwritten because the binding was tight. Now it uses the much looser
    `BuildPackages.coreutils` which is just fine as a richer build dep
    doesn't cause any problems and avoids a rebuild.

So, in conclusion I'd say the conservatism payed off. Onward to actually
raking the muck in the next PR!

[1]: https://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html
2017-01-24 11:37:56 -05:00
Nicolas B. Pierron da8cf2662a Fix missing overlays argument in stdenv/linux/default.nix 2017-01-16 01:17:33 +01:00
Nicolas B. Pierron f5dfe78a1e Add overlays mechanism to Nixpkgs.
This patch add a new argument to Nixpkgs default expression named "overlays".

By default, the value of the argument is either taken from the environment variable `NIXPKGS_OVERLAYS`,
or from the directory `~/.nixpkgs/overlays/`.  If the environment variable does not name a valid directory
then this mechanism would fallback on the home directory.  If the home directory does not exists it will
fallback on an empty list of overlays.

The overlays directory should contain the list of extra Nixpkgs stages which would be used to extend the
content of Nixpkgs, with additional set of packages.  The overlays, i-e directory, files, symbolic links
are used in alphabetical order.

The simplest overlay which extends Nixpkgs with nothing looks like:

```nix
self: super: {
}
```

More refined overlays can use `super` as the basis for building new packages, and `self` as a way to query
the final result of the fix-point.

An example of overlay which extends Nixpkgs with a small set of packages can be found at:
  https://github.com/nbp/nixpkgs-mozilla/blob/nixpkgs-overlay/moz-overlay.nix

To use this file, checkout the repository and add a symbolic link to
the `moz-overlay.nix` file in `~/.nixpkgs/overlays` directory.
2017-01-16 01:17:33 +01:00
John Ericson abaf790ea9 stdenv/booter.nix: Add longer note explaining indexing 2017-01-13 13:47:17 -05:00
John Ericson ff35560460 linux stdenv: Inline stage funs to conform to new convention
Code is just moved around
2017-01-13 13:23:26 -05:00
John Ericson b10f415c21 linux stdenv: Remove stray use of stage0 to bootstrap more elegantly 2017-01-13 13:23:26 -05:00
John Ericson 6a45e911c4 linux stdenv: Utilize overrides and prevStage better
`gcc-unwrapped` basically replaces `gccPlain`. It may seem like an ugly
polution to stick it in all-packages, but a future PR will enshrine this
`*-unwrapped` pattern. In any event, the long term goal is stdenvs might
need to tweak how compilers are booted and wrapped, but the code to build
the unwrapped compilers themselves should be generic.
2017-01-13 13:23:25 -05:00
David Grayson 0f33b9f7f1 top-level: Do stdenvOverrides in stage.nix even if crossSystem exists.
Instead, the cross stdenv will patch up the override field -- the complexity
is now confined to the one place it matters.
2017-01-13 13:23:25 -05:00
John Ericson 3e197f7d81 top-level: Normalize stdenv booting
Introduce new abstraction, `stdenv/booter.nix` for composing bootstraping
stages, and use it everywhere for consistency. See that file for more doc.

Stdenvs besides Linux and Darwin are completely refactored to utilize this.
Those two, due to their size and complexity, are minimally edited for
easier reviewing.

No hashes should be changed.
2017-01-13 13:23:23 -05:00
John Ericson 0ef8b69d12 top-level: Modernize stdenv.overrides giving it self and super
Document breaking change in 17.03 release notes
2017-01-13 10:36:11 -05:00
Tuomas Tynkkynen bde8632106 coreutils: Build with libattr to support xattrs
Fixes #21649
2017-01-05 00:55:35 +02:00
John Ericson 19fbe80c3d top-level: avoid another assert false while we're at it 2016-12-15 17:09:46 -05:00
John Ericson 7960a1b1b8 linux stdenv: Avoid assert false
On one hand, don't want to pass garbage that affects hash, on the other
hand footguns are bad.

Now, factored out the derivation so only need to pass in what is used.
2016-12-15 17:09:29 -05:00