Commit graph

310 commits

Author SHA1 Message Date
John Ericson 2364c22ec9 hardening: line order, spacing, and pointless quoting for consistency 2018-04-10 16:33:47 -04:00
John Ericson 4c76d87871 hardenning: Rejigger ifs and explicit declare and unset -v 2018-04-10 16:33:47 -04:00
Charles Strahan 386e77dae9
hardening: simplify reporting of disabled flags 2018-04-10 15:27:13 -04:00
Charles Strahan 273ce83f29
hardening: make requested fixes 2018-04-10 13:04:46 -04:00
Shea Levy c69d8bf5e6
treewide: Remove gnat support.
See discussion in 6ac7b19c97.
2018-03-08 13:56:36 -05:00
Charles Strahan 806edaa0a2
hardening: ld wrapper changes, setup-hook, etc 2018-03-06 19:21:10 -05:00
Charles Strahan 634c748050
hardening: initial cross support 2018-03-06 18:03:13 -05:00
Charles Strahan fc46895e86
hardening: allow user supplied flags to override
Put hardening flags before user supplied flags.
2018-03-06 00:30:09 -05:00
Charles Strahan cc7ce57f86
hardening: clarify the whitelist logic
Per @Ericson2314's suggestion [1], make it more clear that the active
hardenings are decided via whitelist; the blacklist is merely for the
debug messages.

1: 36d5ce41d4 (r133279731)
2018-03-06 00:30:09 -05:00
Charles Strahan 9920923cde
hardening: fix careless bugs
I got a substitution backwards (used '+' instead of '-').

Also, this now works under `set -u` (had to fix a couple unbound
variable references).
2018-03-06 00:30:08 -05:00
Charles Strahan 0937df463f
hardening: fix bug/typo 2018-03-06 00:30:08 -05:00
Charles Strahan 9fe17b2153
hardening: fix #18995 2018-03-06 00:30:00 -05:00
John Ericson 4a29081a94
Merge pull request #35071 from oxij/stdenv/infopages
stdenv, bash: fixing info pages and stuff
2018-02-26 18:06:11 -05:00
Jan Malakhovski f1074211ce cc-wrapper: allow building without documentation, propagate info pages 2018-02-26 22:42:33 +00:00
Vladimír Čunát f5ce8f86df
Revert "Merge staging at '8d490ca9934d0' into master"
This reverts commit fc23242220, reversing
changes made to 754816b84b.
We don't have many binaries yet.  Comment on the original merge commit.
2018-02-26 22:53:18 +01:00
Nikolay Amiantov 9a9c2e6579 cc-wrapper: fix bool handling for empty and zero values
Before the code would fail silently for zero values and with some output for
empties. We now currently handle both via defaulting value to zero and making
`let` return success error code when there's no syntax error.
2018-02-21 23:54:31 +03:00
Vladimír Čunát 1fcd92ce92
Merge branch 'master' into staging
A few thousand rebuilds from master, again.
Hydra: ?compare=1422362
2017-12-31 09:53:49 +01:00
John Ericson 469fd89832 stdenv-setup: Ease the transition with native builds
- All deps go on the PATH

 - CC and Bintools wrappers with their host != depender's host still get their
   setup hooks run.

 - Environment hooks get applied to all packages

This isn't so elegent, but eases the transition on a very significant
PR.
2017-12-30 22:04:23 -05:00
John Ericson 3a50395ef2 {bintools,cc}-wrapper: extraPackages should be depsTargetTargetPropagated
They are libraries used by programs built with these tools, not used by the
tools themselves.
2017-12-30 22:04:21 -05:00
John Ericson a036473a0a {bintools,cc}-wrapper: Fix setup hook to respect the role of the cc-compiler
We now have the information to properly determine the role the
cc-wrapper dependency has, by taking advantage of `offset`. No longer
use the soon-to-be-deprecated crossConfig environment variable, the
temp hack used before this change.
2017-12-30 22:04:21 -05:00
Dmitry Kalinkin 6d9769663d
cc-wrapper: allow compilers to specify unsupported hardening modes
Fixes: 0fd7ef61b2 ('clang_34: Disable hardening bits (#28543)')
2017-12-30 12:23:25 -05:00
Orivej Desh 7c58e8dfc2 Merge branch 'master' into staging
* master: (125 commits)
  scummvm: fix eval
  tinycc: 0.9.27pre-20171016 -> 0.9.27
  Update terraform provider versions
  vscode: 1.18.1 -> 1.19.0
  linux: 4.14.6 -> 4.14.7
  scummvm: 1.9.0 -> 2.0.0
  cmst: 2017.03.18 -> 2017.09.19
  albert: 0.14.7 -> 0.14.14
  obs-studio: fix vlc plugin
  ffmpeg, ffmpeg-full: 3.4 -> 3.4.1
  uchiwa: 0.26.3 -> 1.1.0
  linux-testing: 4.15-rc3 -> 4.15-rc4
  steam: override nss, nspr, fixes #32781
  ponyc: 0.20.0 -> 0.21.0
  pythonPackages.pwntools: disable tests
  gnome3.gnome-tweak-tool: 3.26.3 → 3.26.4
  vim-rhubarb: init at 2017-06-28
  atom: depend on libsecret
  nvidia-settings: Make sure binary can find libXv.so
  backblaze-b2: 0.6.2 -> 1.1.0
  ...
2017-12-18 15:56:03 +00:00
Dylan Simon 0c62b7cd74 cc-wrapper: don't set cxx_stdlib when nativeTools is true
There are no gcc paths on nativeTools, and cc isn't set.
2017-12-17 04:23:54 +00:00
John Ericson 7ef4448c97 Merge commit '9d8f9b2e531bf95a700a949d879927fb6996ffc9' into binutils-wrapper 2017-12-13 16:08:36 -05:00
John Ericson 2bba929062 bintools-wrapper: Import separately from cc-wrapper 2017-12-13 16:08:18 -05:00
John Ericson 8e557ed2c5 bintools-wrapper: Init
Factor a bintools (i.e. binutils / cctools) wrapper out of cc-wrapper. While
only LD is wrapped, the setup hook defines environment variables on behalf of
other utilites.
2017-12-13 16:08:18 -05:00
John Ericson 4f869bccc1 cc-wrapper: Don't treat "-" alone as a flag
It means stdin, and is morally equivalent to passing a file. e.g.

  $ echo 'int main(void) { return 0; }' | gcc -x c -

will compile and link a binary.
2017-12-13 16:08:17 -05:00
John Ericson bdd6c037c0 cc-wrapper: Use separate mangler for "bool" variables
This avoids any `NIX_FOOBAR=1 1` not triggering conditions.
2017-12-13 16:08:17 -05:00
John Ericson fc7ed86915 cc-wrapper: Pull variable mangler into utils.sh
In preparation for splitting out bintools-wrapper
2017-12-13 16:08:13 -05:00
John Ericson 45d4b27d02 cc-wrapper: GNAT wrapper stop caring about -m32
It need not concern itself with 32-bit dynamic linking; ld-wrapper handles that now.
2017-12-11 19:12:28 -05:00
Will Dietz 115bf9d2cf cc-wrapper: don't add broken cflags to clang
On non-GNU (gcc) compilers, there is no "/lib/gcc/..."
so when this is eventually expanded this is empty
resulting in an incomplete "-idirafter " that
eats the next argument:

-idirafter -B/nix/store/wamjwwdvkmhbf4f2902nhw8jxxzv0hy3-clang-wrapper-4.0.1/bin/
2017-11-30 18:43:43 -06:00
John Ericson 43e1137397 cc-wrapper: Define new- and old-style cross env vars
For example, `BUILD_CC` and `CC_FOR_BUILD`
2017-11-28 15:57:05 -05:00
John Ericson f4cb1e2ffc cc-wrapper: Export env vars for objdump and readelf in setup-hook
Also fix alphabetical order
2017-11-28 15:56:21 -05:00
John Ericson 994cb76850 cc-wrapper: Don't leave CMD defined after setup hook
Rename to `cmd` too, as uppercase typically means the variable is
exported.
2017-11-28 15:56:17 -05:00
John Ericson 43e00f7d18 cc-wrapper: Fix stray binPrefix -> targetPrefix
I thought my sed in e755a8a27d was
exhaustive, but it was not.
2017-11-27 03:30:15 -05:00
John Ericson e755a8a27d treewide: Use targetPrefix instead of prefix for platform name prefixes
Certain tools, e.g. compilers, are customarily prefixed with the name of
their target platform so that multiple builds can be used at once
without clobbering each other on the PATH. I was using identifiers named
`prefix` for this purpose, but that conflicts with the standard use of
`prefix` to mean the directory where something is installed. To avoid
conflict and confusion, I renamed those to `targetPrefix`.
2017-11-27 03:15:50 -05:00
John Ericson 0101856765
Merge pull request #30549 from obsidiansystems/bintools
treewide: Introduce stdenv.cc.bintools
2017-11-08 14:20:48 -05:00
Vladimír Čunát c8c6a1edb5
Merge #31209: cc-wrapper: Fix if dynamicLinker not found 2017-11-06 13:07:43 +01:00
Bojan Nikolic 3a63fc1258 Remove trailing line 2017-11-05 23:00:24 +00:00
Bojan Nikolic 2ed054885b Remove the unnecessary second conditional 2017-11-05 22:57:44 +00:00
John Ericson 70d91badf5 treewide: Depend on stdenv.cc.bintools instead of binutils directly
One should do this when needed executables at build time. It is more
honest and cross-friendly than refering to binutils directly.
2017-11-05 17:10:53 -05:00
Nikolay Amiantov 5f3b84e979 cc-wrapper: disable POSIX compatibility 2017-11-05 12:19:37 +02:00
Bojan Nikolic 3e9daece1d nixos/cc-wrapper: Fix bug if dynamicLinker not found
If a dynamic linker for target is not found the generated script fails
due to unbound variable error (due to "set -u"). Correct by specifying
default value with dynamicLinker:- and not generating ldflagsBefore if
no linker is found.

This problem was found when cross compiling to mingw32 targets
2017-11-03 21:36:43 +00:00
Dan Peebles b426c85ce2 Get rid of most @rpath nonsense on Darwin
This requires some small changes in the stdenv, then working around the
weird choice LLVM made to hardcode @rpath in its install name, and then
lets us remove a ton of annoying workaround hacks in many of our Go
packages. With any luck this will mean less hackery going forward.
2017-10-08 16:13:46 -04:00
John Ericson fdbda216b1 cc-wrapper: Clean up dynamic linking with x86 multilib
It's better layering to do everything in ld-wrapper. Also, use numeric
comparisons for `relocatable`.
2017-09-28 20:05:26 -04:00
John Ericson d349f9a340 cc-wrapper: Use stdenvNoCC to build
cc-wrapper may wrap a cc-compiler, but it doesn't need one to build
itself. (c.f. expand-response-params is a separate derivation.) This
helps avoid cycles on the cross stuff, in addition to removing a
useless dependency edge.

I could have been super careful with overrides in the stdenv to avoid
the mass rebuild, but I don't think it's worth it.
2017-09-26 14:08:21 -04:00
John Ericson 0d3d2a01d2 cc-wrapper: Add set -x tracing for NIX_DEBUG >= 7 2017-09-26 11:24:19 -04:00
John Ericson 127a5f3357 treewide: Use (( "${NIX_DEBUG:-0}" >= 1) )) consistently 2017-09-26 11:24:19 -04:00
John Ericson 6c74ee68b9 cc-wrapper, cc-wrapper-old: Simplify shell logic
1. `crossDrv` is now the default so we don't need to worry about that in
   build != host builds.

2. shell is the build time shell, so `wrapCCCross` doesn't need to
   worry, as build == host.

3. `shell.shellPath` will always be appended where useful.

4. Complicated `shell == ""` logic served no purpose.
2017-09-19 16:45:24 -04:00
John Ericson 13fc982e65 cc-wrapper: Use same dynamic loader on all Darwin, not just x86_64
In practice, this is correct because iOS is on ARM and puts the loader
there.
2017-09-18 19:25:58 -04:00
John Ericson a44bbc72e6 cc-wrapper: Remove obsolete assertion
This was just causing evaluation problems on cross.
2017-09-18 19:22:57 -04:00
Eelco Dolstra ec8d41f08c
Revert "Merge pull request #28557 from obsidiansystems/binutils-wrapper"
This reverts commit 0a944b345e, reversing
changes made to 61733ed6cc.

I dislike these massive stdenv changes with unclear motivation,
especially when they involve gratuitous mass renames like NIX_CC ->
NIX_BINUTILS. The previous such rename (NIX_GCC -> NIX_CC) caused
months of pain, so let's not do that again.
2017-09-07 12:51:21 +02:00
John Ericson dbf6d20d64 binutils-wrapper: Import separately from cc-wrapper 2017-09-01 11:44:56 -04:00
John Ericson 40e9b2a7e6 binutils-wrapper: Init
Factor a binutils wrapper out of cc-wrapper. While only LD is wrapped,
the setup hook defines environment variables on behalf of other
utilites.
2017-09-01 11:44:55 -04:00
John Ericson fbb7d335db cc-wrapper: Use separate mangler for "bool" variables
This avoids any `NIX_FOOBAR=1 1` not triggering conditions.
2017-09-01 11:44:54 -04:00
John Ericson 1f5807d760 cc-wrapper: Pull variable mangler into utils.sh
In preparation for splitting out binutils-wrapper
2017-09-01 11:44:54 -04:00
John Ericson 94c0267fc1 cc-wrapper: Clean up dynamic linking with x86 multilib
It's better layering to do everything in ld-wrapper.
2017-09-01 11:44:54 -04:00
John Ericson 3d3a6e0fac cc-wrapper: Remove support for NIX_LDFLAGS_HARDEN
It has long been deprecated
2017-08-31 13:54:32 -04:00
John Ericson 46fd4bcb14 cc-wrapper: Remove {START,EXEC}_HOOK
These are no longer used by anything
2017-08-31 13:54:07 -04:00
John Ericson 97a48835b7 mkDerivation, cc-wrapper: Check hardening flag validity in Nix
This becomes necessary if more wrappers besides cc-wrapper start
supporting hardening flags. Also good to make the warning into an
error.

Also ensure interface is being used right: Not as a string, not just in
bash.
2017-08-30 17:53:42 +02:00
John Ericson 822a8d0148 cc-wrapper: Remove redundant hardening
GCC just passes `-z ...` flags to ld unaltered, and they are already
passed to LD anyways. On the other hand, `-pie` affects gcc behavior
too.
2017-08-30 17:53:42 +02:00
Daiderd Jordan 92652b4d79
darwin-CF: use @rpath for library id and add an rpath entry for CF based on NIX_COREFOUNDATION_RPATH 2017-08-28 23:24:58 +02:00
John Ericson df7c305c4c cc-wrapper: Leverage the setup script instead of buildCommand 2017-08-25 15:10:05 -04:00
John Ericson dc47e763c1 cc-wrapper: Use set -u for better maintainability 2017-08-25 15:10:05 -04:00
John Ericson a470be5a16 expand-response-params: Build more normally 2017-08-25 15:10:02 -04:00
John Ericson 287fce6402 expand-response-params: Pull out of cc-wrapper
No hashes were changed
2017-08-25 11:21:09 -04:00
Daniel Peebles 4f7f48fb1a Merge pull request #28521 from obsidiansystems/cc-wrapper-dead-code
cc-wrapper: Remove dead code
2017-08-24 16:05:51 -04:00
Tuomas Tynkkynen 0c0fad6141 treewide: Consistently call ARM 'arm'
No need for silly differences.
2017-08-24 01:17:01 +03:00
John Ericson 08f3a60e7b cc-wrapper: Remove dead code
ccPath is only defined below, so this condition would never be true.

Worse, that's not quite true: what if somebody happend to have `/clang`
and no sandboxing. Boy, wouldn't that be annoying to debug!
2017-08-23 17:09:47 -04:00
John Ericson 0135e61b4c Merge remote-tracking branch 'upstream/master' into staging
That way the tarball job succeeds
2017-08-21 18:42:57 -04:00
John Ericson c035711072 cc-wrapper: Remove unused params
Ensured hashes unchanged and eval succeeds in tarball job
2017-08-21 18:40:41 -04:00
Bojan Nikolic ad8d5649a2 set +u workaround for empty arrays
libDirs can be empty, which in combination with "set -u" of
9f1e009975 will cause a variable unbound
error on old bash versions
2017-08-21 12:27:26 +01:00
Daiderd Jordan 5d3527c148
cc-wrapper: don't export CPP
CPP defaults to $(CC) -E and setting it explicitly seems to cause
issues in a number of packages like gcc and gdb.

https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html
2017-08-16 21:38:32 +02:00
John Ericson 3c9cf282bb cc-wrapper: Improve set -u compliance 2017-08-14 14:44:32 -04:00
Vladimír Čunát 6899c7fdb9
Merge branch 'master' into gcc-6 2017-08-12 10:09:41 +02:00
Daiderd Jordan bf8c125170
cc-wrapper: fix LD_DYLD_PATH on darwin
Having multiple compilers in the build environment would result in an
invalid LD_DYLD_PATH like /usr/lib/dyld/usr/lib/dyld.
Since the path is hardcoded in XNU it can't be anything but
/usr/lib/dyld anyway.
2017-08-10 00:22:58 +02:00
Orivej Desh 771f28bac8 cc-wrapper: Fix support for qtbase-setup-hook
Revert https://github.com/NixOS/nixpkgs/pull/27657#issuecomment-318161946
due to https://github.com/NixOS/nixpkgs/pull/28021#issuecomment-321045542
2017-08-09 18:17:46 +02:00
Orivej Desh a6e7dffd1f cc-wrapper: Fix standalone ld 2017-08-08 11:03:51 +02:00
Orivej Desh c8e9dcc8a4 cc-wrapper: Fix standalone gcc
This ensures that all salted variables are defined even if the wrapped program
is invoked outside nix-build environment.
2017-08-08 11:03:51 +02:00
Orivej Desh c8f7f18e69 cc-wrapper: Fix adding directories to rpath
This fixes a bug introduced in #27831: `for path in "$dir"/lib*.so` assumed that
all libs match `lib*.so`, but 07674788d6 started
adding libs that match `*.so` and `*.so.*`.
2017-08-08 11:03:51 +02:00
Robin Gloster 3e981b9e33
Revert "cc-wrapper: fix set -u errors"
This reverts commit 67a41eafe9.

see #28021 for discussion
2017-08-08 10:20:00 +02:00
Robin Gloster 67a41eafe9
cc-wrapper: fix set -u errors
cc @Ericson2314
2017-08-08 10:15:34 +02:00
John Ericson 810fb0c968 cc-wrapper: Don't slurp NIX_DONT_SET_RPATH twice
Otherwise we end with a definition like `1 1`, which doesn't trigger the right
conditional.
2017-08-07 11:09:25 -04:00
John Ericson 2493454e13 cc-wrapper: Use set -u throughout
Now is an opportune time to do this, as the infixSalt conversion in
`add-flags.sh` ensures that all the relevant `NIX_*` vars will be
defined even if empty.
2017-08-07 03:05:51 -04:00
John Ericson 42f35503b5 cc-wrapper: Make hygienic
See the added comments for what exactly has been done.
2017-08-07 03:05:50 -04:00
John Ericson 9f1e009975 cc-wrapper: Unconditionally use @infixSalt@ accross the board
This is basically a sed job, in preparation of the next commit. The
rules are more or less:

  - s"NIX_(.._WRAPPER_)?([a-zA-Z0-9@]*)"NIX_\1@infixSalt@_\2"g

  - except for non-cc-wrapper-specific vars like `NIX_DEBUG`
2017-08-07 03:05:50 -04:00
John Ericson 5ba3972add cc-wrapper: Stop preWrapping
This is an ugly temp hack for cross compilation, but now we have something better on the way.

Bind `infixSalt` as an environment variable as it will be used in it.
2017-08-07 03:05:50 -04:00
Vladimír Čunát f779df87b1
Merge branch 'master' into staging
Let's drop i686-linux here as well.
2017-08-06 10:03:25 +02:00
John Ericson 6463fd3d7e cc-wrapper: Pass shellcheck and other cleanups
In many cases, this involved taking @orivej's and @edolstra's recent
ld-wrapper improvements, and applying then elsewhere.
2017-08-04 12:47:29 -04:00
John Ericson 8ba6012dfd Merge PR #27536 2017-08-03 17:25:30 -04:00
John Ericson cd6c452ade cc-wrapper: Don't hardcode MacOS min version for delegates 2017-08-03 16:37:27 -04:00
John Ericson 2e61d8b71c cc-wrapper: Handle more -l.... gotchas 2017-08-03 16:37:27 -04:00
Orivej Desh 07674788d6 Respect NIX_DONT_SET_RPATH when .so appears in the command line
Unified processing of command line arguments in ld-wrapper broke support for
`NIX_DONT_SET_RPATH` and revealed that ld-wrapper adds the directory of its
`-plugin` argument to runpath. This pull request fixes that. It treats
`dir/libname.so` as `-L dir -l name`, because this is how `ld.so` interprets
resulting binary: with `dir` in `RUNPATH` and the bare `libname.so` (without
`dir`) in `NEEDED`, it looks for `libname.so` in each `RUNPATH` and chooses the
first, even when the linker was invoked with an absolute path to `.so`.
2017-08-01 11:57:33 +02:00
John Ericson 1cd74ac70a cc-wrapper: Make sure symbol tables of child delegators is not empty
As described in https://github.com/NixOS/nixpkgs/issues/18461, MacOS no
longer accepts dylibs which only reexport other dylibs, because their
symbol tables are empty. To get around this, we define an object file
with a single "private extern" symbol, which hopefully won't clobber
anything.
2017-07-31 17:02:56 -04:00
John Ericson 9242c8b331 cc-wrapper: Use two scripts instead of patch 2017-07-31 17:02:56 -04:00
John Ericson 0c37778c2c cc-wrapper: WIP linking hack for mac OS
Probably best to override Haskell packages set, or anything else
linking a lot of libraries, with this.
2017-07-31 17:02:56 -04:00
Domen Kožar 61b5b7fd2f Merge pull request #27657 from orivej/optimize-rpath
ld-wrapper: Optimize expanding rpath
2017-07-31 14:43:48 +02:00
Orivej Desh 4964b104b9 ld-wrapper: Optimize expanding rpath
The time to expand rpath was proportional to the number of -L flags times the
number of -l flags.  Now it is proportional to their sum (assuming constant
number of files in each directory in an -L flag).

Issue reported by @nh2 at https://github.com/NixOS/nixpkgs/issues/27609#issuecomment-317916623
2017-07-28 20:50:30 +00:00
John Ericson ea7d13cf1a stdenv-setup and misc hooks: Work with bash-3.4 for MacOS nix-shell
This is a temporary measure until this impurity is removed from Nix.
2017-07-26 09:08:01 -04:00
John Ericson f6f40e3fe5 stdenv-setup and misc pkgs: Revert to space-deliminated propagated-* files
We cannot switch to line-delimited yet, because certain Nix commands do
not read in the entire file, but just the first line.
2017-07-26 09:07:55 -04:00
Eelco Dolstra 47821f1cf0
cc-wrapper: More quadratic performance fixes
This eliminates the slow lookup of whether we've already seen an rpath
/ library path entry.

Issue #27609.
2017-07-25 18:46:49 +02:00
Eelco Dolstra 70d4d22806
cc-wrapper: Add a "man" output
... and propagate cc.man from there.
2017-07-25 16:12:58 +02:00
Eelco Dolstra b9c403c531
stdenvLinux: Remove bootstrapTools from closure
The "expand-response-params" program had an unnecessary reference to
bootstrapTools, bloating the closure from 205 to 314 MiB.
2017-07-25 14:21:17 +02:00
Eelco Dolstra aa4a92d2df
cc-wrapper/ld-wrapper: Minor speedup in string concatenation
There is still a O(n) pattern match in ld-wrapper, so we should
probably rewrite that code to use associative arrays.
2017-07-24 14:48:26 +02:00
John Ericson b087618ac0 Revert "stdenv: Store one package per line in nix-support/propagated-*"
As @oxij points out in [1], this breakage is especially serious because
it changes the contents of built environments without a corresonding
change in their hashes. Also, the revert is easier than I thought.

This reverts commit 3cb745d5a6.

[1]: https://github.com/NixOS/nixpkgs/pull/27427#issuecomment-317293040
2017-07-24 01:05:30 -04:00
John Ericson 3739858571 cc-wrapper: Use new bash uppercase subsitution syntax in setup hook
While this requires newer bash, stdenv's setup.sh now does across the
board anyways. This way is more concise.
2017-07-15 20:33:10 -04:00
John Ericson 093cc00cdd cc-wrapper: Always export environment variables for binutils
Before, this only happened when cross compiling.
2017-07-15 16:01:03 -04:00
John Ericson aca5ba405e cc-wrapper: Unify and improve dynamic linker flag logic
Besides deduplicating overlapping logic, clear warning messages were
added for:

 - No glob/path for dynamic linker provided (use default glob)

 - Glob did not expand to anything (don't append flag)

 - glob expanded to multiple things (take first, like before)
2017-07-14 15:21:34 -04:00
John Ericson 3cb745d5a6 stdenv: Store one package per line in nix-support/propagated-*
This makes those files a bit easier to read. Also, for what it's worth,
it brings us one baby step closer to handling spaces in store paths.

Also, I optimized handling of many transitive deps with read. Probably,
not very beneficial, but nice to enforce the pkg-per-line structure.
Doing so let me find much dubious code and fix it.

Two misc notes:

 - `propagated-user-env-packages` also needed to be adjusted as
   sometimes it is copied to/from the propagated input files.

 - `local fd` should ensure that file descriptors aren't clobbered
   during recursion.
2017-07-10 13:32:13 -04:00
Daiderd Jordan 980346592c
Merge branch 'staging' into master 2017-07-08 22:22:17 +02:00
John Ericson a302d7360f top-level: {build,host,target}Platform are defined in the stdenv instead
See #27069 for a discussion of this
2017-07-07 12:55:02 -04:00
Ryan Trinkle aad0ebb194 cc-wrapper: expand-response-params: fix parsing of quotes that are quoted and escaped 2017-07-03 23:16:10 -04:00
Orivej Desh 2bc7b4e134 cc-wrapper: simplify expandResponseParams parser
Import from b2446902fe
2017-07-03 21:51:23 +00:00
Ryan Trinkle d07f30f628 cc-wrapper: improve response file parsing speed 2017-06-30 15:20:53 -04:00
John Ericson 16be434b0b Merge accepted cross compilation PRs into staging 2017-06-28 23:17:21 -04:00
John Ericson c23028bdad cc-wrapper: Also export env var for windres, when it is present 2017-06-28 18:31:37 -04:00
David McFarland 63a1a2ac63 stdenv: disable -fPIC on cygwin
It's always on, and you get a warning if you specify it
2017-06-26 09:33:44 -03:00
Franz Pletz d42f502c84
Revert "cc-wrapper: add stackcheck hardening (stack clash)"
This reverts commit 4150f5e8ba.

The stack protector hardening breaks the llvm_4 build.
2017-06-26 01:12:38 +02:00
Franz Pletz a4252649bf
Revert "cc-wrapper: fix darwin"
This reverts commit c3a8595276.

The stack protector hardening breaks the llvm_4 build.
2017-06-26 01:12:11 +02:00
John Ericson 87fab3d6a5 Merge some merged cross-compilation PRs into into staging 2017-06-23 20:24:27 -04:00
John Ericson 12795a7068 misc: Remove almost all uses of stdenv.{ccCross,binutils} 2017-06-22 17:53:52 -04:00
John Ericson c2e2152afc cc-wrapper: Port over finding of dynamic linker from gcc-cross-wrapper
At the next break, we can deduplicate and go for the best of both
worlds.
2017-06-22 17:53:36 -04:00
John Ericson 0aee847858 cc-wrapper: Make the cross compiler use a crossEnvHook
This is a hack until stdenv's setup.hs gets overhauled
2017-06-22 17:52:29 -04:00
Ross MacLeod c1fb71b74a cc-wrapper: when cross building, wedge in setting AR, AS, and LD env vars to their cross building versions
I think it's ok to export things which aren't wrapped. The cc-wrapper
can be thought of as responsible for all of binutils and the c
compiler, only wrapping those binaries which are necessary to
interposition---as opposed to all binaries it thinks are relevaant.

Conversely, adding the setup hook to the unwrapped compilers would be
unforunate as hooks are ugly hacks and the compilers themselves take
a long time to rebuild. Better to wholely separate "pure packages" from
hacks.
2017-06-22 17:52:28 -04:00
John Ericson 8a7433fca4 cc-wrapper: Temporarily got to great lengths to avoid mass-rebuild
Eventually we should avoid this "pre-wrapping" and just update those
files in nixpkgs. This mass-rebuild change is best done along with
those needed to reduce the disparity between native and cross (i.e.
making native the "identity cross").
2017-06-22 17:52:28 -04:00
John Ericson 0f0383394d cc-wrapper: Salt environment variables with LLVM triples
We now (on cross) require per-target flag interposition by putting the
triple in the names of the relevant environment variables, e.g:

export NIX_arm_unknown_linux_gnu_CFLAGS_COMPILE=...

The wrapper also has a `infixSalt` attribute (and "_" prefixed and
suffixed variants) to assist downstream packages.

Note how that the dashes are replaced to keep the identifier valid.
Using names like this allows us to keep the settings for different
compilers seperate.

I think it might be even better to use names like `NIX_{BUILD,HOST}...`
using the platform's role rather than the platform itself, but this
would be more work as the previous stages' tools would have to be re-
wrapped to take on their new role. I therefore didn't do this for now,
but that route should be thoroughly explored in the future.
2017-06-22 17:52:28 -04:00
John Ericson 04a3cad968 cc-wrapper: Add support for no libc whatsoever 2017-06-22 17:52:27 -04:00
John Ericson d70e7263f4 cc-wrapper: Simplify and correct logic to chose dynamic linker library 2017-06-22 17:52:27 -04:00
John Ericson 459f1c60f5 cc-wrapper: Learn about target prefixes
This is first step towards getting rid of gcc-wrapper-cross
2017-06-22 17:52:27 -04:00
John Ericson c4ba2e3ef6 cc-wrapper: Remove stdenv.is* for targetPlatform.is*
Modify bootstrapping stdenvs to make sure `targetPlatform` also passed.
2017-06-22 17:52:27 -04:00
Franz Pletz c3a8595276
cc-wrapper: fix darwin
Clang doesn't support -fstack-check=specific, only -fstack-check. Still a
noop, though.
2017-06-22 02:44:04 +02:00
Franz Pletz 4150f5e8ba
cc-wrapper: add stackcheck hardening (stack clash)
This fixes the Stack Clash issue rediscovered by Qualys. See
https://www.qualys.com/2017/06/19/stack-clash/stack-clash.txt
for more information on the topic, specifically section III.

We don't have the kernel mitigation available because it is a Grsecurity
feature which we don't support anymore. Other distributions like Gentoo
Hardened and Arch already have `-fstack-check` enabled by default.

See the Gentoo page on Stack Clash for more information on this solution:
https://wiki.gentoo.org/wiki/Hardened/Gentoo_Hardened_and_Stack_Clash

This unfortunately doesn't apply to clang because `-fstack-check` is a
noop there. Note that the GCC implementation also has problems that could
be exploited to circumvent these checks but it is still better than
keeping it disabled.
2017-06-22 00:41:53 +02:00
Orivej Desh 5413bfa8e3 Speed up parsing @args.rsp compiler arguments
Improves upon #25205

https://gist.github.com/pbogdan/9d6986bf931b58a70d75e14eb40ee8a1 parsing time is
reduced from one minute to one second
2017-06-14 18:56:44 +00:00
Jörg Thalheim e2197465c1
cc-wrapper: externalize default_cxx_stdlib_compile
This value is require to get c++ std include path for libclang based tools (vim plugins in my case).
I currently extract it this with this rather command:

```
eval echo $(nix-instantiate --eval --expr 'with (import <nixpkgs>) {}; clang.default_cxx_stdlib_compile')
```

it did not trigger any recompilation on my system.
2017-06-08 19:50:40 +01:00
Orivej Desh e9adf383e6 Fix parsing @args.rsp compiler arguments 2017-04-25 13:29:43 +00:00
Tuomas Tynkkynen 7c8a060c09 stdenv: Bringup aarch64 architecture support 2017-01-25 00:01:51 +02:00
John Ericson 09401d44eb cc-wrapper: Remove unneeded crossAttrs 2017-01-24 11:37:56 -05:00
David McFarland 175461e09b cc-wrapper: don't expand $ in response files
This makes the response file handling more consistent with GCC.

For example, a reponse file may contain:

    "-Wl,$ORIGIN"

GCC will treat this as a double quoted string and not expand the
variable reference.  Previously, cc-wrapper would expand the variable
in the same was as if the string was provided on the command line.
2016-11-19 20:21:32 -04:00
Shea Levy 225602d9f2 Merge branch 'response-files' of git://github.com/corngood/nixpkgs into staging 2016-10-31 10:07:30 -04:00
David McFarland 1ad1edbb32 cc-wrapper: expand response files
Fixes #11762
2016-10-31 01:12:07 -03:00
David McFarland 2b6659e13e cc-wrapper: fix detection of unsupported linker flags on cygwin 2016-10-18 17:52:20 -03:00
Shea Levy eca0f17ad2 nix-buffer support improvements.
Use inherit-local, add per-package elisp hooks.
2016-10-07 10:31:37 -04:00
Markus Hauck aafbd40577 Replace hard coded /bin/bash occurrences 2016-10-04 20:15:37 +02:00
Franz Pletz c0fa26ef3b Merge remote-tracking branch 'origin/master' into hardened-stdenv 2016-08-24 11:01:53 +02:00
Franz Pletz 8576aea57c cc-wrapper: fix detection of unsupported linker flags 2016-08-24 10:59:52 +02:00
Franz Pletz 3c06e5f6f7 cc-wrapper: check ld hardening capabilities in stdenv 2016-08-23 18:13:31 +02:00
Robin Gloster 07604ad631 add-hardening.sh: fix quotation 2016-08-23 15:27:51 +00:00
Robin Gloster 335d0097cf cc-wrapper: add-{flags,hardening} -> add-{flags,hardening}.sh 2016-08-23 15:27:51 +00:00
Nikolay Amiantov 24f695ffaa cc-wrapper: handle -nostdinc{++,} 2016-08-17 13:19:46 +03:00
Robin Gloster 5185bc1773 Merge remote-tracking branch 'upstream/master' into hardened-stdenv 2016-07-15 14:41:01 +00:00