Commit graph

225 commits

Author SHA1 Message Date
Chuck a45b3ad89f
makeWrapper: Remove unused extraFlagsArray feature
There is a bug in this feature: It allows extra arguments to leak in
from the environment. For example:

  $ export extraFlagsArray=date
  $ man ls

Note that you get the man page for date rather than for ls. This happens
because 'man' happens to use a wrapper (to add groff to its PATH).

An attempt to fix this was made in 5ae18574fc in PR #19328 for
issue #2537, but 1. That change didn't actually fix the problem because
it addressed makeWrapper's environment during the build process, not the
constructed wrapper script's environment after installation, and 2. That
change was apparently accidentally lost when merged with 7ff6eec5fd.

Rather than trying to fix the bug again, we remove the extraFlagsArray
feature, since it has never been used in the public repo in the ten
years it has been available.

wrapAclocal continues to use its own, separate flavor of extraFlagsArray
in a more limited context. The analogous bug there was fixed in
4d7d10da6b in 2011.
2019-09-25 09:23:09 -07:00
Jan Tojnar f2718dbbc0
wrapGAppsHook: address some shellcheck suggestions
These should not cause issues in practice but it is good idea to handle them.

* prefix and targetOffset are mandatory, as they are always set by the generic builder.
* wrapPrefixVariables and dontWrapGApps are now defaulting to empty value, as they are not mandatory.
2019-09-10 16:24:11 +02:00
Jan Tojnar c54ba9aec9
wrapGAppsHook: simplify taking env vars
using indirect variable access must be better than eval
2019-09-10 16:24:11 +02:00
Jan Tojnar 3c1563f49d
wrapGAppsHook: add wrapGApp helper 2019-09-10 16:24:10 +02:00
Lily Ballard 43dade238f
installShellFiles: init (#65211)
This is a new package that provides a shell hook to make it easy to
declare manpages and shell completions in a manner that doesn't require
remembering where to actually install them. Basic usage looks like

  { stdenv, installShellFiles, ... }:
  stdenv.mkDerivation {
    # ...
    nativeBuildInputs = [ installShellFiles ];
    postInstall = ''
      installManPage doc/foobar.1
      installShellCompletion --bash share/completions/foobar.bash
      installShellCompletion --fish share/completions/foobar.fish
      installShellCompletion --zsh share/completions/_foobar
    '';
    # ...
  }

See source comments for more details on the functions.
2019-09-04 23:19:17 +02:00
Benjamin Esham 3a9b0bd634 add shortenPerlShebang function
This setup hook modifies a Perl script so that any "-I" flags in its shebang
line are rewritten into a "use lib ..." statement on the next line. This gets
around a limitation in Darwin, which will not properly handle a script whose
shebang line exceeds 511 characters.
2019-08-18 16:22:23 -07:00
Daniel Schaefer 7c3d7521f0 patchShebangs: Exit if no arguments were given
Commit "patchShebangs: Allow for multiple arguments" 4a1e51f957
removed the check. We don't want to break existing usages so this
introduces it again with a successful exit code.
2019-05-26 16:05:06 +02:00
Daniel Schaefer 4a1e51f957 patchShebangs: Allow for multiple arguments
It's tempting to think patchShebangs supports multiple arguments.
Without this patch it just silently ignores all but the first. Now it
patches the shebangs in all of its arguments.

Fixes: #57695
2019-05-25 16:15:54 +02:00
Daniel Schaefer 81d3bd60a4 patchShebangs: Explain that script must be executable 2019-05-25 16:15:54 +02:00
Matthew Bauer 87944c3125
Merge pull request #56744 from matthewbauer/macos-10-12
Update macOS to 10.12
2019-04-26 22:20:03 -04:00
Matthew Bauer a1533dfc41 patch-shebangs: fix dev detection
$output = $outputDev is the right condition. ${!outputDev} is only
useful if you want a path.
2019-04-26 21:54:48 -04:00
Matthew Bauer 61bc03c017 Revert "Revert "patch-shebangs: respect cross compilation""
This reverts commit 9c4b11e9a0.
2019-04-26 21:54:48 -04:00
Graham Christensen a682d720c7
Merge pull request #60215 from nmattia/nm-tmp-timestamp
patchShebangs: create timestamp in tmp dir
2019-04-25 12:50:16 -04:00
Nicolas Mattia cc9b4029bc patchShebangs: create timestamp in tmp dir
Creating the timestamp in the patched script's directory has a few
drawbacks:

* if "foo.timestamp" already exists, it will be overwritten
* it requires the directory to be writable
2019-04-25 17:07:58 +02:00
Wael M. Nasreddine 5af0780492
Merge remote-tracking branch 'origin/master' into staging
* origin/master: (693 commits)
  buildGoModule: use go_1_12 instead of go_1_11 (#58103)
  gitAndTools.lab: 0.15.2 -> 0.15.3 (#58091)
  signal-desktop: 1.22.0 -> 1.23.0
  added missing semicolon to documentation
  terminus_font_ttf: 4.46.0 -> 4.47.0
  buildGoModule: remove SSL env vars in favor of cacert in buildInputs (#58071)
  dav1d: init at 0.2.1
  dropbox-cli: 2018.11.28 -> 2019.02.14
  atlassian-confluence: 6.14.1 -> 6.14.2
  maintainers: update email for dywedir
  python.pkgs.hglib: use patch to specify hg path (#57926)
  chkrootkit: 0.52 -> 0.53
  radare2-cutter: 1.7.2 -> 1.8.0
  autorandr: 1.7 -> 1.8
  pythonPackages.pyhepmc: fix build
  llvm-polly/clang-polly: use latest llvm
  apulse: 0.1.11.1 -> 0.1.12, cleanup
  factorio: experimental 0.17.14 → 0.17.16 (#58000)
  sequeler: 0.6.7 -> 0.6.8
  nasc: 0.5.1 -> 0.5.2
  ...
2019-03-21 21:01:25 -07:00
Danylo Hlynskyi de0612c46c
auto-patchelf: don't use grep -q, as it causes Broken pipe (#56958)
This rare sitation was caught when building zoom-us package:
```
automatically fixing dependencies for ELF files
/nix/store/71d65fplq44y9yn2fvkpn2d3hrszracd-auto-patchelf-hook/nix-support/setup-hook: line 213: echo: write error: Broken pipe
/nix/store/71d65fplq44y9yn2fvkpn2d3hrszracd-auto-patchelf-hook/nix-support/setup-hook: line 210: echo: write error: Broken pipe
```

The worst is that derivation continued and resulted into broken package:
https://github.com/NixOS/nixpkgs/pull/55566#issuecomment-470065690

I hope, replacing `grep -q` with `grep` will remove this race condition.
2019-03-20 14:57:59 +02:00
Jan Tojnar 0ea220b9bd
wrapGAppsHook: add package’s own modules to GIO_EXTRA_MODULES
When a package provides both executables and gio modules, it is quite
probable the executables will need those modules. wrapGAppsHook wraps
executables with GIO_EXTRA_MODULES picked up from dependencies
but forgets about the package being built. Let’s add to consideration.

Closes: https://github.com/NixOS/nixpkgs/issues/50254
2019-03-13 23:03:53 +01:00
Frederik Rietdijk 7257dedd7c Merge master into staging-next 2019-02-13 12:33:29 +01:00
deliciouslytyped ca1062ea24
Document undocumented argument to makeWrapper() 2019-02-11 21:45:02 +01:00
Alyssa Ross 21d991b1fd
Merge pull request #54578 from alyssais/gcj-darwin
gcj: fix build on Darwin
2019-02-08 16:54:35 +00:00
Bruce Toll bbb2f93cee wrap-gapps-hook.sh: only wrap links when required
Unless dontWrapGapps is set, the wrap-gapps-hook.sh will currently
wrap all executables (and symbolic links to executables) found under
the target directories: bin and libexec.

As a result, if a symbolic link in a target directory points to an
executable in a target directory, both will get wrapped.  This
causes an extra shell/exec when following the symbolic link,
as well as increasing the size of the final executable's environment.

To avoid wrapping a link to an already wrapped executable, this
commit splits the determination of what gets wrapped into two phases:

1. All binaries under the target directories are wrapped and logged
   with "Wrapping program ..."

2. All links to executables under the target directories are
   identified and checked to see if they reference an executable
   under one of the target directories.

   If yes, the required wrapping has already been performed on
   the associated binary (in phase 1), so no wrapping is done
   and "Not wrapping link: ... (already wrapped)" is logged.

   If no, the link points at an executable that hasn't been
   wrapped, so the link is wrapped and "Wrapping link: ..." is logged.

As an example, the yelp package has a bin directory that contains
an executable "yelp" and a symbolic link "gnome-help" -> "yelp".

Prior to this commit, the bin directory would contain these files
after wrapping:

  gnome-help          -- wrapper to exec .gnome-help-wrapped
  .gnome-help-wrapped -- a symbolic link to yelp
  yelp                -- wrapper to exec .yelp-wrapped
  .yelp-wrapped       -- the original yelp binary

After this commit, the bin directory will instead contain:

  gnome-help          -- a symbolic link to yelp
  yelp                -- wrapper to exec .yelp-wrapped
  .yelp-wrapped       -- the original yelp binary

NOTE: The primary motivation for this commit is to avoid obscuring
the fact that two or more paths are simple aliases and expected to
behave identically. It also reduces the likelihood of hitting
limits related to environment variable size.

LIMITATION: The method used above is intended to be conservative
and will still wrap symbolic links to other symbolic links when
the ultimate target is outside of bin or libexec.
2019-01-31 20:14:30 -05:00
Alyssa Ross b9b83748f3
gcj: fix build on Darwin
Partially addresses https://github.com/NixOS/nixpkgs/issues/29715.
2019-01-30 18:19:17 +00:00
Piotr Bogdan 93545a8910 pruneLibtoolFiles: check if prefix exists 2018-12-09 22:44:29 +00:00
Piotr Bogdan 9cd155f1e2 pruneLibtoolFiles: more permissive regex for detecting libtool files 2018-12-09 00:13:04 +00:00
Bjørn Forsman ad28261ad2 audit-tmpdir hook: clean up whitespace 2018-12-02 11:45:34 +01:00
Bjørn Forsman ed0cee645f audit-tmpdir hook: use abspath matching on RPATH entries
This ensures that RPATH entries like "/foo/build/bar" doesn't trigger a
match when TMPDIR is "/build/bar". (I've had this problem with a
prebuilt package.)
2018-12-02 11:45:18 +01:00
John Ericson ed71691a81
Merge pull request #51217 from mayflower/source-date-epoch-fix-warning
set-source-date-epoch-to-latest: fix warning
2018-11-29 15:20:22 -05:00
Robin Gloster 34faed5bb0
set-source-date-epoch-to-latest: fix warning
If there was no older file than $NIX_BUILD_TOP this would result in a
warning, e.g. with nix-info.

```
/nix/store/15kgcm8hnd99p7plqzx7p4lcr2jni4df-set-source-date-epoch-to-latest.sh: line 13: [: : integer expression expected
```
2018-11-29 09:49:08 +01:00
Frederik Rietdijk 9db2421d1f Merge master into staging-next 2018-11-29 08:12:56 +01:00
aszlig 4a6e3e4185
autoPatchelfHook: Skip on missing segment headers
If the file in question is not a shared object file but an ELF, we
really want to skip the file, because we won't have anything to patch
there.

For example if the file is created via "gcc -c -o foo.o foo.c", we don't
get a segment header and so far autoPatchelf was trying to patch such a
file.

By checking for missing segment headers, we're now no longer going to
attempt patching such a file.

Signed-off-by: aszlig <aszlig@nix.build>
Reported-by: Sander van der Burg <svanderburg@gmail.com>
2018-11-26 01:58:36 +01:00
aszlig 9f23a63f79
autoPatchelfHook: Fix type of norecurse variable
While declaring it as an array doesn't do any harm in our usage, it
might be a bit confusing when reading the code.

Signed-off-by: aszlig <aszlig@nix.build>
2018-11-26 01:13:59 +01:00
aszlig 2faf905f98
autoPatchelfHook: Add addAutoPatchelfSearchPath
This function is useful if autoPatchelf is invoked during some of the
phases of a build and allows to add arbitrary shared objects to the
search path.

So far the same functionality was in autoPatchelf itself, but not
available as a separate function, so when adding shared objects to the
dependency cache one would have to do so manually.

The function also has the --no-recurse flag, which prevents recursing
into subdirectories.

Signed-off-by: aszlig <aszlig@nix.build>
2018-11-25 16:22:32 +01:00
aszlig 3ca35ce0b2
autoPatchelfHook: Add --no-recurse flag
This is to be used with the autoPatchelf command and allows to only
patch a specific file or directory without recursing into
subdirectories.

Apart from being able to run the command in a standalone way, as
detailled in the previous commit this is also needed for the Android SDK
emulator, because according to @svanderburg there are subdirectories we
don't want to patch.

The reason why I didn't use GNU getopt is that it might not be available
on all operating systems and the getopts bash builtin doesn't support
long arguments. Apart from that, the implementation for recognizing the
flag is pretty trivial and it's also using bash builtins only, so if we
want to do something really fancy someday, we can still change it.

Signed-off-by: aszlig <aszlig@nix.build>
2018-11-20 00:11:29 +01:00
aszlig e4fbb244ee
autoPatchelfHook: Allow to prevent automatic run
If you want to only run autoPatchelf on a specific path and leave
everything else alone, we now have a $dontAutoPatchelf environment
variable, which causes the postFixup hook to not run at all.

The name "dontAutoPatchelf" probably is a bit weird in conjunction with
putting "autoPatchelfHook" in nativeBuildInputs, but unless someone
comes up with a better name I keep it that way because it's consistent
with all the other dontStrip, dontPatchShebangs, dontPatchELF and
whatnot.

A specific example where this is needed is when building the Android SDK
emulator, which contains a few ARM binaries in subdirectories that
should not be patched. If we were to run autoPatchelf on all outputs
unconditionally we'd run into errors because some ARM libraries couldn't
be found.

Signed-off-by: aszlig <aszlig@nix.build>
2018-11-20 00:07:38 +01:00
aszlig d03e4ffdbf
autoPatchelfHook: Make easier to run autoPatchelf
The autoPatchelf main function which is run against all of the outputs
was pretty much tailored towards this specific setup-hook and was
relying on $prefix to be set globally.

So if you wanted to run autoPatchelf manually - let's say during
buildPhase - you would have needed to run it like this:

  prefix=/some/directory autoPatchelf

This is now more intuitive and all you need to do is run the following:

  autoPatchelf /some/directory

Signed-off-by: aszlig <aszlig@nix.build>
2018-11-19 17:18:27 +01:00
zimbatm 551aecfa83
tmpdir audit: only fail with files referenced below (#35068)
On Linux the `$TMPDIR` is `/build`. The TMPDIR audit looks for `$TMPDIR`
in the build output, which will then fail with packages like
/buildkite-agent.

This fixes the heuristic to look for `$TMPDIR/` instead.
2018-11-16 22:35:56 +01:00
Daiderd Jordan bdec3ed049
Revert "Revert "Revert "patch-shebangs: respect cross compilation"""
Completely breaks darwin. Every package in the stdenv that has shebangs
in the output will end up with references to bootstrap-tools.

This reverts commit bde99096a8.
2018-11-14 23:37:32 +01:00
Daiderd Jordan c9223a17bc
Revert "patch-shebangs: use --build for auto patch shebangs"
Completely breaks darwin. Every package in the stdenv that has shebangs
in the output will end up with references to bootstrap-tools.

This reverts commit eb7c50a993.
2018-11-14 23:37:31 +01:00
Matthew Bauer f9a6963d9a
Merge pull request #50244 from tathougies/travis/wrap-correctly
make-wrapper should use runtimeShell, not $SHELL, for cross-compilation
2018-11-13 13:55:26 -06:00
Travis Athougies 9531a32b60 make-wrapper should use runtimeShell, not bash, for cross-compilation 2018-11-11 10:25:05 -08:00
Matthew Bauer c8aff96110
Merge pull request #49608 from matthewbauer/cross-patch-shebangs-2
Restore cross-patch-shebangs branch
2018-11-07 13:37:02 -06:00
aszlig c64624b843
autoPatchelfHook: Correctly detect PIE binaries
I originally thought it would just be enough to just check for an INTERP
section in isExecutable, however this would mean that we don't detect
statically linked ELF files, which would break our recent improvement to
gracefully handle those.

In theory, we are only interested in ELF files that have an INTERP
section, so checking for INTERP would be enough. Unfortunately the
isExecutable function is already used outside of autoPatchelfHook, so we
can't easily get rid of it now, so let's actually strive for more
correctness and make isExecutable actually match ELF files that are
executable.

So what we're doing instead now is to check whether either the ELF type
is EXEC *or* we have an INTERP section and if one of them is true we
should have an ELF executable, even if it's statically linked.

Along the way I also set LANG=C for the invocations of readelf, just to
be sure we don't get locale-dependent output.

Tested this with the following command (which contains almost[1] all the
packages using autoPatchelfHook), checking whether we run into any
library-related errors:

  nix-build -E 'with import ./. { config.allowUnfree = true; };
    runCommand "test-executables" {
      drvs = [
        anydesk cups-kyodialog3 elasticsearch franz gurobi
        masterpdfeditor oracle-instantclient powershell reaper
        sourcetrail teamviewer unixODBCDrivers.msodbcsql17 virtlyst
        vk-messenger wavebox zoom-us
      ];
    } ("for i in $drvs; do for b in $i/bin/*; do " +
       "[ -x \"$b\" ] && timeout 10 \"$b\" || :; done; done")
  '

Apart from testing against library-related errors I also compared the
resulting store paths against the ones prior to this commit. Only
anydesk and virtlyst had the same as they didn't have self-references,
everything else differed only because of self-references, except
elasticsearch, which had the following PIE binaries:

  * modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/autoconfig
  * modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/autodetect
  * modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/categorize
  * modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/controller
  * modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/normalize

These binaries were now patched, which is what this commit is all about.

[1]: I didn't include the "maxx" package (MaXX Interactive Desktop)
     because the upstream URLs are no longer existing and I couldn't
     find them elsewhere on the web.

Signed-off-by: aszlig <aszlig@nix.build>
Fixes: https://github.com/NixOS/nixpkgs/issues/48330
Cc: @gnidorah (for MaXX Interactive Desktop)
2018-11-03 08:07:42 +01:00
Matthew Bauer eb7c50a993 patch-shebangs: use --build for auto patch shebangs
In strictDeps=false, autoPatchshebangs should use
--build (corresponding to PATH) to lookup commands. This restores the
previous behavior of patchshebangs so that we don’t break stuff that
isn’t careful in the buildInputs vs. nativeBuildInputs distinction.
Unfortunately this won’t work under cross compilation.
2018-11-02 00:27:14 -05:00
Matthew Bauer bde99096a8 Revert "Revert "patch-shebangs: respect cross compilation""
This reverts commit 9c4b11e9a0.
2018-11-02 00:27:14 -05:00
Jörg Thalheim f10b935f84
breakpointHook: add for debugging failing builds
Usuage: Add breakpointHook to your `buildInputs` like this:

  stdenv.mkDerivation rec {
    # ...
    buildInputs = [ breakpointHook ];
  });

When the build fails as show in this example:

  pkgs.hello.overrideAttrs (old: {
    buildInputs = [ breakpointHook ];
    postPatch = ''
      false
    '';
  });

It will halt execution printing the following message:

build failed in patchPhase with exit code 1
To attach to this build run the following command as root:

   cntr attach -t command cntr-/nix/store/ynyb4n82x2r7sldd58pbb405jdqh5f00-hello-2.10

Installing cntr and running the command will provide shell access to the
build sandbox of failed build:

sudo cntr attach -t command cntr-/nix/store/ynyb4n82x2r7sldd58pbb405jdqh5f00-hello-2.10
WARNING: bad ownership on /nix/var/nix/profiles/per-user/root, should be 1000
[nixbld@localhost:/var/lib/cntr]$

At /var/lib/cntr the sandbox filesystem is mounted. All commands and
files of the system are still accessible within the shell.
To execute commands from the sandbox use the `cntr exec` subcommand.
2018-10-25 10:19:41 +01:00
Will Dietz f7db287960 patch-shebangs.sh: use more robust 'for each file' loop, check for dir
The latter is to avoid warnings printed by find if it doesn't exist.
2018-09-28 11:21:51 -05:00
Will Dietz 286381f072 patch-shebangs: simplify a bit per reviewer suggestion 2018-09-28 11:17:33 -05:00
Will Dietz 830f9fabd4 patch-shebangs: use isScript to safely check for shebang start
Fixes commonly encountered errors about broken pipes or null-bytes in
command-substitution.
2018-09-28 11:15:36 -05:00
Daiderd Jordan 9c4b11e9a0
Revert "patch-shebangs: respect cross compilation"
This causes problems for packages built using a bootstrap stdenv,
resulting in references to /bin/sh or even bootstrap-tools. The darwin
stdenv is much stricter about what requisites/references are allowed but
using /bin/sh on linux is also undesirable.

eg. https://hydra.nixos.org/build/81754896

    $ nix-build -A xz
    $ head -n1 result-bin/bin/xzdiff
    #!/nix/store/yvc7kmw98kq547bnqn1afgyxm8mxdwhp-bootstrap-tools/bin/sh

This reverts commit f06942327a.
2018-09-25 21:11:54 +02:00
John Ericson c701d6cb21 Merge remote-tracking branch 'upstream/master' into staging 2018-09-25 14:20:27 -04:00