Commit graph

3106 commits

Author SHA1 Message Date
Frederik Rietdijk eeaf3a131f Merge master into staging-next 2018-10-09 15:37:22 +02:00
Linus Heckemann 9cc18fa7f9 debian vm tools: use snapshot.debian.org
snapshot.debian.org actually keeps track of all of the updates as they
come in rather than doing arbitrary (?) snapshots.
2018-10-08 18:05:09 +02:00
Justin Humm 64d0676fe1
buildRustPackage: fix regex for separating lib and bin
E.g. exa was wrongly put into /lib, as it matches

  .*.a

but not

  .*\.a
2018-10-07 22:14:19 +02:00
Sarah Brofeldt 2e38f5fc6e
Merge pull request #47448 from kalbasit/nixpkgs_add-bazel-watcher
bazel-watcher: init at 0.5.0
2018-10-04 00:00:47 +02:00
Edward Tate 6ad43a0bce
buildRustPackage now correctly installs binaries to bin and libraries to lib. 2018-10-03 16:27:10 +02:00
Samuel Leathers 024eb9a5a5 trivial builders: adding usage documentation for functions 2018-10-02 22:09:09 +02:00
Frederik Rietdijk 6ce04af137 Merge master into staging 2018-10-02 18:22:37 +02:00
Daiderd Jordan 1383c08f2c
Merge branch 'master' into staging-next 2018-10-01 19:42:07 +02:00
Sarah Brofeldt b256df4937 dockerTools: Use nix instead of nixUnstable 2018-10-01 09:51:52 +02:00
lewo 56b4db9710
Merge pull request #47411 from graham-at-target/multi-layered-images-crafted
Multi-Layered Docker Images
2018-10-01 09:48:24 +02:00
Wael M. Nasreddine 86a5535b2f
bazel-watcher: init at 0.5.0 2018-09-29 13:33:00 -07:00
Wael M. Nasreddine 90b7b4a509
build-bazel-package: remove any .git, .svn and .hg from external 2018-09-29 13:28:15 -07:00
Wael M. Nasreddine 18aa9b0b65
build-bazel-package: prefix bazel with the USER variable
Bazel computes the default value of output_user_root before parsing the
flag[0]. The computation of the default value involves getting the $USER
from the environment. I don't have that variable when building with
sandbox enabled.

[0]: 9323c57607/src/main/cpp/startup_options.cc (L123-L124)
2018-09-29 13:28:12 -07:00
Will Dietz f7db287960 patch-shebangs.sh: use more robust 'for each file' loop, check for dir
The latter is to avoid warnings printed by find if it doesn't exist.
2018-09-28 11:21:51 -05:00
Will Dietz 286381f072 patch-shebangs: simplify a bit per reviewer suggestion 2018-09-28 11:17:33 -05:00
Will Dietz 830f9fabd4 patch-shebangs: use isScript to safely check for shebang start
Fixes commonly encountered errors about broken pipes or null-bytes in
command-substitution.
2018-09-28 11:15:36 -05:00
Graham Christensen fb2d153dac
dockerTools: test buildLayeredImage 2018-09-27 14:19:43 -04:00
Graham Christensen 4fe9006190 dockerTools.buildLayeredImage: init
Create a many-layered Docker Image.

Implements much less than buildImage:

 - Doesn't support specific uids/gids
 - Doesn't support runninng commands after building
 - Doesn't require qemu
 - Doesn't create mutable copies of the files in the path
 - Doesn't support parent images

If you want those feature, I recommend using buildLayeredImage as an
input to buildImage.

Notably, it does support:

 - Caching low level, common paths based on a graph traversial
   algorithm, see referencesByPopularity in
   0a80233487993256e811f566b1c80a40394c03d6
 - Configurable number of layers. If you're not using AUFS or not
   extending the image, you can specify a larger number of layers at
   build time:

       pkgs.dockerTools.buildLayeredImage {
         name = "hello";
         maxLayers = 128;
         config.Cmd = [ "${pkgs.gitFull}/bin/git" ];
       };

 - Parallelized creation of the layers, improving build speed.
 - The contents of the image includes the closure of the configuration,
   so you don't have to specify paths in contents and config.

   With buildImage, paths referred to by the config were not included
   automatically in the image. Thus, if you wanted to call Git, you
   had to specify it twice:

       pkgs.dockerTools.buildImage {
         name = "hello";
         contents = [ pkgs.gitFull ];
         config.Cmd = [ "${pkgs.gitFull}/bin/git" ];
       };

   buildLayeredImage on the other hand includes the runtime closure of
   the config when calculating the contents of the image:

       pkgs.dockerTools.buildImage {
         name = "hello";
         config.Cmd = [ "${pkgs.gitFull}/bin/git" ];
       };

Minor Problems

 - If any of the store paths change, every layer will be rebuilt in
   the nix-build. However, beacuse the layers are bit-for-bit
   reproducable, when these images are loaded in to Docker they will
   match existing layers and not be imported or uploaded twice.

Common Questions

 - Aren't Docker layers ordered?

   No. People who have used a Dockerfile before assume Docker's
   Layers are inherently ordered. However, this is not true -- Docker
   layers are content-addressable and are not explicitly layered until
   they are composed in to an Image.

 - What happens if I have more than maxLayers of store paths?

   The first (maxLayers-2) most "popular" paths will have their own
   individual layers, then layer #(maxLayers-1) will contain all the
   remaining "unpopular" paths, and finally layer #(maxLayers) will
   contain the Image configuration.
2018-09-26 17:54:14 -04:00
Graham Christensen fd045173ce referencesByPopularity: init to sort packages by a cachability heuristic
Using a simple algorithm, convert the references to a path in to a
sorted list of dependent paths based on how often they're referenced
and how deep in the tree they live. Equally-"popular" paths are then
sorted by name.

The existing writeReferencesToFile prints the paths in a simple
ascii-based sorting of the paths.

Sorting the paths by graph improves the chances that the difference
between two builds appear near the end of the list, instead of near
the beginning. This makes a difference for Nix builds which export a
closure for another program to consume, if that program implements its
own level of binary diffing.

For an example, Docker Images. If each store path is a separate layer
then Docker Images can be very efficiently transfered between systems,
and we get very good cache reuse between images built with the same
version of Nixpkgs. However, since Docker only reliably supports a
small number of layers (42) it is important to pick the individual
layers carefully. By storing very popular store paths in the first 40
layers, we improve the chances that the next Docker image will share
many of those layers.*

Given the dependency tree:

    A - B - C - D -\
     \   \   \      \
      \   \   \      \
       \   \ - E ---- F
        \- G

Nodes which have multiple references are duplicated:

    A - B - C - D - F
     \   \   \
      \   \   \- E - F
       \   \
        \   \- E - F
         \
          \- G

Each leaf node is now replaced by a counter defaulted to 1:

    A - B - C - D - (F:1)
     \   \   \
      \   \   \- E - (F:1)
       \   \
        \   \- E - (F:1)
         \
          \- (G:1)

Then each leaf counter is merged with its parent node, replacing the
parent node with a counter of 1, and each existing counter being
incremented by 1. That is to say `- D - (F:1)` becomes `- (D:1, F:2)`:

    A - B - C - (D:1, F:2)
     \   \   \
      \   \   \- (E:1, F:2)
       \   \
        \   \- (E:1, F:2)
         \
          \- (G:1)

Then each leaf counter is merged with its parent node again, merging
any counters, then incrementing each:

    A - B - (C:1, D:2, E:2, F:5)
     \   \
      \   \- (E:1, F:2)
       \
        \- (G:1)

And again:

    A - (B:1, C:2, D:3, E:4, F:8)
     \
      \- (G:1)

And again:

    (A:1, B:2, C:3, D:4, E:5, F:9, G:2)

and then paths have the following "popularity":

    A     1
    B     2
    C     3
    D     4
    E     5
    F     9
    G     2

and the popularity contest would result in the paths being printed as:

    F
    E
    D
    C
    B
    G
    A

* Note: People who have used a Dockerfile before assume Docker's
Layers are inherently ordered. However, this is not true -- Docker
layers are content-addressable and are not explicitly layered until
they are composed in to an Image.
2018-09-26 15:50:10 -04:00
Graham Christensen 2bf0ee3b2b dockertools: tarsum: turn in to a buildInput 2018-09-26 15:50:04 -04:00
Daiderd Jordan 9c4b11e9a0
Revert "patch-shebangs: respect cross compilation"
This causes problems for packages built using a bootstrap stdenv,
resulting in references to /bin/sh or even bootstrap-tools. The darwin
stdenv is much stricter about what requisites/references are allowed but
using /bin/sh on linux is also undesirable.

eg. https://hydra.nixos.org/build/81754896

    $ nix-build -A xz
    $ head -n1 result-bin/bin/xzdiff
    #!/nix/store/yvc7kmw98kq547bnqn1afgyxm8mxdwhp-bootstrap-tools/bin/sh

This reverts commit f06942327a.
2018-09-25 21:11:54 +02:00
John Ericson c701d6cb21 Merge remote-tracking branch 'upstream/master' into staging 2018-09-25 14:20:27 -04:00
Sarah Brofeldt ded8f28c3a Revert "virtualization/qemu-vm: fix and improve virtio/scsi switching"
This reverts commit f777d2b719.
cc #34409
This breaks evaluation of the tested job:
attribute 'diskInterface' missing, at /nix/store/5k9kk52bv6zsvsyyvpxhm8xmwyn2yjvx-source/pkgs/build-support/vm/default.nix:316:24
2018-09-25 11:10:10 +02:00
John Ericson 2b4b7d4ef3
Merge pull request #47233 from oxij/tree/mass-rebuild-noop-cleanups
treewide: mass rebuild noop cleanups
2018-09-25 00:04:52 -04:00
aszlig 19e83bc2ba
Merge autoPatchelfHook improvements (#47222)
This includes the initialy commit was done by @Mic92 plus a few fixes
from my side. So essentially this avoids patching statically linked
executables and also speeds up searching for ELF files altogether.

I've tested this by comparing the outputs of all the derivations which
make use of this hook using the following Nix expression:

  let
    getPackagesForRev = rev: with import (builtins.fetchGit {
      url = ./.;
      inherit rev;
    }) { config.allowUnfree = true; }; [
      cups-kyodialog3 elasticsearch franz gurobi javacard-devkit
      masterpdfeditor maxx oracle-instantclient powershell reaper
      teamviewer unixODBCDrivers.msodbcsql17 virtlyst wavebox zoom-us
    ];

    pkgs = import <nixpkgs> {};
    baseRev = "ef764eb0d8314b81a012dae04642b4766199956d";

  in pkgs.runCommand "diff-contents" {
    chset = pkgs.lib.zipListsWith (old: new: pkgs.runCommand "diff" {
      inherit old new;
      nativeBuildInputs = [ pkgs.nukeReferences ];
    } ''
      mkdir -p "''${NIX_STORE#/}"
      cp --no-preserve=all -r "$old" "''${NIX_STORE#/}"
      cp --no-preserve=all -r "$new" "''${NIX_STORE#/}"
      find "''${old#/}" "''${new#/}" \
        \( -type f -exec nuke-refs {} + \) -o \( -type l -delete \)
      mkdir "$out"
      echo "$old" > "$out/old-path"
      echo "$new" > "$out/new-path"
      diff -Nur "''${old#/}" "''${new#/}" > "$out/diff" || :
    '') (getPackagesForRev baseRev) (getPackagesForRev "");
  } ''
    err=0
    for c in $chset; do
      if [ -s "$c/diff" ]; then
        echo "$(< "$c/old-path") -> $(< "$c/new-path")" \
             "differs, report: $c/diff" >&2
        err=1
      fi
    done
    [ $err -eq 0 ] && touch "$out"
  ''

With these changes there is only one derivation which has altered
contents, which is "franz". However the reason why it has differing
contents is not directly because of the autoPatchelfHook changes, but
because the "env-vars" file from the builder is in
"$out/opt/franz/env-vars" (Cc: @gnidorah) and we now have different
contents for NIX_CFLAGS_COMPILE and other environment variables.

I also tested this against a random static binary and the hook no longer
tries to patch it.

Merges: #47222
2018-09-25 05:21:01 +02:00
aszlig b4526040a2
autoPatchelfHook: Silence errors in isExecutable
The "maxx" package recursively runs isExecutable on a bunch of files and
since the change to use "readelf" instead of "file" a lot of errors like
this one are printed during build:

  readelf: Error: Not an ELF file - it has the wrong magic bytes at the
  start

While the isExecutable was never meant to be used outside of the
autoPatchelfHook, it's still a good idea to silence the errors because
whenever readelf fails, it clearly indicates that the file in question
is not a valid ELF file.

Signed-off-by: aszlig <aszlig@nix.build>
2018-09-25 04:48:12 +02:00
aszlig 9920215d00
autoPatchelfHook: Only check PT_INTERP on execs
If the ELF file is not an executable, we do not get a PT_INTERP section,
because after all, it's a *shared* library.

So instead of checking for PT_INTERP (to avoid statically linked
executables) for all ELF files, we add another check to see if it's an
executable and *only* skip it when it is and there's no PT_INTERP.

Signed-off-by: aszlig <aszlig@nix.build>
2018-09-25 04:42:34 +02:00
John Ericson 22ce614112
Merge pull request #47238 from obsidiansystems/overrideScope-order
lib: Deprecate `overrideScope` in lieu of `overrideScope'` taking arguments in the conventional order
2018-09-24 18:04:18 -04:00
John Ericson b9dce11712 lib: Make overrideScope' which takes arguments in the conventional order
The `overrideScope` bound by `makeScope` (via special `callPackage`)
took an override in the form `super: self { … }`. But this is
dangerously close to the `self: super { … }` form used by *everything*
else, even other definitions of `overrideScope`! Since that
implementation did not even share any code either until I changed it
recently in 3cf43547f4, this inconsistency
is almost certainly an oversight and not intentional.

Unfortunately, just as the inconstency is hard to debug if one just
assumes the conventional order, any sudden fix would break existing
overrides in the same hard-to-debug way. So instead of changing the
definition a new `overrideScope'` with the conventional order is added,
and old `overrideScope` deprecated with a warning saying to use
`overrideScope'` instead. That will hopefully get people to stop using
`overrideScope`, freeing our hand to change or remove it in the future.
2018-09-24 17:50:11 -04:00
Vladimír Čunát 3a09a4b579
Merge branch 'master' into staging
Hydra: ?compare=1480463
2018-09-24 21:18:23 +02:00
Jörg Thalheim 58a97dfb49 autoPatchelfHook: do not patch statically linked files
Also speed up quite significantly due less forking.
2018-09-23 21:33:43 +01:00
Jan Malakhovski b2c7a5a271 bintools-wrapper, cc-wrapper, stdenv: infer propagateDoc automatically
02c09e0171 (NixOS/nixpkgs#44558) was reverted in
c981787db9 but, as it turns out, it fixed an issue
I didn't know about at the time: the values of `propagateDoc` options were
(and now again are) inconsistent with the underlying things those wrappers wrap
(see NixOS/nixpkgs#46119), which was (and now is) likely to produce more instances
of NixOS/nixpkgs#43547, if not now, then eventually as stdenv changes.

This patch (which is a simplified version of the original reverted patch) is the
simplest solution to this whole thing: it forces wrappers to directly inspect the
outputs of the things they are wrapping instead of making stdenv guess the correct
values.
2018-09-23 17:29:56 +00:00
Frederik Rietdijk 14673d61ed Merge master into staging 2018-09-23 09:33:09 +02:00
Stefan Junker f777d2b719 virtualization/qemu-vm: fix and improve virtio/scsi switching 2018-09-22 23:29:19 +02:00
John Ericson aba5f4974a
Merge pull request #47145 from obsidiansystems/fetchzip-cross
fetchzip: Use unzip from buildPackages
2018-09-21 14:52:17 -04:00
John Ericson b97242238d fetchzip: Use unzip from buildPackages
Additionally, the manual path manipulation becomes no longer needed.
2018-09-21 12:55:32 -04:00
Graham Christensen aedc651903
dockerTools.buildImage: test that created=now makes an unstable date 2018-09-20 13:06:14 -04:00
Graham Christensen a32d7e0c74 dockerTools.buildImage: support impure dates
Because dates are an impurity, by default buildImage will use a static
date of one second past the UNIX Epoch. This can be a bit frustrating
when listing docker images in the CLI:

    $ docker image list
    REPOSITORY   TAG      IMAGE ID       CREATED        SIZE
    hello        latest   08c791c7846e   48 years ago   25.2MB

If you want to trade the purity for a better user experience, you can
set created to now.

    pkgs.dockerTools.buildImage {
      name = "hello";
      tag = "latest";
      created = "now";
      contents = pkgs.hello;

      config.Cmd = [ "/bin/hello" ];
    }

and now the Docker CLI will display a reasonable date and sort the
images as expected:

    $ docker image list
    REPOSITORY   TAG      IMAGE ID       CREATED              SIZE
    hello        latest   de2bf4786de6   About a minute ago   25.2MB
2018-09-20 18:26:02 +02:00
Jack Kelly af5eab6ea6 dockerTools.pullImage: correct default arch 2018-09-19 16:13:15 +10:00
John Ericson 7319013ea1 Merge remote-tracking branch 'upstream/master' into staging 2018-09-18 16:55:42 -04:00
Silvan Mosberger 50578abfc5
fetchcargo: Fix cargo-vendor-normalise for darwin 2018-09-17 20:23:50 +02:00
Andreas Rammhold fc5e595003
buildRustCrate: added some edge cases with binaries
This commit adds test based on real-world crates (brotli).
There were a few more edge cases that were missing beforehand. Also it
turned out that we can get rid of the `finalBins` list since that will
now be handled during runtime.
2018-09-13 22:00:29 +02:00
Andreas Rammhold 0f95d05548 buildRustCrate: add test cases 2018-09-13 20:28:39 +02:00
Andreas Rammhold fdc2017f1c buildRustCrate: binary heuristic should be able to treat spaces 2018-09-13 20:28:39 +02:00
Andreas Rammhold 1371815060 buildRustCrate: extracted builder scripts into dedicated files
The build expression got quiet large over time and to make it a bit
easier to grasp the different scripts involved in the build are now
separated from the nix file.
2018-09-13 20:28:39 +02:00
Andreas Rammhold 0c50140da5 buildRustCrate: add heuristic to picking the right source files
Cargo has a few odd (old) ways of picking source files if the `bin.path`
attribute isn't given in the Cargo.toml. This commit adds support for
some of those. The previous behaviour always defaulted to `src/main.rs`
which was not always the right choice.

Since there is  look-ahead into the unpacked sources before running the
actual builder the path selection logic has to be embedded within the
build script.

`buildRustCrate` currently supports two ways of running building
binaries when processing a crate:

- Explicit definition of all the binaries (& optionally the paths to
their respective `main.rs`) and,
- if not binary was explictly configured all files matching the patterns
  `src/main.rs`, `src/bin/*.rs`.

When the explicit list is given without path information paths are now
being picked from a list of candidates. The first match wins. The order
is the same as within the cargo compatibility code.

If the crate does not provide any libraries the path `src/{bin_name}.rs`
is also considered.

All underscores within the binary names are translated into dashes (`-`)
before the lookups are made. This seems to be a common convention.
2018-09-13 20:28:39 +02:00
Andreas Rammhold 87462d6be4
vmTools: update debian repositories to stable Release.xz urls
Previously the Release.xz URL would show up with a new hash whenever
debian releases an update. By using archive.org we should have a stable
source for those. I wasn't able to find the equivalent in the debian
world. Maybe they don't keep all the different Release files around..
2018-09-13 11:10:23 +02:00
Symphorien Gibol a3e1da17cb cargo-vendor-normalise: add a small install check 2018-09-11 23:44:14 +02:00
Jörg Thalheim 7bfa20198a fetchcargo: add type checking to cargo-vendor-normalise.py 2018-09-11 23:44:14 +02:00
Symphorien Gibol f20b229aa1 fectchcargo: don't break old sha256 2018-09-11 23:44:14 +02:00
Symphorien Gibol ccf72b8537 fetchcargo: normalise cargo config to ensure determinism 2018-09-11 23:44:14 +02:00
Justin Humm b66ef28841 buildRustPackage, fetchcargo: optionally use vendor config from cargo-vendor
By setting useRealVendorConfig explicitly to true, the actual (slightly
modified) config generated by cargo-vendor is used.

This solves a problem, where the static vendor config in
pkgs/build-support/rust/default.nix would not sufficiently replace all
crates Cargo is looking for.

As useRealVendorConfig (and writeVendorConfig in fetchcargo) default to
false, there should be no breakage in existing cargoSha256 hashes.

Nethertheless, imho using this new feature should become standard. A
possible deprecation path could be:

- introduce this patch
- set useRealVendorConfig explicitly to false whereever cargoSha256 is
  set but migration is not wanted yet.
- after some time, let writeVendorConfig default to true
- when useRealVendorConfig is true everywhere cargoSha256 is set and
  enough time is passed, `assert cargoVendorDir == null ->
  useRealVendorConfig;`, remove old behaviour
- after some time, remove all appearences of useRealVendorConfig and the
  parameter itself
2018-09-11 23:44:14 +02:00
Matthew Bauer e258c8d8dd
Merge pull request #45698 from kamilchm/filename-in-usnupported-shebang-error
Show the filename on unsupported shebang error
2018-09-10 14:52:38 -05:00
Jan Malakhovski b7bd0561be Merge branch 'master' into staging 2018-09-08 22:08:32 +00:00
Shea Levy 18337f3ece
Merge branch 'no-toPath' 2018-09-06 08:09:53 -04:00
Profpatsch 4616ef1f41 skaware: switch from git repos to tarballs
It should be more performant this way.
2018-09-06 11:53:22 +02:00
Profpatsch 0071ae1d4f skawarePackages: factor out the common parts
Introduce a `skawarePackages.buildPackage` function that contains the
common setup, removing a lot of duplication.
In particular, we require that the build directory has to be empty
after the `fixupPhase`, to make sure every relevant file is moved to
the outputs.

A next step would be to deduplicate the `configureFlags` attributes
and only require a `skawareInputs` field.
2018-09-06 11:53:22 +02:00
Matthew Bauer a6877783f6
Merge pull request #43833 from matthewbauer/cross-patch-shebangs
patch-shebangs: respect cross compilation
2018-09-05 14:52:43 -05:00
Jan Malakhovski 4092708261 treewide: cleanup some references to bash 2018-09-04 22:05:02 +00:00
Alyssa Ross 4af7278bc9 lib: ensure directories of linkFarm links exist (#45628)
There's no reason `linkFarm` can't be used for symlinks in
subdirectories, except that currently it doesn't ensure the directory
of the link exists. This backwards-compatible change expands the utility
of the function.
2018-09-01 14:53:23 +02:00
John Ericson 0828e2d8c3 treewide: Remove usage of remaining redundant platform compatability stuff
Want to get this out of here for 18.09, so it can be deprecated
thereafter.
2018-08-30 17:20:32 -04:00
John Ericson 2c2f1e37d4 reewide: Purge all uses stdenv.system and top-level system
It is deprecated and will be removed after 18.09.
2018-08-30 17:20:32 -04:00
Kamil Chmielewski 383de74f88 patch-shebangs: filename on unsupported shebang
Show the filename on unsupported shebang error.
Simplifies debugging packages with large set of scripts.
2018-08-29 21:47:27 +02:00
Samuel Dionne-Riel ba9db083ac
Merge pull request #26416 from lverns/make-genericName-optional
make-desktopitem: make genericName optional
2018-08-27 19:29:13 -04:00
Alyssa Ross f4745bef6c makeWrapper: document --set-default 2018-08-24 19:46:16 +02:00
Vladimír Čunát e78fd23564
Merge branch 'master' into staging
Hydra: ?compare=1474932
2018-08-22 20:57:14 +02:00
Matthew Bauer 13c8acc3db Revert "Merge pull request #44767 from obsidiansystems/wrapper-env-var-path"
This reverts commit 89efc27f57, reversing
changes made to d0f11020ca.
2018-08-22 01:14:53 +02:00
volth caf2cae44b fetchpatch: patchutils -> buildPackages.patchutils 2018-08-21 17:21:25 -04:00
Matthew Bauer 379fc894de Merge remote-tracking branch 'origin/master' into staging 2018-08-21 15:41:53 -05:00
Will Dietz 9c35796ee3
Merge pull request #45396 from dtzWill/fix/audit-tmpdir-quote
audit-tmpdir: fix processing of files with spaces, quote variables
2018-08-21 15:21:36 -05:00
CrystalGamma 72d161f548 [RFC] ppc64le enablement (#45340)
* ppc64le enablement

* gcc, glibc: properly handle __float128

* lib/systems, stdenv: syntax cleanup

* gcc7: remove ugly hack

* gcc: add/update __float128 flags

* stdenv: add another pair of quotes for consistency

* gcc: move __float128 flag for ppc64le-glibc into common/platform-flags.nix
2018-08-21 15:31:34 -04:00
Vladimír Čunát 765d695b89
Merge branch 'staging-next'
Security fixes for a few packages are included.
2018-08-21 15:36:02 +02:00
John Ericson 7d85ade0cc treewide: Purge stdenv.platform and top-level platform
Progress towards #27069
2018-08-20 15:22:46 -04:00
Will Dietz 79a86ca11c audit-tmpdir: fix processing of files with spaces, quote variables 2018-08-20 13:08:32 -05:00
John Ericson 89efc27f57
Merge pull request #44767 from obsidiansystems/wrapper-env-var-path
{cc,bintools}-wrapper, ghc, libgcc: Define wrapper env vars as full paths
2018-08-17 16:12:26 -04:00
Vladimír Čunát 14aa936ec5
Merge branch 'staging-next' into staging 2018-08-17 20:53:27 +02:00
Vladimír Čunát cbabebcc2e
Merge branch 'master' into staging-next
Hydra: ?compare=1473892
2018-08-17 13:45:21 +02:00
Stewart Mackenzie efac36aa88 carnix overrides: add gmp to rink-rs buildInputs & correct crateBin 2018-08-15 13:20:04 +08:00
Jörg Thalheim 78777fbd6b
Merge pull request #44981 from Ekleog/rust-patch-bis
buildRustPackage: allow patches to fix Cargo.lock
2018-08-14 14:23:51 +02:00
Vladimír Čunát ded9a4b1aa
Merge branch 'staging-next' into staging
Commits from master, conflict resolutions, etc.
2018-08-13 20:57:50 +02:00
Léo Gaspard 48e5fbe8ee
buildRustPackage: allow patches to fix Cargo.lock 2018-08-13 22:07:58 +09:00
Jörg Thalheim cfff3eb6c4 Revert "buildRustPackage: allow patches to fix Cargo.lock"
This reverts commit b6e881ab72.

We need to fix checksums for this pull request first.

Also see https://github.com/NixOS/nixpkgs/pull/44967
2018-08-13 12:26:33 +02:00
Jörg Thalheim 3dc78e6ae9
Merge pull request #44967 from Ekleog/rust-patch
buildRustPackage: allow patches to fix Cargo.lock
2018-08-13 12:22:59 +02:00
Léo Gaspard b6e881ab72 buildRustPackage: allow patches to fix Cargo.lock 2018-08-13 14:44:30 +09:00
Vladimír Čunát 00df25ee57
Merge branch 'master' into staging-next
Hydra: ?compare=1472947
2018-08-12 10:33:41 +02:00
xeji 12eb1e96ce
Merge pull request #44870 from symphorien/tests-eval
Fix evaluation of two tests
2018-08-10 19:06:20 +02:00
Symphorien Gibol 0d13dc3654 nixos/tests/hocker-fetchdocker: fix evaluation
it still does not build
2018-08-10 15:22:47 +02:00
Jan Malakhovski b135329dc5 treewide: random cleanups 2018-08-10 12:56:31 +00:00
Eelco Dolstra fde7296a47
bintools-wrapper: propagated-build-inputs -> propagated-user-env-packages 2018-08-09 13:07:14 +02:00
Eelco Dolstra fa41297209
Revert "cc-wrapper: propagate man and info to propagated-build-inputs"
This reverts commit 28ad0703f3.
2018-08-09 12:58:16 +02:00
Eelco Dolstra c981787db9
Revert "cc-wrapper, bintools-wrapper: simply symlink man and info outputs"
This reverts commit 02c09e0171.
2018-08-09 12:57:38 +02:00
John Ericson 044a73bbe6 bintools-wrapper: Define env vars with full path 2018-08-08 17:16:15 -04:00
John Ericson 1dc0404d6e cc-wrapper: Define env vars with full path 2018-08-06 20:38:13 -04:00
Jan Malakhovski 02c09e0171 cc-wrapper, bintools-wrapper: simply symlink man and info outputs
See discussion in #44516.
2018-08-06 20:50:16 +00:00
Jan Malakhovski 28ad0703f3 cc-wrapper: propagate man and info to propagated-build-inputs 2018-08-05 19:20:02 +00:00
Matthew Bauer f06942327a patch-shebangs: respect cross compilation
This hopefully makes patchShebangs respect cross compilation. It
introduces the concept of the HOST_PATH. Nothing is ever executed on
it but instead used as a way to get the proper path using ‘command
-v’. Needs more testing.

/cc @ericson2314 @dtzwill

Fixes #33956
Fixes #21138
2018-07-31 15:38:35 -04:00
Vladimír Čunát 73959b68c2
Re-Revert "Merge #44221: default for NIX_CXXSTDLIB_COMPILE"
This reverts commit fd81a2ecb6.
Moved from master to staging.
2018-07-31 09:48:16 +02:00
Vladimír Čunát fd81a2ecb6
Revert "Merge #44221: default for NIX_CXXSTDLIB_COMPILE"
This reverts commit 034c9816d5, reversing
changes made to 5afe87ed7a.
Huge rebuild, moving to staging.
2018-07-31 09:43:52 +02:00
James Deikun bd63de114e stdenv/build-support: support .tbz and .txz tarballs 2018-07-30 15:30:16 -04:00
Eduard-Mihai Burtescu 35e0ca9b24
Use the default for NIX_CXXSTDLIB_COMPILE even if defined (but empty). 2018-07-30 10:50:55 +03:00
Frederik Rietdijk dfeeac7f7c remove file that was accidentally added 2018-07-29 17:19:23 +02:00
Matthew Bauer 96ce1e03a4 {cc,bintools}-wrapper: also replace . in config
Some configs will have dots for version numbers. To normalize we can
just use _ again.
2018-07-28 19:54:09 -04:00
Nick Novitski c58b11d229 dockerTools.pullImage: control OS and architecture 2018-07-27 12:29:31 -07:00
Periklis Tsirakidis 04f0c25222 Fix pname to ename propagation if null 2018-07-27 08:15:46 +02:00
John Q Crosscompiler 7cc62144b2
systems: Allow detection of powerpc and sparc 2018-07-26 09:33:36 -04:00
Frederik Rietdijk ae9932e431 php: get rid of composableDerivation
Some bugs may have creeped in during the conversion.
One flag I had trouble with and so removed was:

    "--enable-embedded-mysqli"
2018-07-26 11:49:56 +02:00
Lorenzo Manacorda 7a1a0036e6 buildMaven: Check for authenticated attribute
The `authenticated` attribute is not always present in the
`project-info.json` produced by maven2nix[0]

We therefore check for its presence, and default it to false.

[0]: https://github.com/NixOS/mvn2nix-maven-plugin/issues/5#issuecomment-311846950
2018-07-25 16:00:52 +02:00
Frederik Rietdijk 099c13da1b Merge staging-next into master (#44009)
* substitute(): --subst-var was silently coercing to "" if the variable does not exist.

* libffi: simplify using `checkInputs`

* pythonPackges.hypothesis, pythonPackages.pytest: simpify dependency cycle fix

* utillinux: 2.32 -> 2.32.1

https://lkml.org/lkml/2018/7/16/532

* busybox: 1.29.0 -> 1.29.1

* bind: 9.12.1-P2 -> 9.12.2

https://ftp.isc.org/isc/bind9/9.12.2/RELEASE-NOTES-bind-9.12.2.html

* curl: 7.60.0 -> 7.61.0

* gvfs: make tests run, but disable

* ilmbase: disable tests on i686. Spooky!

* mdds: fix tests

* git: disable checks as tests are run in installcheck

* ruby: disable tests

* libcommuni: disable checks as tests are run in installcheck

* librdf: make tests run, but disable

* neon, neon_0_29: make tests run, but disable

* pciutils: 3.6.0 -> 3.6.1

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/pciutils/versions.

* mesa: more include fixes

mostly from void-linux (thanks!)

* npth: 1.5 -> 1.6

minor bump

* boost167: Add lockfree next_prior patch

* stdenv: cleanup darwin bootstrapping

Also gets rid of the full python and some of it's dependencies in the
stdenv build closure.

* Revert "pciutils: use standardized equivalent for canonicalize_file_name"

This reverts commit f8db20fb3a.
Patching should no longer be needed with 3.6.1.

* binutils-wrapper: Try to avoid adding unnecessary -L flags

(cherry picked from commit f3758258b8895508475caf83e92bfb236a27ceb9)
Signed-off-by: Domen Kožar <domen@dev.si>

* libffi: don't check on darwin

libffi usages in stdenv broken darwin. We need to disable doCheck for that case.

* "rm $out/share/icons/hicolor/icon-theme.cache" -> hicolor-icon-theme setup-hook

* python.pkgs.pytest: setupHook to prevent creation of .pytest-cache folder, fixes #40273

When `py.test` was run with a folder as argument, it would not only
search for tests in that folder, but also create a .pytest-cache folder.
Not only is this state we don't want, but it was also causing
collisions.

* parity-ui: fix after merge

* python.pkgs.pytest-flake8: disable test, fix build

* Revert "meson: 0.46.1 -> 0.47.0"

With meson 0.47.0 (or 0.47.1, or git)
things are very wrong re:rpath handling
resulting in at best missing libs but
even corrupt binaries :(.

When we run patchelf it masks the problem
by removing obviously busted paths.
Which is probably why this wasn't noticed immediately.

Unfortunately the binary already
has a long series of paths scribbled
in a space intended for a much smaller string;
in my testing it was something like
lengths were 67 with 300+ written to it.

I think we've reported the relevant issues upstream,
but unfortunately it appears our patches
are what introduces the overwrite/corruption
(by no longer being correct in what they assume)

This doesn't look so bad to fix but it's
not something I can spend more time on
at the moment.

--

Interestingly the overwritten string data
(because it is scribbled past the bounds)
remains in the binary and is why we're suddenly
seeing unexpected references in various builds
-- notably this is is the reason we're
seeing the "extra-utils" breakage
that entirely crippled NixOS on master
(and probably on staging before?).

Fixes #43650.

This reverts commit 305ac4dade.

(cherry picked from commit 273d68eff8f7b6cd4ebed3718e5078a0f43cb55d)
Signed-off-by: Domen Kožar <domen@dev.si>
2018-07-24 15:04:48 +01:00
Thomas Tuegel 7131e353e6
melpaBuild: use Emacs package names to satisfy package-build
package-build expects the recipe file name to match the Emacs package
name. `melpaBuild` takes an extra argument `ename` for the Emacs package
name (default: `pname`, the Nix package name) which is used to name the recipe
file.

Fixes: #43893
See also: #43609
2018-07-23 06:51:51 -05:00
Jörg Thalheim 218298b30f
Merge branch 'master' into unused5 2018-07-21 15:41:22 +01:00
volth 52f53c69ce pkgs/*: remove unreferenced function arguments 2018-07-21 02:48:04 +00:00
volth 6d2857a311 [bot] treewide: remove unused 'inherit' in let blocks 2018-07-20 19:38:19 +00:00
Frederik Rietdijk 1a6af9f88e
Merge pull request #43857 from volth/unused
[bot] treewide: remove unreferenced code
2018-07-20 21:06:32 +02:00
volth 87f5930c3f [bot]: remove unreferenced code 2018-07-20 18:48:37 +00:00
Matthew Bauer fddd90e9ea
Merge pull request #43538 from timokau/fetchpatch-fix
fetchpatch: quote excludes
2018-07-19 00:35:09 -04:00
Matthew Bauer 76999cc40e treewide: remove aliases in nixpkgs
This makes the command ‘nix-env -qa -f. --arg config '{skipAliases =
true;}'’ work in Nixpkgs.

Misc...

- qtikz: use libsForQt5.callPackage

  This ensures we get the right poppler.

- rewrites:

  docbook5_xsl -> docbook_xsl_ns
  docbook_xml_xslt -> docbook_xsl

diffpdf: fixup
2018-07-18 23:25:20 -04:00
aszlig caccc40ad0
vmTools: Fix BusyBox runtime error in initrd
With the recent update of BusyBox to version 1.29.0 in
d6aa506e3b there is now a new dependency
on libresolv.

This now throws a runtime error when executing ash, eg. whenever we do
something like this:

nix-build -E 'with import ./. {}; vmTools.runInLinuxVM hello'

The resulting error will be:

  .../ash: error while loading shared libraries: libresolv.so.2: cannot
           open shared object file: No such file or directory

I tried to override BusyBox with enableStatic, but that still requires
parts of glibc:

  Static linking against glibc, can't use --gc-sections
  Trying libraries: crypt m resolv
   Library crypt is not needed, excluding it
   Library m is needed, can't exclude it (yet)
   Library resolv is needed, can't exclude it (yet)
   Library m is needed, can't exclude it (yet)
   Library resolv is needed, can't exclude it (yet)
  Final link with: m resolv

In the long term maybe switching to a more minimal C library such as
musl would make more sense, but for now I just added libresolv.so to the
initrd which fixes the runtime error.

Signed-off-by: aszlig <aszlig@nix.build>
Cc: @edolstra, @rbvermaa
Signed-off-by: aszlig <aszlig@nix.build>
2018-07-18 03:02:55 +02:00
Thomas Tuegel 9cfbef8bf2
Merge pull request #43609 from ttuegel/emacs-packages-2018-07-14
Emacs package updates
2018-07-17 17:53:54 -05:00
Timo Kaufmann 3058513941 fetchpatch: add option to revert a patch 2018-07-16 22:46:54 +02:00
Frederik Rietdijk 3c09808160 Merge master into staging-next 2018-07-16 07:43:37 +02:00
Thomas Tuegel d3cea48608
emacsPackagesNg: 2018-07-15 update
All package sets are simultaneously updated to accommodate changes to
package-build. Due to new restrictions in package-build, all packages using
`melpaBuild` must now provide a recipe file, even those packages which are not
included in upstream MELPA.
2018-07-15 20:52:41 -05:00
aszlig f1fbf818c4
autoPatchelfHook: Run after patchelf's setup hook
So far the runtimeDependencies variable has been rather useless unless
you also set dontPatchelf, because the patchelf setup hook ran *after*
the autoPatchelfHook and thus stripped off the additional RPATHs added
using runtimeDependencies.

I did this by moving the autoPatchelfHook to be run in postFixup instead
of fixupOutput, however I needed to replicate the for loop that runs the
hook on all outputs.

Until we have a way to influence order of execution for hooks I've
marked this with an XXX so that we can use fixupOutput again.

Tested this against all packages that use autoPatchelfHook using the
following and checking whether the output contains any errors concerning
shared libraries:

nix-build -E 'with import ./. { config.allowUnfree = true; };
  runCommand "test-executables" {
    drvs = [
      masterpdfeditor franz zoom-us anydesk teamviewer maxx
      oracle-instantclient cups-kyodialog3 virtlyst powershell
    ];
  } "for i in $drvs; do for b in $i/bin/*; do \"$b\" || :; done; done"
'

Signed-off-by: aszlig <aszlig@nix.build>
Fixes: https://github.com/NixOS/nixpkgs/issues/43082
Cc: @Ericson2314
2018-07-16 01:52:28 +02:00
Timo Kaufmann 8f9b985e60 fetchpatch: fail on empty patch
Since this is probably never the desired case and has led to actual
issues, see the comments at:

af1313e915

This might also happen when pulling a patch from GitHub or a similar web
interface without explicitly selecting the "raw" format.
2018-07-15 14:14:21 +02:00
Timo Kaufmann 1ddab0efb1 fetchpatch: escape excludes and includes
Excludes and includes are implemented by passing the parameters to the
respective flags of `filterdiff`. Those were passed unescaped until now.
Since those flags expect patterns (similar to shell globs), something
like `/some/path/*` might be used to exclude or include all files in
some path. Without escaping the shell would expand the `*`, leading to
unexpected behaviour.
2018-07-15 09:58:47 +02:00
John Ericson aa6adfc324 fetchpatch: Add includes to compliment excludes, and require that both not be non-empty.
This commit was originally introduced as part of #41420 and then
reverted with the rest of that PR. However there was no reason to revert
his particular commit.
2018-07-15 09:56:41 +02:00
Vladimír Čunát 0f01215203
Merge branch 'master' into staging-next
Hydra: ?compare=1468896
2018-07-14 18:15:30 +02:00
Uri Baghin ed98822350 bazel-deps: init at 2018-05-31 (#43018) 2018-07-09 22:38:45 +00:00
Antoine Eiche d44b778d10 dockerTools.examples: explicitly set image tag to fix docker-tools tests
docker-tools tests load images without specifying any tag
value. Docker then uses the image with tag "latest" which doesn't
exist anymore since commit 39e678e24e.
2018-07-06 16:38:42 +02:00
Mathias Schreck 39e678e24e dockerTools.buildImage: add option to use nix output hash as tag 2018-07-06 15:15:09 +02:00
Matthew Bauer 79602a2f44
Merge pull request #42851 from jmitchell/fix/vm-debian
vmTools: update Debian minor version to 8.11
2018-07-05 21:50:17 -04:00
Peter Simons 6e07a3a19a
Revert "haskell generic-builder: Use strictDeps always" 2018-07-04 13:18:21 +02:00
Matthew Bauer e8e26becac
Merge pull request #42887 from matthewbauer/libsecurity-closure-reductions
Libsecurity closure reductions
2018-07-02 18:34:53 -04:00
John Ericson 57bb96d659 fetchpatch: Add includes to compliment excludes, and require that both not be non-empty. 2018-07-02 15:51:13 -04:00
Shea Levy 98ddba156c
buildRustCrate: Add some commentary about target_os. 2018-07-02 11:22:47 -04:00
Shea Levy cb692ff813
Merge branch 'feature/fix-build-rust-create-darwin' of git://github.com/marsam/nixpkgs
Set target_os properly on darwin.
2018-07-02 11:20:25 -04:00
Vladimír Čunát c1ffc65d1a
Merge branch 'master' into staging
This apparently fixes some broken src fetches (gnuradio, twisted).
2018-07-02 11:10:26 +02:00
Jacob Mitchell 8d5ef42772 vmTools: update Debian minor version to 8.11 2018-07-01 15:47:09 -07:00
Vladimír Čunát f7781f5293
Merge branch 'master' into staging 2018-06-30 01:41:59 +02:00
Daniel Barlow f9f6ddc083 make-closure needs build system mkdir and jq
Make make-closure work when cross-compiling
2018-06-29 23:22:24 +00:00
Timo Kaufmann 082169ab02 fetchurl: add sageupstream mirror 2018-06-30 01:20:44 +02:00
Guillaume Maudoux ea23a1ed4e fetchgit: adapt regex to git 2.18.0 2018-06-28 07:01:14 -05:00
Matthew Bauer f194659ddb trivial-builders: disallow sub in requireFile
The requireFile call was being substituted from the binary cache. We
do not want this to happen as the user needs to download the file
themselves.
2018-06-26 21:49:08 -04:00
Matthew Bauer 4f6d61e5cf cc-wrapper: disable stackprotector for mingw
It seems to break things.
2018-06-23 22:05:26 -04:00
Justin Bedő 1e0a2eafbf singularity-tools: create mount points for image building (#41470) 2018-06-18 23:50:06 +02:00
Matthew Justin Bauer 176380dbbb
Merge pull request #42129 from timbertson/fetchsubmodules
nix-prefetch-git: fix output `fetchSubmodules` property
2018-06-17 16:19:30 -04:00
Tim Cuthbertson c5454933a1 nix-prefetch-git: fix output fetchSubmodules property 2018-06-17 21:19:27 +10:00
Graham Christensen cbae3d171b
vsenv, vs90wrapper: delete 2018-06-15 22:20:54 -04:00
Vladimír Čunát 31530c185e
Merge a subset of staging (security)
In particular, this contains Firefox-related and libgcrypt updates.
Other larger rebuilds would apparently need lots of time to catch up
on Hydra, due to nontrivial rebuilds in other branches than staging.
2018-06-15 08:48:02 +02:00
Matthew Bauer ad55409266 cc-wrapper: fix eval error on mac
When doing cross to linux, we will not have a GCC compiler (yet). We
can hopefully skip the cxx stdlib stuff for now.
2018-06-14 19:29:30 -04:00
Uri Baghin 274bb96073 bazel: add darwin support 2018-06-12 23:23:51 -04:00
Cole Mickens a44a9fdad6 azure: stop carrying qemu-220 patch 2018-06-12 02:06:03 -07:00
Orivej Desh fd97db43bc
pruneLibtoolFiles: init setup hook (#41819)
A .la file specifies linker flags to link with the library it describes. Its
"dependency_libs" field lists the libraries that this library depends upon.
This list often contains "-l" flags without corresponding "-L" flags. Many
packages in Nixpkgs deal with this in one of these ways:
- delete .la file [1]
- clear dependency_libs [2]
- add -L flags to dependency_libs [3]
- propagate dependencies [4]

Sometimes "dependency_libs" contain wrong "-L" flags pointing to the "dev"
output with headers rather than to the main output with libraries. They have to
be edited or deleted to reduce closure size [5].

Deleting .la files is often but not always safe [6].  Atomatically deleting as
many of them as possible is complex [7].  Deleting .la files that describe
shared rather than static libraries is probably safe; but clearing their
"dependency_libs" field achieves the same effect with less potential for
unintended consequences.  This is the approach that may be enabled for all
Nixpkgs.

[1] 2a79d296d3
[2] c83a530985
[3] 9e0dcf3bd9
[4] 01134e698f
[5] f6c73f1e37
[6] https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Handling_Libtool_Archives
[7] https://github.com/gentoo/gentoo/blob/fb1f2435/eclass/ltprune.eclass
2018-06-11 18:11:02 +00:00
Daniel Peebles 363363298c
Merge pull request #17757 from copumpkin/fetchurl-user-agent
fetchurl: add user agent
2018-06-11 12:49:06 -04:00
Yegor Timoshenko f44012ba10
lib.release -> lib.trivial.release 2018-06-11 02:14:18 +00:00
Yegor Timoshenko 5a232b0f5b
Leverage lib.release 2018-06-11 01:50:06 +00:00
Yegor Timoshenko a26d9e3ba8
Readd --retry 3 2018-06-11 01:48:00 +00:00
Benjamin Hipple d81ab0ab7d alsa: setup mirrors for src downloads (#41761)
This commit adds a list of supported mirrors for all alsa projects, as described
on the download section of the alsa-project hompage:
http://alsa-project.org/main/index.php/Download
2018-06-10 20:56:20 +02:00
aszlig ff5cecf821
autoPatchelfHook: Patch PIC exes/libs as well
If there is a shared object or executable that's using
position-independent code, the file's mime type is
"application/x-pie-executable", so until this change its dependencies
wouldn't be patched.

This simply adds the mime type to the search loop.

Signed-off-by: aszlig <aszlig@nix.build>
2018-06-08 00:27:31 +02:00
John Ericson c69c959cd9
Merge pull request #41429 from lopsided98/fix-env-hook
cc-wrapper, bintools-wrapper: unbreak include/link paths when cross-compiling
2018-06-04 22:42:27 -04:00
lewo 0644b4d948 dockerTools.pullImage: expose image* attributes (#41366)
Attributes `imageName` and `imageTag` are exposed if the image is
built by our Nix tools but not if the image is pulled. So, we expose
these attributes for convenience and homogeneity.
2018-06-03 22:58:23 +02:00
Ben Wolsieffer 645f03b949 cc-wrapper, bintools-wrapper: use getHostRoleEnvHook instead of getTargetRoleEnvHook
Fixes include and link paths when cross compiling.
2018-06-03 10:54:32 -04:00
John Ericson f303ee29cc
Merge pull request #41065 from Ralith/sanitizer-fix
clang_6: fix sanitizers under libstdc++
2018-05-29 13:08:44 -04:00
Orivej Desh 5ac2382487 mirrors: remove obsolete mysql mirror
It does not contain any of the versions currently in Nixpkgs.
2018-05-29 13:41:23 +00:00
Jan Malakhovski ad35019501 Merge branch 'master' into staging
Fixed conflicts:
- lib/systems/for-meta.nix: in favor of staging
- pkgs/os-specific/darwin/xcode/default.nix: in favor of master
2018-05-26 00:20:17 +00:00
Benjamin Saunders b0d0b1adfe clang_6: fix sanitizers under libstdc++ 2018-05-24 15:04:30 -07:00
lewo 2e98e0c003
Merge pull request #40947 from samueldr/fix/34779
dockerTools: fixes extraCommands for mkRootLayer.
2018-05-24 21:22:31 +02:00
Samuel Dionne-Riel 902b0593be tests/docker-tools: Adds regression test for #34779 2018-05-24 12:23:51 -04:00
Antoine Eiche 8f71ce7e80 skopeo: 0.1.29 -> 0.1.30
Skopeo used by our docker tools was patched to work in the build
sandbox (it used /var/tmp which is not available in the sandbox).
Since this temporary directory can now be set at build time, we remove
the patch from our docker tools.
2018-05-24 15:33:52 +02:00
Samuel Dionne-Riel 60737bd319 dockerTools: fixes extraCommands for mkRootLayer.
The extraCommands was, previously, simply put in the body of the script
using nix expansion `${extraCommands}` (which looks exactly like bash
expansion!).

This causes issues like in #34779 where scripts will eventually create
invalid bash.

The solution is to use a script like `run-as-root`.

 * * *

Fixes #34779
2018-05-24 06:51:26 -04:00
John Ericson 205fc55ea2 Merge remote-tracking branch 'upstream/staging' into compiler-rt 2018-05-24 02:58:00 -04:00
Benjamin Saunders 46eeef1898 tests.cc-wrapper: verify building with sanitizers 2018-05-24 02:57:44 -04:00
John Ericson d7144e708b Merge remote-tracking branch 'upstream/master' into staging 2018-05-23 16:00:04 -04:00
Jörg Thalheim 273c882f53
Merge pull request #39214 from seppeljordan/add-nix-prefetch-github
Add nix-prefetch-github
2018-05-23 16:33:03 +01:00
Shea Levy 6da6accd30
treewide: Remove uses of builtins.toPath.
toPath has confusing semantics and is never necessary; it can always
either just be omitted or replaced by pre-concatenating `/.`. It has
been marked as "!!! obsolete?" for more than 10 years in a C++
comment, hopefully removing it will let us properly deprecate and,
eventually, remove it.
2018-05-22 16:42:02 -04:00
Sebastian Jordan aca3198c70 nix-prefetch-github: init -> 1.3 2018-05-22 20:55:07 +02:00
John Ericson db4d77779c Merge remote-tracking branch 'upstream/master' into staging 2018-05-21 20:21:48 -04:00
John Ericson 9ec53a397f requireFile: Use stdenvNoCC
There's no need for a C compiler
2018-05-21 19:26:36 -04:00
Vladimír Čunát 392e6de7d0
Merge branch 'master' into staging 2018-05-20 13:20:53 +02:00
P-E-Meunier aa1d7961e7 curl-sys: fix linking against zlib 2018-05-20 11:30:06 +01:00
P-E-Meunier c0e2f7bbbe buildRustCrate: add extraLinkFlags parameter
This is useful when build scripts do not apply linking flags
2018-05-20 11:29:34 +01:00
Yegor Timoshenko 16ed09a10e
Merge branch 'master' into fetchurl-user-agent 2018-05-18 10:39:44 +03:00
Tuomas Tynkkynen 003473613a Merge remote-tracking branch 'upstream/master' into staging
Conflicts:
	pkgs/top-level/all-packages.nix
2018-05-18 03:54:38 +03:00
Matthew Bauer 768bd58a48 crate-overrides: curl-sys needs zlib 2018-05-17 14:20:29 -05:00
Matthew Bauer e0fccdcc8d rust: add more sys overrides 2018-05-17 14:20:29 -05:00
John Ericson ddbe9191ef Merge remote-tracking branch 'upstream/master' into staging
Keep the dontCheck because the test suite fails, get rid of the LDFLAGS
hack because we don't need it!
2018-05-16 15:16:08 -04:00
Alexandre Esteves d273691f6d trivial-builders.nix: support directories in requireFile
Allow recursive hash for requireFile so it can support directories (#40568)
2018-05-16 01:41:13 -05:00
John Ericson 5e17335bd7 Merge remote-tracking branch 'upstream/staging' into strictDeps 2018-05-14 23:33:03 -04:00
John Ericson 330ca731e8 treewide: Get rid of all uses of crossConfig
The hack of using `crossConfig` to enforce stricter handling of
dependencies is replaced with a dedicated `strictDeps` for that purpose.
(Experience has shown that my punning was a terrible idea that made more
difficult and embarrising to teach teach.)

Now that is is clear, a few packages now use `strictDeps`, to fix
various bugs:

 - bintools-wrapper and cc-wrapper
2018-05-14 23:30:37 -04:00
Matthew Bauer 6748534d83 Merge remote-tracking branch 'upstream/master' into staging 2018-05-08 09:36:00 -05:00
John Ericson 4f7cdd35d5
Merge pull request #40139 from obsidiansystems/modular-setup-hooks
treewide: Modular setup hooks
2018-05-07 15:32:10 -04:00
John Ericson 8b0fce8cb1 {bintools,cc}-wrapper: Factor out role accumulation logic 2018-05-07 15:10:45 -04:00
John Ericson 2110c0bd30 treewide: Use pkgs/build-support/roles.bash to remove copy pasta
Also fix some setup hooks that unnecessarily used environment hooks,
which revolted in the same variable being modified too many times.
2018-05-07 15:10:37 -04:00
aszlig 42a0b11450
dockerTools.pullImage: Fix build with sandboxing
Regression introduced in 736848723e.

This commit most certainly hasn't been tested with sandboxing enabled
and breaks not only pullImage but also the docker-tools NixOS VM test
because it doesn't find it's certificate path and also relies on
/var/tmp being there.

Fixing the certificate path is the easiest one because it can be done
via environment variable.

I've used overrideAttrs for changing the hardcoded path to /tmp (which
is available in sandboxed builds and even hardcoded in Nix), so that
whenever someone uses Skopeo from all-packages.nix the path is still
/var/tmp.

The reason why this is hardcoded to /var/tmp can be seen in a comment in
vendor/github.com/containers/image/storage/storage_image.go:

  Do not use the system default of os.TempDir(), usually /tmp, because
  with systemd it could be a tmpfs.

With sandboxed builds this isn't the case, however for using Nix without
NixOS this could turn into a problem if this indeed is the case.

So in the long term this needs to have a proper solution.

In addition to that, I cleaned up the expression a bit.

Tested by building dockerTools.examples.nixFromDockerHub and the
docker-tools NixOS VM test.

Signed-off-by: aszlig <aszlig@nix.build>
Cc: @nlewo, @Mic92, @Profpatsch, @globin, @LnL7
2018-05-06 04:57:24 +02:00
adisbladis f786072420
Merge pull request #39957 from enumatech/fix/nix-prefetch-git-spaces
nix-prefetch-git: fix handling of submodules with spaces
2018-05-04 23:36:58 +08:00
Lionello Lunesu 9fe26eed9e nix-prefetch-git: fix handling of submodules with spaces
The script would parse the output of `git submodule status` but
didn't handle paths with spaces in them. This would result in the
following error when trying to determine the URL of the submodule:

error: key does not contain a section: .url
2018-05-04 23:16:51 +08:00
Michael Bishop 51998c675a runInLinuxVM: fix ext4 and crc32c-intel interactions 2018-05-03 20:08:48 -03:00
John Ericson cf06e42d1c Merge remote-tracking branch 'upstream/master' into staging 2018-05-03 16:35:36 -04:00
zimbatm f7abcb0752
fetchs3: allow to name the derivation output (#39823)
* fetchs3: add configurable name

Change the default from "foo" to the basename of the s3 URL and make it
configurable.

* fetchs3: fix error on missing credentials.session_token

The session token should default to null instead of failing

* fetchs3: make use of the region argument

Set it to null if you don't want to use it

* fetchs3: prefer local build

Fetcher-types spend more time on network than CPU
2018-05-03 11:08:25 +01:00
Matthew Justin Bauer eeb016e8f0
Merge branch 'staging' into fix-ncurses-darwin-extensions 2018-05-02 15:40:38 -05:00
Antoine Eiche 736848723e dockerTools.pullImage: Skopeo pulls images by digest
Skopeo is used to pull images from a Docker registry (instead of a
Docker deamon in a VM).

An image reference is specified with its name and its digest which is
an immutable image identifier (unlike image name and tag).

Skopeo can be used to get the digest of an image, for instance:
$ skopeo inspect docker://docker.io/nixos/nix:1.11 | jq -r '.Digest'
2018-05-02 21:32:20 +02:00