Commit graph

3742 commits

Author SHA1 Message Date
(cdep)illabout ae607de23d linkFarmFromDrvs: Add a trivial builder to create a linkFarm from a list of derivations 2020-02-18 16:35:28 +01:00
Andreas Rammhold 453589696b
Merge pull request #79816 from andir/buildRustCrate-no-override-dep
buildRustCrate: remove superfluous dependency overrides
2020-02-18 15:18:44 +01:00
Bas van Dijk 4e0a6bc4be symlinkJoin: fix "bash: Argument list too long" error
When the `paths` argument is too big `symlinkJoin` will fail with:

```
while setting up the build environment: executing '/nix/store/rm1hz1lybxangc8sdl7xvzs5dcvigvf7-bash-4.4-p23/bin/bash': Argument list too long
```

This is fixed by passing `paths` as a file instead of as an
environment variable.
2020-02-17 11:45:44 +01:00
Benjamin Hipple 6d881472ef rust: Fix for legacy fetch cargo
See inline comment and #79975 for details.
2020-02-15 17:41:35 -08:00
Jan Tojnar 28faf5bc86
nix-prefetch-git: Fix output mangling
`git repack` and `git gc` sometimes print “Nothing new to pack.”
to stdout, which breaks redirecting output to JSON file.

Let’s move the stdout of all git calls where it is not used to stderr
so that we still receive the info but it does not pollute our output.
2020-02-14 22:15:36 +01:00
Bjørn Forsman d394c82cde
build-fhs-userenv: runCommand -> runCommandLocal (#77253)
It's a trivial derivation, no need to build remotely (it's slower that
way).
2020-02-14 20:16:49 +02:00
lewo d49804cf45
Merge pull request #80102 from nlewo/fix-78744
dockerTools.buildLayeredImage: store all paths passed in final layer (fix 78744)
2020-02-14 09:48:52 +01:00
Mario Rodas 7696369bec
docker-slim: move expression to outside of build-support (#80078) 2020-02-14 09:42:18 +01:00
Antoine Eiche baa78de594 nixosTests.docker-tools: add bulk-layer test
A regression test for https://github.com/NixOS/nixpkgs/issues/78744.
2020-02-14 09:26:26 +01:00
Richard Wallace 3b65b3f6d6 dockerTools.buildLayeredImage: store all paths passed in final layer
Fixes #78744

My previous change broke when there are more packages than the maximum
number of layers. I had assumed that the `store-path-to-layer.sh` was
only ever passed a single store path, but that is not the case if
there are multiple packages going into the final layer. To fix this, we
loop through the paths going into the final layer, appending them to the
tar file and making sure they end up at the right path.
2020-02-14 09:26:26 +01:00
Benjamin Hipple eb11feaa0b treewide: change fetchCargoTarball default to opt-out
Changes the default fetcher in the Rust Platform to be the newer
`fetchCargoTarball`, and changes every application using the current default to
instead opt out.

This commit does not change any hashes or cause any rebuilds. Once integrated,
we will start deleting the opt-outs and recomputing hashes.

See #79975 for details.
2020-02-13 22:41:37 -08:00
Jan Tojnar f19a101a73
nix-prefetch-git: report deepClone & leaveDotGit
Just like in be30ba8e0e
we want `fetchgit (builtins.fromJSON (builtins.readFile ./src.json))` to work.
2020-02-14 06:05:43 +01:00
Benjamin Hipple c1542fc4cd rust: Replace migration README with GitHub issue
The readme was nice to discuss in the implementation PR, but now that this is
merged it's better to have an issue that can be linked against in PRs and
doesn't require further merges to update status.

Ported with a status update in #79975
2020-02-12 22:28:53 -05:00
Frederik Rietdijk 424697d512 Merge master into staging-next 2020-02-12 09:55:31 +01:00
Andreas Rammhold be5597fc9d
buildRustCrate: remove superfluous dependency overrides
By overriding each dependency on every level of the dependency tree we
are creating a lot of unnecessary instances of the same derivation

Looking at the output size of `nix-instantiate --trace-function-calls
-vvvv …` and the execution time I got about a 10x improvement after
applying this change.

It was probably good intentions that lead to these overrides but in
practice no tooling (that I know of) really needs this. `carnix` and
`crate2nix` are fine without those overrides. Furthermore I believe that
it is the job of the tooling around `buildRustCrate` to provide a
coherent set of overrides. By not enforcing all of the overrides, debug
flags, verbosity, … to be the same throughout the closure we also allow
consumers to override specific aspects of the crates. Some (older?)
crates might need different `crateOverrides` then newer crates with the
same name. Currently such situations can not (easily) be implemented
with the override in-place.
2020-02-11 11:48:45 +01:00
Benjamin Hipple 2115a2037c fetchcargo: use flat tar.gz file for vendored src instead of recursive hash dir
This has several advantages:

1. It takes up less space on disk in-between builds in the nix store.
2. It uses less space in the binary cache for vendor derivation packages.
3. It uses less network traffic downloading from the binary cache.
4. It plays nicely with hashed mirrors like tarballs.nixos.org, which only
   substitute --flat hashes on single files (not recursive directory hashes).
5. It's consistent with how simple `fetchurl` src derivations work.
6. It provides a stronger abstraction between input src-package and output
   package, e.g., it's harder to accidentally depend on the src derivation at
   runtime by referencing something like `${src}/etc/index.html`. Likewise, in
   the store it's harder to get confused with something that is just there as a
   build-time dependency vs. a runtime dependency, since the build-time
   src dependencies are tarred up.

Disadvantages are:
1. It takes slightly longer to untar at the start of a build.

As currently implemented, this attaches the compacted vendor.tar.gz feature as a
rider on `verifyCargoDeps`, since both of them are relatively newly implemented
behavior that change the `cargoSha256`.

If this PR is accepted, I will push forward the remaining rust packages with a
series of treewide PRs to update the `cargoSha256`s.
2020-02-10 10:17:29 -05:00
Benjamin Hipple caff183402 fetchzip, fetchgit: cleanup handling of optional features and whitespace
No intended functional difference, just trying to polish to make this easier to
extend and more aligned with other nix conventions.
2020-02-08 21:51:46 -05:00
Andreas Rammhold 56e11bc8df
buildRustCrate: remap the current build dir to / for (more) reproducible builds 2020-02-06 01:18:59 +01:00
Uri Baghin 10d1a04b02 bazel-deps: remove 2020-02-04 12:35:47 +11:00
Uri Baghin 3dc9c1b885 bazel-remote: 2019-01-12 -> 2020-01-29 2020-02-04 02:26:15 +01:00
worldofpeace 8ecbde9def
Merge pull request #78945 from jbedo/singularity-img
singularity-tools: fix buildImage
2020-02-03 15:19:47 -05:00
Justin Bedo 56c22350c0
singularity-tools: fix buildImage 2020-02-02 09:18:39 +11:00
Silvan Mosberger 0a351c3f65
dockerTools.*: Assertion against building for Darwin (#77952)
dockerTools.*: Assertion against building for Darwin
2020-01-31 21:17:40 +01:00
lewo 86f8732194
Merge pull request #78834 from nlewo/fix-two-layers-image
Fix dockerTools.buildLayerImage with 2 layers
2020-01-30 21:10:52 +01:00
Antoine Eiche 01a68479cc dockerTools.buildLayeredImage: assert maxLayers > 1
Since a layer is reserved for "customization", the image can not
contains less than 2 layers.

The user gets the following message at evaluation:

    nix-instantiate nixos/tests/docker-tools.nix
    trace: the maxLayers argument of dockerTools.buildLayeredImage function must be greather than 1 (current value: 1)
2020-01-30 21:09:52 +01:00
Antoine Eiche 283bcc1003 dockerTools.buildLayeredImage: fix image with only 2 layers
A test is also added to ensure an image with 2 layers can be built.
2020-01-30 21:09:44 +01:00
Andreas Rammhold a57d0fe0bb
buildRustCrate: fix #78412
`build.rs` files might create files. Those files are supposed to go into
`OUT_DIR` (envirionment variable) and not be overlayed onto the source
tree.
2020-01-28 14:07:58 +01:00
Andreas Rammhold 19698d15ce
buildRustCrateTests: add regression test for #74071 2020-01-28 14:07:58 +01:00
Frederik Rietdijk 7562c2112d
Merge pull request #77739 from NixOS/staging-next
Staging next
2020-01-28 14:03:29 +01:00
Andreas Rammhold cd27940df0
Merge pull request #78188 from andir/buildRustCrate-support-rlib
buildRustCrate: support `rlib` dependency type
2020-01-28 13:47:57 +01:00
Frederik Rietdijk dce0ca29d9 Merge master into staging-next 2020-01-28 10:46:13 +01:00
Mario Rodas c3ccac15eb
libredirect: fix build on darwin
Fix build failure on darwin due to absence of `O_TMPFILE`.
2020-01-23 20:20:20 -05:00
Andreas Rammhold 78faab1be0 buildRustCrateTests: add test case for rlib linking 2020-01-21 17:46:32 +01:00
Andreas Rammhold 406e0c9d51
buildRustCrateTests: fix some formatting issues 2020-01-21 17:32:48 +01:00
Andreas Rammhold d6a8b55fb0
buildRustCrate: treat rlib crates just like lib crates
Both version provide `rlib` files to link against. Previously we would
try to find a matching shared library in the `lib` output.
2020-01-21 17:22:59 +01:00
Andreas Rammhold 60fea726de
Merge pull request #77822 from andir/buildRustCrate-remove-custom-lib-path
buildRustCrate: remove custom lib path heuristic
2020-01-21 17:21:55 +01:00
Silvan Mosberger b6a9211bf4
dockerTools.*: Assertion against building for darwin
Building a docker image with darwin binaries just yields a confusing
error when ran:

  standard_init_linux.go:211: exec user process caused "exec format error"

This change prevents people from building such images in the first place
2020-01-20 19:14:12 +01:00
Jonas Schürmann 61870727de mirrors.nix: Use HTTPS in maven mirrors
Since 2020-01-15, the maven central repository does no longer support HTTP.
See https://blog.sonatype.com/central-repository-moving-to-https for details.
2020-01-20 09:54:14 +01:00
Mario Rodas 21d0b2067f
Merge pull request #76831 from mbrgm/docker-slim
docker-slim: init at 1.26.1
2020-01-18 07:51:03 -05:00
Andreas Rammhold 69c96adc53
buildRustCrateTests: use releaseTools.aggregate
Previously I did use `runCommand` to do the same. Using
releaseTools.aggregate seems a lot saner and we might get nicer hydra
output of the tests that are failing.
2020-01-16 13:24:15 +01:00
Andreas Rammhold 29a8575e3d
buildRustCrate: remove one of the odd library filename cases
It used to be the case (ref missing) that cargo did treat
`src/$libName.rs` as an alternative to `src/lib.rs` when the latter
wasn't present. Recently I failed to reproduce that with vanilla cargo
and it started to cause pain with some crates of the form:

some_crate/
 `- src
   `- main.rs
   `- some_crate.rs

We would build `src/some_crate.rs` and thing it is a library while that
might not be the actual case. This crate is a valid `bin` crate not a
`lib` crate as far as I can tell from the samples I took.

I removed support for the previously required heuristic and commented
out the test cases in case we will need them again. We could crawl in
the Git history but chances are that the next person looking into this
doesn't know about the history.
2020-01-16 13:24:13 +01:00
Anders Kaseorg 3cd8ce3bce treewide: Fix unsafe concatenation of $LD_LIBRARY_PATH
Naive concatenation of $LD_LIBRARY_PATH can result in an empty
colon-delimited segment; this tells glibc to load libraries from the
current directory, which is definitely wrong, and may be a security
vulnerability if the current directory is untrusted.  (See #67234, for
example.)  Fix this throughout the tree.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2020-01-15 09:47:03 +01:00
Andreas Rammhold cb007e69a1 stdenv: make symlinks that refer to the same output relative
While looking at the graph of all the outputs in my personal binary
cache it became obvious that we have a lot of self references within the
package set. That isn't an isuse by itself. However it increases the
size of the binary cache for every (reproducible) build of a package
that carries references to itself. You can no longer deduplicate the
outputs since they are all unique. One of the ways to get rid of (a few)
references is to rewrite all the symlinks that are currently used to be
relative symlinks. Two build of something that didn't really change but
carries a self-reference can the be store as the same NAR file again.

I quickly hacked together this change to see if that would yield and
success. My bash scripting skills are probably not great but so far it
seem to somewhat work.
2020-01-15 09:26:40 +01:00
Benjamin Hipple 6e8c377562 rustPlatform.buildRustPackage: cleaner output on verifyCargoDeps (#77567)
When this fails, the user may want to copy-paste the path to the "bad"
Cargo.lock file to inspect. The trailing `.` on `$cargoDeps.` gets caught in
most terminal copy-pastes. Since half the lines already don't have it, this
removes it from all of them for consistent output.
2020-01-12 17:19:17 +00:00
Antoine Eiche da261e3631 dockerTools.buildLayeredImage: fix typo in comments 2020-01-11 09:02:30 +01:00
lewo 0d983f9f99
Merge pull request #75911 from Simspace/parallel-docker-buildlayeredimage
when building a layered docker image, ignore it if tar encounters cha…
2020-01-11 08:55:34 +01:00
Andreas Rammhold 3e61906e1c
buildRustCrate: slight "rewording" and reformatting
There is no point in reinventinb builtins through `filterAttrs` or the
like. Lets just stick to what we already have in our toolbelt.
2020-01-07 11:57:34 +01:00
Andreas Rammhold a3a51763f9
buildRustCrate: add buildTests flag to tell rustc to build tests instead of binaries
This helps us instruct rustc to build tests instead of binaries. The
actual build will then ONLY produce test executables. This is a first
step towards having rust crate tests within nixpkgs.

We default back to only a single output in test cases since that is the
only reasonable thing to do here.

Producing libraries or binaries in addition to tests would theoretically
be feasible but usually generates different dependency trees. It is very
common to have some libraries in `[dev-depdendencies]` within Cargo.toml
just for your tests. To not start mixing things up going with a
dedicated derivation for the test build sounds like the best choice for
now.

To use this you must provide a proper test dependency chain to
`buildRustCrate` (as you would usually do with your non-test inputs).
And then set the `buildTests` attribute to `true`. The derivation will
then contain all tests that were built in `$out/tests`. All common test
patterns and directories should be supported and tested by this change.

Below is an example how you would run a single test from the derivation.
This commit contains some more examples in the `buildRustCrateTests`
attribute set that might be helpful.

```
let
  drv = buildRustCrate {
     …
     buildTests true;
  };
in runCommand "test-my-crate" {} ''
  touch $out
  exec ${drv}/tests/my-test
''
```
2020-01-07 11:57:34 +01:00
Andreas Rammhold 6383b42dae
buildRustCrate: fixup usage of builtins.filterSource
While unifying most of the lib function calls I accidentially changed
the filterSource functions as well. Since there were no tests I ended
up forgetting about this case (even thought I ran into it…).
2020-01-07 00:49:48 +01:00
Marius Bergmann d2394a4789 docker-slim: init at 1.26.1
Co-authored-by: Mario Rodas <marsam@users.noreply.github.com>
Co-authored-by: Oleksii Filonenko <brightone@protonmail.com>
2020-01-03 11:18:00 +01:00
Andreas Rammhold 9f03cb8562
Merge pull request #75563 from andir/cleanup-buildRustCrate
Cleanup buildRustCrate expression
2020-01-02 13:42:33 +01:00
John Ericson cfd013813e
Merge pull request #74090 from obsidiansystems/ghcjs-cross-without-cc
stdenv, haskell: bonafied GHCJS cross compilation without stdenv.cc
2019-12-30 16:40:43 -08:00
Richard Wallace 3be767593b dockerTools.buildLayeredImage: fix building layered images in parallel
when tar'ing store paths into layered archives when building layered
images, don't use the absolute nix store path so that tar won't complain
if something new is added to the nix store

when building the final docker image, ignore any file changes tar
detects in the layers. they are all immutable and the only thing that
might change is the number of hard links due to store optimization
2019-12-30 14:47:11 -07:00
Robin Gloster ac8eaa8507
treewide: fix *Flags 2019-12-30 04:50:37 +01:00
Alyssa Ross b9d274b89d rustPlatform.fetchcargo: expose 2019-12-23 18:27:56 +00:00
Alyssa Ross fdfbb4671e rustPlatform: forward unpackPhase to fetchcargo
If a custom unpackPhase is used for the package, it needs to also be
used for fetchcargo so the same source is available for vendoring.
2019-12-23 18:27:56 +00:00
Alyssa Ross 839c9e9344 rustPlatform: forward fetchcargo args to stdenv
Most stdenv wrappers already work like this -- it allows greater
customisation.  We just have to be careful to remove arguments we're
using that shouldn't be passed to stdenv.  I've been conservative
here, because fetchcargo checksums shouldn't change lightly.
2019-12-23 18:27:56 +00:00
Frederik Rietdijk f79e10061f Merge master into staging-next 2019-12-19 09:19:50 +01:00
Graham Christensen 64453c8dbd
Merge pull request #75781 from grahamc/dockertools/remove-implementation-detail-layers
dockertools.buildLayeredImage: remove implementation detail layers
2019-12-17 16:03:11 -05:00
Frederik Rietdijk bf2877b9ab Merge master into staging-next 2019-12-17 17:16:52 +01:00
Michael Raskin 26df2f4e9e
Merge pull request #74862 from alyssais/fetchgittiles
fetchFromGitiles: init
2019-12-17 15:23:08 +00:00
Frederik Rietdijk 7aedd744d8 Merge master into staging-next 2019-12-16 20:28:10 +01:00
Graham Christensen 75b8f3820d
Merge pull request #75779 from grahamc/dockertools/fixup-max-layers
dockerTools.buildLayeredImage: update maxlayers from 24 to 100 to match documentation
2019-12-16 14:11:36 -05:00
Graham Christensen 9c02760855
dockerTools.buildLayeredImage: update maxlayers from 24 to 100 to match documentation
mkManyPureLayers already was changed, and this function was not updated.
2019-12-16 13:14:21 -05:00
Graham Christensen 77452740c6
docker examples: Drop unneeded contents list 2019-12-16 13:11:27 -05:00
Graham Christensen 12e2416380
dockerTools.buildLayeredImage: Exclude top level implementation detail layers 2019-12-16 13:03:15 -05:00
Graham Christensen 700f4c5388
dockerTools.buildLayeredImage: prepare to exclude some paths
Without changing behavior, since this code is fiddly, make it possible
to add a filtering step before packaging individual paths.
2019-12-16 12:57:04 -05:00
Graham Christensen aec80dddc0
dockerTools.buildLayeredImage: pass a list of closures to mkManyPureLayers so it can exclude the top-most level
Before, every docker image had three extra layers:

1. A `closure` layer which is an internal implementation detail of
   calculating the closure of the container
2. a `name-config.json` layer which is the images' run-time
   configuration, and has no business being *in* the image as a layer.
3. a "bulk-layers" layer which is again and implementation detail
   around collecting the image's closure.

None of these layers need to be in the final product.
2019-12-16 12:48:05 -05:00
Graham Christensen f6d75f550e
dockerTools.buildLayeredImage: tweak formatting on contentsEnv 2019-12-16 12:36:45 -05:00
Alyssa Ross a50653295d buildSkawarePackage: pass through extra args
This allows things like hooks other than postInstall to be passed
through to mkDerivation, which is very useful when customising or
debugging a package.
2019-12-16 13:27:10 +00:00
worldofpeace 9665977f4c makeDesktopItem: add desktop file validation
This uses desktop-file-validate in desktop-file-utils.
It can be turned off if wanted.
2019-12-15 19:04:10 -05:00
Jan Tojnar aa3cb8b00e
Merge branch 'staging-next' into staging 2019-12-14 23:09:23 +01:00
Jan Tojnar 429561978b
Merge branch 'master' into staging-next 2019-12-14 23:09:06 +01:00
Chuck 6498f468af WriteHaskell: Strip binaries
This reduces the size of Hello World [1] from 3.06 MiB to 678 KiB.

[1] As measured by nix-shell -p 'writers.writeHaskellBin "hello" {} "main = putStrLn \"hello\""' --run 'ls -l `which hello`'
2019-12-14 09:32:34 +01:00
Frederik Rietdijk dfdf1597a7 Merge master into staging-next 2019-12-13 11:43:39 +01:00
Andreas Rammhold 89647059dc
Merge pull request #75181 from wamserma/fix-modules-shrunk
modules-shrunk: preserve module priorities from build
2019-12-12 22:53:14 +01:00
Andreas Rammhold 2eaaf7aafd
buildRustCrate: move common build functions to a dedicated file
This means we aren't rebuilding hat file for each crate we are building
and the buildPhase expression is a lot easier to comprehent.
2019-12-12 13:55:04 +01:00
Andreas Rammhold 3f49d7a3ea
buildRustCrate: deduplicate dependency override code
The previous lines were only different in the kind of dependencies but
otherwise exactly the same. It makes the entire thing a bit more
readable by moving this into a function that takes care of this.
2019-12-12 01:03:41 +01:00
Andreas Rammhold 6ad22f5b4d
buildRustCrate: use less bash for the build script
We can get rid of a bunch of workarounds that were in the build script
before by just passing on the `crateBin` attribute.

Before we converted the list of attributes to a string only to convert
it back in bash during the build phase. We can do the entire looping
through builds in Nix and thus need no conversion and parsing of
attributes over and over again.

The big part that still remains bash is the heuristic that cargo
introduced and that we can't do at eval time.
2019-12-12 01:03:11 +01:00
Andreas Rammhold 5ad83267ed
buildRustCrate: reflow the way extraRustcOpts is constructed
This should make it more obvious that we have three parts to it and not
just one long gibberish string that makes up all of it.
2019-12-11 23:27:58 +01:00
Andreas Rammhold d37f001164
buildRustCrate: rename makeDeps function to mkRustcDepArgs
This should carry the function better then `makeDeps` as it isn't
producing deps but the rustc arguments required to link against those.
2019-12-11 23:23:55 +01:00
Andreas Rammhold f4aeabd04a
buildRustCrate: document and cleanup the symbol seeding
That code had been in the derivation for a while but no explanation was
given why that is needed. It might be helpful to our future selfs to
document why things are done the way they are.
2019-12-11 23:23:55 +01:00
Andreas Rammhold db55d1f89d
buildRustCrate: use tr instead of sed (it reads a bit nicer)
I already have a few changes in here that will trigger rebuilds so I
might as well do that substitution now.
2019-12-11 22:40:19 +01:00
Andreas Rammhold 50b2ef28f7
buildRustCrate: move the color loggign & remove some runtime checks
The expression is already long and confusing enough without the color
stuff sprinkled in. Moving it to a dedicated file makes sense.

I switched a bit of the color support code to pure Nix since there
wasn't much point in doing that in bash while we can just do it in Nix.
2019-12-11 22:35:44 +01:00
Andreas Rammhold 0aac0e8d2c
buildRustCrate: builtins -> lib where possible
We can just use `lib` instead of `builtins` in all cases but the
`hashString` case. Also changed a few lines to make use of some optional
helpers from lib.
2019-12-11 22:01:36 +01:00
worldofpeace 25879b3ebb
Merge pull request #75450 from KoviRobi/fix-wrap-gapps-hook-shell-quoting
wrapGAppsHook: don't add empty variables (see also #75443)
2019-12-10 23:57:57 -05:00
Frederik Rietdijk f3618342ec Merge staging-next into staging 2019-12-10 19:01:27 +01:00
Kovacsics Robert 2c8c8f2961 wrapGAppsHook: don't add empty variables
Adding empty variables can lead to this problem:

```diff
 wrapProgram \
     ./pye_menu_shell \
     --prefix PATH : /nix/store/4c3z5r6yxsf2cxwwyazhdn92xixn4j5b-python3-3.7.5/bin:/nix/store/b3l3niilvqcxcsbxmd6sgqk1dy1rk81c-pye-menu-1.0/bin:/nix/store/y8j1cfj8d9r5rbbxc22w7hnfjw5f4fd3-cairo-1.16.0-dev/bin:/nix/store/6mg7lfbdh9pgx7pbxr3544qqbrigdl1q-freetype-2.10.1-dev/bin:/nix/store/gpszqcy0xi0lavbbjdq82zkkjp3jbp2a-bzip2-1.0.6.0.1-bin/bin:/nix/store/031c5pk5lzabgmpqpyd46hzi625as6bp-libpng-apng-1.6.37-dev/bin:/nix/store/f8kl7kmpv130aw9zm542p74a3hg0yc13-fontconfig-2.12.6-bin/bin:/nix/store/bqp30vkncmm222mjvwggz0s7p318sflj-expat-2.2.7-dev/bin:/nix/store/w57xa8g4s4aviwmqwgra7m5hwj2b005m-glib-2.60.7-dev/bin:/nix/store/v5d4966ahvfir2hwpv003022f3pb7vik-gettext-0.19.8.1/bin:/nix/store/qpvxhl1jr0fxnrx9idnpdagqs00m5m2z-glib-2.60.7/bin \
     --set PYTHONNOUSERSITE true \
     --set GDK_PIXBUF_MODULE_FILE /nix/store/7ddlakx6xjczqbfs80xjd14f30fzadws-gdk-pixbuf-2.38.1/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache \
     --prefix XDG_DATA_DIRS : /nix/store/0snjc1qg89zqn3v35l9d55xrykh9nj5c-gtk+3-3.24.10/share/gsettings-schemas/gtk+3-3.24.10:/nix/store/b41z51vdv11n6df8ki5vj8dynxw98f9l-gsettings-desktop-schemas-3.32.0/share/gsettings-schemas/gsettings-desktop-schemas-3.32.0:/nix/store/0snjc1qg89zqn3v35l9d55xrykh9nj5c-gtk+3-3.24.10/share/gsettings-schemas/gtk+3-3.24.10 \
-     --prefix GST_PLUGIN_SYSTEM_PATH_1_0 : \
+     --prefix GST_PLUGIN_SYSTEM_PATH_1_0 : "" \
     --prefix GI_TYPELIB_PATH : /nix/store/0snjc1qg89zqn3v35l9d55xrykh9nj5c-gtk+3-3.24.10/lib/girepository-1.0:/nix/store/z29l5xaaxh1s0697mcldj71ab0zshry1-librsvg-2.44.15/lib/girepository-1.0:/nix/store/pija1xzm7izxfb5m2hvhvlwp1l38ffxa-gobject-introspection-1.60.2/lib/girepository-1.0 \
-     --prefix GRL_PLUGIN_PATH :
+     --prefix GRL_PLUGIN_PATH : ""
```
Where the diff is to highlight the problem: we don't have a valid value
for GST_PLUGIN_SYSTEM_PATH_1_0 or GRL_PLUGIN_PATH, and instead of
passing the empy string, the empty string gets unquoted somewhere, so we
end up passing no arguments, thus the parser in wrapProgram takes
--prefix as the argument of GST_PLUGIN_SYSTEM_PATH_1_0, and then
GI_TYPELIB_PATH is missing it's --prefix so wrapProgram complains/dies.

The easiest change is to not add empty arguments to the wrapper
2019-12-10 17:19:10 +00:00
Jan Tojnar 6b73d29775
Merge pull request #73266 from demin-dmitriy/fix-libredirect-open-bug
libredirect: fix argument forwarding in `open*` functions
2019-12-10 00:46:21 +01:00
Alyssa Ross ee49b45a7b fetchFromGitLab: improve readability 2019-12-09 01:33:16 +00:00
Lancelot SIX 337cd5cc8a vmTools.fillDiskWithRPMs: fix cpio use
When updating to cpio-2.13 in fe758f5fa3,
a patch from SUSE was dropped. This patch was intended to resolve
CVE-2015-1197, and introduced the '--extract-over-symlink' option to
disable its own effects.

The CVE-2015-1197 was fixed in cpio-2.13 release[1] by other means,
making this patch useless.

Given that this patch is no longer used, we do not need to disable its
effects anymore with the `--extract-over-symlink` argument switch.

This Commit fixes #74984

[1] https://lists.gnu.org/archive/html/info-gnu/2019-11/msg00002.html
2019-12-08 21:03:39 +01:00
Bjørn Forsman e488670764 makeDesktopItem: use runCommandLocal
This derivation only creates a simple text file, so it makes sense to do
it locally.

On my setup this reduces build time from 2.2s to 1.2s.
2019-12-08 20:47:27 +01:00
Markus S. Wamser 34b04a6364 modules-shrunk: preserve module priorities from build
depmod looks for files modules.order and modules.builtin which are
generated at kernel build time but were previously not passed to
the modules-shrunk derivation
2019-12-07 23:04:17 +01:00
tomberek 81b0a20dfa buildImageWithNixDb: export USER (#74959)
dockerTools.buildImageWithNixDb: export USER

Changes to Nix user detection (./src/nix-channel/nix-channel.cc#L-166)
cause this function to error. Exporting USER fixes this.
2019-12-07 10:06:42 +01:00
lewo 4c1f7320ad
Merge pull request #75038 from nlewo/fix-dockertools
dockerTools.examples.nix: set USER in the container config
2019-12-06 08:59:15 +01:00
Frederik Rietdijk c9a19b5f7e Merge staging-next into staging 2019-12-05 10:19:54 +01:00
Antoine Eiche 09a669478f dockerTools.examples.nix: set USER in the container config
A USER is required by Nix.
See 9348f9291e/src/libutil/util.cc (L478).
2019-12-05 09:45:51 +01:00
Frederik Rietdijk 51ef7c3e49 Merge master into staging-next 2019-12-05 09:14:08 +01:00
Mateusz Kowalczyk 1451a52a38 Remove myself (fuuzetsu) from maintainer lists
I haven't been doing any maintenance for a long time now and not only
do I get notified, it also creates a fake impression that all these
packages had at least one maintainer when in practice they had none.
2019-12-05 16:29:48 +09:00
worldofpeace b07e790963
Merge pull request #74969 from marsam/remove-empty-inherits
treewide: remove empty inherits
2019-12-04 23:17:08 +00:00
Profpatsch c5c5465fe4 pkgs/build-support/trivial-builders: remove runCommandCCLocal
We shouldn’t force the user to have a C compiler in scope, just
because the derivation is forced to build locally. That can’t be
counted as “lightweight” anymore.

Co-Authored-By: Silvan Mosberger<contact@infinisil.com>
2019-12-04 21:17:01 +01:00
Profpatsch 8deaf41d60 pkgs/build-support/trivial-builders: add runCommandLocal
A definition I’ve been copy-pasting everywhere so far, so it’s finally
time to add it to nixpkgs.

I’m using a remote builder for my regular nix builds, so trivial
`runCommand`s which first try a substitution and then copy the inputs
to the builder to run for 0.2s are quite noticable.

If we just always build these, we gain some build time, so let’s make
it easy to switch from remote to local.
2019-12-04 21:17:01 +01:00
Mario Rodas e44d7c14f5
treewide: remove empty let blocks 2019-12-04 06:00:00 -05:00
Alyssa Ross fe16f7d7f9
fetchFromGitiles: init
This has the same motivation as fetchFromGitHub/fetchFromGitLab --
it's cheaper to download a tarball of a single revision than it is to
download a whole history.

I could have gone with domain/group/repo, like fetchFromGitLab, but it
would have made implementation more difficult, and this syntax means
it's a drop-in replacement for fetchgit, so I decided it wasn't worth
it.
2019-12-02 22:44:33 +00:00
Matthew Bauer 162a935d3c
Merge pull request #74226 from bhpdt/fix/idirafter-stdenv
stdenv: Fix gcc -idirafter shell glob in cc-wrapper
2019-12-02 13:41:28 -05:00
Frederik Rietdijk 9f84415541 Merge staging-next into staging 2019-11-30 08:58:01 +01:00
Jörg Thalheim 435c3ecde7
rust: add support for armv6l-linux and armv7l-linux (#73472)
rust: add support for armv6l-linux and armv7l-linux
2019-11-29 12:47:15 +00:00
Florian Klink a905fcb40d buildBazelPackage: set $USER environment variable (#74538)
tensorflow assumes $USER to be set to something, otherwise it complains
like this:

```
FATAL: $USER is not set, and unable to look up name of current user: (error: 0): Success
Traceback (most recent call last):
  File "./configure.py", line 1602, in <module>
    main()
  File "./configure.py", line 1399, in main
    _TF_MAX_BAZEL_VERSION)
  File "./configure.py", line 478, in check_bazel_version
    ['bazel', '--batch', '--bazelrc=/dev/null', 'version'])
  File "./configure.py", line 156, in run_shell
    output = subprocess.check_output(cmd)
  File "/nix/store/drr8qcgiccfc5by09r5zc30flgwh1mbx-python3-3.7.5/lib/python3.7/subprocess.py", line 411, in check_output
    **kwargs).stdout
  File "/nix/store/drr8qcgiccfc5by09r5zc30flgwh1mbx-python3-3.7.5/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
```

Spotted while changing the hash of its fixed-output derivation on
purpose.

We could also set this in the tensorflow-specific part, but very likely,
other programs will fail as well.
2019-11-28 16:14:45 -08:00
Andrew Dunham 5e24b9e4ed buildBazelPackage: also set the SSL_CERT_FILE environment variable 2019-11-28 13:39:20 -08:00
Andreas Rammhold 059faab65a
Merge pull request #73803 from andir/buildRustCrate-lib-output
buildRustCrate: add lib output
2019-11-28 15:36:45 +01:00
David Wood 4d26c18306 cc-wrapper: expose wrapper script as overridable attribute (#65813) 2019-11-27 14:15:56 -05:00
Andrew Dunham 3f86f21207 buildBazelPackage: allow specifying whether to remove rules_cc 2019-11-27 10:43:34 +01:00
Dima df7f3e0f98 nix-prefetch-git: list --branch-name in help
Listing the option and brief explanation.
Addresses: https://github.com/NixOS/nixpkgs/issues/74182
2019-11-26 20:43:23 +01:00
Andreas Rammhold 1b748554d5
buildRustCrate: add lib output
This cuts down the dependency tree on some rust builds where a crate not
just exposes a binary but also a library. `$out/lib` contained a bunch
of extra support files that among other information carry linker flags
(including the full path to link-time dependencies). Worst case this led
to some binary outputs depending on the full build closure of rust
crates.

Moving all the `$out/lib` files to `$lib/lib` solves this nicely.

`lib` might be a bit weird here as they are most of the time just rlib
files (rust libraries). Those are essential only required during
compilation but they can also be shared objects (like with traditional
C-style packages). Which is why I went with `lib` for the new output.

One of the caveats we are running into here is that we do not (always)
know ahead of time of a crate produces just a library or just a binary.
Cargo allows for some ambiguity regarding whether or not a crate
provides one, two, … binaries and libraries as it's outputs. Ideally we
would be able to rely on the `crateType` entirely but so far that isn't
the case. More work on that area might show how difficult that actually
is.
2019-11-26 15:05:01 +01:00
Ben Hipple 5f589d8d5d stdenv: Fix gcc -idirafter shell glob in cc-wrapper
If an empty string is passed to `-idirafter`, it breaks gcc. This commit makes
the stdenv less fragile by expanding out the shell glob and ensuring no empty
arguments get passed.
2019-11-26 00:55:16 +00:00
John Ericson f191360ad0 Merge remote-tracking branch 'upstream/staging-next' into staging 2019-11-25 15:59:05 -05:00
Jason Felice 7a77629490 java classpath hook: allow unbound $CLASSPATH 2019-11-24 23:30:05 -05:00
John Ericson 6bc456c91c Merge remote-tracking branch 'upstream/master' into ghcjs-cross-without-cc 2019-11-25 00:23:07 +00:00
John Ericson 63bd851e95 stdenv: Introduce hasCC attribute
Before, we'd always use `cc = null`, and check for that. The problem is
this breaks for cross compilation to platforms that don't support a C
compiler.

It's a very subtle issue. One might think there is no problem because we
have `stdenvNoCC`, and presumably one would only build derivations that
use that. The problem is that one still wants to use tools at build-time
that are themselves built with a C compiler, and those are gotten via
"splicing". The runtime version of those deps will explode, but the
build time / `buildPackages` versions of those deps will be fine, and
splicing attempts to work this by using `builtins.tryEval` to filter out
any broken "higher priority" packages (runtime is the default and
highest priority) so that both `foo` and `foo.nativeDrv` works.

However, `tryEval` only catches certain evaluation failures (e.g.
exceptions), and not arbitrary failures (such as `cc.attr` when `cc` is
null). This means `tryEval` fails to let us use our build time deps, and
everything comes apart.

The right solution is, as usually, to get rid of splicing. Or, baring
that, to make it so `foo` never works and one has to explicitly do
`foo.*`. But that is a much larger change, and certaily one unsuitable
to be backported to stable.

Given that, we instead make an exception-throwing `cc` attribute, and
create a `hasCC` attribute for those derivations which wish to
condtionally use a C compiler: instead of doing `stdenv.cc or null ==
null` or something similar, one does `stdenv.hasCC`. This allows quering
without "tripping" the exception, while also allowing `tryEval` to work.

No platform without a C compiler is yet wired up by default. That will
be done in a following commit.
2019-11-25 00:12:38 +00:00
John Ericson 6d3b7458cc
Merge pull request #74065 from Ericson2314/cc-versions
compilers, binutils: Add version and pname
2019-11-24 18:35:13 -05:00
John Ericson 814f9104d7
Merge pull request #72657 from cleverca22/vc4
Initial implementation of vc4 cross-compile
2019-11-24 16:04:15 -05:00
Craig Hall da3c053482 {cc,bintools}-wrapper: Inherit compiler version 2019-11-24 18:32:26 +00:00
John Ericson 747d5a3dbf Merge branch 'wrapper-pname-support-19.09' into wrapper-pname-support 2019-11-24 18:00:29 +00:00
John Ericson 4a0a297861 bintools-wrapper: Fix version 2019-11-24 17:57:06 +00:00
John Ericson d0d5136cce Merge remote-tracking branch 'upstream/master' into wrapper-pname-support 2019-11-24 17:25:07 +00:00
Craig Hall 99537e994f {cc,bintools}-wrapper: use cc pname/version if set 2019-11-24 16:33:21 +00:00
Ben Wolsieffer 83ac9c07e4 rust: add support for armv6l-linux and armv7l-linux 2019-11-23 19:19:31 -05:00
Jan Tojnar 937ec235b7
appimage: do not use alias 2019-11-23 21:28:01 +01:00
Herwig Hochleitner 42eeca722c emacs package setup hook: Fix Fix
A trailing separator in emacs load paths has semantics

cc @Ericson2314 @adisbladis https://github.com/NixOS/nixpkgs/pull/73287

This partially reverts commit ca782498a9
2019-11-23 19:49:56 +01:00
Michael Bishop 4aa1ffae04
initial implementation of vc4 cross-compile 2019-11-19 22:19:15 -04:00
Jan Tojnar da76deffd1
Merge branch 'master' into staging-next 2019-11-14 23:25:36 +01:00
John Ericson bca2e8255e treewide: CAML_LD_LIBRARY_PATH may be undefined 2019-11-14 13:44:07 -05:00
adisbladis e578b8499f fetchgitPrivate: Remove fetcher
Since Nix 2.0 we have `builtins.fetchGit` which is a much better
option since it runs in the evaluator and has access to the regular
users ssh keys.
2019-11-14 09:34:12 -08:00
Frederik Rietdijk d688c7cd05 Merge staging-next into staging 2019-11-12 14:32:56 +01:00
John Ericson 16f0fe7fe3 Merge remote-tracking branch 'upstream/master' into mingw-mcfthreads 2019-11-11 20:48:14 -05:00
Demin Dmitriy b6e37c3146 libredirect: fix access return type
`access` should return `int` not `int*`. Actually compiler produced
identical assembly with any of those types, so by luck it "just worked".
2019-11-12 04:29:11 +03:00
Demin Dmitriy 6432f92e42 libredirect: fix argument forwarding in open* functions
Flag `O_TMPFILE` was added in Linux 3.11. It affects whether or not
`mode` argument should be passed.
2019-11-12 04:29:11 +03:00
John Ericson 38ebb8ff82 fetchurl: Eliminate pointless cross differences 2019-11-11 00:25:24 -05:00
John Ericson f7fdc997bf Merge commit 'afa48f16f265fd3e88073bca7929e1e103bd3dc3' into bash-no-undef-vars 2019-11-08 13:58:27 -05:00
John Ericson ca782498a9 emacs package setup hook: Fix
1. Make the test more robust

2. EMACSLOADPATH may be initially undefined.

3. did `targetOffset` twice when meant `hostOffset` too
2019-11-08 13:28:59 -05:00
John Ericson fae8d2627d cc-wrapper, bintools-wrapper: Remove now unneeded set {+,-}u 2019-11-04 19:57:14 -05:00
John Ericson c5cf009996 multiple-outputs setup hook: Remove accidentally added tab
I don't want to just rebase this away because the original commit is
also in #72074.
2019-11-04 19:51:04 -05:00
Sebastian Jordan 8db9c3ef45 nix-prefetch-github: Delete redundant source, build from python3Packages instead 2019-11-04 09:14:00 +01:00
John Ericson 7eecf4f8fb find-xml-catalogues: Ensure XML_CATALOG_FILES is defined 2019-11-01 14:44:44 -04:00
John Ericson 2811b032d6 treewide: Make still dont* Variables are optional in most cases
Go beyond the obvious setup hooks now, with a bit of sed, with a skipped case:

 - cc-wrapper's `dontlink`, because it already is handled.

Also, in nix files escaping was manually added.

EMP
2019-11-01 14:44:44 -04:00
John Ericson 545e7518a8 bintools-wrapper: Don't stop set -u-ing
Same justification as previous commit.
2019-11-01 14:44:43 -04:00
John Ericson 43a98868ae common setup hooks: set -u robustness
Explicitly handle `dont*` and friends not being defined.
2019-11-01 14:44:43 -04:00
Jörg Thalheim 56240d7f20
Merge pull request #71899 from decentriq/aslemmer/build-rust-package-add-target
build-support/rust: Add target option
2019-11-01 15:46:47 +00:00
exfalso a588b1dfbc build-support/rust: Add target option 2019-10-24 11:39:08 +01:00
Lassulus f417dd9c89
Merge pull request #71586 from Lassulus/writers
writers: fix writeC & tests
2019-10-23 21:45:29 +02:00