Commit graph

110 commits

Author SHA1 Message Date
John Ericson 9c213398b3 lib: Clean up how linux and gcc config is specified
Second attempt of 8929989614589ee3acd070a6409b2b9700c92d65; see that
commit for details.

This reverts commit 0bc275e634.
2021-01-23 10:01:28 -05:00
Jonathan Ringer 0bc275e634
Revert "lib: Clean up how linux and gcc config is specified"
This is a stdenv-rebuild, and should not be merged
into master

This reverts commit 8929989614.
2021-01-22 14:07:06 -08:00
John Ericson 8929989614 lib: Clean up how linux and gcc config is specified
The `platform` field is pointless nesting: it's just stuff that happens
to be defined together, and that should be an implementation detail.

This instead makes `linux-kernel` and `gcc` top level fields in platform
configs. They join `rustc` there [all are optional], which was put there
and not in `platform` in anticipation of a change like this.

`linux-kernel.arch` in particular also becomes `linuxArch`, to match the
other `*Arch`es.

The next step after is this to combine the *specific* machines from
`lib.systems.platforms` with `lib.systems.examples`, keeping just the
"multiplatform" ones for defaulting.
2021-01-21 22:44:09 -05:00
Daiderd Jordan 788f61cf3e
Merge pull request #85545 from LnL7/meta-available-flags
meta: expose availability flags in derivation metadata
2020-11-28 18:57:47 +01:00
Silvan Mosberger 560bb92473
Merge pull request #76794 from dudebout/document-nix-env-multiple-output-install-bug
document nix-env bug relating to multiple output installation
2020-09-05 15:40:26 +02:00
Nicolas Dudebout 611258f063 document nix-env bug relating to multiple output installation 2020-09-05 05:31:54 -04:00
Silvan Mosberger 911497988f
Merge pull request #95536 from Infinisil/inputDerivation
mkDerivation: Introduce .inputDerivation for shell.nix build convenience
2020-08-31 15:46:41 +02:00
Silvan Mosberger 3e1a40df75
mkDerivation: Introduce .inputDerivation for shell.nix build convenience
This introduces the .inputDerivation attribute on all derivations
created with mkDerivation. This is another derivation that can always
build successfully and whose runtime dependencies are the build time
dependencies of the original derivation.

This allows easy building and distributing of all derivations needed to
enter a nix-shell with

  nix-build shell.nix -A inputDerivation
2020-08-16 00:24:48 +02:00
Aaron Janse eb970b6241 mkDerivation: handle Redox in cmakeFlags 2020-08-10 19:52:54 -07:00
John Ericson 162c0c3e61 mkDerivation: Don't need to specify pkg-config for meson any more
Env var will work fine.
2020-05-27 16:07:58 +00:00
John Ericson 05d26adb0a mkDerivation mesonFlags: Fix arm cpu families
In my haste to unbreak eval, I screwed up and got the bit-widths,
backwards.
2020-04-28 22:56:20 -04:00
John Ericson cf858e6d57 mkDerivation mesonFlags: Fix cross file logic to handle more cases
Otherwise eval breaks
2020-04-28 16:51:44 -04:00
John Ericson 8245230753 meson: Make target-agnostic
The cross file is added in the `mkDerivation`. It isn't nice putting
build tool-specific stuff here, but our current architecture gives us
little alternative.
2020-04-28 10:55:33 -04:00
Daiderd Jordan 950ac2bc8f
meta: expose availability flags in derivation metadata
Currently it's not possible to determine the reason why a package is
unavailable without evaluating nixpkgs multiple times with different
settings. eg.

    nix-repl> :p android-studio.meta
    { available = false; broken = false; unfree = true; unsupported = true; ... }

The following snippet is an example that uses this information to query
the availability information of all packages in nixpkgs, giving an
overview of all the packages currently marked as broken, etc.

    { pkgs }:

    with import <nixpkgs/lib>;

    let

      mapPkgs =
        let mapPkgs' = path: f: mapAttrs (n: v:
          let result = builtins.tryEval (v ? meta); in
          if !result.success then {} else
          if isDerivation v then f (path ++ [n]) v else
          if isAttrs v && v.recurseForDerivations or false then mapPkgs' (path ++ [n]) f v else
          {}
        );
        in mapPkgs' [];

      getMeta = path: drv:
        if drv.meta ? available then
          let meta = {
            pkg = concatStringsSep "." path;
            inherit (drv.meta) broken unfree unsupported insecure;
          };
          in builtins.trace meta.pkg meta
          else {};

      metaToList = attrs: flatten (map (v: if v ? pkg then v else metaToList v) (attrValues attrs));

    in metaToList (mapPkgs getMeta pkgs)
2020-04-19 12:53:04 +02:00
arcnmx 8cedc7fc6c stdenv: correct cross CMAKE_SYSTEM_NAME
As described in cmake cross instructions, CMAKE_SYSTEM_NAME should be
set to "Generic" if there is no applicable OS:

https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/CrossCompiling#setting-up-the-system-and-toolchain
2019-07-27 15:23:31 -07:00
volth f3282c8d1e treewide: remove unused variables (#63177)
* treewide: remove unused variables

* making ofborg happy
2019-06-16 19:59:05 +00:00
Matthew Bauer 7f23d9fc5f
Merge pull request #59755 from matthewbauer/patches-in-make-derivation
make-derivation: put patches in all derivations
2019-04-23 18:33:04 -04:00
volth b0193379e5
make-derivation.nix: minor
it seems to change nothing (.platform is always there), just to be consisted with the style of other checks
2019-04-21 20:05:13 +00:00
volth b1c3581b46
make-derivation.nix: @matthewbauer's review 2019-04-21 19:16:28 +00:00
volth 5d87bc2650 fix bootstrap when platform.gcc.arch=="skylake" 2019-04-21 16:37:49 +00:00
Matthew Bauer c7ccb9f197 make-derivation: put patches in all derivations
This puts patches in all derivations even if it unspecified by the
derivation. By default it will be an empty list. This simplifies
overrides, as we can now assume that patches is a valid name so that
this works:

self: super: {
  mypkg = super.pkg.overrideAttrs (o: {
    patches = o.patches ++ [ ./my-very-own.patch ];
  });
}

That is, you don’t need to provide a default "or []", make-derivation
provides one for you.

Unfortunately, this is a mass rebuild.
2019-04-16 22:51:26 -04:00
Graham Christensen 817c933878
check-env: don't execute check-meta.nix 15,000 times
Generated from https://github.com/NixOS/nix/pull/2761:

```
                                                                                          ns     calls ns/call
- /home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:22:5 591200 15026 39.3451
+ /home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:22:5 8744   308   28.3896
```

more, generated by:

```
$ NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1 nix-instantiate ./pkgs/top-level/release.nix -A unstable > before 2>&1
$ jq -r '.functions | map((.name + ":" + .file + ":" + (.line|tostring) + ":" + (.column|tostring) + " " + (.count|tostring))) | .[]' before | sort  > before.list
```

applying this patch, then:

```
$ NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1 nix-instantiate ./pkgs/top-level/release.nix -A unstable > after 2>&1
$ jq -r '.functions | map((.name + ":" + .file + ":" + (.line|tostring) + ":" + (.column|tostring) + " " + (.count|tostring))) | .[]' after | sort  > after.list
```

and then diffing before.list and after.list to get:

```
                                                                                                        calls
- :/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:4:1               7513
+ :/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:4:1               154

- mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:23                7513
+ mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:23                154

- mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:26                7513
+ mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:26                154

- onlyLicenses:/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:21:18 15026
+ onlyLicenses:/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:21:18 308
```

The following information is from `NIX_SHOW_STATS=1 GC_INITIAL_HEAP_SIZE=4g nix-env -f ./outpaths.nix -qaP --no-name --out-path --arg checkMeta true`:

| stat                       | before         | after          | Δ               | Δ%      |
|:---------------------------|---------------:|---------------:|:----------------|--------:|
| **cpuTime**                |        179.915 |        145.543 | 🡖 34.372        | -19.10% |
| **envs-bytes**             |  3,900,878,824 |  3,599,483,208 | 🡖 301,395,616   |  -7.73% |
| **envs-elements**          |    214,426,071 |    185,881,709 | 🡖 28,544,362    | -13.31% |
| **envs-number**            |    136,591,891 |    132,026,846 | 🡖 4,565,045     |  -3.34% |
| **gc-heapSize**            | 11,400,048,640 | 12,314,890,240 | 🡕 914,841,600   |   8.02% |
| **gc-totalBytes**          | 25,976,902,560 | 24,510,740,176 | 🡖 1,466,162,384 |  -5.64% |
| **list-bytes**             |  1,665,290,080 |  1,665,290,080 | 0               |         |
| **list-concats**           |      7,264,417 |      7,264,417 | 0               |         |
| **list-elements**          |    208,161,260 |    208,161,260 | 0               |         |
| **nrAvoided**              |    191,359,386 |    179,693,661 | 🡖 11,665,725    |  -6.10% |
| **nrFunctionCalls**        |    119,665,062 |    116,348,547 | 🡖 3,316,515     |  -2.77% |
| **nrLookups**              |     80,996,257 |     76,069,825 | 🡖 4,926,432     |  -6.08% |
| **nrOpUpdateValuesCopied** |    213,930,649 |    213,930,649 | 0               |         |
| **nrOpUpdates**            |     12,025,937 |     12,025,937 | 0               |         |
| **nrPrimOpCalls**          |     88,105,604 |     86,451,598 | 🡖 1,654,006     |  -1.88% |
| **nrThunks**               |    196,842,044 |    175,126,701 | 🡖 21,715,343    | -11.03% |
| **sets-bytes**             |  7,678,425,776 |  7,285,767,928 | 🡖 392,657,848   |  -5.11% |
| **sets-elements**          |    310,241,340 |    294,373,227 | 🡖 15,868,113    |  -5.11% |
| **sets-number**            |     29,079,202 |     27,601,310 | 🡖 1,477,892     |  -5.08% |
| **sizes-Attr**             |             24 |             24 | 0               |         |
| **sizes-Bindings**         |              8 |              8 | 0               |         |
| **sizes-Env**              |             16 |             16 | 0               |         |
| **sizes-Value**            |             24 |             24 | 0               |         |
| **symbols-bytes**          |     16,474,666 |     16,474,676 | 🡕 10            |   0.00% |
| **symbols-number**         |        376,426 |        376,427 | 🡕 1             |   0.00% |
| **values-bytes**           |  6,856,506,288 |  6,316,585,560 | 🡖 539,920,728   |  -7.87% |
| **values-number**          |    285,687,762 |    263,191,065 | 🡖 22,496,697    |  -7.87% |

The following information is from `NIX_SHOW_STATS=1 GC_INITIAL_HEAP_SIZE=4g nix-instantiate ./nixos/release-combined.nix -A tested`:

| stat                       | before         | after          | Δ               | Δ%     |
|:---------------------------|---------------:|---------------:|:----------------|-------:|
| **cpuTime**                |        256.071 |        237.531 | 🡖 18.54         | -7.24% |
| **envs-bytes**             |  7,111,004,192 |  7,041,478,520 | 🡖 69,525,672    | -0.98% |
| **envs-elements**          |    346,236,940 |    339,588,487 | 🡖 6,648,453     | -1.92% |
| **envs-number**            |    271,319,292 |    270,298,164 | 🡖 1,021,128     | -0.38% |
| **gc-heapSize**            |  8,995,291,136 | 10,110,009,344 | 🡕 1,114,718,208 | 12.39% |
| **gc-totalBytes**          | 37,172,737,408 | 36,878,391,888 | 🡖 294,345,520   | -0.79% |
| **list-bytes**             |  1,886,162,656 |  1,886,163,472 | 🡕 816           |  0.00% |
| **list-concats**           |      6,898,114 |      6,898,114 | 0               |        |
| **list-elements**          |    235,770,332 |    235,770,434 | 🡕 102           |  0.00% |
| **nrAvoided**              |    328,829,821 |    326,618,157 | 🡖 2,211,664     | -0.67% |
| **nrFunctionCalls**        |    240,850,845 |    239,998,495 | 🡖 852,350       | -0.35% |
| **nrLookups**              |    144,849,632 |    142,126,339 | 🡖 2,723,293     | -1.88% |
| **nrOpUpdateValuesCopied** |    251,032,504 |    251,032,504 | 0               |        |
| **nrOpUpdates**            |     17,903,110 |     17,903,110 | 0               |        |
| **nrPrimOpCalls**          |    140,674,913 |    139,485,975 | 🡖 1,188,938     | -0.85% |
| **nrThunks**               |    294,643,131 |    288,678,022 | 🡖 5,965,109     | -2.02% |
| **sets-bytes**             |  9,464,322,192 |  9,456,172,048 | 🡖 8,150,144     | -0.09% |
| **sets-elements**          |    377,474,889 |    377,134,877 | 🡖 340,012       | -0.09% |
| **sets-number**            |     50,615,607 |     50,616,875 | 🡕 1,268         |  0.00% |
| **sizes-Attr**             |             24 |             24 | 0               |        |
| **sizes-Bindings**         |              8 |              8 | 0               |        |
| **sizes-Env**              |             16 |             16 | 0               |        |
| **sizes-Value**            |             24 |             24 | 0               |        |
| **symbols-bytes**          |      3,147,102 |      3,147,064 | 🡖 38            | -0.00% |
| **symbols-number**         |         82,819 |         82,819 | 0               |        |
| **values-bytes**           | 11,147,448,768 | 10,996,111,512 | 🡖 151,337,256   | -1.36% |
| **values-number**          |    464,477,032 |    458,171,313 | 🡖 6,305,719     | -1.36% |
2019-04-11 19:21:38 -04:00
Frederik Rietdijk 2fcb11a244 Merge staging-next into master 2019-03-01 09:06:20 +01:00
Matthew Bauer b86e62d30d llvm: support cross compilation with useLLVM flag
You can build (partially) with LLVM toolchain using the useLLVM flag.
This works like so:

  nix-build -A hello --arg crossSystem '{ system =
    "aarch64-unknown-linux-musl"; useLLVM = true }'

also don’t separate debug info in lldClang

It doesn’t work currently with that setup hook. Missing build-id?
2019-02-26 19:45:35 -05:00
Daiderd Jordan 7ec53a932f
stdenv: only set __darwinAllowLocalNetworking on darwin
This is a darwin only nix attribute for sandbox builds, it can be
ignored on other platforms to avoid unnecessary rebuilds.
2019-02-14 21:38:08 +01:00
Matthew Bauer 5c09d977c7 Merge remote-tracking branch 'origin/master' into staging 2019-02-09 12:14:06 -05:00
Vladimír Čunát ff101946a8
Merge branch 'master' into staging-next
Hydra nixpkgs: ?compare=1503899
2019-02-06 14:24:46 +01:00
Matthew Bauer c6f8f8d98d make-derivation: only modify name when name is given
This preserves Nix’s native error handling of missing name:

  error: derivation name missing
2019-02-04 09:15:03 -05:00
Vincent Laporte 0d17ecce2c
mkDerivation: cleaner handling of the name argument 2019-02-01 16:02:42 +00:00
Vladimír Čunát 5effa4e0f9
Merge branch 'master' into staging-next
Comments on conflicts:
- llvm: d6f401e1 vs. 469ecc70 - docs for 6 and 7 say the default is
  to build all targets, so we should be fine
- some pypi hashes: they were equivalent, just base16 vs. base32
2019-02-01 09:22:29 +01:00
Matthew Bauer 8020bd6869
Merge pull request #35884 from dtzWill/fix/man-in-outputsToInstall
default to including "man" in outputsToInstall
2019-01-31 14:55:12 -05:00
Matthew Bauer e2fe4c2d49 make-derivation: fix ordering of conditionals
cross should have higher precedence
2019-01-28 11:38:30 -05:00
Matthew Bauer 9fd1c170cc make-derivation: try to fix stdenv.cc == null conditional
sometimes this gets an infinite recursion error
2019-01-28 10:41:19 -05:00
Matthew Bauer 7e589e5594 make-derivation: fix position in trace
For a long time now, tracing has been broken in Nixpkgs. So when you
have an eval error you would get something like this:

  error: while evaluating the attribute 'buildInputs' of the derivation 'hello-2.10' at /home/mbauer/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:185:11:
  while evaluating 'chooseDevOutputs' at /home/mbauer/nixpkgs/lib/attrsets.nix:474:22, called from undefined position:
  while evaluating 'optionals' at /home/mbauer/nixpkgs/lib/lists.nix:257:5, called from /home/mbauer/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:132:17:

This is coming from how Nix handles string context and how
make-derivation messes with the "name" attribute. This commit should
restore the old behavior so you get a nice line number like:

  error: while evaluating the attribute 'buildInputs' of the derivation 'hello-2.10' at /home/mbauer/nixpkgs/pkgs/applications/misc/hello/default.nix:4:3:
  while evaluating 'chooseDevOutputs' at /home/mbauer/nixpkgs/lib/attrsets.nix:474:22, called from undefined position:
  while evaluating 'optionals' at /home/mbauer/nixpkgs/lib/lists.nix:257:5, called from /home/mbauer/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:132:17:

NOTE: This will still be broken for cross compilation due to the
prefixes we are adding to name.
2019-01-28 09:25:47 -05:00
Matthew Bauer ae16dd1a15
stdenv/make-derivation: don't hide broken packages
This behavior ended up breaking the handleEvalIssue functionality by hiding those packages. So something like this:

$ nix-env -iA nixpkgs.zoom-us

would silently fail, without telling the user how to fix it! Regardless, this "bug" should be handled in Nix - not Nixpkgs.

Fixes #38952.
2019-01-27 17:02:17 -05:00
Matthew Bauer 8babcc3d44
Merge branch 'master' into fix/man-in-outputsToInstall 2019-01-26 10:51:12 -05:00
Frederik Rietdijk a1a5ea5943 stdenv: make checkInputs native
We can't run the checkPhase when build != host, so we may as well make
the checkInputs native.

This signicantly improves the situation of Python packages when enabling
strictDeps.
2019-01-13 14:43:18 +01:00
Jan Tojnar c5881ec2c9
Merge remote-tracking branch 'upstream/master' into staging 2018-11-30 20:09:45 +01:00
Matthew Bauer 28e2277305 make-derivation: remove selfConsistent check
version is set in lots of places but might not need to be in a name.

Alternative to #50364.
2018-11-30 17:58:33 +01:00
Frederik Rietdijk 1828a5c5ba Merge master into staging-next 2018-11-30 17:46:21 +01:00
Matthew Bauer e6834171b9
Merge pull request #51183 from matthewbauer/make-derivation-no-assertions
Fix breaking changes to make-derivation
2018-11-29 19:35:55 -06:00
Vaibhav Sagar b5504e26b0 pkgs/stdenv/generic/make-derivation.nix: s/targetPlatform/hostPlatform/ 2018-11-29 17:54:59 -05:00
Matthew Bauer a56fe056ec make-derivation: don’t add host suffix if there is no c compiler
Some trivial builders use the name attr to choose the exec name
produced. For example nixos-install,

  nixos-install = makeProg {
    name = "nixos-install";
    src = ./nixos-install.sh;
    nix = config.nix.package.out;
    path = makeBinPath [ nixos-enter ];
  };

When cross compiling, this puts the prog in,

  /bin/nixos-install-powerpc64le-unknown-linux-gnu
2018-11-28 12:44:10 -06:00
Matthew Bauer d64f288154 make-derivation: don’t assert on separateDebugInfo
When separateDebugInfo = true & !stdenv.hostPlatform.isLinux, we
always gave an error. There is no reason to do this. Instead, just
don’t add separate debug info when we aren’t on Linux.
2018-11-28 12:29:00 -06:00
Matthew Bauer 13e45aeed3 make-derivation: remove selfConsistent check
version is set in lots of places but might not need to be in a name.

Alternative to #50364.
2018-11-28 12:26:21 -06:00
Vladimír Čunát a5de78b7d7
Merge branch 'master' into staging-next 2018-11-26 10:28:00 +01:00
Matthew Bauer 7eeb02d47b Revert "make-derivation: add disallowedReferences in strictDeps"
This reverts commit 8dbfb61e46.
Also reverts commit fc99c337ed.

Fixes #50915
2018-11-25 15:53:28 -06:00
Frederik Rietdijk c31cb577ae Merge master into staging-next 2018-11-22 09:57:08 +01:00
Matthew Bauer fc99c337ed make-derivation: don’t disallow propagated native inputs
propagateNativeBuildInputs will end up going in the output derivation.
This case is allowed to end up in references because of that. Sorry
for the disruption!

Fixes #50865
2018-11-21 09:39:47 -06:00
Frederik Rietdijk 1d3bff25db Merge staging-next into staging 2018-11-11 14:28:08 +01:00