Commit graph

245 commits

Author SHA1 Message Date
John Ericson a302d7360f top-level: {build,host,target}Platform are defined in the stdenv instead
See #27069 for a discussion of this
2017-07-07 12:55:02 -04:00
John Ericson afc2023993 stdenv: Have mkDerivation pull the "extra" arguments from stdenv instead
Something more elaborate is needed for the "*Platform" arguments.
2017-07-07 12:16:51 -04:00
John Ericson 4cf4d7180d stdenv: Conservatively move mkDerivation into it's own file 2017-07-07 12:16:51 -04:00
Vladimír Čunát e8e57452f4 stdenv: separate all meta-checking code (~200 lines)
Only cosmetic changes are done otherwise.
Real refactoring is left for later.

There's a small slow-down on my machine:
$ time nix-env -qa -P >/dev/null
gets from ~2.8 to ~3.5 seconds (negligible change in RAM).
That's most likely caused by sharing less computation between different
mkDerivation calls, and I plan to improve that soon.
2017-07-07 12:16:26 -04:00
Vladimír Čunát dfc004e69c lib.lists.mutuallyExclusive: add function 2017-07-07 12:02:29 -04:00
Vladimír Čunát 5afcdc88fa stdenv: simple refactor to get rid of pos'
Suggested by Ericson2314.
2017-07-07 12:02:29 -04:00
Vladimír Čunát 7fdf18e892 stdenv: refactor (no change in semantics)
This just moves some expressions around in preparation to further changes.
2017-07-07 12:02:29 -04:00
John Ericson ad8d8fb2f5 stdenv: Simplify dependency code
This is a bit simpler now, but more importantly it scales better when I
double the number of sorts of dependencies as part of my cross
compilation work.
2017-06-29 17:45:08 -04:00
Vladimír Čunát 8004e79415
Merge branch 'master' into staging 2017-05-24 03:24:06 +02:00
John Ericson eaa509f33a stdenv: Rename isGNU to isHurd as GNU is a userland
Elsewhere, things called GNU indeed includes GNU/Linux or GNU/Hurd, but this
predicate was defined excluding Linux regardless of userland.
2017-05-22 13:55:26 -04:00
John Ericson 1dc6f15de9 stdenv: define is* predicates with hostPlatform.is*
This is a saner default until stdenv's are removed altogether
2017-05-22 00:25:02 -04:00
John Ericson c5c6606048 lib: Infer libc field of platform if not specified
This is especially useful when not cross compiling. It means we can
remove the `stdenv.isGlibc` predicate too.

Additionally, use this to simplify the logic to choose the
appropriate libiconv derivation.
2017-05-22 00:25:02 -04:00
Vladimír Čunát ef5844be6c
stdenv: disable audit-tmpdir on non-Linux for now
Without changing any hashes.
2017-05-06 13:19:07 +02:00
Eelco Dolstra 94d164dd7f
Add a setup hook for detecting $TMPDIR references in RPATHs and wrapper scripts 2017-05-04 20:23:57 +02:00
Domen Kožar e057e5927e Merge pull request #25427 from aneeshusa/fix-meta-priority-types
Fix meta priority types
2017-05-02 09:38:32 +02:00
Aneesh Agrawal d3acf9891c stdenv: More useful error message on bad meta attrs
This helps in debugging meta attribute type errors,
which are now enforced as of commit
90b9719f4f.
2017-05-02 01:45:30 -04:00
Dan Peebles f3a05a0fb3 stdenv: disable checkMeta by default until issues resolved
https://github.com/NixOS/nixpkgs/pull/25304#issuecomment-298385426
2017-05-01 13:51:12 -04:00
Eric Sagnes 7004919243 stdenv-generic: add meta attributes checks 2017-04-29 17:07:01 +09:00
Dan Peebles 90b9719f4f treewide: fix the remaining issues with meta attributes 2017-04-29 04:24:34 +00:00
Dan Peebles 1a4ca220e1 treewide: fix assorted issues revealed by the meta checker
Turns out a couple of the licenses were wrong, as well as being strings.
2017-04-28 23:07:42 -04:00
Dan Peebles 32ae4bfc20 stdenv-generic: add meta attribute checking
This is turned off by default but I think we should fix all packages to
respect it and then turn it on by default
2017-04-28 18:12:18 -04:00
Tuomas Tynkkynen ce56c99edc mkDerivation: Don't pass buildInputs to stdenv builder in nativeBuildInputs
When not cross compiling, nativeBuildInputs and buildInputs have
identical behaviour. Currently that is implemented by having
mkDerivation do a concatenation of those variables in Nix code and pass
that to the builder via the nativeBuildInputs attribute.

However, that has some annoying side effects, like `foo.buildInputs`
evaluating to `[ ]` even if buildInputs were specified in the nix
expression for foo.

Instead, pass buildInputs and nativeBuildInputs in separate variables as
usual, and move the logic of cross compilation vs. native compilation to
the stdenv builder script. This is probably a tiny bit uglier but
fixes the previous problem.

Issue #4855.
2017-03-02 03:26:48 +02:00
Graham Christensen a9c875fc2e
nixpkgs: allow packages to be marked insecure
If a package's meta has `knownVulnerabilities`, like so:

    stdenv.mkDerivation {
      name = "foobar-1.2.3";

      ...

      meta.knownVulnerabilities = [
        "CVE-0000-00000: remote code execution"
        "CVE-0000-00001: local privilege escalation"
      ];
    }

and a user attempts to install the package, they will be greeted with
a warning indicating that maybe they don't want to install it:

    error: Package ‘foobar-1.2.3’ in ‘...default.nix:20’ is marked as insecure, refusing to evaluate.

    Known issues:

     - CVE-0000-00000: remote code execution
     - CVE-0000-00001: local privilege escalation

    You can install it anyway by whitelisting this package, using the
    following methods:

    a) for `nixos-rebuild` you can add ‘foobar-1.2.3’ to
       `nixpkgs.config.permittedInsecurePackages` in the configuration.nix,
       like so:

         {
           nixpkgs.config.permittedInsecurePackages = [
             "foobar-1.2.3"
           ];
         }

    b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
    ‘foobar-1.2.3’ to `permittedInsecurePackages` in
    ~/.config/nixpkgs/config.nix, like so:

         {
           permittedInsecurePackages = [
             "foobar-1.2.3"
           ];
         }

Adding either of these configurations will permit this specific
version to be installed. A third option also exists:

  NIXPKGS_ALLOW_INSECURE=1 nix-build ...

though I specifically avoided having a global file-based toggle to
disable this check. This way, users don't disable it once in order to
get a single package, and then don't realize future packages are
insecure.
2017-02-24 07:41:05 -05:00
Graham Christensen 59d61ef34a Revert "nixpkgs: allow packages to be marked insecure" 2017-02-23 09:41:42 -05:00
Graham Christensen 38771badd3
nixpkgs: allow packages to be marked insecure
If a package's meta has `knownVulnerabilities`, like so:

    stdenv.mkDerivation {
      name = "foobar-1.2.3";

      ...

      meta.knownVulnerabilities = [
        "CVE-0000-00000: remote code execution"
        "CVE-0000-00001: local privilege escalation"
      ];
    }

and a user attempts to install the package, they will be greeted with
a warning indicating that maybe they don't want to install it:

    error: Package ‘foobar-1.2.3’ in ‘...default.nix:20’ is marked as insecure, refusing to evaluate.

    Known issues:

     - CVE-0000-00000: remote code execution
     - CVE-0000-00001: local privilege escalation

    You can install it anyway by whitelisting this package, using the
    following methods:

    a) for `nixos-rebuild` you can add ‘foobar-1.2.3’ to
       `nixpkgs.config.permittedInsecurePackages` in the configuration.nix,
       like so:

         {
           nixpkgs.config.permittedInsecurePackages = [
             "foobar-1.2.3"
           ];
         }

    b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
    ‘foobar-1.2.3’ to `permittedInsecurePackages` in
    ~/.config/nixpkgs/config.nix, like so:

         {
           permittedInsecurePackages = [
             "foobar-1.2.3"
           ];
         }

Adding either of these configurations will permit this specific
version to be installed. A third option also exists:

  NIXPKGS_ALLOW_INSECURE=1 nix-build ...

though I specifically avoided having a global file-based toggle to
disable this check. This way, users don't disable it once in order to
get a single package, and then don't realize future packages are
insecure.
2017-02-17 20:49:49 -05:00
Eelco Dolstra 9d6a55aefd
~/.nixpkgs -> ~/.config/nixpkgs
The former is still respected as a fallback for config.nix for
backwards compatibility (but not for overlays because they're a new
feature).
2017-02-01 16:07:55 +01:00
Tuomas Tynkkynen 18599495c4 stdenv: make is64bit evaluate true on aarch64
This should fix the NSS build.
2017-01-29 20:28:14 +02:00
Tuomas Tynkkynen 7c8a060c09 stdenv: Bringup aarch64 architecture support 2017-01-25 00:01:51 +02:00
John Ericson bf17d6dacf top-level: Introduce buildPackages for resolving build-time deps
[N.B., this package also applies to the commits that follow it in the same
PR.]

In most cases, buildPackages = pkgs so things work just as before. For
cross compiling, however, buildPackages is resolved as the previous
bootstrapping stage. This allows us to avoid the mkDerivation hacks cross
compiling currently uses today.

To avoid a massive refactor, callPackage will splice together both package
sets. Again to avoid churn, it uses the old `nativeDrv` vs `crossDrv` to do
so. So now, whether cross compiling or not, packages with get a `nativeDrv`
and `crossDrv`---in the non-cross-compiling case they are simply the same
derivation. This is good because it reduces the divergence between the
cross and non-cross dataflow. See `pkgs/top-level/splice.nix` for a comment
along the lines of the preceding paragraph, and the code that does this
splicing.

Also, `forceNativeDrv` is replaced with `forceNativePackages`. The latter
resolves `pkgs` unless the host platform is different from the build
platform, in which case it resolves to `buildPackages`. Note that the
target platform is not important here---it will not prevent
`forcedNativePackages` from resolving to `pkgs`.

--------

Temporarily, we make preserve some dubious decisions in the name of preserving
hashes:

Most importantly, we don't distinguish between "host" and "target" in the
autoconf sense. This leads to the proliferation of *Cross derivations
currently used. What we ought to is resolve native deps of the cross "build
packages" (build = host != target) package set against the "vanilla
packages" (build = host = target) package set. Instead, "build packages"
uses itself, with (informally) target != build in all cases.

This is wrong because it violates the "sliding window" principle of
bootstrapping stages that shifting the platform triple of one stage to the
left coincides with the next stage's platform triple. Only because we don't
explicitly distinguish between "host" and "target" does it appear that the
"sliding window" principle is preserved--indeed it is over the reductionary
"platform double" of just "build" and "host/target".

Additionally, we build libc, libgcc, etc in the same stage as the compilers
themselves, which is wrong because they are used at runtime, not build
time. Fixing this is somewhat subtle, and the solution and problem will be
better explained in the commit that does fix it.

Commits after this will solve both these issues, at the expense of breaking
cross hashes. Native hashes won't be broken, thankfully.

--------

Did the temporary ugliness pan out? Of the packages that currently build in
`release-cross.nix`, the only ones that have their hash changed are
`*.gcc.crossDrv` and `bootstrapTools.*.coreutilsMinimal`. In both cases I
think it doesn't matter.

 1. GCC when doing a `build = host = target = foreign` build (maximally
    cross), still defines environment variables like `CPATH`[1] with
    packages.  This seems assuredly wrong because whether gcc dynamically
    links those, or the programs built by gcc dynamically link those---I
    have no idea which case is reality---they should be foreign. Therefore,
    in all likelihood, I just made the gcc less broken.

 2. Coreutils (ab)used the old cross-compiling infrastructure to depend on
    a native version of itself. When coreutils was overwritten to be built
    with fewer features, the native version it used would also be
    overwritten because the binding was tight. Now it uses the much looser
    `BuildPackages.coreutils` which is just fine as a richer build dep
    doesn't cause any problems and avoids a rebuild.

So, in conclusion I'd say the conservatism payed off. Onward to actually
raking the muck in the next PR!

[1]: https://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html
2017-01-24 11:37:56 -05:00
John Ericson 0ef8b69d12 top-level: Modernize stdenv.overrides giving it self and super
Document breaking change in 17.03 release notes
2017-01-13 10:36:11 -05:00
Domen Kožar 45f579b9e7 allowUnfree: mention the solution that works for nix-shell as well 2016-11-16 15:14:19 +01:00
Domen Kožar 62edf873aa Merge pull request #18660 from aneeshusa/add-override-attrs
mkDerivation: add overrideAttrs function
2016-10-30 11:32:15 +01:00
Joachim F 3d5630fac9 Merge pull request #19769 from groxxda/license
stdenv.hasLicense: ? supports nested lookup
2016-10-24 15:19:12 +02:00
Alexander Ried a0ac2ae35e stdenv: throwEvalHelp performance (#19779) 2016-10-22 20:24:56 +02:00
Alexander Ried 43ce115ca9 stdenv.hasLicense: ? supports nested lookup
this avoids one copy of the attrset
2016-10-22 02:43:13 +02:00
Aneesh Agrawal 39b64b52ed mkDerivation: add overrideAttrs function
This is similar to `overrideDerivation`, but overrides the arguments to
`mkDerivation` instead of the underlying `derivation` call.

Also update `makeOverridable` so that uses of `overrideAttrs` can be
followed by `override` and `overrideDerivation`, i.e. they can be
mix-and-matched.
2016-10-02 11:08:34 -04:00
Chris Martin 10f2befa58 stdenv.mkDerivation: add comments w/ manual links (#18707) 2016-09-18 11:20:53 +02:00
Tuomas Tynkkynen 5326e85f3d stdenv.mkDerivation: Use chooseDevOutputs 2016-08-29 14:49:51 +03:00
Tuomas Tynkkynen ff9491917f stdenv: Add platforms 2016-08-28 18:04:09 +03:00
Vladimír Čunát 1dc36904d8 Merge #14920: windows improvements, mainly mingw 2016-05-05 08:30:19 +02:00
Nikolay Amiantov d4794c3630 stdenv: clarify how outputsToInstall is chosen
See https://github.com/NixOS/nixpkgs/pull/14694/files#r60013871
2016-04-25 13:24:39 +03:00
Nikolay Amiantov 5ff40ddedf add get* helper functions and mass-replace manual outputs search with them 2016-04-25 13:24:39 +03:00
Vladimír Čunát 6e7787e666 stdenv for windows: auto-link dependency DLLs
For every *.{exe,dll} in $output/bin/ we try to find all (potential)
transitive dependencies and symlink those DLLs into $output/bin
so they are found on invocation.
(DLLs are first searched in the directory of the running exe file.)

The links are relative, so relocating whole /nix/store won't break them.
The hook is activated on cygwin and when cross-compiling to mingw.
2016-04-23 10:52:00 +02:00
Vladimír Čunát 2995439003 buildEnv: respect meta.outputsToInstall
As a result `systemPackages` now also respect it.
Only nix-env remains and that has a PR filed:
    https://github.com/NixOS/nix/pull/815
2016-04-07 15:59:44 +02:00
Vladimír Čunát 3342f717da stdenv: set meta.outputsToInstall unless overridden 2016-04-07 15:59:43 +02:00
Vladimír Čunát 716aac2519 Merge branch 'staging' into closure-size 2016-01-19 09:55:31 +01:00
Eelco Dolstra 81e530a749 Set SOURCE_DATE_EPOCH to latest source file
This provides a timestamp that's more useful than 1970-01-01 yet still
deterministic.
2016-01-05 17:21:48 +01:00
Luca Bruno 5b0352a6a4 Merge branch 'master' into closure-size 2015-12-11 18:31:00 +01:00
Domen Kožar 781b9eab61 allow{reason}: a bit more explicit error message 2015-12-09 17:01:46 +01:00
Vladimír Čunát 263fd55d4b Merge recent staging built on Hydra
http://hydra.nixos.org/eval/1231884
Only Darwin jobs seem to be queued now,
but we can't afford to wait for that single build slave.
2015-12-05 11:11:51 +01:00