Commit graph

2745 commits

Author SHA1 Message Date
aszlig 42a0b11450
dockerTools.pullImage: Fix build with sandboxing
Regression introduced in 736848723e.

This commit most certainly hasn't been tested with sandboxing enabled
and breaks not only pullImage but also the docker-tools NixOS VM test
because it doesn't find it's certificate path and also relies on
/var/tmp being there.

Fixing the certificate path is the easiest one because it can be done
via environment variable.

I've used overrideAttrs for changing the hardcoded path to /tmp (which
is available in sandboxed builds and even hardcoded in Nix), so that
whenever someone uses Skopeo from all-packages.nix the path is still
/var/tmp.

The reason why this is hardcoded to /var/tmp can be seen in a comment in
vendor/github.com/containers/image/storage/storage_image.go:

  Do not use the system default of os.TempDir(), usually /tmp, because
  with systemd it could be a tmpfs.

With sandboxed builds this isn't the case, however for using Nix without
NixOS this could turn into a problem if this indeed is the case.

So in the long term this needs to have a proper solution.

In addition to that, I cleaned up the expression a bit.

Tested by building dockerTools.examples.nixFromDockerHub and the
docker-tools NixOS VM test.

Signed-off-by: aszlig <aszlig@nix.build>
Cc: @nlewo, @Mic92, @Profpatsch, @globin, @LnL7
2018-05-06 04:57:24 +02:00
adisbladis f786072420
Merge pull request #39957 from enumatech/fix/nix-prefetch-git-spaces
nix-prefetch-git: fix handling of submodules with spaces
2018-05-04 23:36:58 +08:00
Lionello Lunesu 9fe26eed9e nix-prefetch-git: fix handling of submodules with spaces
The script would parse the output of `git submodule status` but
didn't handle paths with spaces in them. This would result in the
following error when trying to determine the URL of the submodule:

error: key does not contain a section: .url
2018-05-04 23:16:51 +08:00
Michael Bishop 51998c675a runInLinuxVM: fix ext4 and crc32c-intel interactions 2018-05-03 20:08:48 -03:00
zimbatm f7abcb0752
fetchs3: allow to name the derivation output (#39823)
* fetchs3: add configurable name

Change the default from "foo" to the basename of the s3 URL and make it
configurable.

* fetchs3: fix error on missing credentials.session_token

The session token should default to null instead of failing

* fetchs3: make use of the region argument

Set it to null if you don't want to use it

* fetchs3: prefer local build

Fetcher-types spend more time on network than CPU
2018-05-03 11:08:25 +01:00
Antoine Eiche 736848723e dockerTools.pullImage: Skopeo pulls images by digest
Skopeo is used to pull images from a Docker registry (instead of a
Docker deamon in a VM).

An image reference is specified with its name and its digest which is
an immutable image identifier (unlike image name and tag).

Skopeo can be used to get the digest of an image, for instance:
$ skopeo inspect docker://docker.io/nixos/nix:1.11 | jq -r '.Digest'
2018-05-02 21:32:20 +02:00
Linus Heckemann 75cfbdf33b buildFHSUserEnv: change to root directory after chroot
Fixes #38525
2018-04-28 14:51:07 +01:00
John Ericson ba52ae5048 treewide: isArm -> isAarch32
Following legacy packing conventions, `isArm` was defined just for
32-bit ARM instruction set. This is confusing to non packagers though,
because Aarch64 is an ARM instruction set.

The official ARM overview for ARMv8[1] is surprisingly not confusing,
given the overall state of affairs for ARM naming conventions, and
offers us a solution. It divides the nomenclature into three levels:

```
ISA:             ARMv8   {-A, -R, -M}
                 /    \
Mode:     Aarch32     Aarch64
             |         /   \
Encoding:   A64      A32   T32
```

At the top is the overall v8 instruction set archicture. Second are the
two modes, defined by bitwidth but differing in other semantics too, and
buttom are the encodings, (hopefully?) isomorphic if they encode the
same mode.

The 32 bit encodings are mostly backwards compatible with previous
non-Thumb and Thumb encodings, and if so we can pun the mode names to
instead mean "sets of compatable or isomorphic encodings", and then
voilà we have nice names for 32-bit and 64-bit arm instruction sets
which do not use the word ARM so as to not confused either laymen or
experienced ARM packages.

[1]: https://developer.arm.com/products/architecture/a-profile
2018-04-25 15:28:55 -04:00
Matthew Justin Bauer ab92a474a9
Merge pull request #38822 from matthewbauer/netbsd
Introducing NetBSD userland
2018-04-24 14:46:01 -05:00
Matthew Bauer 31ef995e37 bsd: init netbsd & openbsd userland
Adds a couple of useful NetBSD and OpenBSD derivations. Some of these
will be integrated into Nixpkgs later.

Noncomprehensive list:

- netbsd.getent
- netbsd.getconf
- netbsd.fts
- openbsd.mg
- netbsd.compat (can replace libbsd)
2018-04-24 14:16:35 -05:00
Matthew Bauer 0dc26d0e7e cvs: support ssh access
hacky wrapper handles ssh issues in nix builders
2018-04-24 14:16:29 -05:00
Henry Till afd3dbcce8 vmTools: update Debian 9 names and hashes 2018-04-22 21:32:17 -04:00
Matthew Justin Bauer bb4a8eb6d3
Merge pull request #39003 from P-E-Meunier/carnix-0.7
Carnix: 0.6 -> 0.7
2018-04-19 12:09:55 -05:00
pe@pijul.org ec40f193ac disable parallel rustc (-C codegen-units=1) 2018-04-16 16:16:28 +02:00
pe@pijul.org 8e87f73e36 Update to 0.7.2 2018-04-16 16:07:47 +02:00
pe@pijul.org 29a3059746 Carnix 0.7 2018-04-16 14:11:25 +02:00
Jean-Philippe Braun 9751771c73 dockerTools.buildImage: add /nix/store with correct permissions
Fixes #38835.
2018-04-16 10:19:01 +02:00
Linus Heckemann 4a30f2efec requireFile: exit with non-zero error message
Since the script running is a failure condition, we should fail the
build properly, not leaving it up to the missing output to determine
that the build went wrong.  This should partly address #38952 — nix
build will print out the build log on non-zero exits.
2018-04-15 13:58:05 +01:00
Shea Levy 603a369b89
Revert "buildSetupcfg: Include unzip for zip sources."
Misunderstood the error I was seeing.

This reverts commit 0d3eb70133.
2018-04-14 20:12:22 -04:00
Shea Levy 0d3eb70133
buildSetupcfg: Include unzip for zip sources. 2018-04-14 20:09:51 -04:00
John Ericson 800cb8ae71
Merge pull request #38881 from obsidiansystems/sierra-hack
cc-wrapper: More intelligent sierra hack
2018-04-13 14:39:56 -04:00
John Ericson 1a72330ab0 cc-wrapper: Utilize patched cctools ld for more robust macOS Sierra hack
Also fix numberous bugs, such as:

 - Not getting confused on more flags taking file arguments.

 - Ensuring children reexport their children, but the original
   binary/library doesn't.

 - Not spawning children when it turns out we just dynamically link
   under the threshold but our total number of inputs exceeeds it.

 - Children were always named `libunnamed-*`, when that name was
   supposed to be the last resort only.

ld-wrapper's own RPATH check hardcodes `.so`, but darwin uses `.dylib`
*and* (in practice due to lousy build systems) `.so`. We don't care
however because we never inject `--rpath` like that in practice on
Darwin. Hopefully someday we won't on linux either.
2018-04-13 13:17:03 -04:00
Shea Levy da8fc391a0
pythonPackages.buildSetupcfg: Allow disabling tests. 2018-04-13 12:25:10 -04:00
aszlig de581b99ca
kernel: Fix running kernels *with* modules
Pull request #38470 added support for running/building kernels without
modules. This got merged in 38e04bbf29 but
unfortunately while this works perfectly on kernels without modules it
also makes sure that *every* kernel gets no modules.

So all of our VM tests fail since that merge with something like this:

machine# loading module loop...
machine# modprobe: FATAL: Module loop not found in directory /lib/modules/4.14.33
machine# loading module vfat...
machine# modprobe: FATAL: Module vfat not found in directory /lib/modules/4.14.33
machine# loading module nls_cp437...
machine# modprobe: FATAL: Module nls_cp437 not found in directory /lib/modules/4.14.33
machine# loading module nls_iso8859-1...
machine# modprobe: FATAL: Module nls_iso8859-1 not found in directory /lib/modules/4.14.33
machine# loading module fuse...
machine# modprobe: FATAL: Module fuse not found in directory /lib/modules/4.14.33
machine# loading module dm_mod...
machine# modprobe: FATAL: Module dm_mod not found in directory /lib/modules/4.14.33

I shortly tested this against the "misc" VM test and the test is working
again.

In the long term (and I currently don't have time for this) it would be
better to also have a VM test which tests a kernel without modules.

Signed-off-by: aszlig <aszlig@nix.build>
Cc: @roberth, @7c6f434c
2018-04-12 15:43:53 +02:00
Michael Raskin 38e04bbf29
Merge pull request #38470 from roberth/linux-without-modules
linux module handling: support kernels without modules
2018-04-12 06:31:28 +00:00
Shea Levy 0901b3e195
Add setupcfg2nix and supporting infrastructure 2018-04-11 12:08:26 -04:00
Matthew Bauer 6c064e6b1f Revert "Merge pull request #28029 from cstrahan/hardening-fix"
This reverts commit 0dbc006760, reversing
changes made to cb7f774265.

Should go into staging.
2018-04-10 19:07:27 -05:00
John Ericson 0dbc006760
Merge pull request #28029 from cstrahan/hardening-fix
hardening: fix #18995
2018-04-10 19:48:02 -04:00
John Ericson ac4d74b6d9 hardening: Reindent 2018-04-10 16:33:47 -04:00
John Ericson 21818ae592 hardening: Tiny reindent 2018-04-10 16:33:47 -04:00
John Ericson 2364c22ec9 hardening: line order, spacing, and pointless quoting for consistency 2018-04-10 16:33:47 -04:00
John Ericson 4c76d87871 hardenning: Rejigger ifs and explicit declare and unset -v 2018-04-10 16:33:47 -04:00
Charles Strahan 386e77dae9
hardening: simplify reporting of disabled flags 2018-04-10 15:27:13 -04:00
Charles Strahan 273ce83f29
hardening: make requested fixes 2018-04-10 13:04:46 -04:00
Eelco Dolstra 8787c131ed
vmTools: Add crc32c_generic to the initrd
This is necessary due to a e2fsprogs update
(e6114781b0fad5345a2430fac3587d618273bda2) that causes mke2fs to
enable a feature (metadata_csum) that depends on crc32c.

https://hydra.nixos.org/build/72636785
2018-04-10 14:31:05 +02:00
Ryan Trinkle 1034aa8e9c
Merge pull request #25148 from obsidiansystems/docker-dirlinks
dockerTools: optionally preserve directory symlinks
2018-04-09 17:44:09 -04:00
Kevin Cox 4499513e54
rust: Allow setting cargoSha256 to null.
Setting the hash to null is a convenient way to bypass the hash check
while developing. It looks like the ability to do this was inadvertently
removed while adding vendor directory support.

This still checks that the user is explicitly setting the value but
allows null as a valid option.
2018-04-07 22:48:55 +01:00
Robert Hensing 30bff42231 linux module handling: support kernels without modules 2018-04-05 17:00:00 +02:00
Jörg Thalheim 8a8a73701d
Merge pull request #37928 from VShell/patch-2
buildRustCrate: fix equality testing
2018-03-28 09:34:41 +01:00
Jörg Thalheim d12cab3bb1 buildRustCrate: remove ancient test guards
Let's leave x"" to the 1990s, where they belong
2018-03-28 09:24:22 +01:00
Will Fancher d390ee74e3 Added bionic dynamic linker 2018-03-27 21:24:27 -04:00
Shell Turner 8cc6897ae9
buildRustCrate: fix equality testing
Use string equality instead of integer equality.
2018-03-27 20:08:48 +01:00
Sarah Brofeldt 4874ce1701 dockerTools.tarsum: Fix upstream import 2018-03-26 18:47:31 +02:00
Eelco Dolstra 7b539c0629
Fix typo 2018-03-22 13:57:41 +01:00
lewo ea6f55f83b
Merge pull request #36906 from nlewo/pr/docker-reproducible
Improve Docker image build reproducibility
2018-03-21 08:13:26 +01:00
Tuomas Tynkkynen ef64208eba Merge commit '3ab2949' from staging into master
Conflicts:
	pkgs/development/compilers/llvm/6/llvm.nix
	pkgs/servers/home-assistant/component-packages.nix
2018-03-15 22:30:56 +02:00
lewo 65e5bc713b
Merge pull request #36845 from jbedo/singularity
singularity: 2.4 -> 2.4.2
2018-03-14 10:19:10 +01:00
Tuomas Tynkkynen 2fec9c6e29 Merge remote-tracking branch 'upstream/master' into staging
Conflicts:
	pkgs/development/tools/build-managers/conan/default.nix
2018-03-13 23:04:18 +02:00
Franz Pletz 4f17851fb2
fetchurl: remove broken samba mirror 2018-03-13 17:15:33 +01:00
Antoine Eiche ac0c491836 dockerTools: add --sort=name options on all tar calls
This is to go to a reproducible image build.
Note without this options image are identical from the Docker point of
view but generated docker archives could have different hashes.
2018-03-13 13:46:47 +01:00