Commit graph

151 commits

Author SHA1 Message Date
Luke Granger-Brown 48c23a086b dockerTools.pullImage: force disable skopeo's progress bar
skopeo will disable the progress bar if it detects that stdout isn't a
TTY - in order to make it think that stdout _isn't_ a TTY and therefore
avoid it printing a lot of "…" on separate lines, we pipe the output
through cat.

This changes the output from:

…
…
…
…
…
…

to the eminently more useful and less spammy:

Getting image source signatures
Copying blob sha256:[snip]
Copying blob sha256:[snip]
Copying blob sha256:[snip]
Copying config sha256:[snip]
Writing manifest to image destination
Storing signatures
2021-07-03 01:40:38 +00:00
Robert Hensing 1834bc8711
Merge pull request #125216 from hercules-ci/follow-up-115491
dockerTools: Fix passthru image tag
2021-06-02 16:58:49 +02:00
Robert Hensing 7f6a395070
Merge pull request #125211 from edwtjo/docker-fetch-wotls
build-support/docker: pass tlsVerify to support http registries
2021-06-01 15:07:48 +02:00
Robert Hensing ff55c41fac dockerTools: Fix passthru image tag
It should match the actual image tag.
This fixes the problem introduced in 00996b5e03
https://github.com/NixOS/nixpkgs/pull/115491#pullrequestreview-672789901
2021-06-01 14:42:21 +02:00
Edward Tjörnhammar ab4649e9d1
build-support/docker: pass tlsVerify to support http registries 2021-06-01 14:34:24 +02:00
Robert Hensing 5259d66b74 dockerTools: Allow omitting all store paths
Adds includeStorePaths, allowing the omission of the store paths.
You generally want to leave it on, but tooling may disable this
to insert the store paths more efficiently via other means, such
as bind mounting the host store.
2021-05-26 15:11:42 +02:00
Robert Hensing 69de7cc12a dockerTools: Format 2021-05-26 15:11:42 +02:00
Robert Hensing 578acc7a42
Merge pull request #118018 from considerate/master
dockerTools: Implement merging of image tarballs
2021-04-16 09:17:44 +02:00
Jörg Thalheim 65a40ca547
Merge pull request #102725 from thefloweringash/dockertools-proc
dockerTools: fix absent /proc during runAsRoot
2021-04-12 14:14:24 +01:00
Viktor Kronvall b2aa1f9d7a dockerTools: preserve order of images in manifest 2021-04-07 23:20:14 +09:00
Viktor Kronvall dcc9aef015 dockerTools: take a list of images in mergeImages 2021-04-07 22:49:44 +09:00
Viktor Kronvall ecc293ff7a dockerTools: Implement merging of image tarballs
The `docker load` command supports loading tarballs that contain
multiple docker images with their respective image names and tags. This
enables distributing these images as a single file which simplifies the
release of software when an application requires multiple services to
run.

However, pkgs.dockerTools only create tarballs with a single docker
image and there exists is no mechanism in nixpkgs to combine the created
tarballs. This commit implements merging of tarballs in a way that is
compatible with `docker load`.
2021-04-07 22:49:44 +09:00
Robert Hensing 58b21dea78
Merge pull request #116749 from vroad/docker-layered-image-fakeroot
dockerTools.streamLayeredImage: add fakeRootCommands option
2021-04-07 15:02:24 +02:00
Louis Blin aae8588182 dockerTools.buildLayeredImage: support fromImage
It is now possible to pass a `fromImage` to `buildLayeredImage` and
`streamLayeredImage`, similar to what `buildImage` currently supports.

This will prepend the layers of the given base image to the resulting
image, while ensuring that at most `maxLayers` are used. It will also
ensure that environment variables from the base image are propagated
to the final image.
2021-03-23 14:50:42 +00:00
vroad 057c7a2d58 dockerTools.streamLayeredImage: source $stdenv/setup before running fakeRootCommands 2021-03-19 11:01:10 +09:00
vroad 5199c7e6da
dockerTools.streamLayeredImage: simplify inherit statements
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2021-03-19 10:56:00 +09:00
vroad b4d92811be dockerTools.streamLayeredImage: add fakeRootCommands option 2021-03-19 10:25:26 +09:00
Benjamin Hipple 17a9f368e3
Merge pull request #115793 from lbpdt/feature/docker-tools-layered-image-name-slashes
dockerTools.buildLayeredImage: image names with registry/ prefix
2021-03-11 14:58:26 -05:00
Louis Blin 419a4fa596 dockerTools.buildLayeredImage: image names with registry/ prefix
When using `buildLayeredImage`, it is not possible to specify an image
name of the form `<registry>/my/image`, although it is a valid name.

This is due to derivations under `buildLayeredImage` using that image
name as their derivation name, but slashes are not permitted in that
context.

A while ago, #13099 fixed that exact same problem in `buildImage` by
using `baseNameOf name` in derivation names instead of `name`. This
change does the same thing for `buildLayeredImage`.
2021-03-10 17:44:24 +00:00
Louis Blin bf56388c92 dockerTools.buildLayeredImage: configurable store root
`stream_layered_image.py` currently assumes that the store root will be
at `/nix/store`, although the user might have configured this
differently. This makes `buildLayeredImage` unusable with stores having
a different root, as they will fail an assertion in the python script.

This change updates that assertion to use `builtins.storeDir` as the
source of truth about where the store lives, instead of assuming
`/nix/store`.
2021-03-10 16:44:53 +00:00
Chris Roberts 00996b5e03 dockerTools: Do not lowercase image tag in buildImage
Closes #115455
2021-03-09 09:06:04 +00:00
Robert Hensing f3d006c1d4
Merge pull request #115083 from osener/fix-dockertools-on-darwin
dockerTools: fix build on Darwin
2021-03-04 16:50:26 +01:00
Ozan Sener 78f322f2d0 dockerTools: fix build on Darwin
Fixes #110665

Introduced by #109420
2021-03-04 16:17:48 +01:00
Ben Siraphob 98f26993f2 treewide: remove stdenv where not needed 2021-03-04 19:54:50 +07:00
Johan Thomsen 793e77d4e2 dockertools: sort tar-members by name for reproducibility 2021-02-18 09:51:41 +01:00
Pavol Rusnak 90f7338112
treewide: stdenv.lib -> lib 2021-01-24 01:49:49 +01:00
Robert Hensing 9797139cbb dockerTools: typo
Italian.
2021-01-15 14:23:51 +01:00
Milan 0d724ba2fc
dockerTools: fix build (#109420) 2021-01-15 03:56:38 +01:00
Robert Hensing a31607f904 dockerTools: Don't apologize
Warning about future breaking changes is wrong.

 - It suggests that the maintainers don't value backwards compatibility.
   They do.
 - It implies that other parts of Nixpkgs won't ever break. They will.
 - It implies that a well-defined "public" interface exists. It doesn't.
 - If the reasons above didn't apply, it should have been in the manual
   instead.

Breaking changes will come, especially to the interface. That can be the
only way we can make progress without breaking the image _contents_.

I don't think dockerTools is any different from most of Nixpkgs in
these regards.
2021-01-06 13:02:19 +01:00
Robert Hensing 5cacf0fcec dockerTools: use go.GOARCH as default arch 2020-12-15 02:15:35 -08:00
Terin Stock 8f66dc94a7 dockerTools: normalize arch to GOARCH
Docker (via containerd) and the the OCI Image Configuration imply and
suggest, respectfully, that the architecture set in images matches those
of GOARCH in the Go Language document.

This changeset updates the implimentation of getArch in dockerTools to
return GOARCH values, to satisfy Docker.

Fixes: #106695
2020-12-15 02:14:01 -08:00
Florian Klink e054694925 dockerTools.binSh: init 2020-12-02 14:57:23 +01:00
Florian Klink f7ee2706c2 dockerTools.fakeNss: init
This provides a /etc/passwd and /etc/group that contain root and nobody.

Useful when packaging binaries that insist on using nss to look up
username/groups (like nginx).

The current nginx example used the `runAsRoot` parameter to setup
/etc/group and /etc/passwd (which also doesn't exist in
buildLayeredImage), so we can now just use fakeNss there and use
buildLayeredImage.
2020-12-02 14:56:07 +01:00
Graham Christensen bc49a0815a
utillinux: rename to util-linux 2020-11-24 12:42:06 -05:00
adisbladis 6847a5c12d
referencesByPopularity: Don't use buildPackages at call-site
It's cleaner to reference buildPackages directly in
build-support/docker directly.
2020-11-24 02:55:40 +01:00
adisbladis 8a3b33baed
dockerTools: Set correct architecture when cross compiling 2020-11-19 18:13:21 +01:00
adisbladis 4313ac6b29
dockerTools.buildLayeredImage: Fix cross compilation 2020-11-19 14:32:22 +01:00
Andrew Childs 5b82b8d094 dockerTools: fix absent /proc during runAsRoot
The chroot environment under mnt had /dev and /sys via bind mounts,
but nothing setting up /proc. The `--mount-proc` argument to unshare
defaults to /proc, which is outside of the chroot envirnoment.
2020-11-04 16:17:57 +09:00
Michael 24c5059468 docker: Fix broken pipe on docker layer creation
Add `-p` to the `tee` command to avoid exiting on breaking pipe due to
tarsum finishing before tar which creating docker layers.
2020-11-03 12:33:11 +01:00
Robert Hensing 11b0106cf9 dockerTools: add isExe to streamLayeredImage result 2020-10-05 10:48:17 +02:00
Utku Demir ae82f81bfa
dockerTools.streamLayeredImage: Store the customisation layer as a tarball
This fixes as issue described here[1], where permissions set by 'extraCommands'
were ignored by Nix.

[1] https://github.com/NixOS/nixpkgs/pull/91084#issuecomment-669834938
2020-09-04 16:53:23 +12:00
Robert Hensing 8c0459f611 dockerTools: Always set imageTag attribute
The image tag can be specified or generated from the output hash.
Previously, a generated tag could be recovered from the evaluated
image with some string operations.

However, with the introduction of streamLayeredImage, it's not
feasible to compute the generated tag yourself.

With this change, the imageTag attribute is set unconditionally,
for the buildImage, buildLayeredImage, streamLayeredImage functions.
2020-07-11 16:58:25 +02:00
Utku Demir 4bff5b7f3d
dockerTools: Properly quote a shell variable 2020-06-21 12:35:39 +12:00
Utku Demir 307804d97d
dockerTools.streamLayeredImage: comments 2020-06-21 12:35:38 +12:00
Utku Demir 4e0109f873
Set mtime to get more deterministic builds 2020-06-21 12:35:37 +12:00
Utku Demir 560201da66
Implement dockerTools.streamLayeredImage 2020-06-21 12:35:32 +12:00
lewo a498da343a
Merge pull request #87154 from utdemir/buildimage-optimizations
Some performance optimizations to dockerTools.build{,Layered}Image
2020-05-19 15:39:25 +02:00
zowoq 42232493a3 dockerTools: pass insecure-policy and tmpdir to skopeo 2020-05-16 08:46:13 +10:00
Utku Demir f5a90a7aab
dockerTools.buildImage: Preserve environment variables from the parent image 2020-05-08 21:49:16 +12:00
Utku Demir f12346d493
dockerTools: Calculate tarsum's on the fly
Calculating the tarsum after creating a layer is inefficient, since
we have to read the tarball we've just written from the disk.

This commit simultaneously calculates the tarsum while creating the
tarball.
2020-05-07 11:50:07 +12:00