Commit graph

44 commits

Author SHA1 Message Date
Graham Christensen bc49a0815a
utillinux: rename to util-linux 2020-11-24 12:42:06 -05:00
Graham Christensen d78aa080f5
make-disk-image: support legacy+gpt 2020-10-30 15:50:24 -04:00
Emery Hemingway 3f922834b8 nixos/lib/make-disk-image.nix: abritary format input
Pass unrecognized format types as the output file extension to
qemu-img. The motivation is support for "vdi" output.
2020-07-31 15:33:08 +02:00
lassulus 883a6079fd make-disk-image: add hybrid and dynamic sized images 2020-07-27 19:45:55 +02:00
lassulus 8d05772134
nixos make-disk-image: fix permissions of /build
This was broken in 460c0d6 (PR #90431); now the nixos-unstable channel
should get unblocked.
vcunat modified this commit to use env-var instead of hardcoding /build
2020-07-10 10:31:26 +02:00
Emery Hemingway b8a9e3eea1 nixos/lib/make-disk-image.nix: VDI support 2020-06-13 11:08:48 +05:30
Eelco Dolstra 39c0e489d3
Revert "make-disk-image: replace nixos-install by nix-env calls"
This reverts commit e9bf955fd6. We use
nixos-install to ensure that make-disk-image produces the same result
as a regular installation (9802da517f)
and to reduce code duplication. If there is something broken in
nixos-install, it should be fixed there.
2019-10-07 17:21:28 +02:00
Christian Kögler e9bf955fd6 make-disk-image: replace nixos-install by nix-env calls
nixos-install can only be called from a machine
2019-09-29 19:57:19 +02:00
Travis Athougies a66ef3aa3d Add options to build disk image function (#50239)
* add options to build disk image function

* Revert suffix changes
2019-01-24 01:47:55 +02:00
Matthew Bauer f05d8f31ec make-disk-image: use filterSource instead of cleanSource
cleanSource does not appear to work correctly in this case. The path
does not get coerced to a string, resulting in a dangling symlink
produced in channel.nix.  Not sure why, but this
seems to fix it.

Fixes #51025.

/cc @elvishjericco
2019-01-07 16:28:50 -06:00
volth 2e979e8ceb [bot] nixos/*: remove unused arguments in lambdas 2018-07-20 20:56:59 +00:00
Eelco Dolstra cc2eeef4ab
Fix installing the Nixpkgs channel on the installation media
And test that it got installed correctly.
2018-02-27 19:58:23 +01:00
Eelco Dolstra d12c9911df
Merge remote-tracking branch 'origin/master' into nix-2.0 2018-02-22 17:28:51 +01:00
Jan Malakhovski 2e6b796761 nixos: rename config.system.nixos* -> config.system.nixos.* 2018-02-18 12:56:30 +00:00
Eelco Dolstra 9802da517f
make-disk-image.nix: Use nixos-install again
Since nixos-install doesn't require any special privileges anymore,
this Just Works. No more need for fakeroot / nixos-prepare-root.
2018-02-07 19:39:13 +01:00
Tuomas Tynkkynen 962e79ef32 nixos/make-disk-image.nix: Support EFI images
- Add a new parameter `imageType` that can specify either "efi" or
  "legacy" (the default which should see no change in behaviour by
  this patch).

- EFI images get a GPT partition table (instead of msdos) with a
  mandatory ESP partition (so we add an assert that `partitioned`
  is true).

- Use the partx tool from util-linux to determine exact start + size
  of the root partition. This is required because GPT stores a secondary
  partition table at the end of the disk, so we can't just have
  mkfs.ext4 create the filesystem until the end of the disk.

- (Unrelated to any EFI changes) Since we're depending on the
  `-E offset=X` option to mkfs which is only supported by e2fsprogs,
  disallow any attempts of creating partitioned disk images where
  the root filesystem is not ext4.
2018-01-22 11:18:23 +02:00
Tuomas Tynkkynen 758b4c1ea4 make-disk-image.nix: Really fix write permissions in the store
I think the current one applies the -exec only to those that match
'-type d'. Let's switch it to something that humans can understand...
2017-12-20 17:28:46 +02:00
Tuomas Tynkkynen c9f71974f8 make-disk-image.nix: Remove write permissions from /nix/store
Fakeroot seems to always give the owner write bit to any files touched
inside it (presumably to easily simulate the fact that root can still
modify such files). So do an explicit chmod to remove them.

This should finally solve #32242 after the EC2 images are regenerated
with this change.

https://hydra.nixos.org/build/66143116
2017-12-17 01:19:05 +02:00
Tuomas Tynkkynen 0d27df280f build-support/vm: Use devtmpfs, not static device nodes
In 2017, there is no reason to create a static /dev.
2017-12-12 14:31:50 +02:00
Orivej Desh 470a7752d8 make-disk-image: fix eval 2017-11-13 14:56:30 +00:00
Orivej Desh 27f5cc959b make-disk-image: embed compression switch into the format name 2017-11-12 18:39:36 +00:00
Orivej Desh 608de48fe4 make-disk-image: add compressed option to compress qcow2 images 2017-11-12 18:28:06 +00:00
Orivej Desh a2d3ee77b8 make-disk-image: clean up 2017-11-12 18:24:26 +00:00
Orivej Desh a84ce72fdc make-disk-image: do not use faketime
Since https://github.com/lkl/linux/pull/394 cptofs preserves the source time,
which is 1970-01-01T00:00:01Z for /nix/store and recent for other files.

This reverts commit f5b3f2c5a7.
2017-11-11 20:57:14 +00:00
Orivej Desh f218ef647c make-disk-image: clean up 2017-11-11 20:24:26 +00:00
Dan Peebles f5b3f2c5a7 make-disk-iamge: use faketime to get a consistent timestamp
This fixes #28768 because during an image build, Nix sees bad store
timestamps and attempts to fix them, but can't fix them on a running
system (due to being inside a builder). Since timestamps on the store
are supposed to be 1 anyway, if we fix this, that fixes image building
inside booted images made this way.

Note that this adds quite a bit of noise to the output, because running
`cptofs` under `faketime` causes a bunch of seemingly spurious error
messages and my attempts to suppress them all failed. We'll fix it when
`cptofs` gets a native timestamp preservation feature.
2017-11-10 05:15:37 +00:00
Bjørn Forsman 0ff4bb5f87 nixos: run parted with --script option
-s, --script: never prompts for user intervention

Sometimes the NixOS installer tests fail when they invoke parted, e.g.
https://hydra.nixos.org/build/62513826/nixlog/1. But instead of exiting
right there, the tests hang until the Nix builder times out (and kills
the build). With this change the tests would instead fail immediately,
which is preferred.

While at it, use "parted --script" treewide, so nobody gets build
timeout due to parted error (or misuse). (Only nixos/ use it, and only
non-interactive.)

A few instances already use the short option "-s", convert them to long
option "--short".
2017-10-14 15:29:02 +02:00
Dan Peebles e2589b3ca2 Deduplicate some filterSource invocations
This version should have more conventional regexes that work across many
platforms and regex engines. This is an issue because up until Nix 1.11,
Nix called out to the libc regex matcher, which behaved differently on
Darwin and Linux. And in Nix 1.12, we're moving to std::regex which will
also behave differently here.

And yes, I do actually evaluate make-disk-image.nix on Darwin ;)
2017-08-29 20:27:04 -04:00
Dan Peebles 5f372ef67f amazon-image: support VPC/VHD as an output format for the builder 2017-08-10 22:57:26 +00:00
Dan Peebles f1708a9d7d make-disk-image: change to be less VM-centric
This changes much of the make-disk-image.nix logic (and thus most NixOS
image building) to use LKL to set up the target directory structure rather
than a Linux VM. The only work we still do in a VM is less IO-heavy stuff
that while still time-consuming, is less of the overall load. The goal is
to kill more of that stuff, but that will require deeper changes to NixOS
activation scripts and switch-to-configuration.pl, and I don't want to
bite off too much at once.
2017-04-24 02:30:00 +00:00
Dan Peebles e798f573f0 make-disk-image.nix: set last fsck time on ext4 images to enable resize-on-startup 2017-02-26 02:02:22 +00:00
Dan Peebles 49641e0de0 make-disk-image.nix: support additional filesystem contents
This makes make-disk-image.nix slightly more consistent with other image
builders we have. Unfortunately I duplicated some code in doing so, but
this is temporary duplication on the path to consolidating everything.
See https://github.com/NixOS/nixpkgs/issues/23052 for more details on that.

I'm also exposing the option in the amazon-image.nix maintainer module.
2017-02-22 23:49:49 +00:00
Domen Kožar b6363c7bc8 make-disk-image: make store validity fix optional
This is useful for EC2 AMI generation to speedup the process.

In my case it removes 13min out of 45min when generating an image
on EC2.
2016-12-07 13:30:20 +01:00
Domen Kožar e5cca82d79 make-disk-image: run tune2fs after umount to skip fsck
tune2fs marks the filesystem as clean to prevent resize2fs from
complaining.

But we were invoking it before we mounted the filesystem, so the
counters would increase to 1 and it broke the functionality.

By moving the call after the mount, I have confirmed it works by:

   $ nix-build nixos/tests/ec2.nix

cc @rbvermaa @edolstra
2016-12-07 13:30:20 +01:00
obadz 24f8cf08cc nixos/lib/make-disk-image: refactor to use nixos-install
- Replace hand-rolled version of nixos-install in make-disk-image by an
  actual call to nixos-install
- Required a few cleanups of nixos-install
- nixos-install invokes an activation script which the hand-rolled version
  in make-disk-image did not do. We remove /etc/machine-id as that's
  a host-specific, impure, output of the activation script

Testing:

nix-build '<nixpkgs/nixos/release.nix>' -A tests.installer.simple passes

Also tried generating an image with:

nix-build -E 'let
    pkgs = import <nixpkgs> {};
    lib = pkgs.lib;
    nixos = import <nixpkgs/nixos> {
      configuration = {
        fileSystems."/".device = "/dev/disk/by-label/nixos";
        boot.loader.grub.devices = [ "/dev/sda" ];
        boot.loader.grub.extraEntries = '"''"'
          menuentry "Ubuntu" {
             insmod ext2
             search --set=root --label ubuntu
             configfile /boot/grub/grub.cfg
          }
        '"''"';
      };
    };
  in import <nixpkgs/nixos/lib/make-disk-image.nix> {
    inherit pkgs lib;
    config = nixos.config;
    diskSize = 2000;
    partitioned = false;
    installBootLoader = false;
  }'

Then installed the image:
$ sudo df if=./result/nixos.img of=/dev/sdaX bs=1M
$ sudo resize2fs /dev/disk/by-label/nixos
$ sudo mount /dev/disk/by-label/nixos /mnt
$ sudo mount --rbind /proc /mnt/proc
$ sudo mount --rbind /dev /mnt/dev
$ sudo chroot /mnt /nix/var/nix/profiles/system/bin/switch-to-configuration boot

[ … optionally do something about passwords … ]

and successfully rebooted to that image.

Was doing all this from inside a Ubuntu VM with a single user nix install.
2016-08-16 15:31:16 +01:00
obadz 760b2b9048 nixos/make-disk-image: add ability to defer bootloader install until image has been flashed 2016-08-15 20:01:55 +01:00
Tuomas Tynkkynen 60f5659dad treewide: Use correct output in ${config.nix.package}/bin 2016-04-25 16:44:37 +02:00
Eelco Dolstra 0d3738cdcc Fix the EC2 test
We now generate a qcow2 image to prevent hitting Hydra's output size
limit. Also updated /root/user-data -> /etc/ec2-metadata/user-data.

http://hydra.nixos.org/build/33843133
2016-03-30 21:50:23 +02:00
Eelco Dolstra 5cc7bcda30 Combine OVA generation steps
Previously this was done in three derivations (one to build the raw
disk image, one to convert to OVA, one to add a hydra-build-products
file). Now it's done in one step to reduce the amount of copying
to/from S3. In particular, not uploading the raw disk image prevents
us from hitting hydra-queue-runner's size limit of 2 GiB.
2016-03-15 14:15:12 +01:00
Domen Kožar f1508b3a23 nova-image: use make-disk-image.nix 2015-10-26 16:16:45 +01:00
Eelco Dolstra 106738b196 Give more memory for the disk image builder
http://hydra.nixos.org/build/26480662
2015-10-06 15:03:26 +02:00
Eelco Dolstra 64aed5e78f Fix Nix database in generated images
This prevents seeing lots of warnings about missing hashes/sizes in the
database when running "nix-store --verify --check-contents" for the
first time.
2015-09-28 14:47:48 +02:00
Eelco Dolstra b3347287be Use make-disk-image.nix for VirtualBox images 2015-09-28 14:31:19 +02:00
Eelco Dolstra aeb31b97ad Update AMI generator
The EBS and S3 (instance-store) AMIs are now created from the same
image. HVM instance-store AMIs are also generated.

Disk image generation has been factored out into a function
(nixos/lib/make-disk-image.nix) that can be used to build other kinds
of images.
2015-09-27 21:06:40 +02:00