Commit graph

83 commits

Author SHA1 Message Date
lewo 3fb4eb1c43 nixos/dockerPreloader: preload docker images (#49379)
This module permits to preload Docker image in a VM in order to reduce
OIs on file copies. This module has to be only used in testing
environments, when the test requires several Docker images such as in
Kubernetes tests. In this case,
`virtualisation.dockerPreloader.images` can replace the
`services.kubernetes.kubelet.seedDockerImages` options.

The idea is to populate the /var/lib/docker directory by mounting qcow
files (we uses qcow file to avoid permission issues) that contain images.

For each image specified in
config.virtualisation.dockerPreloader.images:
1. The image is loaded by Docker in a VM
2. The resulting /var/lib/docker is written to a QCOW file

This set of QCOW files can then be used to populate the
/var/lib/docker:
1. Each QCOW is mounted in the VM
2. Symlink are created from these mount points to /var/lib/docker
3. A /var/lib/docker/image/overlay2/repositories.json file is generated
4. The docker daemon is started.
2018-11-03 01:00:53 +01:00
Sarah Brofeldt ded8f28c3a Revert "virtualization/qemu-vm: fix and improve virtio/scsi switching"
This reverts commit f777d2b719.
cc #34409
This breaks evaluation of the tested job:
attribute 'diskInterface' missing, at /nix/store/5k9kk52bv6zsvsyyvpxhm8xmwyn2yjvx-source/pkgs/build-support/vm/default.nix:316:24
2018-09-25 11:10:10 +02:00
Stefan Junker f777d2b719 virtualization/qemu-vm: fix and improve virtio/scsi switching 2018-09-22 23:29:19 +02:00
Sarah Brofeldt 78fb4bd644 nixos/qemu-vm: Don't explicitly mknod /dev/vda*
And don't need to source the uevent files anymore either since $MAJOR
or $MINOR aren't used elsewhere.

[dezgeg: The reason these are no longer needed is that 0d27df280f
switched /tmp to a devtmpfs which automatically creates such device
nodes]
2018-08-16 14:19:41 +03:00
aszlig fb9f5e4a03
nixos/testing: Fix output of systemd messages
Regression introduced by d4468bedb5.

No systemd messages are shown anymore during VM test runs, which is not
very helpful if you want to find out about failures.

There is a bit of a conflict between testing and the change that
introduced the regression. While the mentioned commit makes sure that
the primary console is tty0 for virtualisation.graphics = false, our VM
tests need to have the serial console as primary console.

So in order to support both, I added a new virtualisation.qemu.consoles
option, which allows to specify those options using the module system.

The default of this option is to use the changes that were introduced
and in test-instrumentation.nix we use only the serial console the same
way as before.

For test-instrumentation.nix I didn't add a baudrate to the serial
console because I can't find a reason on top of my head why it should
need it. There also wasn't a reason stated when that was introduced in
7499e4a5b9.

Signed-off-by: aszlig <aszlig@nix.build>
Cc: @flokli, @dezgeg, @edolstra
2018-05-10 08:21:20 +02:00
xeji 3d6f752c51
Merge pull request #39099 from teto/qemu_agent
qemu-guest-agent: init module
2018-05-10 01:22:52 +02:00
Florian Klink 7499e4a5b9 modules/virtualisation/qemu-vm: set console baudrate to 115200 2018-05-07 19:23:47 +03:00
Florian Klink d4468bedb5 modules/virtualisation/qemu-vm: always enable serial console
Always enable both tty and serial console, but set preferred console
depending on cfg.graphical.
Even in qemu graphical mode, you can switch to the serial console via
Ctrl+Alt+3.

With that being done, you also don't need to specify
`systemd.services."serial-getty@ttyS0".enable = true;` either as described in
https://nixos.wiki/wiki/Cheatsheet#Building_a_service_as_a_VM_.28for_testing.29,
as systemd automatically spawns a getty on consoles passwd via cmdline.

This also means, vms built by 'nixos-rebuild build-vm' can simply be run
properly in nographic mode by appending `-nographic` to `result/bin/run-*-vm`,
without the need to explicitly add platform-specific QEMU_KERNEL_PARAMS.
2018-05-07 19:23:47 +03:00
Florian Klink 36f4b7f6ba virtualisation/qemu-vm.nix: remove ttys
unused.
2018-05-07 19:23:47 +03:00
Florian Klink 046e374c99 virtualisation/qemu-vm.nix: use lib.optionalString 2018-05-07 19:23:47 +03:00
aszlig f148c5c4a1
nixos/tests: Fix QEMU flags for SCSI disks
The ability to specify "-drive if=scsi" has been removed in QEMU version
2.12 (introduced in 3e3b39f173).

Quote from https://wiki.qemu.org/ChangeLog/2.12#Incompatible_changes:

> The deprecated way of configuring SCSI devices with "-drive if=scsi"
> on x86 has been removed. Use an appropriate SCSI controller together
> "-device scsi-hd" or "-device scsi-cd" and a corresponding "-blockdev"
> parameter instead.

So whenever the diskInterface is "scsi" we use the new way to specify
the drive and fall back to the deprecated way for the time being. The
reason why I'm not using the new way for "virtio" and "ide" as well is
because there is no simple generic way anymore to specify these.

This also turns the type of the virtualisation.qemu.diskInterface option
to be an enum, so the user knows which values are allowed but we can
also make sure the right value is provided to prevent typos.

I've tested this against a few non-disk-related NixOS VM tests but also
the installer.grub1 test (because it uses "ide" as its drive interface),
the installer.simple test (just to be sure it still works with
"virtio") and all the tests in nixos/tests/boot.nix.

In order to be able to run the grub1 test I had to go back to
8b1cf100cd (which is a known commit where
that test still works) and apply the QEMU update and this very commit,
because right now the test is broken.

Apart from the tests here in nixpkgs, I also ran another[1] test in
another repository which uses the "scsi" disk interface as well (in
comparison to most of the installer tests, this one actually failed
prior to this commit).

All of them now succeed.

[1]: 9b5a119972/tests/system/kernel/bfq.nix

Signed-off-by: aszlig <aszlig@nix.build>
Cc: @edostra, @grahamc, @dezgeg, @abbradar, @ts468
2018-04-30 03:02:59 +02:00
Tuomas Tynkkynen dda74d9e50 nixos/qemu-vm: Always add a virtio RNG device to the quest
mke2fs has this annoying property that it uses getrandom() to get random
numbers (for whatever purposes) which blocks until the kernel's secure
RNG has sufficient entropy, which it usually doesn't in the early boot
(except if your CPU supports RDRAND) where we may need to create the
root disk.

So let's give the VM a virtio RNG to avoid the boot getting stuck at
mke2fs.
2018-04-28 12:48:06 +03:00
Matthieu Coudron ca0604190e qemu-guest-agent: init module
Allow out of band communication between qemu VMs and the host.
Useful to retrieve IPs of VMs from the host (for instance when libvirt can't analyze
DHCP requests because VMs are configured with static addresses or when
there is connectivity default).
2018-04-27 18:32:15 +09:00
John Ericson ba52ae5048 treewide: isArm -> isAarch32
Following legacy packing conventions, `isArm` was defined just for
32-bit ARM instruction set. This is confusing to non packagers though,
because Aarch64 is an ARM instruction set.

The official ARM overview for ARMv8[1] is surprisingly not confusing,
given the overall state of affairs for ARM naming conventions, and
offers us a solution. It divides the nomenclature into three levels:

```
ISA:             ARMv8   {-A, -R, -M}
                 /    \
Mode:     Aarch32     Aarch64
             |         /   \
Encoding:   A64      A32   T32
```

At the top is the overall v8 instruction set archicture. Second are the
two modes, defined by bitwidth but differing in other semantics too, and
buttom are the encodings, (hopefully?) isomorphic if they encode the
same mode.

The 32 bit encodings are mostly backwards compatible with previous
non-Thumb and Thumb encodings, and if so we can pun the mode names to
instead mean "sets of compatable or isomorphic encodings", and then
voilà we have nice names for 32-bit and 64-bit arm instruction sets
which do not use the word ARM so as to not confused either laymen or
experienced ARM packages.

[1]: https://developer.arm.com/products/architecture/a-profile
2018-04-25 15:28:55 -04:00
Nikolay Amiantov 220d0decaf qemu-vm service: quote arguments in qemu runner 2018-03-26 14:01:49 +03:00
Matthieu Coudron 33b0ad83e9 qemu-vm: passing QEMU_NET_OPTS would fail
because of a change in #36850.
spotted by @jtojnar b7a2333ebe (commitcomment-28134992)
2018-03-17 23:21:27 +09:00
Tuomas Tynkkynen ef64208eba Merge commit '3ab2949' from staging into master
Conflicts:
	pkgs/development/compilers/llvm/6/llvm.nix
	pkgs/servers/home-assistant/component-packages.nix
2018-03-15 22:30:56 +02:00
Tuomas Tynkkynen 9e78baf5c7 nixos/qemu-vm: Add virtualized display + HID devices on AArch64 2018-03-14 21:18:20 +02:00
Tuomas Tynkkynen 2fec9c6e29 Merge remote-tracking branch 'upstream/master' into staging
Conflicts:
	pkgs/development/tools/build-managers/conan/default.nix
2018-03-13 23:04:18 +02:00
Matthieu Coudron b7a2333ebe qemu-vm: removes warning when running build-vm
get rid of deprecated qemu commands
2018-03-12 17:29:55 +09:00
Shea Levy fec543436d
nixos: Move uses of stdenv.shell to runtimeShell. 2018-03-01 14:38:53 -05:00
Tuomas Tynkkynen 71631a922b runInLinuxVM: Use QEMU command line that works on other architectures
... by moving the existing definition to qemu-flags.nix and reusing
that.
2018-01-30 16:57:27 +02:00
Tuomas Tynkkynen 8e83158f12 nixos/qemu: Deduplicate QEMU serialDevice into qemu-flags.nix 2018-01-30 16:57:27 +02:00
Tuomas Tynkkynen b2e315f97f nixos/qemu-vm: Pass gic-version=host for AArch64
This is required on the ThunderX CPUs on the Packet.net Type-2A
machines that have a GICv3. For some reason the default is to create a
GICv2 independent of the host hardware...
2017-12-06 17:29:09 +02:00
Tuomas Tynkkynen e56da6a464 nixos/qemu-vm: Use different serial port on ARM/AArch64 2017-11-26 11:13:21 +02:00
Tuomas Tynkkynen 0a2eda7a7e nixos/qemu-vm: Don't try enabling graphics/mouse on non-x86
Needs to be figured out some day, just disable them for now.
2017-11-26 11:13:21 +02:00
Tuomas Tynkkynen 43cb964e17 nixos/qemu-vm: Set QEMU command line depending on architecture 2017-11-26 11:13:21 +02:00
Eelco Dolstra 8f349a3bf3
Add function closureInfo to replace pathsFromGraph
Unlike pathsFromGraph, on Nix 1.12, this function produces a
registration file containing correct NAR hash/size information.

https://hydra.nixos.org/build/62832723
2017-10-25 15:38:14 +02:00
Joachim Fasting 252dcd62f3
OVMF: separate output for ovmf binaries
OVMF{,CODE,VARS}.fd are now available in a dedicated fd output, greatly
reducing the closure in the common case where only those files are used (a
few MBs versus several hundred MBs for the full OVMF).

Note: it's unclear why `dontPatchELF` is now necessary for the build to
pass (on my end, at any rate) but it doesn't make much sense to run this
fixup anyway,

Note: my reading of xen's INSTALL suggests that --with-system-ovmf should
point directly to the OVMF binary.  As such, the previous invocation was
incorrect (it pointed to the root of the OVMF tree).  In any case, I have
only built xen with `--with-system-ovmf`, I have not tested it.

Fixes https://github.com/NixOS/nixpkgs/issues/25854
Closes https://github.com/NixOS/nixpkgs/pull/25855
2017-05-20 12:33:48 +02:00
Graham Christensen 4585fdb9d4
qemu module: add virtualisation.cores option
QEMU can allow guests to access more than one host core at a time.
Previously, this had to be done via ad-hoc arguments:

    virtualisation.qemu.options = ["-smp 12"];

Now you can simply specify:

    virtualisation.cores = 12;
2017-04-24 15:23:46 -04:00
Thomas Strobel b9a7aacef7 improve: modules/virtualisation/qemu-vm.nix
disk image for qemu VM with bootloader:
* remove redundant command
* improve readability
* improve execution speed
* make output more reproducible
2017-03-04 11:31:47 +01:00
Thomas Strobel 0a8d9779c5 fix: "nixos-rebuild build-vm-with-bootloader" 2017-03-03 19:14:20 +01:00
Eelco Dolstra a4ec1841da
VM tests: veryloose -> cache=loose 2017-02-13 12:18:10 +01:00
Nikolay Amiantov 9eb540b807 qemu-vm module: fix boot.tmpOnTmpfs
This option caused systemd to mount /tmp on top of /tmp/{xchg,shared}.

Fixes #21490.
2017-02-03 15:02:34 +03:00
Eelco Dolstra bbd03e236a
Use looser 9pfs caching in VM tests/builds
This can give significant speed ups, see
7e20254412.
2016-12-29 21:26:16 +01:00
Eelco Dolstra ea46420fc0
Use overlayfs instead of unionfs-fuse in the VM tests
Overlayfs is quite a bit faster, e.g. with it the KDE 5 test takes ~7m
instead of ~30m on my laptop (which is still not great, since plain
9pfs is ~4m30s).
2016-12-21 20:49:08 +01:00
Eelco Dolstra a02bb00156
Enable virtualisation.writableStore by default
This works around:

  machine: must succeed: nix-store -qR /run/current-system | grep nixos-
  machine# error: changing ownership of path ‘/nix/store’: Invalid argument

Probably Nix shouldn't be anal about the ownership of the store unless
it's trying to build/write to the store.

http://hydra.nixos.org/build/45093872/nixlog/17/raw
(cherry picked from commit 57a0f140643cde409022e297ed05e05f8d34d778)
2016-12-20 10:52:47 +01:00
Eelco Dolstra f173da375d
Use only one build of qemu in VM tests
Previously we were using two or three (qemu_kvm, qemu_test, and
qemu_test with a different dbus when minimal.nix is included).

(cherry picked from commit 8bfa4ce82ea7d23a1d4c6073bcc044e6bf9c4dbe)
2016-12-20 10:52:46 +01:00
Eelco Dolstra aad5d1f9a7
virtualisation.qemu.program: Remove
This option is defined in qemu-vm.nix, but that module is not always
imported.

http://hydra.nixos.org/build/44817443
(cherry picked from commit 03c55005dfd6fbcd5cf8e00128a3bb6336b3bc0f)
2016-12-20 10:52:46 +01:00
Jörg Thalheim 1590461887 ntp: make timesyncd the new default
- most nixos user only require time synchronisation,
  while ntpd implements a battery-included ntp server (1,215 LOCs of C-Code vs 64,302)
- timesyncd support ntp server per interface (if configured through dhcp for instance)
- timesyncd is already included in the systemd package, switching to it would
  save a little disk space (1,5M)
2016-12-17 00:00:45 +01:00
aszlig 6cfb3b6364
nixos/tests: Use a patched QEMU for testing
The reason to patch QEMU is that with latest Nix, tests like "printing"
or "misc" fail because they expect the store paths to be owned by uid 0
and gid 0.

Starting with NixOS/nix@5e51ffb1c2, Nix
builds inside of a new user namespace. Unfortunately this also means
that bind-mounted store paths that are part of the derivation's inputs
are no longer owned by uid 0 and gid 0 but by uid 65534 and gid 65534.

This in turn causes things like sudo or cups to fail with errors about
insecure file permissions.

So in order to avoid that, let's make sure the VM always gets files
owned by uid 0 and gid 0 and does a no-op when doing a chmod on a store
path.

In addition, this adds a virtualisation.qemu.program option so that we
can make sure that we only use the patched version if we're *really*
running NixOS VM tests (that is, whenever we have imported
test-instrumentation.nix).

Tested against the "misc" and "printing" tests.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-11-17 17:16:16 +01:00
Nikolay Amiantov ff22705793 treewide: replace several /sbin paths by /bin 2016-08-19 17:56:45 +03:00
Shea Levy 9adad8612b Revert "Merge branch 'modprobe-fix' of git://github.com/abbradar/nixpkgs"
Was meant to go into staging, sorry

This reverts commit 57b2d1e9b0, reversing
changes made to 760b2b9048.
2016-08-15 19:05:52 -04:00
Nikolay Amiantov 1afd250676 treewide: replace several /sbin paths by /bin 2016-08-16 00:19:25 +03:00
Nikolay Amiantov 399db54e35 nixos/qemu: don't recreate extra disks 2016-07-19 05:20:02 +03:00
Eelco Dolstra 845c9b50bf boot.initrd.luks.devices: Change into an attribute set
This allows setting options for the same LUKS device in different
modules. For example, the auto-generated hardware-configuration.nix
can contain

  boot.initrd.luks.devices.crypted.device = "/dev/disk/...";

while configuration.nix can add

  boot.initrd.luks.devices.crypted.allowDiscards = true;

Also updated the examples/docs to use /disk/disk/by-uuid instead of
/dev/sda, since we shouldn't promote the use of the latter.
2016-05-25 18:04:21 +02:00
Tuomas Tynkkynen 60f5659dad treewide: Use correct output in ${config.nix.package}/bin 2016-04-25 16:44:37 +02:00
Tuomas Tynkkynen d0c127487f qemu-img: Fix module paths
They are compressed nowadays.

Not sure if these are really needed since nobody noticed they were
broken, but anyway...
2016-04-22 10:42:31 +03:00
Tuomas Tynkkynen 01854a850a treewide: Replace module_init_tools -> kmod
The former is deprecated and doesn't handle compressed kernel modules,
so all current usages of it are broken.
2016-04-22 10:40:57 +03:00
Eelco Dolstra 4d760edb94 Add FIXME 2016-02-09 16:15:57 +01:00