nixpkgs/nixos/lib/test-driver/Machine.pm

727 lines
20 KiB
Perl
Raw Normal View History

* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
package Machine;
use strict;
use threads;
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
use Socket;
use IO::Handle;
use POSIX qw(dup2);
use FileHandle;
use Cwd;
use File::Basename;
use File::Path qw(make_path);
use File::Slurp;
use Time::HiRes qw(clock_gettime CLOCK_MONOTONIC);
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
my $showGraphics = defined $ENV{'DISPLAY'};
my $sharedDir;
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
sub new {
my ($class, $args) = @_;
my $startCommand = $args->{startCommand};
2015-06-10 10:02:48 +00:00
my $name = $args->{name};
if (!$name) {
$startCommand =~ /run-(.*)-vm$/ if defined $startCommand;
$name = $1 || "machine";
}
if (!$startCommand) {
# !!! merge with qemu-vm.nix.
$startCommand =
2013-07-31 12:53:27 +00:00
"qemu-kvm -m 384 " .
"-net nic,model=virtio \$QEMU_OPTS ";
nixos/tests: Fix QEMU flags for SCSI disks The ability to specify "-drive if=scsi" has been removed in QEMU version 2.12 (introduced in 3e3b39f173f9abc99da84084a1f4657c9de885bd). Quote from https://wiki.qemu.org/ChangeLog/2.12#Incompatible_changes: > The deprecated way of configuring SCSI devices with "-drive if=scsi" > on x86 has been removed. Use an appropriate SCSI controller together > "-device scsi-hd" or "-device scsi-cd" and a corresponding "-blockdev" > parameter instead. So whenever the diskInterface is "scsi" we use the new way to specify the drive and fall back to the deprecated way for the time being. The reason why I'm not using the new way for "virtio" and "ide" as well is because there is no simple generic way anymore to specify these. This also turns the type of the virtualisation.qemu.diskInterface option to be an enum, so the user knows which values are allowed but we can also make sure the right value is provided to prevent typos. I've tested this against a few non-disk-related NixOS VM tests but also the installer.grub1 test (because it uses "ide" as its drive interface), the installer.simple test (just to be sure it still works with "virtio") and all the tests in nixos/tests/boot.nix. In order to be able to run the grub1 test I had to go back to 8b1cf100cd8badad6e1b6d4650b904b88aa870db (which is a known commit where that test still works) and apply the QEMU update and this very commit, because right now the test is broken. Apart from the tests here in nixpkgs, I also ran another[1] test in another repository which uses the "scsi" disk interface as well (in comparison to most of the installer tests, this one actually failed prior to this commit). All of them now succeed. [1]: https://github.com/openlab-aux/vuizvui/blob/9b5a119972e9c2d327500638d89063f4fce243ec/tests/system/kernel/bfq.nix Signed-off-by: aszlig <aszlig@nix.build> Cc: @edostra, @grahamc, @dezgeg, @abbradar, @ts468
2018-04-29 23:43:17 +00:00
if (defined $args->{hda}) {
if ($args->{hdaInterface} eq "scsi") {
$startCommand .= "-drive id=hda,file="
. Cwd::abs_path($args->{hda})
. ",werror=report,if=none "
. "-device scsi-hd,drive=hda ";
} else {
$startCommand .= "-drive file=" . Cwd::abs_path($args->{hda})
. ",if=" . $args->{hdaInterface}
. ",werror=report ";
}
}
$startCommand .= "-cdrom $args->{cdrom} "
if defined $args->{cdrom};
$startCommand .= "-device piix3-usb-uhci -drive id=usbdisk,file=$args->{usb},if=none,readonly -device usb-storage,drive=usbdisk "
if defined $args->{usb};
$startCommand .= "-bios $args->{bios} "
if defined $args->{bios};
$startCommand .= $args->{qemuFlags} || "";
}
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
my $tmpDir = $ENV{'TMPDIR'} || "/tmp";
unless (defined $sharedDir) {
$sharedDir = $tmpDir . "/xchg-shared";
make_path($sharedDir, { mode => 0700, owner => $< });
}
my $allowReboot = 0;
$allowReboot = $args->{allowReboot} if defined $args->{allowReboot};
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
my $self = {
startCommand => $startCommand,
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
name => $name,
allowReboot => $allowReboot,
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
booted => 0,
pid => 0,
connected => 0,
socket => undef,
stateDir => "$tmpDir/vm-state-$name",
monitor => undef,
log => $args->{log},
redirectSerial => $args->{redirectSerial} // 1,
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
};
mkdir $self->{stateDir}, 0700;
bless $self, $class;
return $self;
}
sub log {
my ($self, $msg) = @_;
$self->{log}->log($msg, { machine => $self->{name} });
}
sub nest {
my ($self, $msg, $coderef, $attrs) = @_;
$self->{log}->nest($msg, $coderef, { %{$attrs || {}}, machine => $self->{name} });
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
}
sub name {
my ($self) = @_;
return $self->{name};
}
sub stateDir {
my ($self) = @_;
return $self->{stateDir};
}
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
sub start {
my ($self) = @_;
return if $self->{booted};
$self->log("starting vm");
# Create a socket pair for the serial line input/output of the VM.
my ($serialP, $serialC);
socketpair($serialP, $serialC, PF_UNIX, SOCK_STREAM, 0) or die;
# Create a Unix domain socket to which QEMU's monitor will connect.
my $monitorPath = $self->{stateDir} . "/monitor";
unlink $monitorPath;
my $monitorS;
socket($monitorS, PF_UNIX, SOCK_STREAM, 0) or die;
bind($monitorS, sockaddr_un($monitorPath)) or die "cannot bind monitor socket: $!";
listen($monitorS, 1) or die;
# Create a Unix domain socket to which the root shell in the guest will connect.
my $shellPath = $self->{stateDir} . "/shell";
unlink $shellPath;
my $shellS;
socket($shellS, PF_UNIX, SOCK_STREAM, 0) or die;
bind($shellS, sockaddr_un($shellPath)) or die "cannot bind shell socket: $!";
listen($shellS, 1) or die;
# Start the VM.
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
my $pid = fork();
die if $pid == -1;
if ($pid == 0) {
close $serialP;
close $monitorS;
close $shellS;
if ($self->{redirectSerial}) {
open NUL, "</dev/null" or die;
dup2(fileno(NUL), fileno(STDIN));
dup2(fileno($serialC), fileno(STDOUT));
dup2(fileno($serialC), fileno(STDERR));
}
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
$ENV{TMPDIR} = $self->{stateDir};
$ENV{SHARED_DIR} = $sharedDir;
$ENV{USE_TMPDIR} = 1;
$ENV{QEMU_OPTS} =
($self->{allowReboot} ? "" : "-no-reboot ") .
"-monitor unix:./monitor -chardev socket,id=shell,path=./shell " .
"-device virtio-serial -device virtconsole,chardev=shell " .
"-device virtio-rng-pci " .
($showGraphics ? "-serial stdio" : "-nographic") . " " . ($ENV{QEMU_OPTS} || "");
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
chdir $self->{stateDir} or die;
exec $self->{startCommand};
die "running VM script: $!";
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
}
# Process serial line output.
close $serialC;
threads->create(\&processSerialOutput, $self, $serialP)->detach;
sub processSerialOutput {
my ($self, $serialP) = @_;
while (<$serialP>) {
chomp;
s/\r$//;
print STDERR $self->{name}, "# $_\n";
$self->{log}->{logQueue}->enqueue({msg => $_, machine => $self->{name}}); # !!!
}
}
eval {
local $SIG{CHLD} = sub { die "QEMU died prematurely\n"; };
2015-06-10 10:02:48 +00:00
# Wait until QEMU connects to the monitor.
accept($self->{monitor}, $monitorS) or die;
# Wait until QEMU connects to the root shell socket. QEMU
# does so immediately; this doesn't mean that the root shell
# has connected yet inside the guest.
accept($self->{socket}, $shellS) or die;
$self->{socket}->autoflush(1);
};
die "$@" if $@;
2015-06-10 10:02:48 +00:00
$self->waitForMonitorPrompt;
$self->log("QEMU running (pid $pid)");
2015-06-10 10:02:48 +00:00
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
$self->{pid} = $pid;
$self->{booted} = 1;
}
# Send a command to the monitor and wait for it to finish. TODO: QEMU
# also has a JSON-based monitor interface now, but it doesn't support
# all commands yet. We should use it once it does.
sub sendMonitorCommand {
my ($self, $command) = @_;
$self->log("sending monitor command: $command");
syswrite $self->{monitor}, "$command\n";
return $self->waitForMonitorPrompt;
}
# Wait until the monitor sends "(qemu) ".
sub waitForMonitorPrompt {
my ($self) = @_;
my $res = "";
my $s;
while (sysread($self->{monitor}, $s, 1024)) {
$res .= $s;
last if $res =~ s/\(qemu\) $//;
}
return $res;
}
# Call the given code reference repeatedly, with 1 second intervals,
# until it returns 1 or a timeout is reached.
sub retry {
my ($coderef) = @_;
my $n;
for ($n = 899; $n >=0; $n--) {
return if &$coderef($n);
sleep 1;
}
die "action timed out after $n seconds";
}
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
sub connect {
my ($self) = @_;
return if $self->{connected};
$self->nest("waiting for the VM to finish booting", sub {
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
$self->start;
my $now = clock_gettime(CLOCK_MONOTONIC);
local $SIG{ALRM} = sub { die "timed out waiting for the VM to connect\n"; };
alarm 600;
readline $self->{socket} or die "the VM quit before connecting\n";
alarm 0;
2015-06-10 10:02:48 +00:00
$self->log("connected to guest root shell");
# We're interested in tracking how close we are to `alarm`.
$self->log(sprintf("(connecting took %.2f seconds)", clock_gettime(CLOCK_MONOTONIC) - $now));
$self->{connected} = 1;
});
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
}
sub waitForShutdown {
my ($self) = @_;
return unless $self->{booted};
$self->nest("waiting for the VM to power off", sub {
waitpid $self->{pid}, 0;
$self->{pid} = 0;
$self->{booted} = 0;
$self->{connected} = 0;
});
}
sub isUp {
my ($self) = @_;
return $self->{booted} && $self->{connected};
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
}
sub execute_ {
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
my ($self, $command) = @_;
2015-06-10 10:02:48 +00:00
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
$self->connect;
print { $self->{socket} } ("( $command ); echo '|!=EOF' \$?\n");
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
my $out = "";
while (1) {
my $line = readline($self->{socket});
die "connection to VM lost unexpectedly" unless defined $line;
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
#$self->log("got line: $line");
if ($line =~ /^(.*)\|\!\=EOF\s+(\d+)$/) {
$out .= $1;
$self->log("exit status $2");
return ($2, $out);
}
$out .= $line;
}
}
sub execute {
my ($self, $command) = @_;
my @res;
$self->nest("running command: $command", sub {
@res = $self->execute_($command);
});
return @res;
}
sub succeed {
my ($self, @commands) = @_;
my $res;
foreach my $command (@commands) {
$self->nest("must succeed: $command", sub {
my ($status, $out) = $self->execute_($command);
if ($status != 0) {
$self->log("output: $out");
die "command `$command' did not succeed (exit code $status)\n";
}
$res .= $out;
});
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
}
return $res;
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
}
sub mustSucceed {
succeed @_;
}
sub waitUntilSucceeds {
my ($self, $command) = @_;
$self->nest("waiting for success: $command", sub {
retry sub {
my ($status, $out) = $self->execute($command);
return 1 if $status == 0;
};
});
}
sub waitUntilFails {
my ($self, $command) = @_;
$self->nest("waiting for failure: $command", sub {
retry sub {
my ($status, $out) = $self->execute($command);
return 1 if $status != 0;
};
});
}
sub fail {
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
my ($self, $command) = @_;
$self->nest("must fail: $command", sub {
my ($status, $out) = $self->execute_($command);
die "command `$command' unexpectedly succeeded"
if $status == 0;
});
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
}
sub mustFail {
fail @_;
}
2012-10-04 16:34:44 +00:00
sub getUnitInfo {
my ($self, $unit, $user) = @_;
my ($status, $lines) = $self->systemctl("--no-pager show \"$unit\"", $user);
2012-10-04 16:34:44 +00:00
return undef if $status != 0;
my $info = {};
foreach my $line (split '\n', $lines) {
$line =~ /^([^=]+)=(.*)$/ or next;
$info->{$1} = $2;
}
return $info;
}
sub systemctl {
my ($self, $q, $user) = @_;
if ($user) {
$q =~ s/'/\\'/g;
return $self->execute("su -l $user -c \$'XDG_RUNTIME_DIR=/run/user/`id -u` systemctl --user $q'");
}
return $self->execute("systemctl $q");
}
# Fail if the given systemd unit is not in the "active" state.
sub requireActiveUnit {
my ($self, $unit) = @_;
$self->nest("checking if unit $unit has reached state 'active'", sub {
my $info = $self->getUnitInfo($unit);
my $state = $info->{ActiveState};
if ($state ne "active") {
die "Expected unit $unit to to be in state 'active' but it is in state $state\n";
};
});
}
2012-10-04 16:34:44 +00:00
# Wait for a systemd unit to reach the "active" state.
sub waitForUnit {
my ($self, $unit, $user) = @_;
2012-10-04 16:34:44 +00:00
$self->nest("waiting for unit $unit", sub {
retry sub {
my $info = $self->getUnitInfo($unit, $user);
2012-10-04 16:34:44 +00:00
my $state = $info->{ActiveState};
die "unit $unit reached state $state\n" if $state eq "failed";
if ($state eq "inactive") {
# If there are no pending jobs, then assume this unit
# will never reach active state.
my ($status, $jobs) = $self->systemctl("list-jobs --full 2>&1", $user);
if ($jobs =~ /No jobs/) { # FIXME: fragile
# Handle the case where the unit may have started
# between the previous getUnitInfo() and
# list-jobs.
my $info2 = $self->getUnitInfo($unit);
die "unit $unit is inactive and there are no pending jobs\n"
if $info2->{ActiveState} eq $state;
}
}
2012-10-04 16:34:44 +00:00
return 1 if $state eq "active";
};
});
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
}
2012-10-04 16:34:44 +00:00
sub waitForJob {
my ($self, $jobName) = @_;
return $self->waitForUnit($jobName);
2012-10-04 16:34:44 +00:00
}
# Wait until the specified file exists.
sub waitForFile {
my ($self, $fileName) = @_;
$self->nest("waiting for file $fileName", sub {
retry sub {
my ($status, $out) = $self->execute("test -e $fileName");
return 1 if $status == 0;
}
});
}
sub startJob {
my ($self, $jobName, $user) = @_;
$self->systemctl("start $jobName", $user);
2012-10-04 16:34:44 +00:00
# FIXME: check result
}
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
sub stopJob {
my ($self, $jobName, $user) = @_;
$self->systemctl("stop $jobName", $user);
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
}
# Wait until the machine is listening on the given TCP port.
sub waitForOpenPort {
my ($self, $port) = @_;
$self->nest("waiting for TCP port $port", sub {
retry sub {
my ($status, $out) = $self->execute("nc -z localhost $port");
return 1 if $status == 0;
}
});
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
}
# Wait until the machine is not listening on the given TCP port.
sub waitForClosedPort {
my ($self, $port) = @_;
retry sub {
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
my ($status, $out) = $self->execute("nc -z localhost $port");
return 1 if $status != 0;
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
}
}
sub shutdown {
my ($self) = @_;
return unless $self->{booted};
print { $self->{socket} } ("poweroff\n");
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
$self->waitForShutdown;
}
sub crash {
my ($self) = @_;
return unless $self->{booted};
2015-06-10 10:02:48 +00:00
$self->log("forced crash");
$self->sendMonitorCommand("quit");
$self->waitForShutdown;
}
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
# Make the machine unreachable by shutting down eth1 (the multicast
# interface used to talk to the other VMs). We keep eth0 up so that
# the test driver can continue to talk to the machine.
sub block {
my ($self) = @_;
$self->sendMonitorCommand("set_link virtio-net-pci.1 off");
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
}
# Make the machine reachable.
sub unblock {
my ($self) = @_;
$self->sendMonitorCommand("set_link virtio-net-pci.1 on");
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
}
# Take a screenshot of the X server on :0.0.
sub screenshot {
my ($self, $filename) = @_;
my $dir = $ENV{'out'} || Cwd::abs_path(".");
$filename = "$dir/${filename}.png" if $filename =~ /^\w+$/;
my $tmp = "${filename}.ppm";
my $name = basename($filename);
$self->nest("making screenshot $name", sub {
$self->sendMonitorCommand("screendump $tmp");
system("pnmtopng $tmp > ${filename}") == 0
or die "cannot convert screenshot";
unlink $tmp;
}, { image => $name } );
}
# Get the text of TTY<n>
sub getTTYText {
my ($self, $tty) = @_;
my ($status, $out) = $self->execute("fold -w\$(stty -F /dev/tty${tty} size | awk '{print \$2}') /dev/vcs${tty}");
return $out;
}
# Wait until TTY<n>'s text matches a particular regular expression
sub waitUntilTTYMatches {
my ($self, $tty, $regexp) = @_;
$self->nest("waiting for $regexp to appear on tty $tty", sub {
retry sub {
my ($retries_remaining) = @_;
if ($retries_remaining == 0) {
$self->log("Last chance to match /$regexp/ on TTY$tty, which currently contains:");
$self->log($self->getTTYText($tty));
}
return 1 if $self->getTTYText($tty) =~ /$regexp/;
}
});
}
# Debugging: Dump the contents of the TTY<n>
sub dumpTTYContents {
my ($self, $tty) = @_;
$self->execute("fold -w 80 /dev/vcs${tty} | systemd-cat");
}
# Take a screenshot and return the result as text using optical character
# recognition.
sub getScreenText {
my ($self) = @_;
system("command -v tesseract &> /dev/null") == 0
or die "getScreenText used but enableOCR is false";
my $text;
$self->nest("performing optical character recognition", sub {
my $tmpbase = Cwd::abs_path(".")."/ocr";
my $tmpin = $tmpbase."in.ppm";
$self->sendMonitorCommand("screendump $tmpin");
my $magickArgs = "-filter Catrom -density 72 -resample 300 "
. "-contrast -normalize -despeckle -type grayscale "
. "-sharpen 1 -posterize 3 -negate -gamma 100 "
. "-blur 1x65535";
my $tessArgs = "-c debug_file=/dev/null --psm 11 --oem 2";
$text = `convert $magickArgs $tmpin tiff:- | tesseract - - $tessArgs`;
my $status = $? >> 8;
unlink $tmpin;
die "OCR failed with exit code $status" if $status != 0;
});
return $text;
}
# Wait until a specific regexp matches the textual contents of the screen.
sub waitForText {
my ($self, $regexp) = @_;
$self->nest("waiting for $regexp to appear on the screen", sub {
retry sub {
my ($retries_remaining) = @_;
if ($retries_remaining == 0) {
$self->log("Last chance to match /$regexp/ on the screen, which currently contains:");
$self->log($self->getScreenText);
}
return 1 if $self->getScreenText =~ /$regexp/;
}
});
}
# Wait until it is possible to connect to the X server. Note that
# testing the existence of /tmp/.X11-unix/X0 is insufficient.
sub waitForX {
my ($self, $regexp) = @_;
$self->nest("waiting for the X11 server", sub {
retry sub {
my ($status, $out) = $self->execute("journalctl -b SYSLOG_IDENTIFIER=systemd | grep 'Reached target Current graphical'");
return 0 if $status != 0;
($status, $out) = $self->execute("[ -e /tmp/.X11-unix/X0 ]");
return 1 if $status == 0;
}
});
}
sub getWindowNames {
my ($self) = @_;
my $res = $self->mustSucceed(
q{xwininfo -root -tree | sed 's/.*0x[0-9a-f]* \"\([^\"]*\)\".*/\1/; t; d'});
return split /\n/, $res;
}
sub waitForWindow {
my ($self, $regexp) = @_;
$self->nest("waiting for a window to appear", sub {
retry sub {
my @names = $self->getWindowNames;
my ($retries_remaining) = @_;
if ($retries_remaining == 0) {
$self->log("Last chance to match /$regexp/ on the the window list, which currently contains:");
$self->log(join(", ", @names));
}
foreach my $n (@names) {
return 1 if $n =~ /$regexp/;
}
}
});
}
sub copyFileFromHost {
my ($self, $from, $to) = @_;
my $s = `cat $from` or die;
$s =~ s/'/'\\''/g;
$self->mustSucceed("echo '$s' > $to");
}
my %charToKey = (
'A' => "shift-a", 'N' => "shift-n", '-' => "0x0C", '_' => "shift-0x0C", '!' => "shift-0x02",
'B' => "shift-b", 'O' => "shift-o", '=' => "0x0D", '+' => "shift-0x0D", '@' => "shift-0x03",
'C' => "shift-c", 'P' => "shift-p", '[' => "0x1A", '{' => "shift-0x1A", '#' => "shift-0x04",
'D' => "shift-d", 'Q' => "shift-q", ']' => "0x1B", '}' => "shift-0x1B", '$' => "shift-0x05",
'E' => "shift-e", 'R' => "shift-r", ';' => "0x27", ':' => "shift-0x27", '%' => "shift-0x06",
'F' => "shift-f", 'S' => "shift-s", '\'' => "0x28", '"' => "shift-0x28", '^' => "shift-0x07",
'G' => "shift-g", 'T' => "shift-t", '`' => "0x29", '~' => "shift-0x29", '&' => "shift-0x08",
'H' => "shift-h", 'U' => "shift-u", '\\' => "0x2B", '|' => "shift-0x2B", '*' => "shift-0x09",
'I' => "shift-i", 'V' => "shift-v", ',' => "0x33", '<' => "shift-0x33", '(' => "shift-0x0A",
'J' => "shift-j", 'W' => "shift-w", '.' => "0x34", '>' => "shift-0x34", ')' => "shift-0x0B",
'K' => "shift-k", 'X' => "shift-x", '/' => "0x35", '?' => "shift-0x35",
'L' => "shift-l", 'Y' => "shift-y", ' ' => "spc",
'M' => "shift-m", 'Z' => "shift-z", "\n" => "ret",
);
sub sendKeys {
my ($self, @keys) = @_;
foreach my $key (@keys) {
$key = $charToKey{$key} if exists $charToKey{$key};
$self->sendMonitorCommand("sendkey $key");
}
}
sub sendChars {
my ($self, $chars) = @_;
$self->nest("sending keys $chars", sub {
$self->sendKeys(split //, $chars);
});
}
# Sleep N seconds (in virtual guest time, not real time).
sub sleep {
my ($self, $time) = @_;
$self->succeed("sleep $time");
}
# Forward a TCP port on the host to a TCP port on the guest. Useful
# during interactive testing.
sub forwardPort {
my ($self, $hostPort, $guestPort) = @_;
$hostPort = 8080 unless defined $hostPort;
$guestPort = 80 unless defined $guestPort;
$self->sendMonitorCommand("hostfwd_add tcp::$hostPort-:$guestPort");
}
* Stuff for automatic and manual testing of NixOS VMs. lib/build-vms.nix contains a function `buildVirtualNetwork' that takes a specification of a network of machines (as an attribute set of NixOS machine configurations) and builds a script that starts each configuration in a separate QEMU/KVM VM and connects them together in a virtual network. This script can be run manually to test the VMs interactively. There is also a function `runTests' that starts and runs the virtual network in a derivation, and then executes a test specification that tells the VMs to do certain things (i.e., letting one VM send an HTTP request to a webserver on another VM). The tests are written in Perl (for now). tests/subversion.nix shows a simple example, namely a network of two machines: a webserver that runs the Subversion subservice, and a client. Apache, Subversion and a few other packages are built with coverage analysis instrumentation. For instance, $ nix-build tests/subversion.nix -A vms $ ./result/bin/run-vms starts two QEMU/KVM instances. When they have finished booting, the webserver can be accessed from the host through http://localhost:8081/. It also has a small test suite: $ nix-build tests/subversion.nix -A report This runs the VMs in a derivation, runs the tests, and then produces a distributed code coverage analysis report (i.e. it shows the combined coverage on both machines). The Perl test driver program is in lib/test-driver. It executes commands on the guest machines by connecting to a root shell running on port 514 (provided by modules/testing/test-instrumentation.nix). The VMs are connected together in a virtual network using QEMU's multicast feature. This isn't very secure. At the very least, other processes on the same machine can listen to or send packets on the virtual network. On the plus side, we don't need to be root to set up a multicast virtual network, so we can do it from a derivation. Maybe we can use VDE instead. (Moved from the vario repository.) svn path=/nixos/trunk/; revision=16899
2009-08-31 14:25:12 +00:00
1;