Commit graph

51 commits

Author SHA1 Message Date
rnhmjoj 61b7cab481
treewide: use perl.withPackages when possible
Since 03eaa48 added perl.withPackages, there is a canonical way to
create a perl interpreter from a list of libraries, for use in script
shebangs or generic build inputs. This method is declarative (what we
are doing is clear), produces short shebangs[1] and needs not to wrap
existing scripts.

Unfortunately there are a few exceptions that I've found:

  1. Scripts that are calling perl with the -T switch. This makes perl
  ignore PERL5LIB, which is what perl.withPackages is using to inform
  the interpreter of the library paths.

  2. Perl packages that depends on libraries in their own path. This
  is not possible because perl.withPackages works at build time. The
  workaround is to add `-I $out/${perl.libPrefix}` to the shebang.

In all other cases I propose to switch to perl.withPackages.

[1]: https://lwn.net/Articles/779997/
2021-03-31 21:35:37 +02:00
Michael Raskin 9486375887 nixos/service-runner: fix evaluation
Follow-up to: nixos/systemd: allow preStart with other ExecStartPre cmdlines #109976

As the additional ExecStartPre and ExecStartPost are now lists, update
their processing by service-runner.nix
2021-02-05 13:02:56 +01:00
Andreas Rammhold f4d7493162
nixos/tests: only apply qemu parameters if the options are defined
This fixes an eval error that occurred on hydra with the small channel
and the `nixos.tests.boot.biosCdrom.x86_64-linux` attribute:

> $ nix-instantiate nixos/release-small.nix -A nixos.tests.boot.biosCdrom.x86_64-linux
> warning: unknown setting 'experimental-features'
> error: The option `virtualisation.qemu' does not exist. Definition values:
> - In `/home/andi/dev/nixos/nixpkgs/nixos/modules/testing/test-instrumentation.nix':
>     {
>       consoles = [ ];
>       package = {
> 	_type = "override";
> 	content = <derivation /nix/store/q72h2cdcb9zjgiay5gdgzwddjkbjr7xq-qemu-host-cpu-only-for-vm-tests-5.1.0.drv>;
>     ...
> (use '--show-trace' to show detailed location information)

In bc2188b we changed test test-instrumentation to also set the QEMU
package that is being used. That change unfortunately caused us to
always assing values to the virtualisation.qemu.package option even when
the option is not defined. The original code was explicitly testing for
the consoles case but the then newly extended version did not adjust the
check as the intention was probably not clear.

With this commit we are always ensuring the entire virtualisation.qemu
section exists and can thus drop the individual tests for each of the
sections since the QEMU module always defines both the package and the
consoles option when it's root is defined..
2020-10-25 13:42:01 +01:00
Maximilian Bosch aab534b894
nixos/nixos-build-vms: use pkgs.qemu for virtualisation
When I test a change e.g. in the module system manually, I usually use
`nixos-build-vms(8)` which also gives me a QEMU window where I can play
around in the freshly built VM.

It seems as this has changed recently when the default package for
non-interactive VM tests using the same framework was switched to
`pkgs.qemu_test` to reduce the closure size. While this is a reasonable
decision for our CI tests, I think that you really want a QEMU window of
the VM by default when using `nixos-build-vms(8)`.

[1] bc2188b083
2020-10-23 17:37:57 +02:00
rnhmjoj bc2188b083
nixos: fix qemu_test being used in normal VMs
This is an attempt to fixup PR #49403.
2020-10-21 16:38:04 +02:00
Andreas Rammhold 8ed57ac916
nixos/tests: make sure we use the qemu_test package to provide the Guest Agent
This reduces the closure size for the minimal test by a lot since we no
longer have to build the regular QEMU for even the simplest test.
2020-10-19 18:03:01 +02:00
Florian Klink 98d6b55fdc nixos/testing: remove remaining coverage-data logic
This isn't used anymore as per
https://github.com/NixOS/nixpkgs/pull/72354#discussion_r451031449.
2020-09-05 16:07:59 +02:00
Maximilian Bosch 2fbddb0ccb
nixos/test-instrumentation: properly import options for qemu-check
If `qemu-vm.nix` is imported, the option `virtualisation.qemu.consoles`
should be set to make sure that the machine's output isn't rendered on
the graphical window of QEMU.

This is needed when interactively running a NixOS test or in conjunction
with `nixos-build-vms(8)`.

The patch 2578557530 tries to only do this
if the option actually exists, however this condition used to be always
false since `options` wasn't imported in the module and pointed to
`lib.options` due to the `with lib;`-clause.
2020-08-18 12:26:49 +02:00
Robert Hensing 2578557530 nixos/test-instrumentation.nix: Fix evaluation error
Discovered via https://github.com/NixOS/nixpkgs/pull/82743 which
improved option checking, causing an evaluation error that was
hard to understand without running the evaluation manually.
2020-08-16 13:50:53 +02:00
Robert Hensing 43521ac965 nixos/service-runner.nix: Allow quotes in commands + test 2020-02-28 14:26:29 +01:00
rnhmjoj f4c497ae06
nixos/testing: remove leftover of virtualisation.qemu.program 2019-08-29 10:12:13 +02:00
Andreas Rammhold 7508490770
nixos/test: remove the stateVersion statement from the test-instrumentation
We set stateVersion to `mkDefault 18.03` in
`nixos/modules/testing/test-instrumentation.nix` and in
`modules/installer/cd-dvd/installation-cd-base.nix`.

Accessing the stateVersion in the module system from within the tests
results in the following error:
> The unique option `system.stateVersion' is defined multiple times, in
> `nixpkgs/nixos/modules/installer/cd-dvd/installation-cd-base.nix' and
> `nixpkgs/nixos/modules/testing/test-instrumentation.nix'.

There are other tests that use it as well. Namely the radicale test also
verifies behaviour between state versions is as expected. It switches a
package default value. Others switched on the state directory default.
It seems like having the timesyncd switch as part of every rendered
activationScript might cause this weird error.

Removing this line seems like a reasonable thing to do since we actually
set the default to the very same value in the module system. This line
should have been no-op besides the issue that we've two statements
setting it in this very specific case.
2019-06-03 15:05:24 +02:00
danbst 27982b408e types.optionSet: deprecate and remove last usages 2019-01-31 00:41:10 +02:00
Samuel Dionne-Riel 3aab228d09 Revert "Add ssh backdoor to VM tests infrastructure."
This reverts commit d6e3db44cf.

See #53935 for explanations. In short, it may be causing issues with
tests on the build infrastructure.
2019-01-19 13:24:39 -05:00
volth bb9557eb7c lib.makePerlPath -> perlPackages.makePerlPath 2018-12-15 03:50:31 +00:00
Tuomas Tynkkynen d6e3db44cf Add ssh backdoor to VM tests infrastructure.
Thanks to @dezgeg for prototype implementation, I've
cleaned it up and added documentation.
2018-09-28 10:53:08 +01:00
Brian Olsen 9540b1c535 nixos/tests: Set DefaultTimeoutStartSec very high (#44916)
DefaultTimeoutStartSec is normally set to 90 seconds and works fine. But
when running NixOS tests on a very slow machine (like a VM without
nested virtualisation support) this default is to low and causes
systemd units to fail spuriously. One symptom of this issue are tests
at times failing with "timed out waiting for the VM to connect".

Since the VM connect timeout is 300 seconds I also set
DefaultTimeoutStartSec to this which is ridiculously high.
2018-08-29 12:12:12 +02:00
Tuomas Tynkkynen 96190535e5 Revert "nixos: rename system.{stateVersion,defaultChannel} -> system.nixos.\1"
This reverts commit 095fe5b43d.

Pointless renames considered harmful. All they do is force people to
spend extra work updating their configs for no benefit, and hindering
the ability to switch between unstable and stable versions of NixOS.

Like, what was the value of having the "nixos." there? I mean, by
definition anything in a NixOS module has something to do with NixOS...
2018-07-28 00:12:55 +03:00
volth 2e979e8ceb [bot] nixos/*: remove unused arguments in lambdas 2018-07-20 20:56:59 +00:00
volth 87f5930c3f [bot]: remove unreferenced code 2018-07-20 18:48:37 +00:00
Florian Klink fff5923686 nixos/modules: users.(extraUsers|extraGroup->users|group) 2018-06-30 03:02:58 +02:00
Uli Baum 88fa50c2f2 nixos/tests: set default stateVersion in test VMs
... to avoid lots of warnings during eval
introduced by 1f0b6922d3
2018-05-14 00:01:15 +02:00
aszlig fb9f5e4a03
nixos/testing: Fix output of systemd messages
Regression introduced by d4468bedb5.

No systemd messages are shown anymore during VM test runs, which is not
very helpful if you want to find out about failures.

There is a bit of a conflict between testing and the change that
introduced the regression. While the mentioned commit makes sure that
the primary console is tty0 for virtualisation.graphics = false, our VM
tests need to have the serial console as primary console.

So in order to support both, I added a new virtualisation.qemu.consoles
option, which allows to specify those options using the module system.

The default of this option is to use the changes that were introduced
and in test-instrumentation.nix we use only the serial console the same
way as before.

For test-instrumentation.nix I didn't add a baudrate to the serial
console because I can't find a reason on top of my head why it should
need it. There also wasn't a reason stated when that was introduced in
7499e4a5b9.

Signed-off-by: aszlig <aszlig@nix.build>
Cc: @flokli, @dezgeg, @edolstra
2018-05-10 08:21:20 +02:00
Tuomas Tynkkynen 8e83158f12 nixos/qemu: Deduplicate QEMU serialDevice into qemu-flags.nix 2018-01-30 16:57:27 +02:00
Peter Hoeg 40af7718f8 tests: logToJournal has been renamed 2017-12-06 17:45:40 +08:00
Tuomas Tynkkynen e56da6a464 nixos/qemu-vm: Use different serial port on ARM/AArch64 2017-11-26 11:13:21 +02:00
Joerg Thalheim 00344162a7 Revert "nixos/test-instrumentation: softcode DISPLAY"
This reverts commit 4531ec5955.

This broke waitForX: b11ac5d29d (commitcomment-25486347)

cc @Lassulus
2017-11-08 17:04:39 +00:00
lassulus 4531ec5955 nixos/test-instrumentation: softcode DISPLAY 2017-11-07 23:01:59 +01:00
Vladimír Čunát 7c7c83e233
buildLinux: allow overriding stdenv on each call 2017-08-20 08:24:52 +02:00
Graham Christensen 3cec7d10df
kdm: drop service 2017-02-11 13:55:09 -05:00
Eelco Dolstra aad5d1f9a7
virtualisation.qemu.program: Remove
This option is defined in qemu-vm.nix, but that module is not always
imported.

http://hydra.nixos.org/build/44817443
(cherry picked from commit 03c55005dfd6fbcd5cf8e00128a3bb6336b3bc0f)
2016-12-20 10:52:46 +01:00
aszlig 1471426749
nixos/test-instrumentation: Fix eval of boot tests
The boot tests import test-instrumentation.nix directly to create a VM
image that only contains things such as the backdoor and serial console
the same way as used by other NixOS VM tests.

With one difference though: It doesn't need nor want to have 9p
filesystems mounted, because we actually want to test an image rather
than re-using most stuff from the host's store.

Change tested against the boot.uefiUsb and ipv6 tests, just that it
becomes clear we don't break either the tests with 9p nor the boot
tests (which were already broken but now succeed).

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-12-15 21:07:19 +01:00
aszlig 6cfb3b6364
nixos/tests: Use a patched QEMU for testing
The reason to patch QEMU is that with latest Nix, tests like "printing"
or "misc" fail because they expect the store paths to be owned by uid 0
and gid 0.

Starting with NixOS/nix@5e51ffb1c2, Nix
builds inside of a new user namespace. Unfortunately this also means
that bind-mounted store paths that are part of the derivation's inputs
are no longer owned by uid 0 and gid 0 but by uid 65534 and gid 65534.

This in turn causes things like sudo or cups to fail with errors about
insecure file permissions.

So in order to avoid that, let's make sure the VM always gets files
owned by uid 0 and gid 0 and does a no-op when doing a chmod on a store
path.

In addition, this adds a virtualisation.qemu.program option so that we
can make sure that we only use the patched version if we're *really*
running NixOS VM tests (that is, whenever we have imported
test-instrumentation.nix).

Tested against the "misc" and "printing" tests.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-11-17 17:16:16 +01:00
Eelco Dolstra b786b00023 KDE test: Bump kdm start timeout
Hopefully this will fix random failures like
http://hydra.nixos.org/build/36249079.
2016-05-27 11:22:27 +02:00
Eelco Dolstra c99608c638 Add an option to write the X session log to the journal
... rather than ~/.xsession-errors. It might make sense to make this
the default, in order to eliminate ad hoc, uncentralised, poorly
discoverable log files.
2016-05-24 21:46:38 +02:00
Eelco Dolstra 6f6e2c430f test-instrumentation.nix: Only clear $PAGER in the backdoor shell 2016-02-23 11:56:09 +01:00
Eelco Dolstra 6d02d7e740 NixOS tests: Increase hung_task_timeout_secs
This hopefully fixes intermittent test failures like

  http://hydra.nixos.org/build/29962437

  router# [  240.128835] INFO: task mke2fs:99 blocked for more than 120 seconds.
  router# [  240.130135]       Not tainted 3.18.25 #1-NixOS
  router# [  240.131110] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

assuming that these are caused by high load on the host.
2016-01-04 16:53:42 +01:00
aszlig 00934bb908
nixos/tests: Revert setting min_free_kbytes
This reverts commit 02b568414d.

With a5bc11f and 6353f58 in place, we really don't need this anymore.

After running about 500 VM tests on my Hydra, it still didn't improve
very much.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-12-14 17:40:28 +01:00
aszlig 02b568414d
nixos/test-instrumentation: Set vm.min_free_kbytes
We hit page allocation failures a lot at random for VM tests, in case of
my own Hydra when it comes to the installer tests. The reason for this
is that once the memory of the VM gets heavily fragmented the kernel is
unable to allocate new pages.

Setting vm.min_free_kbytes to 16MB forces the kernel to keep a minimum
of 16 MB free.

I've done some testing accross repeated runs of the installer tests with
and without vm.min_free_kbytes set. So accross 30 test runs for each
settings, all of the tests with the option being set passed while 14
tests without that sysctl option triggered page allocation failures.

Sure, running 30 tests is not a guarantee that 16MB is enough, but we'll
see how it turns out in the long run across all VM tests.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-12-13 03:18:10 +01:00
Eelco Dolstra 24ce7ff3ea test-instrumentation.nix: Prevent calling a pager 2015-01-15 14:39:29 +01:00
aszlig 8e516de3e7
nixos: Fix priorities of initialHashedPassword.
Regression introduced in f496c3cbe4.

Previously when we used security.initialRootPassword, the default
priority for this option was 1001, because it was a default value set by
the option itself.

With the mentioned commit, it is no longer an option default but a
mkDefault, which is priority 1000.

I'm setting this to 150 now, as test-instrumentation.nix is using this
for overriding other options and because I think it still makes it
possible to simple-override it, because if no priority is given, we get
priority 100.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-11-04 05:19:07 +01:00
Eelco Dolstra f496c3cbe4 Obsolete security.initialPassword
You can now set users.extraUsers.root.initialHashedPassword instead.
2014-11-03 12:36:56 +01:00
Eelco Dolstra d5b5d763cd Fix evaluation 2014-08-06 19:44:13 +02:00
Eelco Dolstra 2b9ea7fd90 test-instrumentation.nix: Set an empty root password
This makes it easier to log in during interactive sessions.
2014-08-06 14:15:23 +02:00
Eelco Dolstra eab25b104a test-instrumentation.nix: Improve logging
In particular, don't clobber the serial console with duplicate output
from systemd and the journal, and increase the log level.
2014-08-06 14:15:18 +02:00
Shea Levy b3cfb9084b Get all lib functions from lib, not pkgs.lib, in modules 2014-07-02 12:28:18 -04:00
Eelco Dolstra 2fbb9aba43 Fix the installer test
http://hydra.nixos.org/build/10419676
2014-04-20 01:56:11 +02:00
Eelco Dolstra 29027fd1e1 Rewrite ‘with pkgs.lib’ -> ‘with lib’
Using pkgs.lib on the spine of module evaluation is problematic
because the pkgs argument depends on the result of module
evaluation. To prevent an infinite recursion, pkgs and some of the
modules are evaluated twice, which is inefficient. Using ‘with lib’
prevents this problem.
2014-04-14 16:26:48 +02:00
Shea Levy d18bc25b95 Rename linuxManualConfig to buildLinux
Signed-off-by: Shea Levy <shea@shealevy.com>
2014-01-21 20:05:55 -05:00
Eelco Dolstra 2b0aea1793 Allow running NixOS services outside of systemd
The attribute ‘config.systemd.services.<service-name>.runner’
generates a script that runs the service outside of systemd.  This is
useful for testing, and also allows NixOS services to be used outside
of NixOS.  For instance, given a configuration file foo.nix:

  { config, pkgs, ... }:

  { services.postgresql.enable = true;
    services.postgresql.package = pkgs.postgresql92;
    services.postgresql.dataDir = "/tmp/postgres";
  }

you can build and run PostgreSQL as follows:

  $ nix-build -A config.systemd.services.postgresql.runner -I nixos-config=./foo.nix
  $ ./result

This will run the service's ExecStartPre, ExecStart, ExecStartPost and
ExecStopPost commands in an appropriate environment.  It doesn't work
well yet for "forking" services, since it can't track the main
process.  It also doesn't work for services that assume they're always
executed by root.
2013-11-18 18:04:17 +01:00