Commit graph

253 commits

Author SHA1 Message Date
Arian van Putten cbc45b5981 nixos: Make nesting.children work in NixOS tests
We differentiate between modules and baseModules in  the
VM builder for NixOS tests. This way, nesting.children, eventhough
it doesn't inherit from parent, still has enough config to
actually complete the test. Otherwise, the qemu modules
would not be loaded, for example, and a nesting.children
statement would not evaluate.
2019-05-29 12:50:49 +02:00
Arian van Putten d50b434234 nixos: Make 'nesting.clone' work in NixOS tests
Because nesting.clone calls 'eval-config.nix' manually,
without the 'extraArgs' argument that provides the 'nodes'
argument to nixos modules in nixos tests, evaluating
of 'nesting.clone' definitions would fail with the following error

while evaluating the module argument `nodes' in "<redacted>"
while evaluating the attribute '_module.args.nodes' at undefined position:
attribute 'nodes' missing, at <redacted./nixpkgs/lib/modules.nix:163:28

by not using 'extraArgs' but a nixos module instead, the nodes parameter
gets propagated to the 'eval-config.nix' call that  'nesting.clone'
makes too -  getting rid of the error.

See  https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/system/activation/top-level.nix#L13-L23
See  https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/build-vms.nix#L27
See  https://github.com/NixOS/nixpkgs/issues/20886#issuecomment-495952149
2019-05-26 00:20:21 +02:00
Arian van Putten 2e75a7b516 nixos: doc: optionally include all modules in manual generation
Before this change `man 5 configuration.nix` would only show options of modules in
the `baseModules` set, which consists only of the list of modules in
`nixos/modules/module-list.nix`

With this change applied and `documentation.nixos.includeAllModules` option enabled
all modules included in `configuration.nix` file will be used instead.

This makes configurations with custom modules self-documenting. It also means
that importing non-`baseModules` modules like `gce.nix` or `azure.nix`
will make their documentation available in `man 5 configuration.nix`.

`documentation.nixos.includeAllModules` is currently set to `false` by
default as enabling it usually uncovers bugs and prevents evaluation.
It should be set to `true` in a release or two.

This was originally implemented in #47177, edited for more configurability,
documented and rebased onto master by @oxij.
2019-03-05 09:41:40 +00:00
Matthew Bauer 878965731f
nixos/lib/testing.nix: config defaults to {}
Fixes #51858

56e12aae54 ends up passing config to pkgs. Unfortunately this might be null and pkgs/top-level/default.nix assumes it is an attrset. To fix this, we just make the default for config = {}. Thanks to @kristoff3r for tracking this down.

/cc @domenkozar
2019-02-06 20:04:24 -05:00
Travis Athougies a66ef3aa3d Add options to build disk image function (#50239)
* add options to build disk image function

* Revert suffix changes
2019-01-24 01:47:55 +02:00
Samuel Dionne-Riel 50555a6d35
Merge pull request #54330 from samueldr/fix/disable-tests-shell-access
Revert "Add ssh backdoor to VM tests infrastructure."
2019-01-19 14:20:15 -05:00
Samuel Dionne-Riel 3aab228d09 Revert "Add ssh backdoor to VM tests infrastructure."
This reverts commit d6e3db44cf.

See #53935 for explanations. In short, it may be causing issues with
tests on the build infrastructure.
2019-01-19 13:24:39 -05:00
Danylo Hlynskyi 23a13b562c
kernel config: add support for CONFIG_SQUASHFS_ZSTD (#52967)
Also, allow override `make-squashfs.nix` compression parameters.
2019-01-17 15:24:44 +02:00
Silvan Mosberger 497e6f1705
Merge pull request #51661 from eonpatapon/testing-hosts
test: set machines fqdn in /etc/hosts
2019-01-12 20:41:13 +01:00
Samuel Dionne-Riel 2646a64fbc
Merge pull request #53827 from samueldr/feature/data-in-logs
tests: Logs timing in tests
2019-01-12 12:54:56 -05:00
Samuel Dionne-Riel 3b68ddb6fe
Merge pull request #53828 from samueldr/feature/double-alarm-time
tests: Wait for shell for twice as long (10m)
2019-01-12 12:08:14 -05:00
Samuel Dionne-Riel b28b37eb00 tests: Wait for shell for twice as long (10m)
See #49441 for an earlier attempt, which was subsequently reverted. I am
assuming that doubling the time will be sufficient if the machine is
overloaded since so many of the tests already pass at 5 minutes, while
still not holding back failures for needlessly long.
2019-01-11 22:40:19 -05:00
Samuel Dionne-Riel 5d93e2c01c test-driver: Logs time taken for nests 2019-01-11 22:36:31 -05:00
Samuel Dionne-Riel 1fe0018df8 test-driver: Adds time it took to connect to guest in logs
This will make it possible to track whether the time is generous or not
when ran on hydra.
2019-01-11 22:36:31 -05:00
Matthew Bauer 04373fd3cc
Merge pull request #52594 from matthewbauer/fix-51025
make-disk-image: use filterSource instead of cleanSource
2019-01-07 16:29:58 -06:00
Matthew Bauer f05d8f31ec make-disk-image: use filterSource instead of cleanSource
cleanSource does not appear to work correctly in this case. The path
does not get coerced to a string, resulting in a dangling symlink
produced in channel.nix.  Not sure why, but this
seems to fix it.

Fixes #51025.

/cc @elvishjericco
2019-01-07 16:28:50 -06:00
Frederik Rietdijk 9618abe87c Merge master into staging-next 2019-01-04 21:13:19 +01:00
Matthew Bauer 74312c7ef5
Merge pull request #52760 from akru/master
lib/make-ext4-fs: more efficient store maker
2019-01-03 15:07:27 -06:00
Alexander Krupenkin 2f0c495c31
lib/make-ext4-fs: more efficient store maker 2018-12-24 23:21:15 +03:00
Jan Tojnar ef935fa101
Merge branch 'master' into staging 2018-12-24 15:02:29 +01:00
msteen 8d217ede58 fix infinite recursion caused by the unnecessary inspection of options + fix is parent of mount point check (#51541) 2018-12-24 14:05:55 +01:00
Michael Raskin ede54f9144
Merge pull request #52379 from erikarvstedt/tesseract
Major tesseract improvements
2018-12-20 14:41:48 +00:00
Erik Arvstedt 8d1ba999cb
tesseract: rename to tesseract4, add alias
This is more consistent with the naming of the most popular versioned pkgs.
2018-12-19 18:09:56 +01:00
Frederik Rietdijk 9ab61ab8e2 Merge staging-next into staging 2018-12-19 09:00:36 +01:00
Maximilian Bosch 6c6341335b
nixos/test-driver: fix wording in error message about invalid node names
Since 113a6b9325 the test driver
explicitly ensures if the node names won't break the resulting Perl
script at runtime. This slightly improves the correctness of the error
message.
2018-12-18 23:46:54 +01:00
Franz Pletz 670c5ac8ef
Merge pull request #46806 from Ma27/disallow-dash-separators-in-machine-declarations
nixos/testing: disallow special chars in machine names in network expressions
2018-12-18 01:03:34 +00:00
Maximilian Bosch 113a6b9325
nixos/testing: disallow special chars in machine names in network expressions
These names are referenced by Perl variables inside the testing
frameworks which don't allow chars like `-` as character inside. An exemplary
expression may look like this:

```
{
  x11-vm = {
    services.xserver.enable = true;
  };
}
```

This expression evaluates, e.g. when running `nixos-build-vms`, but when
trying to run `./result/bin/nixos-run-vms`, an error like this occurs:

```
starting VDE switch for network 1
running the VM test script
error: Can't modify subtraction (-) in scalar assignment at (eval 17) line 1, at EOF
Bareword "test" not allowed while "strict subs" in use at (eval 17) line 1.
Can't modify subtraction (-) in scalar assignment at (eval 17) line 1, at EOF
Bareword "test" not allowed while "strict subs" in use at (eval 17) line 1.
vde_switch: EOF on stdin, cleaning up and exiting
cleaning up
```

This can be very confusing for beginners, this change breaks evaluation
if such names are used for machines.
2018-12-18 01:58:56 +01:00
volth bb9557eb7c lib.makePerlPath -> perlPackages.makePerlPath 2018-12-15 03:50:31 +00:00
Samuel Dionne-Riel ad27b068d7 installer: Adds AArch64 UEFI installer support. 2018-12-12 19:26:34 -05:00
Jean-Philippe Braun 764f16461b test: set machines fqdn in /etc/hosts 2018-12-07 14:22:21 +01:00
Samuel Dionne-Riel 70488665fa
Merge pull request #51207 from samueldr/fix/sd-image-slimming
sd-image: Slims the ext4 filesystem even more.
2018-12-06 23:35:09 +00:00
Samuel Dionne-Riel 61bdaad9a9 sd-image: Slims the ext4 filesystem even more.
This is to try and squeeze more lost space from the image, so that hydra
starts building it again.

The fsck previous to the resize2fs is required so resize2fs works.

The one afterwards is a sanity check.

Using `-M` from resize2fs will not give much saved space due to a known
(in the manual) issue.

```
[samueldr@aarch64:~/nixpkgs]$ ls -lh result-*/*/*.img
-r--r--r-- 1 root root 2.2G Jan  1  1970 result-original/sd-image/nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img
-r--r--r-- 1 root root 2.1G Jan  1  1970 result-M/sd-image/nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img
-r--r--r-- 1 root root 1.9G Jan  1  1970 result-slimmed/sd-image/nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img
```

```
[samueldr@aarch64:~/nixpkgs]$ nix path-info -S ./result-original
/nix/store/c8k9n78gylx293rjh762fr05a069kxp2-nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img   3844125000

[samueldr@aarch64:~/nixpkgs]$ nix path-info -S ./result-slimmed
/nix/store/962238skj5mnzhrsmjy23dyzmxk77sp4-nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img   3447473208
```
2018-11-30 19:11:49 -05:00
Domen Kožar 56e12aae54
Fix nixops evaluation
See https://discourse.nixos.org/t/nixops-stopped-working/1527
2018-11-26 23:12:31 +00:00
Ding Xiang Fei b011049cf6 Merge branch 'master' of https://github.com/nixos/nixpkgs into tarball-closureinfo 2018-11-26 12:04:07 +08:00
Ding Xiang Fei ceececbd04 Merge branch 'master' of https://github.com/nixos/nixpkgs into tarball-closureinfo 2018-11-14 12:32:28 +08:00
Robert Hensing cdca66d7e8 Add pkgs.nixosTest 2018-11-13 10:43:05 +01:00
Léo Gaspard cae5598611
meta.tests: drop meta.needsVMSupport
Its job is already handled by `requiredSystemFeatures`
2018-11-11 23:11:46 +09:00
Léo Gaspard 6c68fbd4e1
tests: refactor to carry the package set as an argument
This way, the package set will be possible to pass without re-importing
all the time
2018-11-11 23:11:45 +09:00
Ding Xiang Fei 4259f7575e use closure-info for building system tarball 2018-11-07 12:52:53 +08:00
Timo Kaufmann 6141939d6e
Merge pull request #44439 from Ekleog/meta-tests
[RFC] Use `meta.tests` to link from packages to the tests that test them
2018-11-07 00:05:22 +01:00
Sarah Brofeldt 95486ca306 Revert "NixOS tests: Wait for shell for 10x longer (50m)"
This reverts commit 9bc10e1291.
2018-11-05 08:58:08 +00:00
Matthew Bauer d3c2247b77
Merge pull request #47679 from NixOS/docs-make-channel
Document make-channel.nix
2018-11-04 16:51:28 -06:00
Robert Hensing 17e863d017 Remove incorrect sentence from make-channel comment
Besides being incorrect, claims like these tend to stick around
after they become incorrect.
2018-11-04 20:45:49 +01:00
Léo Gaspard 02e1f00ffd
dovecot, opensmtpd: add link to test in meta.tests
Rationale
---------

Currently, tests are hard to discover. For instance, someone updating
`dovecot` might not notice that the interaction of `dovecot` with
`opensmtpd` is handled in the `opensmtpd.nix` test.

And even for someone updating `opensmtpd`, it requires manual work to go
check in `nixos/tests` whether there is actually a test, especially
given not so many packages in `nixpkgs` have tests and this is thus most
of the time useless.

Finally, for the reviewer, it is much easier to check that the “Tested
via one or more NixOS test(s)” has been checked if the file modified
already includes the list of relevant tests.

Implementation
--------------

Currently, this commit only adds the metadata in the package. Each
element of the `meta.tests` attribute is a derivation that, when it
builds successfully, means the test has passed (ie. following the same
convention as NixOS tests).

Future Work
-----------

In the future, the tools could be made aware of this `meta.tests`
attribute, and for instance a `--with-tests` could be added to
`nix-build` so that it also builds all the tests. Or a `--without-tests`
to build without all the tests. @Profpatsch described in his NixCon talk
such systems.

Another thing that would help in the future would be the possibility to
reasonably easily have cross-derivation nix tests without the whole
NixOS VM stack. @7c6f434c already proposed such a system.

This RFC currently handles none of these concerns. Only the addition of
`meta.tests` as metadata to be used by maintainers to remember to run
relevant tests.
2018-10-30 21:31:39 +09:00
Sarah Brofeldt 9bc10e1291 NixOS tests: Wait for shell for 10x longer (50m) 2018-10-30 09:22:42 +01:00
Eelco Dolstra 0d15004cba
Merge pull request #49401 from aherrmann/stringify-modules-path
nixos/lib/eval-config.nix: toString modulesPath
2018-10-29 16:21:09 +01:00
Andreas Herrmann 044ceae280 nixos/lib/eval-config.nix: toString modulesPath
Referencing modulesPath in NixOS configurations can cause evaluation
errors in restricted mode.  If used as `${modulesPath}` (as in all
use-sites in nixpkgs) the modules subtree is copied into its own store
path. Access to this path will be forbidden in restricted mode.

Converting to a string solves this issue.
`${builtins.toString modulesPath}` will point to a subdirectory of the
nixpkgs tree out of which evalModules is called.

This change converts modulesPath to a string by default so that the
call-site doesn't have to anymore.
2018-10-29 15:46:20 +01:00
Linus Heckemann bbffdfcf72 Document make-channel.nix 2018-10-02 21:52:43 +02:00
Tuomas Tynkkynen d6e3db44cf Add ssh backdoor to VM tests infrastructure.
Thanks to @dezgeg for prototype implementation, I've
cleaned it up and added documentation.
2018-09-28 10:53:08 +01:00
Jan Malakhovski 0f3b89bbed nixos: doc: move non-service parts of service.nixosManual to documentation.nixos 2018-09-23 20:50:47 +00:00