Commit graph

473 commits

Author SHA1 Message Date
talyz 64358cb0e9 nixos/utils: Handle arbitrary secrets in JSON output files
Introduce new functions which allows modules to define options where,
if the input is an attrset and the output is JSON, the user can define
arbitrary secrets.
2019-09-06 16:56:25 +02:00
volth 08f68313a4 treewide: remove redundant rec 2019-08-28 11:07:32 +00:00
Frederik Rietdijk 5061fe0c2c Merge staging-next into staging 2019-08-28 08:26:42 +02:00
volth 35d68ef143 treewide: remove redundant quotes 2019-08-26 21:40:19 +00:00
Clemens Fruhwirth f664823a1a make-disk-image.nix: Handle cptofs error explicitly
Because the copy process inside the VM does not reliably
give "No space" error message leaving the user wondering what
went wrong:

unable to create directory /mnt/0000fe01///nix/store/yknzxx7w2ck9p30k81gpi5yfjlrq41lr-libsecret-0.18.7/share/locale/ro: Success
[    5.462365] reboot: Restarting system
error processing entry /build/root/nix/store/yknzxx7w2ck9p30k81gpi5yfjlrq41lr-libsecret-0.18.7/share/locale/ro, aborting
error processing entry /build/root/nix/store/yknzxx7w2ck9p30k81gpi5yfjlrq41lr-libsecret-0.18.7/share/locale, aborting
error processing entry /build/root/nix/store/yknzxx7w2ck9p30k81gpi5yfjlrq41lr-libsecret-0.18.7/share, aborting
error processing entry /build/root/nix/store/yknzxx7w2ck9p30k81gpi5yfjlrq41lr-libsecret-0.18.7, aborting
error processing entry /build/root/nix/store, aborting
error processing entry /build/root/nix, aborting
builder for '/nix/store/fsdvqxq92iai7f3w8wcsncgfwag7cj2l-libvirtd-ssh-image.drv' failed with exit code 228
2019-08-26 13:09:53 +02:00
danbst d80cd26ff9 Merge branch 'master' into flip-map-foreach 2019-08-18 18:00:25 +03:00
Domen Kožar dcd50c0ea0
pkgs.lib -> lib 2019-08-12 11:46:53 +02:00
Domen Kožar 6cf861c617
make-options-doc: add asciidoc variant 2019-08-10 20:11:04 +02:00
Domen Kožar 3a93fcfd1e
make-options-doc: expose Nix set 2019-08-10 14:24:11 +02:00
Domen Kožar 5cfd034af0
Extract NixOS options documentation generation to a function
Motivation is to support other repositories containing nixos
modules that would like to generate options documentation:

- nix-darwin
- private repos
- arion
- ??
2019-08-08 16:18:09 +02:00
Nikola Knezevic d0ef94258d Make hostname in tests overridable
The original form effectively forbade any NixOS configuration that is under
test to explicitly set the hostname.
2019-08-07 10:31:13 +02:00
Danylo Hlynskyi 7585496eff
Merge branch 'master' into flip-map-foreach 2019-08-05 14:09:28 +03:00
danbst 0f8596ab3f mass replace "flip map -> forEach"
See `forEach`-introduction commit.
```
rg 'flip map ' --files-with-matches | xargs sed -i 's/flip map /forEach /g'
```
2019-08-05 14:03:38 +03:00
danbst 91bb646e98 Revert "mass replace "flip map -> foreach""
This reverts commit 3b0534310c.
2019-08-05 14:01:45 +03:00
Andrew Childs d2144755a4 nixos-test-driver: allow configuration of net frontend and backend
When IPXE tests were added, an option was added for configuring only
the frontend, and the backend configuration was dropped entirely. This
caused most installer tests to fail.
2019-07-22 13:44:27 +03:00
Nikolay Amiantov 81d35a9d7e nixos-test-driver: support netRomFile
Needed for UEFI PXE netboot testing.
2019-07-15 19:33:26 +03:00
danbst 3b0534310c mass replace "flip map -> foreach"
See `foreach`-introduction commit.
```
rg 'flip map ' --files-with-matches | xargs sed -i 's/flip map /foreach /g'
```
2019-07-14 13:46:10 +03:00
worldofpeace 3f4a353737 treewide: use dontUnpack 2019-07-01 04:23:51 -04:00
Samuel Dionne-Riel a0aff40f65 make-ext4-fs: Allows populating with custom files
This will allow adding /boot files to the rootfs.
2019-06-16 17:47:30 -04:00
Arian van Putten cbc45b5981 nixos: Make nesting.children work in NixOS tests
We differentiate between modules and baseModules in  the
VM builder for NixOS tests. This way, nesting.children, eventhough
it doesn't inherit from parent, still has enough config to
actually complete the test. Otherwise, the qemu modules
would not be loaded, for example, and a nesting.children
statement would not evaluate.
2019-05-29 12:50:49 +02:00
Arian van Putten d50b434234 nixos: Make 'nesting.clone' work in NixOS tests
Because nesting.clone calls 'eval-config.nix' manually,
without the 'extraArgs' argument that provides the 'nodes'
argument to nixos modules in nixos tests, evaluating
of 'nesting.clone' definitions would fail with the following error

while evaluating the module argument `nodes' in "<redacted>"
while evaluating the attribute '_module.args.nodes' at undefined position:
attribute 'nodes' missing, at <redacted./nixpkgs/lib/modules.nix:163:28

by not using 'extraArgs' but a nixos module instead, the nodes parameter
gets propagated to the 'eval-config.nix' call that  'nesting.clone'
makes too -  getting rid of the error.

See  https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/system/activation/top-level.nix#L13-L23
See  https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/build-vms.nix#L27
See  https://github.com/NixOS/nixpkgs/issues/20886#issuecomment-495952149
2019-05-26 00:20:21 +02:00
Arian van Putten 2e75a7b516 nixos: doc: optionally include all modules in manual generation
Before this change `man 5 configuration.nix` would only show options of modules in
the `baseModules` set, which consists only of the list of modules in
`nixos/modules/module-list.nix`

With this change applied and `documentation.nixos.includeAllModules` option enabled
all modules included in `configuration.nix` file will be used instead.

This makes configurations with custom modules self-documenting. It also means
that importing non-`baseModules` modules like `gce.nix` or `azure.nix`
will make their documentation available in `man 5 configuration.nix`.

`documentation.nixos.includeAllModules` is currently set to `false` by
default as enabling it usually uncovers bugs and prevents evaluation.
It should be set to `true` in a release or two.

This was originally implemented in #47177, edited for more configurability,
documented and rebased onto master by @oxij.
2019-03-05 09:41:40 +00:00
Matthew Bauer 878965731f
nixos/lib/testing.nix: config defaults to {}
Fixes #51858

56e12aae54 ends up passing config to pkgs. Unfortunately this might be null and pkgs/top-level/default.nix assumes it is an attrset. To fix this, we just make the default for config = {}. Thanks to @kristoff3r for tracking this down.

/cc @domenkozar
2019-02-06 20:04:24 -05:00
Travis Athougies a66ef3aa3d Add options to build disk image function (#50239)
* add options to build disk image function

* Revert suffix changes
2019-01-24 01:47:55 +02:00
Samuel Dionne-Riel 50555a6d35
Merge pull request #54330 from samueldr/fix/disable-tests-shell-access
Revert "Add ssh backdoor to VM tests infrastructure."
2019-01-19 14:20:15 -05:00
Samuel Dionne-Riel 3aab228d09 Revert "Add ssh backdoor to VM tests infrastructure."
This reverts commit d6e3db44cf.

See #53935 for explanations. In short, it may be causing issues with
tests on the build infrastructure.
2019-01-19 13:24:39 -05:00
Danylo Hlynskyi 23a13b562c
kernel config: add support for CONFIG_SQUASHFS_ZSTD (#52967)
Also, allow override `make-squashfs.nix` compression parameters.
2019-01-17 15:24:44 +02:00
Silvan Mosberger 497e6f1705
Merge pull request #51661 from eonpatapon/testing-hosts
test: set machines fqdn in /etc/hosts
2019-01-12 20:41:13 +01:00
Samuel Dionne-Riel 2646a64fbc
Merge pull request #53827 from samueldr/feature/data-in-logs
tests: Logs timing in tests
2019-01-12 12:54:56 -05:00
Samuel Dionne-Riel 3b68ddb6fe
Merge pull request #53828 from samueldr/feature/double-alarm-time
tests: Wait for shell for twice as long (10m)
2019-01-12 12:08:14 -05:00
Samuel Dionne-Riel b28b37eb00 tests: Wait for shell for twice as long (10m)
See #49441 for an earlier attempt, which was subsequently reverted. I am
assuming that doubling the time will be sufficient if the machine is
overloaded since so many of the tests already pass at 5 minutes, while
still not holding back failures for needlessly long.
2019-01-11 22:40:19 -05:00
Samuel Dionne-Riel 5d93e2c01c test-driver: Logs time taken for nests 2019-01-11 22:36:31 -05:00
Samuel Dionne-Riel 1fe0018df8 test-driver: Adds time it took to connect to guest in logs
This will make it possible to track whether the time is generous or not
when ran on hydra.
2019-01-11 22:36:31 -05:00
Matthew Bauer 04373fd3cc
Merge pull request #52594 from matthewbauer/fix-51025
make-disk-image: use filterSource instead of cleanSource
2019-01-07 16:29:58 -06:00
Matthew Bauer f05d8f31ec make-disk-image: use filterSource instead of cleanSource
cleanSource does not appear to work correctly in this case. The path
does not get coerced to a string, resulting in a dangling symlink
produced in channel.nix.  Not sure why, but this
seems to fix it.

Fixes #51025.

/cc @elvishjericco
2019-01-07 16:28:50 -06:00
Frederik Rietdijk 9618abe87c Merge master into staging-next 2019-01-04 21:13:19 +01:00
Matthew Bauer 74312c7ef5
Merge pull request #52760 from akru/master
lib/make-ext4-fs: more efficient store maker
2019-01-03 15:07:27 -06:00
Alexander Krupenkin 2f0c495c31
lib/make-ext4-fs: more efficient store maker 2018-12-24 23:21:15 +03:00
Jan Tojnar ef935fa101
Merge branch 'master' into staging 2018-12-24 15:02:29 +01:00
msteen 8d217ede58 fix infinite recursion caused by the unnecessary inspection of options + fix is parent of mount point check (#51541) 2018-12-24 14:05:55 +01:00
Michael Raskin ede54f9144
Merge pull request #52379 from erikarvstedt/tesseract
Major tesseract improvements
2018-12-20 14:41:48 +00:00
Erik Arvstedt 8d1ba999cb
tesseract: rename to tesseract4, add alias
This is more consistent with the naming of the most popular versioned pkgs.
2018-12-19 18:09:56 +01:00
Frederik Rietdijk 9ab61ab8e2 Merge staging-next into staging 2018-12-19 09:00:36 +01:00
Maximilian Bosch 6c6341335b
nixos/test-driver: fix wording in error message about invalid node names
Since 113a6b9325 the test driver
explicitly ensures if the node names won't break the resulting Perl
script at runtime. This slightly improves the correctness of the error
message.
2018-12-18 23:46:54 +01:00
Franz Pletz 670c5ac8ef
Merge pull request #46806 from Ma27/disallow-dash-separators-in-machine-declarations
nixos/testing: disallow special chars in machine names in network expressions
2018-12-18 01:03:34 +00:00
Maximilian Bosch 113a6b9325
nixos/testing: disallow special chars in machine names in network expressions
These names are referenced by Perl variables inside the testing
frameworks which don't allow chars like `-` as character inside. An exemplary
expression may look like this:

```
{
  x11-vm = {
    services.xserver.enable = true;
  };
}
```

This expression evaluates, e.g. when running `nixos-build-vms`, but when
trying to run `./result/bin/nixos-run-vms`, an error like this occurs:

```
starting VDE switch for network 1
running the VM test script
error: Can't modify subtraction (-) in scalar assignment at (eval 17) line 1, at EOF
Bareword "test" not allowed while "strict subs" in use at (eval 17) line 1.
Can't modify subtraction (-) in scalar assignment at (eval 17) line 1, at EOF
Bareword "test" not allowed while "strict subs" in use at (eval 17) line 1.
vde_switch: EOF on stdin, cleaning up and exiting
cleaning up
```

This can be very confusing for beginners, this change breaks evaluation
if such names are used for machines.
2018-12-18 01:58:56 +01:00
volth bb9557eb7c lib.makePerlPath -> perlPackages.makePerlPath 2018-12-15 03:50:31 +00:00
Samuel Dionne-Riel ad27b068d7 installer: Adds AArch64 UEFI installer support. 2018-12-12 19:26:34 -05:00
Robert Hensing 4272188364 nixos/lib/eval-config.nix: Add extraModules parameter for opt-in purity 2018-12-10 22:02:57 +01:00
Jean-Philippe Braun 764f16461b test: set machines fqdn in /etc/hosts 2018-12-07 14:22:21 +01:00
Samuel Dionne-Riel 70488665fa
Merge pull request #51207 from samueldr/fix/sd-image-slimming
sd-image: Slims the ext4 filesystem even more.
2018-12-06 23:35:09 +00:00
Samuel Dionne-Riel 61bdaad9a9 sd-image: Slims the ext4 filesystem even more.
This is to try and squeeze more lost space from the image, so that hydra
starts building it again.

The fsck previous to the resize2fs is required so resize2fs works.

The one afterwards is a sanity check.

Using `-M` from resize2fs will not give much saved space due to a known
(in the manual) issue.

```
[samueldr@aarch64:~/nixpkgs]$ ls -lh result-*/*/*.img
-r--r--r-- 1 root root 2.2G Jan  1  1970 result-original/sd-image/nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img
-r--r--r-- 1 root root 2.1G Jan  1  1970 result-M/sd-image/nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img
-r--r--r-- 1 root root 1.9G Jan  1  1970 result-slimmed/sd-image/nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img
```

```
[samueldr@aarch64:~/nixpkgs]$ nix path-info -S ./result-original
/nix/store/c8k9n78gylx293rjh762fr05a069kxp2-nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img   3844125000

[samueldr@aarch64:~/nixpkgs]$ nix path-info -S ./result-slimmed
/nix/store/962238skj5mnzhrsmjy23dyzmxk77sp4-nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img   3447473208
```
2018-11-30 19:11:49 -05:00
Domen Kožar 56e12aae54
Fix nixops evaluation
See https://discourse.nixos.org/t/nixops-stopped-working/1527
2018-11-26 23:12:31 +00:00
Ding Xiang Fei b011049cf6 Merge branch 'master' of https://github.com/nixos/nixpkgs into tarball-closureinfo 2018-11-26 12:04:07 +08:00
Ding Xiang Fei ceececbd04 Merge branch 'master' of https://github.com/nixos/nixpkgs into tarball-closureinfo 2018-11-14 12:32:28 +08:00
Robert Hensing cdca66d7e8 Add pkgs.nixosTest 2018-11-13 10:43:05 +01:00
Léo Gaspard cae5598611
meta.tests: drop meta.needsVMSupport
Its job is already handled by `requiredSystemFeatures`
2018-11-11 23:11:46 +09:00
Léo Gaspard 6c68fbd4e1
tests: refactor to carry the package set as an argument
This way, the package set will be possible to pass without re-importing
all the time
2018-11-11 23:11:45 +09:00
Ding Xiang Fei 4259f7575e use closure-info for building system tarball 2018-11-07 12:52:53 +08:00
Timo Kaufmann 6141939d6e
Merge pull request #44439 from Ekleog/meta-tests
[RFC] Use `meta.tests` to link from packages to the tests that test them
2018-11-07 00:05:22 +01:00
Sarah Brofeldt 95486ca306 Revert "NixOS tests: Wait for shell for 10x longer (50m)"
This reverts commit 9bc10e1291.
2018-11-05 08:58:08 +00:00
Matthew Bauer d3c2247b77
Merge pull request #47679 from NixOS/docs-make-channel
Document make-channel.nix
2018-11-04 16:51:28 -06:00
Robert Hensing 17e863d017 Remove incorrect sentence from make-channel comment
Besides being incorrect, claims like these tend to stick around
after they become incorrect.
2018-11-04 20:45:49 +01:00
Léo Gaspard 02e1f00ffd
dovecot, opensmtpd: add link to test in meta.tests
Rationale
---------

Currently, tests are hard to discover. For instance, someone updating
`dovecot` might not notice that the interaction of `dovecot` with
`opensmtpd` is handled in the `opensmtpd.nix` test.

And even for someone updating `opensmtpd`, it requires manual work to go
check in `nixos/tests` whether there is actually a test, especially
given not so many packages in `nixpkgs` have tests and this is thus most
of the time useless.

Finally, for the reviewer, it is much easier to check that the “Tested
via one or more NixOS test(s)” has been checked if the file modified
already includes the list of relevant tests.

Implementation
--------------

Currently, this commit only adds the metadata in the package. Each
element of the `meta.tests` attribute is a derivation that, when it
builds successfully, means the test has passed (ie. following the same
convention as NixOS tests).

Future Work
-----------

In the future, the tools could be made aware of this `meta.tests`
attribute, and for instance a `--with-tests` could be added to
`nix-build` so that it also builds all the tests. Or a `--without-tests`
to build without all the tests. @Profpatsch described in his NixCon talk
such systems.

Another thing that would help in the future would be the possibility to
reasonably easily have cross-derivation nix tests without the whole
NixOS VM stack. @7c6f434c already proposed such a system.

This RFC currently handles none of these concerns. Only the addition of
`meta.tests` as metadata to be used by maintainers to remember to run
relevant tests.
2018-10-30 21:31:39 +09:00
Sarah Brofeldt 9bc10e1291 NixOS tests: Wait for shell for 10x longer (50m) 2018-10-30 09:22:42 +01:00
Eelco Dolstra 0d15004cba
Merge pull request #49401 from aherrmann/stringify-modules-path
nixos/lib/eval-config.nix: toString modulesPath
2018-10-29 16:21:09 +01:00
Andreas Herrmann 044ceae280 nixos/lib/eval-config.nix: toString modulesPath
Referencing modulesPath in NixOS configurations can cause evaluation
errors in restricted mode.  If used as `${modulesPath}` (as in all
use-sites in nixpkgs) the modules subtree is copied into its own store
path. Access to this path will be forbidden in restricted mode.

Converting to a string solves this issue.
`${builtins.toString modulesPath}` will point to a subdirectory of the
nixpkgs tree out of which evalModules is called.

This change converts modulesPath to a string by default so that the
call-site doesn't have to anymore.
2018-10-29 15:46:20 +01:00
Linus Heckemann bbffdfcf72 Document make-channel.nix 2018-10-02 21:52:43 +02:00
Tuomas Tynkkynen d6e3db44cf Add ssh backdoor to VM tests infrastructure.
Thanks to @dezgeg for prototype implementation, I've
cleaned it up and added documentation.
2018-09-28 10:53:08 +01:00
Jan Malakhovski 0f3b89bbed nixos: doc: move non-service parts of service.nixosManual to documentation.nixos 2018-09-23 20:50:47 +00:00
Graham Christensen ca7391daf2
Merge pull request #46341 from obsidiansystems/fix-46320
nixpkgs module: Fix defaulting of `localSystem` and `system`
2018-09-08 09:16:41 -04:00
John Ericson 9f9723b179 nixpkgs module: Fix defaulting of localSystem and system
Take two of #40708 (4fe2898608).

That PR attempted to bidirectionally default `config.nixpkgs.system` and
`config.nixpkgs.localSystem.system` to each be updated by the other. But
this is not possible with the way the module system works. Divergence in
certain cases in inevitable.

This PR is more conservative and just has `system` default `localSystem`
and `localSystem` make the final call as-is. This solves a number of
issues.

 - `localSystem` completely overrides `system`, just like with nixpkgs
 proper. There is no need to specify `localSystem.system` to clobber the
 old system.

 - `config.nixpkgs.localSystem` is exactly what is passed to nixpkgs. No
 spooky steps.

 - `config.nixpkgs.localSystem` is elaborated just as nixpkgs would so
 that all attributes are available, not just the ones the user
 specified.

The remaining issue is just that `config.nixpkgs.system` doesn't update
based on `config.nixpkgs.localSystem.system`. It should never be
referred to lest it is a bogus stale value because
`config.nixpkgs.localSystem` overwrites it.

Fixes #46320
2018-09-07 16:43:56 -04:00
Shea Levy 18337f3ece
Merge branch 'no-toPath' 2018-09-06 08:09:53 -04:00
John Ericson 2c2f1e37d4 reewide: Purge all uses stdenv.system and top-level system
It is deprecated and will be removed after 18.09.
2018-08-30 17:20:32 -04:00
xeji b2dc75cd03
Merge pull request #43736 from volth/patch-208
qemu: 2.12.1 -> 3.0.0
2018-08-26 01:28:12 +02:00
Will Fancher 47d2f92a05 make-ext4-fs: Fix cross compiling 2018-08-21 14:05:23 +03:00
volth 341250fa10 qemu: 2.12.1 -> 3.0.0 2018-08-20 22:02:02 +00:00
volth 2e979e8ceb [bot] nixos/*: remove unused arguments in lambdas 2018-07-20 20:56:59 +00:00
Andrew Dunham d7bfd04301 sdImage: make partition ID/UUID configurable 2018-07-12 17:02:03 +03:00
Matthew Bauer 753c63dbc1 qemu-flags: default to qemu-kvm
this is the "generic" version & should work everywhere.

also add darwin case
2018-05-31 12:34:15 -04:00
Shea Levy 6da6accd30
treewide: Remove uses of builtins.toPath.
toPath has confusing semantics and is never necessary; it can always
either just be omitted or replaced by pre-concatenating `/.`. It has
been marked as "!!! obsolete?" for more than 10 years in a C++
comment, hopefully removing it will let us properly deprecate and,
eventually, remove it.
2018-05-22 16:42:02 -04:00
Tuomas Tynkkynen bc828721ed nixos/lib/make-ext4-fs: Add a sanity check
I ended up with a corrupted image with the debugfs contraption once, and
given I couldn't reproduce the problem I suppose that happens if the
filesystem of the builder runs out of space.

At least in this instance fsck could detect it, so let's add it as a
sanity check.
2018-05-05 19:30:54 +03:00
aszlig f148c5c4a1
nixos/tests: Fix QEMU flags for SCSI disks
The ability to specify "-drive if=scsi" has been removed in QEMU version
2.12 (introduced in 3e3b39f173).

Quote from https://wiki.qemu.org/ChangeLog/2.12#Incompatible_changes:

> The deprecated way of configuring SCSI devices with "-drive if=scsi"
> on x86 has been removed. Use an appropriate SCSI controller together
> "-device scsi-hd" or "-device scsi-cd" and a corresponding "-blockdev"
> parameter instead.

So whenever the diskInterface is "scsi" we use the new way to specify
the drive and fall back to the deprecated way for the time being. The
reason why I'm not using the new way for "virtio" and "ide" as well is
because there is no simple generic way anymore to specify these.

This also turns the type of the virtualisation.qemu.diskInterface option
to be an enum, so the user knows which values are allowed but we can
also make sure the right value is provided to prevent typos.

I've tested this against a few non-disk-related NixOS VM tests but also
the installer.grub1 test (because it uses "ide" as its drive interface),
the installer.simple test (just to be sure it still works with
"virtio") and all the tests in nixos/tests/boot.nix.

In order to be able to run the grub1 test I had to go back to
8b1cf100cd (which is a known commit where
that test still works) and apply the QEMU update and this very commit,
because right now the test is broken.

Apart from the tests here in nixpkgs, I also ran another[1] test in
another repository which uses the "scsi" disk interface as well (in
comparison to most of the installer tests, this one actually failed
prior to this commit).

All of them now succeed.

[1]: 9b5a119972/tests/system/kernel/bfq.nix

Signed-off-by: aszlig <aszlig@nix.build>
Cc: @edostra, @grahamc, @dezgeg, @abbradar, @ts468
2018-04-30 03:02:59 +02:00
John Ericson ba52ae5048 treewide: isArm -> isAarch32
Following legacy packing conventions, `isArm` was defined just for
32-bit ARM instruction set. This is confusing to non packagers though,
because Aarch64 is an ARM instruction set.

The official ARM overview for ARMv8[1] is surprisingly not confusing,
given the overall state of affairs for ARM naming conventions, and
offers us a solution. It divides the nomenclature into three levels:

```
ISA:             ARMv8   {-A, -R, -M}
                 /    \
Mode:     Aarch32     Aarch64
             |         /   \
Encoding:   A64      A32   T32
```

At the top is the overall v8 instruction set archicture. Second are the
two modes, defined by bitwidth but differing in other semantics too, and
buttom are the encodings, (hopefully?) isomorphic if they encode the
same mode.

The 32 bit encodings are mostly backwards compatible with previous
non-Thumb and Thumb encodings, and if so we can pun the mode names to
instead mean "sets of compatable or isomorphic encodings", and then
voilà we have nice names for 32-bit and 64-bit arm instruction sets
which do not use the word ARM so as to not confused either laymen or
experienced ARM packages.

[1]: https://developer.arm.com/products/architecture/a-profile
2018-04-25 15:28:55 -04:00
John Ericson 53686e8995
Merge pull request #38485 from obsidiansystems/nixos-nixpkgs-options
nixpkgs module: Clean up platform options
2018-04-19 14:59:58 -04:00
Robert Schütz 80fc5f2a24 Merge branch 'master' into staging 2018-04-10 09:13:36 +02:00
Tuomas Tynkkynen 487be791d7 nixos/make-ext4-fs: Use closureInfo
exportReferencesGraph is deprecated and doesn't have the generated
initial Nix database contain the SHA256 of the contents of the store
paths, which breaks various things under Nix 2.0.
2018-04-10 00:31:36 +03:00
John Ericson c6f7d43678 nixpkgs module: Clean up platform options
- `localSystem` is added, it strictly supercedes system

 - `crossSystem`'s description mentions `localSystem` (and vice versa).

 - No more weird special casing I don't even understand

TEMP
2018-04-06 12:41:44 -04:00
Frederik Rietdijk 23741692fa Merge master into staging 2018-04-04 19:36:39 +02:00
Will Dietz 0086036141 Machine.pm: tweak to look for graphical target in waitForX
Current message may not be printed
(nothing in systemd contains that string,
 it comes from elsewhere)
2018-04-03 13:59:16 -05:00
Michael Bishop 3c9e579d1e
make-system-tarball: allow alternate compression methods 2018-04-03 11:30:43 -03:00
Will Dietz ddd10e0836 nixos testing: significantly reduce deps needed for OCR tests
Instead of imagemagick built with many libraries
(notably librsvg which these days requires rust)
use imagemagick_light with support for libtiff added.
2018-03-31 00:24:15 -05:00
adisbladis c4c1d979e9
Merge pull request #35844 from adisbladis/jquery-mv
jquery & jquery-ui: Move to nixos/lib/testing/
2018-03-06 15:34:34 +08:00
Eelco Dolstra cc2eeef4ab
Fix installing the Nixpkgs channel on the installation media
And test that it got installed correctly.
2018-02-27 19:58:23 +01:00
adisbladis b7ec621e91
jquery & jquery-ui: Move to nixos/lib/testing/
These (outdated) derivations are only used by nixos/lib/testing.nix.

If we want to provide jquery & jquery-ui packages this is better done
in nodePackages.
2018-02-27 20:07:35 +08:00
Eelco Dolstra d12c9911df
Merge remote-tracking branch 'origin/master' into nix-2.0 2018-02-22 17:28:51 +01:00
Michael Fellinger 8c9a8d6805 Document the makeTest length restriction 2018-02-20 16:36:16 +02:00
Michael Fellinger 0d988d6735 Only allow test names of up to 50 characters 2018-02-20 16:36:16 +02:00
Michael Raskin 1e23a5a6e9
Merge pull request #34795 from oxij/os/nixosLabel-pt2
nixos: labels part 2
2018-02-19 14:45:17 +00:00
Jan Malakhovski 2e6b796761 nixos: rename config.system.nixos* -> config.system.nixos.* 2018-02-18 12:56:30 +00:00
rnhmjoj c1bed05e34
nixos/networking-interfaces: rename IP addresses/routes options 2018-02-17 14:01:24 +01:00
Franz Pletz 54c7ca34c4
nixos/testing: use the same qemu in the test driver
The qemu_test package is also used for running a NixOS qemu VM.
2018-02-11 23:40:54 +01:00
Eelco Dolstra 9802da517f
make-disk-image.nix: Use nixos-install again
Since nixos-install doesn't require any special privileges anymore,
this Just Works. No more need for fakeroot / nixos-prepare-root.
2018-02-07 19:39:13 +01:00
Eelco Dolstra df117acab7
ISO images: Initialize the Nix database with correct NAR hashes/sizes
The boot test now runs "nix verify" to ensure that all hashes are
correct.
2018-02-07 16:54:23 +01:00
Shea Levy 943592f698
Add setFunctionArgs lib function.
Among other things, this will allow *2nix tools to output plain data
while still being composable with the traditional
callPackage/.override interfaces.
2018-01-31 14:02:19 -05:00
Tuomas Tynkkynen 71631a922b runInLinuxVM: Use QEMU command line that works on other architectures
... by moving the existing definition to qemu-flags.nix and reusing
that.
2018-01-30 16:57:27 +02:00
Tuomas Tynkkynen 8e83158f12 nixos/qemu: Deduplicate QEMU serialDevice into qemu-flags.nix 2018-01-30 16:57:27 +02:00
Tuomas Tynkkynen 1ce1380497 nixos/qemu-flags: Take a 'pkgs' parameter
I'm gonna use it in the next commit.
2018-01-30 16:57:27 +02:00
Tuomas Tynkkynen 962e79ef32 nixos/make-disk-image.nix: Support EFI images
- Add a new parameter `imageType` that can specify either "efi" or
  "legacy" (the default which should see no change in behaviour by
  this patch).

- EFI images get a GPT partition table (instead of msdos) with a
  mandatory ESP partition (so we add an assert that `partitioned`
  is true).

- Use the partx tool from util-linux to determine exact start + size
  of the root partition. This is required because GPT stores a secondary
  partition table at the end of the disk, so we can't just have
  mkfs.ext4 create the filesystem until the end of the disk.

- (Unrelated to any EFI changes) Since we're depending on the
  `-E offset=X` option to mkfs which is only supported by e2fsprogs,
  disallow any attempts of creating partitioned disk images where
  the root filesystem is not ext4.
2018-01-22 11:18:23 +02:00
John Ericson eec050f395
Merge pull request #33577 from dtzWill/fix/cross-2
Minor cross fixes, 2
2018-01-09 12:36:53 -05:00
Ben Gamari a2215da9a1 make-ext4-fs: Dependencies are nativeBuildInputs 2018-01-09 11:25:19 -05:00
Ben Gamari d680678d0a makeSquashfs: Inputs are nativeBuildInputs 2018-01-09 11:25:19 -05:00
Jörg Thalheim 814b38541f
Merge pull request #32845 from Ma27/test-driver/allow-user-units
test-driver: support testing user units
2018-01-04 09:40:41 +01:00
Maximilian Bosch e538e00404
test-driver: support testing user units
It is quite complicated to test services using the test-driver when
declaring user services with `systemd.user.services` such as many
X11-based services like `xautolock.service`.

This change adds an optional `$user` parameter to each systemd-related
function in the test-driver and runs `systemctl --user` commands using
`su -l $user -c ...` and sets the `XDG_RUNTIME_DIR` variable
accordingly and a new function named `systemctl` which is able to run a
systemd command with or without a specified user.

The change can be confirmed with a simple VM declaration like this:

```
import ./nixos/tests/make-test.nix ({ pkgs, lib }:

with lib;

{
  name = "systemd-user-test";

  nodes.machine = {
    imports = [ ./nixos/tests/common/user-account.nix ];

    services.xserver.enable = true;
    services.xserver.displayManager.auto.enable = true;
    services.xserver.displayManager.auto.user = "bob";
    services.xserver.xautolock.enable = true;
  };

  testScript = ''
    $machine->start;
    $machine->waitForX;

    $machine->waitForUnit("xautolock.service", "bob");
    $machine->stopJob("xautolock.service", "bob");
    $machine->startJob("xautolock.service", "bob");
    $machine->systemctl("list-jobs --no-pager", "bob");
    $machine->systemctl("show 'xautolock.service' --no-pager", "bob");
  '';
})
```
2018-01-02 20:14:27 +01:00
Tuomas Tynkkynen 758b4c1ea4 make-disk-image.nix: Really fix write permissions in the store
I think the current one applies the -exec only to those that match
'-type d'. Let's switch it to something that humans can understand...
2017-12-20 17:28:46 +02:00
Tuomas Tynkkynen c9f71974f8 make-disk-image.nix: Remove write permissions from /nix/store
Fakeroot seems to always give the owner write bit to any files touched
inside it (presumably to easily simulate the fact that root can still
modify such files). So do an explicit chmod to remove them.

This should finally solve #32242 after the EC2 images are regenerated
with this change.

https://hydra.nixos.org/build/66143116
2017-12-17 01:19:05 +02:00
Tuomas Tynkkynen 6ed6d323fe make-squashfs.nix: Improve invalid path check messages
Now the mtime problems are gone, but EC2 Hydra builders are still having
some problems:

https://hydra.nixos.org/build/66043835
2017-12-15 19:57:40 +02:00
Tuomas Tynkkynen 0d27df280f build-support/vm: Use devtmpfs, not static device nodes
In 2017, there is no reason to create a static /dev.
2017-12-12 14:31:50 +02:00
Tuomas Tynkkynen 574526d510 make-squashfs: Hack to avoid building images where nixos-install won't run
https://github.com/NixOS/nixpkgs/issues/32242
2017-12-03 03:37:45 +02:00
Tuomas Tynkkynen 8762490337 nixos/qemu: Add virtio-rng-pci to test-driver
Otherwise AArch64 KVM guests don't have enough entropy at boot.
2017-11-26 11:13:21 +02:00
Orivej Desh 50d8720d2a nixos/testing: fix after --set ceased to evaluate the value 2017-11-19 14:15:19 +00:00
Orivej Desh 470a7752d8 make-disk-image: fix eval 2017-11-13 14:56:30 +00:00
Orivej Desh 27f5cc959b make-disk-image: embed compression switch into the format name 2017-11-12 18:39:36 +00:00
Orivej Desh 608de48fe4 make-disk-image: add compressed option to compress qcow2 images 2017-11-12 18:28:06 +00:00
Orivej Desh a2d3ee77b8 make-disk-image: clean up 2017-11-12 18:24:26 +00:00
Orivej Desh a84ce72fdc make-disk-image: do not use faketime
Since https://github.com/lkl/linux/pull/394 cptofs preserves the source time,
which is 1970-01-01T00:00:01Z for /nix/store and recent for other files.

This reverts commit f5b3f2c5a7.
2017-11-11 20:57:14 +00:00
Orivej Desh f218ef647c make-disk-image: clean up 2017-11-11 20:24:26 +00:00
Dan Peebles f5b3f2c5a7 make-disk-iamge: use faketime to get a consistent timestamp
This fixes #28768 because during an image build, Nix sees bad store
timestamps and attempts to fix them, but can't fix them on a running
system (due to being inside a builder). Since timestamps on the store
are supposed to be 1 anyway, if we fix this, that fixes image building
inside booted images made this way.

Note that this adds quite a bit of noise to the output, because running
`cptofs` under `faketime` causes a bunch of seemingly spurious error
messages and my attempts to suppress them all failed. We'll fix it when
`cptofs` gets a native timestamp preservation feature.
2017-11-10 05:15:37 +00:00
Joerg Thalheim 00344162a7 Revert "nixos/test-instrumentation: softcode DISPLAY"
This reverts commit 4531ec5955.

This broke waitForX: b11ac5d29d (commitcomment-25486347)

cc @Lassulus
2017-11-08 17:04:39 +00:00
lassulus 4531ec5955 nixos/test-instrumentation: softcode DISPLAY 2017-11-07 23:01:59 +01:00
Peter Simons 757a759005 Merge pull request #30166 from LumiGuide/graphite-1.0.2
Fix graphite crash by upgrading from 0.9.15 -> 1.0.2
2017-10-18 12:58:55 +01:00
Bjørn Forsman 0ff4bb5f87 nixos: run parted with --script option
-s, --script: never prompts for user intervention

Sometimes the NixOS installer tests fail when they invoke parted, e.g.
https://hydra.nixos.org/build/62513826/nixlog/1. But instead of exiting
right there, the tests hang until the Nix builder times out (and kills
the build). With this change the tests would instead fail immediately,
which is preferred.

While at it, use "parted --script" treewide, so nobody gets build
timeout due to parted error (or misuse). (Only nixos/ use it, and only
non-interactive.)

A few instances already use the short option "-s", convert them to long
option "--short".
2017-10-14 15:29:02 +02:00
aszlig 20487112ed
nixos: Fix output path generation of runInMachine
Regression introduced by a02bb00156.

The fix is done by disabling writableStore, because the latter will set
up an overlayfs on the Nix store within the VM, which in turn will
discard all the outputs of the resulting output path.

However in runInMachine we actually *want* the contents of the generated
path and also don't want a writable store within the VM (except of
course for $out, which is writable anyway).

I've added a small regression test to verifify the output in
nixos/tests/run-in-machine.nix to make sure this won't break again in
the future.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2017-10-11 20:08:11 +02:00
Bas van Dijk 5b8ff5ed49 graphite: 0.9.15 -> 1.0.2
Fixes: #29961

Also added the option:

  services.graphite.web.extraConfig

for configuring graphite_web.
2017-10-08 03:03:22 +02:00
Dan Peebles e2589b3ca2 Deduplicate some filterSource invocations
This version should have more conventional regexes that work across many
platforms and regex engines. This is an issue because up until Nix 1.11,
Nix called out to the libc regex matcher, which behaved differently on
Darwin and Linux. And in Nix 1.12, we're moving to std::regex which will
also behave differently here.

And yes, I do actually evaluate make-disk-image.nix on Darwin ;)
2017-08-29 20:27:04 -04:00
Dan Peebles 5f372ef67f amazon-image: support VPC/VHD as an output format for the builder 2017-08-10 22:57:26 +00:00
Tuomas Tynkkynen 1d72474df7 make-ext4-fs: Fix modification timestamps in image
Use '-f' to make timestamp calls to always return the same time. Also
Nix uses the timestamp of '1' instead of epoch directly.
2017-07-29 22:03:43 +03:00
Graham Christensen 3f40fcabbf
nixos tests: waitForWindow: output a list of windows we see prior to the final check
machine: must succeed: xwininfo -root -tree | sed 's/.*0x[0-9a-f]* \"\([^\"]*\)\".*/\1/; t; d'
    machine: exit status 0
    machine: Last chance to match /(?^:dfiirst configuration)/ on the the window list, which currently contains:
    machine: [i3 con] container around 0xf8a5f0, i3: first configuration, [i3 con] floatingcon around 0xf8c260, [i3 con] container around 0xf8a380, i3bar for output Virtual-1, [i3 con] bottom dockarea Virtual-1, [i3 con] workspace 1, [i3 con] content Virtual-1, [i3 con] top dockarea Virtual-1, [i3 con] output Virtual-1, [i3 con] workspace __i3_scratch, [i3 con] content __i3, [i3 con] pseudo-output __i3, i3
2017-06-20 21:16:35 -04:00
Graham Christensen 1b833015b7
nixos tests: waitForText: output the detected screen content prior to the last attempt
machine: Last chance to match /(?^:BALICE)/ on the screen, which currently contains:
    machine: performing optical character recognition
    machine: sending monitor command: screendump /tmp/nix-build-vm-test-run-sddm.drv-0/ocrin.ppm
    machine: Session Layout

    O O

    0 1 : 0 9

    Wednesday, June 21, 2017

    |_ I

    Select your user and enter password
2017-06-20 21:10:34 -04:00
Graham Christensen 348785eec0
nixos tests: waitUntilTTYMatches: Log TTY contents on last try
If the test has not passed yet, on the last attempt it now outputs:

    machine: Last chance to match /logine: / on TTY2, which currently contains:
    machine: running command: fold -w$(stty -F /dev/tty2 size | awk '{print $2}') /dev/vcs2
    machine: exit status 0
    machine:

    <<< Welcome to NixOS 17.09.git.a804ef4 (x86_64) - tty2 >>>

    machine login:

to help debug the problem. Notice the "logine" typo in my check.
2017-06-20 20:57:39 -04:00
Graham Christensen 56435c1404
nixos tests: retry: Count down to 0, and pass remaining attempts to the sub
Allows test functions to output diagnostic information on failure.
2017-06-20 20:54:33 -04:00
Eelco Dolstra 0996ea8a68
NixOS VM tests: Don't create a setgid group in vde_switch
Nix no longer allows this for security reasons.

http://hydra.nixos.org/build/53993125
2017-06-07 11:57:36 +02:00
Volth e5fbeb2f11 test-driver: sendKeys() support for capital letters 2017-05-19 13:56:24 +00:00
Dan Peebles f1708a9d7d make-disk-image: change to be less VM-centric
This changes much of the make-disk-image.nix logic (and thus most NixOS
image building) to use LKL to set up the target directory structure rather
than a Linux VM. The only work we still do in a VM is less IO-heavy stuff
that while still time-consuming, is less of the overall load. The goal is
to kill more of that stuff, but that will require deeper changes to NixOS
activation scripts and switch-to-configuration.pl, and I don't want to
bite off too much at once.
2017-04-24 02:30:00 +00:00
aszlig a443bdc0a6
nixos/testing: Improve quality of OCR
First of all, we're now using ImageMagick to improve the screenshot so
that Tesseract has an esier time to recognize the text. The resulting
image of this post-processing is a scaled up black-and-white version
with the backgrounds almost entirely removed and the text edges a bit
blurred, so the screen shots now more or less resemble an image from a
scanner rather. This is what Tesseract is trained for by default.

As mentioned in the previous commit we now also use Tesseract 4, which
further improves the quality of text recognition.

I've spent countless hours just to test different postprocessing
variants and testing what works best for our tests and this is the one
that worked best so far. It's certainly not perfect and I'd like to
avoid the scaling step but we're way better off than before.

In addition to this, the OCR process is now done without an intermediate
file, solely using pipes.

I've tested this using the following VM tests which have OCR enabled:

 * nixos/tests/chromium.nix -A stable
 * nixos/tests/emacs-daemon.nix
 * nixos/tests/installer.nix -A luksroot
 * nixos/tests/lightdm.nix
 * nixos/tests/plasma5.nix
 * nixos/tests/sddm.nix

All of the tests still succeed and comparing some of the recognition
results to the earlier results it now also detects a lot more text than
before this commit.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2017-04-11 03:21:53 +02:00
aszlig 288a79187c
tesseract: Reintroduce enableLanguages
I've removed that attribute in 68bc260ca2,
because the language files no longer were distributed as seperate files,
but if we for example only want to use the English training data, the
closure size of Tesseract gets quite large (around 1.2 GB), which is a
bit much just to be able to run NixOS VM tests.

For this reason I've also switched the VM tests back to using only the
English language.

Tested using the following VM tests (the ones that have OCR enabled) on
x86_64-linux:

 * nixos/tests/chromium.nix -A stable
 * nixos/tests/emacs-daemon.nix
 * nixos/tests/installer.nix -A luksroot
 * nixos/tests/lightdm.nix
 * nixos/tests/plasma5.nix
 * nixos/tests/sddm.nix

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2017-04-11 03:21:26 +02:00
Nikolay Amiantov 1f0ce0eaf8 makeTest: don't rely on makeWrapper not quoting newlines 2017-03-20 16:15:38 +03:00
Dan Peebles e798f573f0 make-disk-image.nix: set last fsck time on ext4 images to enable resize-on-startup 2017-02-26 02:02:22 +00:00
Dan Peebles 49641e0de0 make-disk-image.nix: support additional filesystem contents
This makes make-disk-image.nix slightly more consistent with other image
builders we have. Unfortunately I duplicated some code in doing so, but
this is temporary duplication on the path to consolidating everything.
See https://github.com/NixOS/nixpkgs/issues/23052 for more details on that.

I'm also exposing the option in the amazon-image.nix maintainer module.
2017-02-22 23:49:49 +00:00
aszlig a5e9668c5c
nixos/test: Fix escaping for copyFileFromHost
A long-time issue and one of the reasons I've never used that function
before. So let's remove that todo-comment and escape the contents
properly.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Cc: @edolstra
2017-02-17 19:45:49 +01:00
Graham Christensen 59e77daf5b
nixos tests: make getTTYText smart about tty size 2017-02-12 18:27:06 -05:00
volth 50ec3fe1ac
test-driver: support punctuation in sendChars 2017-01-08 09:17:05 -05:00
Chris Martin 3956435d0a eval-config.nix: configuration->configuration.nix
`configuration` seems to be a reference to an argument that was
removed seven years ago in commit 2892aed7.

`configuration.nix` makes it a big more clear what we're referring to.
2017-01-04 19:15:47 +01:00
Joachim Fasting 48f270db52
make-system-tarball.sh: use numeric uids
Ownership should now always be 0/0.

Pointed out by @dezgeg at
https://github.com/NixOS/nixpkgs/pull/21305#discussion_r93258149
2017-01-01 06:03:05 +01:00
Eelco Dolstra 996e0ef777
test-driver: Show test duration 2016-12-29 16:23:27 +01:00
Lluís Batlle i Rossell 897b10998b Tune mksquashfs for smaller isos.
Bigger block size, more suitable for xz.
2016-12-28 16:07:16 +01:00
Joachim Fasting 168fe999e7
make-system-tarball: create reproducible tarballs
Ensure that archive members are added in sorted order with a fixed
mtime.  This allows `nix-build --check` to succeed (when building a
tarball of a simple system configuration).

We also remove env-vars which doesn't appear to do much apart from
capture a bunch of store paths we probably don't want.

This is an alternative to
4b78a5b5fb
2016-12-20 15:26:21 +01:00
Eelco Dolstra f173da375d
Use only one build of qemu in VM tests
Previously we were using two or three (qemu_kvm, qemu_test, and
qemu_test with a different dbus when minimal.nix is included).

(cherry picked from commit 8bfa4ce82ea7d23a1d4c6073bcc044e6bf9c4dbe)
2016-12-20 10:52:46 +01:00
Eelco Dolstra aad5d1f9a7
virtualisation.qemu.program: Remove
This option is defined in qemu-vm.nix, but that module is not always
imported.

http://hydra.nixos.org/build/44817443
(cherry picked from commit 03c55005dfd6fbcd5cf8e00128a3bb6336b3bc0f)
2016-12-20 10:52:46 +01:00
aszlig 68bc260ca2
tesseract: 3.02.02 -> 3.04.01
From the upstream changelog:

 * Tesseract development is now done with Git and hosted at github.com
   (Previously we used Subversion as a VCS and code.google.com for
   hosting).

So let's move over to the GitHub repository, where the organisation also
includes a full repository for tessdata, so we no longer need to fetch
it one-by-one.

The build also got significantly simpler, because we no longer need to
run autoconf, neither do we need to patch the configure script for
Leptonica headers.

This also has the advantage that we don't need to use the
enableLanguages attribute for the test runner anymore.

Full upstream changelog can be found at:

https://github.com/tesseract-ocr/tesseract/blob/c4d273d33cc36e/ChangeLog

Tested against all NixOS tests with enabled OCR (chromium, emacs-daemon,
installer.luksroot and lightdm).

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Cc: @viric
2016-12-19 22:25:38 +01:00
Domen Kožar b6363c7bc8 make-disk-image: make store validity fix optional
This is useful for EC2 AMI generation to speedup the process.

In my case it removes 13min out of 45min when generating an image
on EC2.
2016-12-07 13:30:20 +01:00
Domen Kožar e5cca82d79 make-disk-image: run tune2fs after umount to skip fsck
tune2fs marks the filesystem as clean to prevent resize2fs from
complaining.

But we were invoking it before we mounted the filesystem, so the
counters would increase to 1 and it broke the functionality.

By moving the call after the mount, I have confirmed it works by:

   $ nix-build nixos/tests/ec2.nix

cc @rbvermaa @edolstra
2016-12-07 13:30:20 +01:00
Graham Christensen cb74fd75d7
login test: Create and use direct reads of the TTY contents. 2016-11-30 00:17:18 -05:00
Franz Pletz 6de991bd95
nixos: compress squashfs with xz 2016-11-23 02:24:13 +01:00
aszlig b4e2b6bc6a
nixos/lib/testing: Fix unsetting $xchg
Regression introduced by 4dcb685af9.

Unsetting the environment variable shortly before using it is not going
to end up very well, so let's just filter out the variable from the
output of export and unset it shortly afterwards.

This fixes the runInMachine NixOS test.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-09-11 17:31:12 +02:00
Eelco Dolstra 4dcb685af9 runInMachine: Unset another variable
Thanks @cstrahan.
2016-09-09 20:02:56 +02:00
Eelco Dolstra 75baee8523 runInMachine: Support passAsFile
We need to rewrite attributes passed via files to their location in
/tmp/xchg in the VM. Otherwise functions like runCommand don't work.
2016-09-08 15:38:56 +02:00
Jan Malakhovski b267785c43 nixos: generalize copy-paste from stage-1 and zfs to utils 2016-08-23 18:14:06 +00:00
Jan Malakhovski 65d26c4dc1 nixos: apply toposort to fileSystems to support bind and move mounts
And use new `config.system.build.fileSystems` property everywhere.
2016-08-23 18:14:05 +00:00
Tuomas Tynkkynen 282277dbc8 treewide: Use more makeBinPath 2016-08-23 01:18:10 +03:00
obadz 24f8cf08cc nixos/lib/make-disk-image: refactor to use nixos-install
- Replace hand-rolled version of nixos-install in make-disk-image by an
  actual call to nixos-install
- Required a few cleanups of nixos-install
- nixos-install invokes an activation script which the hand-rolled version
  in make-disk-image did not do. We remove /etc/machine-id as that's
  a host-specific, impure, output of the activation script

Testing:

nix-build '<nixpkgs/nixos/release.nix>' -A tests.installer.simple passes

Also tried generating an image with:

nix-build -E 'let
    pkgs = import <nixpkgs> {};
    lib = pkgs.lib;
    nixos = import <nixpkgs/nixos> {
      configuration = {
        fileSystems."/".device = "/dev/disk/by-label/nixos";
        boot.loader.grub.devices = [ "/dev/sda" ];
        boot.loader.grub.extraEntries = '"''"'
          menuentry "Ubuntu" {
             insmod ext2
             search --set=root --label ubuntu
             configfile /boot/grub/grub.cfg
          }
        '"''"';
      };
    };
  in import <nixpkgs/nixos/lib/make-disk-image.nix> {
    inherit pkgs lib;
    config = nixos.config;
    diskSize = 2000;
    partitioned = false;
    installBootLoader = false;
  }'

Then installed the image:
$ sudo df if=./result/nixos.img of=/dev/sdaX bs=1M
$ sudo resize2fs /dev/disk/by-label/nixos
$ sudo mount /dev/disk/by-label/nixos /mnt
$ sudo mount --rbind /proc /mnt/proc
$ sudo mount --rbind /dev /mnt/dev
$ sudo chroot /mnt /nix/var/nix/profiles/system/bin/switch-to-configuration boot

[ … optionally do something about passwords … ]

and successfully rebooted to that image.

Was doing all this from inside a Ubuntu VM with a single user nix install.
2016-08-16 15:31:16 +01:00
obadz 760b2b9048 nixos/make-disk-image: add ability to defer bootloader install until image has been flashed 2016-08-15 20:01:55 +01:00
zimbatm 6df0bff908 nixos: throw an error on invalid shell package
All shell packages must export the shellPath passthru
2016-07-04 15:12:27 +01:00
Vladimír Čunát 6b27ceb006 Merge 'master' into staging and re-revert merge
... from staging to master, reverted temporarily in aa9a04883e.
2016-06-23 12:09:03 +02:00
Vladimír Čunát aa9a04883e Revert "Merge branch 'staging'" due to glibc
The main output started to retain dependency on bootstrap-tools; see
https://github.com/NixOS/nixpkgs/pull/15867#issuecomment-227949096

This reverts commit c05d829598, reversing
changes made to f073df60d6.
2016-06-23 09:25:10 +02:00
Vladimír Čunát c05d829598 Merge branch 'staging' 2016-06-22 10:49:56 +02:00
Vladimír Čunát 61b8d27b4c make-wrapper: fixup #14753 changes to make tests work
The tests need to expand passed variable and very carefully.
I could see no other easy way than to change single-quoting in
makeWrapper to double-quoting.
The tests now fail with the same problem as on master...
2016-06-19 13:38:46 +02:00
zimbatm 2974b6f4c8 Use shell packages to select the user's shell
The string type is still available for backward-compatiblity.
2016-06-12 20:35:34 +01:00
aszlig cb796ccd09
nixos/test-driver/Logger: Replace invalid UTF-8
Regression introduced by d84741a4bf.

The mentioned commit actually is a good thing, because we now get the
output from the X session.

Unfortunately, for the i3wm test, the i3-config-wizard prints out the
raw keyboard symbols directly coming from xcb, so the output isn't
necessarily proper UTF-8.

As the XML::Writer already expects valid UTF-8 input, we assume that
everything that comes into sanitise() will be UTF-8 from the start. So
we just decode() it using FB_DEFAULT as the check argument so that
every invalid character is replaced by the unicode replacement
character:

https://en.wikipedia.org/wiki/Specials_(Unicode_block)#Replacement_character

We simply re-oncode it again afterwards and return it, so we should
always get out valid UTF-8 in the log XML.

For more information about FB_DEFAULT and FB_CROAK, have a look at:

http://search.cpan.org/~dankogai/Encode-2.84/Encode.pm#Handling_Malformed_Data

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-05-26 14:24:33 +02:00
Eelco Dolstra ad29b72686 test-driver: Fix "unit X is inactive and there are no pending jobs"
This was causing many random test failures.
2016-05-24 16:33:14 +02:00
Tuomas Tynkkynen 60f5659dad treewide: Use correct output in ${config.nix.package}/bin 2016-04-25 16:44:37 +02:00
Eelco Dolstra 9153d8ed64 Fix X11 tests broken by the removal of -ac
Probably missed a few. Also adding xauth to the system path (it was
already in the closure).
2016-04-12 19:13:47 +02:00
Eelco Dolstra 0d3738cdcc Fix the EC2 test
We now generate a qcow2 image to prevent hitting Hydra's output size
limit. Also updated /root/user-data -> /etc/ec2-metadata/user-data.

http://hydra.nixos.org/build/33843133
2016-03-30 21:50:23 +02:00
Domen Kožar c56c3b6596 nixos/lib/testing.nix: make 'config' a free variable
cc @edolstra
2016-03-29 14:16:54 +01:00
Eelco Dolstra 10293b87a9 Combine ISO generation steps
This folds adding hydra-build-products into the actual ISO generation,
preventing an unnecessary download of the ISO.
2016-03-15 14:15:12 +01:00
Eelco Dolstra 5cc7bcda30 Combine OVA generation steps
Previously this was done in three derivations (one to build the raw
disk image, one to convert to OVA, one to add a hydra-build-products
file). Now it's done in one step to reduce the amount of copying
to/from S3. In particular, not uploading the raw disk image prevents
us from hitting hydra-queue-runner's size limit of 2 GiB.
2016-03-15 14:15:12 +01:00
Profpatsch 7f44b58609 wheter → whether
Nice weather today, isn’t it?
2016-03-07 03:06:54 +01:00
Eelco Dolstra f3d94cfc23 Revert "Add the tool "nixos-typecheck" that can check an option declaration to:"
This reverts commit cad8957eab. It
breaks NixOps, but more importantly, such major changes to the module
system really need to be reviewed.
2016-03-01 20:52:06 +01:00
Thomas Strobel cad8957eab Add the tool "nixos-typecheck" that can check an option declaration to:
- Enforce that an option declaration has a "defaultText" if and only if the
   type of the option derives from "package", "packageSet" or "nixpkgsConfig"
   and if a "default" attribute is defined.

 - Enforce that the value of the "example" attribute is wrapped with "literalExample"
   if the type of the option derives from "package", "packageSet" or "nixpkgsConfig".

 - Warn if a "defaultText" is defined in an option declaration if the type of
   the option does not derive from "package", "packageSet" or "nixpkgsConfig".

 - Warn if no "type" is defined in an option declaration.
2016-02-29 01:09:00 +01:00
Eelco Dolstra 146c727924 waitForUnit: Fail if the unit reaches inactive state with no pending jobs
This prevents waitForUnit for looping for many minutes before giving
up.
2015-12-30 15:17:52 +01:00
Timofei Kushnir c7f4092ed3 Enable to create hybrid ISO without UEFI boot 2015-11-28 08:55:47 +03:00
Domen Kožar f1508b3a23 nova-image: use make-disk-image.nix 2015-10-26 16:16:45 +01:00
Eelco Dolstra 106738b196 Give more memory for the disk image builder
http://hydra.nixos.org/build/26480662
2015-10-06 15:03:26 +02:00
Eelco Dolstra 64aed5e78f Fix Nix database in generated images
This prevents seeing lots of warnings about missing hashes/sizes in the
database when running "nix-store --verify --check-contents" for the
first time.
2015-09-28 14:47:48 +02:00
Eelco Dolstra b3347287be Use make-disk-image.nix for VirtualBox images 2015-09-28 14:31:19 +02:00
Eelco Dolstra aeb31b97ad Update AMI generator
The EBS and S3 (instance-store) AMIs are now created from the same
image. HVM instance-store AMIs are also generated.

Disk image generation has been factored out into a function
(nixos/lib/make-disk-image.nix) that can be used to build other kinds
of images.
2015-09-27 21:06:40 +02:00
Tobias Geerinckx-Rice e87fa04ddc nixos: move left-behind allowUnfree comment 2015-09-03 01:17:10 +02:00
Eelco Dolstra 1e18b75170 Prefer local build for the test driver script 2015-08-26 15:04:47 +02:00
Nicolas B. Pierron c47e89623b Update option-usages.nix expression to work with newer version of the module system. 2015-08-09 17:52:34 +02:00
Eelco Dolstra 2cd7c1f198 Unify NixOS and Nixpkgs channel structure
This is primarily to ensure that

  -I nixpkgs=https://nixos.org/channels/nixos-unstable/nixexprs.tar.xz

and

  -I nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/nixos-unstable.tar.gz

and

  -I nixpkgs=https://nixos.org/channels/nixpkgs-unstable/nixexprs.tar.xz

behave consistently.

It also allows installing packages via "nix-env -iA nixos.<pkg>"
rather than "nixos.pkgs.<pkg>". It would be even better to allow
"nixpkgs.<pkg>", but that requires a change to nix-channel.

Fixes #7659.
2015-08-05 17:37:11 +02:00
Eelco Dolstra 8a5000cc78 nixUnstable: Update to 1.10pre4200_76cc8e9
Also unify the nixStable and nixUnstable expressions.
2015-07-30 12:08:20 +02:00
viric 982ce5ed58 Merge pull request #8978 from dezgeg/pr-arm-images
ARM SD card image expressions
2015-07-29 14:13:57 +02:00
Eelco Dolstra 7c30ce9562 Remove zipTwoLists
This function is redundant (we also have zipLists).
2015-07-28 15:04:18 +02:00
Tuomas Tynkkynen df86813d97 nixos: Add derivations for SD card installation images on ARM
The resulting image can be copied to a SD card with `dd` and is directly
bootable by a suitably configured U-Boot. Though depending on the board, some
extra steps are required for copying U-Boot itself to the SD card.

Inside the image is a partition table, with a FAT32 /boot and a normal
writable EXT4 rootfs. It's possible to directly reuse the SD image's
partition layout and "install" NixOS on the same SD card by replacing
the default configuration.nix and nixos-rebuild, and actually is the
preferred way to use these images. To assist in this installation
method, the boot scripts on the image automatically resize the rootfs
partition to fit the SD card on the first boot.

The SD images come in two flavors; one for the ARMv6 Raspberry Pi,
and one multiplatform image for all the boards supported by the
mainline kernel's multi_v7_defconfig config target. At the moment, these
have been tested on:
    - Raspberry Pi Model B (512MB model)
    - NVIDIA Jetson TK1
    - Linksprite pcDuino3 Nano

To build, run:

nix-build '<nixpkgs/nixos>' -A config.system.build.sdImage \
    -I nixos-config='<nixpkgs/nixos/modules/installer/cd-dvd/sd-image-armv7l-multiplatform.nix>'
2015-07-26 00:31:20 +03:00
Joachim Schiele 0731489953 all tests: added meta.maintainers section 2015-07-12 12:29:51 +02:00
Dan Peebles 10e75453b2 Factor the NixOS channel building code out into its own file, so I can use it elsewhere 2015-06-13 21:55:29 -04:00
Dan Peebles b6c589b2da Simple EC2 user-data VM test 2015-06-11 22:54:04 -04:00
Daniel Peebles 6e73884c31 Merge pull request #8204 from copumpkin/modulesPath
An importable modulesPath, once again
2015-06-10 12:32:04 -04:00
Eelco Dolstra 0f35f9bb69 Drop obsolete boot=on parameter 2015-06-10 15:28:55 +02:00
Dan Peebles b13cb54614 Bring an importable modulesPath back from the dead, in the only way I know 2015-06-08 01:55:49 +00:00
aszlig 2475a6a239
nixos/test-driver: Fix bashism in Machine.pm.
Thanks to @bjornfor for the note.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-22 08:10:17 +02:00
aszlig f12208c5a4
nixos/testing: Use stripped down tesseract only.
Only include the English language for the VM tests, because we most
likely won't need other languages. At least for now.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-22 08:01:17 +02:00
aszlig 8be00dc71d
nixos/test-driver: Make tesseract OCR optional.
By default this is now enabled, and it has to be explicitely enabled
using "enableOCR = true". If it is set to false, any usage of
getScreenText or waitForText will fail with an error suggesting to pass
enableOCR.

This should get rid of the rather large dependency on tesseract which
we don't need for most tests.

Note, that I'm using system("type -P") here to check whether tesseract
is in PATH. I know it's a bashism but we already have other bashisms
within the test scripts and we also run it with bash, so IMHO it's not a
problem here.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-22 07:21:58 +02:00
aszlig 8bd025a75e
nixos/test-driver: Add new waitForText function.
As promised in the previous commit, this can be used similarly to
$machine->waitForWindow, where you supply a regular expression and it's
retrying OCR until the regexp matches.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-21 19:55:21 +02:00
aszlig 235c2228ca
nixos/test-driver: Add new getScreenText function.
Basically, this creates a screenshot and throws tesseract at it to
recognize the characters from the screenshot. In order to produce a
result that is well enough, we're using lanczos scaling and scale the
image up to 400% of its original size.

This provides the base functionality for a new Machine method which will
be called waitForText. I originally had that idea long ago when writing
the VM tests for VirtualBox and Chromium, but thought it would be
disproportionate to the case.

The downside however is that VM tests now depend on tesseract, but given
the average runtime of our tests it really shouldn't have a too big
impact and it's only a runtime dependency after all.

Another issue is that the OCR process takes quite some time to finish,
but IMHO it's better (as in more deterministic) than to rely on sleep().

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-21 19:55:21 +02:00
Eelco Dolstra 13dcb35233 VM tests: Don't produce output on failure
Producing output makes it hard to restart builds that failed for
intermittent reasons (like timeouts). So let's not do that.
2015-04-10 11:18:30 +02:00
Nicolas B. Pierron 6de931a0f8 Merge rename.nix changes. 2015-04-03 23:12:12 +02:00
Bob van der Linden c57a912016 nixos: test: add tests for booting installation iso in various ways 2015-03-24 22:09:07 +01:00
Bob van der Linden 16a936be90 nixos: iso-image: filename escaping in xorriso 2015-03-24 22:01:51 +01:00
Bob van der Linden 8d092270d2 nixos: iso-image: use syslinux bootloader for USB booting support
This changes the bootloader for iso generation from Grub to
syslinux. In addition this adds USB booting support, so that
"dd" can be used to burn the generated ISO to USB thumbdrives
instead of needing applications like UnetBootin.
2015-03-24 22:01:04 +01:00
Nicolas B. Pierron 05e8a48fb4 Document and rename internal option of modules. 2015-03-15 14:45:42 +01:00
Shea Levy f69ce50529 Move most extra args out of eval-config.nix 2015-03-12 23:42:57 +01:00
Shea Levy 0a0a29fd0b Add comments about the module system interface
Ideally the module system could be configured pretty much completely by
the contents of the modules themselves, so add comments about avoiding
complicating it further and possibly removing now-redundant
configurability from the existing interface.
2015-03-12 23:42:57 +01:00
Shea Levy e3eff53037 evalModules: Add internal option for the check argument 2015-03-12 23:42:57 +01:00
Shea Levy e4a06f35b1 nixos: Don't evaluate twice to get the value of config.nixpkgs 2015-03-12 23:42:57 +01:00
Sander van der Burg 8d29cb2b00 In a test VM it's impossible to connect to itself, so fix this 2015-02-19 21:13:49 +00:00
Mathijs Kwik 73f18fd42f nixos: allow adding extra modules through environment
This is useful for adding extra functionality or defaults to _every_
nixos evaluation.

My use case is overriding behaviour for all nixos tests, for example
setting packageOverrides to newer versions and changing some default
dependencies/settings.

By making this accessible through an environment variable, this can now
be fully accomplished externally. No more need to fork
nixos/nixpkgs (which becomes a maintenance burden), just use the channel
instead and plug in via this envvar.
2014-12-16 19:13:15 +01:00
Jaka Hudoklin a782b890d5 nixos/make-system-tarball: extra commands to be executed before archive 2014-12-11 23:17:27 +01:00
Luca Bruno 7885c2ed12 Revert "nixos: iso-image: use syslinux bootloader for USB booting support"
This reverts commit 455db168b3.
2014-11-08 14:33:27 +01:00
Bob van der Linden 455db168b3 nixos: iso-image: use syslinux bootloader for USB booting support
This changes the bootloader for iso generation from Grub to
syslinux. In addition this adds USB booting support, so that
"dd" can be used to burn the generated ISO to USB thumbdrives
instead of needing applications like UnetBootin.
2014-11-02 17:35:16 +01:00
Eelco Dolstra 51c349d0cc test-driver: Use netpbm instead of imagemagick
Imagemagick pulls in 100s of megabytes of dependencies.
2014-09-25 00:07:30 +02:00
Bjørn Forsman 8e8f95beda nixos/test-reports: use local copy of jquery
The current way test reports get jquery,

  src="//ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.min.js"

only works when getting reports over http:// or https://, not file://.
Change it so that it works for all protocols by using a local copy of
jquery.

This fixes the issue where locally created and browsed test reports
cannot be navigated properly; clicking the '+' symbol to expand
sub-sections doesn't work.
2014-09-07 15:47:59 +02:00
William A. Kennington III 2de0cf2549 nixos/utils: Don't remove first character if not a / 2014-09-03 18:40:05 -07:00
Michael Raskin d140851472 Merge pull request #3779 from offlinehacker/docker
full nixos inside docker
2014-09-02 01:04:05 +04:00
William A. Kennington III 9a697d775a Revert "Revert "Fix syntax error in nixos/lib/build-vms.nix, introduced by 86c0f8c""
This reverts commit 2f697bf693.
2014-08-31 09:46:26 -07:00
William A. Kennington III 3d037ebb94 Revert "Revert "Merge pull request #3182 from wkennington/master.ipv6""
This reverts commit ea8910652f.
2014-08-31 09:46:16 -07:00
Rob Vermaas ea8910652f Revert "Merge pull request #3182 from wkennington/master.ipv6"
This reverts commit b23fd65854, reversing
changes made to 43654cba2c.
2014-08-31 10:58:54 +02:00
Rob Vermaas 2f697bf693 Revert "Fix syntax error in nixos/lib/build-vms.nix, introduced by 86c0f8c"
This reverts commit 704e91bab0.
2014-08-31 10:58:50 +02:00
Rickard Nilsson 704e91bab0 Fix syntax error in nixos/lib/build-vms.nix, introduced by 86c0f8c 2014-08-30 19:54:37 +02:00
William A. Kennington III 86c0f8c549 Refactor nixos files relying on the old ipAddress / prefixLength / subnetMask attributes 2014-08-30 07:33:38 -07:00
Jaka Hudoklin d77150df30 nixos: make-system-tarball, add option for extra arguments for tar
Sometimes extra arguments when making tarball are required,
for example if making a container owner of files has to be changed
to root.
2014-08-25 03:29:07 +02:00
Peter Simons 2d326e5032 Merge remote-tracking branch 'origin/master' into staging.
Conflicts:
	pkgs/desktops/e18/enlightenment.nix
2014-08-04 16:51:47 +02:00
Eelco Dolstra b599deedb4 makeTest: Pass nodes attribute
This allows doing things like:

  nix-build nixos/tests/misc.nix -A nodes.machine.config.system.build.initialRamdisk
2014-07-30 14:30:25 +02:00
Eelco Dolstra 06fc1ec34d Merge remote-tracking branch 'origin/master' into staging
Conflicts:
	pkgs/servers/serfdom/default.nix
2014-07-01 11:25:41 +02:00
Eelco Dolstra 40f7b0f9df Another attempt to eradicate ensureDir
See c556a6ea46.
2014-06-30 14:56:10 +02:00
Florian Friesdorf 6a3e6164bd support for nixos test names 2014-06-28 16:05:03 +02:00
aszlig 622673495b
nixos/log2html: Remove schema from jQuery URLs.
This allows viewing test logs for example when using a Hydra running
with HTTPS.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-06-27 09:41:11 +02:00
Eelco Dolstra 48601269a9 test-driver: Set context to include machines created by createMachine 2014-05-21 15:55:50 +02:00
Eelco Dolstra 019e9d8a3d Fix simpleTest function 2014-04-24 15:46:10 +02:00
Eelco Dolstra bfda72c2f9 Fix waitForX 2014-04-17 17:07:03 +02:00