Commit graph

157 commits

Author SHA1 Message Date
xeji f8d047da50
nixos/release-combined.nix: make tests consistent (#40131)
... with release-small.nix
Tests for small channel should be a subset of tests for full channel.
2018-05-10 19:36:58 +02:00
Vladimír Čunát 88695c6e94
Revert "release-combined: don't include tests.chromium"
This reverts commit 1d06254998.
We now get a build and maybe have a workaround #39570; /cc #39476.
2018-04-27 02:42:12 +02:00
Matthew Justin Bauer 1d06254998
release-combined: don't include tests.chromium
The chromium package will frequently time out . Note that this is unrelated to the test but a side effect because the test depends on chromium.

https://hydra.nixos.org/job/nixos/trunk-combined/nixos.tests.chromium.x86_64-linux

See also #39476
2018-04-26 10:14:51 -05:00
Vladimír Čunát e59b91bc9a
nixos/release: fix evaluation if missing x86
... in supportedSystems
2018-03-07 16:06:30 +01:00
Robert Hensing 209f8b1acd nixos/release*.nix: Clean nixpkgs sources by default
Currently, when building NixOS from a git clone, Nix has to copy
the entire repo at >1GB into the store by default. That is not
necessary and causes a dumping large path message.
If you need the old behaviour for some reason, you will have to
specify it by passing the path to your repo explicitly as the
nixpkgs argument like this:

  --arg nixpkgs '{outPath = ./.; revCount = 56789; shortRev = "gfedcba"; }'
2018-02-09 18:32:26 +00:00
Tuomas Tynkkynen 93aa83deb4 nixos/release-combined.nix: Make 32-bit iso_minimal a channel blocker
In practice it already is since the channel update script barfs out if
the job fails:

9f96e1e25d/mirror-nixos-branch.pl (L132)
2018-02-07 20:00:19 +02:00
Tuomas Tynkkynen 8353ebe073 nixos/release.nix: Introduce callSubTestsOnTheseSystems
The existing callSubTests seems to already have special-cased code to
allow enabling subtests on a single specific system by looking at the
`system` attribute in the test arguments. Replace it with a new version
similar to the callTestOnTheseSystems because:

- It's consistent with the existing functions for creating
  system-specific tests (though admittedly, the callSubTests special
  case for `system` predates them)
- This approach allows limiting to multiple system types, the previous
  one inherently allows only one system type.
- This also fixes the problem that if you pass in e.g.
  supportedSystems = [ "aarch64-linux" ], you end up with a
  tests.chromium job that silently runs on x86_64-linux.
- Finally, this causes renames of the jobs like:
  tests.chromium -> tests.chromium.x86_64-linux to be consistent with
  the rest of the tests.
2018-01-23 12:24:33 +02:00
Tuomas Tynkkynen 4fd0a3a43d nixos/release.nix: More refactoring for multi-arch
Currently, even if you pass supportedSystems = [ "aarch64-linux" ] you
end up with e.g. `nixos.tests.docker` which actually silently runs on
x86_64-linux. Using the new callTestOnTheseSystems fixes that.

As a side-effect, this also causes a rename of
`nixos.tests.docker` -> `nixos.tests.docker.x86_64-linux`, which is IMHO
a good thing since it's makes them consistent with the rest of the
tests.
2018-01-19 10:35:49 +02:00
obadz edcf51a359
Merge pull request #31805 from gleber/make-switch-to-configuration-pure
nixos/system: make switch-to-configuration script pure
2017-11-19 19:03:52 +00:00
Franz Pletz f367bb4d6d
Merge pull request #31569 from gleber/add-mutable-users-test
nixos/tests: add a test for config.users.mutableUsers.
2017-11-19 19:54:28 +01:00
Gleb Peregud f8fb6a679e nixos/release: add nixos/tests/switch-test.nix to tested job 2017-11-19 19:50:08 +01:00
Gleb Peregud cea8302182 nixos/release: include mutableUsers test to release tests. 2017-11-19 19:20:50 +01:00
Sarah Brofeldt 58d07e2d6b php: Add nixos test to ensure php-pcre works in httpd 2017-11-12 08:05:27 +01:00
Franz Pletz 874a3c033c
nixos: limited support archs are not release-critical
Lately failing i686 tests like firefox have been blocking channel
releases. We're still building the tests for systems with limited
support but won't delay a channel release if they fail.
2017-10-14 22:09:35 +02:00
Joachim Fasting e05459584e
nixos/release-combined: remove basic kernel tests
Arguably, breaking linux-latest should not block a release.  Also, booting
the kernel + basic sanity checking is implicitly exercised by every other
vm test.
2017-09-16 12:45:30 +02:00
Joachim Fasting ffd56ba4f6
nixos/tests: move kernel-params test to misc 2017-09-16 12:45:28 +02:00
Joachim Fasting c85cf60c83
nixos/tests: move sysctl test to misc 2017-09-16 12:45:23 +02:00
Vladimír Čunát d6f9c0b353
nixos tests: restrict sysctl and plasma5 to x86_64-linux
- sysctl is new and never succeeded on i686-linux
  > cannot stat /proc/sys/net/core/bpf_jit_enable: No such file or directory
- testing plasma5 on i686 would defeat part of the reason why we ended
  supporting i686 (lots of stuff built on Hydra)
2017-09-11 07:35:19 +02:00
Franz Pletz 0d206c0b6c
nixos/release: xmonad isn't release-critical 2017-09-10 11:01:26 +02:00
Vladimír Čunát 94036cabda
nixos/release*: fixup evaluation of the tested job 2017-09-07 21:52:28 +02:00
Tim Steinbach 7faad2dce8 Merge pull request #29040 from NeQuissimus/tests_sys
More basic system tests
2017-09-06 15:04:10 -04:00
Tim Steinbach a54b2e3ba2 Merge pull request #29002 from NeQuissimus/docker_edge_module_test
docker: Package in module, tests
2017-09-06 15:03:56 -04:00
Tim Steinbach 0857e4c84d Merge pull request #28989 from NeQuissimus/tests_xmonad
Add XMonad test
2017-09-06 15:03:41 -04:00
Tim Steinbach b4ccef2163
tests: Add environment 2017-09-05 19:05:37 -04:00
Tim Steinbach 3e2975e892
tests: Add kernelParams 2017-09-05 19:04:43 -04:00
Tim Steinbach 04b0f3255f
tests: Add sysctl 2017-09-05 19:03:54 -04:00
Tim Steinbach 52341c03e8
tests: linux-copperhead is not release-blocking 2017-09-05 12:15:23 -04:00
Tim Steinbach 656ec9de0e
Add basic kernel tests 2017-09-05 10:38:07 -04:00
Tim Steinbach 380ed98bd7
docker: Add test for docker-edge, check for proper versions in tests 2017-09-04 19:02:44 -04:00
Tim Steinbach e153fa84a5
Add XMonad test 2017-09-04 14:03:20 -04:00
Tuomas Tynkkynen affcf92287 nixos/tests: Make the UEFI boot test release-critical
Seems to be working fine, based on the history:
https://hydra.nixos.org/job/nixos/trunk-combined/nixos.tests.installer.simpleUefiGummiboot.i686-linux/all
https://hydra.nixos.org/job/nixos/trunk-combined/nixos.tests.installer.simpleUefiGummiboot.x86_64-linux/all
2017-08-19 08:46:48 +02:00
Graham Christensen 7d0b001d4a nixos,nixpkgs: only build essentials on i686 2017-08-05 12:06:05 +02:00
Graham Christensen 5fd4ae36e2
nixos release-combined: only build zfsroot tests on x86_64-linux 2017-06-30 15:21:30 -04:00
Graham Christensen b0a4c2c33f
nixos: installer.nix test: test ZFS install use case 2017-06-28 19:45:20 -04:00
aszlig 7a99036cef
nixos/release-combined: Add keymap tests
We really want to break channel updates whenever we break something like
this, because this actually will hit machines out there and can be very
much annoying (we had broken keymaps a few times which is why I
introduced these tests in the first place).

Just to be sure I don't break channel updates with this commit, I ran
all of the keymap tests and they all succeeded.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2017-06-22 03:33:35 +02:00
aszlig dc87b5587e
nixos: Add a VM test for the SLiM display manager
This is currently our default display manager, so I'm adding this to the
"tested" job as well to ensure we don't ship broken revisions where X is
most likely not working.

The test uses a custom SLiM theme that's specifically tailored for good
OCR results (mainly white background and black fonts without anything
else), because our default NixOS theme has a very small contrast between
background and fonts in some places.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2017-04-23 19:25:27 +02:00
Thomas Tuegel 352e335f3d
nixos/sddm: test automatic and manual login 2017-03-24 10:39:13 -05:00
Thomas Tuegel 5dbbb60d4f
nixos/plasma5: re-enable test
Fixes #24240.

I do not know the cause of the test failures, but the tests pass now.
2017-03-24 10:38:59 -05:00
Vladimír Čunát c71f8f6ead
release-combined: disable plasma tests temporarily
... to get a channel bump after a long time; see #24240.
2017-03-23 16:10:04 +01:00
Tuomas Tynkkynen 74c4e30842 Revert "nixos: build for aarch64-linux"
This reverts commit b806e25d65.

This seems to push Hydra's memory usage out of the roof fail nixos
evaluating with:

Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS

Let's revert this for now. It's not a big deal at all since the
nixpkgs-unstable jobset is still building the packages.
2017-03-20 10:30:51 +02:00
Graham Christensen b806e25d65
nixos: build for aarch64-linux 2017-03-13 17:34:33 -04:00
Thomas Tuegel 044c7d091b Merge pull request #23388 from ttuegel/nixos-plasma5
NixOS: Plasma 5 tests and warnings
2017-03-03 09:50:08 -06:00
Thomas Tuegel 0da421ce17
nixos/tests: fix Plasma 5 test 2017-03-02 07:01:42 -06:00
Vladimír Čunát 45344fdf19
tested job: drop the hibernate test on i686 for now
/cc #23107.
2017-03-02 07:28:47 +01:00
Graham Christensen b1a05a0865
nixos: drop references to kde4
Excluding modules/programs/environment.nix for PATHand QT_PLUGIN_PATH to allow the programs to continue running.
2017-02-11 14:01:13 -05:00
Thomas Tuegel e5a79b0eae
nixos/tests/kde5: rename from sddm-kde5 and run by default
(cherry picked from commit 701f02a6eed902835e3d2a27838723cd9a7bb66a)
2016-09-09 10:12:38 -05:00
Eelco Dolstra d8625f6d25 Make the NFSv4 tests release-critical
We can probably drop NFSv3...
2016-09-07 14:15:57 +02:00
obadz ab08440f9c Revert "Temporarily disable chromium test"
This reverts commit 9e6eec201b.

Verified locally that cd063d7 fixes the problem.
cc @edolstra @aszlig
2016-08-23 14:16:58 +01:00
Eelco Dolstra 9e6eec201b Temporarily disable chromium test
It's hanging and blocking the unstable channel.

http://hydra.nixos.org/build/38984676

@aszlig
2016-08-23 13:01:34 +02:00
Nikolay Amiantov 74a5c99904 nixos tests: add hibernation test to blockers 2016-08-21 17:34:00 +03:00
Eelco Dolstra a78ecb0d33 Remove nixos.tests.boot.biosUsb.* as release blockers
These have a high random failure rate, blocking channel updates. Issue
2016-07-21 11:44:55 +02:00
obadz de80d0544c ecryptfs: add test to release-combined.nix 2016-07-14 11:52:50 +02:00
Domen Kožar 467cd6f3a4 Make i3wm test a release blocker
Catch issues like
331fa2feff
2016-05-26 10:47:15 +01:00
aszlig eb6e366446
nixos/release-combined: Add boot-stage1 test
We don't want to push out a channel update whenever this test fails,
because that might have unexpected and confused side effects and it
*really* means that stage 1 of our boot up is broken.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-05-06 16:56:54 +02:00
Vladimír Čunát 25960a52c3 tested job: fix evaluation of chromium tests
It's a bit inconsistent now, but I want mainly unblock the channel.
/cc maintainer @aszlig.
2016-05-06 10:56:17 +02:00
Tuomas Tynkkynen df310048e5 release-combined.nix: More Chromium test evaluation fixes
Follow-up to f35e9386bd.
2016-05-05 06:23:12 +03:00
Domen Kožar db179647d6 make all boot tests release critical #14902
(cherry picked from commit 9d1662c6c90059b63fad01b3f55a0df94af7cde4)
Signed-off-by: Domen Kožar <domen@dev.si>
2016-04-23 13:12:57 +01:00
obadz d946a28c55 Revert "NixOS release: Don't depend on chromium"
This reverts commit 3135af2511.

Closes #14732
2016-04-19 18:20:49 +01:00
Eelco Dolstra 3135af2511 NixOS release: Don't depend on chromium
This is failing because it exceeds the hydra-queue-runner size limit.

http://hydra.nixos.org/build/33303819
2016-03-15 16:06:09 +01:00
Eelco Dolstra 55e71f45cc Fix NixOS tested job 2016-03-15 15:43:57 +01:00
aszlig 1f89aa6c7f
nixos/tests: Remove tests.boot.boot* prefixes
As @bobvanderlinden suggests in #13585:

"Looks like that cleans things up quite a bit! Just one aesthetics note,
the boot tests could now be renamed from boot.bootBiosCdrom to
boot.biosCdrom in nixos/tests/boot.nix:L33.

That makes them more consistent with the other tests."

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 19:02:36 +01:00
aszlig e45c211b50
nixos/tests/chromium: Split up into subtests
This makes it easier to test just a specific channel rather than to
force testing all builds down the users/testers throat. Especially this
makes it easier to test NixOS channel upgrades only against the Chromium
stable channel instead of just removing the beta/dev channels from the
tests entirely (as done in 69ec09f38a).

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2016-03-01 01:04:51 +01:00
aszlig 7eea91fbc0
nixos/release: Use callSubTests for subtests
Should clean up a lot of these redundant lines for various sub-tests.

Note that the tests.boot* are now called tests.boot.boot*, but otherwise
all the test attribute names should stay the same.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Cc: @edolstra
Cc: @wkennington
Cc: @bobvanderlinden
2016-03-01 01:04:42 +01:00
aszlig 4c61faa009
nixos/tests/networking: Add a test for loopback
It serves as a regression test, because right now if you enable
networking.useNetworkd the default loopback interface doesn't get
assigned any IP addresses.

To be sure, I have bisected this and it has been introduced with the
update to systemd 228 in 1da87d4.

Only the "scripted" networking tests have to succeed in order to trigger
a channel update of nixos-unstable, so I'm leaving this test as broken
and we have to figure out next what's the *exact* reason for the
breakage.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-12-18 07:09:42 +01:00
Thomas Tuegel 34f2addf11 nixos: add sddm test to combined constituents 2015-12-11 07:08:33 -06:00
Eelco Dolstra a0d7a458b1 Don't block releases on nixos.tests.gnome3.i686-linux
http://hydra.nixos.org/build/26702440

Issue #10353.
2015-10-14 13:55:43 +02:00
Eelco Dolstra d234c64032 Disable lightdm test
It's hanging in the OCR step for hours.

@aszlig

http://hydra.nixos.org/build/24268601
2015-08-17 10:48:04 +02:00
Eelco Dolstra 606d34e45a Revert "Revert "Drop tests.installer.grub1 as release critical""
This reverts commit cd28c5517c. Still broken:

http://hydra.nixos.org/build/23928152
2015-07-22 15:20:40 +02:00
Eelco Dolstra 4c0e44c34c Make OpenJDK release-critical
Currently there are no tests that depend on the JDK. Since we don't
want a release with a broken JDK, make it an explicit dependency of
the "tested" jobs.
2015-07-14 16:03:29 +02:00
William A. Kennington III e861889e99 nixos/tests/installer: Add a testcase for fat32 /boot 2015-07-08 12:05:31 -07:00
Eelco Dolstra cd28c5517c Revert "Drop tests.installer.grub1 as release critical"
This reverts commit 1907ca8a2a. See

1907ca8a2a (commitcomment-9719007)
2015-06-12 01:37:45 +02:00
Eelco Dolstra 8cf5d48d03 Make the boot test release-critical 2015-06-10 15:28:56 +02:00
aszlig f0d0617827
nixos: Add lightdm test to the "tested" job.
This is to prevent bugs such as #7902 from hitting the channels again.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-28 10:27:29 +02:00
aszlig 7808efe5b7
nixos/release: Add luksroot test to "tested" job.
We want to avoid getting broken LUKS systems into the latest channel, so
let's ensure that the channel update won't happen if LUKS support is
broken again.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-21 13:21:41 +02:00
aszlig 1567f56a5a
nixos: Re-add swraid installer tests to releases.
It has been removed by 71a197bc6e.

I'm reintroducing the test mainly because it actually *is* useful,
because right now, machines with mdraid will not boot. In order to
prevent such things from happening in the future, we should *not* remove
this VM test again.

If it really goes back to failing randomly, we should really try to fix
it instead of removing it again.

Of course it fails right now because of the mdraid bump in 7719f7f.
Also, if you want to have additional info about the reasons, have a look
at the commit message of 666cf992f0.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-04-01 08:50:38 +02:00
Eelco Dolstra 0461f35894 Rename scrubDrv -> hydraJob and make it more effective
It now strictly evaluates all remaining attributes, preventing
unevaluated thunks that cannot be garbage-collected. It's also applied
to all jobs in Nixpkgs' release.nix.

This reduces hydra-eval-jobs' memory consumption on the 14.12
release-combined jobset from 5.1 GB to 2.0 GB.
2015-03-20 19:46:18 +01:00
Eelco Dolstra 1907ca8a2a Drop tests.installer.grub1 as release critical
This test sometimes fails with

  Kernel panic - not syncing: assertion "i && sym_get_cam_status(cp->cmd) == DID_SOFT_ERROR" failed: file "/tmp/nix-build-linux-3.14.32.drv-0/linux-3.14.32/drivers/scsi/sym53c8xx_2/sym_hipd.c", line 3399

after "sd 2:0:0:0: ABORT operation timed-out."

Since we don't care all that much about GRUB 1 anymore, don't make the
release depend on it.

http://hydra.nixos.org/build/19563197
2015-02-12 16:20:00 +01:00
Domen Kožar 3d74b38101 add chromium test to the tested job
(cherry picked from commit fa4e45f20cc7049ed99fd81d96fc1f67b2a870f6)
Signed-off-by: Domen Kožar <domen@dev.si>
2014-12-14 13:47:19 +01:00
William A. Kennington III a403379b88 nixos/tests: Add networking tests for basic functionality 2014-11-29 19:53:37 -08:00
Shea Levy 92837f2dbf Ditch efi installer tests
They've been broken forever due to weirdness in qemu tests, and they don't add much over installer tests
2014-10-07 12:20:56 -04:00
Eelco Dolstra 1cd727180e Add all default NixOS packages to the channel
The job ‘dummy’ depends on the default contents of
‘environment.systemPackages’, thus ensuring that those packages all
end up in the channel.
2014-09-25 00:25:54 +02:00
William A. Kennington III 8250059a9f nixos/tests/nat: Add tests for standalone and firewall based nat 2014-09-18 14:39:46 -07:00
Luca Bruno e5b7b97a67 nixos tested: only build on supportedSystems 2014-09-03 12:51:10 +02:00
William A. Kennington III 3944239362 nixos/tests-installer: Add a test for btrfs default volumes and bind mounts 2014-09-02 09:16:13 -07:00
William A. Kennington III 36614ff3e2 Revert "Revert "Merge pull request #2449 from wkennington/master.grub""
This reverts commit 94205f5f21.

Conflicts:
	nixos/modules/system/boot/loader/grub/install-grub.pl
2014-09-02 09:16:13 -07:00
Michael Raskin 94205f5f21 Revert "Merge pull request #2449 from wkennington/master.grub"
This reverts commit 469f22d717, reversing
changes made to 0078bc5d8f.

Conflicts:
	nixos/modules/installer/tools/nixos-generate-config.pl
	nixos/modules/system/boot/loader/grub/install-grub.pl
	nixos/release.nix
	nixos/tests/installer.nix

I tried to keep apparently-safe code in conflicts.
2014-08-31 12:58:37 +04:00
William A. Kennington III babcd70c36 nixos/release-combined: Add required installer tests 2014-08-28 13:35:37 -07:00
Eelco Dolstra d9e6c07c5b Remove the containers test from the release-critical builds
Too many random failures...
2014-08-05 10:25:42 +02:00
Eelco Dolstra 0bc7ec6e28 Remove obsolete udisks test
Closes #2716.
2014-05-22 12:11:53 +02:00
Eelco Dolstra 3b616e378a release.nix: Drop officialRelease flag 2014-05-01 15:05:14 +02:00
Eelco Dolstra a96f4920d5 Don't make the EFI tests release-critical
They're failing on i686: http://hydra.nixos.org/build/10712961
2014-04-30 16:48:20 +02:00
Eelco Dolstra e9be441b62 Merge the EFI test into tests/installer.nix 2014-04-30 11:26:39 +02:00
William A. Kennington III 936481a12e nixos: Add support for changing supported systems
release.nix and release-combined.nix current hardcode the systems which
they are built for. This change introduces an argument to the
expressions called supportedSystems, which allows the builder to choose
which architectures he wants to build. By default, this uses the same
linux x86_64 and i686 architectures.
2014-04-29 10:53:36 +02:00
Domen Kožar cd31cff9f4 Make gnome3 test as release critical 2014-04-26 13:31:07 +02:00
Eelco Dolstra 537c034e8f Make some tests release-critical 2014-04-25 17:08:58 +02:00
Eelco Dolstra 894e2dfb25 Add a test for udisks2 2014-04-19 14:37:05 +02:00
Eelco Dolstra 2f51ca9609 Add a regression test for udisks 2014-04-07 13:22:12 +02:00
Eelco Dolstra 1f6bfa19ad Gnome 3 should not be a release blocker 2014-04-07 12:24:17 +02:00
Shea Levy 1425fa5b3b Disable efi tests again
OVMF sucks
2014-03-01 09:51:28 -05:00
Shea Levy 8e97e38677 Reenable efi tests 2014-02-12 07:13:37 -05:00