Commit graph

29 commits

Author SHA1 Message Date
Joerg Thalheim 00344162a7 Revert "nixos/test-instrumentation: softcode DISPLAY"
This reverts commit 4531ec5955.

This broke waitForX: b11ac5d29d (commitcomment-25486347)

cc @Lassulus
2017-11-08 17:04:39 +00:00
lassulus 4531ec5955 nixos/test-instrumentation: softcode DISPLAY 2017-11-07 23:01:59 +01:00
Bas van Dijk 5b8ff5ed49 graphite: 0.9.15 -> 1.0.2
Fixes: #29961

Also added the option:

  services.graphite.web.extraConfig

for configuring graphite_web.
2017-10-08 03:03:22 +02:00
Graham Christensen 3f40fcabbf
nixos tests: waitForWindow: output a list of windows we see prior to the final check
machine: must succeed: xwininfo -root -tree | sed 's/.*0x[0-9a-f]* \"\([^\"]*\)\".*/\1/; t; d'
    machine: exit status 0
    machine: Last chance to match /(?^:dfiirst configuration)/ on the the window list, which currently contains:
    machine: [i3 con] container around 0xf8a5f0, i3: first configuration, [i3 con] floatingcon around 0xf8c260, [i3 con] container around 0xf8a380, i3bar for output Virtual-1, [i3 con] bottom dockarea Virtual-1, [i3 con] workspace 1, [i3 con] content Virtual-1, [i3 con] top dockarea Virtual-1, [i3 con] output Virtual-1, [i3 con] workspace __i3_scratch, [i3 con] content __i3, [i3 con] pseudo-output __i3, i3
2017-06-20 21:16:35 -04:00
Graham Christensen 1b833015b7
nixos tests: waitForText: output the detected screen content prior to the last attempt
machine: Last chance to match /(?^:BALICE)/ on the screen, which currently contains:
    machine: performing optical character recognition
    machine: sending monitor command: screendump /tmp/nix-build-vm-test-run-sddm.drv-0/ocrin.ppm
    machine: Session Layout

    O O

    0 1 : 0 9

    Wednesday, June 21, 2017

    |_ I

    Select your user and enter password
2017-06-20 21:10:34 -04:00
Graham Christensen 348785eec0
nixos tests: waitUntilTTYMatches: Log TTY contents on last try
If the test has not passed yet, on the last attempt it now outputs:

    machine: Last chance to match /logine: / on TTY2, which currently contains:
    machine: running command: fold -w$(stty -F /dev/tty2 size | awk '{print $2}') /dev/vcs2
    machine: exit status 0
    machine:

    <<< Welcome to NixOS 17.09.git.a804ef4 (x86_64) - tty2 >>>

    machine login:

to help debug the problem. Notice the "logine" typo in my check.
2017-06-20 20:57:39 -04:00
Graham Christensen 56435c1404
nixos tests: retry: Count down to 0, and pass remaining attempts to the sub
Allows test functions to output diagnostic information on failure.
2017-06-20 20:54:33 -04:00
Volth e5fbeb2f11 test-driver: sendKeys() support for capital letters 2017-05-19 13:56:24 +00:00
aszlig a443bdc0a6
nixos/testing: Improve quality of OCR
First of all, we're now using ImageMagick to improve the screenshot so
that Tesseract has an esier time to recognize the text. The resulting
image of this post-processing is a scaled up black-and-white version
with the backgrounds almost entirely removed and the text edges a bit
blurred, so the screen shots now more or less resemble an image from a
scanner rather. This is what Tesseract is trained for by default.

As mentioned in the previous commit we now also use Tesseract 4, which
further improves the quality of text recognition.

I've spent countless hours just to test different postprocessing
variants and testing what works best for our tests and this is the one
that worked best so far. It's certainly not perfect and I'd like to
avoid the scaling step but we're way better off than before.

In addition to this, the OCR process is now done without an intermediate
file, solely using pipes.

I've tested this using the following VM tests which have OCR enabled:

 * nixos/tests/chromium.nix -A stable
 * nixos/tests/emacs-daemon.nix
 * nixos/tests/installer.nix -A luksroot
 * nixos/tests/lightdm.nix
 * nixos/tests/plasma5.nix
 * nixos/tests/sddm.nix

All of the tests still succeed and comparing some of the recognition
results to the earlier results it now also detects a lot more text than
before this commit.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2017-04-11 03:21:53 +02:00
aszlig a5e9668c5c
nixos/test: Fix escaping for copyFileFromHost
A long-time issue and one of the reasons I've never used that function
before. So let's remove that todo-comment and escape the contents
properly.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Cc: @edolstra
2017-02-17 19:45:49 +01:00
Graham Christensen 59e77daf5b
nixos tests: make getTTYText smart about tty size 2017-02-12 18:27:06 -05:00
volth 50ec3fe1ac
test-driver: support punctuation in sendChars 2017-01-08 09:17:05 -05:00
Graham Christensen cb74fd75d7
login test: Create and use direct reads of the TTY contents. 2016-11-30 00:17:18 -05:00
Eelco Dolstra ad29b72686 test-driver: Fix "unit X is inactive and there are no pending jobs"
This was causing many random test failures.
2016-05-24 16:33:14 +02:00
Eelco Dolstra 9153d8ed64 Fix X11 tests broken by the removal of -ac
Probably missed a few. Also adding xauth to the system path (it was
already in the closure).
2016-04-12 19:13:47 +02:00
Eelco Dolstra 146c727924 waitForUnit: Fail if the unit reaches inactive state with no pending jobs
This prevents waitForUnit for looping for many minutes before giving
up.
2015-12-30 15:17:52 +01:00
Dan Peebles b6c589b2da Simple EC2 user-data VM test 2015-06-11 22:54:04 -04:00
Eelco Dolstra 0f35f9bb69 Drop obsolete boot=on parameter 2015-06-10 15:28:55 +02:00
aszlig 2475a6a239
nixos/test-driver: Fix bashism in Machine.pm.
Thanks to @bjornfor for the note.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-22 08:10:17 +02:00
aszlig 8be00dc71d
nixos/test-driver: Make tesseract OCR optional.
By default this is now enabled, and it has to be explicitely enabled
using "enableOCR = true". If it is set to false, any usage of
getScreenText or waitForText will fail with an error suggesting to pass
enableOCR.

This should get rid of the rather large dependency on tesseract which
we don't need for most tests.

Note, that I'm using system("type -P") here to check whether tesseract
is in PATH. I know it's a bashism but we already have other bashisms
within the test scripts and we also run it with bash, so IMHO it's not a
problem here.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-22 07:21:58 +02:00
aszlig 8bd025a75e
nixos/test-driver: Add new waitForText function.
As promised in the previous commit, this can be used similarly to
$machine->waitForWindow, where you supply a regular expression and it's
retrying OCR until the regexp matches.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-21 19:55:21 +02:00
aszlig 235c2228ca
nixos/test-driver: Add new getScreenText function.
Basically, this creates a screenshot and throws tesseract at it to
recognize the characters from the screenshot. In order to produce a
result that is well enough, we're using lanczos scaling and scale the
image up to 400% of its original size.

This provides the base functionality for a new Machine method which will
be called waitForText. I originally had that idea long ago when writing
the VM tests for VirtualBox and Chromium, but thought it would be
disproportionate to the case.

The downside however is that VM tests now depend on tesseract, but given
the average runtime of our tests it really shouldn't have a too big
impact and it's only a runtime dependency after all.

Another issue is that the OCR process takes quite some time to finish,
but IMHO it's better (as in more deterministic) than to rely on sleep().

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-05-21 19:55:21 +02:00
Bob van der Linden c57a912016 nixos: test: add tests for booting installation iso in various ways 2015-03-24 22:09:07 +01:00
Eelco Dolstra 51c349d0cc test-driver: Use netpbm instead of imagemagick
Imagemagick pulls in 100s of megabytes of dependencies.
2014-09-25 00:07:30 +02:00
Eelco Dolstra bfda72c2f9 Fix waitForX 2014-04-17 17:07:03 +02:00
Eelco Dolstra f0a9703f77 journalctl no longer parses the flag "-bu" 2014-04-17 10:56:10 +02:00
Eelco Dolstra de57c0eb66 Shut up Perl warning 2014-03-18 14:10:31 +01:00
Shea Levy 51de280c0a nixos X tests: wait for logind to link a session to the server
There seems to be some race causing failures if an X command gets in before slim starts the session

Signed-off-by: Shea Levy <shea@shealevy.com>
2014-01-22 14:23:56 -05:00
Eelco Dolstra 5c1f8cbc70 Move all of NixOS to nixos/ in preparation of the repository merge 2013-10-10 13:28:20 +02:00
Renamed from lib/test-driver/Machine.pm (Browse further)