Commit graph

39929 commits

Author SHA1 Message Date
aszlig aa65a7057f
vm/windows: Properly escape shell command.
Security-wise it's not a big issue because we're still sandboxed, but I
really don't want to write something like \\\\\\\\192.168.0.2\\\\share
in order to set up network shares.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:51:57 +01:00
aszlig cfa859d792
vm/windows: Don't init /nix/store on install.
We're going to do this during the suspendedVM phase, so we're able to
more easily change the shares without reinstalling the whole VM.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:51:57 +01:00
aszlig 3e91192f07
vm/windows: Wait for VDE switch to startup.
This could possibly cause flapping whenever qemu is too fast in starting
up. As we are running with the shell's -e flag, the socat check also
ensures that the VDE switch is properly started and causes the whole
build to fail, should it not start up within 20 seconds.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:51:56 +01:00
aszlig c731467e2c
vm/windows: Split install into several stages.
These stages are in particular:

 * Install of the bare Windows VM with Cygwin and shut down.
 * Boot up the same VM again without the installation media and dump the
   VMs memory to state.gz.
 * Resume from state.gz and build whatever we want to build.

Every single stage involves a new "controller", which is more like an
abstraction on the Nix side that constructs the madness described in
276b72fb93.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:51:56 +01:00
aszlig 5105e7f0bf
vm/windows: Update sha256 of Cygwin's setup.ini.
This is kinda stupid to do every little time the file is automatically
regenerated upstream. But let's see how often that happens and whether
it will become a major annoyance or not, and if yes, we might be forced
to include it in our source tree.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:50:55 +01:00
aszlig 9b1862ca1f
vm/windows: Move creating SSH key into install/.
This SSH key is specifically only for accessing the installed Cygwin
within the Windows VM, so we only need to expose the private key. Yes,
you heard right, the private key. It's not security-relevant because the
machine is completely read-only, only exposed to the filesystem and
networking is not available.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:50:54 +01:00
aszlig 4e21215d52
vm/windows: Move the installer into install/.
At least the largest portion of the installer, because in the end we
don't want the installer to *actually* save the state but only prepare
the base image.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:50:54 +01:00
aszlig 276b72fb93
vm: Introduce new Windows VM installer for Cygwin.
After quite a lot of fighting against Windows and its various
limitations, this new is the base architecture for installing and
accessing the Windows VM and thus the Cygwin environment inside it:

                .------------.
          .---> | vde_switch |
          |     `-[#]----[#]-'
          |        |      |
         ,'       .'      `---.___
       ,'    192.168.0.1          `.
       |          |            192.168.0.2
      ,'    _____[#]____           |
    ,'     |            |   ______[#]______
    |      | Windows VM |  |    .--'       |
    |      |____________|  |    |          |
    |             |  /|\   |  .-|          |
    | .---------. |   |    |  | |          |
  .-|-| manager |-'   |    |  | |          |
  | | `---------'     |    |  | |          |
  | |                 |    |  | |          |
  | | .-------------. |    | Samba         |
  | | | BOOTSTRAP   | |    |  | |          |
  | | |-------------| |    |  | |   .------|
  | `-| spawn VMs   |-+--> |  | `---| xchg | <-------.
  |   |-------------|      |  | .---^------|         |
  |   | install     |---.  |  `-| nixstore | <----.  |
  |   |-------------|   |  |    `----------|      |  |
  |---| suspend VM  |   |  |               |      |  |
  |   `------.------'   |  | Controller VM |      |  |
  |          |          |  |_______________|      |  |
  |       .--'          |         /|\            VirtIO
  |       |           __|__________:____________  |  |
  |      \|/         |  |          `.           | |  |
  | .------------.   |  |           :           | |  |
  | | REAL BUILD |   |  |   .-------^--------.  | |  |
  | |------------|   |  `-> | serial console |  | |  |
  `-| revive VM  |   |      `----------------'  | |  |
    |------------|   |------------.             | |  |
    | build      |-->| /nix/store >>>-----------|-'  |
    |------------|   |------------|             |    |
    | collect    |<--| xchg       >>>-----------|----'
    `-----.------'   |------------'             |
          |          |                          |
         \|/         |    |  |  __   ___  |     |
                     |    |--| |  | (__  -|-    |
    F I N I S H E D  |    |  | |__| ___)  |     |
                     |__________________________|

This might look a bit overwhelming, but let me try to explain:

We're starting at the base derivation ("BOOTSTRAP" above), where we
actually install the Cygwin envirenment. Over there we basically fire up
a vde_switch process and two virtual machines: One is the Windows
machine, the other is a NixOS machine, which serves as some kind of
proxy between the host and the Windows machine.

The reason we're doing this, is because we don't have a lot of options
for sharing files between a stock Windows machine and the host. In
earlier experiments, I've tried to communicate with the Windows guest by
using pipes and OpenSSH, but obviously this wasn't a big speed rush (or
to say it bluntly: It was fucking slow).

Using TCP/IP directly for accessing the guest would have been another
option, but it could lead to possible errors when the port or a range of
ports are in use at the Host system. Also, we would need to punch a hole
into the sandbox of the Nix builder (as it doesn't allow networking),
which in turn will possibly undermine deterministic builds/runs (well,
at least as deterministic as it can be, we're running Windows,
remember?).

So, let's continue: The responsibility of the NixOS (controller) VM is
to just wait until an SSH port becomes available on the Windows VM,
whereas the Windows VM itself is installed using an unattended
installation file provided via a virtual floppy image.

With the installation of the basic Windows OS, we directly install
Cygwin and start up an OpenSSH service.

At this point the bootstrapping is almost finished and as soon as the
port is available, the controller VM sets up Samba shares and makes it
available as drive letters within Windows and as bind mounts (for
example /nix/store) within Cygwin.

Finally we're making a snapshot of the memory of the Windows VM in order
to revive it within a few seconds when we want to build something.

Now, the build process itself is fairly straightforward: Revive VM and
build based on existing store derivations and collect the result _and_
the exit code from the xchg share/directory.

Conclusion: This architecture may sound a bit complicated, but we're
trying to achieve deterministic and reproducable builds and/or test
runs.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:50:53 +01:00
Shea Levy c11f32acdf scotty: jailbreak
Needed when overriding aeson to the new version, which in turn is needed
if using scotty on a project that also uses lens 4.0.
2014-02-25 11:14:07 -05:00
Shea Levy 9ef06bd387 Add libjenkins haskell package 2014-02-25 11:09:26 -05:00
Shea Levy 1f42fa7cc0 Add hspec-expectations-lens haskell package 2014-02-25 10:49:15 -05:00
Shea Levy 793328e1ee Mediawiki: Add some needed rewrites 2014-02-25 09:13:40 -05:00
Shea Levy 0d4a9e3aa6 Allow httpd subservices to set the document root
Only the main service OR one of the subservices can set the document
root. This is used by mediawiki when it is hosted at the root of the
vhost.
2014-02-25 07:44:45 -05:00
Peter Simons ec09c7d34e haskell-cpphs: update to version 1.18.2 2014-02-25 00:25:28 +01:00
Peter Simons f49ce8adc7 haskell-threepenny-gui: update to version 0.4.1.0 2014-02-25 00:25:28 +01:00
Peter Simons 38d51bdf3b haskell-multirec: update to version 0.7.4 2014-02-25 00:25:28 +01:00
Peter Simons e4f4e42896 haskell-multiarg: update to version 0.24.0.4 2014-02-25 00:25:28 +01:00
Peter Simons 7fc5eea0f3 haskell-x509: update to version 1.4.9 2014-02-25 00:25:28 +01:00
Peter Simons 3b3afdce57 haskell-shakespeare-text: update to version 1.0.2 2014-02-25 00:25:28 +01:00
Peter Simons 3912c15aa8 haskell-io-streams: update to version 1.1.4.2 2014-02-25 00:25:28 +01:00
Peter Simons efcaa15444 haskell-hoogle: update to version 4.2.29 2014-02-25 00:25:28 +01:00
Peter Simons 75a2d1b2f0 haskell-crypto-pubkey-types: update to version 0.4.2.1 2014-02-25 00:25:28 +01:00
Peter Simons eedfd78218 haskell-asn1-encoding: update to version 0.8.1.3 2014-02-25 00:25:28 +01:00
Peter Simons c20be9610c haskell-fb: re-generate package with cabal2nix 2014-02-25 00:25:27 +01:00
Peter Simons bac47e8168 haskell-lens: update to version 4.0.4 2014-02-25 00:25:27 +01:00
viric 0c11d4aa1f Merge pull request #1448 from pSub/dwm-remove-patch
Removed confnotify-6.0.patch.
2014-02-24 22:48:48 +01:00
Peter Simons 2955a48107 python-packages.nix: strip trailing whitespace 2014-02-24 22:04:40 +01:00
Peter Simons 199afbe062 python-logilab-common: update to version 0.61.0 2014-02-24 22:04:40 +01:00
Peter Simons ddbeda0142 python-logilab-astng: update to version 0.24.3 2014-02-24 22:04:40 +01:00
Peter Simons 02a475bf75 pylint: update from 0.26.0 to 0.28.0 2014-02-24 22:04:40 +01:00
Peter Simons dfff43fe59 nspr: update from 4.10.2 to 4.10.3 2014-02-24 22:04:40 +01:00
Peter Simons 65aaf865a7 nasm: update from 2.10 to 2.11.01 2014-02-24 22:04:40 +01:00
Peter Simons 684e06c2e6 mtools: update from 4.0.17 to 4.0.18 2014-02-24 22:04:40 +01:00
Peter Simons 20a150a1df memtest86: update from 4.1.0 to 4.3.3 2014-02-24 22:04:40 +01:00
Peter Simons ce365835d7 links2: update from 2.5 to 2.8 2014-02-24 22:04:40 +01:00
Peter Simons 17f7c664f6 libzip: update from 0.11.1 to 0.11.2 2014-02-24 22:04:40 +01:00
Peter Simons 2dc6bcbc40 libselinux: update from 2.2.1 to 2.2.2 2014-02-24 22:04:40 +01:00
Peter Simons 9a409ee3fd libpcap: update from 1.4.0 to 1.5.3 2014-02-24 22:04:40 +01:00
Peter Simons 77f03b973d libmilter: update from 8.14.4 to 8.14.8 2014-02-24 22:04:40 +01:00
Peter Simons 9f714fbafa iw: update from 3.11 to 3.14 2014-02-24 22:04:40 +01:00
Peter Simons 4d7ee5246c isocodes: update from 3.49 to 3.51 2014-02-24 22:04:40 +01:00
Peter Simons 01aaf816c9 acpid: update from 2.0.20 to 2.0.21 2014-02-24 22:04:40 +01:00
Shea Levy a0a9430869 Add mariadb 2014-02-24 15:34:57 -05:00
Peter Simons b05d3e77f1 spamassassin: update from 3.3.2 to 3.4.0 2014-02-24 21:17:55 +01:00
Peter Simons df4919f070 duplicity: update from 0.6.22 to 0.6.23 2014-02-24 21:17:55 +01:00
Peter Simons abca8f64f2 lxc: update from 1.0.0.beta4 to 1.0.0 2014-02-24 21:17:55 +01:00
Lluís Batlle i Rossell 9c36f01edd Updating toxic and toxcore to newer code. 2014-02-24 19:18:15 +01:00
Rickard Nilsson d5211b0e0e Make initialRootPassword overrideable in all virtualisation modules, not just virtualbox. 2014-02-24 18:05:26 +01:00
Peter Simons 51c9dbc9f3 python-wrapper: add 'ignoreCollisions' parameter (which default to 'false') 2014-02-24 12:41:12 +01:00
Rob Vermaas aa23a99436 Add stackdriver-statsd-backend/statsd-librato-backend node packages and update existing node packages. 2014-02-24 12:02:21 +01:00