Commit graph

1004 commits

Author SHA1 Message Date
aszlig d16dae8d32
vm/windows: Use &&, not set -e in runAndSuspend.
We now have proper quoting, so we no longer need the workaround using
"set -e".

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:51:59 +01:00
aszlig 084238c6ec
vm/windows: Fix perms of the Samba shares.
We now map all guest accounts to the root user, because in the end the
permissions of the current user boil down to the build user of the Nix
builder of the host. That way it's not possible to gain more permissions
at all and just makes the VM communication a lot easier.

However, setting "writable" to yes instead of "read only" to no doesn't
change anything here, I just found it to be clearer.

Also, we now no longer need to have a "nobody" user.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:51:59 +01:00
aszlig 5241bb1d5e
vm/windows: Avoid shadowing the Nix store.
This is done by putting the non-initrd /nix/store into a subdirectory,
which we then chroot to and pass along the SSH command.

Also, we now collect the exit code after the chroot command and power
off the VM thereafter, because the store is no longer shadowed and we
have still access to the busybox inside the initrd.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:51:58 +01:00
aszlig a1d7974acd
vm/windows: Remove Samba from installMode.
This should trim down possible dependencies on the base installation and
hereby reduce the need for reinstallation of the damn VM to only changes
that affect the Windows installation and the base Cygwin + OpenSSH
setup.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:51:58 +01:00
aszlig fedf13e6cf
vm/windows: Implement and use "xchg" share.
This now finally introduces our xchg share and also uses it for
exchanging state while suspending a VM. However, accessing the _real_
Nix store still isn't possible because we're shadowing the directory in
the initrd.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:51:58 +01:00
aszlig 0ce1fd07fe
vm/windows: Reintroduce setting up /nix/store.
Now we're doing this at the point where we're saving the VM state.
Unfortunately it's not quite right, because the controller VM is shut
down _before_ we're saving the state, so the share gets disconnected
despite autodisconnect being deactivated during setup.

We can get around this issue by finally introducing the xchg share,
which is the last item to be implemented before we can merge to master.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:51:57 +01:00
aszlig aa65a7057f
vm/windows: Properly escape shell command.
Security-wise it's not a big issue because we're still sandboxed, but I
really don't want to write something like \\\\\\\\192.168.0.2\\\\share
in order to set up network shares.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:51:57 +01:00
aszlig cfa859d792
vm/windows: Don't init /nix/store on install.
We're going to do this during the suspendedVM phase, so we're able to
more easily change the shares without reinstalling the whole VM.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:51:57 +01:00
aszlig 3e91192f07
vm/windows: Wait for VDE switch to startup.
This could possibly cause flapping whenever qemu is too fast in starting
up. As we are running with the shell's -e flag, the socat check also
ensures that the VDE switch is properly started and causes the whole
build to fail, should it not start up within 20 seconds.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:51:56 +01:00
aszlig c731467e2c
vm/windows: Split install into several stages.
These stages are in particular:

 * Install of the bare Windows VM with Cygwin and shut down.
 * Boot up the same VM again without the installation media and dump the
   VMs memory to state.gz.
 * Resume from state.gz and build whatever we want to build.

Every single stage involves a new "controller", which is more like an
abstraction on the Nix side that constructs the madness described in
276b72fb93.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:51:56 +01:00
aszlig 5105e7f0bf
vm/windows: Update sha256 of Cygwin's setup.ini.
This is kinda stupid to do every little time the file is automatically
regenerated upstream. But let's see how often that happens and whether
it will become a major annoyance or not, and if yes, we might be forced
to include it in our source tree.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:50:55 +01:00
aszlig 9b1862ca1f
vm/windows: Move creating SSH key into install/.
This SSH key is specifically only for accessing the installed Cygwin
within the Windows VM, so we only need to expose the private key. Yes,
you heard right, the private key. It's not security-relevant because the
machine is completely read-only, only exposed to the filesystem and
networking is not available.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:50:54 +01:00
aszlig 4e21215d52
vm/windows: Move the installer into install/.
At least the largest portion of the installer, because in the end we
don't want the installer to *actually* save the state but only prepare
the base image.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:50:54 +01:00
aszlig 276b72fb93
vm: Introduce new Windows VM installer for Cygwin.
After quite a lot of fighting against Windows and its various
limitations, this new is the base architecture for installing and
accessing the Windows VM and thus the Cygwin environment inside it:

                .------------.
          .---> | vde_switch |
          |     `-[#]----[#]-'
          |        |      |
         ,'       .'      `---.___
       ,'    192.168.0.1          `.
       |          |            192.168.0.2
      ,'    _____[#]____           |
    ,'     |            |   ______[#]______
    |      | Windows VM |  |    .--'       |
    |      |____________|  |    |          |
    |             |  /|\   |  .-|          |
    | .---------. |   |    |  | |          |
  .-|-| manager |-'   |    |  | |          |
  | | `---------'     |    |  | |          |
  | |                 |    |  | |          |
  | | .-------------. |    | Samba         |
  | | | BOOTSTRAP   | |    |  | |          |
  | | |-------------| |    |  | |   .------|
  | `-| spawn VMs   |-+--> |  | `---| xchg | <-------.
  |   |-------------|      |  | .---^------|         |
  |   | install     |---.  |  `-| nixstore | <----.  |
  |   |-------------|   |  |    `----------|      |  |
  |---| suspend VM  |   |  |               |      |  |
  |   `------.------'   |  | Controller VM |      |  |
  |          |          |  |_______________|      |  |
  |       .--'          |         /|\            VirtIO
  |       |           __|__________:____________  |  |
  |      \|/         |  |          `.           | |  |
  | .------------.   |  |           :           | |  |
  | | REAL BUILD |   |  |   .-------^--------.  | |  |
  | |------------|   |  `-> | serial console |  | |  |
  `-| revive VM  |   |      `----------------'  | |  |
    |------------|   |------------.             | |  |
    | build      |-->| /nix/store >>>-----------|-'  |
    |------------|   |------------|             |    |
    | collect    |<--| xchg       >>>-----------|----'
    `-----.------'   |------------'             |
          |          |                          |
         \|/         |    |  |  __   ___  |     |
                     |    |--| |  | (__  -|-    |
    F I N I S H E D  |    |  | |__| ___)  |     |
                     |__________________________|

This might look a bit overwhelming, but let me try to explain:

We're starting at the base derivation ("BOOTSTRAP" above), where we
actually install the Cygwin envirenment. Over there we basically fire up
a vde_switch process and two virtual machines: One is the Windows
machine, the other is a NixOS machine, which serves as some kind of
proxy between the host and the Windows machine.

The reason we're doing this, is because we don't have a lot of options
for sharing files between a stock Windows machine and the host. In
earlier experiments, I've tried to communicate with the Windows guest by
using pipes and OpenSSH, but obviously this wasn't a big speed rush (or
to say it bluntly: It was fucking slow).

Using TCP/IP directly for accessing the guest would have been another
option, but it could lead to possible errors when the port or a range of
ports are in use at the Host system. Also, we would need to punch a hole
into the sandbox of the Nix builder (as it doesn't allow networking),
which in turn will possibly undermine deterministic builds/runs (well,
at least as deterministic as it can be, we're running Windows,
remember?).

So, let's continue: The responsibility of the NixOS (controller) VM is
to just wait until an SSH port becomes available on the Windows VM,
whereas the Windows VM itself is installed using an unattended
installation file provided via a virtual floppy image.

With the installation of the basic Windows OS, we directly install
Cygwin and start up an OpenSSH service.

At this point the bootstrapping is almost finished and as soon as the
port is available, the controller VM sets up Samba shares and makes it
available as drive letters within Windows and as bind mounts (for
example /nix/store) within Cygwin.

Finally we're making a snapshot of the memory of the Windows VM in order
to revive it within a few seconds when we want to build something.

Now, the build process itself is fairly straightforward: Revive VM and
build based on existing store derivations and collect the result _and_
the exit code from the xchg share/directory.

Conclusion: This architecture may sound a bit complicated, but we're
trying to achieve deterministic and reproducable builds and/or test
runs.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 04:50:53 +01:00
Eelco Dolstra 83ca2c272a fetchurl: Ensure that ‘urls’ is a list 2014-02-19 13:58:42 +01:00
Eelco Dolstra a8ded9d5d3 fetchgit: Require a content hash
Without this, the result will not be a fixed-output derivation and
won't work in general.
2014-02-18 19:13:07 +01:00
Petr Rockai 97721af469 VMs: Add Fedora 17 to the list of distros (only had 16 and 18). 2014-02-15 12:57:21 +01:00
Eelco Dolstra 8ba1fdef00 debian: Update to 7.4 2014-02-14 20:29:43 +01:00
Vladimír Čunát e31523a369 preferLocalBuild: revert a stdenv change 2014-02-10 23:33:49 +01:00
Vladimír Čunát 161d029d7d preferLocalBuild: set to true for wrappers and fetchers 2014-02-10 21:04:17 +01:00
Peter Simons 1a40e97ba6 Merge pull request #1674 from errge/haddockhoogle
Generate hoogle databases beside HTML haddock documentation for Haskell packages
2014-02-06 22:46:12 +01:00
Eelco Dolstra 77376a215e debBuild: Allow setting a maintainer address 2014-02-06 14:36:50 +01:00
Eelco Dolstra 6b613f285c Fix nixBuild 2014-02-06 12:30:26 +01:00
Eelco Dolstra a123a2675b lcov: Filter out /nix/store/* by default 2014-02-05 19:18:33 +01:00
Eelco Dolstra 80647127a3 Turn the coverage analysis stdenv adapters into setup hooks
Stdenv adapters are kinda weird and un-idiomatic (especially when they
don't actually change stdenv).  It's more idiomatic to say

  buildInputs = [ makeCoverageAnalysisReport ];
2014-02-05 19:18:33 +01:00
Eelco Dolstra 7703f04b75 Add a stdenv adapter ‘useGoldLinker’ to force use of Gold 2014-02-05 19:18:33 +01:00
Gergely Risko 2c8e492691 Generate hoogle databases beside HTML haddock documentation for Haskell packages 2014-02-04 18:11:57 +01:00
Rob Vermaas 13e13f279b Do not assume a .rev attribute in available in src in maven-build.nix 2014-01-24 15:52:31 +01:00
Eelco Dolstra 6a8485af6b clang-wrapper: Set $CXX to clang
This is useful for non-Autoconf-based packages, since GNU Make's
default for CXX is "g++".  (The CC default is "cc" so should work fine
with Clang already.)
2014-01-23 11:24:17 +01:00
Eelco Dolstra f4af2a8737 clang-wrapper: Sync with gcc-wrapper 2014-01-23 10:57:49 +01:00
Vladimír Čunát ac6761c908 Merge master into stdenv-updates
Conflicts:
	pkgs/applications/version-management/gource/default.nix
	pkgs/top-level/all-packages.nix
2014-01-20 19:39:28 +01:00
Shea Levy 55e6303d6a Add full llvm build, update libc++ and dragonegg
Some packages in the llvm suite (e.g. compiler-rt) cannot be built
separate from the build of llvm, and while some others (e.g. clang) can
the combined build is much better tested (we've had to work around
annoying issues before). So this puts llvm, clang, clang-tools-extra,
compiler-rt, lld, lldb, and polly all into one big build (llvmFull).
This build includes a static llvm, as dynamic is similarly less tested
and has known failures.

This also updates libc++ and dragonegg. libc++ now builds against
libc++abi as a separate package rather than building it during the
libc++ build.

The clang purity patch is gone. Instead, we simply set --sysroot to
/var/empty for pure builds, as all impure paths are either looked up in
the gcc prefix (which we hard-code at compile time) or in the sysroot.
This also means that if NIX_ENFORCE_PURITY is 0 then clang will look in
the normal Linux paths by default, which is the proper behavior IMO.

polly required an updated isl. When stdenv-updates is merged, perhaps we
can update the isl used by gcc and avoid having two versions.

Since llvm on its own is now separate from the llvm used by clang, I've
removed myself as maintainer from llvm and will leave maintenance of
that to those who are interested in llvm separate from clang.

Signed-off-by: Shea Levy <shea@shealevy.com>
2014-01-20 05:42:38 -05:00
Shea Levy b529a34b4a Add Fedora 20
Signed-off-by: Shea Levy <shea@shealevy.com>
2014-01-19 20:50:46 -05:00
Vladimír Čunát 0a58b512cb Merge master into stdenv-updates
Conflicts (simple):
	pkgs/development/compilers/gcc/4.8/default.nix
	pkgs/development/compilers/llvm/default.nix
	pkgs/development/libraries/gmp/5.1.1.nix
	pkgs/development/libraries/gmp/5.1.3.nix
	pkgs/development/libraries/gmp/5.1.x.nix
	pkgs/top-level/all-packages.nix
2014-01-18 14:46:20 +01:00
Rob Vermaas 1625743902 Fix Fedora8 vm builds. 2014-01-15 21:32:38 +01:00
Eelco Dolstra 8622548160 Add a setup hook for fixing dylib install names on Darwin
Install names need to be absolute paths, otherwise programs that link
against the dylib won't work without setting $DYLD_LIBRARY_PATH.  Most
packages do this correctly, but some (like Boost and ICU) do not.
This setup hook absolutizes all install names.
2014-01-15 13:41:09 +01:00
Rob Vermaas ed9520bfb5 Allow specifying extra qemu flags using QEMU_OPTS. Replace CentOS 6.3 with CentOS 6.5 (6.3 is removed from site). 2014-01-14 22:51:26 +01:00
Eelco Dolstra 6fb27abf33 Merge branch 'java' 2014-01-13 13:16:17 +01:00
Vladimír Čunát a2c316288c Merge master into stdenv-updates
Conflicts:
	pkgs/development/lisp-modules/stumpwm/default.nix (auto-solved)
	pkgs/top-level/all-packages.nix (trivial)
2014-01-12 12:29:24 +01:00
William A. Kennington III 42229f1040 nix-prefetch-git: Convert relative submodule URLS to absolute URLS
nix-prefetch-git does not convert relative submodule urls into absolute
urls based on the parent's origin. This patch adds support for
repositories which are using the relative url syntax.
2014-01-10 17:28:35 -06:00
Eelco Dolstra 51713fbbfc gcc-wrapper: Use -isystem instead of -I
This mostly matters for "gcc -MMD", which distinguishes between user
and system header files.
2014-01-10 13:29:29 +01:00
Eelco Dolstra 16e626a80d Replace lib/java -> share/java 2014-01-09 13:01:38 +01:00
Eelco Dolstra 3abcd7e268 debian: Update to 7.3 2014-01-06 15:51:00 +01:00
Eelco Dolstra 5d6259a973 Add a setup hook that automatically sets up $CLASSPATH
All JARs in $pkg/share/java (for each $pkg in the build inputs) are
added to $CLASSPATH.  Thus, you can say

  buildInputs = [ setJavaClassPath someJavaDependency ];

and the JARs in someJavaDependency will be found automatically by
tools like javac or ant.

Note that the manual used to say that JARs should be installed in
lib/java; this is now share/java, following the Debian policy:

  http://www.debian.org/doc/packaging-manuals/java-policy/x110.html

The directory share/java makes more sense because JARs are
architecture-independent.  (Also, a quick grep shows that we were not
exactly consistent about this in Nixpkgs.)
2014-01-06 14:36:14 +01:00
Eelco Dolstra 4728ec3f99 Fix some references to apacheAntOpenJDK 2014-01-06 14:36:13 +01:00
Shea Levy e6dae5d7f4 Add fetchbower function to go along with bower2nix
Signed-off-by: Shea Levy <shea@shealevy.com>
2014-01-06 08:20:22 -05:00
Vladimír Čunát 9443129b08 Merge master into stdenv-updates
Conflicts:
	pkgs/development/libraries/glibc/2.18/common.nix (take stdenv-updates)
	pkgs/misc/emulators/zsnes/default.nix (more complex, build tested)
	pkgs/top-level/all-packages.nix (auto-solved)
2014-01-04 18:34:53 +01:00
Michael Raskin af9db522cf Update FLTK and TigerVNC; doesn't fix TigerVNC Xvnc build per se 2014-01-01 01:36:57 +04:00
Peter Simons 631ad32b5d pkgs/build-support/cabal: the configure-phase warning check can be
disabled by setting 'strictConfigurePhase' to 'false'

This is necessary for some packages, like dns, because cabal warns about
multiple versions of the same dependency being used, but the usage is fine,
actually, so we want the build to succeed. Packages that depend on 'doctest'
also have this issue <https://github.com/sol/doctest-haskell/issues/69>.
2013-12-29 12:26:05 +01:00
Peter Simons 6209bb8556 pkgs/build-support/cabal: add 'buildTarget' attribute to allow passing flags to "./Setup build" 2013-12-29 12:26:05 +01:00