nixpkgs/pkgs/build-support
aszlig a5bc11f9eb
nixos/vm-tests: Remove msize mount option
This seems to be the root cause of the random page allocation failures
and @wizeman did a very good job on not only finding the root problem
but also giving a detailed explanation of it in #10828.

Here is an excerpt:

  The problem here is that the kernel is trying to allocate a contiguous
  section of 2^7=128 pages, which is 512 KB. This is way too much:
  kernel pages tend to get fragmented over time and kernel developers
  often go to great lengths to try allocating at most only 1 contiguous
  page at a time whenever they can.

  From the error message, it looks like the culprit is unionfs, but this
  is misleading: unionfs is the name of the userspace process that was
  running when the system ran out of memory, but it wasn't unionfs who
  was allocating the memory: it was the kernel; specifically it was the
  v9fs_dir_readdir_dotl() function, which is the code for handling the
  readdir() function in the 9p filesystem (the filesystem that is used
  to share a directory structure between a qemu host and its VM).

  If you look at the code, here's what it's doing at the moment it tries
  to allocate memory:

    buflen = fid->clnt->msize - P9_IOHDRSZ;

    rdir = v9fs_alloc_rdir_buf(file, buflen);

  If you look into v9fs_alloc_rdir_buf(), you will see that it will try
  to allocate a contiguous buffer of memory (using kzalloc(), which is a
  wrapper around kmalloc()) of size buflen + 8 bytes or so.

  So in reality, this code actually allocates a buffer of size
  proportional to fid->clnt->msize. What is this msize? If you follow
  the definition of the structures, you will see that it's the
  negotiated buffer transfer size between 9p client and 9p server. On
  the client side, it can be controlled with the msize mount option.

  What this all means is that, the reason for running out of memory is
  that the code (which we can't easily change) tries to allocate a
  contiguous buffer of size more or less equal to "negotiated 9p
  protocol buffer size", which seems to be way too big (in our NixOS
  tests, at least).

After that initial finding, @lethalman tested the gnome3 gdm test
without setting the msize parameter at all and it seems to have resolved
the problem.

The reason why I'm committing this without testing against all of the
NixOS VM test is basically that I think we can only go better but not
worse than the current state.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-12-14 17:26:24 +01:00
..
agda agda: Remove unused/uneeded abstractions, including postprocess 2015-05-31 01:55:10 +00:00
autonix autonix: JSON manifest format 2015-06-25 09:03:40 -05:00
build-dotnet-package build-dotnet-package: add optional makeWrapperArgs argument 2015-10-29 06:02:18 +03:00
build-fhs-chrootenv buildFHS{Chroot,User}Env: support extraInstallCommands 2015-12-04 00:58:47 +03:00
build-fhs-userenv build-fhs-userenv: don't leak file descriptors 2015-12-10 16:01:04 +00:00
buildenv buildEnv: Allow setting meta attributes 2015-11-29 19:13:19 +02:00
builder-defs Don't fail if env-vars cannot be written to 2015-11-04 16:32:59 +01:00
cc-wrapper Fix compilation of GCC 4.9 on SmartOS. 2015-11-16 17:20:16 +01:00
dotnetbuildhelpers Lay down the foundation for packaging the .NET echosystem 2015-06-06 09:06:22 -04:00
dotnetenv Remove executable bits from non-executable files 2015-01-27 18:54:38 +01:00
emacs melpaBuild: add unpackCmd for single-file packages 2015-12-06 11:50:10 -06:00
fetchadc Add Apple command-line tools 2014-10-02 06:50:09 +01:00
fetchbower Make fetchBower a bit more like a normal mkDerivation, which is easier to test. 2015-07-16 14:42:16 +00:00
fetchbzr build-support: Fix nix-prefetch-* on OS X. 2014-08-27 11:22:34 +02:00
fetchcvs build-support: Fix nix-prefetch-* on OS X. 2014-08-27 11:22:34 +02:00
fetchdarcs fetchbzr, fetchdarcs, fetchhg: use rev attr 2014-06-28 21:06:10 +02:00
fetchegg Infrastructure to build chicken eggs. 2014-10-13 23:29:27 +02:00
fetchfile Update builder.sh 2015-11-25 17:14:03 +08:00
fetchgit fetchgit: follow up to 2cf7069b7d 2015-12-06 15:06:02 +01:00
fetchgitlocal Rewrite fetchgitLocal 2015-10-01 22:00:44 -07:00
fetchgitrevision Remove semicolon causing evaluation error. 2011-01-02 23:52:39 +00:00
fetchhg fetchhg: fixed nix-prefetch-hg interpreter 2015-10-06 22:41:46 +03:00
fetchmtn Fix a few typos 2013-08-11 10:55:53 +00:00
fetchnuget fetchnuget: run fixupPhase to honor propagatedBuildInputs 2015-06-12 17:13:21 +02:00
fetchpatch fetchpatch: Allow to pass a postFetch attribute. 2015-06-23 03:35:30 +02:00
fetchsvn fetchsvn: correctly handle trailing slashes 2015-12-06 11:49:51 -06:00
fetchsvnrevision Live builds support: initially for git and svn. taglib_live is packaged as an example. 2011-01-02 23:23:59 +00:00
fetchsvnssh undo 2008-07-06 15:46:10 +00:00
fetchurl fetchurl: Always use tarballs.nixos.org 2015-12-10 16:06:12 +01:00
fetchzip fetchzip: fix unpacking regression 2015-01-25 20:16:21 -05:00
gcc-cross-wrapper orig-gcc -> orig-cc 2015-03-04 22:15:46 +01:00
gcc-wrapper-old Fix build of gccgo-wrapper on Illumos. 2015-11-16 17:20:12 +01:00
grsecurity Revert "kernel: Remove unsupported 3.10, 3.12, 3.14" 2015-11-19 14:25:16 +01:00
kdewrapper Another attempt to eradicate ensureDir 2014-06-30 14:56:10 +02:00
kernel makeInitrd: Support prepending other initrds 2015-03-25 14:14:46 -07:00
libredirect citrix-receiver: init at 13.2.1, fixes #8458 2015-11-11 22:27:15 +01:00
make-desktopitem makeDesktopItem: add StartupNotify key 2015-12-12 17:25:56 -06:00
make-startupitem Another attempt to eradicate ensureDir 2014-06-30 14:56:10 +02:00
make-symlinks * "." -> "source". 2005-12-05 14:11:09 +00:00
mono-dll-fixer * Remove references to substitute.sh. 2006-12-27 18:17:53 +00:00
native-darwin-cctools-wrapper Darwin: Add mig', lipo', etc. to `stdenvNative'. 2012-04-23 15:47:31 +00:00
nuke-references nuke-references: support -e option 2015-10-18 18:41:11 +03:00
ocaml add an ocaml specific builder to the system 2015-05-22 10:12:21 -07:00
release clang-analyzer: align attrname with pkgname 2015-02-20 22:30:51 +01:00
rust buildRustPackage: fix failure due to branch names with slashes 2015-11-24 20:50:11 +01:00
setup-hooks Fix patchShebangs on SmartOS 2015-11-16 17:20:14 +01:00
src-only A helper to get the source some package is built from. 2008-10-01 15:57:22 +00:00
substitute * Set preferLocalBuild on a few more trivial builders. 2012-04-26 15:17:43 +00:00
substitute-files substituteAllFiles: add derivation 2014-10-16 15:27:25 +01:00
templaterpm Update: new features for nix-template-rpm 2015-02-06 20:12:57 +01:00
upstream-updater More support for SF.net in auto-updater 2014-09-02 14:25:16 +04:00
vm nixos/vm-tests: Remove msize mount option 2015-12-14 17:26:24 +01:00
vsenv * "ensureDir" -> "mkdir -p". "ensureDir" is a rather pointless 2012-01-18 20:16:00 +00:00
build-maven.nix build-maven: Add handling for snapshot version resolution 2015-07-20 09:19:12 -04:00
build-pecl.nix fix phpPackages memcache,memcached,xdebug 2015-02-05 20:28:16 +01:00
replace-dependency.nix replace-dependency.nix: Remove unnecessary carriage return. 2015-11-16 17:52:12 -05:00
source-from-head-fun.nix Remove getConfig helper function 2012-09-19 13:56:56 -04:00
trivial-builders.nix Set ‘allowSubstitutes = false’ on various derivations 2015-07-09 15:10:37 +02:00