Commit graph

16 commits

Author SHA1 Message Date
Damien Diederen 7c2adb1d5c nixos/lib/make-ext4-fs: Fix: `resize2fs -M' can leave insufficient slack
The root filesystem resizing step, `resize2fs -M', does not provide any
control over the amount of slack left in the result.  It can produce an
arbitrarily tight fit, depending on how well the payload aligns with
ext4 data structures.

This is problematic, as NixOS must create a few files and directories
during its first boot, before the root is enlarged to match the size of
the containing SD card.

An overly tight fit can cause failures in the first stage:

    mkdir: can't create directory '/mnt-root/proc': No space left on device

or in the second stage:

    install: cannot create directory '/var': No space left on device

A previous version of `make-ext4-fs' (before PR #79368) was explicitly
"reserving" 16 MiB of free space in the final filesystem.  Manually
calculating the size of an ext4 filesystem is a perilous endeavor,
however, and the method it employed was apparently unreliable.

Reverting is consequently not a good option.

A solution would be to create some sort of "balloon" occupying inodes
and blocks in the image prior to invoking `resize2fs -M', and to remove
these temporary files/directories before the compression step.

This changeset takes the simpler approach of simply dropping the
resizing step.

Note that this does *not* result in a larger image in general, as the
current procedure does not truncate the `.img' file anyway.  In fact, it
has been observed to yield *smaller* compressed images---probably
because of some "noise" left after resizing.  E.g., before-vs-after:

    -r--r--r-- 2 root root 607M  1. Jan 1970  nixos-sd-image-21.11pre-git-x86_64-linux.img.zst

    -r--r--r-- 2 root root 606M  1. Jan 1970  nixos-sd-image-21.11pre-git-x86_64-linux.img.zst
2021-05-31 21:35:51 +02:00
Julian Stecklina fd2047c642 nixos/lib/make-ext4-fs: fix after mkfs.ext4 refactoring
In 9ac1ab10c9 this library function was
refactored to use mkfs.ext4 instead of cptofs. There are two problems:

If populateImageCommands would create no files (which is fine), a cp
invocation would fail due to missing source arguments.

Another problem is that mkfs.ext4 relies on fakeroot to have sane
uid/gids in the generated filesystem image. This currently doesn't
work for cross compiling.
2020-07-26 22:49:55 +02:00
misuzu 9ac1ab10c9 nixos/lib/make-ext4-fs: use mkfs.ext4 instead of cptofs
This fixes image creation on armv7l when image is bigger than 2G.
Also fix some reproducibility issues and other cptofs issues.
2020-07-01 11:32:28 +03:00
Florian Klink 079de14fd3 nixos/make-ext4-fs: increase fudge factor from 1.03 to 1.10
Turns out, on smaller images (~800MiB uncompressed sdcard image size),
the current fudge factor is way too small to even get the system to the
phase where it can resize itself.

I first tried with 1.05, but it wasn't enough.
2020-06-21 09:47:02 +02:00
Richard Marko db894cb8ce nixos/lib/make-ext4-fs: fall back to resize2fs -M if exact resize fails
See also
- https://bugs.launchpad.net/ubuntu/+source/e2fsprogs/+bug/1415077/comments/4
- https://github.com/NixOS/nixpkgs/pull/62262
2020-02-09 17:18:21 +01:00
Bernardo Meurer 70c5a78062
nixos: compress make-ext4-fs with zstd 2019-12-13 16:18:04 -08:00
Samuel Dionne-Riel a0aff40f65 make-ext4-fs: Allows populating with custom files
This will allow adding /boot files to the rootfs.
2019-06-16 17:47:30 -04:00
Alexander Krupenkin 2f0c495c31
lib/make-ext4-fs: more efficient store maker 2018-12-24 23:21:15 +03:00
Samuel Dionne-Riel 61bdaad9a9 sd-image: Slims the ext4 filesystem even more.
This is to try and squeeze more lost space from the image, so that hydra
starts building it again.

The fsck previous to the resize2fs is required so resize2fs works.

The one afterwards is a sanity check.

Using `-M` from resize2fs will not give much saved space due to a known
(in the manual) issue.

```
[samueldr@aarch64:~/nixpkgs]$ ls -lh result-*/*/*.img
-r--r--r-- 1 root root 2.2G Jan  1  1970 result-original/sd-image/nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img
-r--r--r-- 1 root root 2.1G Jan  1  1970 result-M/sd-image/nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img
-r--r--r-- 1 root root 1.9G Jan  1  1970 result-slimmed/sd-image/nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img
```

```
[samueldr@aarch64:~/nixpkgs]$ nix path-info -S ./result-original
/nix/store/c8k9n78gylx293rjh762fr05a069kxp2-nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img   3844125000

[samueldr@aarch64:~/nixpkgs]$ nix path-info -S ./result-slimmed
/nix/store/962238skj5mnzhrsmjy23dyzmxk77sp4-nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img   3447473208
```
2018-11-30 19:11:49 -05:00
Will Fancher 47d2f92a05 make-ext4-fs: Fix cross compiling 2018-08-21 14:05:23 +03:00
Andrew Dunham d7bfd04301 sdImage: make partition ID/UUID configurable 2018-07-12 17:02:03 +03:00
Tuomas Tynkkynen bc828721ed nixos/lib/make-ext4-fs: Add a sanity check
I ended up with a corrupted image with the debugfs contraption once, and
given I couldn't reproduce the problem I suppose that happens if the
filesystem of the builder runs out of space.

At least in this instance fsck could detect it, so let's add it as a
sanity check.
2018-05-05 19:30:54 +03:00
Tuomas Tynkkynen 487be791d7 nixos/make-ext4-fs: Use closureInfo
exportReferencesGraph is deprecated and doesn't have the generated
initial Nix database contain the SHA256 of the contents of the store
paths, which breaks various things under Nix 2.0.
2018-04-10 00:31:36 +03:00
Ben Gamari a2215da9a1 make-ext4-fs: Dependencies are nativeBuildInputs 2018-01-09 11:25:19 -05:00
Tuomas Tynkkynen 1d72474df7 make-ext4-fs: Fix modification timestamps in image
Use '-f' to make timestamp calls to always return the same time. Also
Nix uses the timestamp of '1' instead of epoch directly.
2017-07-29 22:03:43 +03:00
Tuomas Tynkkynen df86813d97 nixos: Add derivations for SD card installation images on ARM
The resulting image can be copied to a SD card with `dd` and is directly
bootable by a suitably configured U-Boot. Though depending on the board, some
extra steps are required for copying U-Boot itself to the SD card.

Inside the image is a partition table, with a FAT32 /boot and a normal
writable EXT4 rootfs. It's possible to directly reuse the SD image's
partition layout and "install" NixOS on the same SD card by replacing
the default configuration.nix and nixos-rebuild, and actually is the
preferred way to use these images. To assist in this installation
method, the boot scripts on the image automatically resize the rootfs
partition to fit the SD card on the first boot.

The SD images come in two flavors; one for the ARMv6 Raspberry Pi,
and one multiplatform image for all the boards supported by the
mainline kernel's multi_v7_defconfig config target. At the moment, these
have been tested on:
    - Raspberry Pi Model B (512MB model)
    - NVIDIA Jetson TK1
    - Linksprite pcDuino3 Nano

To build, run:

nix-build '<nixpkgs/nixos>' -A config.system.build.sdImage \
    -I nixos-config='<nixpkgs/nixos/modules/installer/cd-dvd/sd-image-armv7l-multiplatform.nix>'
2015-07-26 00:31:20 +03:00