Commit graph

1352 commits

Author SHA1 Message Date
Edmund Wu 78f915a002
nixos/kernel.nix: ensure same kernel is used
Originally, changes to the kernel don't propagate to the other
derivation within the same package set. This commit allows for the
changes in the kernel to be propagated.

A distinct example is setting `boot.kernel.randstructSeed` to a non-zero
length string which would result in building 2 kernels, one with the
correct seed and the other with the zero length seed. Then, when using
an out-of-tree kernel driver, it would be built with the zero length
seed which differs from the non-zero length seed used to boot,
contradicting the purpose of the `boot.kernel.randstructSeed`.
2021-01-11 12:57:34 -05:00
Jan Tojnar 2b9372b2ca
Merge branch 'master' into staging-next 2021-01-11 13:36:42 +01:00
Eelco Dolstra 187af93c19
Merge pull request #78430 from puckipedia/fix-initrd-silent
nixos/stage-1: Do not allow missing kernel modules in initrd
2021-01-11 13:18:45 +01:00
Jan Tojnar 88c998fb21
Merge branch 'staging-next' into staging 2021-01-10 23:24:57 +01:00
Jan Tojnar dd72357155
Merge branch 'master' into staging-next 2021-01-10 23:24:33 +01:00
Jörg Thalheim ceda1a5eee
nixos/systemd-boot: add typechecking 2021-01-10 21:13:59 +01:00
Dominik Xaver Hörl 25bef2d8f9 treewide: simplify pkgs.stdenv.lib -> pkgs.lib
The library does not depend on stdenv, that `stdenv` exposes `lib` is
an artifact of the ancient origins of nixpkgs.
2021-01-10 20:12:06 +01:00
Frederik Rietdijk b209617ff0 plasma5Packages: alias to the libsForQt5 used to build the plasma5 desktop
For in NixOS it is beneficial if both plasma5 and pam use the same Qt5
version. Because the plasma5 desktop may use a different version as the
default Qt5 version, we introduce plasma5Packages.
2021-01-10 15:59:45 +01:00
Danielle Lancashire ca7b35d2d9
modules/boot: Specify the type for tmpfs mounts
https://github.com/NixOS/nixpkgs/pull/107497 broke booting on many systems that
use tmpOnTmpfs due to the lack of specifying the mount type.

This commit explicitly adds the mount type, which should fix booting
such systems.

The original change may want to be revisited however too.
2021-01-09 15:32:17 +01:00
github-actions[bot] e251591528
Merge master into staging-next 2021-01-09 01:00:25 +00:00
Florian Klink 964c419bc0
Merge pull request #107497 from helsinki-systems/fix/tmp-mount
nixos/tmp: Make /tmp on ramdisk usable again
2021-01-08 23:54:46 +01:00
Gauvain 'GovanifY' Roussel-Tarbouriech 771ba47d56
boot.loader.grub: add extraInstallCommands option 2021-01-03 09:31:51 +01:00
Matthew Bauer 08135a3caa nixos/binfmt.nix: fix running commands in binfmt
To allow arguments in binfmt, we need to put the command in a shell
script. This uses exec to run the binfmt interpreter.

Fixes #90683
2020-12-31 13:28:01 -06:00
Blaž Hrastnik a6b5693a71 nixos/stage-1: set zstd as the default compressor when supported
Co-authored-by: Atemu <atemu.main@gmail.com>
2020-12-28 14:25:17 +01:00
Janne Heß 56d7e7492c
nixos/tmp: Make /tmp on ramdisk usable again
@poettering decided we only need a limited number of inodes in our /tmp,
so why not limit that for every systemd user? That makes medium-sized nix
builds impossible so this commit restores the old behaviour which is the
kernel default of half the number of physical RAM pages which does not
seem too unreasonable to me.
2020-12-26 13:13:41 +01:00
Florian Klink c674a51382 nixos/systemd: provide libidn2 for systemd-resolved
systemd started using dlopen() for some of their "optional"
dependencies.

Apparently, `libidn2` isn't so optional, and systemd-resolved doesn't
work without libidn2 present, breaking DNS resolution.

Fixes https://github.com/NixOS/nixpkgs/issues/107537

Upstream bug: https://github.com/systemd/systemd/issues/18078
2020-12-25 13:45:25 +01:00
Linus Heckemann 834cc5d5fa nixos/initrd: docbookise "compressor" description 2020-12-17 23:01:08 +01:00
Dominik Xaver Hörl d4ef25db5d nixos/initrd: add compressorArgs, make compressor option public 2020-12-17 11:38:10 +01:00
Linus Heckemann c14f14eeaf
Merge pull request #97505 from mayflower/grub-efi-mirroredboot
nixos/grub: allow multiple "nodev" devices for mirroredBoots
2020-12-04 18:37:50 +01:00
Frederik Rietdijk 0d8491cb2b Merge master into staging-next 2020-11-29 13:51:10 +01:00
Sarah Brofeldt a7a5f7904c
Merge pull request #99173 from johanot/fix-initrd-ssh-commands-test
nixos/initrd-ssh: set more defensive pemissions on sshd test key
2020-11-29 11:27:03 +01:00
Frederik Rietdijk b2a3891e12 Merge master into staging-next 2020-11-27 15:09:19 +01:00
Florian Klink 5b3a952e04
Merge pull request #102938 from cruegge/dev-symlinks
nixos/stage-1: create /dev/std{in,out,err} symlinks
2020-11-25 01:57:21 +01:00
Graham Christensen bc49a0815a
utillinux: rename to util-linux 2020-11-24 12:42:06 -05:00
Florian Klink d22b3ed4bc systemd: switch to unified cgroup hierarchy by default
See https://www.redhat.com/sysadmin/fedora-31-control-group-v2 for
details on why this is desirable, and how it impacts containers.

Users that need to keep using the old cgroup hierarchy can re-enable it
by setting `systemd.unifiedCgroupHierarchy` to `false`.

Well-known candidates not supporting that hierarchy, like docker and
hidepid=… will disable it automatically.

Fixes #73800
2020-11-19 16:56:46 +01:00
Frederik Rietdijk 20f001c01e Merge master into staging-next 2020-11-09 14:33:52 +01:00
Jan Tojnar 8e7fca3a5c
nixos/plymouth: fix eval with aliases disabled
Fallout from https://github.com/NixOS/nixpkgs/pull/101369
2020-11-09 14:00:18 +01:00
Samuel Gräfenstein 88bf1b3e92 nixos/boot: add final newline to pbkdf2-sha512.c 2020-11-09 11:39:28 +00:00
Marek Mahut e02f6bfa26
Merge pull request #100418 from pltanton/master
fido2luks: 0.2.3 -> 0.2.15
2020-11-09 00:22:09 +01:00
Timo Kaufmann 1fd1c2ad88
Merge pull request #96639 from xfix/support-microsoft-usb-keyboards
nixos/availableKernelModules: add microsoft hid
2020-11-05 20:33:49 +01:00
Peter Hoeg 13ed0cce2f nixos/systemd-resolved: fix incorrect user 2020-11-05 22:41:39 +08:00
Christoph Ruegge bcc808c68f Create /dev/std{in,out,err} symlinks in stage-1
This used to be done by udev, but that was removed in
systemd/systemd@6b2229c. The links are created by systemd at the end of
stage-2, but activation scripts might need them earlier.
2020-11-05 15:32:19 +01:00
lf- 644079e707 nixos/modules: deprecation warning for StartLimitInterval in [Service]
This implements
https://github.com/NixOS/nixpkgs/issues/45786#issuecomment-440091879
2020-10-31 16:50:35 +01:00
Niklas Hambüchen c178fe4bbb nixos/modules: Reformat warnings section 2020-10-31 16:50:25 +01:00
lf- b37bbca521 nixos/modules: fix systemd start rate-limits
These were broken since 2016:
f0367da7d1
since StartLimitIntervalSec got moved into [Unit] from [Service].
StartLimitBurst has also been moved accordingly, so let's fix that one
too.

NixOS systems have been producing logs such as:
/nix/store/wf98r55aszi1bkmln1lvdbp7znsfr70i-unit-caddy.service/caddy.service:31:
Unknown key name 'StartLimitIntervalSec' in section 'Service', ignoring.

I have also removed some unnecessary duplication in units disabling
rate limiting since setting either interval or burst to zero disables it
(ad16158c10/src/basic/ratelimit.c (L16))
2020-10-31 01:35:56 -07:00
Graham Christensen 82578fc725
Merge pull request #102172 from grahamc/stage-1-datestamps
stage-1: add datestamps to logs
2020-10-30 16:13:02 -04:00
Graham Christensen b34cf366aa
Merge pull request #102171 from grahamc/faster-ext-resize
stage-1: modprobe ext{2,3,4} before resizing (so resizing takes less than 45 minutes)
2020-10-30 16:12:50 -04:00
Graham Christensen ece5c0f304
stage-1: modprobe ext{2,3,4} before resizing
I noticed booting a system with an ext4 root which expanded to 5T took
quite a long time (12 minutes in some cases, 43(!) in others.)

I changed stage-1 to run `resize2fs -d 62` for extra debug output and
timing information. It revealed the adjust_superblock step taking
almost all of the time:

    [Fri Oct 30 11:10:15 UTC 2020] zero_high_bits_in_metadata: Memory used: 132k/0k (63k/70k), time:  0.00/ 0.00/ 0.00
    [Fri Oct 30 11:21:09 UTC 2020] adjust_superblock: Memory used: 396k/4556k (295k/102k), time: 654.21/ 0.59/ 5.13

but when I ran resize2fs on a disk with the identical content growing
to the identical target size, it would only take about 30 seconds. I
looked at what happened between those two steps in the fast case with
strace and found:

```
   235	getrusage(RUSAGE_SELF, {ru_utime={tv_sec=0, tv_usec=1795}, ru_stime={tv_sec=0, tv_usec=3590}, ...}) = 0
   236	write(1, "zero_high_bits_in_metadata: Memo"..., 84zero_high_bits_in_metadata: Memory used: 132k/0k (72k/61k), time:  0.00/ 0.00/ 0.00
   237	) = 84
   238	gettimeofday({tv_sec=1604061278, tv_usec=480147}, NULL) = 0
   239	getrusage(RUSAGE_SELF, {ru_utime={tv_sec=0, tv_usec=1802}, ru_stime={tv_sec=0, tv_usec=3603}, ...}) = 0
   240	gettimeofday({tv_sec=1604061278, tv_usec=480192}, NULL) = 0
   241	mmap(NULL, 2564096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fa3c7355000
   242	access("/sys/fs/ext4/features/lazy_itable_init", F_OK) = 0
   243	brk(0xf85000)                           = 0xf85000
   244	brk(0xfa6000)                           = 0xfa6000
   245	gettimeofday({tv_sec=1604061278, tv_usec=538828}, NULL) = 0
   246	getrusage(RUSAGE_SELF, {ru_utime={tv_sec=0, tv_usec=58720}, ru_stime={tv_sec=0, tv_usec=3603}, ...}) = 0
   247	write(1, "adjust_superblock: Memory used: "..., 79adjust_superblock: Memory used: 396k/2504k (305k/92k), time:  0.06/ 0.06/ 0.00
   248	) = 79
   249	gettimeofday({tv_sec=1604061278, tv_usec=539119}, NULL) = 0
   250	getrusage(RUSAGE_SELF, {ru_utime={tv_sec=0, tv_usec=58812}, ru_stime={tv_sec=0, tv_usec=3603}, ...}) = 0
   251	gettimeofday({tv_sec=1604061279, tv_usec=939}, NULL) = 0
   252	getrusage(RUSAGE_SELF, {ru_utime={tv_sec=0, tv_usec=520411}, ru_stime={tv_sec=0, tv_usec=3603}, ...}) = 0
   253	write(1, "fix_uninit_block_bitmaps 2: Memo"..., 88fix_uninit_block_bitmaps 2: Memory used: 396k/2504k (305k/92k), time:  0.46/ 0.46/ 0.00
   254	) = 88
```

In particular the access to /sys/fs seemed interesting. Looking
at the source of resize2fs:

```
[root@ip-172-31-22-182:~/e2fsprogs-1.45.5]# rg -B2 -A1 /sys/fs/ext4/features/lazy_itable_init .
./resize/resize2fs.c
923-	if (getenv("RESIZE2FS_FORCE_LAZY_ITABLE_INIT") ||
924-	    (!getenv("RESIZE2FS_FORCE_ITABLE_INIT") &&
925:	     access("/sys/fs/ext4/features/lazy_itable_init", F_OK) == 0))
926-		lazy_itable_init = 1;
```

I confirmed /sys is mounted, and then found a bug suggesting the
ext4 module is maybe not loaded:
https://bugzilla.redhat.com/show_bug.cgi?id=1071909

My home server doesn't have ext4 loaded and had 3T to play with, so
I tried (and succeeded with) replicating the issue locally:

```
[root@kif:/scratch]# lsmod | grep -i ext

[root@kif:/scratch]# zfs create -V 3G rpool/scratch/ext4

[root@kif:/scratch]# time mkfs.ext4 /dev/zvol/rpool/scratch/ext4
mke2fs 1.45.5 (07-Jan-2020)
Discarding device blocks: done
Creating filesystem with 786432 4k blocks and 196608 inodes
Filesystem UUID: 560a4a8f-93dc-40cc-97a5-f10049bf801f
Superblock backups stored on blocks:
	32768, 98304, 163840, 229376, 294912

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

real	0m2.261s
user	0m0.000s
sys	0m0.025s

[root@kif:/scratch]# zfs set volsize=3T rpool/scratch/ext4

[root@kif:/scratch]# time resize2fs -d 62 /dev/zvol/rpool/scratch/ext4
resize2fs 1.45.5 (07-Jan-2020)
fs has 11 inodes, 1 groups required.
fs requires 16390 data blocks.
With 1 group(s), we have 22234 blocks available.
Last group's overhead is 10534
Need 16390 data blocks in last group
Final size of last group is 26924
Estimated blocks needed: 26924
Extents safety margin: 49
Resizing the filesystem on /dev/zvol/rpool/scratch/ext4 to 805306368 (4k) blocks.
read_bitmaps: Memory used: 132k/0k (63k/70k), time:  0.00/ 0.00/ 0.00
read_bitmaps: I/O read: 1MB, write: 0MB, rate: 3802.28MB/s
fix_uninit_block_bitmaps 1: Memory used: 132k/0k (63k/70k), time:  0.00/ 0.00/ 0.00
resize_group_descriptors: Memory used: 132k/0k (68k/65k), time:  0.00/ 0.00/ 0.00
move_bg_metadata: Memory used: 132k/0k (68k/65k), time:  0.00/ 0.00/ 0.00
zero_high_bits_in_metadata: Memory used: 132k/0k (68k/65k), time:  0.00/ 0.00/ 0.00
```

here it got stuck for quite some time ... straceing this 20 minutes in revealed this in a tight loop:

```
getuid()                                = 0
geteuid()                               = 0
getgid()                                = 0
getegid()                               = 0
prctl(PR_GET_DUMPABLE)                  = 1 (SUID_DUMP_USER)
fallocate(3, FALLOC_FL_ZERO_RANGE, 2222649901056, 2097152) = 0
fsync(3)                                = 0
```

it finally ended 43(!) minutes later:

```
adjust_superblock: Memory used: 264k/3592k (210k/55k), time: 2554.03/ 0.16/15.07
fix_uninit_block_bitmaps 2: Memory used: 264k/3592k (210k/55k), time:  0.16/ 0.16/ 0.00
blocks_to_move: Memory used: 264k/3592k (211k/54k), time:  0.00/ 0.00/ 0.00
Number of free blocks: 755396/780023556, Needed: 0
block_mover: Memory used: 264k/3592k (216k/49k), time:  0.05/ 0.01/ 0.00
block_mover: I/O read: 1MB, write: 0MB, rate: 18.68MB/s
inode_scan_and_fix: Memory used: 264k/3592k (216k/49k), time:  0.00/ 0.00/ 0.00
inode_ref_fix: Memory used: 264k/3592k (216k/49k), time:  0.00/ 0.00/ 0.00
move_itables: Memory used: 264k/3592k (216k/49k), time:  0.00/ 0.00/ 0.00
calculate_summary_stats: Memory used: 264k/3592k (216k/49k), time: 16.35/16.35/ 0.00
fix_resize_inode: Memory used: 264k/3592k (222k/43k), time:  0.04/ 0.00/ 0.00
fix_resize_inode: I/O read: 1MB, write: 0MB, rate: 22.80MB/s
fix_sb_journal_backup: Memory used: 264k/3592k (222k/43k), time:  0.00/ 0.00/ 0.00
overall resize2fs: Memory used: 264k/3592k (222k/43k), time: 2570.90/16.68/15.07
overall resize2fs: I/O read: 1MB, write: 1MB, rate: 0.00MB/s
The filesystem on /dev/zvol/rpool/scratch/ext4 is now 805306368 (4k) blocks long.

real	43m1.943s
user	0m16.761s
sys	0m15.069s
```

I then cleaned up and recreated the zvol, loaded the ext4 module, created the ext4 fs,
resized the volume, and resize2fs'd and it went quite quickly:

```
[root@kif:/scratch]# zfs destroy rpool/scratch/ext4

[root@kif:/scratch]# zfs create -V 3G rpool/scratch/ext4

[root@kif:/scratch]# modprobe ext4

[root@kif:/scratch]# time resize2fs -d 62 /dev/zvol/rpool/scratch/ext4

[root@kif:/scratch]# time mkfs.ext4 /dev/zvol/rpool/scratch/ext4
mke2fs 1.45.5 (07-Jan-2020)
Discarding device blocks: done
Creating filesystem with 786432 4k blocks and 196608 inodes
Filesystem UUID: 5b415f2f-a8c4-4ba0-ac1d-78860de77610
Superblock backups stored on blocks:
	32768, 98304, 163840, 229376, 294912

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

real	0m1.013s
user	0m0.001s
sys	0m0.023s

[root@kif:/scratch]# zfs set volsize=3T rpool/scratch/ext4

[root@kif:/scratch]# time resize2fs -d 62 /dev/zvol/rpool/scratch/ext4
resize2fs 1.45.5 (07-Jan-2020)
fs has 11 inodes, 1 groups required.
fs requires 16390 data blocks.
With 1 group(s), we have 22234 blocks available.
Last group's overhead is 10534
Need 16390 data blocks in last group
Final size of last group is 26924
Estimated blocks needed: 26924
Extents safety margin: 49
Resizing the filesystem on /dev/zvol/rpool/scratch/ext4 to 805306368 (4k) blocks.
read_bitmaps: Memory used: 132k/0k (63k/70k), time:  0.00/ 0.00/ 0.00
read_bitmaps: I/O read: 1MB, write: 0MB, rate: 3389.83MB/s
fix_uninit_block_bitmaps 1: Memory used: 132k/0k (63k/70k), time:  0.00/ 0.00/ 0.00
resize_group_descriptors: Memory used: 132k/0k (68k/65k), time:  0.00/ 0.00/ 0.00
move_bg_metadata: Memory used: 132k/0k (68k/65k), time:  0.00/ 0.00/ 0.00
zero_high_bits_in_metadata: Memory used: 132k/0k (68k/65k), time:  0.00/ 0.00/ 0.00
adjust_superblock: Memory used: 264k/1540k (210k/55k), time:  0.02/ 0.02/ 0.00
fix_uninit_block_bitmaps 2: Memory used: 264k/1540k (210k/55k), time:  0.15/ 0.15/ 0.00
blocks_to_move: Memory used: 264k/1540k (211k/54k), time:  0.00/ 0.00/ 0.00
Number of free blocks: 755396/780023556, Needed: 0
block_mover: Memory used: 264k/3592k (216k/49k), time:  0.01/ 0.01/ 0.00
block_mover: I/O read: 1MB, write: 0MB, rate: 157.11MB/s
inode_scan_and_fix: Memory used: 264k/3592k (216k/49k), time:  0.00/ 0.00/ 0.00
inode_ref_fix: Memory used: 264k/3592k (216k/49k), time:  0.00/ 0.00/ 0.00
move_itables: Memory used: 264k/3592k (216k/49k), time:  0.00/ 0.00/ 0.00

calculate_summary_stats: Memory used: 264k/3592k (216k/49k), time: 16.20/16.20/ 0.00
fix_resize_inode: Memory used: 264k/3592k (222k/43k), time:  0.00/ 0.00/ 0.00
fix_resize_inode: I/O read: 1MB, write: 0MB, rate: 5319.15MB/s
fix_sb_journal_backup: Memory used: 264k/3592k (222k/43k), time:  0.00/ 0.00/ 0.00
overall resize2fs: Memory used: 264k/3592k (222k/43k), time: 16.45/16.38/ 0.00
overall resize2fs: I/O read: 1MB, write: 1MB, rate: 0.06MB/s
The filesystem on /dev/zvol/rpool/scratch/ext4 is now 805306368 (4k) blocks long.

real	0m17.908s
user	0m16.386s
sys	0m0.079s
```

Success!
2020-10-30 12:18:23 -04:00
Graham Christensen a179781696
stage-1: add datestamps to logs
When the stage-1 logs get imported in to the journal, they all get
loaded with the same timestamp. This makes it difficult to identify
what might be taking a long time in early boot.
2020-10-30 12:16:35 -04:00
Florian Klink b8d59e93c8 nixos/networkd: allow RouteMetric= in [DHCPv6] section 2020-10-29 19:47:42 +01:00
Anton Plotnikov 1321ae850c
fido2luks: 0.2.3 -> 0.2.15
Also remove interactive flag from initrd, because of broken io.
2020-10-23 11:03:31 +03:00
Marc 'risson' Schmitt 9e6bede5ab nixos/initrd-network: fix /etc/resolv.conf when multiple dns servers from DHCP
Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>
2020-10-19 21:32:58 -07:00
Florian Klink 9d0d99f05b
Merge pull request #95746 from Mic92/cloud-init
cloud-init: 0.7.9 -> 20.2 (python3!)
2020-10-15 22:57:46 +02:00
Johan Thomsen 8f5949fd46 nixos/initrd-ssh: set more defensive pemissions on sshd test key
It looks like the test sshd key can never be used, because of too open
permissions. My guess is that the current test script works fine once
the user defined ssh-key has been copied into initrd.

At "nixos-install" however, the user specified host key is not present
in initrd yet and validation fails.

fixes #91486
2020-09-30 15:47:49 +02:00
Florian Klink 30ddfe6b95 Revert "nixos/systemd-boot: Temporarily ignore errors"
This reverts commit b32701bc54.

That fix has been included upstream in 246.5.
2020-09-27 16:45:27 +02:00
Richard Marko 9b06e9856b raspberrypi-builder: allow passing firmware package as argument 2020-09-26 14:26:16 +02:00
Cole Helbling 4586810487
nixos/stage-1: set up /dev/fd
Otherwise, stage-2-init.sh will complain about not having access to
/dev/fd/62 as of systemd v246.

On IRC, flokli said:

    15:14 <flokli> cole-h: hmmm... I could imagine some of the setup inside /dev has been moved into other parts of systemd
    15:14 <flokli> And given we run systemd much later (outside initramfs only) it doesn't work properly here
    15:17 <flokli> We probably don't invoke udev correctly
2020-09-22 15:33:21 -07:00
Tethys Svensson b32701bc54 nixos/systemd-boot: Temporarily ignore errors
This is a temporary fix for #97433. A more proper fix has been
implemented upstream in systemd/systemd#17001, however until it gets
backported, we are stuck with ignoring the error.

After the backport lands, this commit should be reverted.
2020-09-10 20:56:04 +02:00
Linus Heckemann fdd944526c nixos/grub: allow multiple "nodev" devices for mirroredBoots
For UEFI setups, "device" will generally be the special value "nodev"
which represents not running grub-install at all. Using "nodev" for
boot mirrors should therefore be allowed.
2020-09-09 07:40:08 +02:00
Christian Kampka 2c6753f9d0
Revert "nixos/raspberrypi-builder: fix cross using buildPackages"
The commit enforces buildPackages in the builder but neglects
the fact that the builder is intended to run on the target system.
Because of that, the builder will fail when remotely building a
configuration eg. with nixops or nix-copy-closure.

This reverts commit a6ac6d00f9.
2020-09-08 20:14:13 +02:00
Frederik Rietdijk af81d39b87 Merge staging-next into staging 2020-09-04 20:03:30 +02:00
Anders Kaseorg f4b2c9dfe7 cryptsetup, lvm2, systemd: Break cyclic dependency at a different point
The cyclic dependency of systemd → cryptsetup → lvm2 → udev=systemd
needs to be broken somewhere.  The previous strategy of building
cryptsetup with an lvm2 built without udev (#66856) caused the
installer.luksroot test to fail.  Instead, build lvm2 with a udev built
without cryptsetup.

Fixes #96479.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2020-09-03 12:35:56 -07:00
Janne Heß 8cf4ec8b97
nixos/systemd: Don't use apply for $PATH
When not using apply, other modules can use $PATH as a list instead of
getting a colon-separated list to each /bin directory.
2020-09-03 20:27:55 +02:00
WORLDofPEACE 18348c7829
Merge pull request #96042 from rnhmjoj/loaOf
treewide: completely remove types.loaOf
2020-09-02 08:45:37 -04:00
Orivej Desh 1a68e21d47
nixos/systemd: support adding and overriding tmpfiles.d via environment.etc (#96766)
This allows the user to configure systemd tmpfiles.d via
`environment.etc."tmpfiles.d/X.conf".text = "..."`, which after #93073
causes permission denied (with new X.conf):

```
ln: failed to create symbolic link '/nix/store/...-etc/etc/tmpfiles.d/X.conf': Permission denied
builder for '/nix/store/...-etc.drv' failed with exit code 1
```

or collision between environment.etc and systemd-default-tmpfiles
packages (with existing X.conf, such as tmp.conf):

```
duplicate entry tmpfiles.d/tmp.conf -> /nix/store/...-etc-tmp.conf
mismatched duplicate entry /nix/store/...-systemd-246/example/tmpfiles.d/tmp.conf <-> /nix/store/...-etc-tmp.conf
builder for '/nix/store/...-etc.drv' failed with exit code 1
```

Fixes #96755
2020-09-02 02:54:11 +00:00
rnhmjoj 20d491a317
treewide: completely remove types.loaOf 2020-09-02 00:42:50 +02:00
Konrad Borowski bcc93f02d2 nixos/availableKernelModules: add microsoft hid
This adds support for Microsoft USB keyboards at boot.
2020-08-29 22:28:54 +02:00
Matthew Bauer 25ac498482
Merge pull request #96404 from matthewbauer/gcc-cross
Fix cycle detected in Darwin->Linux cross GCC
2020-08-26 16:17:14 -05:00
Lassulus e453860b8f
Merge pull request #86236 from ThibautMarty/fix-nullOr-types
treewide: fix modules options types where the default is null
2020-08-26 18:21:29 +02:00
Frederik Rietdijk 081bd762e5 Merge staging-next into staging 2020-08-26 08:43:29 +02:00
Frederik Rietdijk f6286dea88 extra-utils: build a full lvm2 without udev support, fixes #96197
dmsetup was missing symbols.
https://github.com/NixOS/nixpkgs/pull/96290#issuecomment-680252830
2020-08-26 08:39:01 +02:00
Linus Heckemann 27f0ca6670 stage-1 find-libs: initialise left to empty array
declare -a is not sufficient to make the array variable actually
exist, which resulted in the script failing when the target object did
not have any DT_NEEDED entries. This in turn resulted in some
initramfs libraries not having their rpaths patched to point to
extra-utils, which in turn broke the extra-utils tests.
2020-08-25 12:10:30 +02:00
Florian Klink 40d2968ebf
Merge pull request #94354 from flokli/systemd-246
systemd: 245.6 -> 246
2020-08-24 12:42:24 +02:00
Jan Tojnar 91104b5417
Merge branch 'master' into staging-next 2020-08-23 02:00:50 +02:00
Lassulus 6a2c73031a
Merge pull request #89353 from wizeman/u/fix-zfs-ebusy
stage-1: retry mounting ZFS root a few times
2020-08-22 15:42:32 +02:00
Jörg Thalheim 4b9a2e13a8
cloud-utils: split of smaller .guest output 2020-08-18 11:37:41 +01:00
Frederik Rietdijk 0ac85bc455 Merge master into staging-next 2020-08-17 14:54:39 +02:00
Florian Klink 36a162edc3
Merge pull request #95342 from flokli/systemd-initctl
nixos/systemd: don't try to install systemd-initctl.{service,socket}
2020-08-16 17:17:18 +02:00
Vladimír Čunát 3937923f81
nixos/systemd.tmpfiles.packages: fix an edge case
symlinkJoin can break (silently) when the passed paths contain symlinks
to directories.  This should work now.

Down-side: when lib/tmpfiles.d doesn't exist for some passed package,
the error message is a little less explicit, because we never get
to the postBuild phase (and symlinkJoin doesn't provide a better way):
/nix/store/HASH-NAME/lib/tmpfiles.d: No such file or directory

Also, it seemed pointless to create symlinks for whole package trees
and using only a part of the result (usually very small part).
2020-08-16 10:23:53 +02:00
Jan Tojnar afe22f645a
Merge branch 'staging-next' into staging 2020-08-13 21:59:15 +02:00
Florian Klink 7361f6f252 nixos/boot: handle systemd-udevd being a symlink to udevadm 2020-08-13 20:51:39 +02:00
Florian Klink 4f7636dafc nixos/systemd: don't try to install systemd-initctl.{service,socket}
These are now only installed by systemd if HAVE_SYSV_COMPAT is true,
which only is the case if you set sysvinit-path and sysvrcnd-path (which
we explicitly unset in the systemd derivation for quite some time)
2020-08-13 20:51:39 +02:00
Florian Klink 2f9d719061 nixos/systemd: remove mymachines nss module from passwd: and group: lines
From the systemd release notes:

nss-mymachines lost support for resolution of users and groups, and now
only does resolution of hostnames. This functionality is now provided by
nss-systemd. Thus, the 'mymachines' entry should be removed from the
'passwd:' and 'group:' lines in /etc/nsswitch.conf (and 'systemd' added
if it is not already there).
2020-08-13 20:51:39 +02:00
Florian Klink e37941e4e4 nixos/systemd: don't try to install systemd-initctl.{service,socket}
Since systemd 246, these are only installed by systemd if
HAVE_SYSV_COMPAT is true, which only is the case if you set
sysvinit-path and sysvrcnd-path (which we explicitly unset in the
systemd derivation for quite some time)
2020-08-13 17:00:24 +02:00
Jörg Thalheim 28224516e5
Merge pull request #92964 from lopsided98/initrd-secrets-compressor 2020-08-13 07:54:01 +01:00
Frederik Rietdijk 7e189c56a1 Merge staging-next into staging 2020-08-13 08:52:48 +02:00
John Ericson 70d68f0478 nixos/systemd: Add support for listenDatagrams
This works exactly analogously to the existing `listenStreams`.
2020-08-11 22:08:19 +00:00
Frederik Rietdijk 46ee7ddcad Merge staging-next into staging 2020-08-11 10:26:59 +02:00
zowoq 8fb410c0ad nixos/*: editorconfig fixes 2020-08-08 10:54:16 +10:00
Frederik Rietdijk cb634b2aa5 Merge staging-next into staging 2020-08-07 23:02:23 +02:00
Jörg Thalheim ba930d8679
nixos/modules: remove trailing whitespace
This leads to ci failure otherwise if the file gets changed.
git-blame can ignore whitespace changes.
2020-08-07 14:45:39 +01:00
Florian Klink c1f77f4544
Merge pull request #91960 from datafoo/fix-issue-91761
nixos/networkd: update options
2020-08-07 00:37:08 +02:00
Florian Klink 8e0b2b9177
Merge pull request #66856 from flokli/systemd-cryptsetup-lvm
systemd: build with cryptsetup support, add cryptsetup generators
2020-08-06 12:06:54 +02:00
volth 7503f280c8
nixos/systemd: fix TUN networking in LXC containers (#81481)
in LXC container /dev/net/tun is pre-available, "dev-net-tun.device" always fails
2020-08-05 21:12:57 -05:00
Florian Klink 72c8ed0389 systemd: build with cryptsetup and cryptsetup-generators
There's a circular dependency to systemd via cryptsetup and lvm2
(systemd -> cryptsetup -> lvm2 -> udev=systemd).

However, cryptsetup only really needs the devmapper component shipped
with lvm2. So build `pkgs.cryptsetup` with a lvm2 that doesn't come with
udev.
2020-08-05 00:46:57 +02:00
Julien Moutinho a7439821bc initrd-network: fix flushBeforeStage2 2020-08-02 09:00:11 +02:00
Maciej Krüger a7a0d79ef3
boot.loader.grub: add theme option
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>

Co-authored-by: Samuel Dionne-Riel <samuel@dionne-riel.com>
2020-08-02 04:03:45 +02:00
worldofpeace b21c16fc5e
Merge pull request #83911 from mkg20001/boot-persistence
stage-1-init: add boot.persistence option
2020-07-30 14:56:06 -04:00
Keshav Kini 5e86bba082 nixos/boot: some documentation improvements
- Give a more accurate description of how fileSystems.<name/>.neededForBoot
  works

- Give a more detailed description of how fileSystems.<name/>.encrypted.keyFile
  works
2020-07-29 14:39:21 -07:00
Vladimír Čunát 7a5c6fee0f
Merge branch 'master' into staging-next
Some rebuilds, e.g. all of haskell.
Hydra nixpkgs: ?compare=1601713
2020-07-22 08:37:19 +02:00
Florian Klink f14799c8e7
Merge pull request #93073 from helsinki-systems/tmpfiles-packages
nixos/systemd: Implement a packages option for tmpfiles
2020-07-20 23:56:41 +02:00
Janne Heß a44b2cdd3a nixos/systemd: Implement a packages option for tmpfiles
Also drop the `portables` tmpfiles because the file is missing in the
systemd derivation.
2020-07-18 00:03:47 +02:00
Jan Tojnar 821dba740e
Merge branch 'staging-next' into staging 2020-07-15 09:29:01 +02:00
Graham Christensen 64dd9c1d6a
stage-2: parameterized systemd executable
This lets users do sneaky things before systemd starts, and
permanently affect the environment in which systemd runs. For example,
we could start systemd in a non-default network namespace by setting
the systemdExecutable to a wrapper script containing:

    #!/bin/sh
    ip netns add virtual
    touch /var/run/netns/physical
    mount -o bind /proc/self/ns/net /var/run/netns/physical
    exec ip netns exec virtual systemd

_note: the above example does literally work, but there are unresolved
problems with udev and dhcp._
2020-07-14 13:18:14 -04:00
datafoo 70407f09da nixos/networkd: use assertRange with 64bits integers 2020-07-13 12:23:51 +02:00
datafoo 6e1a9bbd9b nixos/networkd: delete unnecessary new lines in config files 2020-07-13 12:23:51 +02:00
datafoo bb3ba3e515 nixos/networkd: reoder code to match networkd documentation 2020-07-13 12:23:51 +02:00
datafoo e9d13d3751 nixos/networkd: update options for systemd 245
Fix #91761
2020-07-13 12:22:55 +02:00
datafoo 1f0371c650 nixos/networkd: reformat code with a single option per line
Simplifies greatly the interpretation of commit differences.
2020-07-13 12:22:55 +02:00
Niklas Hambüchen 45325b1d24
install-grub.pl: Fix GRUB always reinstalling. (#92756)
See https://github.com/NixOS/nixpkgs/pull/92520#issuecomment-655157990.

In #85895 I accidentally introduced a comparison between an array reference
and an array, which can never be true. Fixed by dereferencing.
2020-07-13 02:16:43 +02:00
Florian Klink d3a991d410 lvm2: add multiple output support 2020-07-12 23:04:33 +02:00
Florian Klink 3b2b3f7c53 nixos/systemd: add missing defaults, make options nullable
Otherwise evaluation will fail if these are not set.
2020-07-12 12:16:26 +02:00
Florian Klink 94dc0c006e
Merge pull request #92759 from mdlayher/mdl-systemd-watchdog
nixos/systemd: add options for hardware watchdog management
2020-07-12 09:35:25 +02:00
Ben Wolsieffer ab5faea41e nixos/stage-1: fix initrd secrets with custom compressor 2020-07-11 17:24:59 -04:00
Franz Pletz a8fd3c7edd
Merge pull request #85074 from Emantor/fix/boot_kernel_module
boot.initrd.luks: remove x86_64/i586 AES modules
2020-07-11 19:20:32 +02:00
Edmund Wu 3b0608d679
nixos/networkd: correct DHCPv6 UseDNS field 2020-07-10 21:47:51 -04:00
Matt Layher f9ea9c7299 nixos/systemd: add options for hardware watchdog management 2020-07-08 21:43:12 -04:00
Florian Klink 9e5f550af9 nixos/networkd: fix systemd.network.netdevs.<name>.vlanConfig.Id example
This is passed as integer, not string.

Closes https://github.com/NixOS/nixpkgs/issues/91172.
2020-07-08 20:43:17 +02:00
worldofpeace dfe59e0ab2
Merge pull request #90307 from eyJhb/luks-pre-post
boot.initrd.luks.devices: add preOpenCommands and postOpenCommands
2020-07-08 03:02:29 -04:00
Niklas Hambüchen e8c1ff9ef8
grub: Fix incorrect upgrade to new jsonStateLine. See #85895. (#92520)
Fixes error

    Can't use an undefined value as an ARRAY reference at /nix/store/...-install-grub.pl line 642, <FILE> line 5.

with `/boot/grub/state` being:

```
grub
2.04
no
/dev/sda
/boot

```

I am not sure where the trailing empty line can come from; the script does not
seem to write it. In any case, now we handle that situation as well.

Further, ensure that `extraGrubInstallArgs` defaults to the empty array
if its key is not present in the `jsonState`.
2020-07-07 00:53:55 +02:00
Martin Weinelt 8f357b7144
Merge pull request #81224 from chkno/modprobe-dot-d
nixos/modprobe: Fix modprobe configuration manpage link
2020-07-06 23:31:11 +02:00
Chuck 70cb417bac nixos/modprobe: Fix modprobe configuration manpage link 2020-07-06 13:50:11 -07:00
Niklas Hambüchen a90ae331ec install-grub.pl: Add errno messages to all or die errors.
For example, turns the error

    cannot copy /nix/store/g24xsmmsz46hzi6whv7qwwn17myn3jfq-grub-2.04/share/grub/unicode.pf2 to /boot

into the more useful

    cannot copy /nix/store/g24xsmmsz46hzi6whv7qwwn17myn3jfq-grub-2.04/share/grub/unicode.pf2 to /boot: Read-only file system
2020-07-06 22:07:46 +02:00
Niklas Hambüchen 8665b5ab91 grub: Add boot.loader.grub.extraGrubInstallArgs option.
Useful for when you need to build grub modules into your grub kernel
to get a working boot, as shown in the added example.

To store this new value, we switch to more structural JSON approach.

Using one line per value to store in `/boot/grub/state` gets really messy when
the values are arrays, or even worse, can contain newlines (escaping would be
needed). Further, removing a value from the file would get extra messy
(empty lines we'd have to keep for backwards compatibility).

Thus, from now on we use JSON to store all values we'll need in the future.
2020-07-06 22:07:46 +02:00
Niklas Hambüchen 81c15742ce install-grub.pl: Write state file atomically.
Other files were already written atomically, but not this one.
2020-07-06 22:07:46 +02:00
Niklas Hambüchen 0dcad215fe install-grub.pl: Refactor: Extract getList() 2020-07-06 22:07:46 +02:00
Lassulus e0f07f9b8d
Merge pull request #63165 from CRTified/module/initrd-ovpn
nixos/system/boot/initrd-openvpn: New openvpn options for initrd
2020-07-05 14:32:52 +02:00
Samuel Gräfenstein 850d7d1790
nixos/*: wether -> whether 2020-07-04 15:17:03 +02:00
Graham Christensen 105e63469d
Merge pull request #91344 from ElvishJerricco/zfs-encryption-systemd-ask-password
ZFS: Ask for stage 2 encryption passwords using systemd-ask-password
2020-07-02 14:15:18 -04:00
Will Fancher c128229dce plymouth: Enable systemd-ask-password-plymouth 2020-07-02 13:50:23 -04:00
Michele Guerini Rocco dab676b2d7
Merge pull request #65231 from buckley310/grub-password
grub: add support for passwords
2020-07-01 09:04:30 +02:00
CRTified c684398c6a nixos/system/boot/initrd-openvpn: Add openvpn options for initrd
nixos/tests/initrd-openvpn: Add test for openvpn in the initramfs

The module in this commit adds new options that allows the
integration of an OpenVPN client into the initrd.
This can be used e.g. to remotely unlock LUKS devices.

This commit also adds two tests for `boot.initrd.network.openvpn`.
The first one is a basic test to validate that a failing connection
does not prevent the machine from booting.

The second test validates that this module actually creates a valid
openvpn connection.
For this, it spawns three nodes:

  - The client that uses boot.initrd.network.openvpn
  - An OpenVPN server that acts as gateway and forwards a port
    to the client
  - A node that is external to the OpenVPN network

The client connects to the OpenVPN server and spawns a netcat instance
that echos a value to every client.
Afterwards, the external node checks if it receives this value over the
forwarded port on the OpenVPN gateway.
2020-07-01 00:08:55 +02:00
Fabian Möller c07a6f8743
nixos/generic-extlinux-compatible: fix docbook syntax 2020-06-23 20:51:02 +02:00
Sean Buckley 37ec7c488a
grub: add support for passwords
This patch adds support for user accounts/passwords in GRUB 2.
When configured, everything but the default option is password-protected.
2020-06-23 19:01:43 +02:00
Florian Klink 387f3b58d2 hardware.deviceTree: add name
This can be used to explicitly specify a specific dtb file, relative to
the dtb base.

Update the generic-extlinux-compatible module to make use of this option.
2020-06-21 13:48:22 +02:00
Florian Klink bd8137aef1 extlinux-conf-builder.sh: allow a custom FDT to be specified
Some bootloaders might not properly detect the model.
If the specific model is known by configuration, provide a way to
explicitly point to a specific dtb in the extlinux.conf.
2020-06-21 13:48:22 +02:00
Florian Klink 54129e72b4 nixos/generic-extlinux-compatible: introduce boot.loader.generic-extlinux-compatible.populateCmd
This option exposes the builder command used to populate an image,
honoring all options except the -c <path-to-default-configuration>
argument.

Useful to have for sdImage.populateRootCommands.

Special care needs to be taken w.r.t cross - the populate command runs
on the host platform, the activation script on the build platform (so
the builders differ)
2020-06-21 13:41:22 +02:00
Samuel Dionne-Riel f203b8b277
Merge pull request #90119 from samueldr/feature/rpi4-fixups
Fix some pitfalls from the Raspberry Pi 4 specific sd image
2020-06-20 17:02:51 -04:00
Florian Klink a84cbb60f0
Merge pull request #91073 from danielfullmer/systemd-string-format
nixos/systemd-boot: fix incorrect string formatting
2020-06-19 02:23:25 +02:00
Daniel Fullmer 1d4dc149df nixos/systemd-boot: fix incorrect string formatting
Currently, this always writes "default nixos-generation-%d.conf" without
replacing the "%d" in the string.
Python .format() is not equivalent to "%"
2020-06-18 19:58:50 -04:00
Florian Klink 1c9e02b911 nixos/systemd: allow str in systemd.services.<name>.path
Turns out, #75510 was too restrictive.

We also need to allow str here, as some modules set this to
"/run/wrappers" to bring `/run/wrappers/bin` into $PATH of a unit.
2020-06-19 00:02:51 +02:00
Florian Klink c1e7366483
Merge pull request #75510 from helsinki-systems/systemd-path-type
nixos/systemd: Use a proper type for unit paths
2020-06-18 23:50:42 +02:00
Florian Klink d72530162a
Merge pull request #90604 from maralorn/systemd-oneshot-warning
nixos/systemd: Update warning for restarting oneshots
2020-06-16 23:35:22 +02:00
Malte Brandy 0d4134de4a
nixos/systemd: Update warning for restarting oneshots
Restart= can be anything other than on-success and always for onehost units as of
10e72727ee
which is contained in systemd 245.
2020-06-16 22:10:12 +02:00
Florian Klink 42b92250b9 nixos/systemd-boot: fix default boot entry selection
6cd12ebcfe
changed behaviour - now the "default" entry needs to identity an entry
with its full name, including the ".conf".

Reported-In: https://github.com/NixOS/nixpkgs/issues/86422
2020-06-15 20:22:45 -04:00
Daniel Fullmer 7e3519a7cf nixos/systemd-boot: update bootloader if needed 2020-06-15 20:22:45 -04:00
eyjhb 7279428096
boot.initrd.luks.devices: add preOpenCommands and postOpenCommands 2020-06-14 12:03:00 +02:00
Arian van Putten 71a6d32c18 nixos/timesyncd: Make dbus-activatible
Upstream has this alias too; so that dbus activation works.
What I don't fully understand is why this would ever be useful given
this unit is already started way in early boot; even before dbus is up.
But lets just keep behaviour similar to upstream and then ask these
questions to upstream.
2020-06-13 12:23:45 +02:00
Arian van Putten cd1dedac67 nixos/networkd: Make activatible through dbus and netlink
With this systemd buffers netlink messages in early boot from the kernel
itself; and passes them on to networkd for processing once it's started.
Makes sure no routing messages are missed.

Also makes an alias so that dbus can activate this unit. Upstream has
this too.
2020-06-13 12:23:45 +02:00
Arian van Putten 0e18e5db99 nixos/resolved: Include dbus alias of resolved unit
This will make dbus socket activation for it work

When `systemd-resolved` is restarted; this would lead to unavailability
of DNS lookups.  You're supposed to use DBUS socket activation to buffer
resolved requests; such that restarts happen without downtime
2020-06-13 12:23:45 +02:00
Samuel Dionne-Riel 37e50ca635 raspberrypi-builder: ensure scripts fails on error
The way this ends up being called with the raspberry pi 4 image builder
ends up not using the `-e` from the shebang.

In turn, the builds fails during cross-compilation. The wrong coreutils
ends up being used, but this is not made apparent.

The issue I faced is already fixed on master, but this ensures no one
ends up with a failed build "succeeding".
2020-06-11 14:41:18 -04:00
Frederik Rietdijk dc33419285 Merge master into staging-next 2020-06-08 12:06:12 +02:00
worldofpeace d508591039
Merge pull request #86163 from worldofpeace/wallpaper-refactor
Gnome and Pantheon: install nixos wallpapers
2020-06-07 14:44:59 -04:00
Nadrieril e4f445008e
boot: fix order of arguments for hasPrefix (#89751) 2020-06-07 18:43:15 +02:00
Frederik Rietdijk 43f71029cc Merge master into staging-next 2020-06-05 19:40:53 +02:00
Ben Wolsieffer 14eceb5991 nixos/grub: support initrd secrets 2020-06-04 18:30:46 -04:00
Ben Wolsieffer 50a5e5597a nixos/stage-1: make boot.initrd.secrets appear in the manual 2020-06-04 17:53:29 -04:00
Ricardo M. Correia e234135995 stage-1: retry mounting ZFS root a few times
This is a workaround for NixOS/nixpkgs#25383
2020-06-04 14:55:10 +02:00
Florian Klink cfb4d0dfe3
Merge pull request #84032 from teto/fix_kernel_merge
Fix kernel configuration merge
2020-05-22 13:32:22 +02:00
Florian Klink 4e2715934e nixos/binfmt: move systemd-binfmt.service to binfmt module
Also, remove the dangling systemd.services.systemd-binfmt.wants = [
"proc-sys-fs-binfmt_misc.mount" ]; in systemd.nix.

If boot.binfmt.registrations != {}, systemd will install
proc-sys-fs-binfmt_misc.automount, which will auto-mount
`/proc/sys/fs/binfmt_misc` as soon as systemd-binfmt tries to access it.

Fixes https://github.com/NixOS/nixpkgs/issues/87687
Fixes https://github.com/NixOS/nixops/issues/574
2020-05-18 11:18:44 +02:00