Commit graph

176 commits

Author SHA1 Message Date
Pascal Bach ee50c21488
Merge pull request #111768 from misuzu/gitlab-runner-warnings
nixos/gitlab-runner: warn about possible secrets leak
2021-08-02 21:41:50 +02:00
Sandro d5bd34ebf2
treewide: convert phases that contain ":" to dont* = true (#130500) 2021-07-18 23:42:48 +02:00
Franz Pletz f73efb9fb3
Merge pull request #121841 from Pamplemousse/jenkins-cli
jenkins: Create the `jenkins-cli` command
2021-07-06 14:45:29 +00:00
Pamplemousse 4265efef54 nixos/modules/jenkins: Add option to add CLI
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-06-14 20:16:20 -07:00
Bjørn Forsman 4bcb22e17a nixos/jenkins-job-builder: add support for folder jobs
Add support for folder jobs
(https://plugins.jenkins.io/cloudbees-folder/) by reworking the service
to support nested jobs.

This also fixes this deprecation warning (as a happy side effect):

  WARNING:jenkins_jobs.cli.subcommand.test:(Deprecated) The default output behavior of `jenkins-jobs test` when given the --output flag will change in JJB 3.0. Instead of writing jobs to OUTPUT/jobname; they will be written to OUTPUT/jobname/config.xml. The new behavior can be enabled by the passing `--config-xml` parameter
2021-06-03 19:29:57 +02:00
Jörg Thalheim 846f44e880
Merge pull request #121667 from Mic92/buildkite
nixos/buildkite-agents: fix race-condition when installing secrets
2021-06-01 09:31:23 +02:00
Martin Weinelt 7bd65d54f7 treewide: remove nand0p as maintainer
While looking at the sphinx package I noticed it was heavily
undermaintained, which is when we noticed nand0p has been inactive for
roughly 18 months. It is therefore prudent to assume they will not be
maintaining their packages, modules and tests.

- Their last contribution to nixpkgs was in 2019/12
- On 2021/05/08 I wrote them an email to the address listed in the
  maintainer-list, which they didn't reply to.
2021-05-17 01:50:49 +02:00
Jörg Thalheim 503b937542
nixos/buildkite-agents: fix race-condition when installing secrets 2021-05-05 06:56:06 +02:00
Robert Hensing 519a435b08 nixos/hercules-ci-agent: Set default labels 2021-05-04 16:29:05 +02:00
Robert Hensing 4abd56732e nixos/hercules-ci-agent: Set default concurrency to auto 2021-05-04 16:28:31 +02:00
Viktor Kronvall c01046b022 services.buildkite-agents: support multi-tags
The buildkite agent supports multiple tags with the same key. This
functionality is used to have a [single agent listen on multiple
queues](https://buildkite.com/docs/agent/v3/queues#setting-an-agents-queue).

However, having the tags be of type `attrsOf str` means that
we cannot suport this use case. This commit modifies the type
of tags to be `attrsOf (either str (listOf str))` where the list
is expanded into multiple tags with the same key.

Example:
```
{tags = {queue = ["default", "testing"];};}
```
generates
```
tags="queue=default,queue=testing"
```
in the buildkite agent configuration.
2021-04-22 21:23:52 +09:00
Vincent Haupert f4af2f267a
nixos/github-runner: init at v2.277.1 (#116775)
* github-runner: init at 2.277.1

* nixos/github-runner: initial version

* nixos/github-runner: add warning if tokenFile in Nix store

* github-runner: don't accept unexpected attrs

* github-runner: formatting nits

* github-runner: add pre and post hooks to checkPhase

* nixos/github-runner: update ExecStartPre= comment

* nixos/github-runner: adapt tokenFile option description

Also note that not only a change to the option value will trigger a
reconfiguration but also modifications to the file's content.

* nixos/github-runner: remove mkDefault for DynamicUser=

* nixos/github-runner: create a parent for systemd dirs

Adds a parent directory "github-runner/" to all of the systemd lifecycle
directories StateDirectory=, RuntimeDirectory= and LogDirectory=.

Doing this has two motivations:

1. Something like this would required if we want to support multiple
   runners configurations. Please note that this is already possible
   using NixOS containers.
2. Having an additional parent directory makes it easier to remap
   any of the directories. Without a parent, systemd is going to
   complain if, for example, the given StateDirectory= is a symlink.

* nixos/github-runner: use specifier to get abs runtime path

* nixos/github-runner: use hostname as default for option `name`

Until now, the runner registration did not set the `--name` argument if
the configuration option was `null`, the default for the option.
According to GitHub's documentation, this instructs the registration
script to use the machine's hostname.

This commit causes the registration to always pass the `--name` argument
to the runner configuration script. The option now defaults to
`networking.hostName` which should be always set on NixOS.

This change becomes necessary as the systemd service name includes the
name of the runner since fcfa809 and, hence, expects it to be set. Thus,
an unset `name` option leads to an error.

* nixos/github-runner: use types.str for `name` option

Forcing a `name` option to comply with a pattern which could also be
used as a hostname is probably not required by GitHub.

* nixos/github-runner: pass dir paths explicitly for ExecStartPre=

* nixos/github-runner: update variable and script naming

* nixos/github-runner: let systemd choose the user/group

User and group naming restrictions are a complex topic [1] that I don't
even want to touch. Let systemd figure out the username and group and
reference it in our scripts through the USER environment variable.

[1] https://systemd.io/USER_NAMES/

* Revert "nixos/github-runner: use types.str for `name` option"

The escaping applied to the subdirectory paths given to StateDirectory=,
RuntimeDirectory= and LogsDirectory= apparently doesn't use the same
strategy that is used to escape unit names (cf. systemd-escape(1)). This
makes it unreasonably hard to construct reliable paths which work for
StateDirectory=/RuntimeDirectory=/LogsDirectory= and ExecStartPre=.

Against this background, I decided to (re-)apply restrictions to the
name a user might give for the GitHub runner. The pattern for
`networking.hostName` seems like a reasonable choice, also as its value
is the default if the `name` option isn't set.

This reverts commit 193ac67ba337990c22126da24a775c497dbc7e7d.

* nixos/github-runner: use types.path for `tokenFile` option

* nixos/github-runner: escape options used as shell arguments

* nixos/github-runner: wait for network-online.target

* github-runner: ignore additional online tests
2021-04-10 10:17:10 +00:00
Jörg Thalheim 96e4052d70
nixos/hydra: gc-check-reachability no longer exists in nix 2.4 (#117925)
* nixos/hydra: gc-check-reachability no longer exists in nix 2.4

fixup! nixos/hydra: gc-check-reachability no longer exists in nix 2.4

* Update nixos/modules/services/continuous-integration/hydra/default.nix

Co-authored-by: Sandro <sandro.jaeckel@gmail.com>

Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2021-03-29 11:14:16 +02:00
Cole Helbling 10d513e633
nixos/hydra: append application_name to HYDRA_DBI
This will make it easier to track specifically where queries are being
made from (assuming a `log_line_prefix` that includes `%a` in the
postgres configuration).
2021-03-03 16:04:00 -08:00
Robert Hensing 696294ea9f Improve todo 2021-02-10 10:36:00 +01:00
Robert Hensing 460ba46a4a nixos/hercules-ci-agent: Add myself to meta.maintainers 2021-02-10 09:50:27 +01:00
Robert Hensing 0c22957751 nixos/hercules-ci-agent: Improve concurrentTasks option doc 2021-02-10 09:49:57 +01:00
Robert Hensing fbabab7b16 nixos/hercules-ci-agent: Format with nixpkgs-fmt 2021-02-10 09:49:21 +01:00
Robert Hensing c808983caa nixos/hercules-ci-agent: Remove patchNix 2021-02-06 15:35:42 +01:00
misuzu bdc96d3351 nixos/gitlab-runner: warn about possible secrets leak 2021-02-03 17:18:49 +02:00
Ben Wolsieffer 945a461ac2 nixos/buildbot: fix type for pythonPackages
The pythonPackages option is a function that returns a list of packages, but the
type was specified as just a list of packages.
2021-01-31 17:40:12 -05:00
Ben Wolsieffer d877c9b39c nixos/buildbot: add lopsided98 as maintainer 2021-01-31 17:40:11 -05:00
Sandro 548dc3cc62
Merge pull request #111125 from helsinki-systems/gocd-agent
nixos/gocd-agent nixos/gocd-server: add types
2021-01-30 02:22:03 +01:00
Scriptkiddi c1b25f2529
nixos/gocd-agent nixos/gocd-server: add types 2021-01-29 12:59:55 +01:00
Gabriel Gonzalez 80f6769ebc
Don't createHome for the hydra user (#110722)
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2021-01-29 09:22:49 +00:00
misuzu 1e38f0bdf9
nixos/gitlab-runner: add support for 'docker+machine' and 'docker-ssh+machine' executors (#108763) 2021-01-26 00:01:54 +00:00
Fritz Otlinghaus 72691e14ec
nixos/buildbot: add pythonPackages type 2021-01-24 13:20:35 +01:00
Graham Christensen bc49a0815a
utillinux: rename to util-linux 2020-11-24 12:42:06 -05:00
lf- b37bbca521 nixos/modules: fix systemd start rate-limits
These were broken since 2016:
f0367da7d1
since StartLimitIntervalSec got moved into [Unit] from [Service].
StartLimitBurst has also been moved accordingly, so let's fix that one
too.

NixOS systems have been producing logs such as:
/nix/store/wf98r55aszi1bkmln1lvdbp7znsfr70i-unit-caddy.service/caddy.service:31:
Unknown key name 'StartLimitIntervalSec' in section 'Service', ignoring.

I have also removed some unnecessary duplication in units disabling
rate limiting since setting either interval or burst to zero disables it
(ad16158c10/src/basic/ratelimit.c (L16))
2020-10-31 01:35:56 -07:00
Maximilian Bosch 1308817e05
nixos/hydra: remove hydra-migration upgrade path
This should NOT be backported to 20.09!

When 21.03 is released, the DB changes are about a year old and
operators had two release cycles for the upgrade. At this point it
should be fair to remove the compat layer to reduce the complexity of
the module itself.
2020-10-21 18:03:04 +02:00
Augustin Borsu 685ae84cb0 nixos/jenkins: switch to openjdk11 as openjdk14 is not supported 2020-10-04 11:45:02 +02:00
Robert Hensing 346a1b0ec6 nixos/hercules-ci-agent: init 2020-08-23 20:13:15 +02:00
Milan e49fb87b05
nixos/gitlab-runner: add clone-url option (#93894) 2020-07-30 10:24:33 +02:00
Roman Sharapov d53d13b6ee nixos/buildbot: enable configurable keepalive for buildbot worker
In the current implementation, there's no possibility to modify the default
parameter for keepalive. This is a number that indicates how frequently
keepalive messages should be sent from the worker to the buildmaster,
expressed in seconds. The default (600) causes a message to be sent to
the buildmaster at least once every 10 minutes.

If the worker is behind a NAT box or stateful firewall, these messages
may help to keep the connection alive: some NAT boxes tend to forget about
a connection if it has not been used in a while. When this happens, the
buildmaster will think that the worker has disappeared, and builds will
time out. Meanwhile the worker will not realize than anything is wrong.
2020-07-13 13:07:20 -04:00
misuzu fc9f994ee5
nixos/gitlab-runner: add more global options (#86946) 2020-06-29 13:35:21 +00:00
Florian Klink 9538bf50ae nixos/buildbot-master: support reporters, migrate away from status
Since Buildbot 0.9.0, status targets were deprecated and ignored.
There's a very small line on startup explaining that, and status simply
isn't reported. Avoid others the same headaches, and do it right in the
NixOS module.

As there might have been changes in the way reporters are organized, and
configuration might need to be migrated remove the old option, and not
just provide an alias.
2020-06-18 16:49:26 +02:00
Florian Klink 1fd972bd50 nixos/buildbot-master: fix typo in services.buildbot-master.bpPort
It's pbPort, and it's also a connection string, meaning
listen-on-localhost is also possible. Provide an alias for the old
option name, so old configs still work.
2020-06-05 01:29:03 +02:00
misuzu 0b0afcae16 nixos/gitlab-runner: support multiple services 2020-05-02 11:59:57 +02:00
zowoq c59c4e3589 nixos/*: use $out instead of $bin with buildGoPackage 2020-04-28 20:30:29 +10:00
Frederik Rietdijk f8c6921e97
hydra: wrap executables with hydra env vars
We already set the relevant env vars in the systemd services. That does
not help one when executing any of the executables outside a service,
e.g. when creating a new user.
2020-04-11 14:36:42 +02:00
Maximilian Bosch 0f5c38feed
hydra: 2020-03-24 -> 2020-04-07
Also removed `pkgs.hydra-flakes` since flake-support has been merged
into master[1]. Because of that, `pkgs.hydra-unstable` is now compiled
against `pkgs.nixFlakes` and currently requires a patch since Hydra's
master doesn't compile[2] atm.

[1] https://github.com/NixOS/hydra/pull/730
[2] https://github.com/NixOS/hydra/pull/732
2020-04-07 14:11:12 +02:00
Jörg Thalheim 212b574d89
Merge pull request #81298 from Mic92/buildkite
nixos/buildkite-agents: don't run as nogroup
2020-04-02 13:39:34 +01:00
Florian Klink 9faea55282
Merge pull request #74379 from bachp/gitlab-runner-reload
nixos/gitlab-runner: reload on config change
2020-03-31 20:18:28 +02:00
Maximilian Bosch bd5324c4fc
hydra: 2020-02-06 -> 2020-03-{24,27}
Upgrades Hydra to the latest master/flake branch. To perform this
upgrade, it's needed to do a non-trivial db-migration which provides a
massive performance-improvement[1].

The basic ideas behind multi-step upgrades of services between NixOS versions
have been gathered already[2]. For further context it's recommended to
read this first.

Basically, the following steps are needed:

* Upgrade to a non-breaking version of Hydra with the db-changes
  (columns are still nullable here). If `system.stateVersion` is set to
  something older than 20.03, the package will be selected
  automatically, otherwise `pkgs.hydra-migration` needs to be used.

* Run `hydra-backfill-ids` on the server.

* Deploy either `pkgs.hydra-unstable` (for Hydra master) or
  `pkgs.hydra-flakes` (for flakes-support) to activate the optimization.

The steps are also documented in the release-notes and in the module
using `warnings`.

`pkgs.hydra` has been removed as latest Hydra doesn't compile with
`pkgs.nixStable` and to ensure a graceful migration using the newly
introduced packages.

To verify the approach, a simple vm-test has been added which verifies
the migration steps.

[1] https://github.com/NixOS/hydra/pull/711
[2] https://github.com/NixOS/nixpkgs/pull/82353#issuecomment-598269471
2020-03-28 23:33:25 +01:00
Jörg Thalheim 8b7f4fa8a6
nixos/buildkite-agents: don't run as nogroup 2020-02-28 15:34:37 +00:00
Yorick van Pelt 1b351f81f4
nixos/buildkite-agents: fix hooksDir assertion 2020-02-19 12:22:35 +01:00
Yorick van Pelt e242eccb0b
nixos/buildkite-agents: update release notes 2020-02-10 13:36:30 +01:00
Yorick van Pelt f003810989
nixos/buildkite-agents: support multiple buildkite agents 2020-02-10 13:35:14 +01:00
Florian Klink 0daae2e08c nixos/buildkite: drop user option (#78160)
* nixos/buildkite: drop user option

This reverts 8c6b1c3eaa.

Turns out, buildkite-agent has logic to write .ssh/known_hosts files and
only really works when $HOME and the user homedir are in sync.

On top of that, we provision ssh keys in /var/lib/buildkite-agent, which
doesn't work if that other users' homedir points elsewhere (we can cheat
by setting $HOME, but then getent and $HOME provide conflicting
results).

So after all, it's better to only run the system-wide buildkite agent as
the "buildkite-agent" user only - if one wants to run buildkite as
different users, systemd user services might be a better fit.

* nixosTests.buildkite-agent: add node with separate user and no ssh key
2020-01-21 13:21:57 +00:00
Florian Klink 4b73d3c444 nixos/buildkite: make privateSshKeyPath optional
When only cloning public repos, or when the ssh key is provided by
different means, we don't need to manage it here.
2020-01-20 11:10:56 +01:00