Commit graph

162 commits

Author SHA1 Message Date
Cole Helbling 10d513e633
nixos/hydra: append application_name to HYDRA_DBI
This will make it easier to track specifically where queries are being
made from (assuming a `log_line_prefix` that includes `%a` in the
postgres configuration).
2021-03-03 16:04:00 -08:00
Robert Hensing 696294ea9f Improve todo 2021-02-10 10:36:00 +01:00
Robert Hensing 460ba46a4a nixos/hercules-ci-agent: Add myself to meta.maintainers 2021-02-10 09:50:27 +01:00
Robert Hensing 0c22957751 nixos/hercules-ci-agent: Improve concurrentTasks option doc 2021-02-10 09:49:57 +01:00
Robert Hensing fbabab7b16 nixos/hercules-ci-agent: Format with nixpkgs-fmt 2021-02-10 09:49:21 +01:00
Robert Hensing c808983caa nixos/hercules-ci-agent: Remove patchNix 2021-02-06 15:35:42 +01:00
Ben Wolsieffer 945a461ac2 nixos/buildbot: fix type for pythonPackages
The pythonPackages option is a function that returns a list of packages, but the
type was specified as just a list of packages.
2021-01-31 17:40:12 -05:00
Ben Wolsieffer d877c9b39c nixos/buildbot: add lopsided98 as maintainer 2021-01-31 17:40:11 -05:00
Sandro 548dc3cc62
Merge pull request #111125 from helsinki-systems/gocd-agent
nixos/gocd-agent nixos/gocd-server: add types
2021-01-30 02:22:03 +01:00
Scriptkiddi c1b25f2529
nixos/gocd-agent nixos/gocd-server: add types 2021-01-29 12:59:55 +01:00
Gabriel Gonzalez 80f6769ebc
Don't createHome for the hydra user (#110722)
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2021-01-29 09:22:49 +00:00
misuzu 1e38f0bdf9
nixos/gitlab-runner: add support for 'docker+machine' and 'docker-ssh+machine' executors (#108763) 2021-01-26 00:01:54 +00:00
Fritz Otlinghaus 72691e14ec
nixos/buildbot: add pythonPackages type 2021-01-24 13:20:35 +01:00
Graham Christensen bc49a0815a
utillinux: rename to util-linux 2020-11-24 12:42:06 -05:00
lf- b37bbca521 nixos/modules: fix systemd start rate-limits
These were broken since 2016:
f0367da7d1
since StartLimitIntervalSec got moved into [Unit] from [Service].
StartLimitBurst has also been moved accordingly, so let's fix that one
too.

NixOS systems have been producing logs such as:
/nix/store/wf98r55aszi1bkmln1lvdbp7znsfr70i-unit-caddy.service/caddy.service:31:
Unknown key name 'StartLimitIntervalSec' in section 'Service', ignoring.

I have also removed some unnecessary duplication in units disabling
rate limiting since setting either interval or burst to zero disables it
(ad16158c10/src/basic/ratelimit.c (L16))
2020-10-31 01:35:56 -07:00
Maximilian Bosch 1308817e05
nixos/hydra: remove hydra-migration upgrade path
This should NOT be backported to 20.09!

When 21.03 is released, the DB changes are about a year old and
operators had two release cycles for the upgrade. At this point it
should be fair to remove the compat layer to reduce the complexity of
the module itself.
2020-10-21 18:03:04 +02:00
Augustin Borsu 685ae84cb0 nixos/jenkins: switch to openjdk11 as openjdk14 is not supported 2020-10-04 11:45:02 +02:00
Robert Hensing 346a1b0ec6 nixos/hercules-ci-agent: init 2020-08-23 20:13:15 +02:00
Milan e49fb87b05
nixos/gitlab-runner: add clone-url option (#93894) 2020-07-30 10:24:33 +02:00
Roman Sharapov d53d13b6ee nixos/buildbot: enable configurable keepalive for buildbot worker
In the current implementation, there's no possibility to modify the default
parameter for keepalive. This is a number that indicates how frequently
keepalive messages should be sent from the worker to the buildmaster,
expressed in seconds. The default (600) causes a message to be sent to
the buildmaster at least once every 10 minutes.

If the worker is behind a NAT box or stateful firewall, these messages
may help to keep the connection alive: some NAT boxes tend to forget about
a connection if it has not been used in a while. When this happens, the
buildmaster will think that the worker has disappeared, and builds will
time out. Meanwhile the worker will not realize than anything is wrong.
2020-07-13 13:07:20 -04:00
misuzu fc9f994ee5
nixos/gitlab-runner: add more global options (#86946) 2020-06-29 13:35:21 +00:00
Florian Klink 9538bf50ae nixos/buildbot-master: support reporters, migrate away from status
Since Buildbot 0.9.0, status targets were deprecated and ignored.
There's a very small line on startup explaining that, and status simply
isn't reported. Avoid others the same headaches, and do it right in the
NixOS module.

As there might have been changes in the way reporters are organized, and
configuration might need to be migrated remove the old option, and not
just provide an alias.
2020-06-18 16:49:26 +02:00
Florian Klink 1fd972bd50 nixos/buildbot-master: fix typo in services.buildbot-master.bpPort
It's pbPort, and it's also a connection string, meaning
listen-on-localhost is also possible. Provide an alias for the old
option name, so old configs still work.
2020-06-05 01:29:03 +02:00
misuzu 0b0afcae16 nixos/gitlab-runner: support multiple services 2020-05-02 11:59:57 +02:00
zowoq c59c4e3589 nixos/*: use $out instead of $bin with buildGoPackage 2020-04-28 20:30:29 +10:00
Frederik Rietdijk f8c6921e97
hydra: wrap executables with hydra env vars
We already set the relevant env vars in the systemd services. That does
not help one when executing any of the executables outside a service,
e.g. when creating a new user.
2020-04-11 14:36:42 +02:00
Maximilian Bosch 0f5c38feed
hydra: 2020-03-24 -> 2020-04-07
Also removed `pkgs.hydra-flakes` since flake-support has been merged
into master[1]. Because of that, `pkgs.hydra-unstable` is now compiled
against `pkgs.nixFlakes` and currently requires a patch since Hydra's
master doesn't compile[2] atm.

[1] https://github.com/NixOS/hydra/pull/730
[2] https://github.com/NixOS/hydra/pull/732
2020-04-07 14:11:12 +02:00
Jörg Thalheim 212b574d89
Merge pull request #81298 from Mic92/buildkite
nixos/buildkite-agents: don't run as nogroup
2020-04-02 13:39:34 +01:00
Florian Klink 9faea55282
Merge pull request #74379 from bachp/gitlab-runner-reload
nixos/gitlab-runner: reload on config change
2020-03-31 20:18:28 +02:00
Maximilian Bosch bd5324c4fc
hydra: 2020-02-06 -> 2020-03-{24,27}
Upgrades Hydra to the latest master/flake branch. To perform this
upgrade, it's needed to do a non-trivial db-migration which provides a
massive performance-improvement[1].

The basic ideas behind multi-step upgrades of services between NixOS versions
have been gathered already[2]. For further context it's recommended to
read this first.

Basically, the following steps are needed:

* Upgrade to a non-breaking version of Hydra with the db-changes
  (columns are still nullable here). If `system.stateVersion` is set to
  something older than 20.03, the package will be selected
  automatically, otherwise `pkgs.hydra-migration` needs to be used.

* Run `hydra-backfill-ids` on the server.

* Deploy either `pkgs.hydra-unstable` (for Hydra master) or
  `pkgs.hydra-flakes` (for flakes-support) to activate the optimization.

The steps are also documented in the release-notes and in the module
using `warnings`.

`pkgs.hydra` has been removed as latest Hydra doesn't compile with
`pkgs.nixStable` and to ensure a graceful migration using the newly
introduced packages.

To verify the approach, a simple vm-test has been added which verifies
the migration steps.

[1] https://github.com/NixOS/hydra/pull/711
[2] https://github.com/NixOS/nixpkgs/pull/82353#issuecomment-598269471
2020-03-28 23:33:25 +01:00
Jörg Thalheim 8b7f4fa8a6
nixos/buildkite-agents: don't run as nogroup 2020-02-28 15:34:37 +00:00
Yorick van Pelt 1b351f81f4
nixos/buildkite-agents: fix hooksDir assertion 2020-02-19 12:22:35 +01:00
Yorick van Pelt e242eccb0b
nixos/buildkite-agents: update release notes 2020-02-10 13:36:30 +01:00
Yorick van Pelt f003810989
nixos/buildkite-agents: support multiple buildkite agents 2020-02-10 13:35:14 +01:00
Florian Klink 0daae2e08c nixos/buildkite: drop user option (#78160)
* nixos/buildkite: drop user option

This reverts 8c6b1c3eaa.

Turns out, buildkite-agent has logic to write .ssh/known_hosts files and
only really works when $HOME and the user homedir are in sync.

On top of that, we provision ssh keys in /var/lib/buildkite-agent, which
doesn't work if that other users' homedir points elsewhere (we can cheat
by setting $HOME, but then getent and $HOME provide conflicting
results).

So after all, it's better to only run the system-wide buildkite agent as
the "buildkite-agent" user only - if one wants to run buildkite as
different users, systemd user services might be a better fit.

* nixosTests.buildkite-agent: add node with separate user and no ssh key
2020-01-21 13:21:57 +00:00
Florian Klink 4b73d3c444 nixos/buildkite: make privateSshKeyPath optional
When only cloning public repos, or when the ssh key is provided by
different means, we don't need to manage it here.
2020-01-20 11:10:56 +01:00
Florian Klink 70308a7daf nixos/buildkite-agent: add gnutar, gzip and git to runtimePackages
These are required for nix to do builtins.fetchTarball and
builtins.fetchGit, so most likely we want them to be around.
2020-01-20 10:30:24 +01:00
Florian Klink 7838f00824 nixos/buildkite: stop using deprecated option 2020-01-20 10:30:22 +01:00
Florian Klink 8c6b1c3eaa nixos/buildkite-agent: add "user" option
This allows buildkite-agent to run as another user.

It'll still run builds from /var/lib/buildkite-agent and setup things in
there.
2020-01-19 21:21:15 +01:00
Florian Klink 63c310c07e nixos/buildkite-agent: change option meta-data into tags attrset
This is called tags in the buildkite 3.
2020-01-17 23:11:26 +01:00
Yorick van Pelt e1ad9d1685 nixos/buildkite-agent: add a shell attribute to specify the shell being used
This gets passed to BUILDKITE_SHELL, which will specify the shell being
used to executes script in.

Defaults to `${pkgs.bash}/bin/bash -e -c`, matching how buildkite
behaves on other distros.
2020-01-17 18:40:11 +01:00
Florian Klink 99d0f4c380 nixos/buildkite-agent: tweak TimeoutStopSec and KillMode
This improves behaviour when the service is being stopped.
2020-01-17 18:40:11 +01:00
Florian Klink 9237767277 nixos/buildkite-agent: actually use services.buildkite-agent.package
We were currently just using pkgs.buildkite-agent, no matter what was
configured in services.buildkite-agent.package
2020-01-17 18:40:11 +01:00
Florian Klink c154dd5101 nixos/buildkite-agent: stop deploying public keys, make ssh private key optional.
SSH public keys aren't needed to clone private repos, and if we only
need to configure a single attribute, there's no need for the "openssh"
attrset anymore.
2020-01-17 18:40:11 +01:00
Mark Karpov a8cdfd30ff
Make buildkite agent service compatible with buildkite-agent3 2020-01-15 17:16:44 +01:00
Bas van Dijk b0b37569a7
Merge pull request #77669 from basvandijk/hydra-sync-buildMachinesFiles-with-upstream
hydra: only set buildMachinesFiles when nix.buildMachines is defined
2020-01-15 09:39:25 +01:00
Bas van Dijk ad554b3e4e hydra: only set buildMachinesFiles when nix.buildMachines is defined
This applies [hydra PR #432](https://github.com/NixOS/hydra/pull/432)
to the NixOS module in nixpkgs:

```
commit 4efd078977e5ea20e1104783efc324cba11690bc
Author: Bas van Dijk <v.dijk.bas@gmail.com>
Date:   Sun Dec 11 15:35:38 2016 +0100

    Only set buildMachinesFiles when nix.buildMachines is defined
```
2020-01-14 12:32:40 +01:00
Bas van Dijk f34aaa6646 hydra: remove the build-use-substitutes option
The following commit from 2016 in hydra removed the `--option
build-use-substitutes` from the hydra-queue-runner service:

```
commit ee2e9f5335c8c0288c102975b506f6b275793cfe
Author: Eelco Dolstra <edolstra@gmail.com>
Date:	Fri Oct 7 20:23:05 2016 +0200

    Update to reflect BinaryCacheStore changes

    BinaryCacheStore no longer implements buildPaths() and ensurePath(),
    so we need to use copyPath() / copyClosure().
```

It would be better if the hydra module in NixOS matches the upstream
module.
2020-01-14 12:14:25 +01:00
worldofpeace 2af959e579 nixos/jenkins/slave: fix eval 2020-01-12 15:17:05 -05:00
worldofpeace 79de730e77 nixos/buildbot: fix eval
Fixes https://github.com/NixOS/nixpkgs/issues/77309
2020-01-08 10:06:08 -05:00