nixos/kubernetes: adding manual section for kubernetes and writing release note for NixOS 19.03
This commit is contained in:
parent
e2380e79e1
commit
8d62d7972f
|
@ -23,5 +23,6 @@
|
||||||
<xi:include href="linux-kernel.xml" />
|
<xi:include href="linux-kernel.xml" />
|
||||||
<xi:include href="../generated/modules.xml" xpointer="xpointer(//section[@id='modules']/*)" />
|
<xi:include href="../generated/modules.xml" xpointer="xpointer(//section[@id='modules']/*)" />
|
||||||
<xi:include href="profiles.xml" />
|
<xi:include href="profiles.xml" />
|
||||||
|
<xi:include href="kubernetes.xml" />
|
||||||
<!-- Apache; libvirtd virtualisation -->
|
<!-- Apache; libvirtd virtualisation -->
|
||||||
</part>
|
</part>
|
||||||
|
|
127
nixos/doc/manual/configuration/kubernetes.xml
Normal file
127
nixos/doc/manual/configuration/kubernetes.xml
Normal file
|
@ -0,0 +1,127 @@
|
||||||
|
<chapter xmlns="http://docbook.org/ns/docbook"
|
||||||
|
xmlns:xlink="http://www.w3.org/1999/xlink"
|
||||||
|
xmlns:xi="http://www.w3.org/2001/XInclude"
|
||||||
|
version="5.0"
|
||||||
|
xml:id="sec-kubernetes">
|
||||||
|
<title>Kubernetes</title>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
The NixOS Kubernetes module is a collective term for a handful of
|
||||||
|
individual submodules implementing the Kubernetes cluster components.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
There are generally two ways of enabling Kubernetes on NixOS.
|
||||||
|
One way is to enable and configure cluster components appropriately by hand:
|
||||||
|
<programlisting>
|
||||||
|
services.kubernetes = {
|
||||||
|
apiserver.enable = true;
|
||||||
|
controllerManager.enable = true;
|
||||||
|
scheduler.enable = true;
|
||||||
|
addonManager.enable = true;
|
||||||
|
proxy.enable = true;
|
||||||
|
flannel.enable = true;
|
||||||
|
};
|
||||||
|
</programlisting>
|
||||||
|
Another way is to assign cluster roles ("master" and/or "node") to the host.
|
||||||
|
This enables apiserver, controllerManager, scheduler, addonManager,
|
||||||
|
kube-proxy and etcd:
|
||||||
|
<programlisting>
|
||||||
|
<xref linkend="opt-services.kubernetes.roles"/> = [ "master" ];
|
||||||
|
</programlisting>
|
||||||
|
While this will enable the kubelet and kube-proxy only:
|
||||||
|
<programlisting>
|
||||||
|
<xref linkend="opt-services.kubernetes.roles"/> = [ "node" ];
|
||||||
|
</programlisting>
|
||||||
|
Assigning both the master and node roles is usable if you want a single
|
||||||
|
node Kubernetes cluster for dev or testing purposes:
|
||||||
|
<programlisting>
|
||||||
|
<xref linkend="opt-services.kubernetes.roles"/> = [ "master" "node" ];
|
||||||
|
</programlisting>
|
||||||
|
Note: Assigning either role will also default both
|
||||||
|
<xref linkend="opt-services.kubernetes.flannel.enable"/> and
|
||||||
|
<xref linkend="opt-services.kubernetes.easyCerts"/> to true.
|
||||||
|
This sets up flannel as CNI and activates automatic PKI bootstrapping.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
As of kubernetes 1.10.X it has been deprecated to open
|
||||||
|
non-tls-enabled ports on kubernetes components. Thus, from NixOS 19.03 all
|
||||||
|
plain HTTP ports have been disabled by default.
|
||||||
|
While opening insecure ports is still possible, it is recommended not to
|
||||||
|
bind these to other interfaces than loopback.
|
||||||
|
|
||||||
|
To re-enable the insecure port on the apiserver, see options:
|
||||||
|
<xref linkend="opt-services.kubernetes.apiserver.insecurePort"/>
|
||||||
|
and
|
||||||
|
<xref linkend="opt-services.kubernetes.apiserver.insecureBindAddress"/>
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<note>
|
||||||
|
<para>
|
||||||
|
As of NixOS 19.03, it is mandatory to configure:
|
||||||
|
<xref linkend="opt-services.kubernetes.masterAddress"/>.
|
||||||
|
The masterAddress must be resolveable and routeable by all cluster nodes.
|
||||||
|
In single node clusters, this can be set to <literal>localhost</literal>.
|
||||||
|
</para>
|
||||||
|
</note>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
Role-based access control (RBAC) authorization mode is enabled by default.
|
||||||
|
This means that anonymous requests to the apiserver secure port will
|
||||||
|
expectedly cause a permission denied error. All cluster components must
|
||||||
|
therefore be configured with x509 certificates for two-way tls communication.
|
||||||
|
The x509 certificate subject section determines the roles and permissions
|
||||||
|
granted by the apiserver to perform clusterwide or namespaced operations.
|
||||||
|
See also:
|
||||||
|
<link
|
||||||
|
xlink:href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/">
|
||||||
|
Using RBAC Authorization</link>.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
The NixOS kubernetes module provides an option for automatic certificate
|
||||||
|
bootstrapping and configuration,
|
||||||
|
<xref linkend="opt-services.kubernetes.easyCerts"/>.
|
||||||
|
The PKI bootstrapping process involves setting up a certificate authority
|
||||||
|
(CA) daemon (cfssl) on the kubernetes master node. cfssl generates a CA-cert
|
||||||
|
for the cluster, and uses the CA-cert for signing subordinate certs issued to
|
||||||
|
each of the cluster components. Subsequently, the certmgr daemon monitors
|
||||||
|
active certificates and renews them when needed. For single node Kubernetes
|
||||||
|
clusters, setting <xref linkend="opt-services.kubernetes.easyCerts"/> = true
|
||||||
|
is sufficient and no further action is required. For joining extra node
|
||||||
|
machines to an existing cluster on the other hand, establishing initial trust
|
||||||
|
is mandatory.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
To add new nodes to the cluster:
|
||||||
|
On any (non-master) cluster node where
|
||||||
|
<xref linkend="opt-services.kubernetes.easyCerts"/> is enabled, the helper
|
||||||
|
script <literal>nixos-kubernetes-node-join</literal> is available on PATH.
|
||||||
|
Given a token on stdin, it will copy the token to the kubernetes
|
||||||
|
secrets directory and restart the certmgr service. As requested
|
||||||
|
certificates are issued, the script will restart kubernetes cluster
|
||||||
|
components as needed for them to pick up new keypairs.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
<note>
|
||||||
|
<para>
|
||||||
|
Multi-master (HA) clusters are not supported by the easyCerts module.
|
||||||
|
</para>
|
||||||
|
</note>
|
||||||
|
|
||||||
|
<para>
|
||||||
|
In order to interact with an RBAC-enabled cluster as an administrator, one
|
||||||
|
needs to have cluster-admin privileges. By default, when easyCerts is
|
||||||
|
enabled, a cluster-admin kubeconfig file is generated and linked into
|
||||||
|
<literal>/etc/kubernetes/cluster-admin.kubeconfig</literal> as determined by
|
||||||
|
<xref linkend="opt-services.kubernetes.pki.etcClusterAdminKubeconfig"/>.
|
||||||
|
<literal>export KUBECONFIG=/etc/kubernetes/cluster-admin.kubeconfig</literal>
|
||||||
|
will make kubectl use this kubeconfig to access and authenticate the cluster.
|
||||||
|
The cluster-admin kubeconfig references an auto-generated keypair owned by
|
||||||
|
root. Thus, only root on the kubernetes master may obtain cluster-admin
|
||||||
|
rights by means of this file.
|
||||||
|
</para>
|
||||||
|
|
||||||
|
</chapter>
|
|
@ -54,6 +54,13 @@
|
||||||
</itemizedlist>
|
</itemizedlist>
|
||||||
<para>to <literal>false</literal> and enable your preferred display manager.</para>
|
<para>to <literal>false</literal> and enable your preferred display manager.</para>
|
||||||
</note>
|
</note>
|
||||||
|
<para>
|
||||||
|
A major refactoring of the Kubernetes module has been completed.
|
||||||
|
Refactorings primarily focus on decoupling components and enhancing
|
||||||
|
security. Two-way TLS and RBAC has been enabled by default for all
|
||||||
|
components, which slightly changes the way the module is configured.
|
||||||
|
See: <xref linkend="sec-kubernetes"/> for details.
|
||||||
|
</para>
|
||||||
</listitem>
|
</listitem>
|
||||||
</itemizedlist>
|
</itemizedlist>
|
||||||
</section>
|
</section>
|
||||||
|
@ -564,6 +571,40 @@
|
||||||
provisioning.
|
provisioning.
|
||||||
</para>
|
</para>
|
||||||
</listitem>
|
</listitem>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
The use of insecure ports on kubernetes has been deprecated.
|
||||||
|
Thus options:
|
||||||
|
<varname>services.kubernetes.apiserver.port</varname> and
|
||||||
|
<varname>services.kubernetes.controllerManager.port</varname>
|
||||||
|
has been renamed to <varname>.insecurePort</varname>,
|
||||||
|
and default of both options has changed to 0 (disabled).
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
Note that the default value of
|
||||||
|
<varname>services.kubernetes.apiserver.bindAddress</varname>
|
||||||
|
has changed from 127.0.0.1 to 0.0.0.0, allowing the apiserver to be
|
||||||
|
accessible from outside the master node itself.
|
||||||
|
If the apiserver insecurePort is enabled,
|
||||||
|
it is strongly recommended to only bind on the loopback interface. See:
|
||||||
|
<varname>services.kubernetes.apiserver.insecurebindAddress</varname>.
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
The option <varname>services.kubernetes.apiserver.allowPrivileged</varname>
|
||||||
|
and <varname>services.kubernetes.kubelet.allowPrivileged</varname> now
|
||||||
|
defaults to false. Disallowing privileged containers on the cluster.
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
|
<listitem>
|
||||||
|
<para>
|
||||||
|
The kubernetes module does no longer add the kubernetes package to
|
||||||
|
<varname>environment.systemPackages</varname> implicitly.
|
||||||
|
</para>
|
||||||
|
</listitem>
|
||||||
</itemizedlist>
|
</itemizedlist>
|
||||||
</section>
|
</section>
|
||||||
</section>
|
</section>
|
||||||
|
|
Loading…
Reference in a new issue