infra/home-cluster
2
0 Fork 0
Code Issues Pull Requests 40 Actions Packages Projects Releases Wiki Activity

feat: distribution is a dict and nodes and cluster are top level (#1288)

Browse Source 
* feat: distribution is a dict and nodes and cluster are top level

Signed-off-by: Devin Buhl <devin@buhl.casa>

* chore: housekeeping

Signed-off-by: Devin Buhl <devin@buhl.casa>

---------

Signed-off-by: Devin Buhl <devin@buhl.casa>
This commit is contained in:
Devin Buhl authored and GitHub committed 2024-02-01 20:31:42 -05:00
1 parent cba35dc66a
commit 89673cd17b
55 files changed
+330 -302
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/tests/config-k0s.yaml
+17 -14
View File
@@ -1,23 +1,26 @@
---
skip_tests: true
distribution: k0s
distribution:
type: k0s
timezone: Etc/UTC
nodes:
host_network: 10.10.10.0/24
dns_servers: ["1.1.1.1"]
search_domain: "fake"
inventory:
- name: k8s-controller-0
address: 10.10.10.100
controller: true
ssh_username: fake
- name: k8s-worker-0
address: 10.10.10.101
controller: false
ssh_username: fake
cluster:
nodes:
host_network: 10.10.10.0/24
dns_servers: ["1.1.1.1"]
search_domain: "fake"
inventory:
- name: k8s-controller-0
address: 10.10.10.100
controller: true
ssh_username: fake
- name: k8s-worker-0
address: 10.10.10.101
controller: false
ssh_username: fake
pod_network: 10.69.0.0/16
service_network: 10.96.0.0/16
endpoint_vip: 10.10.10.254
.github/tests/config-k3s-ipv4.yaml
+18 -14
View File
@@ -1,23 +1,27 @@
---
skip_tests: true
distribution: k3s
distribution:
type: k3s
timezone: Etc/UTC
nodes:
host_network: 10.10.10.0/24
dns_servers: ["1.1.1.1"]
search_domain: "fake"
inventory:
- name: k8s-controller-0
address: 10.10.10.100
controller: true
ssh_username: fake
- name: k8s-worker-0
address: 10.10.10.101
controller: false
ssh_username: fake
cluster:
nodes:
host_network: 10.10.10.0/24
dns_servers: ["1.1.1.1"]
search_domain: "fake"
inventory:
- name: k8s-controller-0
address: 10.10.10.100
controller: true
ssh_username: fake
- name: k8s-worker-0
address: 10.10.10.101
controller: false
ssh_username: fake
pod_network: 10.69.0.0/16
service_network: 10.96.0.0/16
endpoint_vip: 10.10.10.254
.github/tests/config-k3s-ipv6.yaml
+17 -14
View File
@@ -1,23 +1,26 @@
---
skip_tests: true
distribution: k3s
distribution:
type: k3s
timezone: Etc/UTC
nodes:
host_network: 10.10.10.0/24
dns_servers: ["1.1.1.1"]
search_domain: "fake"
inventory:
- name: k8s-controller-0
address: 10.10.10.100
controller: true
ssh_username: fake
- name: k8s-worker-0
address: 10.10.10.101
controller: false
ssh_username: fake
cluster:
nodes:
host_network: 10.10.10.0/24
dns_servers: ["1.1.1.1"]
search_domain: "fake"
inventory:
- name: k8s-controller-0
address: 10.10.10.100
controller: true
ssh_username: fake
- name: k8s-worker-0
address: 10.10.10.101
controller: false
ssh_username: fake
pod_network: 10.42.0.0/16,fd7f:8f5:e87c:a::/64
service_network: 10.43.0.0/16,fd7f:8f5:e87c:e::/112
endpoint_vip: 10.10.10.254
.github/tests/config-talos.yaml
+28 -25
View File
@@ -1,34 +1,37 @@
---
skip_tests: true
distribution: talos
distribution:
type: talos
talos:
schematics:
enabled: true
id: df491c50a5acc05b977ef00c32050e1ceb0df746e40b33c643ac8a9bfb7c7263
customization: |-
extraKernelArgs:
- net.ifnames=0
systemExtensions:
officialExtensions:
- siderolabs/intel-ucode
- siderolabs/i915-ucode
timezone: Etc/UTC
nodes:
host_network: 10.10.10.0/24
dns_servers: ["1.1.1.1"]
search_domain: "fake"
inventory:
- name: k8s-controller-0
address: 10.10.10.100
controller: true
talos_disk_device: fake
- name: k8s-worker-0
address: 10.10.10.101
controller: false
talos_disk_device: fake
cluster:
nodes:
host_network: 10.10.10.0/24
dns_servers: ["1.1.1.1"]
search_domain: "fake"
inventory:
- name: k8s-controller-0
address: 10.10.10.100
controller: true
talos_disk_device: fake
- name: k8s-worker-0
address: 10.10.10.101
controller: false
talos_disk_device: fake
talos:
schematics:
enabled: true
id: df491c50a5acc05b977ef00c32050e1ceb0df746e40b33c643ac8a9bfb7c7263
customization: |-
extraKernelArgs:
- net.ifnames=0
systemExtensions:
officialExtensions:
- siderolabs/intel-ucode
- siderolabs/i915-ucode
pod_network: 10.69.0.0/16
service_network: 10.96.0.0/16
endpoint_vip: 10.10.10.254
README.md
+6 -61
View File
@@ -214,7 +214,7 @@ You have two different options for setting up your local workstation.
### 🔧 Stage 3: Bootstrap configuration
> [!NOTE]
> The [config.sample.yaml](./config.sample.yaml) file contain necessary information that is **vital** to the bootstrap process.
> The [config.sample.yaml](./config.sample.yaml) file contains config that is **vital** to the bootstrap process.
1. Generate the `config.yaml` from the [config.sample.yaml](./config.sample.yaml) configuration file.
@@ -222,72 +222,17 @@ You have two different options for setting up your local workstation.
task init
```
#### 🔧 Stage 3: Flux
2. Fill out the `config.yaml` configuration file using the comments in that file as a guide.
📍 _Using [SOPS](https://github.com/getsops/sops) with [Age](https://github.com/FiloSottile/age) allows us to encrypt secrets and use them with Flux._
1. Create a Age private / public key (this file is gitignored)
```sh
task sops:age-keygen
```
2. Fill out the appropriate vars in `config.yaml`
#### Stage 3: Flux with Cloudflare DNS
> [!NOTE]
> To use `cert-manager` with the Cloudflare DNS challenge you will need to create a API Token.
1. Head over to Cloudflare and create a API Token by going [here](https://dash.cloudflare.com/profile/api-tokens).
2. Under the `API Tokens` section click the blue `Create Token` button.
3. Click the blue `Use template` button for the `Edit zone DNS` template.
4. Name your token something like `home-kubernetes`
5. Under `Permissions`, click `+ Add More` and add each permission below:
```text
Zone - DNS - Edit
Account - Cloudflare Tunnel - Read
```
6. Limit the permissions to a specific account and zone resources.
7. Fill out the appropriate vars in `config.yaml`
#### Stage 3: Flux with Cloudflare Tunnel
> [!NOTE]
> To expose services to the internet you will need to create a [Cloudflare Tunnel](https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/).
1. Authenticate cloudflared to your domain
```sh
cloudflared tunnel login
```
2. Create the tunnel
```sh
cloudflared tunnel create k8s
```
3. In the `~/.cloudflared` directory there will be a json file with details you need. Ignore the `cert.pem` file.
4. Fill out the appropriate vars in `config.yaml`
#### Stage 3: Finishing up
1. Complete filling out the rest of the `config.yaml` configuration file.
2. Once done run the following command which will verify and generate all the files needed to continue.
3. Run the following command which will generate all the files needed to continue.
```sh
task configure
```
3. Push you changes to git
4. Push you changes to git
📍 **Verify** all the `*.sops.yaml` and `*.sops.yaml` files under `./kubernetes` directory is **encrypted** with SOPS
📍 _**Verify** all the `./kubernetes/**/*.sops.*` files are **encrypted** with SOPS_
```sh
git add -A
@@ -295,7 +240,7 @@ You have two different options for setting up your local workstation.
git push
```
4. Continue on to ⚡ [**Stage 4**](#-stage-4-prepare-your-nodes-for-kubernetes)
5. Continue on to ⚡ [**Stage 4**](#-stage-4-prepare-your-nodes-for-kubernetes)
### ⚡ Stage 4: Prepare your nodes for Kubernetes
bootstrap/scripts/plugin.py
+11 -1
View File
@@ -6,10 +6,15 @@ from typing import Any
from typing import Any
from netaddr import IPNetwork
from bcrypt import hashpw, gensalt
import makejinja
import validation
def encrypt(value: str) -> str:
return hashpw(value.encode(), gensalt(rounds=10)).decode("ascii")
def nthhost(value: str, query: int) -> str:
value = IPNetwork(value)
try:
@@ -20,6 +25,7 @@ def nthhost(value: str, query: int) -> str:
return False
return value
def import_filter(file: Path) -> Callable[[dict[str, Any]], bool]:
module_path = file.relative_to(Path.cwd()).with_suffix("")
module_name = str(module_path).replace("/", ".")
@@ -31,6 +37,7 @@ def import_filter(file: Path) -> Callable[[dict[str, Any]], bool]:
spec.loader.exec_module(module)
return module.main
class Plugin(makejinja.plugin.Plugin):
def __init__(self, data: dict[str, Any], config: makejinja.config.Config):
self._data = data
@@ -45,12 +52,15 @@ class Plugin(makejinja.plugin.Plugin):
validation.validate(data)
def filters(self) -> makejinja.plugin.Filters:
return [nthhost]
return [encrypt, nthhost]
def path_filters(self):
return [self._mjfilter_func]
def _mjfilter_func(self, path: Path) -> bool:
return not any(
path.is_relative_to(excluded_dir) for excluded_dir in self._excluded_dirs
bootstrap/scripts/validation.py
+11 -8
View File
@@ -10,7 +10,6 @@ GLOBAL_CLI_TOOLS = ["age", "cloudflared", "flux", "sops", "jq", "kubeconform", "
TALOS_CLI_TOOLS = ["talosctl", "talhelper"]
K0S_CLI_TOOLS = ["k0sctl"]
def required(*keys: str):
def wrapper_outter(func: Callable):
@wraps(func)
@@ -22,7 +21,6 @@ def required(*keys: str):
return wrapper
return wrapper_outter
def _validate_network(network: str, family: int) -> str:
try:
network = netaddr.IPNetwork(network)
@@ -40,9 +38,10 @@ def validate_python_version() -> None:
@required("distribution")
def validate_cli_tools(distribution: str, **_) -> None:
if distribution not in DISTRIBUTIONS:
raise ValueError(f"Invalid distribution {distribution}")
def validate_cli_tools(distribution: dict, **_) -> None:
distro = distribution.get("type")
if distro not in DISTRIBUTIONS:
raise ValueError(f"Invalid distribution {distro}")
for tool in GLOBAL_CLI_TOOLS:
if not which(tool):
raise ValueError(f"Missing required CLI tool {tool}")
@@ -55,9 +54,10 @@ def validate_cli_tools(distribution: str, **_) -> None:
@required("distribution")
def validate_distribution(distribution: str, **_) -> None:
if distribution not in DISTRIBUTIONS:
raise ValueError(f"Invalid distribution {distribution}")
def validate_distribution(distribution: dict, **_) -> None:
distro = distribution.get("type")
if distro not in DISTRIBUTIONS:
raise ValueError(f"Invalid distribution {distro}")
@required("timezone")
@@ -98,6 +98,9 @@ def validate_cluster_networks(cluster: dict, feature_gates: dict, **_) -> None:
def massage_config(data: dict) -> dict:
data["distribution"] = data.get("distribution", {})
data["nodes"] = data.get("nodes", [])
data["cluster"] = data.get("cluster", {})
data["flux"] = data.get("flux", {})
data["cloudflare"] = data.get("cloudflare", {})
data["feature_gates"] = data.get("feature_gates", {})
bootstrap/templates/.sops.yaml.j2
+2 -2
View File
@@ -1,7 +1,7 @@
{% if flux.enabled %}
---
creation_rules:
{% if distribution in ['talos'] %}
{% if distribution.type in ["talos"] %}
- # IMPORTANT: This rule MUST be above the others
path_regex: talos/.*\.sops\.ya?ml
key_groups:
@@ -13,7 +13,7 @@ creation_rules:
key_groups:
- age:
- "{{ flux.sops_age_public_key }}"
{% if distribution in ['k0s', 'k3s'] %}
{% if distribution.type in ["k0s", "k3s"] %}
- path_regex: ansible/.*\.sops\.ya?ml
key_groups:
- age:
bootstrap/templates/ansible/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("distribution") in ['k0s', 'k3s']
main = lambda data: data.get("distribution", {}).get("type", "k3s") in ["k0s", "k3s"]
bootstrap/templates/ansible/inventory/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("distribution") in ['k0s', 'k3s']
main = lambda data: data.get("distribution", {}).get("type", "k3s") in ["k0s", "k3s"]
bootstrap/templates/ansible/inventory/group_vars/kubernetes/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("distribution") in ['k3s']
main = lambda data: data.get("distribution", {}).get("type", "k3s") in ["k3s"]
bootstrap/templates/ansible/inventory/group_vars/master/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("distribution") in ['k3s']
main = lambda data: data.get("distribution", {}).get("type", "k3s") in ["k3s"]
bootstrap/templates/ansible/inventory/group_vars/worker/.mjfilter.py
+10 -1
View File
@@ -1 +1,10 @@
main = lambda data: data.get("distribution") in ['k3s']
main = lambda data: (
data.get("distribution").get("type", "k3s") in ["k3s"] and
len(
list(
filter(
lambda item: "controller" in item and item["controller"] is False, data.get("nodes").get("inventory")
)
)
) > 0
)
bootstrap/templates/ansible/inventory/hosts.yaml.j2
+3 -3
View File
@@ -3,17 +3,17 @@ kubernetes:
children:
master:
hosts:
{% for item in cluster.nodes.inventory %}
{% for item in nodes.inventory %}
{% if item.controller %}
"{{ item.name }}":
ansible_user: "{{ item.ssh_username }}"
ansible_host: "{{ item.address }}"
{% endif %}
{% endfor %}
{% if cluster.nodes.inventory | selectattr('controller', 'equalto', False) | list | length %}
{% if nodes.inventory | selectattr('controller', 'equalto', False) | list | length %}
worker:
hosts:
{% for item in cluster.nodes.inventory %}
{% for item in nodes.inventory %}
{% if not item.controller %}
"{{ item.name }}":
ansible_user: "{{ item.ssh_username }}"
bootstrap/templates/ansible/playbooks/cluster-installation.yaml.j2
+1 -1
View File
@@ -1,4 +1,4 @@
{% if distribution in ['k3s'] %}
{% if distribution.type in ["k3s"] %}
---
- name: Cluster Installation
hosts: kubernetes
bootstrap/templates/ansible/playbooks/cluster-kube-vip.yaml.j2
+3 -3
View File
@@ -10,7 +10,7 @@
ansible.builtin.pause:
seconds: 5
tasks:
{% if distribution in ['k3s'] %}
{% if distribution.type in ["k3s"] %}
- name: Ensure Kubernetes is running
ansible.builtin.include_role:
name: xanmanning.k3s
@@ -21,9 +21,9 @@
- name: Upgrade kube-vip
ansible.builtin.template:
src: templates/kube-vip-ds.yaml.j2
{% if distribution in ['k3s'] %}
{% if distribution.type in ["k3s"] %}
dest: "{% raw %}{{ k3s_server_manifests_dir }}{% endraw %}/kube-vip-ds.yaml"
{% elif distribution in ['k0s'] %}
{% elif distribution.type in ["k0s"] %}
dest: "/var/lib/k0s/manifests/kube-vip/kube-vip-ds.yaml"
{% endif %}
mode: preserve
bootstrap/templates/ansible/playbooks/cluster-nuke.yaml.j2
+1 -1
View File
@@ -1,4 +1,4 @@
{% if distribution in ['k3s'] %}
{% if distribution.type in ["k3s"] %}
---
- name: Cluster Nuke
hosts: kubernetes
bootstrap/templates/ansible/playbooks/cluster-prepare.yaml.j2
+3 -3
View File
@@ -1,4 +1,4 @@
{% if distribution in ['k3s'] %}
{% if distribution.type in ["k3s"] %}
---
- name: Prepare System
hosts: kubernetes
@@ -62,8 +62,8 @@
mode: '0644'
dest: /etc/resolv.conf
content: |
search {{ cluster.nodes.search_domain|default('.', true) }}
{% for item in cluster.nodes.dns_servers %}
search {{ nodes.search_domain|default('.', true) }}
{% for item in nodes.dns_servers | default(['1.1.1.1', '1.0.0.1']) %}
nameserver {{ item }}
{% endfor %}
bootstrap/templates/ansible/playbooks/cluster-rollout-update.yaml.j2
+8 -8
View File
@@ -12,9 +12,9 @@
seconds: 5
tasks:
- name: Details
{% if distribution in ['k3s'] %}
{% if distribution.type in ["k3s"] %}
ansible.builtin.command: "k3s kubectl get node {% raw %}{{ inventory_hostname }}{% endraw %} -o json"
{% elif distribution in ['k0s'] %}
{% elif distribution.type in ["k0s"] %}
ansible.builtin.command: "k0s kubectl get node {% raw %}{{ inventory_hostname }}{% endraw %} -o json"
{% endif %}
register: kubectl_get_node
@@ -32,9 +32,9 @@
- name: Cordon
kubernetes.core.k8s_drain:
name: "{% raw %}{{ inventory_hostname }}{% endraw %}"
{% if distribution in ['k3s'] %}
{% if distribution.type in ["k3s"] %}
kubeconfig: /etc/rancher/k3s/k3s.yaml
{% elif distribution in ['k0s'] %}
{% elif distribution.type in ["k0s"] %}
kubeconfig: /var/lib/k0s/pki/admin.conf
{% endif %}
state: cordon
@@ -43,9 +43,9 @@
- name: Drain
kubernetes.core.k8s_drain:
name: "{% raw %}{{ inventory_hostname }}{% endraw %}"
{% if distribution in ['k3s'] %}
{% if distribution.type in ["k3s"] %}
kubeconfig: /etc/rancher/k3s/k3s.yaml
{% elif distribution in ['k0s'] %}
{% elif distribution.type in ["k0s"] %}
kubeconfig: /var/lib/k0s/pki/admin.conf
{% endif %}
state: drain
@@ -78,9 +78,9 @@
- name: Uncordon
kubernetes.core.k8s_drain:
name: "{% raw %}{{ inventory_hostname }}{% endraw %}"
{% if distribution in ['k3s'] %}
{% if distribution.type in ["k3s"] %}
kubeconfig: /etc/rancher/k3s/k3s.yaml
{% elif distribution in ['k0s'] %}
{% elif distribution.type in ["k0s"] %}
kubeconfig: /var/lib/k0s/pki/admin.conf
{% endif %}
state: uncordon
bootstrap/templates/ansible/playbooks/tasks/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("distribution") in ['k3s']
main = lambda data: data.get("distribution", {}).get("type", "k3s") in ["k3s"]
bootstrap/templates/ansible/playbooks/templates/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("distribution") in ['k3s']
main = lambda data: data.get("distribution", {}).get("type", "k3s") in ["k3s"]
bootstrap/templates/ansible/requirements.yaml.j2
+1 -1
View File
@@ -8,7 +8,7 @@ collections:
version: 8.3.0
- name: kubernetes.core
version: 3.0.0
{% if distribution in ['k3s'] %}
{% if distribution.type in ["k3s"] %}
roles:
- name: xanmanning.k3s
src: https://github.com/PyratLabs/ansible-role-k3s
bootstrap/templates/kubernetes/apps/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("flux").get("enabled") == True
main = lambda data: data.get("flux", {}).get("enabled", False) == True
bootstrap/templates/kubernetes/apps/cert-manager/cert-manager/issuers/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("cloudflare").get("enabled") == True
main = lambda data: data.get("cloudflare", {}).get("enabled", False) == True
bootstrap/templates/kubernetes/apps/flux-system/.mjfilter.py
-1
View File
@@ -1 +0,0 @@
main = lambda data: data.get("flux").get("github").get("webhook").get("enabled") == True
bootstrap/templates/kubernetes/apps/flux-system/kustomization.yaml.j2
+2
View File
@@ -3,4 +3,6 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ./namespace.yaml
{% if flux.github.webhook.enabled %}
- ./webhooks/ks.yaml
{% endif %}
bootstrap/templates/kubernetes/apps/flux-system/webhooks/.mjfilter.py
+6
View File
@@ -0,0 +1,6 @@
main = lambda data: (
data.get("flux", {})
.get("github", {})
.get("webhook", {})
.get("enabled", False) == True
)
bootstrap/templates/kubernetes/apps/kube-system/kubelet-csr-approver/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("distribution") in ['talos']
main = lambda data: data.get("distribution", {}).get("type", "k3s") in ["talos"]
bootstrap/templates/kubernetes/apps/kube-system/kustomization.yaml.j2
+2 -2
View File
@@ -4,11 +4,11 @@ kind: Kustomization
resources:
- ./namespace.yaml
- ./cilium/ks.yaml
{% if distribution in ['talos'] %}
{% if distribution.type in ["talos"] %}
- ./kubelet-csr-approver/ks.yaml
{% endif %}
- ./metrics-server/ks.yaml
{% if distribution in ['k0s', 'talos'] %}
{% if distribution.type in ["k0s", "talos"] %}
- ./spegel/ks.yaml
{% endif %}
- ./reloader/ks.yaml
bootstrap/templates/kubernetes/apps/kube-system/spegel/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("distribution") in ['k0s', 'talos']
main = lambda data: data.get("distribution", {}).get("type", "k3s") in ["k0s", "talos"]
bootstrap/templates/kubernetes/apps/kube-system/spegel/app/helmrelease.yaml.j2
+2 -2
View File
@@ -24,10 +24,10 @@ spec:
keepHistory: false
values:
spegel:
{% if distribution in ['k0s'] %}
{% if distribution.type in ["k0s"] %}
containerdSock: /run/k0s/containerd.sock
containerdRegistryConfigPath: /var/lib/k0s/containerd/certs.d
{% elif distribution in ['talos'] %}
{% elif distribution.type in ["talos"] %}
containerdSock: /run/containerd/containerd.sock
containerdRegistryConfigPath: /etc/cri/conf.d/hosts
{% endif %}
bootstrap/templates/kubernetes/apps/network/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("cloudflare").get("enabled") == True
main = lambda data: data.get("cloudflare", {}).get("enabled", False) == True
bootstrap/templates/kubernetes/apps/system-upgrade/k0s/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("distribution") in ['k0s']
main = lambda data: data.get("distribution", {}).get("type", "k3s") in ["k0s"]
bootstrap/templates/kubernetes/apps/system-upgrade/k0s/app/plan.yaml.j2
+1 -1
View File
@@ -18,7 +18,7 @@ spec:
controllers:
discovery:
selector: {}
{% if cluster.nodes.inventory | selectattr('controller', 'equalto', False) | list | length %}
{% if nodes.inventory | selectattr('controller', 'equalto', False) | list | length %}
workers:
discovery:
selector: {}
bootstrap/templates/kubernetes/apps/system-upgrade/k3s/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("distribution") in ['k3s']
main = lambda data: data.get("distribution", {}).get("type", "k3s") in ["k3s"]
bootstrap/templates/kubernetes/apps/system-upgrade/kustomization.yaml.j2
+4 -4
View File
@@ -3,15 +3,15 @@ apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ./namespace.yaml
{% if distribution in ['k3s', 'talos'] %}
{% if distribution.type in ['k3s', 'talos'] %}
- ./system-upgrade-controller/ks.yaml
{% endif %}
{% if distribution in ['k0s'] %}
{% if distribution.type in ["k0s"] %}
- ./k0s/ks.yaml
{% endif %}
{% if distribution in ['k3s'] %}
{% if distribution.type in ["k3s"] %}
- ./k3s/ks.yaml
{% endif %}
{% if distribution in ['talos'] and cluster.nodes.talos.schematics.enabled %}
{% if distribution.type in ["talos"] and nodes.talos.schematics.enabled %}
- ./talos/ks.yaml
{% endif %}
bootstrap/templates/kubernetes/apps/system-upgrade/system-upgrade-controller/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("distribution") in ['k3s', 'talos']
main = lambda data: data.get("distribution", {}).get("type", "k3s") in ['k3s', 'talos']
bootstrap/templates/kubernetes/apps/system-upgrade/system-upgrade-controller/app/rbac.yaml.j2
+1 -1
View File
@@ -11,7 +11,7 @@ subjects:
- kind: ServiceAccount
name: system-upgrade
namespace: system-upgrade
{% if distribution in ['talos'] %}
{% if distribution.type in ["talos"] %}
---
apiVersion: talos.dev/v1alpha1
kind: ServiceAccount
bootstrap/templates/kubernetes/apps/system-upgrade/talos/.mjfilter.py
+7 -6
View File
@@ -1,6 +1,7 @@
main = lambda data: data.get("distribution") in ['talos'] and (
data.get("cluster", {})
.get("nodes", {})
.get("talos", {})
.get("schematics", {})
.get("enabled", False) == True)
main = lambda data: (
data.get("distribution", {}).get("type", "k3s") in ["talos"] and
data.get("distribution", {})
.get("talos", {})
.get("schematics", {})
.get("enabled", False) == True
)
bootstrap/templates/kubernetes/apps/system-upgrade/talos/app/plan.yaml.j2
+1 -1
View File
@@ -88,6 +88,6 @@ spec:
args:
- --nodes=$(NODE_IP)
- upgrade
- --image=factory.talos.dev/installer/{{ cluster.nodes.talos.schematics.id }}:$(SYSTEM_UPGRADE_PLAN_LATEST_VERSION)
- --image=factory.talos.dev/installer/{{ nodes.talos.schematics.id }}:$(SYSTEM_UPGRADE_PLAN_LATEST_VERSION)
- --preserve=true
- --wait=false
bootstrap/templates/kubernetes/bootstrap/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("flux").get("enabled") == True
main = lambda data: data.get("flux", {}).get("enabled", False) == True
bootstrap/templates/kubernetes/flux/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("flux").get("enabled") == True
main = lambda data: data.get("flux", {}).get("enabled", False) == True
bootstrap/templates/kubernetes/flux/repositories/helm/kustomization.yaml.j2
+2 -2
View File
@@ -12,10 +12,10 @@ resources:
- ./jetstack.yaml
- ./metrics-server.yaml
- ./openebs.yaml
{% if distribution in ['talos'] %}
{% if distribution.type in ["talos"] %}
- ./postfinance.yaml
{% endif %}
- ./stakater.yaml
{% if distribution in ['k0s', 'talos'] %}
{% if distribution.type in ["k0s", "talos"] %}
- ./xenitab.yaml
{% endif %}
bootstrap/templates/kubernetes/flux/repositories/helm/postfinance.yaml.j2
+1 -1
View File
@@ -1,4 +1,4 @@
{% if distribution in ['talos'] %}
{% if distribution.type in ["talos"] %}
---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
bootstrap/templates/kubernetes/flux/repositories/helm/xenitab.yaml.j2
+1 -1
View File
@@ -1,4 +1,4 @@
{% if distribution in ['k0s', 'talos'] %}
{% if distribution.type in ["k0s", "talos"] %}
---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
bootstrap/templates/kubernetes/flux/vars/cluster-settings.yaml.j2
+1 -1
View File
@@ -7,7 +7,7 @@ metadata:
data:
TIMEZONE: "{{ timezone }}"
CLUSTER_CIDR: "{{ cluster.pod_network.split(',')[0] }}"
NODE_CIDR: "{{ cluster.nodes.host_network }}"
NODE_CIDR: "{{ nodes.host_network }}"
{% if feature_gates.dual_stack_ipv4_first %}
CLUSTER_CIDR_V6: "{{ cluster.pod_network.split(',')[1] }}"
{% endif %}
bootstrap/templates/kubernetes/k0s/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("distribution") in ['k0s']
main = lambda data: data.get("distribution", {}).get("type", "k3s") in ["k0s"]
bootstrap/templates/kubernetes/k0s/k0sctl.yaml.j2
+1 -1
View File
@@ -5,7 +5,7 @@ metadata:
name: home-kubernetes
spec:
hosts:
{% for item in cluster.nodes.inventory %}
{% for item in nodes.inventory %}
{% if item.controller %}
- role: controller+worker
{% else %}
bootstrap/templates/kubernetes/k0s/scripts/apply-system.sh.j2
+2 -2
View File
@@ -35,8 +35,8 @@ apt-get install -y --no-install-recommends \
chattr -i /etc/resolv.conf
rm -f /etc/resolv.conf
cat <<EOF > /etc/resolv.conf
search {{ cluster.nodes.search_domain|default('.', true) }}
{% for item in cluster.nodes.dns_servers %}
search {{ nodes.search_domain|default('.', true) }}
{% for item in nodes.dns_servers | default(['1.1.1.1', '1.0.0.1']) %}
nameserver {{ item }}
{% endfor %}
EOF
bootstrap/templates/kubernetes/talos/.mjfilter.py
+1 -1
View File
@@ -1 +1 @@
main = lambda data: data.get("distribution") in ['talos']
main = lambda data: data.get("distribution", {}).get("type", "k3s") in ["talos"]
bootstrap/templates/kubernetes/talos/talconfig.yaml.j2
+15 -15
View File
@@ -22,7 +22,7 @@ cniConfig:
name: none
nodes:
{% for item in cluster.nodes.inventory %}
{% for item in nodes.inventory %}
- hostname: "{{ item.name }}"
ipAddress: "{{ item.address }}"
{% if item.talos_disk_device.startswith('/') %}
@@ -35,26 +35,26 @@ nodes:
networkInterfaces:
- interface: eth0
dhcp: false
{% if cluster.nodes.talos.vlan %}
{% if distribution.talos.vlan %}
vlans:
- vlanId: {{ cluster.nodes.talos.vlan }}
- vlanId: {{ distribution.talos.vlan }}
addresses:
- "{{ item.address }}/{{ cluster.nodes.host_network.split('/') | last }}"
- "{{ item.address }}/{{ nodes.host_network.split('/') | last }}"
mtu: 1500
routes:
- network: 0.0.0.0/0
gateway: "{{ cluster.nodes.host_network | nthhost(1) }}"
gateway: "{{ nodes.host_network | nthhost(1) }}"
{% if item.controller %}
vip:
ip: "{{ cluster.endpoint_vip }}"
{% endif %}
{% else %}
addresses:
- "{{ item.address }}/{{ cluster.nodes.host_network.split('/') | last }}"
- "{{ item.address }}/{{ nodes.host_network.split('/') | last }}"
mtu: 1500
routes:
- network: 0.0.0.0/0
gateway: "{{ cluster.nodes.host_network | nthhost(1) }}"
gateway: "{{ nodes.host_network | nthhost(1) }}"
{% if item.controller %}
vip:
ip: "{{ cluster.endpoint_vip }}"
@@ -63,10 +63,10 @@ nodes:
{% endfor %}
controlPlane:
{% if cluster.nodes.talos.schematics.enabled %}
{% if distribution.talos.schematics.enabled %}
schematic:
customization:
{{ cluster.nodes.talos.schematics.customization | indent(6, first=True) }}
{{ distribution.talos.schematics.customization | indent(6, first=True) }}
{% endif %}
patches:
# Configure containerd
@@ -110,7 +110,7 @@ controlPlane:
rotate-server-certificates: true
nodeIP:
validSubnets:
- "{{ cluster.nodes.host_network }}"
- "{{ nodes.host_network }}"
# Enable KubePrism
- &kubePrismPatch |-
@@ -125,7 +125,7 @@ controlPlane:
machine:
network:
nameservers:
{% for item in cluster.nodes.dns_servers %}
{% for item in nodes.dns_servers | default(['1.1.1.1', '1.0.0.1']) %}
- {{ item }}
{% endfor %}
@@ -165,7 +165,7 @@ controlPlane:
extraArgs:
listen-metrics-urls: http://0.0.0.0:2381
advertisedSubnets:
- "{{ cluster.nodes.host_network }}"
- "{{ nodes.host_network }}"
# Disable default API server admission plugins.
- |-
@@ -183,12 +183,12 @@ controlPlane:
allowedKubernetesNamespaces:
- system-upgrade
{% if cluster.nodes.inventory | selectattr('controller', 'equalto', False) | list | length %}
{% if nodes.inventory | selectattr('controller', 'equalto', False) | list | length %}
worker:
{% if cluster.nodes.talos.schematics.enabled %}
{% if distribution.talos.schematics.enabled %}
schematic:
customization:
{{ cluster.nodes.talos.schematics.customization | indent(6, first=True) }}
{{ distribution.talos.schematics.customization | indent(6, first=True) }}
{% endif %}
patches:
- *containerdPatch
bootstrap/templates/partials/cilium-values-full.partial.yaml.j2
+5 -5
View File
@@ -6,9 +6,9 @@ cluster:
id: 1
containerRuntime:
integration: containerd
{% if distribution in ['k3s'] %}
{% if distribution.type in ["k3s"] %}
socketPath: /var/run/k3s/containerd/containerd.sock
{% elif distribution in ['k0s'] %}
{% elif distribution.type in ["k0s"] %}
socketPath: /var/run/k0s/containerd.sock
{% endif %}
endpointRoutes:
@@ -60,13 +60,13 @@ ipv6NativeRoutingCIDR: "${CLUSTER_CIDR_V6}"
ipv6:
enabled: true
{% endif %}
{% if distribution in ['k3s'] %}
{% if distribution.type in ["k3s"] %}
k8sServiceHost: 127.0.0.1
k8sServicePort: 6444
{% elif distribution in ['k0s'] %}
{% elif distribution.type in ["k0s"] %}
k8sServiceHost: localhost
k8sServicePort: 7443
{% elif distribution in ['talos'] %}
{% elif distribution.type in ["talos"] %}
k8sServiceHost: 127.0.0.1
k8sServicePort: 7445
{% endif %}
bootstrap/templates/partials/cilium-values-init.partial.yaml.j2
+5 -5
View File
@@ -6,9 +6,9 @@ cluster:
id: 1
containerRuntime:
integration: containerd
{% if distribution in ['k3s'] %}
{% if distribution.type in ["k3s"] %}
socketPath: /var/run/k3s/containerd/containerd.sock
{% elif distribution in ['k0s'] %}
{% elif distribution.type in ["k0s"] %}
socketPath: /var/run/k0s/containerd.sock
{% endif %}
endpointRoutes:
@@ -23,13 +23,13 @@ ipv6NativeRoutingCIDR: "{{ cluster.pod_network_v6 }}"
ipv6:
enabled: true
{% endif %}
{% if distribution in ['k3s'] %}
{% if distribution.type in ["k3s"] %}
k8sServiceHost: 127.0.0.1
k8sServicePort: 6444
{% elif distribution in ['k0s'] %}
{% elif distribution.type in ["k0s"] %}
k8sServiceHost: localhost
k8sServicePort: 7443
{% elif distribution in ['talos'] %}
{% elif distribution.type in ["talos"] %}
k8sServiceHost: 127.0.0.1
k8sServicePort: 7445
{% endif %}
bootstrap/templates/partials/kubelet-csr-approver-values.partial.yaml.j2
+1 -1
View File
@@ -1,2 +1,2 @@
providerRegex: ^({{ (cluster.nodes.inventory | map(attribute='name') | join('|')) }})$
providerRegex: ^({{ (nodes.inventory | map(attribute='name') | join('|')) }})$
bypassDnsResolution: true
config.sample.yaml
+112 -72
View File
@@ -1,59 +1,76 @@
---
# (Required) Distribution can either be k3s, k0s, or talos
distribution: ""
# (Required) IANA formatted timezone (e.g. America/New_York)
#
# (Required) Distribution represents the Kubernetes distribution layer and any additional customizations
#
distribution:
# (Required) Distribution can either be k3s, k0s, or talos
type: k3s
# (Optional) Talos Specific Options
talos: {}
# schematics:
# # (Required) Disabling the schematic will disable the system-upgrade-controller and any
# # additional Talos customization you have defined.
# enabled: false
# # (Required) For use with the system-upgrade-controller generate a schematic ID
# # based on your System Extension requirements https://factory.talos.dev/
# id: ""
# # (Required) Additional NodeConfigs to apply to all nodes
# # See: https://budimanjojo.github.io/talhelper/latest/reference/configuration/#nodeconfigs
# customization: |-
# extraKernelArgs:
# - net.ifnames=0
# systemExtensions:
# officialExtensions:
# - siderolabs/intel-ucode
# - siderolabs/i915-ucode
# # (Optional) Add vlan tag to network master device
# # See: https://www.talos.dev/latest/advanced/advanced-networking/#vlans
# vlan: 1
#
# (Required) Timezone is your IANA formatted timezone (e.g. America/New_York)
#
timezone: ""
# (Required) Cluster details
#
# (Required) Nodes represents the physical or virtual machines layer and any additional customizations
#
nodes:
# (Required) CIDR your nodes are on (e.g. 192.168.1.0/24)
host_network: ""
# (Optional) The DNS server to use for the cluster, this can be an existing
# local DNS server or a public one.
# Default is ["1.1.1.1", "1.0.0.1"]
# If using a local DNS server make sure it meets the following requirements:
# 1. your nodes can reach it
# 2. it is configured to forward requests to a public DNS server
# 3. you are not force redirecting DNS requests to it - this will break cert generation over DNS01
# If using multiple DNS servers make sure they are setup the same way, there is no
# guarantee that the first DNS server will always be used for every lookup.
dns_servers: []
# (Optional) The DNS search domain to use for the nodes.
# Default is "."
# Use the default or leave empty to avoid possible DNS issues inside the cluster.
search_domain: ""
# (Required) Use only 1, 3 or more ODD number of controller nodes, recommended is 3
# Worker nodes are optional
inventory: []
# - name: "" # Name of the node (must match [a-z0-9-\.]+)
# address: "" # IP address of the node
# controller: true # (Required) Set to true if this is a controller node
# ssh_username: "" # (Required: k3s/k0s) SSH username of the node
# talos_disk_device: "" # (Required: Talos) Device path or serial number of the disk for this node
# ...
#
# (Required) Cluster represents the Kubernetes cluster layer and any additional customizations
#
cluster:
# (Required) The nodes section is used to configure your cluster nodes
nodes:
# (Required) CIDR your nodes are on (e.g. 192.168.1.0/24)
host_network: ""
# (Required) The DNS server to use for the cluster, this can be an existing
# local DNS server or a public one
# If using a local DNS server make sure it meets the following requirements:
# 1. your nodes can reach it
# 2. it is configured to forward requests to a public DNS server
# 3. you are not force redirecting DNS requests to it - this will break cert generation over DNS01
# If using multiple DNS servers make sure they are setup the same way, there is no
# guarantee that the first DNS server will always be used for every lookup.
dns_servers:
- "1.1.1.1"
# (Optional) The DNS search domain to use for the nodes.
# Leave this option blank to avoid possible DNS issues inside the cluster.
search_domain: ""
# (Required) Use only 1, 3 or more ODD number of controller nodes, recommended is 3
# Worker nodes are optional
inventory: []
# - name: "" # Name of the node (must match [a-z0-9-\.]+)
# address: "" # IP address of the node
# controller: true # (Required) Set to true if this is a controller node
# ssh_username: "" # (Required: k3s/k0s) SSH username of the node
# talos_disk_device: "" # (Required: Talos) Device Path or Serial number of Disk for this node
# ...
# (Optional) Talos Specific Options
talos: {}
# schematics:
# # (Required) Disabling the schematic will disable the system-upgrade-controller and any
# # additional customizations you have defined.
# enabled: false
# # (Required) For use with the system-upgrade-controller generate a schematic ID
# # based on your System Extension requirements https://factory.talos.dev/
# id: ""
# # (Required) Additional NodeConfigs to apply to all nodes
# # See: https://budimanjojo.github.io/talhelper/latest/reference/configuration/#nodeconfigs
# customization: |-
# extraKernelArgs:
# - net.ifnames=0
# systemExtensions:
# officialExtensions:
# - siderolabs/intel-ucode
# - siderolabs/i915-ucode
# # (Optional) Add vlan tag to network master device
# # See: https://www.talos.dev/latest/advanced/advanced-networking/#vlans
# vlan: 1
# (Required) The pod CIDR for the cluster, this must NOT overlap with any
# existing networks and is usually a /16 (64K IPs).
# If you want to use IPv6 check the advanced flags below
@@ -69,55 +86,73 @@ cluster:
# if you want to call the Kube API by hostname rather than IP
tls_sans: []
# (Required) Flux details
# IMPORTANT: All options are ignored if flux is disabled and no resources
# outside Cilium and kube-vip (k0s/k3s) will be deployed to your cluster.
#
# (Optional) Flux details - Flux is used to manage the cluster configuration.
#
flux:
# (Required) Disable to use a different tool (e.g. kubectl, Argo, Kluctl)
enabled: true
# (Required) Options for Github
# (Required) Age Public Key (e.g. age1...)
# 1. Generate a new key with the following command:
# > task sops:age-keygen
# 2. Copy the public key and paste it below
sops_age_public_key: ""
# (Required) Options for GitHub
github:
# (Required) Github repository URL (for private repos use the ssh:// URL)
# (Required) GitHub repository URL (for private repos use the ssh:// URL)
address: ""
# (Required) Github repository branch
# (Required) GitHub repository branch
branch: main
# (Required) Options for Flux Github webhook
# (Required) Options for Flux GitHub webhook
webhook:
# (Required) Enable to use Github push-based sync
# (Required) Enable to setup GitHub push-based sync after cluster creation
enabled: true
# Token for Github push-based sync (openssl rand -hex 16)
# Token for GitHub push-based sync
# 1. Generate a new token with the following command:
# > openssl rand -hex 16
# 2. Copy the token and paste it below
token: ""
# (Required) Private key for Flux to access the Github repository
# (Required) Private key for Flux to access the GitHub repository
private:
# (Required) Enable to use a private Github repository
# (Required) Enable to use a private GitHub repository
enabled: false
# Private key for Flux to access the Github repository
# Private key for Flux to access the GitHub repository
# 1. Generate a new key with the following command:
# ssh-keygen -t ecdsa -b 521 -C "github-deploy-key" -f github-deploy.key -q -P ""
# 2. Make sure to paste public key from "github-deploy.key.pub"
# into the deploy keys section of your repository settings
# > ssh-keygen -t ecdsa -b 521 -C "github-deploy-key" -f github-deploy.key -q -P ""
# 2. Make sure to paste public key from "github-deploy.key.pub" into
# the deploy keys section of your repository settings.
# 3. Uncomment and paste the private key below
# key: |
# -----BEGIN OPENSSH PRIVATE KEY-----
# ...
# -----END OPENSSH PRIVATE KEY-----
# (Required) Age Public Key (e.g. age15uzrw396e67z9wdzsxzdk7ka0g2gr3l460e0slaea563zll3hdfqwqxdta)
sops_age_public_key: ""
#
# (Optional) Cloudflare details - Cloudflare is used for DNS, TLS certificates and tunneling.
#
# (Required) Cloudflare details - Cloudflare is used for DNS, TLS certificates and tunneling.
# IMPORTANT: All options are ignored if cloudflare is disabled and no resources
# outside a bare Flux setup will be deployed to your cluster if Flux is enabled.
cloudflare:
# (Required) Disable to use a different DNS provider
enabled: true
# (Required) Cloudflare Domain
domain: ""
# (Required) Cloudflare API Token (NOT API Key)
# 1. Head over to Cloudflare and create a API Token by going to
# https://dash.cloudflare.com/profile/api-tokens
# 2. Under the `API Tokens` section click the blue `Create Token` button.
# 3. Click the blue `Use template` button for the `Edit zone DNS` template.
# 4. Name your token something like `home-kubernetes`
# 5. Under `Permissions`, click `+ Add More` and add each permission below:
# `Zone - DNS - Edit`
# `Account - Cloudflare Tunnel - Read`
# 6. Limit the permissions to a specific account and zone resources.
# 7. Click the blue `Continue to Summary` button and then the blue `Create Token` button.
# 8. Copy the token and paste it below.
token: ""
# (Required) Optionals for Cloudflare Acme
acme:
# (Required) Email you want to be associated with the ACME account (used for TLS certs via letsencrypt.org)
# (Required) Any email you want to be associated with the ACME account (used for TLS certs via letsencrypt.org)
email: ""
# (Required) Use the ACME production server when requesting the wildcard certificate.
# By default the ACME staging server is used. This is to prevent being rate-limited.
@@ -133,6 +168,11 @@ cloudflare:
# in your nodes host network that is NOT being used. This is announced over L2.
gateway_vip: ""
# (Required) Options for Cloudflare Tunnel
# 1. Authenticate cloudflared to your domain
# > cloudflared tunnel login
# 2. Create the tunnel
# > cloudflared tunnel create k8s
# 3. Copy the AccountTag, TunnelID, and TunnelSecret from the tunnel configuration file and paste them below
tunnel:
# (Required) Cloudflare Account ID (cat ~/.cloudflared/*.json | jq -r .AccountTag)
account_id: ""