Cloud Provisioning: Kubernetes
1. Purpose
This file describes how to provision Kubernetes clusters in cloud environments. It complements the local and native environment documents with public-cloud and multi-cloud deployment options.
The focus is reproducible provisioning for workshops, labs, and technical evaluations. It provides:
- a copyable Bash provisioning script with dry-run as the default mode,
- provider-specific CLI examples for AWS, Azure, Google Cloud, and additional providers,
- managed Kubernetes variants such as EKS, AKS, GKE, OKE, DOKS, LKE, and IKS,
- self-managed variants on cloud VMs using kubeadm, k3s, RKE2, k0s, and Talos,
- IaC paths with Terraform/OpenTofu, Pulumi, Cluster API, Crossplane, and GitOps,
- validation, troubleshooting, teardown, and cost-control guidance.
This file is not intended as a production runbook to execute blindly. It is a workshop and architecture guide. Before any real execution, you must validate quotas, costs, network ranges, security requirements, and organizational policies.
2. Recommendation
For most workshops, a managed Kubernetes service is the cleanest cloud option:
| Goal | Recommendation | Rationale |
|---|---|---|
| Fast workshop on AWS | EKS with eksctl or Terraform |
Fast entry point, controlled teardown, strong ecosystem. |
| Fast workshop on Azure | AKS with Azure CLI or Terraform | Good integration with Resource Groups, Managed Identity, Azure Container Registry, and Azure Monitor. |
| Fast workshop on Google Cloud | GKE Autopilot | Low infrastructure maintenance, strong defaults, focus on Kubernetes objects. |
| Production-like cluster comparison | EKS, AKS, and GKE, each via Terraform | Same operating model, comparable parameters, traceable state. |
| Multi-cloud lifecycle | Cluster API or Terraform plus GitOps | Declarative fleet management instead of manual provider-specific click paths. |
| Cloud VM lab | kubeadm or k3s on VMs | More control over the control plane, runtime, CNI, and node operations. |
| Edge or small cloud VMs | k3s, RKE2, or k0s | Lighter operating models than a full kubeadm setup. |
| Immutable OS / strong hardening | Talos Linux | Operation through an API instead of SSH, clear machine state. |
If you only want to practice Kubernetes objects, Deployments, Services, probes, RBAC, Ingress, and Helm, use managed Kubernetes. If you want to understand cluster bootstrap, the control plane, etcd, CNI, CSI, kubelet, or node-level debugging, use cloud VMs with kubeadm, k3s, RKE2, k0s, or Talos.
3. Target Architecture
Cloud provisioning consists of multiple layers. A Kubernetes cluster is only one part of the overall environment.
flowchart TD
Repo[Workshop Files] --> Script[Provisioning Script]
Script --> ProviderCLI[Provider CLI]
Script --> IaC[Terraform/OpenTofu or Pulumi]
ProviderCLI --> Network[Cloud Network]
ProviderCLI --> IAM[Identity and Permissions]
ProviderCLI --> Cluster[Managed Kubernetes or Cloud VMs]
IaC --> Network
IaC --> IAM
IaC --> Cluster
Cluster --> Kubeconfig[kubeconfig]
Cluster --> Addons[Add-ons]
Addons --> Workloads[Workshop Workloads]
Workloads --> Validation[Validation]
Validation --> Cleanup[Teardown]
You should always treat cloud Kubernetes as the interaction of the following domains:
| Domain | Typical Resources | Failure Mode When Neglected |
|---|---|---|
| Identity | IAM roles, service principals, workload identity, ServiceAccounts | Cluster can be created, but nodes or add-ons do not receive required permissions. |
| Network | VPC/VNet, subnets, routing, NAT, security groups, firewalls | Nodes do not join, LoadBalancer remains pending, pods cannot reach external services. |
| Cluster | Control plane, node pools, version, upgrade policy | Version drift, unexpected upgrades, unplanned capacity. |
| Add-ons | CNI, CSI, DNS, Metrics Server, Ingress Controller, Gateway Controller | Workloads run, but service discovery, storage, or ingress do not work. |
| Operations | Logging, metrics, alerts, backup, audit | Failures remain invisible or cannot be reconstructed afterwards. |
| Security | RBAC, Pod Security, secrets, network isolation, image policies | Labs are later reused accidentally as insecure blueprints. |
| Cost | Tags, budgets, quotas, automatic deletion | Test clusters keep running and create unnecessary costs. |
4. Decision Model
flowchart TD
Start[You want to provision Kubernetes in the cloud] --> NeedControl{Do you want to operate the control plane and nodes yourself?}
NeedControl -- No --> Managed{Do you want as little node operation as possible?}
Managed -- Yes --> Auto[Autopilot/Automatic/Auto Mode]
Managed -- No --> Standard[Managed Kubernetes with node pools]
NeedControl -- Yes --> VM{Do you need full OS control?}
VM -- Yes --> Kubeadm[kubeadm on cloud VMs]
VM -- Prefer lightweight --> Dist[k3s/RKE2/k0s/Talos]
Auto --> IaC{Repeatable for teams?}
Standard --> IaC
Kubeadm --> IaC
Dist --> IaC
IaC -- Yes --> Terraform[Terraform/OpenTofu, Pulumi, or Cluster API]
IaC -- No --> CLI[Provider CLI for labs]
4.1 Cloud Paths
| Path | Example | You Get | You Operate Yourself | Suitable For |
|---|---|---|---|---|
| Managed Kubernetes | EKS, AKS, GKE, OKE, DOKS, LKE, IKS | Managed control plane, provider integration | Node pools, add-ons, policies, workloads | Standard workshops and production-like platforms. |
| Serverless/Autopilot/Automatic | GKE Autopilot, AKS Automatic, EKS Auto Mode/Fargate | Less node management | Policies, limits, application design, provider-specific constraints | App-oriented labs, cost/ops comparison. |
| VM + kubeadm | EC2, Azure VM, Compute Engine, OCI Compute | Full control over Kubernetes components | Everything: OS, runtime, control plane, etcd, CNI, upgrades | Cluster bootstrap, admin, and troubleshooting labs. |
| VM + k3s/RKE2/k0s | Small cloud VMs, edge, remote labs | Simplified cluster bootstrap | OS, distribution, network, storage, add-ons | Lightweight environments. |
| Talos | Cloud VMs with Talos image | Immutable Kubernetes OS | Talos configuration, API operation, cluster lifecycle | Hardening, declarative machine operation. |
| kOps | Mainly AWS, partially other providers | Kubernetes cluster plus cloud infrastructure | kOps state, network, upgrades | Traditional self-managed clusters on AWS. |
| Cluster API | Management cluster plus provider | Declarative cluster fleet | Management cluster, provider controllers | Multi-cluster and multi-cloud lifecycle. |
| Crossplane | Kubernetes as control plane for cloud resources | Cloud resources through Kubernetes APIs | Crossplane providers, compositions, RBAC | Platform teams and self-service. |
| Terraform/OpenTofu | IaC state plus provider | Reproducible infrastructure | State, modules, plan/apply process | Standardized provisioning. |
| Pulumi | IaC in programming languages | Reproducible infrastructure as code | Stack state, secrets, runtime | Teams with a strong software-engineering focus. |
5. Architecture Overview
5.1 Managed Kubernetes
flowchart LR
Operator[kubectl, Helm, Terraform] --> API[Cloud Provider API]
API --> CP[Managed Kubernetes Control Plane]
API --> NP[Node Pools]
NP --> Nodes[Worker Nodes]
CP --> KubeAPI[Kubernetes API]
KubeAPI --> Addons[Add-ons: DNS, CNI, CSI, Ingress]
Addons --> Apps[Workloads]
Apps --> LB[Cloud Load Balancer]
Apps --> Disk[Cloud Disks]
5.2 Cloud VMs with Self-Managed Kubernetes
flowchart TD
Script[Provisioning Script] --> VMs[Cloud VMs]
Script --> Firewall[Security Groups/Firewalls]
Script --> LB[Optional API Load Balancer]
VMs --> Runtime[containerd or Distribution Runtime]
Runtime --> Bootstrap[kubeadm/k3s/RKE2/k0s/Talos]
Bootstrap --> CP[Control Plane]
Bootstrap --> Workers[Worker Nodes]
CP --> Etcd[etcd]
Workers --> Pods[Pods]
Pods --> CNI[CNI]
Pods --> CSI[CSI]
5.3 IaC and GitOps
flowchart TD
Git[Git Repository] --> Plan[Terraform/OpenTofu Plan or Pulumi Preview]
Plan --> Apply[Apply/Up]
Apply --> Infra[Cloud Infrastructure]
Infra --> Cluster[Kubernetes Cluster]
Cluster --> Bootstrap[Bootstrap Namespace]
Bootstrap --> GitOps[Argo CD or Flux]
GitOps --> Addons[Add-ons]
GitOps --> Apps[Applications]
6. Minimum Requirements
6.1 Local Workstation
You need at least:
- Linux, macOS, WSL2, or an admin VM,
bash,curl,jq,git,openssl,ssh,tar,unzip,kubectlmatching the target cluster version,helmif you install add-ons,- the provider CLI for the selected provider,
- access to a cloud account with budget and permissions,
- a clean strategy for secrets and kubeconfig files.
6.2 Provider CLIs
| Provider | CLI | Typical Purpose |
|---|---|---|
| AWS | aws, eksctl, optionally rosa |
EKS, EKS Auto Mode, IAM, VPC, kubeconfig, ROSA. |
| Azure | az |
AKS, Resource Groups, VNets, Managed Identity, kubeconfig. |
| Google Cloud | gcloud |
GKE Autopilot, GKE Standard, IAM, project and network management. |
| DigitalOcean | doctl |
DOKS, node pools, kubeconfig, Load Balancer. |
| Akamai/Linode | linode-cli |
LKE, node pools, kubeconfig. |
| Oracle Cloud | oci |
OKE, compartment, VCN, node pools. |
| IBM Cloud | ibmcloud, ks plugin |
IBM Cloud Kubernetes Service and OpenShift clusters. |
| Civo | civo |
Civo Kubernetes. |
| Vultr | vultr-cli |
Vultr Kubernetes Engine. |
| Scaleway | scw |
Scaleway Kapsule/Kosmos. |
| Hetzner | hcloud |
Cloud VMs for k3s, kubeadm, or Talos; no classic managed-Kubernetes path. |
| OVHcloud | ovh or Terraform |
OVHcloud Managed Kubernetes Service. |
| Alibaba Cloud | aliyun |
ACK. |
| Tencent Cloud | tccli |
TKE. |
6.3 Cloud Quotas
Check before the workshop:
- vCPU limit per region,
- number of public IPs,
- NAT Gateway or egress costs,
- Load Balancer quota,
- disk quota,
- IAM and policy creation,
- available Kubernetes versions,
- available VM sizes,
- approved regions,
- organizational policies for public endpoints.
6.4 Versioning
Pin versions deliberately. Do not blindly use latest.
Recommended rules:
- For managed Kubernetes, query supported versions per provider.
- For kubeadm, k3s, RKE2, k0s, and Talos, document the target version in the repository.
- For Terraform/OpenTofu, pin provider and module versions.
- For Helm charts, pin chart versions and image tags.
- For production-like labs, test upgrades and teardown.
Examples:
# AWS EKS: check available add-on and version information
aws eks describe-addon-versions --region eu-central-1 | jq '.addons[].addonName' | sort -u
# Azure AKS: check available versions per region
az aks get-versions --location westeurope -o table
# GKE: check available versions per region
gcloud container get-server-config --region europe-west3 --format=json | jq '.validMasterVersions[0:5]'
# DigitalOcean Kubernetes: check available options
doctl kubernetes options versions
# Linode/Akamai LKE: check available versions
linode-cli lke versions-list
7. Security, Cost, and Governance Rules
7.1 Cost Rules
Cloud provisioning creates real costs. Follow these rules:
- Every cluster gets tags/labels such as
owner,workshop,ttl,environment, andcost-center. - Every lab cluster has a documented teardown path.
- Dry-run is the default; real execution requires
--execute. - Use small node sizes for fundamentals labs.
- Avoid NAT Gateways, large Load Balancers, GPU nodes, and premium disks unless you explicitly need them.
- Configure budgets and alerts in the cloud account.
- Explicitly clean up Load Balancers, volumes, snapshots, public IPs, and DNS records after cluster teardown.
7.2 Security Rules
- No production secrets in workshop clusters.
- No long-lived cloud root credentials on workstations.
- Use SSO, temporary credentials, or service principals with minimal permissions.
- Treat kubeconfig files as confidential.
- Public Kubernetes API endpoints must be explicitly approved and restricted.
- Prefer Workload Identity, IRSA, Managed Identity, or comparable mechanisms over static cloud keys in pods.
- Do not grant permanent admin rights to workshop participants.
- Enable audit logging and API access visibility in production-like environments.
7.3 Governance Rules
| Topic | Rule |
|---|---|
| Naming scheme | <team>-<purpose>-<provider>-<region>-<number> |
| Regions | Predefined allow-list; no arbitrary regions. |
| Tags | Required for owner, TTL, environment, and cost center. |
| Versions | Documented, verified, not implicit. |
| Network | Avoid CIDR collisions with corporate networks. |
| Access | Role-based, not shared admin access. |
| Deletion | Script-supported and validated. |
8. Tool Setup
8.1 Base Packages on Ubuntu/WSL/VM
sudo apt update
sudo apt install -y \
ca-certificates curl gnupg lsb-release git jq unzip tar gzip \
openssh-client openssl make bash-completion
8.2 Install kubectl
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl
sudo mv kubectl /usr/local/bin/kubectl
kubectl version --client
8.3 Install Helm
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
helm version
8.4 Install Terraform or OpenTofu
Example using Terraform via HashiCorp packages:
sudo apt update
sudo apt install -y gnupg software-properties-common
wget -O- https://apt.releases.hashicorp.com/gpg | \
gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg >/dev/null
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | \
sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update
sudo apt install -y terraform
terraform version
8.5 Provider CLI Quick Check
# Check AWS CLI
aws --version
aws sts get-caller-identity
# Check eksctl
eksctl version
# Check Azure CLI
az version
az account show -o table
# Check Google Cloud CLI
gcloud version
gcloud auth list
# Check DigitalOcean CLI
doctl version
doctl account get
# Check Linode CLI
linode-cli --version
linode-cli profile view
# Check Oracle Cloud CLI
oci --version
oci iam region list --all
# Check IBM Cloud CLI
ibmcloud version
ibmcloud account show
9. Common Parameters
Use common variables for workshop scripts. This allows you to compare providers without rewriting each example completely.
export CLOUD_PROVIDER="aws" # aws, azure, gcp, digitalocean, linode, oci, ibm, civo, vultr, scaleway, hetzner
export CLUSTER_MODE="managed" # managed, autopilot, terraform, vm-kubeadm, vm-k3s, vm-rke2, vm-k0s, talos
export CLUSTER_NAME="k8s-workshop-cloud"
export REGION="eu-central-1"
export RESOURCE_GROUP="rg-k8s-workshop"
export PROJECT_ID="my-gcp-project"
export NODE_COUNT="2"
export NODE_SIZE="small"
export K8S_VERSION=""
export OWNER="your.name@example.com"
export TTL="8h"
export ENVIRONMENT="workshop"
9.1 Mapping Common Regions
| Provider | Example Region in Europe | Note |
|---|---|---|
| AWS | eu-central-1 |
Frankfurt. |
| Azure | westeurope or germanywestcentral |
Choose region based on quota and company policy. |
| Google Cloud | europe-west3 or europe-west1 |
Frankfurt or Belgium. |
| DigitalOcean | fra1, ams3, lon1 |
Check DOKS options first. |
| Akamai/Linode | eu-central, eu-west |
Check region via linode-cli regions list. |
| Oracle Cloud | eu-frankfurt-1 |
Compartment and VCN IDs required. |
| IBM Cloud | eu-de |
Check zone/region by service. |
| Civo | FRA1, LON1 |
Check region via CLI. |
| Vultr | fra, ams, lhr |
Check region via CLI. |
| Scaleway | fr-par, nl-ams, pl-waw |
Check Kapsule configuration. |
| Hetzner | fsn1, nbg1, hel1 |
For VM-based clusters. |
10. Central Provisioning Script
This script is built for workshops. It is intentionally defensive:
- Dry-run is the default.
- Real execution requires
--execute. - Provider-specific dependencies are checked.
- kubeconfig is validated after provisioning.
- Teardown is explicit via
--action delete. - Incomplete provider variants return clear guidance instead of creating half-finished resources.
10.1 Example Calls
# Only show what would happen
bash cloud-kubernetes-provision.sh --provider aws --mode managed --cluster workshop-eks
# Actually create EKS with eksctl
bash cloud-kubernetes-provision.sh \
--provider aws \
--mode managed \
--cluster workshop-eks \
--region eu-central-1 \
--nodes 2 \
--node-size t3.medium \
--execute
# Actually create AKS
bash cloud-kubernetes-provision.sh \
--provider azure \
--mode managed \
--cluster workshop-aks \
--region westeurope \
--resource-group rg-workshop-aks \
--nodes 2 \
--node-size Standard_B2s \
--execute
# Actually create GKE Autopilot
bash cloud-kubernetes-provision.sh \
--provider gcp \
--mode autopilot \
--cluster workshop-gke \
--region europe-west3 \
--project my-gcp-project \
--execute
# Create DigitalOcean Kubernetes
bash cloud-kubernetes-provision.sh \
--provider digitalocean \
--mode managed \
--cluster workshop-doks \
--region fra1 \
--nodes 2 \
--node-size s-2vcpu-4gb \
--execute
# Validate cluster
bash cloud-kubernetes-provision.sh --provider aws --action validate --cluster workshop-eks --region eu-central-1
# Delete cluster
bash cloud-kubernetes-provision.sh --provider aws --action delete --cluster workshop-eks --region eu-central-1 --execute
10.2 Script
Save the following content as scripts/cloud-kubernetes-provision.sh.
#!/usr/bin/env bash
set -Eeuo pipefail
# Cloud Kubernetes provisioning for workshops.
# Default: dry-run. Real execution only with --execute.
ACTION="create"
PROVIDER="${CLOUD_PROVIDER:-aws}"
MODE="${CLUSTER_MODE:-managed}"
CLUSTER_NAME="${CLUSTER_NAME:-k8s-workshop-cloud}"
REGION="${REGION:-eu-central-1}"
RESOURCE_GROUP="${RESOURCE_GROUP:-rg-k8s-workshop}"
PROJECT_ID="${PROJECT_ID:-}"
NODE_COUNT="${NODE_COUNT:-2}"
NODE_SIZE="${NODE_SIZE:-}"
K8S_VERSION="${K8S_VERSION:-}"
OWNER="${OWNER:-unknown}"
TTL="${TTL:-8h}"
ENVIRONMENT="${ENVIRONMENT:-workshop}"
DRY_RUN="true"
log() { printf '[%s] %s\n' "$(date +%H:%M:%S)" "$*"; }
fail() { printf 'ERROR: %s\n' "$*" >&2; exit 1; }
need() { command -v "$1" >/dev/null 2>&1 || fail "Required command is missing: $1"; }
run() {
if [[ "$DRY_RUN" == "true" ]]; then
printf '[dry-run] %s\n' "$*"
else
log "Executing: $*"
"$@"
fi
}
usage() {
cat <<'USAGE'
Usage:
cloud-kubernetes-provision.sh [options]
Options:
--provider <aws|azure|gcp|digitalocean|linode|oci|ibm|civo|vultr|scaleway|hetzner>
--mode <managed|autopilot|terraform|vm-kubeadm|vm-k3s|vm-rke2|vm-k0s|talos>
--action <create|validate|delete|versions|kubeconfig>
--cluster <name>
--region <region>
--resource-group <name> Azure Resource Group
--project <id> Google Cloud Project ID
--nodes <count>
--node-size <size>
--k8s-version <version>
--owner <name or email>
--ttl <duration>
--environment <name>
--execute Disable dry-run
-h, --help
Examples:
./cloud-kubernetes-provision.sh --provider aws --mode managed --cluster demo
./cloud-kubernetes-provision.sh --provider azure --mode managed --cluster demo --execute
./cloud-kubernetes-provision.sh --provider gcp --mode autopilot --cluster demo --project my-project --execute
USAGE
}
while [[ $# -gt 0 ]]; do
case "$1" in
--provider) PROVIDER="$2"; shift 2 ;;
--mode) MODE="$2"; shift 2 ;;
--action) ACTION="$2"; shift 2 ;;
--cluster) CLUSTER_NAME="$2"; shift 2 ;;
--region) REGION="$2"; shift 2 ;;
--resource-group) RESOURCE_GROUP="$2"; shift 2 ;;
--project) PROJECT_ID="$2"; shift 2 ;;
--nodes) NODE_COUNT="$2"; shift 2 ;;
--node-size) NODE_SIZE="$2"; shift 2 ;;
--k8s-version) K8S_VERSION="$2"; shift 2 ;;
--owner) OWNER="$2"; shift 2 ;;
--ttl) TTL="$2"; shift 2 ;;
--environment) ENVIRONMENT="$2"; shift 2 ;;
--execute) DRY_RUN="false"; shift ;;
-h|--help) usage; exit 0 ;;
*) fail "Unknown option: $1" ;;
esac
done
print_context() {
cat <<EOF
Context:
Provider: $PROVIDER
Mode: $MODE
Action: $ACTION
Cluster: $CLUSTER_NAME
Region: $REGION
ResourceGroup: $RESOURCE_GROUP
Project: ${PROJECT_ID:-not set}
Nodes: $NODE_COUNT
Node Size: ${NODE_SIZE:-provider-default}
K8s Version: ${K8S_VERSION:-provider-default}
Owner: $OWNER
TTL: $TTL
Environment: $ENVIRONMENT
Dry-Run: $DRY_RUN
EOF
}
common_validate() {
need kubectl
run kubectl cluster-info
run kubectl get nodes -o wide
run kubectl get namespaces
run kubectl get pods -A
}
aws_default_node_size() { [[ -n "$NODE_SIZE" ]] && printf '%s' "$NODE_SIZE" || printf 't3.medium'; }
azure_default_node_size() { [[ -n "$NODE_SIZE" ]] && printf '%s' "$NODE_SIZE" || printf 'Standard_B2s'; }
gcp_default_node_size() { [[ -n "$NODE_SIZE" ]] && printf '%s' "$NODE_SIZE" || printf 'e2-standard-2'; }
do_default_node_size() { [[ -n "$NODE_SIZE" ]] && printf '%s' "$NODE_SIZE" || printf 's-2vcpu-4gb'; }
linode_default_node_size() { [[ -n "$NODE_SIZE" ]] && printf '%s' "$NODE_SIZE" || printf 'g6-standard-2'; }
aws_versions() {
need aws
run aws eks describe-addon-versions --region "$REGION"
}
aws_kubeconfig() {
need aws
run aws eks update-kubeconfig --region "$REGION" --name "$CLUSTER_NAME"
}
aws_create_managed() {
need eksctl
local size; size="$(aws_default_node_size)"
local args=(eksctl create cluster --name "$CLUSTER_NAME" --region "$REGION" --nodes "$NODE_COUNT" --node-type "$size" --managed)
[[ -n "$K8S_VERSION" ]] && args+=(--version "$K8S_VERSION")
args+=(--tags "owner=$OWNER,environment=$ENVIRONMENT,ttl=$TTL")
run "${args[@]}"
aws_kubeconfig
}
aws_create_autopilot() {
need aws
cat <<'INFO'
EKS Auto Mode requires several IAM, networking, and cluster parameters. For production-like usage, a dedicated IaC module is recommended.
This script intentionally does not create a half-complete Auto Mode cluster. Use the AWS chapter in the Markdown file or Terraform.
INFO
}
aws_delete() {
if command -v eksctl >/dev/null 2>&1; then
run eksctl delete cluster --name "$CLUSTER_NAME" --region "$REGION"
else
need aws
run aws eks delete-cluster --name "$CLUSTER_NAME" --region "$REGION"
fi
}
azure_versions() {
need az
run az aks get-versions --location "$REGION" -o table
}
azure_kubeconfig() {
need az
run az aks get-credentials --resource-group "$RESOURCE_GROUP" --name "$CLUSTER_NAME" --overwrite-existing
}
azure_create_managed() {
need az
local size; size="$(azure_default_node_size)"
run az group create --name "$RESOURCE_GROUP" --location "$REGION" --tags owner="$OWNER" environment="$ENVIRONMENT" ttl="$TTL"
local args=(az aks create --resource-group "$RESOURCE_GROUP" --name "$CLUSTER_NAME" --location "$REGION" --node-count "$NODE_COUNT" --node-vm-size "$size" --generate-ssh-keys --enable-managed-identity --tags owner="$OWNER" environment="$ENVIRONMENT" ttl="$TTL")
[[ -n "$K8S_VERSION" ]] && args+=(--kubernetes-version "$K8S_VERSION")
run "${args[@]}"
azure_kubeconfig
}
azure_create_autopilot() {
need az
cat <<'INFO'
AKS Automatic has its own requirements and options, which may change quickly.
Use the AKS Automatic chapter as a baseline and check the current Azure CLI help before execution: az aks create --help
INFO
}
azure_delete() {
need az
run az aks delete --resource-group "$RESOURCE_GROUP" --name "$CLUSTER_NAME" --yes --no-wait
}
gcp_versions() {
need gcloud
[[ -n "$PROJECT_ID" ]] && run gcloud config set project "$PROJECT_ID"
run gcloud container get-server-config --region "$REGION" --format=json
}
gcp_kubeconfig() {
need gcloud
[[ -n "$PROJECT_ID" ]] && run gcloud config set project "$PROJECT_ID"
run gcloud container clusters get-credentials "$CLUSTER_NAME" --region "$REGION"
}
gcp_create_managed() {
need gcloud
[[ -n "$PROJECT_ID" ]] && run gcloud config set project "$PROJECT_ID"
local size; size="$(gcp_default_node_size)"
local args=(gcloud container clusters create "$CLUSTER_NAME" --region "$REGION" --num-nodes "$NODE_COUNT" --machine-type "$size" --labels "owner=$OWNER,environment=$ENVIRONMENT,ttl=$TTL")
[[ -n "$K8S_VERSION" ]] && args+=(--cluster-version "$K8S_VERSION")
run "${args[@]}"
gcp_kubeconfig
}
gcp_create_autopilot() {
need gcloud
[[ -n "$PROJECT_ID" ]] && run gcloud config set project "$PROJECT_ID"
local args=(gcloud container clusters create-auto "$CLUSTER_NAME" --region "$REGION" --labels "owner=$OWNER,environment=$ENVIRONMENT,ttl=$TTL")
[[ -n "$K8S_VERSION" ]] && args+=(--cluster-version "$K8S_VERSION")
run "${args[@]}"
gcp_kubeconfig
}
gcp_delete() {
need gcloud
[[ -n "$PROJECT_ID" ]] && run gcloud config set project "$PROJECT_ID"
run gcloud container clusters delete "$CLUSTER_NAME" --region "$REGION" --quiet
}
do_versions() {
need doctl
run doctl kubernetes options versions
}
do_kubeconfig() {
need doctl
run doctl kubernetes cluster kubeconfig save "$CLUSTER_NAME"
}
do_create_managed() {
need doctl
local size; size="$(do_default_node_size)"
local args=(doctl kubernetes cluster create "$CLUSTER_NAME" --region "$REGION" --size "$size" --count "$NODE_COUNT" --tag "owner:$OWNER" --tag "environment:$ENVIRONMENT" --tag "ttl:$TTL")
[[ -n "$K8S_VERSION" ]] && args+=(--version "$K8S_VERSION")
run "${args[@]}"
do_kubeconfig
}
do_delete() {
need doctl
run doctl kubernetes cluster delete "$CLUSTER_NAME" --dangerous --force
}
linode_versions() {
need linode-cli
run linode-cli lke versions-list
}
linode_create_managed() {
need linode-cli
local size; size="$(linode_default_node_size)"
[[ -z "$K8S_VERSION" ]] && fail "For LKE you must set --k8s-version. Available versions: linode-cli lke versions-list"
run linode-cli lke cluster-create --label "$CLUSTER_NAME" --region "$REGION" --k8s_version "$K8S_VERSION" --node_pools.type "$size" --node_pools.count "$NODE_COUNT" --tags "$ENVIRONMENT" --tags "$OWNER"
cat <<'INFO'
Note: Afterwards, determine the cluster ID with:
linode-cli lke clusters-list
and save the kubeconfig with:
linode-cli lke kubeconfig-view <cluster-id> --text | base64 -d > ./kubeconfig-lke.yaml
export KUBECONFIG="$PWD/kubeconfig-lke.yaml"
INFO
}
linode_delete() {
need linode-cli
cat <<'INFO'
LKE deletion requires the cluster ID, not only the label:
linode-cli lke clusters-list
linode-cli lke cluster-delete <cluster-id>
INFO
}
oci_create_hint() {
need oci
cat <<'INFO'
OKE requires compartment, VCN, and subnet OCIDs. Set these values deliberately before creating clusters.
Example:
oci ce cluster create \
--compartment-id "$OCI_COMPARTMENT_ID" \
--name "$CLUSTER_NAME" \
--kubernetes-version "$K8S_VERSION" \
--vcn-id "$OCI_VCN_ID" \
--endpoint-subnet-id "$OCI_ENDPOINT_SUBNET_ID" \
--service-lb-subnet-ids '["ocid1.subnet..."]'
INFO
}
ibm_create_hint() {
need ibmcloud
cat <<'INFO'
IBM Cloud Kubernetes Service uses the ks plugin. Install and check it first:
ibmcloud plugin install container-service
ibmcloud ks cluster versions
ibmcloud ks cluster create --help
INFO
}
terraform_hint() {
need terraform
mkdir -p "generated/$PROVIDER-$CLUSTER_NAME"
cat > "generated/$PROVIDER-$CLUSTER_NAME/README.md" <<EOF
# Terraform starting point for $PROVIDER / $CLUSTER_NAME
This directory was generated as a placeholder. Add provider, networking, and cluster resources based on the provider chapter.
Recommended workflow:
\`\`\`bash
terraform init
terraform fmt -recursive
terraform validate
terraform plan
terraform apply
\`\`\`
EOF
log "Terraform starting point: generated/$PROVIDER-$CLUSTER_NAME"
}
vm_hint() {
cat <<EOF
Mode $MODE requires cloud VMs, SSH access, firewall rules, and internal IP addresses first.
After that, use the chapter on VM-based cloud clusters:
- kubeadm: full Kubernetes bootstrap
- k3s/RKE2/k0s: lighter distributions
- Talos: declarative OS and cluster operation
EOF
}
create() {
case "$MODE" in
terraform) terraform_hint ;;
vm-kubeadm|vm-k3s|vm-rke2|vm-k0s|talos) vm_hint ;;
managed)
case "$PROVIDER" in
aws) aws_create_managed ;;
azure) azure_create_managed ;;
gcp) gcp_create_managed ;;
digitalocean) do_create_managed ;;
linode) linode_create_managed ;;
oci) oci_create_hint ;;
ibm) ibm_create_hint ;;
*) fail "Managed mode for provider '$PROVIDER' is not directly implemented in this script. Use the provider chapter or Terraform." ;;
esac ;;
autopilot)
case "$PROVIDER" in
aws) aws_create_autopilot ;;
azure) azure_create_autopilot ;;
gcp) gcp_create_autopilot ;;
*) fail "Autopilot/Automatic mode is not defined for '$PROVIDER'." ;;
esac ;;
*) fail "Unknown mode: $MODE" ;;
esac
}
versions() {
case "$PROVIDER" in
aws) aws_versions ;;
azure) azure_versions ;;
gcp) gcp_versions ;;
digitalocean) do_versions ;;
linode) linode_versions ;;
*) log "No version query is implemented for provider '$PROVIDER'. Use the provider CLI help." ;;
esac
}
kubeconfig() {
case "$PROVIDER" in
aws) aws_kubeconfig ;;
azure) azure_kubeconfig ;;
gcp) gcp_kubeconfig ;;
digitalocean) do_kubeconfig ;;
linode) linode_create_managed ;;
*) fail "kubeconfig action is not implemented for '$PROVIDER'." ;;
esac
}
delete() {
case "$PROVIDER" in
aws) aws_delete ;;
azure) azure_delete ;;
gcp) gcp_delete ;;
digitalocean) do_delete ;;
linode) linode_delete ;;
*) fail "Delete action is not implemented for '$PROVIDER'. Use the provider chapter and manually check remaining resources." ;;
esac
}
print_context
case "$ACTION" in
create) create ;;
validate) common_validate ;;
delete) delete ;;
versions) versions ;;
kubeconfig) kubeconfig ;;
*) fail "Unknown action: $ACTION" ;;
esac
10.3 Limits of the Script
The script is a common entry point, not a complete platform engine. It is suitable for:
- workshop clusters,
- demo clusters,
- reproducible lab environments,
- orientation around provider tools,
- controlled dry runs.
For production-like environments, use instead:
- Terraform/OpenTofu with remote state and a review process,
- Pulumi with stack configuration and a secrets backend,
- Cluster API with a management cluster,
- Crossplane with clean composition design,
- GitOps for add-ons and workloads,
- dedicated security, logging, backup, and upgrade processes.
11. AWS: Amazon EKS and EC2-Based Variants
AWS provides several Kubernetes paths:
| Variant | Tooling | Suitable For |
|---|---|---|
EKS with eksctl |
eksctl, aws, kubectl |
Fast workshops and reproducible CLI labs. |
| EKS with AWS CLI | aws eks, aws iam, aws ec2 |
Fine-grained control, but significantly more setup. |
| EKS Auto Mode | aws, Terraform |
Less routine operation for compute, storage, and networking. |
| EKS Fargate | eksctl, aws |
Pod execution without own worker nodes for suitable workloads. |
| EKS with Terraform/OpenTofu | Terraform AWS Provider, EKS modules | Team-ready infrastructure with review and state. |
| kOps on AWS | kops, S3 state, Route 53 |
Self-managed Kubernetes plus AWS infrastructure. |
| EC2 + kubeadm | aws, ssh, kubeadm |
Deep admin labs with control plane and etcd. |
| EC2 + k3s/RKE2/k0s/Talos | Provider CLI, SSH or Talos API | Lightweight or hardened clusters on VMs. |
| ROSA | rosa, oc, aws |
Managed OpenShift on AWS. |
11.1 EKS with eksctl
For workshops, eksctl is usually the fastest EKS path.
export AWS_REGION="eu-central-1"
export CLUSTER_NAME="workshop-eks"
aws sts get-caller-identity
eksctl create cluster \
--name "$CLUSTER_NAME" \
--region "$AWS_REGION" \
--nodes 2 \
--node-type t3.medium \
--managed \
--tags owner="$USER",environment=workshop,ttl=8h
aws eks update-kubeconfig \
--region "$AWS_REGION" \
--name "$CLUSTER_NAME"
kubectl get nodes -o wide
Teardown
eksctl delete cluster \
--name "$CLUSTER_NAME" \
--region "$AWS_REGION"
Check for remaining resources afterwards:
aws elbv2 describe-load-balancers --region "$AWS_REGION" | jq '.LoadBalancers[].LoadBalancerName'
aws ec2 describe-volumes --region "$AWS_REGION" --filters Name=status,Values=available | jq '.Volumes[].VolumeId'
aws ec2 describe-addresses --region "$AWS_REGION" | jq '.Addresses[].PublicIp'
11.2 EKS with an eksctl Configuration File
For repeatable labs, YAML configuration is better than long CLI commands.
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
name: workshop-eks
region: eu-central-1
tags:
owner: workshop
environment: lab
ttl: "8h"
managedNodeGroups:
- name: system
instanceType: t3.medium
desiredCapacity: 2
minSize: 1
maxSize: 3
volumeSize: 40
privateNetworking: true
eksctl create cluster -f eksctl-workshop.yaml
11.3 EKS with AWS CLI
The pure AWS CLI path is intentionally more verbose because IAM roles, networking, cluster, and node groups are managed separately.
Typical workflow:
# 1. Create an IAM role for the EKS control plane.
# 2. Provision VPC, subnets, security groups, and routing.
# 3. Create the EKS cluster.
# 4. Create a node group or Fargate profile.
# 5. Write kubeconfig.
# 6. Check add-ons.
Minimal cluster call, if IAM and network already exist:
aws eks create-cluster \
--region eu-central-1 \
--name workshop-eks \
--role-arn arn:aws:iam::<account-id>:role/EKSClusterRole \
--resources-vpc-config subnetIds=subnet-a,subnet-b,securityGroupIds=sg-a,endpointPublicAccess=true,endpointPrivateAccess=true
aws eks wait cluster-active \
--region eu-central-1 \
--name workshop-eks
aws eks update-kubeconfig \
--region eu-central-1 \
--name workshop-eks
Node group:
aws eks create-nodegroup \
--region eu-central-1 \
--cluster-name workshop-eks \
--nodegroup-name system \
--node-role arn:aws:iam::<account-id>:role/EKSNodeRole \
--subnets subnet-a subnet-b \
--scaling-config minSize=1,maxSize=3,desiredSize=2 \
--instance-types t3.medium \
--disk-size 40
11.4 EKS Auto Mode
EKS Auto Mode reduces routine tasks for compute, storage, and networking. Use this path when a workshop should focus more on application and platform behavior and less on node operations.
Checkpoints:
- IAM roles and cluster roles must match the Auto Mode model.
- Networking and subnets must be tagged appropriately.
- Workload requirements must fit the supported operational boundaries.
- Cost model and automatic capacity provisioning must be understood.
11.5 EKS Fargate
EKS Fargate can run pods without your own EC2 worker nodes. This is useful for certain lab or isolated workloads, but it is not suitable for all Kubernetes features.
eksctl create fargateprofile \
--cluster workshop-eks \
--region eu-central-1 \
--name fp-demo \
--namespace demo
11.6 EKS with Terraform/OpenTofu
Recommended structure:
infra/aws-eks/
├── versions.tf
├── providers.tf
├── variables.tf
├── network.tf
├── eks.tf
├── outputs.tf
└── terraform.tfvars.example
Example with a module-based approach:
module "eks" {
source = "terraform-aws-modules/eks/aws"
version = "~> 21.0"
name = var.cluster_name
kubernetes_version = var.kubernetes_version
vpc_id = module.vpc.vpc_id
subnet_ids = module.vpc.private_subnets
eks_managed_node_groups = {
system = {
instance_types = ["t3.medium"]
min_size = 1
max_size = 3
desired_size = 2
}
}
tags = var.tags
}
Workflow:
terraform init
terraform fmt -recursive
terraform validate
terraform plan -out tfplan
terraform apply tfplan
aws eks update-kubeconfig --region eu-central-1 --name workshop-eks
kubectl get nodes -o wide
11.7 kOps on AWS
kOps is suitable when you want to build a self-managed Kubernetes cluster on AWS and have kOps manage the AWS infrastructure as well.
export KOPS_STATE_STORE=s3://my-kops-state-store
export NAME=workshop.example.com
kops create cluster \
--name "$NAME" \
--zones eu-central-1a,eu-central-1b,eu-central-1c \
--node-count 2 \
--node-size t3.medium \
--control-plane-size t3.medium \
--dns-zone example.com
kops update cluster --name "$NAME" --yes --admin
kops validate cluster --wait 10m
Teardown:
kops delete cluster --name "$NAME" --yes
11.8 EC2 + kubeadm
You need at least:
- three EC2 instances for a small HA lab, or one control plane plus workers for a fundamentals lab,
- security groups for SSH, Kubernetes API, kubelet, CNI, and pod network,
- internal DNS names or fixed private IPs,
- optionally a Network Load Balancer in front of the control plane,
containerd,kubeadm,kubelet, andkubectlon the nodes.
Basic bootstrap pattern:
sudo kubeadm init \
--control-plane-endpoint "api.workshop.internal:6443" \
--pod-network-cidr "10.244.0.0/16" \
--upload-certs
mkdir -p "$HOME/.kube"
sudo cp /etc/kubernetes/admin.conf "$HOME/.kube/config"
sudo chown "$(id -u):$(id -g)" "$HOME/.kube/config"
kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml
11.9 ROSA
ROSA is relevant when you want to provision OpenShift rather than upstream Kubernetes on AWS.
Typical workflow:
rosa login
rosa verify quota
rosa create account-roles --mode auto --hosted-cp
rosa create cluster --hosted-cp --cluster-name workshop-rosa --region eu-central-1
rosa describe cluster --cluster workshop-rosa
For Kubernetes workshops without an OpenShift focus, EKS is usually more appropriate. For platform workshops around OpenShift concepts, Operators, Routes, Security Context Constraints, and oc, ROSA is useful.
12. Azure: AKS and VM-Based Variants
Azure provides several main paths for Kubernetes:
| Variant | Tooling | Suitable For |
|---|---|---|
| AKS Standard | az aks |
Standard workshops, production-like clusters, node pools. |
| AKS Automatic | Azure CLI, portal, Terraform depending on support | App-focused clusters with stronger platform automation. |
| AKS + ACR | az acr, az aks |
Image build and deployment labs. |
| AKS + Azure CNI Overlay | az aks |
Network-oriented labs with Azure integration. |
| Azure VM + kubeadm | az vm, ssh, kubeadm |
Admin and bootstrap labs. |
| Azure VM + k3s/RKE2/k0s/Talos | az vm, SSH or Talos API |
Lightweight and special-purpose variants. |
| AKS with Terraform/OpenTofu | Terraform AzureRM Provider | Reproducible team and platform operation. |
| Azure Arc-enabled Kubernetes | az connectedk8s |
Registration of external clusters in Azure. |
12.1 AKS with Azure CLI
export LOCATION="westeurope"
export RESOURCE_GROUP="rg-workshop-aks"
export CLUSTER_NAME="workshop-aks"
az login
az account show -o table
az group create \
--name "$RESOURCE_GROUP" \
--location "$LOCATION" \
--tags owner="$USER" environment=workshop ttl=8h
az aks create \
--resource-group "$RESOURCE_GROUP" \
--name "$CLUSTER_NAME" \
--location "$LOCATION" \
--node-count 2 \
--node-vm-size Standard_B2s \
--generate-ssh-keys \
--enable-managed-identity \
--tags owner="$USER" environment=workshop ttl=8h
az aks get-credentials \
--resource-group "$RESOURCE_GROUP" \
--name "$CLUSTER_NAME" \
--overwrite-existing
kubectl get nodes -o wide
12.2 AKS with ACR Integration
export ACR_NAME="acrworkshop$RANDOM"
az acr create \
--resource-group "$RESOURCE_GROUP" \
--name "$ACR_NAME" \
--sku Basic
az aks update \
--resource-group "$RESOURCE_GROUP" \
--name "$CLUSTER_NAME" \
--attach-acr "$ACR_NAME"
12.3 AKS Node Pools
az aks nodepool add \
--resource-group "$RESOURCE_GROUP" \
--cluster-name "$CLUSTER_NAME" \
--name apps \
--node-count 2 \
--node-vm-size Standard_D2s_v5 \
--labels workload=apps \
--tags owner="$USER" environment=workshop
kubectl get nodes --show-labels
12.4 AKS Automatic
AKS Automatic is intended for scenarios where you want to configure fewer platform details yourself. Before a workshop, check current CLI support and whether your organization allows this variant.
Checkpoints:
- supported regions,
- network model,
- identity model,
- allowed add-ons,
- limitations around node configuration,
- cost model,
- compatibility with Ingress, Gateway API, CSI, and observability.
12.5 AKS with Terraform/OpenTofu
resource "azurerm_resource_group" "this" {
name = var.resource_group_name
location = var.location
tags = var.tags
}
resource "azurerm_kubernetes_cluster" "this" {
name = var.cluster_name
location = azurerm_resource_group.this.location
resource_group_name = azurerm_resource_group.this.name
dns_prefix = var.cluster_name
kubernetes_version = var.kubernetes_version
default_node_pool {
name = "system"
node_count = 2
vm_size = "Standard_B2s"
}
identity {
type = "SystemAssigned"
}
tags = var.tags
}
Workflow:
terraform init
terraform validate
terraform plan -out tfplan
terraform apply tfplan
az aks get-credentials --resource-group rg-workshop-aks --name workshop-aks --overwrite-existing
kubectl get nodes -o wide
12.6 Azure VM + kubeadm
Example VM creation:
az network vnet create \
--resource-group "$RESOURCE_GROUP" \
--name vnet-k8s \
--address-prefixes 10.50.0.0/16 \
--subnet-name subnet-k8s \
--subnet-prefixes 10.50.1.0/24
az vm create \
--resource-group "$RESOURCE_GROUP" \
--name cp-1 \
--image Ubuntu2404 \
--size Standard_B2s \
--vnet-name vnet-k8s \
--subnet subnet-k8s \
--admin-username azureuser \
--generate-ssh-keys
Afterwards, install containerd, kubelet, kubeadm, and kubectl, initialize the control plane, and join worker nodes.
12.7 Teardown
az aks delete \
--resource-group "$RESOURCE_GROUP" \
--name "$CLUSTER_NAME" \
--yes --no-wait
# For dedicated workshop Resource Groups, deleting the entire group is often cleaner:
az group delete \
--name "$RESOURCE_GROUP" \
--yes --no-wait
Check whether public IPs, disks, or Load Balancers still exist:
az resource list --resource-group "$RESOURCE_GROUP" -o table
13. Google Cloud: GKE and Compute Engine Variants
Google Cloud provides two very important GKE modes: Autopilot and Standard.
| Variant | Tooling | Suitable For |
|---|---|---|
| GKE Autopilot | gcloud container clusters create-auto |
App-focused workshops with little node operation. |
| GKE Standard zonal | gcloud container clusters create --zone |
Small labs, not for high availability. |
| GKE Standard regional | gcloud container clusters create --region |
More production-like labs and regional control plane. |
| GKE with Terraform/OpenTofu | Google Provider | Reproducible clusters and network resources. |
| Compute Engine + kubeadm | gcloud compute, SSH, kubeadm |
Deep admin labs. |
| Compute Engine + k3s/RKE2/k0s/Talos | gcloud compute, SSH or Talos API |
Special-purpose and edge labs. |
| GKE Enterprise / Fleet | gcloud container fleet |
Multi-cluster and enterprise management. |
13.1 Prepare Project
export PROJECT_ID="my-gcp-project"
export REGION="europe-west3"
export CLUSTER_NAME="workshop-gke"
gcloud auth login
gcloud config set project "$PROJECT_ID"
gcloud services enable container.googleapis.com compute.googleapis.com iam.googleapis.com
13.2 GKE Autopilot
gcloud container clusters create-auto "$CLUSTER_NAME" \
--region "$REGION" \
--labels owner="$USER",environment=workshop,ttl=8h
gcloud container clusters get-credentials "$CLUSTER_NAME" \
--region "$REGION"
kubectl get nodes -o wide
Autopilot is a good fit when you want to practice Kubernetes objects, resource requests, workload security, and Services without operating node pools.
13.3 GKE Standard Regional
gcloud container clusters create "$CLUSTER_NAME" \
--region "$REGION" \
--num-nodes 1 \
--machine-type e2-standard-2 \
--enable-ip-alias \
--labels owner="$USER",environment=workshop,ttl=8h
gcloud container clusters get-credentials "$CLUSTER_NAME" \
--region "$REGION"
For workshops, --num-nodes 1 on regional clusters must be evaluated carefully because GKE interprets the node count per zone. Check the actual number of nodes after creation:
kubectl get nodes -o wide
gcloud container node-pools list --cluster "$CLUSTER_NAME" --region "$REGION"
13.4 Add GKE Node Pool
gcloud container node-pools create apps \
--cluster "$CLUSTER_NAME" \
--region "$REGION" \
--machine-type e2-standard-2 \
--num-nodes 1 \
--node-labels workload=apps
13.5 GKE with Terraform/OpenTofu
resource "google_container_cluster" "this" {
name = var.cluster_name
location = var.region
remove_default_node_pool = true
initial_node_count = 1
release_channel {
channel = "REGULAR"
}
ip_allocation_policy {}
}
resource "google_container_node_pool" "system" {
name = "system"
location = var.region
cluster = google_container_cluster.this.name
node_count = 2
node_config {
machine_type = "e2-standard-2"
labels = {
workload = "system"
}
}
}
13.6 Compute Engine + kubeadm
VM foundation:
gcloud compute networks create k8s-workshop --subnet-mode custom
gcloud compute networks subnets create k8s-workshop-eu \
--network k8s-workshop \
--region "$REGION" \
--range 10.60.1.0/24
gcloud compute firewall-rules create k8s-workshop-ssh \
--network k8s-workshop \
--allow tcp:22
gcloud compute instances create cp-1 \
--zone europe-west3-a \
--machine-type e2-standard-2 \
--image-family ubuntu-2404-lts-amd64 \
--image-project ubuntu-os-cloud \
--subnet k8s-workshop-eu
After that, follow the same kubeadm, k3s, RKE2, k0s, or Talos bootstrap as with other VM providers.
13.7 Teardown
gcloud container clusters delete "$CLUSTER_NAME" \
--region "$REGION" \
--quiet
# Check remaining resources
gcloud compute forwarding-rules list
gcloud compute disks list
gcloud compute addresses list
14. Additional Managed Kubernetes Providers
The big three providers cover many environments, but other providers are relevant for workshops, price comparisons, edge labs, or regional requirements. The following examples are intentionally written as starting points. Always check current regions, versions, prices, quotas, and CLI help.
14.1 DigitalOcean Kubernetes (DOKS)
DOKS is a pragmatic managed Kubernetes path for small to medium-sized labs.
Primary tool: doctl
export CLUSTER_NAME="workshop-doks"
export REGION="fra1"
doctl auth init
doctl kubernetes options regions
doctl kubernetes options sizes
doctl kubernetes options versions
doctl kubernetes cluster create "$CLUSTER_NAME" \
--region "$REGION" \
--size s-2vcpu-4gb \
--count 2 \
--tag environment:workshop \
--tag ttl:8h
doctl kubernetes cluster kubeconfig save "$CLUSTER_NAME"
kubectl get nodes -o wide
Teardown:
doctl kubernetes cluster delete "$CLUSTER_NAME" --dangerous --force
14.2 Akamai/Linode Kubernetes Engine (LKE)
LKE is a managed Kubernetes offering with CLI, API, and Terraform paths.
Primary tool: linode-cli
export CLUSTER_NAME="workshop-lke"
export REGION="eu-central"
export K8S_VERSION="<version-from-linode-cli-lke-versions-list>"
linode-cli lke versions-list
linode-cli regions list
linode-cli lke cluster-create \
--label "$CLUSTER_NAME" \
--region "$REGION" \
--k8s_version "$K8S_VERSION" \
--node_pools.type g6-standard-2 \
--node_pools.count 2 \
--tags workshop
linode-cli lke clusters-list
# Then insert the cluster ID:
linode-cli lke kubeconfig-view <cluster-id> --text | base64 -d > kubeconfig-lke.yaml
export KUBECONFIG="$PWD/kubeconfig-lke.yaml"
kubectl get nodes -o wide
Teardown:
linode-cli lke cluster-delete <cluster-id>
14.3 Oracle Kubernetes Engine (OKE)
OKE requires OCI-specific IDs for compartment, VCN, and subnets. For workshops, prepare these values in advance.
Primary tool: oci
export OCI_COMPARTMENT_ID="ocid1.compartment..."
export OCI_VCN_ID="ocid1.vcn..."
export OCI_ENDPOINT_SUBNET_ID="ocid1.subnet..."
export OCI_LB_SUBNET_ID="ocid1.subnet..."
export CLUSTER_NAME="workshop-oke"
export K8S_VERSION="<supported-version>"
oci ce cluster create \
--compartment-id "$OCI_COMPARTMENT_ID" \
--name "$CLUSTER_NAME" \
--kubernetes-version "$K8S_VERSION" \
--vcn-id "$OCI_VCN_ID" \
--endpoint-subnet-id "$OCI_ENDPOINT_SUBNET_ID" \
--service-lb-subnet-ids "[\"$OCI_LB_SUBNET_ID\"]"
Generate kubeconfig through the OCI console or CLI after cluster creation. For real environments, deliberately add node pools, network rules, IAM policies, and Load Balancer subnets.
14.4 IBM Cloud Kubernetes Service (IKS)
IKS is managed through the IBM Cloud CLI and the Kubernetes Service plugin.
Primary tool: ibmcloud
ibmcloud login --sso
ibmcloud plugin install container-service
ibmcloud ks cluster versions
ibmcloud ks locations
ibmcloud ks flavors
# Example structure; adjust exact options to account, region, VPC, and cluster type:
ibmcloud ks cluster create vpc-gen2 \
--name workshop-iks \
--zone eu-de-1 \
--flavor bx2.4x16 \
--workers 2
ibmcloud ks cluster config --cluster workshop-iks
kubectl get nodes -o wide
Teardown:
ibmcloud ks cluster rm --cluster workshop-iks
14.5 Civo Kubernetes
Civo is useful for fast labs with a simple CLI. Check region, available versions, and marketplace add-ons first.
Primary tool: civo
civo apikey add workshop <api-key>
civo apikey current workshop
civo region ls
civo kubernetes versions
civo kubernetes create workshop-civo \
--region FRA1 \
--nodes 2 \
--size g4s.kube.medium \
--save \
--switch
kubectl get nodes -o wide
Teardown:
civo kubernetes delete workshop-civo --yes
14.6 Vultr Kubernetes Engine (VKE)
VKE is a managed Kubernetes path for simple lab clusters. Check plans and regions first.
Primary tool: vultr-cli
vultr-cli account info
vultr-cli kubernetes versions
vultr-cli regions list
vultr-cli kubernetes create \
--region fra \
--label workshop-vke \
--version <version> \
--node-pools '[{"node_quantity":2,"label":"system","node_plan":"vc2-2c-4gb"}]'
vultr-cli kubernetes list
vultr-cli kubernetes config <cluster-id> > kubeconfig-vke.yaml
export KUBECONFIG="$PWD/kubeconfig-vke.yaml"
kubectl get nodes -o wide
Teardown:
vultr-cli kubernetes delete <cluster-id>
14.7 Scaleway Kapsule/Kosmos
Scaleway offers Kapsule as managed Kubernetes and Kosmos for multi-cloud node pools. Use the CLI help for current parameters.
Primary tool: scw
scw init
scw k8s version list
scw k8s cluster create \
name=workshop-kapsule \
region=fr-par \
version=<version> \
cni=cilium
scw k8s pool create \
cluster-id=<cluster-id> \
name=system \
node-type=DEV1-M \
size=2
scw k8s kubeconfig get <cluster-id> > kubeconfig-scaleway.yaml
export KUBECONFIG="$PWD/kubeconfig-scaleway.yaml"
kubectl get nodes -o wide
Teardown:
scw k8s cluster delete <cluster-id> with-additional-resources=true
14.8 OVHcloud Managed Kubernetes Service
OVHcloud Managed Kubernetes is often provisioned via the portal, API, or Terraform. For workshops, Terraform is usually cleaner than manual API calls.
Primary tool: ovh or Terraform
# Typical IaC path:
terraform init
terraform plan
terraform apply
# Then write kubeconfig from output or provider resource and validate:
kubectl get nodes -o wide
Keep project ID, region, node pool flavors, and network ranges traceable in Terraform state.
14.9 Alibaba Cloud ACK
ACK is Alibaba Cloud's managed Kubernetes path. For international workshops, check account, region, and compliance requirements particularly carefully.
Primary tool: aliyun
aliyun configure
aliyun cs GET /api/v1/clusters
# Cluster creation requires VPC, subnets, worker configuration, and RAM roles depending on the mode.
# For reproducible environments, prefer Terraform or the official ACK CLI examples.
14.10 Tencent Kubernetes Engine (TKE)
TKE is Tencent Cloud's managed Kubernetes path. Check region, VPC, subnets, cluster type, and IAM first.
Primary tool: tccli
tccli configure
tccli tke DescribeClusters
# For reproducible environments, prefer Terraform or an approved organizational module.
14.11 Hetzner Cloud with k3s, kubeadm, or Talos
Hetzner Cloud is typically not a classic managed Kubernetes path, but it is a very popular VM basis for k3s, kubeadm, RKE2, k0s, or Talos.
Primary tool: hcloud
export HCLOUD_TOKEN="<token>"
hcloud server-type list
hcloud image list
hcloud server create \
--name cp-1 \
--type cx22 \
--image ubuntu-24.04 \
--location fsn1 \
--ssh-key <ssh-key-name> \
--label environment=workshop \
--label ttl=8h
hcloud server list
After that, install k3s, kubeadm, RKE2, k0s, or Talos as described in the VM chapter.
Teardown:
hcloud server delete cp-1 worker-1 worker-2
hcloud load-balancer list
hcloud volume list
15. VM-Based Cloud Clusters with kubeadm, k3s, RKE2, k0s, and Talos
Managed Kubernetes is not always the right learning goal. For admin, architecture, and troubleshooting workshops, you often need real nodes, system services, and controllable control-plane components.
15.1 Common VM Target Architecture
flowchart TD
Cloud[Cloud Provider] --> Net[VPC/VNet/Subnets]
Net --> FW[Security Groups or Firewalls]
Net --> LB[Optional API Load Balancer]
FW --> CP1[Control Plane 1]
FW --> CP2[Control Plane 2]
FW --> CP3[Control Plane 3]
FW --> W1[Worker 1]
FW --> W2[Worker 2]
LB --> CP1
LB --> CP2
LB --> CP3
CP1 --> Etcd[etcd]
CP2 --> Etcd
CP3 --> Etcd
W1 --> Pods[Workloads]
W2 --> Pods
15.2 Minimum Ports
| Purpose | Port | Direction | Note |
|---|---|---|---|
| SSH | 22/tcp | Admin to nodes | Preferably only your own admin IP. |
| Kubernetes API | 6443/tcp | Admin/nodes to control plane | Public only with strong restrictions. |
| etcd | 2379-2380/tcp | Control plane internal | Do not expose publicly. |
| kubelet | 10250/tcp | Control plane to nodes | Do not expose publicly. |
| NodePort | 30000-32767/tcp | Optional external | Use sparingly for labs. |
| CNI | varies | Node to node | Depends on Flannel, Calico, Cilium, Weave, Canal. |
| Ingress | 80/443/tcp | Client to Load Balancer | Through cloud Load Balancer or NodePort. |
15.3 kubeadm on Cloud VMs
kubeadm is useful for production-like learning goals because it makes the control plane, etcd, kubelet, container runtime, and CNI explicit.
Preparation on All Nodes
sudo swapoff -a
sudo sed -i.bak '/ swap / s/^/#/' /etc/fstab
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilter
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
sudo sysctl --system
Container runtime:
sudo apt update
sudo apt install -y containerd
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml >/dev/null
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
sudo systemctl restart containerd
sudo systemctl enable containerd
Kubernetes packages:
sudo apt update
sudo apt install -y apt-transport-https ca-certificates curl gpg
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.36/deb/Release.key | \
sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.36/deb/ /' | \
sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt update
sudo apt install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
Control plane:
sudo kubeadm init \
--control-plane-endpoint "api.k8s.internal:6443" \
--pod-network-cidr "10.244.0.0/16" \
--upload-certs
mkdir -p "$HOME/.kube"
sudo cp /etc/kubernetes/admin.conf "$HOME/.kube/config"
sudo chown "$(id -u):$(id -g)" "$HOME/.kube/config"
CNI example with Flannel:
kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml
CNI example with Cilium:
helm repo add cilium https://helm.cilium.io/
helm repo update
helm install cilium cilium/cilium \
--namespace kube-system \
--set kubeProxyReplacement=false
Worker join:
sudo kubeadm join api.k8s.internal:6443 \
--token <token> \
--discovery-token-ca-cert-hash sha256:<hash>
15.4 k3s on Cloud VMs
k3s is very useful for small remote labs.
Single node:
curl -sfL https://get.k3s.io | sh -
sudo k3s kubectl get nodes
sudo cat /etc/rancher/k3s/k3s.yaml
Server with explicit token:
export K3S_TOKEN="$(openssl rand -hex 24)"
curl -sfL https://get.k3s.io | \
K3S_TOKEN="$K3S_TOKEN" \
sh -s - server --cluster-init
Agent:
curl -sfL https://get.k3s.io | \
K3S_URL="https://<server-ip>:6443" \
K3S_TOKEN="<token>" \
sh -
15.5 RKE2 on Cloud VMs
RKE2 is interesting when you want to work closer to Rancher, stronger security defaults, or enterprise-like operating models.
Server:
curl -sfL https://get.rke2.io | sudo sh -
sudo systemctl enable rke2-server.service
sudo systemctl start rke2-server.service
sudo journalctl -u rke2-server -f
Kubeconfig:
sudo cp /etc/rancher/rke2/rke2.yaml ./rke2.yaml
sudo chown "$USER:$USER" ./rke2.yaml
export KUBECONFIG="$PWD/rke2.yaml"
kubectl get nodes
15.6 k0s on Cloud VMs
k0s is a lightweight Kubernetes distribution with a simple installation model.
curl -sSLf https://get.k0s.sh | sudo sh
sudo k0s install controller --single
sudo k0s start
sudo k0s status
sudo k0s kubeconfig admin > kubeconfig-k0s.yaml
export KUBECONFIG="$PWD/kubeconfig-k0s.yaml"
kubectl get nodes
15.7 Talos on Cloud VMs
Talos is not primarily managed through SSH, but through talosctl and machine configurations.
Basic workflow:
talosctl gen config workshop-talos https://<api-endpoint>:6443
talosctl apply-config \
--insecure \
--nodes <control-plane-ip> \
--file controlplane.yaml
talosctl bootstrap --nodes <control-plane-ip>
talosctl kubeconfig --nodes <control-plane-ip>
kubectl get nodes
Talos fits well when you want to demonstrate immutable infrastructure, clear machine state, and hardening-oriented operating models.
16. Infrastructure as Code
16.1 Why IaC?
Manual CLI commands are good for learning. For repeatable workshops and teams, you need IaC.
| Criterion | CLI | Terraform/OpenTofu | Pulumi | Cluster API |
|---|---|---|---|---|
| Entry barrier | Low | Medium | Medium | High |
| Reproducibility | Medium | High | High | High |
| Reviewability | Medium | High | High | High |
| Multi-cloud | Medium | High | High | High |
| Cluster lifecycle | Provider-dependent | Good | Good | Very good for fleets |
| Drift detection | Weak | Good | Good | Kubernetes-native |
| Secret handling | Manual | Backend-dependent | Stack-dependent | Kubernetes-native plus external systems |
16.2 Terraform/OpenTofu Standard Structure
infra/cloud-kubernetes/
├── README.md
├── versions.tf
├── providers.tf
├── variables.tf
├── locals.tf
├── network.tf
├── cluster.tf
├── nodepools.tf
├── addons.tf
├── outputs.tf
├── terraform.tfvars.example
└── environments/
├── aws-dev.tfvars
├── azure-dev.tfvars
└── gcp-dev.tfvars
16.3 Common Terraform Variables
variable "cluster_name" {
type = string
description = "Name of the Kubernetes cluster."
}
variable "region" {
type = string
description = "Cloud region."
}
variable "kubernetes_version" {
type = string
description = "Desired Kubernetes version."
default = null
}
variable "node_count" {
type = number
description = "Desired number of nodes for simple labs."
default = 2
}
variable "tags" {
type = map(string)
description = "Required tags for governance and cost control."
default = {
environment = "workshop"
ttl = "8h"
}
}
16.4 Terraform Workflow
terraform fmt -recursive
terraform init
terraform validate
terraform plan -out tfplan
terraform apply tfplan
terraform output
Teardown:
terraform plan -destroy -out destroy.tfplan
terraform apply destroy.tfplan
16.5 Remote State
For teams, state does not belong on a local disk.
| Cloud | Typical Backend |
|---|---|
| AWS | S3 with DynamoDB locking or HCP Terraform. |
| Azure | Azure Storage Account with Blob Container. |
| Google Cloud | GCS Bucket. |
| Multi-cloud | HCP Terraform, GitLab/OpenTofu backend, S3-compatible backend. |
16.6 Pulumi
Pulumi is useful if your team prefers to describe infrastructure in TypeScript, Python, Go, C#, or YAML.
Example structure:
pulumi/cloud-kubernetes/
├── Pulumi.yaml
├── Pulumi.dev.yaml
├── index.ts
└── package.json
Typical workflow:
pulumi login
pulumi stack init dev
pulumi config set cloud:region eu-central-1
pulumi preview
pulumi up
pulumi stack output kubeconfig --show-secrets > kubeconfig.yaml
export KUBECONFIG="$PWD/kubeconfig.yaml"
kubectl get nodes
16.7 IaC Acceptance Criteria
An IaC provisioning setup is suitable for workshops and teams when:
fmt,validate, andplanrun reproducibly,- no real secrets are stored in the repository,
- provider and module versions are pinned,
- state backend and locking are defined,
- tags and naming scheme are enforced,
- teardown is tested,
- outputs provide kubeconfig, cluster name, region, and important endpoints,
- add-ons are not installed through unreviewed ad-hoc commands.
17. Cluster API, Crossplane, and GitOps
17.1 Cluster API
Cluster API is a Kubernetes-native approach to create, update, and operate clusters declaratively. It requires a management cluster where Cluster API controllers and providers run.
flowchart TD
Admin[kubectl against Management Cluster] --> Mgmt[Management Cluster]
Mgmt --> CAPI[Cluster API Controller]
CAPI --> Provider[Infrastructure Provider: AWS/Azure/GCP/OpenStack/vSphere/...]
Provider --> Infra[Cloud Infrastructure]
CAPI --> Workload[Workload Cluster]
Workload --> Apps[Apps and Add-ons]
Basic idea:
clusterctl init --infrastructure aws
clusterctl generate cluster workshop-capi \
--kubernetes-version v1.36.0 \
--control-plane-machine-count=1 \
--worker-machine-count=2 \
> workshop-capi.yaml
kubectl apply -f workshop-capi.yaml
kubectl get clusters
kubectl get machines
Use Cluster API when you want to operate many clusters in a standardized way. For a single beginner workshop, it is often too much infrastructure.
17.2 Crossplane
Crossplane exposes cloud resources through Kubernetes APIs. Platform teams can provide abstracted resources such as CompositeCluster, XNetwork, or XPostgresInstance.
flowchart LR
User[Developer Team] --> Claim[Claim: KubernetesCluster]
Claim --> Crossplane[Crossplane]
Crossplane --> Composition[Composition]
Composition --> ProviderAWS[AWS Provider]
Composition --> ProviderAzure[Azure Provider]
Composition --> ProviderGCP[GCP Provider]
ProviderAWS --> EKS[EKS]
ProviderAzure --> AKS[AKS]
ProviderGCP --> GKE[GKE]
Crossplane fits well for self-service platforms. For pure cluster fundamentals, it is an advanced topic.
17.3 GitOps After Provisioning
After cluster creation, not everything should be installed by hand. A clean path is:
- Provision the cluster.
- Validate kubeconfig.
- Create bootstrap namespace.
- Install Argo CD or Flux.
- Connect add-on repository.
- Connect workload repository.
- Detect and correct drift via GitOps.
Argo CD quick start:
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
kubectl -n argocd rollout status deployment/argocd-server
Flux quick start:
flux check --pre
flux bootstrap github \
--owner=<github-user-or-org> \
--repository=<repo> \
--branch=main \
--path=clusters/workshop
18. Networking, DNS, Ingress, and Gateway API
18.1 Cloud Network Decisions
| Decision | Questions |
|---|---|
| Public or private | Is the Kubernetes API publicly reachable or only through VPN/bastion? |
| Subnets | Do you separate control plane, nodes, Load Balancers, and databases? |
| NAT | Do private nodes need outbound internet access? |
| Pod IP model | Does the provider use native pod IPs, overlay, or CNI-specific models? |
| DNS | Who creates DNS records for Load Balancers? |
| TLS | Where does TLS terminate: Load Balancer, Ingress, Gateway, or pod? |
| Egress | How is outbound traffic controlled and logged? |
18.2 Ingress Controller
These controllers are typical for workshops:
| Controller | Suitable For |
|---|---|
| ingress-nginx | General labs, portable, well documented. |
| AWS Load Balancer Controller | EKS with AWS ALB/NLB integration. |
| Application Gateway Ingress Controller | AKS with Azure Application Gateway. |
| GKE Ingress | GKE-native Load Balancer integration. |
| Traefik | k3s and small lab environments. |
| HAProxy Ingress | Advanced routing and performance scenarios. |
Example ingress-nginx:
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx \
--namespace ingress-nginx \
--create-namespace
kubectl -n ingress-nginx get svc
18.3 Gateway API
Gateway API separates responsibilities better than classic Ingress models:
flowchart TD
Infra[Platform/Infra Team] --> GatewayClass[GatewayClass]
Platform[Platform Team] --> Gateway[Gateway]
App[App Team] --> Route[HTTPRoute]
GatewayClass --> Controller[Gateway Controller]
Gateway --> Controller
Route --> Gateway
Controller --> LB[Cloud Load Balancer]
Route --> Service[Service]
Service --> Pods[Pods]
Example structure:
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: public
namespace: ingress
spec:
gatewayClassName: nginx
listeners:
- name: http
protocol: HTTP
port: 80
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: demo
namespace: demo
spec:
parentRefs:
- name: public
namespace: ingress
rules:
- backendRefs:
- name: demo
port: 80
18.4 ExternalDNS
ExternalDNS can create DNS records from Kubernetes objects. Use it only with tightly scoped zones and permissions.
Checkpoints:
- Is the cluster allowed to modify DNS zones?
- Which record types are allowed?
- Are private and public zones separated?
- Are TTLs appropriate?
- Is there protection against accidental wildcard or root-zone changes?
18.5 cert-manager
cert-manager automates certificates. For cloud workshops, HTTP-01 and DNS-01 are relevant.
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm upgrade --install cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--set crds.enabled=true
kubectl -n cert-manager rollout status deployment/cert-manager
19. Storage and Data Persistence
19.1 Cloud Storage Matrix
| Provider | Block Storage | Typical CSI Path |
|---|---|---|
| AWS | EBS | Amazon EBS CSI Driver. |
| Azure | Managed Disks | Azure Disk CSI Driver. |
| Google Cloud | Persistent Disk | GCE Persistent Disk CSI Driver. |
| DigitalOcean | Volumes Block Storage | DigitalOcean CSI. |
| Akamai/Linode | Block Storage | Linode CSI. |
| Oracle Cloud | Block Volumes | OCI CSI. |
| IBM Cloud | Block/File Storage | IBM Cloud CSI. |
| Hetzner | Volumes | hcloud CSI for VM-based clusters. |
19.2 Check StorageClasses
kubectl get storageclass
kubectl describe storageclass
19.3 PVC Test
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: data
namespace: demo
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
kubectl create namespace demo
kubectl apply -f pvc.yaml
kubectl -n demo get pvc
kubectl -n demo describe pvc data
19.4 Stateful Workloads
For production-like cloud workloads, you need:
- explicit StorageClass selection,
- backup and restore test,
- encryption,
- snapshot strategy,
- zone concept,
- PDBs and maintenance windows,
- monitoring for IOPS, latency, and capacity.
20. Observability and Operations
20.1 Minimum Standard
kubectl get nodes -o wide
kubectl top nodes
kubectl top pods -A
kubectl get events -A --sort-by=.lastTimestamp | tail -50
20.2 Metrics Server
Many managed clusters ship with Metrics Server or provide it as an add-on. Check first:
kubectl get deployment metrics-server -n kube-system
kubectl top nodes
Installation, if required:
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
20.3 Prometheus and Grafana
For workshops, kube-prometheus-stack may be sufficient:
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm upgrade --install monitoring prometheus-community/kube-prometheus-stack \
--namespace monitoring \
--create-namespace
20.4 Logs
Cloud-native options:
| Provider | Typical Service |
|---|---|
| AWS | CloudWatch Container Insights, EKS add-ons, Fluent Bit. |
| Azure | Azure Monitor Container Insights. |
| Google Cloud | Cloud Logging and Cloud Monitoring. |
| DigitalOcean | External stacks or marketplace add-ons. |
| IBM/Oracle/Akamai | Provider logging plus own stack. |
Portable workshop path:
- Loki,
- Promtail or Grafana Alloy,
- Grafana,
- structured application logs.
20.5 Operational Signals
| Signal | Tool |
|---|---|
| Node Ready | kubectl get nodes, cloud console. |
| Pod Scheduling | Events, scheduler logs. |
| Rollouts | kubectl rollout status. |
| Resources | Metrics Server, Prometheus. |
| Network | Debug pods, CNI metrics. |
| Storage | PVC events, CSI controller logs. |
| API Access | Audit logs, cloud IAM logs. |
21. Security Baseline
21.1 Identity
| Provider | Workload Identity Model |
|---|---|
| AWS | IAM Roles for Service Accounts or EKS Pod Identity. |
| Azure | Microsoft Entra Workload ID / Managed Identity. |
| Google Cloud | Workload Identity Federation for GKE. |
| Other | Provider-specific Service Accounts or external secret systems. |
Rule: Avoid static cloud keys in Kubernetes Secrets. Use short-lived tokens and Workload Identity where possible.
21.2 RBAC Minimum
kubectl auth can-i get pods -A
kubectl auth can-i '*' '*' --as system:serviceaccount:demo:app
kubectl get clusterrolebinding
A workshop ServiceAccount should not receive cluster-admin privileges.
21.3 Pod Security Standards
Namespace with the Restricted profile:
kubectl label namespace demo \
pod-security.kubernetes.io/enforce=restricted \
pod-security.kubernetes.io/audit=restricted \
pod-security.kubernetes.io/warn=restricted
21.4 SecurityContext Example
apiVersion: apps/v1
kind: Deployment
metadata:
name: demo
namespace: demo
spec:
replicas: 2
selector:
matchLabels:
app: demo
template:
metadata:
labels:
app: demo
spec:
serviceAccountName: demo
securityContext:
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
containers:
- name: app
image: nginx:1.27-alpine
ports:
- containerPort: 8080
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
memory: 256Mi
21.5 Network Isolation
Default deny:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
namespace: demo
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
Allow DNS:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-dns
namespace: demo
spec:
podSelector: {}
policyTypes:
- Egress
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
21.6 Image Security
- No
latesttags for workshops that need to be reproducible. - Use images from trusted registries.
- Introduce signatures and SBOMs in advanced labs.
- Check Admission Policies for registries and SecurityContext.
22. Validation
22.1 Cluster Baseline
kubectl version
kubectl cluster-info
kubectl get nodes -o wide
kubectl get namespaces
kubectl get pods -A
kubectl get events -A --sort-by=.lastTimestamp | tail -50
22.2 Workload Test
kubectl create namespace demo
kubectl create deployment web -n demo --image=nginx:1.27-alpine --replicas=2
kubectl expose deployment web -n demo --port=80 --target-port=80 --type=ClusterIP
kubectl rollout status deployment/web -n demo
kubectl get pods,svc -n demo -o wide
22.3 Internal Service Test
kubectl run curl -n demo --rm -it --image=curlimages/curl:8.10.1 --restart=Never -- \
curl -sS http://web.demo.svc.cluster.local
22.4 LoadBalancer Test
kubectl patch svc web -n demo -p '{"spec":{"type":"LoadBalancer"}}'
kubectl get svc web -n demo -w
If EXTERNAL-IP remains pending, check:
- Cloud Controller Manager,
- Load Balancer quota,
- subnet tags,
- Security Groups or firewalls,
- provider-specific controllers,
- Kubernetes Events for the Service.
22.5 Storage Test
kubectl get storageclass
kubectl apply -f pvc.yaml
kubectl -n demo get pvc
22.6 RBAC Test
kubectl auth can-i get pods -n demo
kubectl auth can-i create deployments -n demo
kubectl auth can-i '*' '*' --all-namespaces
22.7 Validation Log
| Check | Command | Expected Result |
|---|---|---|
| API reachable | kubectl cluster-info |
API Server responds. |
| Nodes ready | kubectl get nodes |
All nodes are Ready. |
| System pods | kubectl get pods -n kube-system |
CoreDNS and CNI are running. |
| DNS | Curl pod to Service DNS | DNS resolves Service. |
| Scheduling | Test Deployment | Pods are scheduled. |
| Service | ClusterIP test | Service routes to pods. |
| LoadBalancer | Service type LoadBalancer | External IP or hostname appears. |
| Storage | PVC | PVC becomes Bound. |
| Metrics | kubectl top nodes |
Metrics are available. |
| Teardown | Provider CLI or IaC destroy | No remaining resources. |
23. Decommissioning
23.1 Basic Rules
- Delete Kubernetes LoadBalancer Services first.
- Then remove add-ons that manage cloud resources.
- Then delete the cluster.
- Then check remaining resources.
- Then clean up state, kubeconfig, and temporary secrets.
23.2 Provider Commands
# AWS EKS
eksctl delete cluster --name workshop-eks --region eu-central-1
# Azure AKS
az aks delete --resource-group rg-workshop-aks --name workshop-aks --yes --no-wait
# Google GKE
gcloud container clusters delete workshop-gke --region europe-west3 --quiet
# DigitalOcean DOKS
doctl kubernetes cluster delete workshop-doks --dangerous --force
# Linode LKE
linode-cli lke cluster-delete <cluster-id>
# Terraform/OpenTofu
terraform plan -destroy -out destroy.tfplan
terraform apply destroy.tfplan
23.3 Find Remaining Resources
# Kubernetes side
kubectl get svc -A --field-selector spec.type=LoadBalancer
kubectl get pvc -A
# AWS
aws elbv2 describe-load-balancers --region eu-central-1
aws ec2 describe-volumes --region eu-central-1 --filters Name=status,Values=available
aws ec2 describe-addresses --region eu-central-1
# Azure
az resource list --tag environment=workshop -o table
# Google Cloud
gcloud compute forwarding-rules list
gcloud compute disks list
gcloud compute addresses list
# DigitalOcean
doctl compute load-balancer list
doctl compute volume list
doctl compute floating-ip list
23.4 Clean Local Traces
kubectl config get-contexts
kubectl config delete-context <context>
kubectl config delete-cluster <cluster>
kubectl config unset users.<user>
rm -f kubeconfig-*.yaml
rm -rf generated/*
24. Troubleshooting
24.1 Cluster Is Not Created
| Symptom | Probable Cause | Check |
|---|---|---|
| Quota error | Not enough vCPUs, IPs, Load Balancers, or disks | Check provider quota page and CLI. |
| IAM error | Missing permissions for roles, cluster, network | aws sts get-caller-identity, az account show, gcloud auth list. |
| Region not supported | Kubernetes version or node size unavailable in region | Query version and size lists. |
| Network error | CIDR collision, missing subnets, blocked routing | Check VPC/VNet/subnets. |
| Node group remains unhealthy | Node role, image, subnet, or security group is wrong | Check provider events and node logs. |
24.2 kubectl Does Not Connect
kubectl config current-context
kubectl config view --minify
kubectl cluster-info
Check:
- Was kubeconfig written?
- Does the context point to the right cluster?
- Is cloud authentication still valid?
- Is the API public or only privately reachable?
- Do you need VPN, bastion, or Cloud Shell?
24.3 Nodes Are Not Ready
kubectl describe node <node>
kubectl get pods -n kube-system -o wide
kubectl get events -A --sort-by=.lastTimestamp | tail -100
Common causes:
- CNI not installed or faulty,
- kubelet cannot reach the runtime,
- Security Groups block node communication,
- DNS or egress missing,
- Cloud Controller Manager or CSI lacks permissions,
- DiskPressure, MemoryPressure, or PIDPressure.
24.4 Pods Remain Pending
kubectl -n demo describe pod <pod>
kubectl get nodes -o wide
kubectl describe node <node>
Check:
- resource requests too high,
- no matching node labels,
- taints without tolerations,
- PVC not bound,
- Pod Security Admission blocks the pod,
- image pull secret missing.
24.5 LoadBalancer Remains Pending
kubectl -n demo describe svc web
kubectl get events -n demo --sort-by=.lastTimestamp
Provider-specific checks:
- AWS: subnet tags, AWS Load Balancer Controller, Security Groups, IAM.
- Azure: Public IP, Load Balancer SKU, Managed Identity, subnet permissions.
- GKE: firewall, Service annotation, quota.
- DOKS/LKE: Load Balancer quota and region.
- VM cluster: MetalLB or your own Load Balancer is missing.
24.6 PVC Remains Pending
kubectl describe pvc -n demo data
kubectl get storageclass
kubectl get events -n demo --sort-by=.lastTimestamp
Check:
- StorageClass exists,
- default StorageClass is set,
- CSI controller is running,
- cloud roles allow disk creation,
- zone/region matches node and volume,
- disk quota is sufficient.
24.7 terraform apply Fails
terraform fmt -recursive
terraform validate
terraform plan
terraform state list
Typical causes:
- provider version incompatible,
- state lock stuck,
- local credentials expired,
- resource already exists,
- name collision,
- organizational policy blocks public resources,
- provider API has eventual consistency.
25. Workshop Labs
25.1 Lab 1: Check Provider Context
Goal: Validate your local CLI environment.
Tasks:
- Install
kubectl,helm,jq, and the relevant provider CLI. - Log in to the provider.
- Check account, region, quotas, and available Kubernetes versions.
- Document region, cluster name, and teardown command.
Acceptance:
kubectl version --client
helm version
aws sts get-caller-identity || true
az account show -o table || true
gcloud auth list || true
25.2 Lab 2: Create Managed Cluster
Goal: Create a small managed Kubernetes cluster.
Tasks:
- Choose EKS, AKS, GKE, DOKS, or LKE.
- Run the provisioning script without
--executefirst. - Run the actual provisioning.
- Write kubeconfig.
- Validate nodes and system pods.
Acceptance:
kubectl get nodes -o wide
kubectl get pods -A
25.3 Lab 3: Workload, Service, and LoadBalancer
Goal: Verify that workloads and cloud LoadBalancers work.
Tasks:
- Create namespace
demo. - Deploy
nginx. - Expose the Service as
LoadBalancer. - Check external reachability.
- Delete the Service and check remaining cloud resources.
Acceptance:
kubectl -n demo get deploy,pod,svc -o wide
curl -I http://<external-ip-or-hostname>
25.4 Lab 4: Storage
Goal: Validate dynamic volume provisioning.
Tasks:
- List StorageClasses.
- Create a PVC.
- Mount the PVC into a pod.
- Write test data.
- Delete pod and verify persistence.
25.5 Lab 5: VM-Based Cluster
Goal: Compare managed Kubernetes with a self-managed VM cluster.
Tasks:
- Create three small cloud VMs.
- Open only required ports.
- Install kubeadm or k3s.
- Install CNI.
- Validate nodes, DNS, and Service routing.
25.6 Lab 6: IaC Workflow
Goal: Create the same cluster reproducibly through Terraform/OpenTofu.
Tasks:
- Create
versions.tf,providers.tf,variables.tf, andcluster.tf. - Run
terraform fmt,validate,plan, andapply. - Write kubeconfig.
- Validate cluster.
- Run
terraform destroy.
25.7 Lab 7: Security Baseline
Goal: Harden a workshop namespace.
Tasks:
- Create namespace
secure-demo. - Enable Pod Security Restricted.
- Create ServiceAccount, Role, and RoleBinding.
- Apply default-deny NetworkPolicy.
- Deploy an app with SecurityContext.
26. Checklists
26.1 Before Provisioning
- Cloud account and budget are approved.
- Region is allowed.
- Quotas are sufficient.
- Naming scheme is defined.
- Tags are defined.
- Network CIDRs do not collide.
- Public API endpoints are assessed.
- CLI login works.
- Kubernetes version is checked.
- Teardown path is documented.
26.2 After Provisioning
- kubeconfig points to the correct cluster.
- Nodes are
Ready. - System pods are running.
- DNS works.
- Test Deployment works.
- Service works.
- LoadBalancer works or is intentionally not installed.
- StorageClass is available or intentionally unavailable.
- Metrics work or are planned as an add-on.
- Tags are visible on cloud resources.
26.3 Before Production-Like Usage
- Network design is reviewed.
- Private API or access restriction is implemented.
- IAM/RBAC follows least privilege.
- Pod Security Standards are active.
- Secret strategy is defined.
- Audit logs are active.
- Observability is installed.
- Backup and restore are tested.
- Upgrade path is documented.
- Cost monitoring is active.
26.4 Teardown
- LoadBalancer Services deleted.
- PVCs and volumes checked.
- Cluster deleted.
- Node pools deleted.
- Public IPs deleted.
- Disks and snapshots checked.
- DNS records removed.
- kubeconfig contexts deleted.
- IaC state updated.
- Cost view checked.
27. Sources and References
This file is aligned with primary sources and official documentation. Before real provisioning, always check the current provider pages because CLI options, versions, regions, and product variants can change.
| Topic | Reference |
|---|---|
| Kubernetes production environments | https://kubernetes.io/docs/setup/production-environment/ |
| kubeadm | https://kubernetes.io/docs/reference/setup-tools/kubeadm/ |
| Creating a kubeadm cluster | https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/ |
| AWS EKS | https://docs.aws.amazon.com/eks/latest/userguide/ |
| EKS with eksctl | https://docs.aws.amazon.com/eks/latest/userguide/getting-started-eksctl.html |
| EKS kubeconfig | https://docs.aws.amazon.com/eks/latest/userguide/create-kubeconfig.html |
| Azure AKS | https://learn.microsoft.com/azure/aks/ |
| AKS with Azure CLI | https://learn.microsoft.com/azure/aks/learn/quick-kubernetes-deploy-cli |
| Google Kubernetes Engine | https://cloud.google.com/kubernetes-engine/docs |
| GKE gcloud cluster commands | https://cloud.google.com/sdk/gcloud/reference/container/clusters |
| DigitalOcean Kubernetes | https://docs.digitalocean.com/products/kubernetes/ |
| doctl Kubernetes | https://docs.digitalocean.com/reference/doctl/reference/kubernetes/ |
| Akamai/Linode LKE | https://techdocs.akamai.com/cloud-computing/docs/linode-kubernetes-engine |
| Oracle Kubernetes Engine | https://docs.oracle.com/en-us/iaas/Content/ContEng/Concepts/contengoverview.htm |
| IBM Cloud Kubernetes Service | https://cloud.ibm.com/docs/containers |
| kOps | https://kops.sigs.k8s.io/ |
| Cluster API | https://cluster-api.sigs.k8s.io/ |
| Terraform EKS Tutorial | https://developer.hashicorp.com/terraform/tutorials/kubernetes/eks |
| Terraform AWS EKS Module | https://registry.terraform.io/modules/terraform-aws-modules/eks/aws/latest |
| Pulumi Kubernetes | https://www.pulumi.com/docs/integrations/clouds/kubernetes/ |
| Gateway API | https://gateway-api.sigs.k8s.io/ |
| Helm | https://helm.sh/docs/ |
| cert-manager | https://cert-manager.io/docs/ |
| Argo CD | https://argo-cd.readthedocs.io/ |
| Flux | https://fluxcd.io/ |