The goal: get a kubernetes cluster up and running, and install the dashboard. (Can you tell I'm a kubernetes beginner?).
The case I purchased didn't quite fit the PoE hats, so it's a little extra jampacked, but I avoided shorting anything, so hopefully we're okay.
OS Setup
I used the latest Raspian Lite, here was my process:
- Find the SD card with
fdisk(it changed with different micro sd's). - Burn image with
dd, specifically: dd bs=4M if=2020-02-13-raspbian-buster-lite.img of=/dev/sdX conv=fsync. - Mount
/dev/sdX1and enable SSH withcd /path/to/mnt/location && touch ssh. - Mount
/dev/sdX2and enable static ip by editing the bottom of the/path/to/mnt/etc/dhcpcd.conf, and uncommenting the following lines at the bottom (mine are edited):
# Example static IP configuration:
interface eth0
static ip_address=192.168.1.20/24
static routers=192.168.1.1
static domain_name_servers=192.168.1.1
Rinse and repeat this process 4 times, I chose 192.168.1.20, ...21, ...22, and ...23. I also reserved those IPs on my router, so they wouldn't get DHCP'd to another device if any of my Pi's go offline. After they're all up, I
sudo raspi-config on each to set the hostnames (kmaster, kworker1, kworker2, kworker3) and set the timezones (unclear if this matters to me). Installing k3s
The cluster is all up and running, so now I can install k3s, first on master (192.168.1.20). After SSHing onto it:
-> curl -sfL https://get.k3s.io | sh -
foo...
bar...
installing...
-> sudo cat /var/lib/rancher/k3s/server/node-token
K104c102328fkj9kw9a01a0422e5fe7ec320b0274f3e35a721cac588b8472e5d1dd::server:19238307020bdb828665190548c71c81
Now, to setup the other pi's as workers, you need to set the token, and URL of the master server. So, export the variables K3S_URL and K3S_TOKEN and install like before.
ssh pi@IP "export K3S_URL="https://192.168.1.20:6443" && export K3S_TOKEN="K104c102328fkj9kw9a01a0422e5fe7ec320b0274f3e35a721cac588b8472e5d1dd::server:19238307020bdb828665190548c71c81" && curl -sfL https://get.k3s.io | sh -"
That installs and starts the k3s-agent service, and connects them to master. With that, we should have a up and running k3s cluster, which can be controlled by
kubectl from the master node.-> sudo kubectl get nodes
NAME STATUS ROLES AGE VERSION
kworker2 Ready <none> 47h v1.17.4+k3s1
kworker3 Ready <none> 47h v1.17.4+k3s1
kworker1 Ready <none> 47h v1.17.4+k3s1
kmaster Ready master 47h v1.17.4+k3s1
Kubernetes dashboard
It's as simple as running this command:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0/aio/deploy/recommended.yaml Then, following this post, you can grab the token like so:
sudo kubectl -n kubernetes-dashboard describe secret $(sudo kubectl -n kubernetes-dashboard get secret | grep admin-user | awk '{print $1}') | awk '/token:/{print $2}'
To actually connect, you need to use
kubectl proxy, and if you run it from your master node, you need to proxy with SSH to view the page from your machine. ssh -L 8001:localhost:8001 pi@IP. Now you can navigate to the dashboard at http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/.Note: this breaks on restart. The problem is something to do with iptables, seemingly related to docker, and can be fixed by flushing the iptables, I have a script for this that I run on restart, if broken.
#!/bin/sh
sudo pkill kubectl
sudo systemctl stop k3s
sudo systemctl stop docker
sudo iptables --flush
sudo iptables -tnat --flush
sudo systemctl start k3s
sudo systemctl start docker
Future plans
I will rewrite the webstack for my site with microservices, so I can learn how to write microservices. I will hopefully get to play with OpenFaaS in that process. I would also like to get a pihole up and running again, which should be easy enough.