My k8s adventure
Intro
The story starts in the middle of 2018 when I’m hearing about sports betting (football) often at that time. It was my first contact with the betting world and I was curious so I sign-up in some russian betting websites that someone advised.
Later I found my self having some difficulties so I started following some facebook groups and started thinking that I could improve my chances, or at least make my life easier, if I automate some stuff.
A side project was born!
I have always had side projects to help me learn, try new things and experiment a bit more than the experience of my daily job allows. I really enjoy this and have a lot of fun with them 😄
It started by being a simple bash script that was reading information about games periodically and was triggering some Ubuntu desktop notifications any time something interesting was up.
Results were not awesome, but it really helped a lot! At the time I add already a okish relation with tipster from a facebook group so I asked him what he thought about this idea and he liked it so much that we started working on it together.
In the following months I put together a lot of stuff. I started without docker, then moved to docker and docker-compose. I started with a monolithic and went for multiple services. Started with some EC2 VM in AWS with docker, then moved to spot instances with auto-scaling and then K8S (installed with kubeadm).
At some point, I’m adding more and more features trying to push further the functionalities of this automation. The AWS bills 💸 are increasing and at some point, because we were still developing and had no customers yes, this is no longer good for me to keep it in the cloud and started to look for alternatives. I never considered abandon the project though.
At this point, the project is already quite big. 2 years passed since I started it
I had built not just the whole backend but also developer a android application and a web frontend. All by my self…
I’m talking about more than 10 services, each with its own responsibility. I can not say they were well built, but it was a huge huge amount of work…. and I was not even being paid yet for it.
Fixing the AWS bill
Raspberry PI 4 had come out recently at the time and it was a solution that seemed to suited all my requirements. The idea was to host my own k8s cluster at home. That way I wouldn’t pay for traffic and since raspberries consume so little electricity, the costs were basically just the investment on the hardware.
I had not much money at the time so I could never for example spend hundreds in Intel NUCs…
To start I calculated that 2 raspberry PI 4 with 4GB RAM would be enough. Which cost me about 200 € …
| Product | Value |
|---|---|
| 2x Raspberry PI 4 4GB RAM | 97.6 € |
| 2x Charger | 14.4 € |
| 9 Layer Acrylic Case + 9 Cooling Fan | 26.8 € |
| 3x Heatsink | 6 € |
| Gigabit Switch 8 ports | 22 € |
| 2x 32GB MicroSD Cards | 14.8€ |
| 4x cat6 cable 1m | 10.1€ |
| — | — |
| Sum | 191.7 € |
yes… I was already thinking on adding more PIs to my cluster 😉
For the OS, I went for ubuntu. I’m used to it since I started working professionally in 2010 so it was an easy choice.
Actually this step is was not as easy at the time. I needed to use a 64bit OS and at the time there was none available for raspberry pi yet.
I used then this https://github.com/satmandu/docker-rpi4-imagebuilder to install the OS at the time.
After we have the OS installed, We need to enable linux cgroup (control group) - a kernel feature - so we can use containers. Among other things It that allows processes to prioritize and limit resources such as CPU, Memory and disk I/O. Container applications such as docker or containerd rely on. So whether you go for ubuntu or debian, be sure to add cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory to your cmd boot file or docker/containerd will likely not work properly.
Regarding the k8s flavor, Initially I kept the same installation process I had in before in AWS with kubeadm, but soon I moved to K3S because it was simpler and light. Seemed ideal for raspberry PIs.
With k3s it is really easy to setup your k8s cluster. You just need to follow their documentation which is basically:
|
|
Then do similar in each of your nodes
|
|
And that was it. I installed k8s/k3s, then installed my apps and I quickly i managed to have what I had in AWS running from my homelab cluster.
This was the beginning of my story with k8s and raspberries at home 😄