Hi all!
I have a couple of months to create and deploy a small cluster for running docker containers.
The cluster will consist of 3 master nodes and some workers. When it is ready, it will consist of about 15 servers.
I have little experience with Docker (managing some containers on my home server), I have spent the last 4 or 5 weeks studying and testing with Kubernetes and I think it’s a little overkill for what it’s going to take. You run the risk of adding unnecessary complexity.
I am seeing that instead Docker Swarm seems easier to set up and manage.
To consider that I will be on my own to manage it.
What do you think?
Thanks!
I haven’t used Docker Swarm (I have barely used Docker Compose), but I have run a couple on-prem Kubernetes clusters (at my house and for clients at my day job) and cloud Kubernetes clusters, so I can speak to how complex it is it set up and run.
My background is systems administration, engineering, IT, and now DevOps. I’ve been using Linux since Ubuntu 6.06.
I set up my Kubernetes cluster with kubeadm because I wanted to learn, and it took me about a weekend to get my single master, two worker cluster up and running. I think you could probably do this using k3s much faster and have less learning curve (you don’t have to care as much about Container Network Interfaces, for example, because k3s makes that decision for you.)
There is a lot of documentation out there on Kubernetes. Helm as a “package manager” (really a templating engine) can be nice if the software you want to deploy has a Helm chart that is well written. Writing your own Helm charts can be a learning process, I’ve modified some but not written one from scratch yet.
Kubernetes releases new versions about quarterly. I’ve done several upgrades on my primary home cluster over the course of the past 2 years and they’ve been pretty smooth, about an hour of time investment
totaleach. And remember, I’m on the more nerdy and complex flavor of Kubernetes. I think with k3s these would be even smoother and quicker.I feel like Kubernetes knowledge is probably more valuable out in the industry if that’s a factor for you. I haven’t come across any Docker Swarm clusters in my DevOps travels, just Kubernetes and some HashiCorp Nomad.
I’m curious to see what folks say about Docker Swarm. If you have any questions about Kubernetes or running your workload on it, I’d be happy to try to help!
K8s all the way. That’s the general consensus in the industry. Swarm is not popular in comparison. My company sells a product in two flavors, SaaS and On Prem. Both run on K8s. We supply Helm charts and whatnot for On Prem installations (including in private cloud). There is good demand for K8s skill in the labor market.
I agree with this. I think single node or not the industry is moving towards Kubernetes for container orchestration. Docker has showed their evil intentions and it’s time to leave them in the past. Even podman has native kubernetes manifest support (albeit limited last i checked) as @rs5th@lemmy.scottlabs.io pointed out there’s good avenues to take if you want to avoid the complexities of kubernetes like k3s.
Is k3s good for production?
We’re using it in production at my day job in a couple of places.
How is it about the reverse proxy like traefik? Are you using nginx ingress with k3s?
Sorry for the many questions! Thanks!
Yep, using ingress-nginx on k3s as well.
Homeland: portainer Work: Openshift
My thoughts is how granular you need permissions to be.
Kubernetes all the way. The Pod definitions look a little different from compose files, and the networking is a little strange at first, but it’s just better in every way.
For one, you don’t get any of the dumb compromises that Docker Swarm has, and the industry has standardized on Kubernetes, so you’ll have much more options with Kubernetes.
Docker swarm is pretty easy to set up and use (and lets you use compose files directly!) and is probably more than enough average self-hosters/homelabbers, but if you want to do something super fancy related to clustering there’s a good chance you’ll hit functionality walls quickly.
Kubernetes is a pain to set up but is very flexible and ‘scalable’ to incredible levels, while being massive overkill for most applications.
I’m not really sure what the state of docker swarm is, but I’d recommend taking a look at Hashicorp Nomad too. It’s between swarm and k8s in complexity, but it’s been great to work with so far.
Kunernetes is what I see most companies using, but it’s also usually overly complicated for what they need.
What kind of workloads do you plan on running?
overly complicated for what they need.
👀 glances at the 4 Kubernetes clusters running in my house / personal VPS
There’s nothing wrong with having an over complicated homelab! I manage dozens of kubernetes clusters at work and it really is a good skill to have for career reasons. My dig at it is mostly that a lot of smaller teams or companies will jump to it without really looking at their other options first.
Oh sure, I totally get it. Once I was fully bought in to k8s and GitOps for one environment, it just made sense to have my other ones use the same tooling.
I thought Swarm is long dead.