I have broken my Kubernetes cluster

Categories: Kubernetes
My Kubernetes cluster is down My single-node baremetal cluster using kubeadm and on calico networking plugin is down (link to the article here). The machine won't startup anymore and there is very mysterius error from cephlib reporting that it was unable to start. Few days ago I was playing with rook (https://github.com/rook/rook). Rook uses ceph to give you easy to use persistent volume experience. Everything was fine at this point until I started doing research on advanced Prometheus configuration.

Read More →

Persistent Volumes in your home Kubernetes cluster

Categories: Kubernetes
Summary We will deploy rook cluster storage service into the baremetal cluster. Prerequisites You should have a Baremetal Kubernetes cluster at least version 1.8. If you don't, I recommend taking a look at: Home Kubernetes cluster in 15 minutes with kubeadm and calico According to rook's documentation you should also have at least 5GB of disk space on the drive you want to use for storage. Let's do it First we need to prepare the baremetal machine for rook's persistent volumes:

Read More →

Watch out for faulty ingress definitions

Categories: Kubernetes
Summary Ingress objects can interfere between each other in the cluster. If you deploy incorrect ingress definition (and kubernetes won't detect the error), Nginx Ingress Controller will fall into restart loop and won't accept any new configuration. This will lead to broken deployments from that time onwards. Description I have been trying out how wildcards in Nginx Ingress Controller work. This is required when you want to have an URL prefixed with the application's language or a country.

Read More →

Kubernetes automatic SSL certificate provisioning

Categories: Kubernetes
Summary There is an open source project from JetStack called kube-lego. It allows you to automatically request SSL certificates for your Kubernetes cluster using Let’s Encrypt free service. Working with Let’s Encrypt using Kube-lego is quite straightforward. Nginx Ingress Controller has built-in support for kube-lego. Having RBAC might seem like a complication but in fact it doesn’t add much of a complexity to the solution. Requirements Kubernetes 1.8.0 or higher with Nginx Ingress Controller deployed 30 minutes of spare time Description Let’s Encrypt is a service that provides you with automatic TLS/SSL certificate provisioning for your website.

Read More →

Home Kubernetes cluster in 15 minutes

Categories: Kubernetes
Description This guide will let you install Kubernetes 1.8 with RBAC enabled on so called “baremetal” along with Calico network plugin. Although there is nothing specific to baremetal, that’s what I will use to install it. This cluster can be then used for small projects, once exposed to the Internet. That’s excatly what I use it for :) Requirements Spare machine with Ubuntu installed and at least a few GB of RAM.

Read More →

Always set limits to containers running in your cluster

Categories: Kubernetes
History about a Prometheus eating up 20GB of RAM 2017-11-30 UPDATE As those few days have passed I have not had any problems with Kubernetes cluster being unresponsive. Therefore this article concludes a few weeks of investigation why machine could entirely freeze. Set limits to your pods so that they won't kill your node. The History WeaveWorks Cloud DaemonSet deploys by default Prometheus to the cluster. Prometheus scrapes metrics out of your cluster and stores them and creates time-series data out of them ( this might not be the accurate description of what Prometheus does, but it's good enough for what just hapened ).

Read More →

Baremetal Kubernetes on Ubuntu unresponsive

Categories: Kubernetes
EDIT 24-11-2017 The solution below did not help me fix freezing cluster. I have identified since then, though, another problem with the setup. Prometheus was eating up all memory of a machine. The link to the article: https://cwienczek.com/lesson-1-always-set-limits-to-containers-running-in-your-cluster/ Summary If you’ve: - installed Kubernetes on your baremetal machine - that machine is running Ubuntu - and your machine is suddenly unresponsive, then: Make sure you don’t have docker 1.

Read More →

Setting up secure helm chart repository on Azure Blob Storage

Categories: Kubernetes
Kubernetes helm repository supports only basic authentication at the time of writing this article. There is, though, another and perhaps simpler way as of helm 2.7.0. Using Azure Blob Storage you can easily make your helm repository private. Requirements Time: ~10 minutes Helm Package Manager 2.7.0-rc1 or later Microsoft Azure account, at least with permissions to create azure storage account Azure CLI, tested on 2.0.19 Darwin Helm chart that you can upload to the cloud Summary Create azure storage account in one of your resource groups Add blob storage container to Azure Storage Account and set access to private Go to Storage account -> Shared Access Signature and generate read-only credentials for helm users The url to your repository will be: https://[azure_storage_name].

Read More →