Summary
Recently I've worked on deploying Kubernetes clusters using Azure acs-engine as a part of our automation effort at the firm. It is a very useful tool that generates ARM (Azure Resource Manager) templates with ready to-use deployments of Virtual Machines and Kubernetes components on them. It also supports other container orchestrators, such as DC/OS or Swar. However, I will focus only on Kubernetes in this article as this is the only ochestrator I use. Acs-engine saves a lot of time compared to other custom solutions used to deploy Kubernetes cluster onto Azure cloud. It also gives a lot of flexibility with the configuration of the cluster. It is, however, open source tool with support being provided only by the community.
This is a list of some of the noteworthy things that will give you more overview of what an acs-engine is and whether this tool can be of use for you or not.
Some of them will also be valid for Azure ACS service as the deployment process is almost the same.
Acs-engine does not have an SLA
Azure can guarantee only SLA for the underlying virtual machines, but not for the cluster created by acs-engine. You need to be aware of that, because when something brakes, you will be the one that is called. Some companies require an SLA on the underlying platform in order not to require to maintain the platform themselves. Acs-engine is a no-go for such companies.
Acs-engine uses hyperkube to deploy Kubernetes components
All Kubernetes master components, except for kubelet, are deployed using hyperkube docker image. Acs-engine leverages a Kubernetes' feature called Static Pods. This enables the Kubernetes components to be brought up by kubelet itself that is deployed to the master node. It is very smart to leverage Kubernetes to bring up itself to live.
By default you cannot change tiller or kubernetes-dashboard deplyoment
By default, acs-engine deploys kubernetes-dashboard and tiller (helm's server component). Because of the way they are deployed, you cannot simply change them using kubectl apply. The cluster will re-apply old configuration if you try this. You would need to ssh into master machine and change manually file
in order to upgrade tiller.
The community says that in the recent version of acs-engine this is fixed by new addon annotation. However, I have not tested if kubectl apply works in the newest version.