Terraform module for creating Kubernetes cluster running on Container Linux by CoreOS in an AWS VPC
make allwill simply spin up a default cluster; and, since it is based on Terraform, customization is much easier than CloudFormation.
The default configuration includes Kubernetes add-ons: DNS, Dashboard and UI.
# prereqs $ brew update && brew install awscli cfssl jq kubernetes-cli terraform
build artifacts and deploy cluster
$ make all
$ kubectl get nodes
$ kubectl get pods --namespace=kube-system
verify dns - run after addons have fully loaded
$ kubectl exec busybox -- nslookup kubernetes
$ make dashboard
obliterate the cluster and all artifacts
$ make clean
component / tool
|Container Linux by CoreOS||1409.7.0, 1465.3.0, 1492.1.0|
|aws-cli||aws-cli/1.11.129 Python/2.7.10 Darwin/16.7.0 botocore/1.5.92|
Quick install prerequisites on Mac OS X with Homebrew:
$ brew update && brew install awscli cfssl jq kubernetes-cli terraform
make allwill create: - AWS Key Pair (PEM file) - AWS VPC with private and public subnets - Route 53 internal zone for VPC - Bastion host - Certificate Authority server - etcd3 cluster bootstrapped from Route 53 - High Availability Kubernetes configuration (masters running on etcd nodes) - Autoscaling worker node group across subnets in selected region - kube-system namespace and addons: DNS, UI, Dashboard
To open dashboard:
To display instance information:
To display status:
To destroy, remove and generally undo everything:
make cleanshould be idempotent - should an error occur simply try running the command again and things should recover from that point.
Tack works in three phases:
The purpose of this phase is to prep the environment for Terraform execution. Some tasks are hard or messy to do in Terraform - a little prep work can go a long way here. Determining the Container Linux by CoreOS AMI for a given region, channel and VM Type for instance is easy enough to do with a simple shell script.
Terraform does the heavy lifting of resource creation and sequencing. Tack uses local modules to partition the work in a logical way. Although it is of course possible to do all of the Terraform work in a single
.tffile or collection of
.tffiles, it becomes unwieldy quickly and impossible to debug. Breaking the work into local modules makes the flow much easier to follow and provides the basis for composing variable solutions down the track - for example converting the worker Auto Scaling Group to use spot instances.
Once the infrastructure has been configured and instantiated it will take some time for it to settle. Waiting for the 'master' ELB to become healthy is an example of this.
Like many great tools, tack has started out as a collection of scripts, makefiles and other tools. As tack matures and patterns crystalize it will evolve to a Terraform plugin and perhaps a Go-based cli tool for 'init-ing' new cluster configurations. The tooling will compose Terraform modules into a solution based on user preferences - think
npm initor better yet yeoman.
curl --cacert /etc/kubernetes/ssl/ca.pem --cert /etc/kubernetes/ssl/k8s-etcd.pem --key /etc/kubernetes/ssl/k8s-etcd-key.pem https://etcd.test.kz8s:2379/health openssl x509 -text -noout -in /etc/kubernetes/ssl/ca.pem openssl x509 -text -noout -in /etc/kubernetes/ssl/k8s-etcd.pem
To access Elasticseach and Kibana first start
$ kubectl proxy Starting to serve on localhost:8001
If you have an existing VPC you'd like to deploy a cluster into, there is an option for this with tack.
In order to test existing VPC support, we need to generate a VPC and then try the overrides with it. After that we can clean it all up. These instructions are meant for someone wanting to ensure that the tack existing VPC code works properly. * Run
make allto generate a VPC with Terraform * Edit terraform.tfstate * Search for the VPC block and cut it out and save it somewhere. Look for "path": ["root","vpc"] * Run
make cleanto remove everything but the VPC and associated networking (we preserved it in the previous step) * Edit as per instructions above * Run
make allto test out using an existing VPC * Cleaning up: * Re-insert the VPC block into terraform.tfstate * Run
make cleanto clean up everything