Monitoring Kubernetes clusters on AWS, GCP and Azure using Prometheus Operator and Grafana
Note: the work on this repository is now based on CoreOS's kube-prometheus and it will be the default option for Kubernetes 1.7.X and up. For 1.5.X and 1.6.X you can deploy a simpler solution, located in
./basicdirectory. The purpose of this project is to provide a simple and interactive method to deploy and configure Prometheus on Kubernetes, especially for the users that are not using Helm.
StatefulSetsfor persistent storage
kubectlconfigured
If you are running Kubernetes 1.12 or higher you will also need to run cAdvisor on your cluster (bound to host port 4194) in order to access resource usage and performance characteristics of running containers.
Clone the repository and checkout the latest release:
curl -L https://git.io/getPrometheusKubernetes | sh -
All the components versions can be configured using the interactive deployment script. Same for the SMTP account or the Slack token.
Some other settings that can be changed before deployment: * Prometheus replicas: default 2 ==>
manifests/prometheus/prometheus-k8s.yaml* persistent volume size: default 40Gi ==>
manifests/prometheus/prometheus-k8s.yaml* allocated memory for Prometheus pods: default 2Gi ==>
manifests/prometheus/prometheus-k8s.yaml* Alertmanager replicas: default 3 ==>
manifests/alertmanager/alertmanager.yaml* Alertmanager configuration: ==>
assets/alertmanager/alertmanager.yaml* custom Grafana dashboards: add yours in
assets/grafana/with names ending in
-dashboard.json* custom alert rules: ==>
assets/prometheus/rules/
Note: please commit your changes before deployment if you wish to keep them. The
deployscript will remove the changes on most of the files.
./deploy
Now you can access the dashboards locally using
kubectl port-forwardcommand, or expose the services using a ingress or a LoadBalancer. Please check the
./toolsdirectory to quickly configure a ingress or proxy the services to localhost.
To remove everything, just execute the
./teardownscript.
assets/prometheus/rules/and execute
scripts/generate-rules-configmap.sh. Then apply the changes using
kubectl apply -f manifests/prometheus/prometheus-k8s-rules.yaml -n monitoring
assets/grafana/and execute
scripts/generate-dashboards-configmap.sh. Then apply the changes using
kubectl apply -f manifests/grafana/grafana-dashboards.cm.yaml.
Note: all the Grafana dashboards should have names ending in
-dashboard.json.
The official documentation for Prometheus Operator custom configuration can be found here: custom-configuration.md If you wish, you can update the Prometheus configuration using the
./tools/custom-configuration/update_configscript.