We are using AWS hosting for our infrastructure. Nodes of Kubernetes cluster are distributed in 3 availability zones, and the lifecycle of nodes, as well as of master nodes, is managed by autoscaling groups, meaning that the platform supports self-healing/autoscaling cluster.
Access to API Kubernetes cluster is accomplished via AWS load-balancer, which distributes requests to 3 master nodes. Each master node is located in separate auto-stalling groups and availability zones. Access rights to Kubernetes cluster functionality are managed by mapping user IAM to Kubernetes RBAC; for this, we use add-on.
Diagram of the AWS infrastructure:
Our Kubernetes cluster installed by Kops can create a production-ready Kubernetes cluster and manage their lifecycle (updates, recovery, reconfiguration).
- Heapster/cAdvisor for metrics
- Reloader forpod reloading on Configmap/Secret updates
- NginX as ingress controller
- external-dns for DNS records which handle exposed services
- aws-iam-authenticator for user management through AWS IAM
Diagram of Kubernetes environment:
Deployment scheme is simple, and it is implemented through Gitlab-CI.
There are upstream repositories which contain the micro-service code. After the commit push/merge event, Gitlab-CI will test the code and build the docker container images with compiled binaries. After the docker is build, a job triggers a downstream repository for continious deplyoment which contains Helm charts for each service. Downstream Gitlab-CI will update Helm release in Kubernetes cluster with the new docker image version.
This repository contains configuration for every service which can be updated by directly pushing changes in this repository.
Diagram of CI/CD:
In the future, we are going to add monitoring and notification stack, such as Prometheus/Grafana/Alertmanager, as well as the application error tracing stack, such as Sentry. This will allow detecting error rates, performance drops, server faults, etc.
Improvements to CI/CD we plan to fulfill: install/tear down test environments for future/load tests or demos as well as spin environments for merge requests.