We create Kubernetes as a cloud service and closely monitor modern tools for container applications developers. Here we discuss some tools that immensely simplify Kubernetes deployments and help create CI/CD.
Kubernetes cluster deployment
Ansible set of roles to deploy and configure Kubernetes. Kubespray supports AWS, GCE, Azure, Mail.Ru Cloud Solutions, OpenStack, and bare metal IaaS. Kubespray is an open source and open development project with kubeadm under the hood.
Kubespray lets you create and orchestrate resources (instances, networks, balancers, etc.), using Ansible only, you won’t need other tools. If you already familiarized yourself with Ansible, this will be of great help.
Kubernetes distribution tool from
version 1.4 and up. Allows initializing Kubernetes clusters in optimal
configurations for the given infrastructure.
Kubeadm does not support the dynamical creation of an infrastructure in the cloud. Its main advantage is that it can run a minimally functional Kubernetes cluster in any environment. However, add-ins and network settings are not included with the Kubeadm, so you will have to configure everything manually or use other tools.
Kops is used to create, delete, update, and support industrial level resilient Kubernetes clusters using the command line. This tool officially supports Amazon Web Services (AWS). GTE support is in beta, and VMware vSphere – in alpha stage. Support for other platforms, including OpenStack, is also in the plans. Kops permits you to fully control the Kubernetes cluster lifecycle – starting from infrastructure setup and up to deleting the cluster.
Kubernetes clusters as a cloud service. Using this tool, you will have a running cluster within minutes with no need to configure it. You can also update it to the desired version. Clusters are easily scalable and support Mail.Ru infrastructure, which is optimized for high-load services.
Price depends on configuration. For example, a test environment with two nodes and one master will cost RUB 3200 a month. It can be tested for free.
Terminal console for a Kubernetes cluster to manage and monitor cluster status online using the good old interface. Kubebox displays usage of pod resources, monitors cluster, shows containers logs, etc. It also allows you to easily switch to target namespace and execute a command in the right container to troubleshoot it or restore normal operation quickly.
Provides UI for performance analysis. Aggregates and generalizes metrics from different sources, presents high-level analytical data to administrators. Kubedash uses Heapster as the data source. It runs as a service by default on all Kubernetes clusters and gathers data and insights on every container.
Open-source UI for Kubernetes that can be used instead of native kubectl console. This tool is useful not only for developers, but also for project managers, as it helps to monitor the projects running on Kubernetes clusters, in a user-friendly interface. This tool offers the possibility to manage running apps and integrates with CI/CD pipelines. Containerum UI will be especially useful for those who are just getting familiar with Kubernetes.
A small bash script to aggregate logs of multiple pods into a single stream. Initially, the Kubetail does not support filtering or selection, but there is a separate fork in Github, that can color logs using MultiTail.
A tool to find malfunctions and monitor clusters Docker Swarm and Kubernetes. Weave Scope automatically generates application topologies and architectures, thus helping to find bottlenecks in applications. You can deploy Weave Scope as a standalone application on a local server or a notebook, or use it as a SaaS in the Weave Scope cloud. Weave Scope allows you to easily group, filter, and search containers by names, tags, or resources consumption. It also has a surprisingly useful function: you can log in to a Kubernetes node as a root user from web-console without using ssh access.
Price: autonomous mode – free; standard SaaS version – 30/month per node (30 days trial); corporate version – 150/month per node.
An open source tool for monitoring and notifications, inspired by the Google Borg Monitor. Prometheus allows to create its metrics (it can integrate with all widespread programming languages) and also contains a large number of preset integrations (exporters) for various technologies: PostgreSQL, MySQL, AWS Cloudwatch, ETCD, and Kubernetes.
Prometheus has become a de-facto monitoring standard for Kubernetes. There is a special Prometheus Operator, enabling users to create Prometheus instances in Kubernetes clusters, including close integration with Grafana and Alertmanager.
Kubernetes operator for Icinga. Searchlight runs periodic checks in the Kubernetes clusters and if something is wrong, it will send you an e-mail, or an SMS, or write to a chat. Searchlight includes a default set of checks, especially for Kubernetes.
Searchlight offers new possibilities for Prometheus monitoring similar to an external black box monitoring service and serves as a backup system in case of full failure of internal systems.
Read-only system panel that can operate with many Kubernetes clusters. Using Kube-ops-view, you can easily navigate between clusters, monitor nodes, and pods condition. Kube-ops-view can animate some processes like creating or deleting pods. It also uses Heapster as the source of data.
Aquasec protects Kubernetes instances during its whole lifecycle. This solution deploys a specific agent in every container that acts as a firewall and closes holes in the container security. Agent interacts with Aquasec central management console that manages security restrictions. Aquasec also allows to configure a flexible security mechanism implementation pipeline in the cloud as well as local environments.
There is another open source tool related to Aquasec, Kube-Bench. It checks Kubernetes environments against a long list of tests from the CIS Kubernetes Benchmark document.
Price: $0.29 per scan.
Another tool that acts as a cloud-native application firewall and analyzes network traffic between containers and services. Twistlock analyzes the standard behavior of the containers and generates rules based on this behavior, so that administrators don’t have to create these rules manually. Twistlock also supports CIS Benchmark for Kubernetes, starting from version 2.2.
Price: starts from $1700 for a yearly license, trial period available.
A component of the Sysdig Container Intelligence Platform shipped as a standalone solution. It provides container visibility and is integrated with orchestration tools, including Kubernetes, Docker, AWS ECS, and Apache Mesos. Sysdig Secure allows user to implement service-aware policies, block attacks, analyze history, and monitor cluster performance. Sysdig Secure is available as a cloud and on-premise application.
Price: free for autonomous use. The price for the Pro version for cloud or local software is set depending on the configuration.
Service that assesses, to what extent the Kubernetes resources use the security features. Kubesec.io verifies the resource configurations against the best practices. User gets full control and recommendations to improve the overall system security. The project web-site contains many links to external sources on container security and Kubernetes.
A simple, but incredibly powerful alias generator for kubectl. By providing more than 800 short aliases for every possible need, it greatly speeds up everyday administration in Kubernetes.
A Kubernetes remote cluster management panel for mobile devices (Android and iOS). The cabin allows you to manage applications, scale deployments, and find malfunctions in the cluster. It helps Kubernetes cluster operators to respond to incidents from anywhere swiftly.
A small open-source utility that adds functionality to Kubectl, allowing to switch contexts and connect to several Kubernetes clusters simultaneously quickly. Kubens allows switching between Kubernetes namespaces. Both tools support autocomplete in bash/zsh/fish.
Optimizes work with kubectl. Provides commands autocomplete and offers options. It can even search and correct commands that were entered incorrectly. Kube-shell displays in-line help regarding the commands being executed.
Kail is short for the Kubernetes Tail. This tool works with Kubernetes clusters and helps to trace Docker logs for appropriate pods. Kail allows to filter pods depending on services, deployments, tags, and other parameters. Pod will be automatically added to the log (or deleted from it) after the launch if it matches the filter criteria.
The most popular open-source CI/CD server in the world. There is a free plugin for it that allows to deploy applications in Kubernetes, perform rolling updates (updates with minimum downtime), as well as Green or Blue application deployment. In this post, the detailed scenario for this configuration is presented.
Popular CI/CD service, created by the JetBrains team. You can use the Kubernetes cluster architecture with this plugin to run TeamCity build agents. The plugin supports TeamCity of version 2017.1.x and newer.
Price: free for up to three build agents and 100 build configurations. $299 per license to use additional build agent and ten additional build configurations.
Visualization and control
Universal web-interface for Kubernetes clusters. This native dashboard makes troubleshooting and monitoring of clusters much easier. To access this dashboard, a secure proxy channel must be created between your system and the Kubernetes API server. Native Kubernetes dashboard rely on the Heapster data collection tool, so it must be installed in the system. Even though Heapster is officially deprecated, there is no full-scale alternative for this tool yet.
This is a web-interface for the application catalog in the Kubernetes cluster. It allows to install, update, or remove Helm charts in one click with no need to use the command line.