By Gary Duan
Kubernetes is one of many key enabling technologies of digital transformation that has tended to remain obscure to non-technical company decision makers.
Related podcast: Securing software containers
Kubernetes is an administration console — an open source project from Google that makes containerized software applications easy to deploy, scale, and manage.
As beneficial as Kubernetes is for orchestrating containerized environments, a maturing set of security best practices must be adhered to for enterprises to ensure that their applications and data are as safe as possible from emerging vulnerabilities and exploits.
The most dangerous attacks on container environments will execute a “kill chain” of events – not striking all at once but instead through a sequence of lateral moves within the dynamic container environment to ultimately take over containers, attack Kubernetes services, or gain unauthorized access.
Attackers are shaping their attacks to take advantage of recently discovered vulnerabilities and systems which have not yet been patched or equipped to counter efforts to exploit them. In addition, the discovery of malicious ‘backdoors’ hidden in popular Docker images is another cause for concern.
Three recent examples illustrate this seemingly endless stream of vulnerabilities that attackers can leverage in a containerized environment: the Dirty Cow exploit, the Linux Stack Clash vulnerability, and the even more recently discovered CVE-2018-1002105 vulnerability in Kubernetes. Here’s how each inflicts damage:
Dirty cow
The name is as silly as the danger is serious: this dirty copy-on-write exploit (hence the Dirty Cow moniker) is capable of compromising the Linux kernel and allowing attackers to overwrite setuid programs and escalate their access rights. Attackers with root access privileges can then carry out whatever further malicious behavior they wish. This exploit is particularly dangerous to container environments, because the attack cannot be stopped by eliminating the container: instead, the attack corrupts the container host and breaks container immutability as well.
Linux stack clash
Duan
This exploit allows attackers to take advantage of the way memory for user space binaries is allocated on the stack. By clashing the stack with another memory region (the heap, etc.), attackers can pass over the stack guard gap, create controlled memory corruption, and use this to both give themselves root access privileges and compromise running containers. Paired with a complementary vulnerability, this is another exploit in which attacks are able to control and/or damage systems and then move laterally into others, with the potential to wreak havoc across container environments.
CVE-2018-1002105
This recently reported critical vulnerability in Kubernetes is particularly worrisome, as enterprises are in fact seeing their Kubernetes APIs exposed to the internet with zero authentication protections guarding them. The result is that attackers are hijacking Kubernetes in customer-facing production environments, and can run their own malicious commands on unsecured Kubernetes clusters. Under attackers’ control, enterprise Kubernetes deployments are being used to distribute ransomware, perform cryptomining, breach sensitive data, and other nefarious and damaging activities.
This vulnerability has been observed exposing Kubernetes clusters across industries and organizations of all kinds. Kubernetes users can determine if their deployment is exposed by applying this quick and simple test.
Not all attacks on the Kubernetes system require critical vulnerabilities to exploit. The recent Tesla crypto-mining attack used an unprotected Kubernetes console to gain access to the underlying servers.
Best Practices
To safeguard container and Kubernetes environments from evolving vulnerabilities and exploits, enterprises should be diligent in maintaining an awareness of known security issues, and apply critical patches the moment they become available. At the same time, proactively following these best practices can mitigate or eliminate the danger and harm done by attacks that leverage the above-mentioned exploits and others.
•Tighten access. Vulnerabilities that enable attackers to take over one container, and then escalate their privileges to infect or control other containers, can be curtailed by ensuring that built-in Kubernetes access restrictions are closely applied. Administrators should implement network segmentation, namespaces, and operating system controls, as well as advanced segmentation capabilities available from third-party network inspection solutions to make sure that a single compromised container doesn’t allow attacks to escalate to others.
•Limit services attack surface. Malicious techniques that give attackers unauthorized access by evading Kubernetes security controls – accomplished through the API server in the case of Kubernetes vulnerability CVE-2018-1002105 – often take advantage of externally exposed Kubernetes services. The kubelet node agent is another common target. As a best practice, expose only those services that must be exposed, and take care in configuring security policies for any visible orchestration service to be sure that authentication safeguards are fully active and effective.
•Extend container security. Continuous and automated monitoring is essential to protecting vast and dynamic Kubernetes environments in all phases of the application lifecycle. Scanning for existing vulnerabilities, monitoring for unauthorized privilege escalations, and implementing detection measures to recognize lateral attack traffic, suspicious connections, and suspicious containers are highly-effective methods of protecting Kubernetes in production. Environments should also undergo regular auditing and testing to ensure they meet industry security standards, including the Kubernetes CIS Benchmark.
By adhering to the latest best practices and remaining attentive to the newest threats, enterprises can better solidfy their Kubernetes deployments with robust protections against known vulnerabilities and exploit – and give themselves the best chance of success when targeted.
About the essayist: Gary Duan is the CTO at NeuVector, a container network security company that protects Kubernetes in production.
