Earlier this week, a severe vulnerability in Kubernetes (CVE-2018-1002105) was disclosed that allows an unauthenticated user to perform privilege escalation and gain full admin privileges on a cluster. The CVE was given the high severity score of 9.8 (out of 10) and it affects all Kubernetes versions from 1.0 onwards, but fixes are available for recent versions.
The first to discover this vulnerability was Darren Shepherd from Rancher, and it was quickly fixed by the community with updates to the major upstream K8s releases v1.10, v1.11, and v1.12 (read on for details). We have also updated Aqua’s open source kube-hunter to hunt for this vulnerability – more details below.
Unauthenticated Remote Privilege Escalation
The flaw that was discovered was in the Kubernetes API Server, which handles all the administration tasks for the cluster. The API server provides various functionality by acting as a reverse proxy to the kubelet running on the compute nodes. The API Server also acts as a reverse proxy when implementing the API Aggregation feature of Kubernetes. API aggregation enables the installation of additional Application Programming Interfaces (APIs) into the core API Server. Those additional APIs are referred to as API extensions by upstream Kubernetes.
This CVE actually enables two different attack scenarios, one that allows normal authenticated users with Pod exec/attach/forward privileges to escalate privileges and become admins. and another that allows a remote unauthenticated user to gain access and escalate privileges via the aggregated API server.
A pod exec/attach/portforward API call can be escalated to perform any API request against the kubelet API on the node specified in the pod spec (e.g. listing all pods on the node, running arbitrary commands inside those pods, and obtaining the command output). Pod exec/attach/portforward permissions are included in the admin/edit/view RBAC roles intended for namespace-constrained users.
An API call to any aggregated API server endpoint can be escalated to perform any API request against that aggregated API server, as long as that aggregated API server is directly accessible from the Kubernetes API server’s network. In default configurations, all users (authenticated and unauthenticated) are allowed to perform discovery API calls that allow this escalation.
What Could an Attacker Do?
There’s no wonder that this CVE scored so highly on the severity scale; a remote user who gains admin privileges can gain ownership of a cluster, which may allow him to perform malicious actions such as:
- Inject malicious code into containers to exfiltrate data, mine for cryptocurrencies, access secrets
- Bring down the entire cluster
- Change admission controller settings such as PodSecurityPolicy to expand their abilities even further
So, anything really.
Mitigation: Immediate K8s cluster update
The first and most immediate action you should take is to upgrade your K8s clusters to the fixed versions noted below. Running with a such a severe vulnerability is very risky.
The affected versions and their fixes are:
- Kubernetes v1.0.x-1.9.x – no fix available
- Kubernetes v1.10.0-1.10.10 – fixed in v1.10.11
- Kubernetes v1.11.0-1.11.4 – fixed in v1.11.5
- Kubernetes v1.12.0-1.12.2 – fixed in v1.12.3
If you’re running k8s in one of the managed cloud versions (Azure AKS and Google GKE for example), your cloud provider will do this for you. Also, Red Hat has been pushing updates to OpenShift customers.
Other configuration-based mitigations, such as disabling access to the API, are highly disruptive and impractical.
How can Aqua help with this vulnerability?
We’ve updated kube-hunter, our open-source tool, which is used to discover all Kubernetes nodes in the network and to run automated penetration testing based on known vulnerabilities and exploitation techniques, Equipped with the new “hunter” that discovers the Kubernetes Privilege Escalation vulnerability (CVE-2018-1002105).
Make sure you run kube-hunter on your network to discover whether your infrastructure is exposed to this vulnerability.
Aqua customers can also easily identify if their Kubernetes or Openshift environments are compromised by scanning one host on their cluster. This is done in the Aqua Console:
- Navigate to the Enforcers section
- Select the relevant Aqua Enforcer
- In the selected enforcer press Scan Now (under the Host Scan)
The CVE is listed because the host is running a vulnerable version of Kubernetes:
What else can be done to secure Kubernetes
It is highly recommended to improve the security posture of your Kubernetes cluster in a way that makes applications less susceptible to future vulnerabilities (i.e. zero days), as well as speed up the detection and response to any suspected breach.
At Aqua, we recommend our users to deploy Aqua’s enforcers on each Kubernetes node and enable the “drift-prevention” security control which prevent attackers to execute arbitrary code on their nodes. Adding similar security layer to Kubernetes environment can reduce the potential risks of unknown vulnerabilities (also known as 0 day attacks). And of course continuously use kube-bench and kube-hunter to find flaws in your k8s cluster configuration.
Unlike previous reported attacks on Kubernetes environments (Tesla, Kubelet backdoor) that mostly resulted in cryptocurrency mining, this new vulnerability can lead to complete cluster takeover, a threat which is in a different order of magnitude. It’s a wake-up call to all Kubernetes users to increase the security levels of their environments by adding additional prevention layers.
Kubernetes is a complex piece of software that has grown in use (and lines of code) very rapidly, so the discovery of a severe vulnerability in Kubernetes should not come as a surprise, but and the community should be applauded for quickly fixing it and communicating it.
