Pascom RCE, AutoWarp, and a GKE Container Escape
A nice chain leading to unauthenticated RCE, a path traversal leading to server-side request forgery, used to hit the application’s API from localhost and leak administrative credentials, then an unescaped argument in a privileged-off task for command injection.
Path Traversal - A fairly well-known attack against Tomcat servers where
..;/ will be normalized as
../ but reserve-proxies like nginx will usually not see it as a traversal and pass it along. This can be used to hit endpoints that Tomcat is serving but that are not reachable through the reverse proxy.
Server-Side Request Forgery - This was a previously known and reported issue in a third-party component. Normally the
/getFavicon endpoint wouldn’t be reachable but the path traversal could be used to make requests to it. It does as the name indicates, takes a
host parameter and attempts to fetch its
host is entirely attacker controlled and without sensitization, it can be used to craft and url.
Sensitive Information in Configuration - Requests to the REST API for the application from localhost do not require authentication, so the prior SSRF could be used to make a request to the
/services/sysinfo/activeconfig endpoint which contains a password for a privileged API user.
Command Injection - Finally with a privileged API user, the scheduled task functionality could be reached. One of the tasks,
task050380 takes two parameters a tar file and a target directory, and extracts the tar file. There is an optional parameter
$deleteOutDir which will first delete files inside the output directory.
$this->execute('rm -rf ' . realpath($targetdir) . '/*');
In this case as the path escaped in this
rm call additional commands can be injected using shell expansions like
$(id) though it does need to be a directory name that can be created.
Two issues, first an XSS requiring two injection points to bypass the web-application firewall and a cache poisoning attack making it possible for the XSS to be stored.
Cross-Site Scripting - The first XSS discovered was the the
gdId cookie would be reflected in the page, however because the WAF would block the request when using a space or including an HTML tags it couldn’t be weaponized on its own. The page also reflected the user’s IP nearby, which can be influenced through
X-Forwarded-For headers, Combining these two leading to XSS.
Cache Poisoning - The application would cache pages if they had a
.css file extension which could be appended to a request without changing the endpoint it resolved to. Allowing the XSS requiring custom headers to be exploitable.
Azure Automation would run an internal service serving JWTs that could be accessed across tenant boundaries.
Each time a tenant automation script would start an orchestrator service would also be started running on a random high port. This service could be used to request a JWT for the tenant. The problem was that as there was no additional authentication layer, any tenant could make requests to these services and obtain the JWT for other tenant.
Escaping to the Node Virtual Machine The first step in the chain was going from container automation scripts run within and accessing the node virtual machine. What they found was there was an allow list of workloads (based on image name and command) that could mount certain host paths.
Specifically a container with an image named
gcr.io.datadoghq/agent* and the command
bash -c could mount (read-only),
/var/run/containerd. This check could by bypassed by providing custom arguments to the container under
args rather than as part of
command. So you control the code
bash will execute inside the datadog image.
As the containerd socket can be mounted inside this image, an attacker can abuse that access to create a new privileged container with full access to the host filesystem. Then, install a systemd service to spawn a reverse shell with full access to the virtual machine including other tenants and their service account tokens and instance metadata. The attacker also gains access to the Kubelet’s credentials.
Full Cluster Takeover
The authors used their tool
sa-hunter to find powerful pods to compromise. Found the
kube-system namespace which is highly privileged (no restrictions) and two pods that are installed by default. So the first step would be to compromise a node virtual machine that is running one of the privileged pods, steal their service account token from the node.
With the service account token one can update an existing deployment’s service account, add a malicious container to the deployment, and read the
Bonus: Invisible Backdoor
A little added bonus to end off the attack, they also document a way of backdooring the system by installing a MutatingAdmissionWebhook which will be called whenever an objects are created or updated in the kubernetes cluster. Giving access to all the pods and secrets and can mutate them. Autopilot admins cannot even list these webhooks and so cannot discover them.