NewApp Monitoring is liveSign up free
AgentbeamBeta
AI agent vulnerability · vLLM

CVE-2026-53923: vLLM: GGUF dequantize kernel int truncation exposes uninitialized GPU memory in multi-tenant serving

mediumCVSS 4.0 5.3 · 3.1 7.5GHSA-5jv2-g5wq-cmr4

vLLM's GGUF dequantize CUDA kernels truncate a 64-bit element count to a 32-bit int. For a GGUF model with a weight tensor over 2^31 elements, part of the output buffer is never written and keeps stale GPU memory, which can hold data from other users' requests on a shared server. Affects vLLM 0.5.5 up to but not including 0.24.0. Upgrade to 0.24.0.

Published Jun 17, 2026 · Updated Jul 17, 2026 · Analysis Oct 26, 2026

What an attacker can do

In csrc/quantization/gguf/gguf_kernel.cu, ggml_dequantize allocates its output with torch::empty({m, n}), then passes m * n to a to_cuda_ggml_t function whose element-count parameter k is an int. When m * n exceeds INT_MAX, the kernel handles only the truncated count. A shape such as [65536, 65536] gives 4,294,967,296 elements. The rest of the buffer keeps whatever was in GPU memory before. ggml_mul_mat_vec_a8, ggml_mul_mat_a8 and ggml_moe_a8 have the same pattern with a truncated col. ggml_moe_a8_vec uses torch::zeros and is not affected.

The attacker crafts a GGUF file with oversized tensors and gets the victim to load it, for example from a model hub. The CVSS 4.0 vector marks user interaction as passive. There is no error or warning when this happens. The stale memory feeds into the model's computation, so data from earlier requests can influence output served to other users. The advisory's description of the leak is conditional: residual memory "may contain" other tenants' data. It does not claim a way to extract it directly.

Who is affected

The vllm pip package from 0.5.5 up to but not including 0.24.0, when serving GGUF-quantized models on CUDA. Deployments that only load models you build or fully trust carry little risk. Multi-tenant servers that load third-party GGUF files carry the most. Check:

pip show vllm

Then list which model files your servers load and where they come from.

vllm

Ecosystem
PyPI
Affected
>= 0.5.5, < 0.24.0
Fixed in
0.24.0
Check
pip show vllm

How to fix it

Upgrade to vLLM 0.24.0:

pip install --upgrade "vllm>=0.24.0"

The fix is pull request #44971, merged to main on June 11, 2026 as commit f219788. It widens k to int64_t in the to_cuda_ggml_t typedef and the 18 dequantize signatures, and widens col, batch, vecs and padded in gguf_kernel.cu. As defense in depth, it zero-fills the outputs of ggml_dequantize, ggml_mul_mat_vec_a8, ggml_mul_mat_a8 and ggml_moe_a8. The advisory lists no workaround. Load GGUF models only from sources you trust.

How to tell if you were hit

The advisory says no error or warning is raised, so there is no log line to find. Inspect GGUF files before loading them: read tensor shapes with the gguf Python package and flag any tensor whose element count exceeds 2,147,483,647. Review which GGUF models were loaded by shared vLLM instances, and where they came from.

Original advisory

From the GitHub Advisory Database (CC BY 4.0).

Summary

Integer truncation of tensor dimensions in vLLM's GGUF dequantize kernels (csrc/quantization/gguf/gguf_kernel.cu) causes partial tensor processing. The output tensor is allocated at full size via torch::empty (uninitialized memory), but the dequantize CUDA kernel processes only a truncated number of elements. The unfilled portion of the output tensor retains whatever was previously in GPU memory. In multi-tenant inference deployments, this residual GPU memory may contain tensor data from other users' inference requests, constituting information disclosure.

Root Cause

The to_cuda_ggml_t function pointer type at ggml-common.h:1067 declares its element count parameter as int (32-bit):

using to_cuda_ggml_t = void (*)(const void * __restrict__ x,
                                dst_t * __restrict__ y,
                                int k,              // 32-bit
                                cudaStream_t stream);

All dequantize kernel functions (dequantize_block_cuda, dequantize_row_q2_K_cuda, etc. in dequantize.cuh) inherit this int k parameter and use it as the kernel launch grid size:

static void dequantize_block_cuda(..., const int k, cudaStream_t stream) {
    const int num_blocks = (k + 2*CUDA_DEQUANTIZE_BLOCK_SIZE - 1) / (2*CUDA_DEQUANTIZE_BLOCK_SIZE);
    dequantize_block<<<num_blocks, CUDA_DEQUANTIZE_BLOCK_SIZE, 0, stream>>>(vx, y, k);
}

In ggml_dequantize() at gguf_kernel.cu:85, the caller passes m * n (an int64_t product) to this int k parameter:

at::Tensor DW = torch::empty({m, n}, options);    // line 80: full-size, UNINITIALIZED
// ...
to_cuda((void*)W.data_ptr(), (scalar_t*)DW.data_ptr(), m * n, stream);  // line 85: m*n truncated to int

When m * n > INT_MAX, the truncated k is smaller than the actual tensor size. The kernel processes k elements. The remaining (m * n) - k elements in DW are never written and contain stale GPU memory.

This is a single root cause -- the int type on the k parameter in to_cuda_ggml_t -- with a single fix: change int k to int64_t k. All dequantize functions inherit this type through the same typedef.

Affected Functions

All in csrc/quantization/gguf/gguf_kernel.cu:

| Function | Line | Allocation | Info Disclosure? |
| `ggml_dequantize` | 74 | `torch::empty({m, n})` at line 80 | Yes -- `m*n` truncated to `int k` at line 85 |
| `ggml_mul_mat_vec_a8` | 91 | `torch::empty({vecs, row})` at line 99 | Yes -- `int col = X.sizes()[1]` at line 94 |
| `ggml_mul_mat_a8` | 207 | `torch::empty({batch, row})` at line 215 | Yes -- `int col = X.sizes()[1]` at line 210 |
| `ggml_moe_a8` | 279 | `torch::empty({tokens*top_k, row})` at line 289 | Yes -- `int col = X.sizes()[1]` at line 285 |

All four functions allocate output tensors with torch::empty (uninitialized) and then run CUDA kernels that use truncated dimension values as loop bounds. The unfilled portion of each output tensor retains stale GPU memory.

ggml_moe_a8_vec (line 382) uses torch::zeros instead of torch::empty, so it is not affected by the info disclosure variant.

Impact: Information Disclosure in Multi-Tenant Serving

vLLM is designed for multi-tenant inference serving. GPU memory is reused across requests from different users. When the dequantize kernel partially fills an output tensor:

  1. The output tensor DW is allocated with torch::empty -- the buffer contains whatever was previously in that GPU memory region
  2. The dequantize kernel fills only a truncated portion of the buffer
  3. The unfilled portion retains residual data from prior GPU operations, which may include tensor data from other users' inference requests
  4. The contaminated tensor proceeds through the model computation
  5. No error or warning is generated -- the partial fill is silent

This is a confidentiality violation. In shared inference deployments (the primary vLLM use case), one user's inference data can leak into another user's model computation through residual GPU memory.

Attacker Control

The attacker crafts a GGUF model file with weight tensor dimensions whose product exceeds INT_MAX (e.g., a matrix with shape [65536, 65536] gives m * n = 4,294,967,296). The model is hosted on HuggingFace or any model hub. The victim loads the model with vLLM for inference serving. The truncation happens automatically during model weight dequantization.

Fix

A fix for this vulnerability was added here: https://github.com/vllm-project/vllm/pull/44971

Credited to Aviral2642, russellb, jperezdealgaba.

Questions

Does this affect non-GGUF models?

The advisory covers only the GGUF dequantize and matmul kernels in gguf_kernel.cu. It does not describe the problem for other quantization formats.

Can an attacker read other users' prompts directly?

The advisory says stale GPU memory may contain tensor data from other requests and enters the model computation. It does not describe a direct way to dump that memory.

Which version fixes CVE-2026-53923?

vLLM 0.24.0. Versions from 0.5.5 up to but not including 0.24.0 are affected.

References

How this page is made: the data panel comes from the GitHub Advisory Database, CISA’s KEV catalog, the NVD and FIRST EPSS, and updates twice a day. The analysis is written by an AI model from the sources listed above and checked by automated rules before it is published. Check versions against the vendor's advisory before you act. Methodology

More vLLM vulnerabilities

All vLLM entries