Most GPU jobs use under 20% memory in 319-GPU cluster

Andrew Sheinberg, Samyak Gupta and Kai Li found in an arXiv preprint published Oct. 28 that more than 70% of 7,399 jobs in a 10-day period used less than 20% of memory.

The Oct. 28 preprint said 50% of jobs in the 10-day sample averaged below 5% Streaming Multiprocessor utilization, suggesting schedulers and multitasking could raise usable capacity without new hardware.

The paper, submitted Oct. 26 to arXiv and not yet peer reviewed, analyzed activity from 150 users across 12 departments. The data came from a university research cluster with 89 nodes and 319 GPUs, after the authors deployed a lightweight monitoring tool over several months.

The authors said their measurements point to capacity that current systems do not capture: "substantial utilization gains are available through better job scheduling and GPU multitasking, independent of hardware or application improvements."

The work targets a practical bottleneck in AI and scientific computing. GPU clusters are expensive shared infrastructure, and the paper says rising demand often leaves users waiting for access. Earlier monitoring systems, according to the authors, usually measured activity every second or minute and mainly supported job-level scheduling rather than tracking how individual GPUs were being used.

Sheinberg, Gupta and Li instead sampled resource use every 100 milliseconds, or a tenth of a second. Their tool uses Nvidia's DCGM library to capture fine-grained compute-utilization metrics inside GPUs. Streaming Multiprocessors, or SMs, are the GPU compute units that run parallel workloads; low average SM use can indicate that a job has reserved hardware it is not keeping busy.

The preprint highlights two separate mismatches. Memory use was low for most jobs, with more than 70% below 20% of GPU memory. Compute use was also low, with half the jobs averaging under 5% SM utilization.

The authors also said 80% of jobs showed a large gap between requested and actual durations, weakening the scheduler's ability to place work efficiently.

For multi-GPU jobs, the paper said the examined workloads showed highly periodic communication patterns. That matters because predictable communication windows may create opportunities to schedule other work around idle periods, though the preprint does not quantify a production gain from doing so.

The limits are material. The paper studies one university research cluster and focuses on one 10-day period, so the results may not transfer directly to commercial cloud fleets, enterprise AI clusters or systems with different user behavior. The authors also frame their conclusions as opportunities, not as proof that a specific scheduler would deliver a stated utilization improvement.

The monitoring software has been released as open-source code by the authors. The paper lists the work as 19 pages with 12 figures and four tables, and says it was originally written in fall 2024.

Source: arXiv cs.AI preprint, Oct. 28, 2026