GPU Activity and SM Occupancy: Why Two Percentages Can Disagree

First identify what the dashboard calls utilization

A dashboard label is insufficient to identify a counter. Ask for the underlying field, device scope and reporting interval before interpreting its percentage.

NVIDIA's NVML utilization structure defines its gpu field by the portion of a sample period during which at least one kernel executes. It documents product-dependent sample periods between 1 second and 1/6 second. A dashboard's refresh interval does not establish that underlying sampling period.

For this field, high activity says kernel execution occupies much of the sampled time. It does not establish that every streaming multiprocessor is full, or count floating-point operations. Other metrics labeled utilization need their own definitions.

References: [N1]

Occupancy measures the resident warp population

A streaming multiprocessor, or SM, executes thread groups called warps. In the occupancy context, active warps are resident execution contexts; active does not mean each warp issues an instruction on every cycle.

Occupancy compares active warps per SM with the maximum active warps supported by that SM. Its denominator is a hardware residency limit, rather than the elapsed time in a dashboard sample. The applicable maximum depends on the device; a number copied from another architecture can change the meaning of the percentage.

A reported average also needs its averaging basis. Retain which SMs, cycles and kernel execution the report covers. An average over a profiled kernel cannot silently become the occupancy of every workload throughout a device-monitoring window.

References: [N3][N4]

Theoretical occupancy describes what can fit

Theoretical occupancy is a calculated upper bound for a kernel configuration on the specified device. Compiled resource requirements and launch configuration matter. Registers and shared memory are finite resources shared among resident work, alongside hardware limits on blocks and warps. A block's resource requirements can therefore limit how many blocks, and consequently warps, can reside together.

Achieved occupancy describes the active warp population observed during execution on the report's averaging basis. Warps enter and finish at different times, so the calculated resident capacity need not stay filled. The two numbers answer whether work could fit and how much residency was observed.

This explains why a theoretical percentage is not an observed execution result. A gap invites a question about the identified kernel and its execution; the percentage alone does not establish the cause. Higher occupancy also does not guarantee higher performance.

References: [N2][N3][N4]

A resident warp still has to become ready and be selected

Nsight Compute's SchedulerStats description separates active warps from eligible warps, which are ready for their next instruction. A warp waiting on an instruction dependency can remain resident without being ready.

The scheduler selects an eligible warp to issue one or more instructions. With no eligible warp, it skips the issue slot. Residency, readiness and selection are therefore distinct observations. Keeping more warps resident can give the scheduler alternatives while others wait, but occupancy alone does not reveal whether those alternatives are ready.

Issuing instructions is closer to execution progress than merely holding resident warps. It still does not count accepted application output: instruction types and the work required for an outcome matter.

References: [N2][N3]

How 90% activity and 25% occupancy can coexist

Consider two wholly fictional reports. The first uses a 100 ms observation window and stipulates that at least one kernel executes during 90 ms of it. Its activity calculation is 90 ms / 100 ms = 90%. This window illustrates the definition; it is not a claim that NVML supports a 100 ms sample period.

The second is an independently stipulated kernel report. Its stated averaging basis gives 16 active warps per SM against a maximum of 64, yielding 16 / 64 = 25% achieved occupancy. The 64-warp maximum is invented for this toy device, not a specification for a named GPU.

The reports can coexist: one describes how often a device has kernel activity, while the other describes resident warps within a kernel execution. Nothing stipulates that the profiled kernel accounts for all activity in the first report, or that the reports share a measurement run.

Multiplying 0.90 by 0.25 gives 0.225 arithmetically. Calling that 22.5% useful compute is unsupported. The scopes and windows are different, and neither input supplies a count of useful operations or accepted output. Even aligning the reports would not turn resident warps into delivered FLOP/s.

References: [N1][N2][N4]

Compare two busy submissions without inventing a winner

Suppose two fictional submissions both report 90% activity using the same field and monitoring method. For an identified kernel on the same invented device, submission A reports 25% achieved occupancy and B reports 50%, with matching occupancy definitions. Their scheduler summaries use the same normalization; A reports fewer cycles without an eligible warp, and more cycles with issued instructions, than B.

This supports two narrow observations: B maintained a larger resident warp population in its kernel report, while A issued instructions on a greater share of the scheduler cycles covered by its report. The higher occupancy does not contradict the lower issue activity. These observations establish neither a cause for the difference nor a faster application.

Before accepting a capacity or tuning claim, ask what changed between the submissions, which kernel instances were covered, and whether the monitoring, profiling and completed-work results came from the same run or separate evaluations. Retain the exact device and software versions, workload settings, counter units and averaging definitions.

Then require the result that matches the purchase decision: the same accepted work completed in less elapsed time, or more accepted output under the same quality and response-time requirement. Use the related GPU benchmark guide for that broader comparison. If only these counters are available, the conclusion remains about activity, residency and issue behavior; the throughput claim remains unestablished. Unused residency capacity also does not establish compute capacity available for another customer's job.

References: [N2][N3][N4]

Carry the selected configuration into the physical project

The related activity-versus-facility guide covers cabinet occupancy, and the Roofline guide explains arithmetic intensity and resource ceilings. Neither denominator should be substituted for resident-warp occupancy in this comparison.

Once the platform team selects equipment, Leviathan's rack-and-stack service provides a route to discuss agreed equipment mounting, IT cabling, labeling and physical installation records. Use the related project inquiry with a nonconfidential equipment list, site, released rack and cable plans, target window and required physical handoff. Application profiling, software tuning and performance acceptance remain with the appointed platform team.

References: [L1]