An Idle GPU-Host CPU Has a Different Latency Question from a Slow CPU

Separate execution speed from the return to execution

A buyer comparing GPU hosts may receive a CPU frequency and a utilization average while needing a different answer: how does the application respond when work arrives after a quiet period? Consider a workload with a CPU stage before its next GPU operation. Finishing that stage slowly while executing and waiting to resume execution are different possible contributions to elapsed time. A slow response alone identifies neither.

Linux 6.18's CPUFreq documentation describes frequency/voltage operating configurations, called P-states in ACPI terminology. CPUIdle instead manages periods without ordinary runnable work, when hardware can suspend instruction execution. Performance control can also be shared across CPUs through a CPUFreq policy object.

Throughout this comparison, hold the model, loaded weights, backend preparation and readiness condition constant. The question is hardware idle behavior with an already prepared application. The related model-warmup guide covers preparation that may recur when application state changes.

References: [CPU-P][CPU-D]

Give residency, exit latency and P-state transition time separate meanings

Linux 6.18 defines target residency as the minimum time in an idle state, including entry, needed to save more energy than a shallower alternative. Exit latency is the maximum interval to resume the first instruction after wakeup. It can need to cover completion of entry before exit. Target residency therefore is not a timer that must expire before entry, and exit latency does not describe completion of the waiting application request.

CPUFreq's cpuinfo_transition_latency describes switching between P-states, in nanoseconds. That is a different transition. Its scaling_cur_freq field commonly reflects the last requested P-state frequency; even architecture-specific reporting need not give the exact instantaneous hardware clock. Ask which frequency field supports an offer's claim.

Reading any of these quantities as an exact end-to-end response duration would skip the remaining application work.

References: [CPU-D][CPU-I][CPU-P]

CPU-GAP20: the same 20% active share conceals different gaps

CPU-GAP20 is a fictional time-accounting example for one logical CPU over 200 ms. Stipulate that active intervals contain the relevant execution and idle intervals contain no other runnable task on that CPU. The interval labels supply no state-entry history. They are invented observations, not measurements of a server or a governor.

Trace A repeats 1 ms active followed by 4 ms idle forty times. Its active time is 40 x 1 ms = 40 ms; its idle time is 40 x 4 ms = 160 ms. Trace B repeats 10 ms active followed by 40 ms idle four times. Its totals are 4 x 10 ms = 40 ms active and 4 x 40 ms = 160 ms idle. Both span 200 ms and are 40 / 200 = 20% active.

For explanation, stipulate an abstract idle state D with a 20 ms target residency and a 0.2 ms exit latency. These numbers describe no actual CPU and recommend no policy. A's forty 4 ms gaps are shorter than the stated residency; B's four 40 ms gaps are longer. B offers longer uninterrupted gaps in this retrospective record, even though the active and idle totals match A's.

That comparison establishes the loss of information in the average. It establishes neither actual entry into D nor its energy consumption. The traces also have different execution bursts; equal active totals do not establish equal request counts, inputs or useful work. They cannot rank two hosts by application performance.

A completed gap is evidence the governor did not have in advance

CPUIdle selects a state before the full idle duration is known. Linux distinguishes a known next timer event from non-timer wakeups that can arrive earlier. Prior idle durations can inform an estimate; the completed trace does not reveal that estimate.

In CPU-GAP20, knowing afterward that B stayed idle for 40 ms does not prove D was selected. Knowing A's eventual gap was 4 ms likewise does not reconstruct what was anticipated at entry. The state decision and the observed hardware result remain missing.

Do not multiply forty gaps or four gaps by 0.2 ms and report an incurred delay. The example has supplied no D-entry count, no actual exit durations and no mapping from those wakeups to application requests. A worst-case parameter cannot fill those gaps.

References: [CPU-I]

Identify what the idle-state evidence actually observes

A logical CPU's idle request alone does not establish core or package residence. Other hardware threads and cores can keep shared units active. Linux also warns that kernel-accounted idle time may exceed actual residence; available hardware residency counters address the physical-state question.

For the offered system, have the platform owner distinguish advertised state support, effective state availability and wakeup-latency constraints, requests to enter states and observed residence. Ask what each supplied observation represents, which hardware unit it covers and its time window. If hardware residence evidence is unavailable, retain that uncertainty when assessing a claim about physical idle behavior.

CPU-GAP20's 160 ms idle total is consequently insufficient for an energy comparison. No power values or physical-state history were provided. For a claimed energy advantage, request matching consumption evidence with its measurement boundary and duration. CPU, complete-server and rack energy claims need their own stated coverage; a CPU activity average cannot supply the missing consumption.

References: [CPU-I][CPU-D]

Compare arrivals after stated gaps with sustained work

For sustained demand, request evidence of the useful CPU-side and application work under that sustained condition. For intermittent demand, retain the first useful response after each relevant class of quiet gap alongside subsequent responses. Define the start event, such as request arrival at a stated interface, and the useful finish, such as completed output. A hardware resume event and an application's useful finish are separate milestones.

Keep the post-gap observations identifiable. A combined statistic dominated by requests during long active bursts leaves the buyer unable to judge the arrivals that motivated this comparison. Ask for the number of observations, their gap conditions and the response distribution for that population, with the summary statistic defined.

The returned evidence should identify the server and CPU configuration, enabled topology, CPU allocation and concurrent work, GPU, firmware, operating system and kernel release, CPUIdle and CPUFreq drivers and policies, GPU driver, application/backend versions, inputs and offered load. It should explain the timing boundary, observation interval, clock basis and measurement method, including relevant measurement overhead. Compare equivalent conditions; changed conditions need an explicit explanation.

Connect response observations and state evidence over corresponding intervals. Their coexistence can support an explanation, but it does not isolate idle exit as the cause of every slow request. Queueing, execution and other dependencies still belong in the application owner's account.

Use the evidence for the demand pattern you are buying

If the requirement is sustained useful work, compare the sustained results against that requirement. An idle-state specification adds no proof that an executing CPU stage finishes sooner. If users arrive intermittently, an offer demonstrated only during sustained work leaves response after the stated gaps unanswered. Request the missing evidence before choosing on that basis.

A claimed energy benefit needs consumption evidence as well as acceptable response for the intended arrivals. CPU-GAP20 explains why matching average activity does not match the opportunities between bursts. It selects no universal governor, idle state or processor.

Use the related core-count article to interpret physical cores and SMT contexts. Use the CPU-request article for Kubernetes placement accounting and runtime CPU-time limits. Those facts can matter to the same host purchase, but they cannot replace the idle-state evidence examined here.

Take the selected system into physical installation planning

Once the platform and application owners have selected the complete equipment configuration, Leviathan's Rack & Stack / Colocation service can discuss agreed equipment mounting, IT cabling, labels and physical installation records. CPU policy selection, software administration and application performance assessment remain with the appointed platform team.

Use the related service and project inquiry below. Share a nonconfidential equipment list, rack positions, released connection requirements, site constraints and required handoff records to discuss physical installation scope and availability.

References: [L1]