--- title: "More GPU FLOPS or More Memory Bandwidth? Read the Workload’s Arithmetic Intensity" description: "Leviathan Systems — Technical content Published 2026-10-20; updated 2026-10-20." canonical: "https://www.leviathansystems.co/articles/gpu-memory-bandwidth-arithmetic-intensity-roofline" last-updated: "2026-10-20" --- # More GPU FLOPS or More Memory Bandwidth? Read the Workload’s Arithmetic Intensity Leviathan Systems — Technical content Published 2026-10-20; updated 2026-10-20. More peak GPU FLOP/s helps only when the operation can use that arithmetic capacity. Arithmetic intensity measures floating-point work per byte transferred at a named memory boundary. In a simplified roofline model, the performance ceiling is the smaller of compute throughput and memory bandwidth multiplied by arithmetic intensity. That relationship explains a possible bottleneck; it does not establish an application's measured speed or choose a GPU. ## Three specifications answer three different questions Capacity asks whether the required data and working storage fit in the available memory. Bandwidth asks how quickly bytes can move across a particular interface. Arithmetic throughput asks how quickly a processor can perform a specified kind of calculation. Adding installed GB does not, by itself, increase the work done per byte moved. Use FLOP for a count of floating-point operations and FLOP/s for a rate. A rate also needs its arithmetic basis: input and accumulation precision, operation type, dense or sparse accounting, and whether the specification covers one GPU or a larger system. A matrix-arithmetic peak is useful only when the operation can use that execution capability. This article connects these quantities for one operation. Use the linked memory-capacity comparison to assess fit and the benchmark guide to compare completed application work. Neither question is replaced by the calculation here. References: [N1](https://docs.nvidia.com/deeplearning/performance/dl-performance-gpu-background/index.html), [N3](https://docs.nersc.gov/tools/performance/roofline/), [N4](https://docs.nvidia.com/nsight-compute/ProfilingGuide/index.html#roofline-charts) ## Name the memory boundary before dividing work by bytes Let F be the floating-point operation count and Q the bytes transferred across the chosen boundary, including reads and writes. Arithmetic intensity I = F / Q, in FLOP/byte. For a device-memory analysis, Q describes traffic to and from the GPU's external device memory, such as HBM. It is not simply the dataset's stored size: repeated transfers count again. Cache traffic needs its own denominator and matching bandwidth. Data reused on chip need not cross the device-memory boundary again for every arithmetic use. The original Roofline paper calls its DRAM-based quantity operational intensity and explicitly counts traffic after cache filtering. NERSC also describes separate intensities for different memory levels. Host-to-device links, GPU-to-GPU communication and the external network are other boundaries. A network rate or NCCL busbw value cannot be inserted as device-memory bandwidth in this calculation. References: [R1](https://www2.eecs.berkeley.edu/Pubs/TechRpts/2008/Archive/EECS-2008-134.pdf), [N3](https://docs.nersc.gov/tools/performance/roofline/) ## Read the roofline as two competing ceilings Let P be the applicable compute ceiling in FLOP/s and B the bandwidth at the chosen memory boundary in bytes/s. The simple roofline ceiling is min(P, B × I). Its balance point is I_balance = P / B. Below that point, the memory term is smaller; above it, the compute term is smaller. At the intersection, the two terms are equal. The units explain the multiplication: bytes/s × FLOP/byte gives FLOP/s. On a roofline plot, the memory term rises with intensity until it meets the horizontal compute ceiling. A larger compute peak raises the horizontal ceiling, but leaves B × I unchanged when B and I stay fixed. This is an idealized bound assuming the stated resource rates and enough overlapping work to approach them. Its shape identifies what could constrain an operation under those assumptions. References: [R1](https://www2.eecs.berkeley.edu/Pubs/TechRpts/2008/Archive/EECS-2008-134.pdf), [N4](https://docs.nvidia.com/nsight-compute/ProfilingGuide/index.html#roofline-charts) ## A fictional processor: 100 TFLOP/s and 2 TB/s Consider a wholly fictional processor with a modeled peak of 100 TFLOP/s for dense FP32 multiply-add work and modeled device-memory bandwidth of 2 TB/s. Count each fused multiply-add as two floating-point operations. Assume both operations below can use this same arithmetic mode. These are invented inputs, not a named GPU's specifications or observed results. Use decimal prefixes throughout: 1 TFLOP/s = 10^12 FLOP/s and 1 TB/s = 10^12 bytes/s. The balance point is therefore (100 × 10^12 FLOP/s) / (2 × 10^12 bytes/s) = 50 FLOP/byte. Operation A has stipulated intensity of 20 FLOP/byte at the device-memory boundary. Its memory term is 2 × 10^12 bytes/s × 20 FLOP/byte = 40 × 10^12 FLOP/s, or 40 TFLOP/s. The smaller of 100 and 40 is 40 TFLOP/s: the model gives a memory-limited ceiling. Operation B has stipulated intensity of 100 FLOP/byte at the same boundary. Its memory term is 200 TFLOP/s, so the compute ceiling of 100 TFLOP/s is smaller. The two operations encounter different limits on the same fictional processor. The numbers describe ceilings for their respective work; they do not show that B finishes sooner, because their total work and actual execution times have not been specified. References: [R1](https://www2.eecs.berkeley.edu/Pubs/TechRpts/2008/Archive/EECS-2008-134.pdf) ## What would more compute or bandwidth change? Keep fictional Operation A at 20 FLOP/byte. Raising P from 100 to 200 TFLOP/s while holding B at 2 TB/s leaves its ceiling at 40 TFLOP/s. The extra arithmetic capacity does not change the smaller memory term. Instead, hold P at 100 TFLOP/s and raise modeled bandwidth to 4 TB/s. The memory term becomes 80 TFLOP/s, below the unchanged 100 TFLOP/s compute ceiling. For Operation B at 100 FLOP/byte, that same bandwidth increase leaves the ceiling at 100 TFLOP/s because compute was already the smaller term. These controlled changes isolate the mechanism. Real products change several properties together, and an implementation may behave differently on each. The example supports asking whether additional bandwidth or compute addresses the suspected limit. References: [R1](https://www2.eecs.berkeley.edu/Pubs/TechRpts/2008/Archive/EECS-2008-134.pdf) ## Reuse and operation shape can change the ratio An original traffic example makes the distinction clear. Suppose a code region performs one million FLOPs while transferring 100,000 bytes at the selected boundary. Its intensity is 10 FLOP/byte. If an alternative implementation performs the same counted work but transfers only 50,000 bytes there, intensity becomes 20 FLOP/byte. The arithmetic change comes from fewer transfers, without increasing installed memory capacity. Matrix multiplication helps explain why shape matters. For a plain product of an M-by-K matrix and a K-by-N matrix, the conventional arithmetic count is 2 × M × N × K FLOPs. Under an idealized one-read-per-input, one-write-per-output assumption, with s bytes per element and no read of an old output, traffic is s × (M × K + K × N + M × N) bytes. Changing the dimensions can change the ratio of work to traffic. That expression is a model of a plain product, not a count of every implementation's device-memory transactions. NVIDIA's matrix guide explains tiled reuse and the tradeoff between reuse and parallelism. References: [N2](https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html) ## Why measured performance can sit below the ceiling Insufficient parallelism can prevent an operation from filling the processor even when its arithmetic intensity is high. NVIDIA's performance guide identifies latency as a separate limitation and warns that actual implementations can reread data and execute instructions absent from the algorithmic FLOP count. A ratio computed from ideal traffic can therefore misdescribe the implemented operation. A cache-based explanation also changes which bytes and bandwidth belong together. NERSC distinguishes vendor specifications from capability measured in the actual execution environment. Keep a specification-based illustration, an empirically characterized roofline and an observed application result labeled as different evidence. For a whole application, ask how much elapsed time the analyzed operation accounts for. Even a convincing device-memory explanation for one region does not explain time spent elsewhere. Ask whether host transfers, communication or other stages dominate the actual run before treating the region's bound as an application forecast. References: [N1](https://docs.nvidia.com/deeplearning/performance/dl-performance-gpu-background/index.html), [N3](https://docs.nersc.gov/tools/performance/roofline/) ## Use the mechanism to ask for better comparison evidence Ask the platform owner which operation and shape the proposed comparison represents, which arithmetic mode supplies P, and which memory boundary supplies B and Q. Ask whether F and Q are theoretical counts or measurements from an identified implementation. Retain those answers alongside the workload result rather than assigning the entire application a permanent memory-bound label. Then use the linked GPU benchmark guide for the actual system comparison. The useful question is whether representative measured work meets the required quality, throughput and response time on the offered configuration. A roofline explanation can make that request more specific; it cannot supply the missing result. Once equipment is selected, the physical installation becomes a separate project package. Leviathan's rack-and-stack service covers agreed equipment mounting, IT cabling, labeling and physical installation records. Use the related project inquiry to share the exact equipment, site, released plans and required physical handoff. Workload tuning and application performance acceptance belong with the appointed platform team. References: [L1](https://www.leviathansystems.co/services/rack-and-stack) ## Primary sources - [N1 — NVIDIA — GPU Performance Background User's Guide](https://docs.nvidia.com/deeplearning/performance/dl-performance-gpu-background/index.html). Checked 2026-10-20. DU-09798-001_v001, February 2023; HTML updated February 1, 2023. Explains arithmetic intensity, overlapping memory and math work, latency and implementation limits. Historical product examples are not current GPU specifications. - [N2 — NVIDIA — Matrix Multiplication Background User's Guide](https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html). Checked 2026-10-20. DU-09799-001_v001, February 2023; HTML updated February 1, 2023. Supports matrix operation counts, simplified traffic reasoning and reuse/parallelism tradeoffs. Library recipes and historical benchmark numbers are outside this comparison. - [N3 — NERSC — Roofline Performance Model](https://docs.nersc.gov/tools/performance/roofline/). Checked 2026-10-20. Definitions of FLOP counts, throughput and traffic by memory level; distinguishes specifications from empirical machine characterization. Only the conceptual discussion is used, without adopting its dated platform or profiling examples. - [N4 — NVIDIA — Nsight Compute Profiling Guide: Roofline Charts](https://docs.nvidia.com/nsight-compute/ProfilingGuide/index.html#roofline-charts). Checked 2026-10-20. Sections 2.9.1–2.9.2 distinguish memory and compute boundaries, their ridge point and a profiled kernel's achieved value. Used to interpret the chart, without prescribing tool settings or inferring measured performance. - [R1 — Williams, Waterman and Patterson — Roofline, UCB/EECS-2008-134](https://www2.eecs.berkeley.edu/Pubs/TechRpts/2008/Archive/EECS-2008-134.pdf). Checked 2026-10-20. October 17, 2008 technical report, Section 3. Establishes the minimum-of-compute-and-memory bound and its DRAM traffic boundary. The article's processor, traffic and hardware-change examples are original fictional calculations. - [L1 — Leviathan Systems — Rack-and-Stack and Colocation Installation](https://www.leviathansystems.co/services/rack-and-stack). Checked 2026-10-20. Published physical installation scope and project inputs. Supports the installation handoff described here, without establishing workload tuning or application-performance acceptance services. ## Related Leviathan resources - [Compare actual GPU workload benchmark evidence](/articles/which-gpu-is-fastest-workload-benchmarks) - [Compare GPU memory capacity by exact variant](/articles/data-center-gpu-memory-capacity-comparison) - [Interpret NCCL communication bandwidth separately](/articles/nccl-bandwidth-validation) - [Review rack-and-stack installation scope](/services/rack-and-stack) - [Discuss physical installation of the selected equipment](/contact?service=rack-and-stack&from=%2Farticles%2Fgpu-memory-bandwidth-arithmetic-intensity-roofline) ## Plan the physical installation of your selected system Share a nonconfidential equipment list, site, released rack and cabling plans, target window and physical handoff requirements so the installation scope and availability can be discussed. [Discuss the scope](/contact?service=rack-and-stack&from=/articles/gpu-memory-bandwidth-arithmetic-intensity-roofline)