Shared GPU Memory Can Serialize Distinct Words in the Same Bank

On-chip placement leaves the bank mapping unresolved

A statement that a kernel uses shared memory identifies a memory space, not the rate at which its next request completes. CUDA's block shared memory lets threads in a block access common storage. Here the question is how one warp's reads use that storage.

NVIDIA's archived CUDA C++ Best Practices Guide 13.1 describes shared memory as on chip and divided into banks that can operate in parallel. Distinct words assigned to one bank cannot all receive that parallel service in the same request. The hardware separates a conflicting request into conflict-free requests.

The guide gives a 32-bank mapping for devices of compute capability 5.x or newer, with consecutive 32-bit words assigned to consecutive banks. The example below applies that stated mapping to one ordinary read of one aligned 32-bit word per participating thread. It does not extend the calculation to every wider, vector, atomic or specialized instruction, or to every GPU architecture.

References: [CM1][CM2]

CM-306: keep the request size and allocation fixed

Consider an original address-accounting example, CM-306. One warp has 32 participating threads, numbered t = 0 through 31. Each thread reads one 32-bit word, equal to four bytes, in the same shared-memory load instruction. All values are already available, with no concurrent writes.

Use the same array of 64 four-byte words in all three cases. Its stipulated allocation is 64 x 4 = 256 bytes. Number its words from 0 through 63, choosing the bank labels so word 0 belongs to bank 0. A word's bank is the remainder after dividing its word index by 32.

Thus words 0 and 32 belong to bank 0, words 1 and 33 to bank 1, and words 31 and 63 to bank 31. A bank contains many possible words; sharing its bank number does not make two words the same address.

Every case produces 32 per-thread results of four bytes each. That aggregate result size is 128 bytes. Count distinct addressed words separately, because the broadcast case repeats a single word across those results.

References: [CM1][CM2]

Word index t gives each bank one word

First let thread t read word t. Threads 0 through 31 therefore read words 0 through 31, mapping to banks 0 through 31 respectively. Every bank receives one distinct requested word. The request has no bank conflict under the stipulated model.

There are 32 distinct words and 32 x 4 = 128 bytes of distinct requested storage. Their byte offsets begin at 0, 4, 8 and continue through 124; the last word ends at offset 127.

The unused second half of the 256-byte allocation has no bearing on this request's bank conflict count. Allocation capacity and the addresses accessed together answer separate questions.

References: [CM1][CM2]

Word index 2*t puts two distinct words in each used bank

Now let thread t read word 2*t. The word indices are 0, 2, 4 and so on through 62. All fit in the same allocation. They are still 32 distinct words, with the same 128 bytes of distinct requested storage.

The bank sequence is 0, 2, 4 and so on through 30, then 0, 2, 4 through 30 again. Only 16 banks participate. Thread 0 reads word 0 and thread 16 reads word 32; both use bank 0, but they need different words. Likewise, threads 15 and 31 need words 30 and 62 from bank 30.

One conflict-free population contains the words requested by threads 0 through 15. Another contains those requested by threads 16 through 31. Each population asks each participating bank for only one word. Two populations are necessary because each used bank must supply two distinct words; two are sufficient for this mapping. This is the modeled two-way bank conflict.

That partition demonstrates the service requirement, without prescribing the hardware's ordering. All 32 requested words remain useful. The problem is their concentration in 16 banks, even though the allocation and requested byte count match the consecutive case.

References: [CM1][CM2]

Word index 0 for every thread is a broadcast

For the third case, every thread reads word 0. All 32 reads now identify the identical four-byte location in bank 0. NVIDIA's Programming Guide explicitly distinguishes this read broadcast from a conflict between different words in one bank.

There are still 32 thread results, amounting to 128 bytes when counted per recipient, but only one distinct addressed word containing four bytes. The request does not require 32 separate distinct-word services from bank 0.

Compare this with the two-word-stride case: that case needs word 0 and word 32 from bank 0, so one broadcast of word 0 cannot satisfy both demands. The deciding property is address identity within the bank, not merely how many threads name the bank. This read exception supplies no rule for concurrent writes.

References: [CM2]

Global-memory coalescing counts a different service unit

The related global-memory coalescing guide asks how a warp's global addresses fit into 32-byte transaction units under NVIDIA's compute-capability 6.0-and-higher rule. CM-306 instead assigns shared-memory words to banks. The number 32 denotes different things: a four-byte word contains 32 bits, this model has 32 banks, and the global coalescing rule uses 32-byte units.

A 32-bank array is not a set of 32-byte global-memory segments. The two conflict-free populations in CM-306 are not two HBM transactions. If values first arrived through global-memory loads, the coalescing and cache behavior of those earlier loads would require their own accounting.

Nsight Compute's Profiling Guide keeps shared-memory service, cache activity and device-memory transfers separate. Its Shared Memory table describes serialized wavefronts and bank conflicts; its Device Memory table defines bytes transferred between L2 and device memory. A global load served from cache need not fetch the same data from HBM on that access. A shared bank conflict is not a cache-miss label.

Moving a working set on chip can avoid some repeated global accesses while leaving conflicting shared reads. Conversely, establishing conflict-free shared reads leaves the earlier global accesses unresolved. Each claim needs evidence at its own memory boundary.

References: [CM1][N1]

Two service populations do not establish twice the kernel time

CM-306 establishes a difference in bank service with equal distinct requested bytes in its first two cases. It assigns no elapsed duration to either request. Dividing 128 bytes by an unspecified time cannot produce an access rate.

The two-way conflict explains why the second request cannot use the same parallel bank service as the first. It does not establish a twofold change in full-kernel time. A kernel also performs other instructions and accesses, and an application's elapsed time covers more than this one request. The example supplies no share of total execution time attributable to these reads.

Suppose a supplier compares two implementations using the same shared allocation and the same logical read volume. Those facts permit both of CM-306's first two mappings, so they cannot establish equal access efficiency. A returned address mapping could establish the modeled conflict difference. It would still leave the application's performance comparison open.

For that comparison, retain the actual GPU and compute capability, software and kernel versions, participating threads and access widths behind the claimed pattern. Reported counters need their names, access types, memory boundaries and observation scope. Then compare the same accepted workload under matching input and output requirements, with elapsed time measured over the claimed kernel or application boundary. The related workload-benchmark guide develops that broader decision.

References: [CM1][N1]

Connect selected equipment to its physical installation

If the platform decision leads to a physical installation project, Leviathan's rack-and-stack service covers agreed equipment mounting, IT cabling, labels and installation records. Use the service and project-inquiry links below with the equipment list, site, released rack and cable plans, required window and physical handoff.

Kernel evaluation and software-performance acceptance remain with the appointed platform team.

References: [L1]