Memory Systems in Modern ASIC/SoCs

Arithmetic and logical operations in computing systems are executed by the Central Processing Unit (CPU). Modern Systems-on-Chip (SoCs) integrate CPU cores from architectures such as ARM, Intel, or MIPS. These cores process digital data stored in memory and generate results that are ultimately saved to another memory location. To understand the types of memory required in a computing system, consider this analogy:

The Analogy of Memory as a Scratchpad Solving a complex math problem typically requires intermediate steps on paper, even if only the final answer matters. Similarly, a processor needs volatile memory (temporary workspace) to execute operations and derive results. Once finalized, data is stored in non-volatile memory (e.g., SSDs, hard drives) for long-term retention.

Hierarchy of Volatile Memory 1. Registers (Flip-Flops):

• The fastest and smallest memory units, directly embedded within the CPU.
• Built using flip-flops (6-8 transistors per bit), they store operands, instructions, or intermediate results during execution.
• Operate at CPU clock speeds (nanosecond access times) but are limited in quantity (e.g., 16–64 general-purpose registers in modern CPUs).
• Managed explicitly by the compiler or assembly code to hold critical data for arithmetic/logic operations.

2. On-Chip Cache (SRAM):

• Static Random-Access Memory (SRAM) acts as a secondary layer, using 6T cells to store bits without refresh cycles.
• Divided into L1, L2, L3 caches, with L1 being the fastest (closest to registers) and L3 the largest but slowest.
• Balances speed and capacity: L1 cache may be 32–64 KB per core, while L3 can reach tens of MB in high-end CPUs.

3. Off-Chip DRAM (DDR Modules):

• Dynamic RAM (DRAM) provides high-density, cost-effective volatile memory but requires periodic refreshing.
• Accessed via a DDR controller and PHY interface, with latencies in the 10–100 nanosecond range.

Cache Architecture and Memory Access When the CPU requests data, it first checks registers, then the L1 cache, followed by L2/L3 caches. If the data is absent, the DDR controller accesses off-chip DRAM. The memory hierarchy ensures that frequently used data resides in faster, closer storage (registers → cache → DRAM).

System-Level Optimization Optimal memory performance in SoCs hinges on:

• Efficient use of registers to minimize cache/memory access.
• Cache sizing and placement (L1/L2/L3) to balance speed and capacity.
• Intelligent DDR controller algorithms for row management, request scheduling, and power efficiency.
• Balancing cost, speed, and density trade-offs between SRAM (on-chip) and DRAM (off-chip).

In summary, modern computing systems rely on a synergistic combination of registers, on-chip SRAM caches, sophisticated DDR controllers, and high-density DRAM modules to deliver the speed and capacity required for complex applications.