Data centers are evolving to solve the challenges of storing, moving and analyzing data rapidly and efficiently. To a large degree, this evolution is being driven by the four high-performance application trends shown in the image below.
Traditional gaming and professional visualization are primarily in the domain of the PC and satisfied by innovations in fast GDDR memory. But with artificial intelligence (AI) training and inference and with high-performance computing, we see increased use of the fastest memory, high bandwidth memory (HBM), in data centers. Application architects in these segments must find the maximum bandwidth possible.
Let’s take a minute and think about why. Data, of course! And lots of it!
The growth of data is accelerating. According to IDC’s Global Datasphere 2021 report, the universe of data was 36 zettabytes of data in 2018, which grew to 64 zettabytes of data in 2020 and will continue to grow to 146 zettabytes in 2024.
Large amounts of data are integral to AI, deep learning and machine learning performance. AI is a game changer for delivering intelligence that can lead to new insights beyond human-driven discovery. Instead of programming compute infrastructure, we can now train it with volumes of data to reach further than ever before, including in areas like smart agriculture, personalized education and precision medicine.
At Micron, we’re delivering new technologies such as GDDR and HBM as part of our Ultra-Bandwidth Solutions to fuel analytics engines and help train the next wave of AI algorithms. For instance, an AI model with more than 1.3 billion parameters cannot fit into a single GPU (even one with 32GB of memory). So, expanding memory capacity enables larger model sizes/more parameters to reside closer to the core compute. By increasing bandwidth and capacity and reducing latency across memory and storage solutions, we are speeding time to insight and helping our customers uncover increased competitive advantage.
The multicomponent, accelerator-based heterogeneous approach
To address the growth of data-intensive workloads and applications, evolved use cases and new business opportunity, data center infrastructure is being redefined. Data centers have traditionally been CPU-centric. Data was processed using memory (such as DDR4) and storage (such as SSDs). As modern use cases and workloads like AI have emerged, this traditional architecture has proven to be a bottleneck to keeping the CPUs fed at the high-performance rates needed.
To address this growing performance gap, dedicated hardware is now being used to off-load some functions of the CPU. This new accelerator-based compute model is becoming central to the evolution of the heterogeneous data center. The process of reallocation in modern data centers deploys a variety of components that focus on specific functionality or on the high variability of data types and formats, thereby dramatically increasing the speed and performance of the entire system.
To those traditional CPUs using DDR4 (and soon DDR5) computing memory, we now add GPUs for acceleration, as well as FPGAs and ASICs for other functions. Modern data centers may use these various compute functions, and also using different types of memory becomes integral to achieving the high performance needed to address a wide range of workloads.
That’s why HBM was introduced – it is driving performance to new levels. Micron HBM2E, the latest generation of HBM and our fastest DRAM, is central to delivering better insights to our customers.
For more on Micron’s new HBM2E memory, watch the video, “Micron HBM2E: Performance Is Everything”
HBM2E: The star of the Ultra-Bandwidth Solutions portfolio
Micron is leading the market with evolved memory solutions to support the demanding and dynamic requirements of ever-growing market applications that are changing our world.
HBM2E uses through-silicon via (TSV) channels to create a vertically stacked DRAM. (For details, check out our tech brief, “Integrating and Operating HBM2E Memory.”) Micron has been researching and developing stacked DRAM for 20 years, generating thousands of patents in the process. (“Stacked Silicon Miracles” with our R&D leader Akshay Singh has more.) In the explorations of our stacked DRAM innovation to come, we intend to develop new products that satisfy the high-performance and low-power demands of more data-intensive workloads.
We vertically stack DRAM for HBM2E and connect the layers with through-silicon via (TSV) channels.
High-bandwidth memory was architected to address industry gaps in bandwidth, power and form factor. It is now the industry standard memory solution for AI and used extensively in the data center. HBM2E is the third standard of the HBM product family: HBM1, HBM2 and now HBM2E. HBM2E provides a very wide, multichannel I/O — that’s 1,024 bits wide — and has a very short physical channel. Essentially, it provides an extreme memory density in a small footprint.
HBM2E sits on an interposer very close to the GPU or CPU, generally encased within the same package or heat-spreading enclosure. With its wide I/O bus and increased density, HBM2E provides the high performance and power efficiency required for the accelerator-based compute model of the modern data center.
For a side-by-side comparison of the data center accelerator memory portfolio and the high-performance memories we offer, check out Table 1 in our white paper, “The Demand for Ultra-Bandwidth Solutions.” We are excited about the enhanced I/O performance, bandwidth and power efficiency in HBM2E that make it a cornerstone of our Ultra-Bandwidth Solutions portfolio.
Summary
As the demands of high-performance applications drive the evolution of next-generation system architectures and the heterogeneous data center, HBM2E memory and our Ultra-Bandwidth Solutions are delivering the critical memory and advanced system performance to transform data into insights. Find us online at micron.com/hbm2e.
