Intel, IBM, OSATs tip the latest in packaging

AI, datacenters and edge computing are among hot markets for advanced packages.

By Mark LaPedus

Several entities are expected to present papers on the latest breakthroughs in IC-packaging and related topics at the upcoming IEEE Electronic Components and Technology Conference (ECTC).

ECTC will take place in Denver from May 28-31. At the event, AMD, Amkor, ASE, Broadcom, IBM, Imec, Intel, Leti, Samsung, Sony, TSMC and many others will present papers. Some of the papers represent technologies that are still in R&D. Others are in production.

The range of topics to be covered at ECTC include 2.5D/3D packaging, chiplets, heterogeneous integration, hybrid bonding, interconnects, photonics and wafer-level packaging. More than 1,500 scientists, engineers and businesspeople from over 20 countries are expected to attend, along with more than 115 exhibitors.

What is packaging?

IC-packaging is an important part of the semiconductor industry. In a semiconductor production facility or fab, a chipmaker manufactures a multitude of different chips.  A given chip line is targeted for a specific application, such as AI, automotive, consumer and wireless.    

In most cases, the chips are then sent to a separate facility, where they are assembled into an IC package. Basically, a chip resides inside a package. A package is a small enclosure that protects the chip from harsh operating conditions.

There are a multitude of different package types in the market. Each type is geared for a specific application.  Some packages are simple, commodity-level products for use in more mature applications. Some are considered mid-range packages, which are used for more advanced systems. Still others are classified as advanced packages, which are optimized for the most complex applications.  

A typical IC package. This is a 28-pin quad-flat no-leads (QFN) chip package. This IC package is upside down to show contacts and thermal/ground pad. Source: Wikipedia

In all cases, the package incorporates different types of connection schemes, such as tiny copper bumps, metal leads or I/Os. These connection schemes provide the electrical signals from the chip to a board.       

Several types of companies provide assembly and packaging services for customers (chipmakers, fabless design houses). The first type are OSATs, which specialize in providing packaging and testing services for customers. Amkor, ASE, JCET and others are considered OSATs. IBM falls in that category.      

Several foundry vendors, such as Intel, Samsung and TSMC, also provide packaging services for customers. Foundries also provide semiconductor manufacturing services as well.  

Nonetheless, packaging has always been important. But in recent times, advanced packaging has taken center stage in the market. Apple, AMD, Intel and others make use of advanced packages in their products.  

There are several reasons why these products use advanced packages. Take the server chips as an example. Generally, these complex chips consist of multiple dies, such as processors, DRAM, I/Os and others. It makes no sense to scatter all of these chips on the board. This would take up too much board space and cause unwanted latency between the chips.

So, chipmakers take these devices (processors, memories, etc.) and integrated them in a single advanced package. This in turn saves board space and reduces the signal latency.

AI, the edge and packaging

Advanced packaging is used for a range of applications. AI is perhaps the hottest market in the arena.   

AI is a complex topic. Today, the most prevalent AI technology deployed in the market is something called deep learning. In a system, deep learning uses multi-layered networks, “called deep neural networks, to simulate the complex decision-making power of the human brain,” according to IBM.   

Deep learning isn’t new. For years, many companies have deployed deep learning algorithms in their systems for a wide range of applications.

Today, generative AI, the latest craze in AI, is considered the most advanced deep learning algorithm. ChatGPT, a chatbot developed by OpenAI, is one example of generative AI.

Generative AI algorithms incorporate large language models that utilize a massive amount of data. Typically, the data is processed in a datacenter. Datacenters are large facilities that are equipped with high-end computers or servers. These servers are processing large amounts of general-purpose data.

For generative AI, datacenters use so-called AI servers to process massive amounts of data. AI servers are equipped with the latest and greatest chips, including high-speed GPUs, memories and other devices.   

Generally, in the server, the GPUs, memories and other chips are not scattered throughout the board. This is inefficient. This implementation would take up too much board space and would likely cause unwanted latency between different chips.

Instead, these devices (GPUs, memories, etc.) are typically integrated in a single advanced package. In this case, the GPU die and a DRAM memory stack sit side-by-side on top of an interposer in the package. The interposer is a structure that provides signals from the dies to the board.

Some call this a 2.5D package. This implementation saves board space and reduces the latency between different chips.

Using AI servers, along with the right chips, generative AI produces what appears to be incredibly fast, human-like conversations with end users.

The problem? Generative AI is prone to errors. Plus, the systems consume too much power. It will take several breakthroughs to solve these problems. Advanced packaging could help some but not all matters.

“Given the growth in large language models (LLMs) for artificial intelligence (AI) applications, a single neural network may now contain billions of parameters for computation. That number is expected to increase further at an unprecedented rate,” according to KAIST, ETRI and Amkor, in an abstract that describes their upcoming paper at ECTC. “For higher-performance, more energy-efficient neural network computing, more processor and memory dies need to be integrated in a single package. However, fabricating these integrated systems is difficult and costly. It is also challenging to increase memory bandwidth, due to the memory bottleneck arising from the relatively slow bandwidth of off-chip memory, and from the excessive energy consumption needed for frequent data access.”

At ECTC, KAIST, ETRI and Amkor will present a paper entitled, “The Energy-Efficient 10-Chiplet AI Hyperscale NPU on Large-Scale Advanced Package.” A chiplet is a term that describes a chip or a die.

Researchers from the KAIST-led group developed an advanced packaging solution for an AI hyperscale processor unit (HPU). The HPU-based package is optimized to handle two neural processor unit (NPU) devices and eight high-bandwidth memories (HBM3). The dual-NPU devices achieve peta-scale performance through energy-efficient tensor cores optimized for AI computation, according to the abstract.

All 10 chiplets are integrated in a package using an interposer. High-density interconnects are used between the NPUs and HBMs to enable near-TB/s bandwidth, according to the abstract.

Others are taking a slightly different approach. At ECTC, ASE will present a paper that extends the capabilities of its Fan-Out-Chip-on-Substrate-Bridge (FOCoS-Bridge) technology.

Fan-out is a type of advanced package. The package incorporates several dies in the same structure. There are different ways to connect the dies in the package.

In this case, ASE makes use of a small piece of silicon, sometimes called a silicon bridge, which connects different dies in the package. At ECTC, ASE for the first time will describe a large chip module (>3X reticle size) containing 10 chiplets and 10 silicon bridge dies.

That’s not the only solution. Generally, the power consumption in today’s datacenters is increasing at an alarming rate. In response, the industry has been working on various ways to process some of the data at the edge of a network. This in turn could offload the processing, thereby reducing the power consumption.

At ECTC, IBM will address these issues by presenting a paper, entitled “Ultra-Compact Computing at the Edge Involving Unobtrusively Small Sub-Millimeter Heterogeneous Integration Packaging.”

In the paper, IBM will describe a heterogeneous integration-based chiplet packaging process. This process will make it possible to build a computer system on a tiny substrate (<1mm2). The system consists of a 32-bit processor, memory, analog I/O with built-in temperature and chemical sensors, energy-harvesting power source, and operating system software.

To demonstrate the packaging process, IBM built heterogeneous chips thinned to 50-100μm, with lateral dimensions of several hundred microns, separated by 20μm gaps, and with 10-20μm-diameter copper pillars.

More packaging

Others will also present papers at ECTC. Intel, for one, will describe an infrared (IR) laser debond technology for 2.5D/3D packages.  

As stated, chips are manufactured in a fab. Then, in some cases, the chips are placed on a thin substrate during the assembly and packaging process. These substrates are fragile and sometimes are unable to withstand the packaging process.

So in some cases, the substrate is bonded to a separate carrier wafer, providing mechanical support for the thin substrate during processing. Then, once the packaging process is completed, the substrate is debonded from the carrier wafer using several methods.

Intel’s IR laser debond technology uses inorganic thin films to control a laser release process from the silicon carrier wafer. Intel has demonstrated the process in various applications, such as backend metal layers, 3µm pitch hybrid-bond interconnects, as well as singulated chiplets. All of this is done without any damage to the structure.