Speed Lightning to data centers and circuit boards

Scientists at IBM Research have announced a new set of advancements in chip assembly and packaging, called co-packaged optics, that promises to improve energy efficiency and boost bandwidth by bringing optical link connections inside devices and within the walls of data centers used to train and deploy large language models. This new process promises to increase the number of optical fibers that can be connected at the edge of a chip, a measure known as beachfront density, by six times. As artificial intelligence demands ever more bandwidth, this innovation will use the world’s first successful polymer optical waveguide to bring the speed and bandwidth of optics all the way to the edge of chips.

Early results suggest that switching from conventional electrical interconnects to co-packaged optics will slash energy costs for training AI models, speed up model training, and dramatically increase energy efficiency for data centers.

Today’s advanced chip and chip packaging technologies typically use electrical signals for the transistors in microelectronics that power phones, computers, and almost everything that we do. Transistors, for their part, have gotten many times smaller over the decades, enabling us to pack more capability into a given space. But even the most capable semiconductor components are only as fast as the connections between them.

These connections make it possible for us to seamlessly use electronic devices in our daily lives — like when we drive our cars, which include chips in nearly every system from the seats to the tires. “Even your refrigerator has electronics in it to help everything operate properly,” says IBM Research engineer John Knickerbocker, a distinguished engineer of chiplets and advanced packaging.

Knickerbocker and his team are thinking smaller, though. Because of optical connectors’ lower cost and higher energy efficiency, they make great candidates for improving the performance of chip-to-chip and device-to-device communication in data centers, where generative AI computing is demanding ever higher and higher bandwidth.

“Large language models have made AI very popular these days across the tech industry,” Knickerbocker says. “And the resulting growth of LLMs — and generative AI more broadly — is requiring exponential growth in high-speed connections between chips and data centers.”