Serialised memory interface gains momentum
The potential for a clogged network is coming as little surprise. The IEEE Ethernet Bandwidth Assessment Report of 2012 predicted a 100-fold increase in network traffic by 2020, requiring networks to support up to 10 terabits per second. With this prediction, the network will need 400-gigabit-per-second link speeds as early as 2015.
While networking system memory has traditionally relied on legacy DRAM and SRAM connected to line card processors through parallel interfaces, higher capacity line cards supporting transmission rates of 100 Gbps and beyond will require scalability that a serial interface provides.
Legacy memory not only presents bandwidth and access-rate constraints, but also locks network line cards into inefficient pin count, board space and power requirements that add cumulatively to system costs. To break free of these constraints, network design engineers are now designing better alternatives.
Serialising the memory access on networking line cards is rapidly gaining momentum as one of these alternatives because it reduces all three limitations. Furthermore, this approach avoids the commercial and logistical quagmire of the technical panacea known as “multi-chip modules” (MCMs).
MCMs deliver the performance required by using advanced and sophisticated packaging technologies. Once the interconnect challenges are solved the user is still faced with a daunting cooling requirement created by the thermal density of the package. However, the ultimate reason why MCM adoption is limited is because of supply chain logistics and inventory ownership. These issues are not insurmountable for mobile phones where the power is low, unit costs are in the single digit range, the market is huge and the lifetime expectation is only a few years. The networking infrastructure market is at the opposite end of the spectrum by comparison.
Today, line cards in high-speed networks aggregate to 100Gbps or more. Although deployments of these systems have been generating headlines for several years, they are just now evolving from the early adoption phase, where performance won out over power and space constraints. These cards are now moving into broad commercial deployment for system vendors and their customers, the network service providers. As 100G moves toward an inflexion point history has shown that system cost reduction measures are the most important consideration in driving card design.
Current 100Gbps line cards are already experiencing memory bottlenecks and commodity DRAM continues to evolve by emphasising storage capacity needed for computing systems. Just as DRAM bandwidth performance increases linearly while networking rates are growing exponentially, faster memory solutions – such as networking SRAM, and speciality DRAM – will also be unable to scale along with expanding network demands. Line rates in Metro, Core and Carrier networks have more than doubled every 18 months, outstripping even the bandwidth and access requirements of the PC processor evolution.
The NPUs, SoCs, ASICs and FPGAs at the heart of the network cards use serialised links for every other interconnect because of performance requirements. Yet, host processors on 100Gbps line cards have continued to rely on legacy memory because of two key aspects – commodity pricing and availability. As engineers strive to keep pace with the ever-increasing demand for high bandwidth content delivery and fast, reliable connectivity, they are turning toward the high efficiency of serial interface technology to connect to a high performance, deterministic memory array architecture.
A straightforward comparison demonstrates the pin count, board space and power limitations of the legacy approach and the advantages of serialisation. In the example of a 200GE over-subscription buffer, a DDR3 implementation requires 20 devices, a 600-pin count, 30W and 1920 mm2 of board space. A higher access rate memory, such as the MoSys Bandwidth Engine, requires only two devices to support the same 200GE buffer over a serial memory interface. Furthermore, the pin count is reduced to 128, power is reduced to 14W and board area is reduced to 720 mm2. (See Table 1).
Table 1. A 200GE over-subscription buffer used on a network line card and how different memory types are implemented. The first three types use parallel interconnects to interface with the network-processing chip. The last type shows how a serial interconnect can be implemented to reduce pin count, board space and power consumption.
High speed serial I/O between the host processor and memory enables designers reduce pin count dramatically. Compared to SRAM, the migration to denser, higher-speed memory types enabled by serialising the memory interface reduces the number of devices required to achieve the throughput, which is also compelling for both board space and power consumption reasons. The leading design consideration today is to reduce the cost per port. Therefore, reducing the power consumption by decreasing the number of memory devices and shrinking board space makes the serial interface the obvious approach to address the I/O constraints and inefficiencies imposed by parallel bus technology. Network designs that continue to rely on the legacy approach will be disadvantaged compared to a new generation of designs based on serial I/O to memory.
Serial interface technology is now pervasive and it has favourable cost points. There are high-speed, serialised interface technologies, such as the GigaChip Interface (GCI), that have achieved industry standard status and are available without royalty fees and licensing costs. When implemented at 15 Gbps, the GCI is the highest data throughput, CRC-protected (Cyclic Redundancy Check) interface available for networking designs. In a 100GE packet arrival interval time, the Bandwidth Engine-2 can receive up to eight GCI frames, which could be used to perform up to sixteen reads and writes, to and from memory for querying tables, buffering data, or update statistics, perform metering, or any combination thereof. This device is an attractive solution for high-capacity line cards, not only because it meets the performance requirements of today’s networks, but it provides the scalability for 400Gbps to 1-Terabit-per-second (Tbps) systems while reducing pin count, board space and power consumption.
Serial interface to memory has been attempted in the past but was thwarted by evolutionary discrete component technology. We have now reached a point where the parallel approach can’t compete with serial. Furthermore, the ability to use discrete serial components preserves the existing infrastructure and supply chain that the entire electronics industry is built upon. To continue down the parallel path requires migration to multi-chip modules (MCM), which have inherent business and logistical hurdles, which have historically prevented its adoption. The time for serial is now.
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
