Reap the benefits of 10GBase-T connectivity in data centers – Part I
Ethernet at 10G speeds has arrived! The growing importance of cloud computing and the increasing utilization of unified data/storage connectivity and server virtualization by enterprise data centers, have conspired to elevate the importance and popularity of 10Gbps Ethernet. Not long ago considered an exotic connectivity option relegated to high-capacity backhaul, more and more applications are taking advantage of the availability and cost-effectiveness of 10GE links. As was the case with three prior generations of Ethernet, the ubiquity, the ready and familiar management tools, and the compelling cost structure are allowing 10G Ethernet (10GE) to quickly dominate the computer networking scene.
Crehan Research, a leading industry analyst of data center technologies, claims over 8 million 10GE ports in data center switches for 2011. And in its January, 2011, report, The Linley Group, another leading industry analyst, predicted robust 10GE growth and estimated that 10GE NIC/LAN-on-motherboard (LOM) shipments alone will surpass 16 million ports in 2014.
Starting in 2002, The Institute of Electrical and Electronics Engineers (IEEE) has created several standards for 10G Ethernet connectivity. The more popular ones include:
* 10GBase-SR uses optical modules with 850nm lasers to work over multi-mode fiber.
* 10GBase-LR uses optical modules with 1310nm lasers to work over single-mode fiber.
* 10GBase-LRM uses optical modules with 1310nm lasers and works over multi-mode fiber.
* 10GBase-ER uses optical modules with 1550nm lasers and works over single mode fiber with reach up to 40km.
* 10GBase-KX4 which operates over four copper backplane lanes with distance up to 1 meter
* 10GBase-KR which operates over a single backplane lane with distance up to 1 meter
* 10GBase-T, which operates over Category 6 (Cat6) and Category 6A (Cat6A) twisted pair copper cabling with distance up to 100 meters.
In addition, a non-IEEE Standard approach called SFP+ Direct Attach has also gained popularity. This method uses a passive twin-ax cable assembly, which connects directly into a SFP+ module housing. The cable assemblies are ordered in pre-specified lengths and come with the SFP+ module form factor connecters attached. Distances between three and 10 meters are supported, depending on the coaxial cable thickness used.
The focus of this two-part article is on the 10GBase-T connectivity option. Of all the options available, 10GBase-T, which is also known as IEEE 802.3an, is arguably the most flexible, economical, backwards-compatible, and user-friendly 10GE connectivity option available. It was designed to operate with the familiar unshielded twisted pair cabling technology, which is already pervasive in environments using and can interoperate directly with 1GE. It is capable of covering, with a single cable type, any distance up to 100 meters and, therefore, reaches 99 percent of the distance requirements in data centers and enterprise environments. In this article, we will explore the basics of 10GBase-T technology, explain the many benefits it brings to the data center, outline the current state of the art, and explore upcoming advances and their implications.
10GBase-T Basics
Ratified in June 2006, IEEE 802.3an provided a stable blueprint for chip manufacturers to develop and introduce compliant and interoperable devices allowing for 10Gbps communications over unshielded twisted pair cabling. 10GBase-T is the fourth generation of so-called BASE-T technologies, which all use RJ45 connectors and unshielded twisted pair cabling to provide 10Mbps, 100Mbps, 1Gbps, and 10Gbps data transmission while being backwards-compatible with prior generations. Because BASE-T devices have used an auto-negotiation protocol defined by the IEEE to determine the capabilities supported by the other end of the link, this backward compatibility has meant that upgrades could be performed one end at a time, allowing quick and easy incremental improvement of network speed without either changing the wiring or forklift upgrades of equipment.
The 10GBase-T transceiver uses full duplex transmission with echo cancellation on each of the four twisted pairs available in standard Ethernet cables, thereby transmitting an effective 2.5Gbps on each pair. These bits are transformed into a bandwidth-reducing line code called 128-DSQ (for double square), which limits the analog bandwidth utilization of the 10GBase-T modem to 400MHz. High-performance line equalization countermands the low-pass filter effects of the transmission channel, and additional digital signal processing (DSP) functions cancel the crosstalk and echo impairments present in the cabling. Additionally, powerful low-density parity check, or LDPC, forward error correction coding rounds out some of the DSP functions and allows nearly error-free detection at close to fundamental limits in signal-to-noise ratio. The block diagram in Figure 1 depicts an example of a 10GBase-T transceiver.
Figure 1: Block diagram of a 10GBase-T transceiver showing the major DSP blocks responsible for line equalization, LDPC forward error correction, and analog line code data transformation.
Since the DSP capabilities of the 10GBase-T transceivers result in full internal pair-to-pair crosstalk cancellation, these transceivers are limited in their performance by alien crosstalk, i.e. the undesired signal coupling between adjacent components and cabling. Bearing this sensitivity in mind, while 10GBase-T transceivers are capable of operation over various types of twisted pair cabling, Cat6A cabling was purpose-built for this type of transmission. Specifically, Cat6A cabling requirements were developed to address the alien crosstalk headroom required to support 10GBase-T over 100 meters of cabling containing up to four-connectors and to deliver positive signal-to-alien crosstalk margin and the extended frequency bandwidth of up to 500MHz.
While Cat6A cabling is ideal, Cat6 cabling, which was primarily targeted to support 100Base-T and 1000Base-T transceivers, may also be applicable to some 10GBase-T applications. Because the alien crosstalk in Cat6 UTP cabling is extremely dependent upon installation practices (e.g., bundling, the use of tie-wraps, and pathway fill), deployment guidelines were developed based upon a ╥typical╙ worst-case environment. These show that 10GBase-T transceivers should operate over Cat6 UTP channel lengths of up to 37 meters and may operate over channel lengths of 37 to 55 meters, depending upon the actual alien crosstalk levels present. The TIA TSB-155-A and ISO/IEC 24750 technical bulletins identify the additional performance headroom, as well as applicable field qualification test requirements and procedures, which must be satisfied by the installed base of Cat6 cabling in order to support 10GBase-T.
The previously mentioned 128-DSQ line code used by 10GBase-T system increases the number of bits per symbol when compared to prior Base-T standards. This is important since an external signal that introduces electromagnetic interference (EMI), couples to the cable common mode and gets converted to a differential signal may cause errors on a 100-meter 10GBase-T link.
Common EMI tests, such as those mandated by the Telcordia GR1089 standard, call for testing with field strengths of 8.5V/m. Measurements using Cat5 and Cat6 unshielded twisted pair cabling in 8.5V/m electromagnetic fields indicate that differential pickup can easily reach 60mV, thereby exceeding the voltage margin at the receiver of a 100-meter 10GBase-T link.
To contend with such EMI events, particularly in an unshielded cabling system, 10GBase-T transceivers support adaptive interference cancellation. Such an algorithm requires three key steps: detecting the interferer, identifying it, and then removing it.
Detection can be based on analysis of the differential signal available at each receiver. This detection must operate in the presence of the desired signal being sent from the transmitter on the other side, thus requiring sophisticated signal processing. A Fourier transform or other techniques applied to the received differential signal can reveal the source of interference that would otherwise go undetected because its voltage levels are lower than that of the transmitter signal.
The EMI signal can be detected with much higher confidence on the common mode signal but the transformers traditionally used for isolation in Base-T Ethernet links do not have a receiver for the common mode signal. Newer generation of 10GBase-T solutions offer a so-called ╥fifth channel╙ receiver dedicated to the common mode signal. This receiver requires an extra transformer beyond what is in the traditional magnetics but can greatly improve the reliability of EMI detection. 10GBase-T PHYs from vendors such as PLX Technology are designed to be configurable for both modes of operation, allowing system makers to determine their own tradeoff on robustness vs. cost.
Once the EMI signal is detected, adaptive filtering techniques can learn the frequency and relative amplitude of the interference and filter it out. The latest generation of 10GBase-T equipment can recover from large EMI events (10V/m fields) in less than 10 microseconds, and simultaneously cancel multiple EMi occurrences and operate error-free after adaptation to the EMI. Figure 2 is a photograph of a test chamber that is used for EMI testing of 10GBase-T enabled equipment.
Figure 2: An EMI test chamber. Networking equipment manufactures test and prequalify 10GBase-T equipment in such chambers to make sure it meets the Telcordia GR1089 standard for Electro-Magnetic Interference. This assures users of such equipment in a data center that EMI will not be a problem.
Summary
10GBase-T technology was designed to provide a backwards-compatible, incremental upgrade for 10Gbps Ethernet, and minimize the disruption that a transition to 10Gbps speed could mean with other interconnection technologies. This focus on easy-transition shows up in everything from support for 100 and 1000Base-T interconnects, to architecturally enabling larger, simplified management domains, to reuse of existing cabling practices.
Note: Part II of this article will explore 10GBase-T technology at a deeper level, and how its deployment is revolutionizing data centers.
About the Authors
Ron Cates is vice president of marketing, networking products, at PLX Technology, Sunnyvale, Calif. a leader in 10GBASE-T transceivers. He has more than 30 years of experience in the semiconductor industry and holds BSEE and MSEE degrees from the University of California at Los Angeles and an MBA from San Diego State University. He can be reached at rcates@plxtech.com.
George Zimmerman is an independent consultant, specializing in physical layer communications technology. He is an acknowledged expert in wireline communications and has been a defining force in the development of 10GBASE-T, Energy Efficient Ethernet, and various DSL technologies. He holds a PhD. in electrical engineering from Caltech, and an undergraduate degree from Stanford University. He can be reached at gzimmerman11@cox.net.
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
