Here Comes 25 Gig Ethernet!
You may have heard plenty of industry buzz over the past few months regarding 25 gigabit Ethernet. That is no surprise considering that several big data centers and cloud computing providers like Google, Microsoft, Broadcom, Arista and Mellanox formed the 25 Gigabit Ethernet Consortium in early 2014.
Shortly thereafter, IEEE formed a 25G Ethernet study group, and on December 10, 2014, the IEEE P802.3by 25 Gb/s Ethernet Task Force was approved to begin developing an amendment that defines MAC parameters and physical layer specifications for full duplex 25G Ethernet transmission in server-to-switch interconnection applications.
While the amendment is not expected to be ratified until 2016, pre-standard 25G Ethernet products are expected as it has been the long-standing trend in the industry. In fact, Broadcom has already announced the availability of a new high-density 25G Ethernet switch for cloud-scale data centers.
While we wait for the IEEE Task Force to define single-lane, 25G PHY operation over copper twinax and multimode optical fiber, which will likely be one of the most expeditious objectives ever met within IEEE, let’s take a closer look at why 25G makes sense from a technology, cost, scalability and flexibility perspective.
We’re Already There
The existing 100G standards such as 100GBASE-LR4 and 100GBASE-ER4 already are comprised of four lanes of 25G over singlemode fiber using course wavelength division multiplexing (CWDM) technology. The 100GBASE-CR4 and 100GBASE-KR4 standards for 4 X 25G over twinax and backplane were also recently ratified, and the 100GBASE-SR4 standard with 4 X 25G per lane over multimode fiber is in the final stage of approval.
100 gigabit QSFP28 SR4 optical transceiver modules are already available for this application and have been successfully deployed in operational networks. With the same footprint as the 40G QSFP+ for four 10G lanes, the QSFP28 essentially offers 250 percent more density. It also supports direct-attach copper (DAC) cable assemblies, active optical cable (AOC) assemblies and transceivers. SFP28 optical transceiver modules for one lane of 25G are also available and most often seen where one QSFP28 cable breaks out to four SFP28 cables. CFP2 and CFP4 transceivers also support a four-lane 25G configuration.
With 4 X 25G per lane component technology already available, the process to go to a single 25G lane is a simple one that requires very minor changes and significantly reduces cost compared to 40G using four 10G lanes. It won’t be long before SFP28 to SFP28 cables hit the market alongside 25G switches.
One of the key drivers for 25G is cloud computing. Not only are cloud providers looking to decrease cost and improve density, but improvements in server speeds has rendered to 10G, which is too slow. Many of these environments have standardized on a top of rack (ToR) topology, and keeping up with these connections at 10G requires more switches in the rack and therefore higher cost. Accordingly, the earliest deployments of 25G will take place in cloud and hyper scale data centers, followed later by enterprise data centers.
25G also works from a scalability standpoint. With up-links migrating to 100G using the same type of technology, it makes sense to increase the switch-server speed at the edge to 25G. With cost benefits over 40G and the technology already gaining traction, a recent five-year forecast by Dell’Oro Group predicts that 25G will take over Ethernet server port sales by 2018.
Beyond Top of Rack
While 25G over twinax DAC assemblies will fulfill the ToR server environments where a distance of 3 to 5 meters is more than adequate, there is also the need for longer distances to support the middle of the row (MoR) topologies to about 15m and end of row (EoR) to 30m. That’s where a future 25GBASE-T application over balanced twisted-pair copper cabling has potential to fill the gap.
At the September meeting of IEEE 802.3, the Task Force is developing 40GBASE-T (P802.3bq) which discussed the technical feasibility and the benefits of 25GBASE-T relative to 40GBASE-T. As a result, a Call for Interest (CFI) was held at the IEEE 802.3 meeting in November. It was agreed upon to form a study group to explore 25GBASE-T Ethernet and potentially extend the work of the P802.3bq Task Force to include it. Some considerations will include maintaining backwards compatibility with 10GBASE-T and forward compatibility with 40GBASE-T through auto-negotiation, as well as keeping the power per port low and supporting Energy Efficient Ethernet.
When it comes to home run applications where servers and storage devices are consolidated into separate areas of a data center, the distances required increase to about 50 to 100m. A single-lane 25G Multimode fiber PHY will be a significant solution to support home run applications and an overall broader range of 25G architectures.
The table below highlights which media will likely best fit the various data center topologies and their average distances for 25G applications.
While standards groups are actively focusing on 25G Ethernet over twisted-pair, twinax and fiber, it’s still too soon to tell how it will shape up. Regardless, 25G Ethernet has broad market potential in the server interconnect world because it’s both economically and technically feasible—on a variety of media.
With 25G, total cost of ownership (TCO) will be lower as switching capabilities will be better utilized, and it is well positioned to offer higher density and lower power with no changes in architecture required. In other words, 25G Ethernet is definitely coming—and it is coming fast.