PAM4 — The High-Speed Signal Interconnection Technology of Next-Generation Data Center

What Is PAM4?

PAM4 (4-Level Pulse Amplitude Modulation) is one of PAM modulation technologies that uses 4 different signal levels for signal transmission. Each symbol period can represent 2 bits of logic information (0, 1, 2, 3), that is, four levels per unit time.

In the data center and short-distance optical fiber transmission, the modulation scheme of NRZ is still adopted, that is, the high and low signal levels are used to represent the (1, 0) information of the digital logic signal to be transmitted, and one bit of logical information can be transmitted per signal symbol period.

However, as the transmission rate evolves from 28Gb/s to a higher rate, the electrical signal transmission on the backplane will cause more severe loss to the high-frequency signal, and higher-order modulation can transmit more data in the same signal bandwidth. Therefore, the industry is increasingly calling for higher-order PAM4 modulation. The PAM4 signal uses four different signal levels for signal transmission, and each symbol period can represent 2 bits of logical information (0, 1, 2, 3). Since the PAM4 signal can transmit 2 bits of information per symbol period, to achieve the same signal transmission capability, the symbol rate of the PAM4 signal only needs to reach half of the NRZ signal, so the loss caused by the transmission channel is greatly reduced. With the development of future technologies, the possibility of using more levels of PAM8 or even PAM16 signals for information transmission is not ruled out.

NRZ vs. PAM4: The comparison of waveforms and eye diagrams between NRZ and PAM4 signals

And then, if the optical signal can also be transmitted by using the PAM4, the clock recovery and pre-emphasized PAM4 signal can be directly realized when the electro-optical transmitting is performed inside the optical module, therefore, the unnecessary step of converting the PAM4 signal into the NRZ signal of 2 times the baud rate and then performing related processing is eliminated, thereby saving the chip design cost.

Why PAM4?

The end-to-end transmission system includes fiber optic and fiber-optic transmission systems. Since the fiber transmission can easily reach the rate of 25Gbd so that the research progress of transmitting PAM4 on the fiber has been progressing slowly. For fiber-optic transmission systems, from NRZ moving to PAM4 is considered in terms of cost. If you do not need to consider the cost, there are other related modulation technologies can be used in the long-distance range, such as DP-QPSK, which can transmit the baud rate signal above 50Gbd for several thousand kilometers. However, in the data center field, the transmission distance is generally only 10km or less. If the optical transceiver using PAM4 technology is adopted, the cost can be greatly reduced.

For 400GE, the largest cost is expected to be optical components and related RF packages. PAM4 technology uses four different signal levels for signal transmission. It can transmit 2 bits of logic information per clock cycle and double the transmission bandwidth, thus effectively reducing transmission costs. For example, 50GE is based on a single 25G optical device, and the bandwidth is doubled through the electrical layer PAM4 technology, which effectively solves the problem of high cost while satisfying the bandwidth improvement. The 200GE/400GE adopts 4/8 channel 25G devices, and the bandwidth can be doubled by PAM4 technology.

For data center applications, reducing the application of the device can significantly reduce costs. The initial goal of adopting higher order modulation formats is to place more complex parts on the circuit side to reduce the optical performance requirements. The use of high-order modulation formats is an effective way to reduce the number of optics used, reduce the performance requirements of optics, and achieve a balance between performance, cost, power, and density in different applications.

In some application scenarios, high-order modulation formats have been used for several years on the line side. However, since the client side needs are different from the line side, so other considerations are needed.

For example, on the client side, the main consideration is the test cost, power consumption and density. On the line side, spectrum efficiency and performance are mainly considered, and cost reduction is not the most important consideration. By using linear components on the client side and the PAM4 modulation format that is directly detected, companies can greatly reduce test complexity and thus reduce costs. Among all high-order modulation formats, the lowest cost implementation is PAM4 modulation with a spectral efficiency of 2 bits/s/Hz.

PAM4

Conclusion

As a popular signal transmission technology for high-speed signal interconnection in next-generation data centers, PAM4 signals are widely used for electrical and optical signal transmission on 200G/400G interfaces. Gigalight has a first-class R&D team in the industry and has overcome the signal integrity design challenges of PAM4 modulation. Gigalight’s 200G/400G PAM4 products include 200G QSFP56 SR4, 200G QSFP56 AOC, 200G QSFP56 FR4, 400G QSFP56-DD SR8, 400G QSFP56-DD AOC, etc.

All of the PAM4 products from Gigalight can be divided into digital PAM4 products and analog PAM4 products. The digital PAM4 products adopt DSP solutions which can support a variety of complex and efficient modulation schemes. The electric port has strong adaptability and good photoelectric performance. And the analog PAM4 products simulate CDR with low power consumption and low cost. Gigalight always adheres to the concept of innovation, innovative technology, and overcomes difficulties. It invests a lot of human resources and material resources in the research and development of next-generation data center products.

Originally published at morph.tilda.ws

What is Data Center Interconnect/Interconnection?

Data Center Interconnection means the implements of Data center Interconnect (DCI) technology. With the DCI technology advances, better and cheaper options have become available and this has created a lot of confusion. This is compounded by the fact that a lot of companies are trying to enter this market because there is a lot of money to be made. This article is written to straighten out some of the confusion.

According to the different applications, there are two parts of data center interconnections. The first is intra-Data Center Interconnect (intra-DCI) which means connections within the data center. It can be within one building or between data center buildings on a campus. Connections can be a few meters up to 10km. The second is inter-Data Center Interconnect (inter-DCI) which means connections between data centers from 10km up to 80km. Of course, connections can be much longer but most of the market activity for inter-DCI is focused on 10km to 80km. Longer connections are considered Metro or Long-haul. For reference, please see the table below.

DCI Distance Fiber Type Optics Technology Optical Transceivers
intra-DCI 300m MMF NRZ/PAM4 QSFP28 SR4
500m SMF QSFP28 PSM4
2km QSFP28 CWDM4
10km QSFP28 LR4
inter-DCI 10km SMF Cohernet QSFP28 4WDM-10
20km QSFP28 4WDM-20
30km to 40km QSFP28 4WDM-40
80km to 2000km CFP2-ACO

Intra-DCI

The big bottlenecks are in the intra-DCI and therefore, the highest volume of optical transceivers are sold here generating the most revenue, however, it is low margin revenue because there is so much competition. In this space, may of the connections are less than 300m and Multi-Mode Fiber (MMF) is frequently used. MMF is thicker, and components are cheaper because the tolerances are not as tight, but the light disperses as it bounces around in the thick cable. Therefore, 300m is the limit for many types of high speed transmission that use MMF. There is a data center transceiver with a transmission distance up to 100m over OM4 MMF for example.

Gigalight 100GBASE-SR4 100m QSFP28 Optical Transceiver

100G QSFP28 SR4 for MMF up to 100m

In a data center, everything is connected to servers by routers and switches. Sometimes a data center can be one large building bigger than a football field and other times data centers are built on a campus of many buildings spanning many blocks. In the case of a campus, the fiber is brought to one hub and the connections are made there. Even if the building you want to connect to might be 200m away, the fiber runs to a hub, which can be more than 1km away, so this type of routing increases the fiber distance. Some of the distances between buildings can be 4km, requiring Single Mode Fiber (SMF), which has a much narrower core, making it more efficient, but also increasing the cost of all related components because the tolerances are tighter. Therefore, with data centers growing, so has the need for SMF as the connections get longer within the data center. With SMF you have the option to drive high bandwidth with coherent technology, and we’ll see more of this in the future. Previously coherent was only used for longer distances, but with cost reductions and greater efficiency versus other solutions, coherent is now being used for shorter reaches in the data center.

Gigalight 100GBASE-LR4 Lite 4km QSFP28 Optical Transceiver

100G QSFP28 LR4L for SMF up to 4km

500m is a new emerging market and because the distance is shorter, a new technology is emerging, and that is silicon photonics modulators. EMLs (Externally Modulated Lasers) perform modulation within the laser, but with silicon photonics, the modulator is outside the laser and it’s a good solutions for distances of 500m. In an EML, the modulator is integrated into the same chip, but is outside the laser cavity, and hence is “external”. For silicon photonics, the laser and modulator are on different chips and usually in different packages. Silicon photonics modulators are based on the CMOS manufacturing process that is high scale and low cost. A continuous wave laser with silicon photonic modulation is very good for 500m applications. EMLs are more suitable for longer reaches, such as 2-10km. Therefore, with data centers growing, so has the need for single mode fiber as the connections get longer within the data center. With SMF you have the option to drive high bandwidth with coherent technology, and we’ll see more of this in the future. Previously coherent was only used for longer distances, but with cost reductions and greater efficiency versus other solutions, coherent is now being used for shorter reaches in the data center.

100GE PSM4 2km QSFP28 Optical Transceiver

100G QSFP28 PSM4 for SMF up to 500m/2km

100GE CWDM4 2km QSFP28 Optical Transceiver

100G QSFP28 CWDM4 for SMF up to 2km

100GBASE-LR4 10km QSFP28 Optical Transceiver

100G QSFP28 LR4 for SMF up to 10km

Inter-DCI

Inter-DCI is typically between 10km and 80km, including 20km and 40km. Before we talk about data center connectivity, let’s talk about why data centers are set up the way they are and why 80km is such an important connection distance. While it is true that a data center in New York might backup to tape in a data center in Oregon, this is considered regular long-haul traffic. Some data centers are geographically situated to serve an entire continent and others are focused on a specific metro area. Currently, the throughput bottleneck is in the metro and this is where data centers and connectivity are most needed.

100GE 4WDM-20 20km QSFP28 Optical Transceiver

100G QSFP28 4WDM-20 for SMF up to 20km

100GE 4WDM-40 40km QSFP28 Optical Transceiver

100G QSFP28 4WDM-40 for SMF up to 40km

Say you have a Fortune 100 retailer and they are running thousands of transactions per second. The farther away a data center is, the more the data is secure because the data center is so far away and separate from natural disasters, but with the increased distance there are more “in flight” transactions are at risk of being lost due to latency. Therefore, for online transactions there might be a primary data center that is central to retail locations and a secondary data center that is around 80km away. It’s far enough away not to be affected by local power outages, tornadoes, etc, but close enough that there is only a few hundred milli-seconds of latency; therefore, in the worst case a small number of transactions would be at risk.

In another example of inter-DCI, as if a certain video is getting a lot of views, the video is not only kept in its central location, but copies of the video are pushed to metro data centers where access is quicker because it’s stored closer to the user, and the traffic doesn’t tie up long haul networks. Metro data centers can grow to a certain size until their sheer size becomes a liability with no additional scale advantage and thus they are broken up into clusters. Once again, to guard against natural disasters and power outages, data centers should be far away. Counterbalancing this, data centers need to have low latency communication between them, so they shouldn’t be too far away. There is a compromise and the magic distance is 80km for a secondary data center, so you’ll hear about 80km data center interconnect a lot.

It used to be that on-off keying could provide sufficient bandwidth between data centers, but now with 4K video and metro bottlenecks, coherent transmission is being used for shorter and shorter distances. Coherent is likely to take over the 10km DCI market. It has already taken over the 80km market but it might take time before coherent comes to 2km. The typical data center bottlenecks are 500m, 2km, and 80km. As coherent moves to shorter distances, this is where the confusion comes.

The optical transceiver modules that were only used within the data center are gaining reach, and they’re running up against coherent solutions that were formerly only used for long distances. Due to the increasing bandwidth and decreasing cost, coherent is being pulled closer into the data center.

The other thing to think about is installing fiber between data centers. Hopefully this is already done, because once you dig, it’s a fixed cost, so you put down as many fibers as you can. Digging just for installing fiber is extremely expensive. In France when they lay fiber, they use another economic driver. Whenever you put in train tracks, you put in fiber at the same time, even if it is not needed. It’s almost for free because they are digging anyway. Fibers are leased to data centers one at a time; therefore, data centers try to get as much bandwidth as possible onto each fiber (this is also a major theme in the industry). You might ask, why not own your own fiber? You need to have a lot of content to own your own fiber. The cost is prohibitive. In order to make the fiber network function, all the nodes need to use the same specification and this is hard. Therefore, carriers are usually the ones to install the full infrastructure.

Article Source: John Houghton, a Silicon Valley entrepreneur, technology innovator, and head of MobileCast Media.