A new 16-ary modulation for super-Nyquist-WDM systems: Dual-polarized quadrature duoquaternary (DP-QDQ) modulation

• A new 16-ary modulation format called quadrature duoquaternary (QDQ) is proposed.
• DP-QDQ can realize super-Nyquist wavelength-division-multiplexing (WDM) transmission.
• DP-QDQ is investigated in 32-GBaud super-Nyquist-WDM systems by simulations.
• DP-QDQ has a high spectral efficiency and a good dispersion tolerance.
• DP-QDQ has a lower sensitivity to the sampling time error.

A partial-response-pulse-shaped 16-ary quadrature amplitude modulation (16QAM) format called quadrature duoquaternary (QDQ) modulation, which can achieve higher spectral efficiency than Nyquist-pulse-shaped 16QAM and realize super-Nyquist wavelength-division-multiplexing (WDM) transmission, is proposed. The dual-polarized QDQ (DP-QDQ) modulation principle and coherent reception based on digital signal processing (DSP) are presented. The performance of the DP-QDQ scheme is investigated in 32-GBaud super-Nyquist-WDM systems by simulation. The simulation results show that DP-QDQ has only a 1.3 dB optical-signal-to-noise-ratio (OSNR) penalty for the 28-GHz-spaced 5-channel super-Nyquist-WDM case relative to the single-channel case. Compared with Nyquist-pulse-shaped 16QAM, DP-QDQ not only has a higher spectral efficiency, but also a lower sensitivity to sampling time error and a better dispersion tolerance. The 28-GHz-spaced 5-channel super-Nyquist-WDM DP-QDQ system can successfully implement 1520-km transmission at the forward-error-correction (FEC) bit-error-rate (BER) requirements.