Imperfect narrow filtering in optical links with phase modulation

In order to reveal the pattern effect in the optical signal transmission it is studied as a random complex process. The different sources of detection error are studied for quadrature phase-shift keying in the absence of nonlinearity: the error in the rectangular filter width, the finite duration of the initial pulses, the deviation of detection point from the bit interval center. The dispersion and diameter of cloud in the constellation diagram are calculated and shown to be less for longer initial pulses. The error of imperfect optical system is proved to be important at a noise level of 11 dB and more. The result is also applicable for 8-PSK, 16-PSK and higher formats.