Optimized linear phase square-root Nyquist FIR filters for CDMA IS-95 and UMTS standards

In this paper, a novel method to design linear phase square-root Nyquist filters is introduced. The design procedure is posed as a quadratic programming problem with linear and quadratic constraints that can be solved using convex optimization packages. The flatness of the filter's pass-band, i.e., the pass-band ripple energy, is formulated as a quadratic function of the impulse response of the filter. By minimizing this quadratic function through a proxy function called the pseudo-ISI, one can optimize the ISI energy of the filter while the frequency response of the filter satisfies a spectral mask. It is shown that the optimization of the pseudo-ISI results in an optimized ISI energy if the spectral mask is tight enough. The Qualcomm's Code Division Multiple Access (CDMA) IS-95 and The European Telecommunications Standards Institute'a Universal Mobile Communications Systems (UMTS) standards are used to define the spectral masks. The designed filters show superior inter-symbol interference and stop-band energies compared to the previously proposed designs for CDMA and UMTS standards.

► A method to design square-root Nyquist linear-phase FIR filters is introduced.
► A proxy function (pseudo-ISI) is introduced to optimize the ISI energy.
► The optimization is posed as a quadratic problem with linear and quadratic constraints.
► The linear constraints represent the spectral masks.
► Square-root Nyquist filters for the CDMA IS-95 and UMTS standards are designed.