A tensor product finite element method for the diffraction grating problem with transparent boundary conditions

We consider the diffraction grating problem in optics, which has been modeled by a boundary value problem governed by a Helmholtz equation with transparent boundary conditions. A tensor-product finite element method is proposed to numerically solve the problem. An FFT-based matrix decomposition algorithm is developed to solve the linear system arising in the vertically layered medium case, which can be used as a preconditioning technique for the general case. Numerical examples are presented to illustrate the accuracy and efficiency of the method.