Optical entangled fractional Fourier transform derived via non-unitary SU(2) bosonic operator realization and its convolution theorem

► We correctly construct two mutual conjugate 3-mode entangled states. ► We introduce a new non-unitary bosonic operator realization. ► We invent a new 3-mode entangled fractional Fourier transform (EFrFT). ► We derive the eigenmodes of EFrFT. ► The convolution theorem for EFrFT is discussed.