On the Rayleigh-Taylor instability in compressible viscoelastic fluids

We mathematically investigate the Rayleigh-Taylor (abbr. RT) instability in the compressible viscoelastic fluid in the presence of a uniform gravitational field in a bounded domain based on Oldroyd-B model. We first analyze the linearized equations around the viscoelastic RT equilibrium solution, and obtain an instability condition. Then we construct solutions of the linearized viscoelastic RT problem that grow in time in the Sobolev space H3 under an instability condition, thus leading to the linear instability. Finally, with the help of the constructed unstable solutions of the linearized viscoelastic RT problem and a local well-posedness result of smooth solutions to the nonlinear viscoelastic RT problem, we mathematically prove the instability of viscoelastic RT (abbr. VRT) problem in the sense of Hadamard.