On the second fundamental theorem of invariant theory for the orthosymplectic supergroup

Let OSp(V) be the orthosymplectic supergroup on an orthosymplectic vector superspace V of superdimension (m|2n). Lehrer and Zhang showed that there is a surjective algebra homomorphism Frr:Br(m−2n)→EndOSp(V)(V⊗r), where Br(m−2n) is the Brauer algebra of degree r with parameter m−2n. The second fundamental theorem of invariant theory in this setting seeks to describe the kernel KerFrr of Frr as a 2-sided ideal of Br(m−2n). In this paper, we show that KerFrr≠0 if and only if r≥rc:=(m+1)(n+1), and give a basis and a dimension formula for KerFrr. We show that KerFrr as a 2-sided ideal of Br(m−2n) is generated by KerFrcrc for any r≥rc, and we provide an explicit set of generators for KerFrcrc. These generators coincide in the classical case with those obtained in recent papers of Lehrer and Zhang on the second fundamental theorem of invariant theory for the orthogonal and symplectic groups. As an application we obtain the necessary and sufficient conditions for the endomorphism algebra Endosp(V)(V⊗r) over the orthosymplectic Lie superalgebra osp(V) to be isomorphic to Br(m−2n).