Consequences of misspecifying the number of latent treatment attendance classes in modeling group membership turnover within ecologically valid behavioral treatment trials

Historically, difficulties in analyzing treatment outcome data from open-enrollment groups have led to their avoidance in use in federally funded treatment trials despite the fact that 79% of treatment programs use open-enrollment groups. Recently, latent class pattern mixture models (LCPMM) have shown promise as a defensible approach for making overall (and attendance-class-specific) inferences from open-enrollment groups with membership turnover. We present a statistical simulation study comparing LCPMMs to longitudinal growth models (LGM) to understand when both frameworks are likely to produce conflicting inferences concerning overall treatment efficacy. LCPMMs performed well under all conditions examined; meanwhile, LGMs produced problematic levels of bias and Type I errors under two joint conditions: moderate to high dropout (30%–50%) and treatment by attendance class interactions exceeding Cohen's d ≈ .2. This study highlights key concerns about using LGM for open-enrollment data: treatment effect overestimation and advocacy for treatments that may be ineffective in reality.