False positives with visual analysis for nonconcurrent multiple baseline designs and ABAB designs: Preliminary findings

• We evaluated false positives with three-tier nonconcurrent multiple baseline (NMBL) and ABAB designs with simulated data.
• False positives were generated for 7.5% of the NMBL graphs and 0% of the ABAB graphs.
• Current rules for the use of NMBL and ABAB designs may need only minor adjustments to minimize false positives.

This study evaluated the probability of generating false positives with three-tier nonconcurrent multiple baseline (NMBL) designs and ABAB designs. For Experiment 1, we generated four sets of three-tier NMBL design graphs. The first, second, and third sets consisted of fixed A-phase data points for all three tiers at 0%, 25% and 50%, respectively, and randomly generated data points in the B phases. The fourth set consisted of randomly generated data points in the A and B phases for all three tiers. Across all four sets (N = 1000), results show that false positives were produced with 7.5% of three-tier NMBL design graphs and were most probable when baseline levels were set at 0% or 25%. For Experiment 2, we generated 3000 ABAB design graphs consisting of three to five data points per phase. Results indicate that no false positives were produced, regardless of the number of data points included in each phase. Results of this study support specific guidelines for the use of NMBL designs and ABAB designs.