The influences of a two-tier test strategy on student learning: A lag sequential analysis approach

• Two-tier tests can enhance students' programming skills.
• Two-tier tests change students' learning strategies.
• Two-tier tests do not change students' intention.
• A lag sequential analysis is a useful approach.

Recently, programming skills have become a core competence. Many teaching strategies were developed to improve programming skills. Among them, online tests were widely applied to enhance students learning. Nonetheless, they may not be able to engage students in deep thinking and reflections. Thus, a two-tier test strategy was proposed to address this issue. However, previous research mainly focused on investigating the effectiveness of the two-tier test strategy but there is a lack of studies that investigate why the two-tier test approach is effective. To this end, we developed an online test, where the two-tier test strategy was implemented. Additionally, an empirical study was conducted to explore the influences of the two-tier test approach on students' learning performance and behavior patterns. Pre-test and post-test scores were applied to assess students' learning performance while a lag sequential analysis was used to analyze behavior patterns. Regarding learning performance, the proposed two-tier test can improve students' programming skills. Regarding behavior patterns, the two-tier test approach facilitates students to develop a learning by reviewing strategy, which is useful to improve their programming skills.