Virtual Success: Examining the Effectiveness of the North Carolina Virtual Public School

Abstract

EXECUTIVE SUMMARY Although virtual education options have rapidly expanded in recent years, little academic research has examined the effectiveness of these courses. My analysis examines the North Carolina Virtual Public School (NCVPS) and specifically uses the state Algebra I and English I end-of-course (EOC) tests to compare the academic outcomes of students in a statewide, entirely virtual classroom to those in a traditional brick-and-mortar setting. I also use a survey of NCVPS students to examine how students experience the virtual system and why they selected that option. I find that students use NCVPS for two distinct reasons. On one hand, certain students use the course as an opportunity to take an overload of courses or to pursue a course not offered in their local school. These students typically live in rural areas, but they are typically not poor and have a history of academic success. A majority of Algebra I NCVPS students fall into this category, because many rural middle schools use NCVPS to offer a high school course they would otherwise be unable to provide. Such students are likely to do well no matter which system they use. On the other hand, certain students use NCVPS as an accommodation, often to keep them on track for graduation. These students often live in urban area and are typically economically disadvantaged with a history of low academic performance. Such students are likely to struggle either virtually or in a traditional classroom. Not surprisingly, my survey results indicate that these students are often very worried about the lack of guidance and support in NCVPS. A complicated picture emerges as I examine NCVPS student survey responses. What certain students see as a benefit, others see as a drawback. Some students love the freedom of the virtual environment while others want more structure. The particular opinions depend on students’ past experience, familiarity with the system, and comfort level with self-directed work. Policymakers need to balance student preferences with the benefits of the virtual classroom. Determining the effectiveness of NCVPS is not as easy as comparing the average results of students using each system. For example, students using NCVPS for Algebra I in ninth grade have substantially lower scores on the Algebra I EOC test than their traditional peers. However, these NCVPS students’ math scores had been on a downward trajectory in the years before they began their virtual Algebra I course, while traditional students’ scores were fairly stable over time. Such differences are interesting as a descriptive exercise in that they illuminate that certain NCVPS students would need more academic support in either the virtual or traditional system. One cannot draw causal conclusions from such differences, however. They probably have more to do with the characteristics of students who choose the virtual option than with the effectiveness of NCVPS. It’s not clear that NCVPS caused the difference in scores. To draw any conclusions, I need a counterfactual: a comparison of what did happen (say, in NCVPS) to what would have happened if the student had been placed in a traditional classroom. Thus, I focus my main analysis on middle schools, several of which only offer NCVPS Algebra I and several of which only offer traditional Algebra I. Removing an element of choice allows a more causal conclusion. I find that eighth grade students perform about as well in Algebra I NCVPS as they do in the traditional classroom. Sixth and seventh grade students, however, struggle in the virtual system relative to comparable peers. These results are consistent across a variety of models, including simple OLS regression, propensity score matching, and a panel approach. My research supports several implications for NCVPS managers, school and district administrators, and state policymakers. 1. Rural middle schools without the capacity to offer a traditional Algebra I course should allow advanced eighth grade students to take the course through NCVPS. My research consistently demonstrates that virtual eighth grades students perform about the same on the EOC test as similar traditional students. Although more research is needed, this finding may also extend to advanced students in other subject areas. On a related note, schools should also allow advanced students to use the virtual option to take an overload of courses if they are deemed capable of handling the additional work. 2. NCVPS does not work for all students, and certain students need special support. My research consistently finds that young students using NCVPS as an opportunity in sixth or seventh grade fare poorly in Algebra I NCVPS relative to similar traditional students. Some students may not be ready academically or developmentally to use a self-directed virtual program. In these cases, young students may benefit from waiting until eighth grade before pursuing NCVPS. Additionally, many students that pursue NCVPS as an accommodation have a history of academic struggles, meaning they likely need an entirely different support system than advanced, high-achieving eighth graders. NCVPS should consider this balance as they assign students to classrooms, and more research should focus on what particular methods and means of support work best for at-risk students in the virtual environment. 3. School and state policymakers need to stay up-to-date with technology needs and advances. Schools should employ better screening to ensure that students have access to the technology they need to succeed in NCVPS. Over 10% of NCVPS students reported having no computer at home. This limitation likely affects less affluent students disproportionately, and these students are not set up for success if they cannot access course content at home. Additionally, student survey responses indicate that many students prefer the support, guidance and physical presence of a traditional classroom. NCVPS could do more to support students’ learning needs. As technology moves forward, NCVPS should learn from best practices of other systems. For instance, perhaps NCVPS could supplement its PowerPoint slides with standardized video lectures that can be used by all teachers year-after-year. A video of a teacher manually working through a problem or topic may be more beneficial than a series of slides.

Overall, North Carolina has developed one of the leading virtual systems in the country. It seems unlikely that virtual education will go away any time soon, and the rapidly expanding interest in virtual options mean that other states will look to other programs as models. North Carolina’s program works well for advanced eighth grade Algebra I students, although it does not work for all students. By focusing on targeted growth and meeting student needs, NCVPS can offer North Carolina students an exceptional program that other states could model.