Program-level measurements

Continuing our exploration of measures of success in online or blended settings, an important item to add to course-specific factors (identified in my previous post) that reflects program-level success is program completion or graduation rate. Conceptions of student success will vary depending on educational level – K-12, community college, 4-year College, and post-graduate degree – as well as credentialing requirements, but completion of the courses required for graduation is a critical measure of program success.

Concerns about students’ persistence in online or blended programs surfaced shortly after institutions started offering these programs. Rovai (2003) explored research on this phenomenon and described a composite model to explain persistence and attrition in online courses and programs.   

A definition of retention that applies to online or blended programs comes from Boston, Ice and Gibson (2011): “the progressive reenrollment in college, whether continuous from one term to the next or temporarily interrupted and then resumed” (¶ 38). Students in online programs may not complete courses each term, based on lack of resources, changes in their profession, personal or professional commitments, etc., but should eventually complete their degree. 

A broader conception of online program success from Kuh et al. (2006) includes the following factors: “academic achievement, engagement in educationally purposeful activities, satisfaction, acquisition of desired knowledge, skills, and competencies, persistence, attainment of educational objectives, and post college performance” (p. 7). This definition reflects a more comprehensive view of student experiences and expectations in online or blended degree programs. 

A popular measurement of student satisfaction in higher education institutions are student course evaluations (SCE). An analysis of 2-years of SCE data comparing our students’ perceptions of graduate course quality resulted in NO statistically significant differences based on course format – hybrid vs online. 

As part of our evaluation of a blended and online graduate degree in educational technology, we analyzed student success, by format, along with GPA and retention, where retention includes the number of students who withdrew after the term started AND those who received a failing grade in the course. For the 2011-2014 academic years, our graduate courses had retention rates of 98.8% in online courses and 97% in hybrid courses. We are still analyzing measures of student program success and will share that information in a future posting.

Other measurements of student success in online programs include interactions and a sense of community. Interactions are often cited in the research as linked with student satisfaction in online educational experiences and we will explore conceptions of interactions in future postings. Exter et al. (2009) report on a study that measured students’ program-level sense of community and its possible link with overall success. A challenge for those implementing and supporting online programs is how to measure students’ sense of community at the program level and actions that can be taken to improve this affective element of students’ experience.  

Knowing how many students complete an online program and graduate is an important indicator of success. Identifying factors contributed to students’ failure to graduate is equally critical, especially if our goal is to improve graduation rates. This becomes especially salient as the marketplace for online courses and programs grows to include non-traditional educational institutions and stakeholders select programs based on their individual criteria. 

Andy

References

Boston, W.E., Ice, P., & Gibson, A.M. (2011, spring). Comprehensive assessment of student retention in online learning environments. Online Journal of Distance Learning Administration, 14(1). Retrieved from: http://www.westga.edu/~distance/ojdla/spring141/index.php

Exter, M.E., Korkmaz, N., Harlin, N.M., & Bichelmeyer, B.A. (2009). Sense of community within a fully online program: Perspectives of graduate students. The Quarterly Review of Distance Education, 10(2), 177-194.

Kuh, G.D., Kinzie, J., Buckley, J.A., Bridges, B.K., & Hayek, J.C. (2006). What matters to student success: A review of the literature. Commissioned Report for the National Symposium on Postsecondary Student Success: Spearheading a Dialog on Student Success. Available online: http://nces.ed.gov/npec/pdf/kuh_team_report.pdf

Rovai, A.P. (2002). In search of higher persistence rates in distance education online programs. The Internet and Higher Education, 6, 1-16.