"All businesses, and certainly institutions of higher education as a subset, should be concerned with the perceived satisfaction of the product or service they provide. Studies that reveal insight into the thoughts of consumers assist in the identification of untargeted markets (Parasuraman, Berry, & Zeitaml, 1990) and are endeavors of commitment to continuing improvement (Hom, 2002) and scholarly understanding. Additionally, studies of this nature provide institutions with a foundation for taking proactive measures concerning the issues unique to their business environment (Tamblyn & Shelton, 1996). This comparative study of student satisfaction, as a supplemental step toward gaining a glimpse of current consumer satisfaction, serves to deliver a critical portion of that foundational element for Kennesaw State University.

Reviews of contemporary literature concentrating on the comparison of students participating in distance-learning (DL) to those engaged in traditional or face-to-face (FTF) classes tend to focus on student satisfaction, student learning/knowledge, and student participation/interaction (Allen, Bouhris, Burrell, & Mabry, 2002; Merisotis & Phipps, 1999; Helford & Lei, 1999). One such study incorporates teacher satisfaction as an additional element in the same effort (Rivera, McAlister, & Rice, 2002). Research indicates that studies attempting to investigate all three of these outcomes simultaneously include quantities of variables that detract from clear conclusions and invite criticism of the experimental design (Merisotis & Phipps, 1999).

As both a logical step toward increasing best practices at Kennesaw State University (KSU), and as an attempt to remove portions of traditional experimental complication, this study narrows the analysis scope solely to the measurement of student satisfaction using the hypothesis:

There are no differences in student satisfaction between online/distance learning courses and non-online/traditional courses.

Further review of relevant studies, such as those mentioned earlier, instruct that when all three outcomes are factored into a single statistical measure, the yield is The No Significant Difference Phenomenon advanced and popularized by Thomas Russell of North Carolina State University (Russell, n.d.). However, when student satisfaction is permitted to stand as its own statistical measure, independent from learning, participation, and teacher satisfaction, noticeably mixed results are observed. While some studies produce “no significant difference” (Allen, et al., 2002), others list student satisfaction in DL classes higher than that of their FTF counterparts (Finlay, Desmet, & Evans, 2004; MacFarland, 2001), and in remaining works, FTF classes enjoy a favorable rating over DL classes (Rivera, McAlister, & Rice, 2002; Helford & Lei, 1999; Vamosi, Pierce, & Slotkin, 2004). Further complicating matters is a study published in 2001 by Wisan, Nazma, & Pscherer establishing a graduating scale of increasing student satisfaction based upon greater numbers of DL courses taken.

Accepting the outward appearance of the conclusions established by these studies should be conducted with cautious skepticism. Quality of research methods and widely differing focus and technique make comparing studies and their relative conclusions a difficult, if not impossible, scholarly exercise (Merisotis & Phipps, 1999).

Aggregately, the only definitive conclusion that may be drawn from the body of currently accessible research is that lack of continuity in both study method and purpose shield investigators from viewing a clear picture of this field. A meta-analysis of over 450 manuscripts conducted by Allen, Bouhris, Burrell, & Mabry in 2002 attempted to overcome some of these methodical shortcomings. The conclusions of this comprehensive study seem to have generally conceded a slightly higher student satisfaction rating toward FTF while more broadly proving The No Significant Difference Phenomenon. Increasingly affordable accessibility to internet technologies combined with unparalleled convenience for users combine to project potentially attractive markets for universities throughout the world. The long term viability of these markets may well hinge on the perceived satisfaction of the relative consumer base. Thus, the need for a standardized, broadly applied test to gain clearer resolution of this field is critically important. The outcome of such a study would yield important information that would provide strategic insight into course designs, financial viability of programs, capital investment, satisfaction of student services and other indicators of institutional success."

REFERENCES

Allen, M., Bourhis, J., Burrell, N., & Mabry, E. (2002). Comparing student satisfaction with distance education to traditional classrooms in higher education: A meta-analysis. American Journal of Distance Education, 16, 83-98.

Finlay, W., Desmet, C., & Evans, L. (2004). Is it the technology or the teacher? A comparison of online and traditional English composition classes. Journal of Educational Computing Research, 31, 163-181.

Helford, P.Q., & Lei, R.M. (1999). Using the Web to deliver and enhance classes: Two case studies. Retrieved on July 21, 2005, from the Helsinki University of Technology Web site: http://www.hut.fi/events/eunis99/TrackA.html

Hom, W.C. (September, 2002). Applying customer satisfaction theory to community college planning of counseling services [electronic version]. Retrieved on July 27, 2005, from the iJournal Web site: http://www.ijournal.us

MacFarland, T.W. (2001). Fall term 1999 Nova Southeastern University students respond to a broad-based satisfaction survey: A comparison of campus-based students and distance education students. Nova Southeastern University, Fort Lauderdale, FL.

Merisotis, J., & Phipps, R.A. (1999). What’s the difference? Outcomes of distance vs. traditional classroom-based learning. Change, 31, 12-17.

Parasuraman, A., Berry, L.L., & Zeithaml, V.A. (1990). Guidelines for conducting service quality research. Marketing Research, 2, 34-45.

Rivera, J.C., McAlister, M.K., & Rice, M.L. (2002). A comparison of student outcomes & satisfaction between traditional & web based course offerings. Retrieved on July 21, 2005, from the Online Journal of Distance Learning Administration Web site: http://www.westga.edu/%7Edistance/ojdla/fall53/fall53.html

Russell, T.R., (n.d.). The no significant difference phonemenon. Retrieved on July 23, 2005, from the International Distance Education Certification Center Web site: http://nosignificantdifference.org/

Tamblyn, A., & Shelton, D. (1996). Market research manual for providers of vocational education and training. Retrieved on July 27, 2005, from the ERIC Web site: http://web11.epnet.com?DeliveryPringSave.asp?tb=1&_ug=sid+ 09C7B34A-CA8D-4008-939F-FA48BF0

Vamosi, A.R., Pierce, B.G., & Slotkin, M.H. (2004). Distance learning in an accounting principles course: Student satisfaction and perceptions of efficacy. Journal of Education for Business, 79, 360-367.

Wisan, G., Nazma, S., & Pscherer, C.P. (2001, June). Comparing online and face-to-face instruction at a large virtual university: Data and issues in the measurement of quality. Paper presented at the 41 st forum of the Association for Institutional Research, Long Beach, CA.

BIBLIOGRAPHY

Abel. R. (2005). Achieving success in internet-supported learning in higher education: Case studies illuminate success factors, challenges, and future directions [electronic version]. Retrieved on July 18, 2005, from the Alliance for Higher Education Competitiveness Web site: http://www.a-hec.org

Faul, A.C., Frey, A.J., & Barber, R. (2004). The effects of Web-assisted instruction in a social work research methods course. Social Work Education, 23, 105-118.

Newlin, M.H., Lavooy, M.J., & Wang, A.Y. (2005). An experimental comparison of conventional and Web-based instructional formats. North American Journal of Psychology, 7, 327-336.

Stein, D. (2004, February 1). Course structure: Most important factor in student satisfaction. Distance Education Report, 8, 4.

Wiesenberg, F., & Massey-Hicks, R. (2005, February). Multimedia course design, instructor preparation increase satisfaction. Online Classroom, 1-3.

Statistically Based Conclusions and Conditions  

Respondents Survey Data Rev. 2:

• 187 total responding participants
• 11 excluded responding participants = 5.9%
• 8 incomplete information = 4.3%
• 2 underage = 1.1%
• 1 disagree with survey premise = .5%
• 176 total qualifying responding participants = 94.1%
• 137 Online responding participants = 77.8% of total qualifying responding participants
• 39 Classroom responding participants = 22.2% of total qualifying responding participants
• 3.5 : 1 Online to Classroom responses

Respondents Survey Data Rev. 3:

• 187 total responding participants
• 63 excluded responding participants = 33.7%
• 52 Online responding participants with zero Classroom counterparts = 27.8%
• 8 incomplete information = 4.3%
• 2 underage = 1.1%
• 1 disagree with survey premise = .5%
• 124 total qualifying responding participants = 66.3%%
• 85 Online responding participants = 68.5% of total qualifying responding participants
• 39 Classroom responding participants = 31.5% of total qualifying responding participants
• 2.2 : 1 Online to Classroom responses

Conclusions/Inferences Based on Population Sizes and Distributions:

• Given the number of sections targeted for this study, the overall data pool is smaller than anticipated
• Zero response rates from CRJU 1101 and POLS 3334 sections suggest:
  • More assistance may be required with the instructors of these sections in technical assimilation
  • More incentive or justification may need to be provided to the instructors of these sections
  • 39 Classroom responding participants is dangerously close to the coveted “30 minimum” usually considered desirable for similar tests
  • Ideally, future studies would strive for a more balanced response rate between Online and Classroom sections. In our test, responses are aggregated such that a 3.5: 1 ratio, although lopsided, does not hurt our conclusions. But, these conclusions could be bolstered with a more even distribution.

Conclusions/Inferences Based on Likert Scale Data from Survey Data Rev. 2:

• Only question #14 displayed a higher rated response average in the Classroom section than the Online
• 12 of the 14 questions presented results that caused support of the null hypothesis = 85.7%
• 2 of the 14 questions presented results that caused the rejection of the null hypothesis = 14.3%
• Question #6 (I found this class academically challenging) presented results that show an average response of 4.759 for Online participants compared to 4.308 for Classroom participants.

This result states that more Online students found this class to be academically challenging than did Classroom students

The variance of .451 represents the most statistically significant variance in the study

Likert Scale equivalents for this question:

4 = somewhat agree

5 = agree

Question #10 (The syllabus for this course was clearly written and easily understood) presented results that show an average response of 5.154 for Online participants compared to 5.496 for Classroom participants.

While a variance of .342 exists between the two test groups, it is important to note the level of polarity in response groups. Both groups are responding near the very top of the established scale.

Noting that the aggregate numerically converted Likert Scale range for this study is 39-234 (strongly disagree to strongly agree) for the Classroom pool:

• Question #6 (I found this class academically challenging) presents the lowest response with a total converted rating of 168 and a non-converted average of 4.308.
• Question #10 (The syllabus for this course was clearly written and easily understood) presents the highest response with a total converted rating of 201 and a non-converted average of 5.154
• The total range of all converted ratings is 168 to 201 for a distance of 33.
• All questions in this survey had total converted ratings that fell within a 16.9% range of the total available scale range.
• The total of all converted ratings for all questions = 2564 for a total survey average of 4.696 for all Classroom respondents

Noting that the aggregate numerically converted Likert Scale range for this study is 137-822 (strongly disagree to strongly agree) for the Online pool:

• Question #13 (I worked harder in this course than I anticipated) and Question #16 (I will recommend this course to others) present the lowest response with a total converted rating of 641 and a non-converted average of 4.679.
• Question #10 (The syllabus for this course was clearly written and easily understood) presents the highest response with a total converted rating of 753 and a non-converted average of 5.496
• The total range of all converted ratings is 641 to 753 for a distance of 112.
• All questions in this survey had total converted ratings that fell within a 16.4% range of the total available scale range.
• The total of all converted ratings for all questions = 9364 for a total survey average of 4.882 for all Online respondents

Conclusions/Inferences Based on Likert Scale Data from Survey Data Rev. 3:

• Only question #14 displayed a higher rated response average in the Classroom section than the Online
• 13 of the 14 questions presented results that caused support of the null hypothesis = 92.9%
• 1 of the 14 questions presented results that caused the rejection of the null hypothesis = 7.1%

Question #6 (I found this class academically challenging) presented results that show an average response of 4.835 for Online participants compared to 4.308 for Classroom participants.

This result states that more Online students found this class to be academically challenging than did Classroom students

The variance of .527 represents the most statistically significant variance in the study

Likert Scale equivalents for this question:

4 = somewhat agree

5 = agree

Noting that the aggregate numerically converted Likert Scale range for this study is 39-234 (strongly disagree to strongly agree) for the Classroom pool:

• Question #6 (I found this class academically challenging) presents the lowest response with a total converted rating of 168 and a non-converted average of 4.308.
• Question #10 (The syllabus for this course was clearly written and easily understood) presents the highest response with a total converted rating of 201 and a non-converted average of 5.154
• The total range of all converted ratings is 168 to 201 for a distance of 33.
• All questions in this survey had total converted ratings that fell within a 16.9% range of the total available scale range.
• The total of all converted ratings for all questions = 2564 for a total survey average of 4.696 for all Classroom respondents

Noting that the aggregate numerically converted Likert Scale range for this study is 85-510 (strongly disagree to strongly agree) for the Online pool:

• Question #16 (I will recommend this course to others) presents the lowest response with a total converted rating of 394 and a non-converted average of 4.635.
• Question #10 (The syllabus for this course was clearly written and easily understood) presents the highest response with a total converted rating of 462 and a non-converted average of 5.435
• The total range of all converted ratings is 394 to 462 for a distance of 68.
• All questions in this survey had total converted ratings that fell within a 16.0% range of the total available scale range.
• The total of all converted ratings for all questions = 5866 for a total survey average of 4.929 for all Online respondents

General Comments

The results of this study seem to support the “No Significant Difference Phenomenon” with one unique wrinkle. Similar studies conducted on this topic, while yielding little or no statistically significant difference; do traditionally yield a slight realistic difference in favor of Classroom sections. Our study upholds the traditional statistical trend of no significant difference, but with a slightly higher realistic difference in favor of Online sections.

While this study is prepared to concede several limitations in its design, it may be considered a significant addition to the existent body of research on this topic.