THE USE OF A COURSE MANAGEMENT SYSTEM IN THE LIGHT OF THE TECHNOLOGY ACCEPTANCE MODEL: A STUDENT PERSPECTIVE

Cheng-Chang Sam Pan
University of Central Florida, Orlando, FL, USA
Stephen Sivo
University of Central Florida, Orlando, FL, USA
James Brophy
University of Central Florida, Orlando, FL, USA

Abstract

The pilot study focuses on five latent factors affecting students’ use of WebCT in a Web-enhanced hybrid undergraduate course at a southeastern university in the United States. An online questionnaire is used to measure two exogenous variables (i.e., subjective norm and computer self-efficacy), three endogenous variables (i.e., perceived ease of use, perceived usefulness, and attitude toward WebCT use), one dependent variable (i.e., actual system use), and eleven demographic items. PROC CALIS is used to analyze the data collected. Results suggest the technology acceptance model may not be applicable to a higher education setting. However, student attitude toward WebCT instruction remains a significant determinant to WebCT use on a non-voluntary basis. Educational achievement (i.e., student final grades) also regressed on the attitude factor as an outcome variable. Suggestions for practitioners and researchers in the field are mentioned.

Introduction

The intense competition among institutions of higher education in terms of student recruitment has encouraged universities to adopt e-learning systems (e.g. WebCT and Blackboard) in order to reach a larger population at great distances (Gary, 2002). For a relatively young metropolitan university, the University of Central Florida (UCF) has experienced a rapid increase in student enrollment. Consequently, significant resources have been invested in the adoption of WebCT at UCF. The intent of this adoption is to offer an alternative educational medium and a nontraditional paradigm to tailor and customize instruction and to better suit differing types of UCF students with varied learning backgrounds. This study concentrates on factors affecting WebCT use from a student perspective, in best hope to represent the causality among the factors. Insights are provided for practitioners and researchers in the causal relationship between student attitude and behavior and interrelatedness among those contributing factors in terms of WebCT use, so as to allow for student success in online courses.

Review of Literature

Determinants of Attitude
One of the fundamental motivation models from the literature is Fishbein and Ajzen’s (1975) theory of reasoned action. The theory specified a causal relationship between individual behavioral intention and actual behavior. With TRA, one can differentiate an individual’s actual behavior from his or her behavioral intention. Behavioral intention is a latent factor that is measured by two other latent factors: attitude toward behavior and subjective norm.

Introduction of the TAM
Rooted in TRA, the technology acceptance model (TAM) by Davis (1989) identifies the following two distinct constructs: perceived usefulness and perceived ease of use. These two constructs directly affect a person’s attitude toward the target system use and indirectly affect actual system use (Davis, 1993). Davis (1993) defined perceived usefulness as “the degree to which an individual believes that using a particular system would enhance his or her job performance” and perceived ease of use as “the degree to which an individual believes that using a particular system would be free of physical and mental effort” (p. 477). Furthermore, attitude toward use of a system is defined as “the degree to which an individual evaluates and associates the target system with his or her job” (p. 476). Actual system use is a behavioral response, measured by the individual’s action in real life. David (1993) states that “frequency of use and amount of time spent using a target system are typical of the usage metrics” (p. 480).

The TAM is employed by Management Information Systems (MIS) practitioners to predict the success or a failure of an information systems project. The assumptions on which the TAM is based comprise:

1. When end users perceive the target system as one that is easy to use and nearly free of mental effort, then they may have a favorable attitude toward using the system. Nevertheless, Sanders and McCormick (1993) argued that an individual must use some of or all of one’s mental resources in order to perform a task.
2. When end users perceive the system as one that is helpful to their job, then they may have a positive attitude toward the system used.
3. When users have a favorable attitude toward the target system, they may use the system frequently and intensely, which means that the system developed is successful.

Attitude-behavior Relationship
Attitude-behavior relationships have been discussed with respect to adoption of information systems projects (e.g., WebCT) and are considered vital in much of the previous research (Albarracin & Wyer, 2000; Bursey & Craig, 2000; Chiou, 2000; Costa, 1999; Fiore, Yah & Yoh, 2000; Robinson, 2001; Rutter, 2000). Intuitively, the voluntary behavior of individuals determines success in any information system. One of the most important factors which regulates end user behaviors (e.g., adoption or rejection of the system) is their attitude toward the system (Harris, 1999). This is also endorsed by Sankaran, Sankaran, and Bui (2000). Sankaran, Sankaran, and Bui argued that there is a positive correlation between a student’s attitude toward course format and his or her learning performance. In other words, students who favor Web-based courses tend to perform better in those courses than in the lecture courses, with other extraneous variables considered. Sanders and Morrison-Shetlar (2001) also found students whose attitudes toward Web-enhanced instruction can play a vital role in influencing the future use of Web course management system (i.e., WebCT).

Subjective Norm
Triandis (1994) defines norms as “[I]deas about what is correct behavior for members of a particular group” (p. 100). Subjective norm represents “perceived external pressures to use (or not use) the system” (Liker & Sindi, 1997, p. 152). It is two-fold: vertical pressure and horizontal pressure, as implied in a study by Anandarajan, Igbaria, and Anakwe (2000). Vertical pressure refers to the social pressure from people who are either superordinate or subordinate to the individual (i.e., a vertical dyads relationship); horizontal pressure refers to the social pressure from people closely related to the individual (e.g., close friends). There is more likelihood for those who report high subjective norm to accept and adopt the new system (Anandarajan, Igbaria, & Anakwe, 2000; Liker & Sindi, 1997).

Computer Self-efficacy
Originally from Bandura’s (1977) self-efficacy theory, computer self-efficacy becomes a pivotal issue in technology acceptance. Venkatesh and Davis (1994) first coined the term computer self-efficacy, which is defined as “[The] self-efficacy…in the specific context of user acceptance of computer technology” (p. 214). Venekatesh and Davis verified that users’ perceived ease of use is strongly regressed on computer self-efficacy in the early stage of technology acceptance. Morris (2001, p. 882) also said “people who believe they are capable of using IT [Information Technology] to accomplish their tasks are more likely to use IT than those who do not share similar self-efficacy beliefs.” This is congruent with what Lee and Witta (2001) reported.

Although the TAM has been validated and re-tested since 1989, studies of the TAM on a non-voluntary basis are rarely conducted. Venkatesh (2000) advised that “Future research should examine mandatory usage contexts to test the boundary conditions of the proposed [technology acceptance] model” (p. 358). Moreover, Sanders & Morrison-Shetlar (2001) reported a complete view on student attitude through various lenses (students’ demographics), but they overlooked at a legitimate question of any educator’s interest. That is, what is the attitude-behavior relationship to do with student achievement? Researchers hope to address these issues in this paper.

Method

Sampling
From a Web-enhanced hybrid General Psychology course, 217 undergraduate students participated on a voluntary basis in this study, which yields a response rate of 48%. The majority of sample subjects are female (about 70%). Regarding academic level, 69.6% are freshmen, 19.8% sophomores, 8.3% juniors, 1.8% seniors, and .5% others. Concerning about ethnicity, 71% are Caucasians, 10.6% African Americans, 9.2% Hispanics, 7.8% Asian Americans, and about 1.5% others. 70% of the participants never take any course using WebCT. More than 90% of the subjects have significant experiences of using a computer for more than four years.

Data Collection
An online questionnaire with seven varied scales was administrated to the target population. The validated questionnaire comprises six scales plus eleven demographic questions. Each of the two factors (perceived usefulness and perceived ease of use) is measured with six question items, as suggested by Davis (1989). A five-item scale, proposed by Davis (1993), is adapted to measure student attitude toward WebCT use. To measure that latent computer self-efficacy factor, Lee (2002) suggested a scale with twenty-seven items, which yields five subscales. Lastly, a four-item scale, suggested by Wolski and Jackson (1999), is used to measure subjective norm. Presumably, students’ learning performance in the presence of WebCT is of any educator’s interest. Students’ final grades are collected on a five-point scale. Concerning actual WebCT use, two variables reported by Davis (1993) are used. They are intensity and frequency variables. Data are stored in a secured ColdFusion server maintained by the college researchers are affiliated with.

Measure
Although those items are adapted from the literature, researchers remain suspicious. PROC FACTOR is used to conduct an exploratory factor analysis on the twenty-nine items, including five composite variables for computer self-efficacy factor and students’ final grades (See Table 1).

Table 1
An Illustration of Rotated Factor Pattern (Standardized Regression Coefficients)

Note: Printed values are multiplied by 100 and rounded to the nearest integer. Values greater than .45 are flagged by an '*'.

Six variables that loaded on Factor 1 are pertinent to student perception of ease of use in WebCT, so it is named Perceived Ease of Use factor. Five variables loaded on Factor 2. They are related to students’ attitude toward WebCT use (i.e., attitude toward courses using WebCT). Factor 2 is then named Attitude toward WebCT factor. Five variables loaded on Factor 3. They are exactly corresponding to the factor previously proposed, so Factor 2 is labeled Computer Self-efficacy factor. Six variables that loaded on Factor 4 are regarding student perception of WebCT’s usefulness. Factor 4 is labeled Perceived Usefulness factor. Three variables (including T346 variable) that loaded on Factor 5 are associated with subjective norm, so Factor 5 is labeled Subjective Norm factor. It is noted that four variables concerning subjective norm factor were initially adapted from the literature. Researchers eliminated T348 variable and kept the remaining for further analysis. Although six factors are suggested, none of the variables that loaded on Factor 6 appears significant (r < .40). For the purpose of this study, researchers decided to keep the two variables (i.e., intensity and frequency) plus GD (i.e., student final grade) for further analysis.

To assess scale reliability with coefficient alpha, PROC CORR is utilized for first five factors. They are .97, .93, .92, .96, and .75 respectively.

Table 2 illustrates six factors and their coefficient alpha. T346 is retained in order to keep three indicators for each construct. Of researchers’ interest, Factor 6 is retained for further analysis, and it is labeled System use, which is made up of T349 (i.e., frequency), T350 (i.e., intensity), and GD (student final grades).

Table 2
An Illustration of Six Factors and Coefficient Alpha

Note: * denotes coefficient alpha is .79 when T346 is removed.

Design of the Study
This research is a structural equation modeling study with quantitative measurement. The causal pathways to be scrutinized are students’ perceived usefulness, perceived ease of use, attitude toward WebCT, actual use of WebCT, computer self-efficacy, and each individual’s subjective norm. Through examination of these factors it is anticipated the researchers be able to test the TAM using UCF student population, and then to extend the TAM by adding two extra factors: computer self-efficacy and subjective norm. Furthermore, they hope to determine predictive capability (i.e., predictabilities) of each of the five latent variables/factors with system use as an outcome variable.

Results

Three stages of the analysis are included. For the first stage of this study, the TAM reported in the literature was fitted to the data with the intention of ultimately developing aspects of the model as originally specified with other predictor variables and with an educational outcome as measured by GD variable. A diagram is provided to illustrate the model hypothesized. (See Figure 1.)

Figure 1. A diagram of the technology acceptance model hypothesized with students’ final grades treated as one outcome variable.

For the first stage of the analysis, the capability of the TAM to fit the data is assessed, followed by the inclusion of an educational outcome and then more predictors thought to participate in both an explanation of technology use and educational achievement. PROC CALIS is used to assess the model hypothesized. A review of the parameters estimated for the models suggest that the two outcome variables, frequency and intensity, do not share enough covariation (r= .086) to allow them both to serve as one common factor. The standard error for the intensity was far too wide (i.e., s > .1) to warrant its specification in the model. For this reason, intensity is removed from the model, and frequency is retained as the sole outcome variable. Moreover, collinearities between T36 and perceived usefulness, T311 and perceived ease of use, and T344 and attitude toward WebCT use occur, because each of the regression weights is found 1, which means it may be a one factor model for Factors 1, 2, and 4. The second stage is to pursue this critical issue.

The second stage of this study involves augmenting the last, revised TAM with another outcome variable to observe the capacity of the model to explain not only frequency, but also educational achievement, as measured by GD variable. In order to resolve the issue of co-linearities, each latent factor is extracted from the original TAM and carefully examined. A second hypothetical model is specified, where all latent factors are presumed to determine T349 (i.e., frequency) and GD (i.e., student final grades), and there is a covariance among each one of them.

Although co-linearities among variables are resolved, the results suggest that this model be hardly fitted to the data. With all five latent factors becoming exogenous and highly correlated between one another, only three paths are found significant: one from attitude toward WebCT use to GD, another from attitude toward WebCT use to T349, and the other from subjective norms to T349. To further explore this model, attitude toward WebCT use and subjective norms factors are retained for the third stage of the analysis. (See Figure 2.)

Figure 2. A model with attitude toward WebCT and subjective norms determining outcome variables.

With the three factors removed, the third stage of the analysis is focused on predictabilities of student attitude toward WebCT use and subjective norms to GD and T349 variables. (See Figure 3.) According to covariance structure analysis estimates, the model is deemed satisfactory. All the fit indices are greater than .9 (Hatcher, 1994). Goodness of Fit Index (GFI) is 0.9164. Bentler's Comparative Fit Index is 0.9472. Additionally, Bentler & Bonett's (1980) Non-normed Index is 0.9280. Both root mean square residual (RMR) and root mean square error of approximation (RMSEA) are less than .1. They are 0.0594. and 0.0992 respectively. The final model accounts for 82% of the total variance of T340, 65.3% of T341, 81.7% of T342, 68.2% of T343, 84.1% of T344, 85.9% of T345, 50.4% of T347, 32.4% of T346, 15% of T349, and 5.7% of GD.

Discussion

In the end, the present study was concerned with the determinants of two outcomes: the frequency of students using WebCT to accomplish class assignments and student grades at the end of the semester. A revision of the TAM model to consider both outcomes while incorporating Self-Efficacy and Subjective Norms unrelentingly delivered warnings of co-linearities grounded in the configuration of the correlations among the variables that were unresolvable by minor model modifications. Perceived ease of use, perceived usefulness, and attitude toward WebCT use appeared not to be determined by one another. Should the hypothetic model specify a causal path between one and another, co-linearity occured. Once those variables involved were removed, co-linearities remained on other variables. The theory-driven model based upon the TAM was pronounced failed. In the end, to posit a model that best explained the two primary effects of concern, all latent factors turned out to be superfluous except Subjective Norms and Student Attitude toward WebCT use. The final model divested of all but the most germane causes though much simpler in form was much more elegant as well. Albeit, all five latent factors were highly related when treated as exogenous, only the two factors mentioned were relevant in any way to the outcomes of interest.

Student attitude toward WebCT is the only variable that determines GD. Its coefficient matrix reports a weak, but significant, ability (????.2396), predicting students’ final grades. That means that about 5.74% of the total variance of student final grades variable is explained by the total model. The attitude construct also determines frequency of using WebCT (??? 0.2430). Subjective norms only determines frequency of using WebCT (??????????. Including the residual term, 14.96% of the total variance of frequency variable is explained. Obviously, the two exogenous variables are not able to entirely explain either GD or frequency variables. As previously implied, two factors specified in the TAM: perceived ease of use and perceived usefulness may not be able to exert their influences on students’ WebCT use in this setting. Neither is computer self-efficacy construct. Although researchers failed to provide appropriate outcome variables to measure system use, the results of the present study are deemed informative for practitioners and researchers in the field. More proper measures: decision support, work integration, and customer service need to be introduced for further explain WebCT use (Doll & Torkadeh, 1998).

Since attitude toward WebCT use and subjective norms determine GD and frequency of using WebCT, it is suggested that the class instructor may need to play a change agent to positively influence students’ attitude toward the WebCT. For instance, in a WebCT-enhanced hybrid course, the instructor and teaching assistants can provide activities to make students believe that the quality of WebCT instruction is comparable to that of the face-to-face instruction. Somehow the students also need to believe the attention they receive from the instructor in the virtual classroom is the same as that in the traditional classroom. Furthermore, since subjective norms can directly impact frequency of using WebCT, the instructor or TAs need to announce to the class that using the Web-related instruction is a requirement in the course (vertical influence) to get the message to come across. As implied in the final model, peer pressure (i.e., horizontal influence) matters. It is advised that each student group leader in the large size class comply with the course convention, defined by the instructor, and play a role model to other group members.

Cautions should be taken in applying these results. Since the subjects are sampled from the UCF student body, the preliminary results can merely apply to other student population in similar settings to UCF. Having said that, this study does shed some light on students’ cognitive and behavioral pattern with respect to WebCT use.

Further research needs to focus on development of system use scales to better measure actual system use. Doll and Torkzadeh (1998) claimed that either the amount of system use or the extent of the use can be problematic or questionable to view these measures as appropriate indicators of skill. In the present study, frequency and intensity were initially proposed as outcome variables of system use. What Doll and Torkzadeh implied may be the reason why one of the variables, intensity, ended up being removed, and that adversely affects the outcome variable scale. Due to the scope of the present study, subjects’ demographic categories are not addressed. Relationships between demographics and attitude toward WebCT instruction may further explain the actual system use in this higher education setting, as Sankaran, Sankaran, and Bui (2000) recommended. Additionally, Shaftel (2000) found that attitude can predict the behavior only in a limited timeframe and behavioral intention is not a good mediating factor in the attitude-behavior relationship. A latent change analysis with the Level and Shape (LS) model should be considered, using multiple questionnaire administrations at different times, using the same population (Raykov & Marcoulides, 2000).

References

Albarracin, D., & Wyer, R. S., Jr. (2000). The cognitive impact of past behavior: Influences on beliefs, attitudes, and future behavioral decisions. Journal of Personality & Social Psychology, 79(1), 5-22.

Anandarajan, M., Igbaria, M., & Anakwe, U. P. (2000). Technology acceptance in the banking industry: A perspective from a less developed country. Information Technology & People, 13(4), 298-312.

Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191-215.

Bursey, M., & Craig, D. (2000). Attitudes, subjective norm, perceived behavioral control, and intentions related to adult smoking cessation after coronary artery bypass graft surgery. Public Health Nursing, 17(6), 460-467.

Chiou, J. (2000). Antecedents and moderators of behavioral intention: Differences between U.S. and Taiwanese students. Genetic, Social, & General Psychology Monographs, 126(1), 105-124.

Costa, L. L. (1999). Association of hospitalized heart failure patients' intention to manage the therapeutic regimen with postdischarge health behaviors and outcomes. Unpublished doctoral dissertation, The Catholic University of America, Washington, D. C.

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-340.

Davis, F. D. (1993). User acceptance of information technology: System characteristics, user perceptions, and behavioral impacts. International Journal of Man Machine Studies, 38, 475-487.

Doll, W. J., & Torkzadeh, G. (1998). Developing a multidimensional measure of system-use in an organizational context. Information & Management, 33, 17-185.

Fiore, A. M., Yah, X., & Yoh, E. (2000). Effects of a product display and environmental fragrancing on approach responses and pleasurable experiences. Psychology & Marketing, 17(1), 27-54.

Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley.

Gary, S. (2002, August). A moving target: E-Learning vendors take aim in a changing environment. Syllabus, 16(1), 28-32.

Harris, R. W. (1999). Attitudes towards end-user computing: a structural equation model. Behaviour & Information Technology, 18(2), 109-125.

Hatcher, L. (1994). A step-by-step approach to using the SAS system for factor analysis and structural equation modeling. Cary, NC: SAS Institute Inc.

Lee, C. (2002). The impact of self-efficacy and task value on satisfaction and performance in a Web-based course. Unpublished doctoral dissertation, University of Central Florida, Florida.

Lee, C., & Witta, E. L. (2001). Online students’ perceived self-efficacy: Does it change? Paper presented at the Association for Education Communications and Technology (AECT) International Convention, Atlanta, GA.

Liker, J. K., & Sindi, A. A. (1997). User acceptance of expert systems: A test of the theory of reasoned action. Journal of Engineering and Technology management, 14, 147-173.

Morris, M. G., Turner, J. M. (2001). Assessing users’ subjective quality of experience with the world wide web: An exploratory examination of temporal changes in technology acceptance. International Journal of Human-Computer Studies, 54, 877-901.

Raykov, T., & Marcoulides, G. A. (2000). A first course in structural equation modeling. Mahwah, New Jersey: Lawrence Erlbaum Associates.

Robinson, L. (2001). Sales force use of technology: An extension of the technology acceptance model, including antecedents and outcomes relevant in professional selling organizations. Unpublished doctoral dissertation, University of South Florida, Florida.

Rutter, D. R. (2000). Attendance and reattendance for breast cancer screening: A prospective 3-year test of the Theory of Planned Behaviour. British Journal of Health Psychology, 5(Pt. 1), 1-13.

Sanders, D., & Morrision-Shetlar, A. I. (2001). Student attitudes toward Web-enhanced instruction in an introductory biology course. Journal of Research on Computing in Education, 33(3), 251-262.

Sanders, M. S., & McCormick, E. J. (1993). Human factors in engineering and design (7th ed.). New York: McGraw-Hill, Inc.

Sankaran, S. R., Sankaran, D., & Bui, T. X. (2000). Effect of student attitude to course format on learning performance: An empirical study in Web vs. lecture instruction. Journal of Instructional Psychology, 27(1), 66-73.

Shaftel, J. S. (2000). Attitude structure and change in the domain of study skills. Unpublished doctoral dissertation, University of Kansas, Kansas.

Triandis, H. C. (1994). Culture and social behavior. New York: McGraw-Hill.

Venkatesh, V. (2000). Determinants of perceived ease of use: Integrating control, intrinsic motivation, and emotion into the technology acceptance model. Information Systems Research, 11(4), 342-365.

Venkatesh, V., & Davis, F. D. (1994). Modeling the determinants of perceived ease of use. Proceedings of the 15th International Conference on Information Systems, Vancouver, British Columbia, Canada, 213-228.

Wolski, S., & Jackson, S. (1999, February). Technology diffusion within educational institutions: applying the technology acceptance model. Paper presented at the 10th Society for Information Technology & Teacher Education International Conference, San Antonio, TX.

TCC 2003 Online Conference
Copyright © 2003 Kapi'olani Community College,
University of Hawaii, All rights reserved.