This study addressed student access by examining success and withdrawal rates in the blended learning courses by comparing them to face-to-face and online modalities over an extended time period at the University of Central Florida. Further, the investigators sought to assess the differences in those success and withdrawal rates with the minority status of students. Secondly, the investigators examined the student end-of-course ratings of blended learning and other modalities by attempting to develop robust if-then decision rules about what characteristics of classes and instructors lead students to assign an “excellent” value to their educational experience. Because of the high stakes nature of these student ratings toward faculty promotion, awards, and tenure, they act as a surrogate measure for instructional quality. Next, the investigators determined the conditional probabilities for students conforming to the identified rule cross-referenced by expected grade, the degree to which they desired to take the course, and course modality.

Methods
Student grades by course modality were recoded into a binary variable with C or higher assigned a value of 1, and remaining values a 0. This was a declassification process that sacrificed some specificity but compensated for confirmation bias associated with disparate departmental policies regarding grade assignment. At the measurement level this was an “on track to graduation index” for students. Withdrawal was similarly coded by the presence or absence of its occurrence. In each case, the percentage of students succeeding or withdrawing from blended, online or face-to-face courses was calculated by minority and non-minority status for the fall 2014 through fall 2015 semesters.

Next, a classification and regression tree (CART) analysis (Brieman et al. 1984) was performed on the student end-of-course evaluation protocol (Appendix 1). The dependent measure was a binary variable indicating whether or not a student assigned an overall rating of excellent to his or her course experience. The independent measures in the study were: the remaining eight rating items on the protocol, college membership, and course level (lower undergraduate, upper undergraduate, and graduate). Decision trees are efficient procedures for achieving effective solutions in studies such as this because with missing values imputation may be avoided with procedures such as floating methods and the surrogate formation (Brieman et al. 1984, Olshen et al. 1995). For example, a logistic regression method cannot efficiently handle all variables under consideration. There are 10 independent variables involved here; one variable has three levels, another has nine, and eight have five levels each. This means the logistic regression model must incorporate more than 50 dummy variables and an excessively large number of two-way interactions. However, the decision-tree method can perform this analysis very efficiently, permitting the investigator to consider higher order interactions. Even more importantly, decision trees represent appropriate methods in this situation because many of the variables are ordinally scaled. Although numerical values can be assigned to each category, those values are not unique. However, decision trees incorporate the ordinal component of the variables to obtain a solution. The rules derived from decision trees have an if-then structure that is readily understandable. The accuracy of these rules can be assessed with percentages of correct classification or odds-ratios that are easily understood. The procedure produces tree-like rule structures that predict outcomes.

The model-building procedure for predicting overall instructor rating
For this study, the investigators used the CART method (Brieman et al. 1984) executed with SPSS 23 (IBM Corp 2015). Because of its strong variance-sharing tendencies with the other variables, the dependent measure for the analysis was the rating on the item Overall Rating of the Instructor, with the previously mentioned indicator variables (college, course level, and the remaining 8 questions) on the instrument. Tree methods are recursive, and bisect data into subgroups called nodes or leaves. CART analysis bases itself on: data splitting, pruning, and homogeneous assessment.

Splitting the data into two (binary) subsets comprises the first stage of the process. CART continues to split the data until the frequencies in each subset are either very small or all observations in a subset belong to one category (e.g., all observations in a subset have the same rating). Usually the growing stage results in too many terminate nodes for the model to be useful. CART solves this problem using pruning methods that reduce the dimensionality of the system.

The final stage of the analysis involves assessing homogeneousness in growing and pruning the tree. One way to accomplish this is to compute the misclassification rates. For example, a rule that produces a .95 probability that an instructor will receive an excellent rating has an associated error of 5.0%.

Implications for using decision trees
Although decision-tree techniques are effective for analyzing datasets such as this, the reader should be aware of certain limitations. For example, since trees use ranks to analyze both ordinal and interval variables, information can be lost. However, the most serious weakness of decision tree analysis is that the results can be unstable because small initial variations can lead to substantially different solutions.

For this study model, these problems were addressed with the k-fold cross-validation process. Initially the dataset was partitioned randomly into 10 subsets with an approximately equal number of records in each subset. Each cohort is used as a test partition, and the remaining subsets are combined to complete the function. This produces 10 models that are all trained on different subsets of the original dataset and where each has been used as the test partition one time only.

Although computationally dense, CART was selected as the analysis model for a number of reasons— primarily because it provides easily interpretable rules that readers will be able evaluate in their particular contexts. Unlike many other multivariate procedures that are even more sensitive to initial estimates and require a good deal of statistical sophistication for interpretation, CART has an intuitive resonance with researcher consumers. The overriding objective of our choice of analysis methods was to facilitate readers’ concentration on our outcomes rather than having to rely on our interpretation of the results.

Results
Institution-level evaluation: Success and withdrawal
The University of Central Florida (UCF) began a longitudinal impact study of their online and blended courses at the start of the distributed learning initiative in 1996. The collection of similar data across multiple semesters and academic years has allowed UCF to monitor trends, assess any issues that may arise, and provide continual support for both faculty and students across varying demographics. Table 1 illustrates the overall success rates in blended, online and face-to-face courses, while also reporting their variability across minority and non-minority demographics.

• ‪Mohamed Elkholany‬‏