METHOD

Graduating Students. During the Spring 1998 semester, approximately 900 undergraduate students and 450 graduate students had completed their application for graduation through the Office of Admissions and Records. A sample of one-third of each group of this population was drawn through quasi-random selection based on student identification numbers for a total of 450 students. Each of the students selected in the sample were contacted by telephone and asked to participate in the Graduation Exit Interview. The order of the list of students was randomized such that every third student on the list was called first, then every second student, followed by the remaining students. Students in the first two groups who received phone messages rather than a direct contact during the initial call were contacted a second time, coinciding with the last group of students. A total of 57 individual student interviews were scheduled. Of this sample, 47 students completed the interviews. One individual's data was removed from the sample, at their request. Therefore, 46 completed interviews were available for analysis.

Faculty. A group of 45 faculty interviewers were assembled based on participation in previous Exit Interviews, expressed interest, and participation in selected university committees. Faculty were contacted via the campus electronic mail system. Follow-up phone calls were conducted to individuals who did not respond to the first message. A total of 15 faculty interviewers agreed to participate. Each of which was asked to conduct two or more student interviews.

Measure

The 1998 Graduation Exit Interview was based on previous versions of individual and group interviews, yet was redesigned for use by both undergraduate and graduate students (see attached Interview). The current interview incorporates closed-ended response categories and open-ended comment areas for each of the questions. This format can be used for both individual and group interviews, and as a survey completed by the student.

Procedure

Interviews. Individual Graduation Exit Interviews were scheduled three weeks prior to graduation. In a single week, four 1.5 hour sessions at midday and four 1.5 hour sessions after normal business hours were scheduled. The interview was designed to be 20 minutes in length to accommodate the schedules of faculty and graduating students. Participants met in the student union, where they were introduced, and received the interview instructions and materials (see Appendix A). Each faculty/student pair was allowed to choose a private location to complete the interview.

Mail Survey. For students who expressed interest, but were unable to attend the arranged sessions with a faculty interviewer, the survey was mailed to them with a cover letter (see attached letter to Graduating Student). In addition, students who scheduled interviews, but failed to attend their scheduled session, were mailed the survey. A total of 30 students were mailed surveys. A postage-paid envelope was included in the survey package. Twelve completed surveys were returned.

RESULTS

There was an 82% completion rate for the scheduled interviews (47/57), and a 40% return rate for the mail surveys (12/30). Of the 59 completed surveys (57+12), approximately 71% of the participants had earned a baccalaureate degree, the remaining group earning either master or doctoral degrees.

The time reported to complete the degree earned averaged six years across degrees.

With regard to student expectations for their academic experience at UNR, most reported their initial expectations being met, and their perception becoming less favorable over time.

Advisement in the academic major was generally reported as less adequate than expected.

The expertise of faculty exceeded the expectation of most students (24%), while their availability was reported as meeting their expectations (26%).

Courses were reported to be of good quality, with most students very satisfied with the size of courses. Variability was reported in satisfaction with the variety of course offerings, though. In the core curriculum measures, approximately half of the respondents completed these requirements at UNR. Some learning gain was generally reported in each of these areas.

Participant's overall satisfaction with UNR was rated highly.

The extent to which gains were made in important domains of learning was positive in all areas, except mathematical skills.

The greatest overall strength of degree programs was consistently reported as the faculty. The area of most suggested changes in degree programs was identified as the curriculum.

Participants reported feeling well prepared for future plans (87.9%), and responded favorably to both choosing UNR again (81%) and recommending it to a friend (89.7%).

Due to the limited breadth of the sample, and its nature (as self-selected), general indications are all that can be validly gleaned. Thus, in general, student perceptions about UNR become more negative over time. Dissatisfaction with academic advisement may account for this trend. Due to the limited reports of learning gains in participant responses, evaluation of student gains in core curriculum areas is difficult and merits further investigation. Of the data available, though, most participants reported some learning gain. Also, gains in mathematical and computer skills seem to lag behind advances in written and oral communication. Faculty strengths are reported as the highlight of participant's academic training, a great compliment to our staff. Finally, although students feel well prepared for their future plans, and leave with a favorable opinion of UNR, they consistently suggest amendments to the curriculum – with anecdotal comments relating to increased employment centered and practical experience.

The voluntary participation of faculty and students made this project possible. The Office of Admissions & Records, the Student Services Office, and the Assessment Office Survey Coordinator facilitated in the development of this project. The author, and the Office of University Assessment would like to acknowledge the contributions of the following UNR faculty and staff members, as well as the departments from which student participants volunteered:

Suggested changes to the current format include changes to the content, as well as the structure and method of the survey. With respect to content, the addition of a demographics section would facilitate the use of statistical comparisons between groups were a sufficiently large sample to be drawn. In addition, the question regarding service utilization should be modified to reflect general categories of services, as opposed to the current format of specifically naming some of the many services available. This change would eliminate overlap with each service's self-study of utilization, and would increase accurate participant responses. Also, the assessment of learning gain in the core curriculum would benefit from the addition of a question specifically related to learning: "Evaluate your learning:" learned a lot, learned some, did not contribute to my knowledge.

The structure of the current interview should be further revised to include additional closed-ended response categories for all questions. These categories should reflect the most common responses provided in the narrative responses over previous survey administrations. Another structural modification which is indicated is the use of a scantron-able form to facilitate data entry and to provide a measure which is aesthetically clear and has a professional appearance.

Finally, the method of administration must be revised in order to meet the goals of the survey: to draw meaningful and valid conclusions regarding the effectiveness of and satisfaction with UNR in recent graduates. Limited sample size and participant self-selection cause a number of threats to the generality of responses, and these threats are clearly observable in the data from previous administrations. In particular, graduates with extreme positive or negative experiences are most likely to respond, eliminating the majority of students.

Thus, the method of administration should be changed such that all students applying for graduation receive the survey when they purchase their program of study form in the Cashiers Office. This would require the cooperation of additional university offices, but would ensure that all students in the population of interest have an equal opportunity to participate. Requiring that the form be returned is not feasible at this time because it is not officially listed as a requirement in the UNR catalog, but revisions of this policy in the future may be warranted, and for the interim, students can be prompted to return this material when they turn in their program of study. Surveys can be forwarded to the Assessment Office from Admissions and Records or the Graduate School. Utilization of this method of administration would require that additional measures be taken to ensure anonymity and confidentiality of responses, however.