Step 2 - Student Learning Outcomes

WHAT DO YOU WANT YOUR PROGRAM MAJORS TO KNOW AND/OR BE ABLE TO DO?

Assessing Student Learning
- Outcomes,
- Performance Indicators, and
- Assessment Methods

STEP 2-a - Student Learning Outcomes (SLOs)

What do you want majors to know and be able to do?
1. Identify Specific Learning Outcomes
2. Strategies for how to proceed
  1. Develop student learning outcome statements for your program
  2. Write a draft of the student learning outcome statements
  3. Discuss the initial draft of student learning outcomes with your colleagues
3. Student Learning Outcome (SLO) examples
4. Prioritize 3 to 5 SLOs for the Assessment Plan

STEP 2-b - Student Performance Indicators

How will you know your majors have achieved the program's learning outcomes?
1. Performanc Indicators; Identify what students will do to demonstrate their competence
2. Standards & Rubrics; Identify how students' level of competence will be measured

STEP 2-c - Assessment Methods

How will you collect the Student Performance Indicator data?
1. Identify how and by whom student performance will be assessed
  1. General Considerations; an assessment method will only be successful if it provides quality information with a reasonable amount of effort
  2. What Assessment Methods to Use; find the ones that meet the criteria of validity, reliability and practicality for each of your program learning outcomes
    - Direct Methods; Direct assessments require students to demonstrate knowledge &/or skill by responding to some measure
    - Indirect Methods; Indirect assessments ask students to reflect on learning experiences rather than demonstrate them specifically

STEP 2-a

Learning Outcomes are the Plan's Key Feature

WHAT DO YOU WANT MAJORS TO KNOW AND/OR BE ABLE TO DO?

2-a) Specify Learning Outcomes

A program must have clear and measurable statements of the (learning) outcomes expected of its majors. Though learning may be assessed at many specific points in the curriculum, outcomes assessment is often focused on how well students demonstrate what they have learned as they are completing their work in the major.

Because student learning/performance is assessed relative to expected outcomes, they form the foundation, the key feature, of a 'program-level' Assessment Plan and should be carefully considered by program faculty.

Strategies for how to proceed

Prioritize 3-5 student learning-outcome statements for your program:
A variety of resources on this website and at other institutions can help you develop student learning outcomes for your program. As a general guide, limit the number of outcomes to just those that resources and time limitations make possible. However, if your program already has a number of clearly defined learning objectives, limit student learning outcomes to 2 or 3 for each program objective. The outcomes listed should be the most significant, incorporating a synthesis of knowledge and skills developed throughout the curriculum. As you consider your program’s Student Learning Outcomes, reviewing Characteristics of Learning Outcomes may be useful.
Write a draft of the student learning-outcome statements:
You are encouraged to look at examples of learning outcomes statements from programs at other institutions.
Discuss the initial draft of student-learning outcomes with your colleagues:
Begin sharing the learning outcomes with faculty colleagues and revise them as discussion proceed. You may want to schedule a faculty meeting to present the draft outcomes in the context of a program assessment plan, allowing plenty of time for a focused discussion of the appropriateness and completeness of the outcomes.
Bringing a draft of proposed learning outcomes to this discussion can be more productive than starting with a blank sheet of paper.
As a faculty, agree on the student-learning outcomes that will guide your program-assessment plan:
When you have agreement on the student learning outcomes, record them on your Program Assessment Worksheet (or start a new one). You are now ready for to consider how to assess your Student Learning Outcomes.

HOW WILL YOU KNOW YOUR STUDENTS HAVE ACHIEVED THE PROGRAM'S LEARNING OUTCOMES?

Once expected learning outcomes are clearly identified, the next step is to identify student behaviors or products. The focus here is on student performance and demonstrating competence.

It is useful to consider measuring student learning as a two-part planning process:

Identifying the student performance to be assessed (what students do that will demonstrate their competence);
Identify how student performance will be assessed (how students' level of competence will be measured).

STEP 2-b

Performance Indicators

2b) Performance Indicators: Identify the student performance to be assessed

The learning outcomes you developed are only statements of what you want students to know and/or be able to do.
It is necessary to specify the kind(s) of performance through which students can demonstrate their competence relative to a learning outcome.
That is, what will students do that indicates how well they have achieved the learning outcome?
Keep in mind that you should be able to measure this performance in ways that are valid, reliable and practical.
Student actions and behaviors are called Performance Indicators and can take a variety of forms, but only behaviors, actions, responses, or products, etc. can be measured.

Preformance Indicators can involve . . .

Answering questions (written or oral)
Analyzing data and drawing conclusions
Summarizing a body of literature and research
Designing a project, study or experiment
Creating a product (e. g., musical composition or presentation)
Carrying out a project (e.g., sociological study, building a prototype)
Defining and defending a position (written or oral)
Demonstrating technical procedures or skills (e.g. in a chemistry laboratory, archeological excavation)
Demonstrating artistic skills (e.g., dance, theatre, portfolios)
Critiquing intellectual, artistic or scientific work of others (written or oral)
Solving problems (e.g., mathematics, engineering, computer science)

These examples of student performance can be designed to reflect different levels of competence and sophistication. Answering questions, for example, can address basic competence (memorizing) by asking students to recall specific factions or information – what Bloom calls the knowledge domain.

On the other hand, students may be asked to analyze, synthesize and evaluate what they have learned or apply their abilities to a specific situation. Analysis, synthesis, application, and evaluation are higher-order cognitive skills in Bloom’s Learning Taxonomy.

Program assessment involves most traditional methods used in Classroom Assessment. The American Psychological Association has evaluated the advantages and disadvantages of most methods for use in Assessment -> Evaluating Assessment Strategies

Strategies for how to proceed

For each of the learning outcomes you identified in Step 2-a, specify one or more performance indicators:
- The performance indicators (examples listed above) MUST be a measurable student behavior that are aligned with (inform) the outcome
- They should help demonstrate how well the student has achieved the program learning outcome
Use multiple indicators for each learning outcome if possible:
- Data from two or three sources are much more trustworthy and informative than is depending on a single assessment
Write drafts of the performance indicator identified for each learning outcome statement:
- On the Program Assessment Worksheet your downloaded or printed in earlier, enter the draft Performance Indicators next to the student learning outcome it informs.
- Examples of Student Performance Indicators linked to their associated Student Learning Outcomes can be seen in this sample Economics assessment plan.

As with developing the learning outcomes in Step 1-a, involving all program faculty in specifying performance indicators is very important.

HOW WILL YOU ASSESS WHAT STUDENT DO?

Next: step 2-c (Assessment Methods)

STEP 2-c

Assessment Methods

2c) Assessment Methods: Describe how student performance will be assessed

In many instances, the kinds of student behaviors in the Student Performance Indicators you developed suggest the kind of assessment procedure used. When deciding how exactly to assess student performance, strive for a good alignment between the assessment measure and the performance being assessed. As mentioned earlier, any assessment should be valid, reliable and practical. In other words, an assessment system will only be successful if it provides quality information with a reasonable amount of effort.

There are some additional general considerations about assessment methods.

Use multiple assessments for any learning outcome.
- Data from two or three sources are much more trustworthy and informative than is depending on a single assessment.
  - If the data agree with each other, you can be fairly confident the student performance being measured is accurately represented by the assessment results.
  - If the data don’t agree, you can be glad you weren’t trusting just one source and can begin to investigate what accounts for the discrepancy.
Even though multiple assessments are desirable, stay focused on quality of the information you obtain.
- If you cannot draw trustworthy conclusions about how well students are achieving the stated outcomes from the information obtained, other assessments should be considered.
- More modest assessments that produce useful information are far preferable to “swimming” in data of little practical value - that is, that will not help you make judgments about how you can improve your program. Sometimes less is more.
When appropriate, use existing assessment methods and instruments.
- Your assessment goals should drive the selection of methods, not the other way around. However . . .
  - Your program may currently use a faculty developed or commercially instrument that is valid and reliable.
  - A capstone course project (embedded assessment) that has been refined over the years may be an excellent tool to assess one or more program outcomes.
Try to pilot test whatever assessment whatever assessment instruments and processes you choose, including standardized exams.
- This may be preaching to the choir, as most of us know from experience, things don’t always go the way the think or hope they will.
- Get the bugs out before investing time, effort and money in a full implementation.
Additionally, you may specify standards or performance levels if that is appropriate.

What Assessment Methods to Use?

The good news is that there are many kinds of assessment methods available. Some careful thought and consideration are needed to find the ones that meet the criteria of validity, reliability and practicality for each of your program learning outcomes. A good place to begin is to consider the most commonly used assessment methods. In general, these methods are of two types – direct and indirect. (See also U. of Wisconsin Assessment Manual)

Direct assessments require students to demonstrate knowledge &/or skill by responding to some measure.
Indirect assessments ask students to reflect on learning experiences rather than demonstrate them specifically.

Both methods are useful and offer different perspectives.

Direct assessments usually occur while the student is still at UNR and are essential to know how well an outcome, e.g., a specified set of skills, has been achieved.
Indirect assessments can also be obtained from alumni or their employers. Alumni, for example, can offer perspectives on the extent to which those same skills are required in their current positions and how competent they were to use them.
The combination of direct and indirect assessments is an opportunity to use multiple assessment measures.

Strategies for how to proceed

For each of the Student Performance Indicators you identified above, specify one or more assessment methods.
The assessment methods you choose need to be valid, reliable and practical ways to measure the indicator. While you can use both direct and indirect assessment methods, any indicator that involves observed student performance (e.g., answering questions, solving problems, demonstrating a skill), should include a direct assessment method. Associated Student Learning Outcomes, Student Performance Indicators and Assessment Methods are shown in the sample Economics Program Assessment Plan.
Write a draft of the assessment methods identified for each performance indicator.
On the Program Assessment Worksheet your downloaded or printed in Step 1, enter the draft the assessment method(s) next to the student performance indicator it measures

Review and critique the Program Assessment Worksheet you have created thus far as you prepare to discuss it with your colleagues, preferably at a faculty meeting or retreat.
- What you have described should be the academic core of your program.
- You should see a logical alignment and congruence throughout.
- Each Student Learning Outcome should be stated in such a way that a student demonstrating the Student Performance Indicators for it will have competently achieved the outcome.
- The Assessment Methods you listed for each Student Performance Indicator will enable you to measure the level of competence.
- Finally, when taken together, what you have described should be the essence of what you expect your graduating majors to know and be able to do.

Implementing the assessment plan you have designed will provide data – the results of assessing student performance. The question now is, how to use these data to improve your program. You are now ready for Step 3.