Search the UMD Libraries website:
Thursday, November 27, 2014
McKeldin CLOSED

Art

CLOSED
Architecture CLOSED
Chemistry
CLOSED
EPSL CLOSED
Media Services

in Hornbake

CLOSED
Special Collections

in Hornbake

CLOSED
MSPAL CLOSED
Shady Grove See here for hours

Assessment Methods

Guidelines for Developing Assessment Methods (Taken From: http://www.unl.edu/svcaa/assessment/learningoutcomes_guidebook.pdf)

Methods for assessing outcomes should:

  • gather evidence that will provide the best representation of the learning that is expected
  • detail what evidence will be gathered, how that evidence will be gathered, and who is responsible for gathering that evidence
  • record and analyze data so there can be reflection on learning across all students
  • reflect the methods used for assessing learning in the curriculum and research approaches used in the discipline
  • be aligned with content and the outcomes of the curriculum
  • provide evidence of the extent to which learning was successful across students

 A. CLASSROOM/COURSE DATA

Assessment Type Advantages Disadvantages
Overall Analysis
  • maximizes faculty autonomy and investment in student learning
  • facilitates prompt feedback
  • can provide immediate feedback to faculty about teaching effectiveness
  • limited by pedagogical constraints of instructor
  • can produce unreliable evaluation results
  • results affected by instructor/departmental evaluation bias
  • generally can promote disconnected course experiences

Objective Tests
(multiple choice, true-false, fill-in-the-blank items)

  • displays good psychometric properties
  • facilitates rapid feedback through ease of scoring
  • develops norms
  • inexpensive
  • comprehensive
  • g
  • usually involves testing low-level knowledge
  • constructing high quality test questions difficult
Essay Tests
  • showcases deeper learning, higher order thought processes
  • requires transfer, integration of learning from other sources
  • can include applications or problem-based learning
  • develops writing skills and critical thinking
  • inexpensive and easy to administer
  • faster to construct than objective tests
  • may disadvantage students with poor writing or thinking skills
  • takes longer to grade and provide feedback
  • produces narrower sample of content knowledge
  • difficult to maintain consistency in evaluation across essays
Embedded Questions and Assignments
  • saves time since assignments will already be required for the course
  • overcomes faculty resistance due to reduced intrusion of external assessment activity
  • encourages faculty to discuss common course outcomes, goals, & objectives
  • promotes shared responsibility for agreeing where embedding should occur
  • can be time-consuming to coordinate effort
  • may be taxing to isolate key aspects of performance

Classroom Assessment Techniques
(1-minute papers, course focus groups, free-writing, etc)

  • convenience
  • provides immediate feedback to faculty about success
  • vividly demonstrates faculty commitment to student satisfaction
  • focus on teacher performance
  • should be combined with other methods for full picture of student learning
  • perceived to sacrifice content coverage for time required to assess
  • focus on retention, not long-term understanding

 

B. INDIVIDUAL PROJECTS/PERFORMANCE ASSESSMENT

Assessment Type Advantages Disadvantages

Oral Presentations
(debate, role play)

  • builds expertise in important communication area of oral expression
  • promotes importance of sharing knowledge
  • enhances oral skills
  • Q&A promotes thinking on your feet
  • assists professor to cover course content
  • may burden students with speech and language difficulties, speaking anxiety
  • may be hard to grade

Graphic Test and Displays
(concept maps, outlines)

  • provides experience in applying and organizing course concepts
  • assists in thinking through organization of information
  • additional grappling with the material enhances recall
  • appeals to visual learners
  • students have limited practice with displaying graphic skills
  • students may not have sufficient experiences in interpreting skills
  • technological sophistication will influence production quality
Posters
  • hold students accountable for independent project
  • reduces grading burden compared to writing projects
  • provides opportunity to integrate communication skills (e.g., writing, graphics, oral defense)
  • can incorporate team effort
  • expert judgment, peer review can be facilitated with criteria
  • may need to make special arrangements for space
  • students may invest money in project for one-shot exposure
  • lack of aesthetic sense may handicap poster effectiveness
  • numbers of posters to be judged can create quality pressures on grading

Structural/Situational Assessments
(guided leaning, in-baskets, critical situations, etc.)

  • provides realistic testing circumstance
  • reality engages and motivates students
  • promotes transfer of information, application
  • taps complex skills
  • difficult to construct and measure
  • locating designed instruments is challenging
  • prone to history/context/age cohort effects
  • students may rely on common sense under pressure rather than their knowledge from the course

C. SUMMATIVE PERFORMANCE ASSESSMENT 

Assessment Type Advantages Disadvantages
Overall Analysis
  • promotes coherence in curriculum planning
  • provides feedback loop to improve quality
  • some strategies can be adapted to student interests
  • some options are labor and/or cost intensive
  • students may not receive direct feedback regarding their performances, thus limiting their own gains from effort expended
  • departments may ignore available data in their planning
Standardized Tests
  • typically one shot assessment
  • facilitates comparisons over time
  • convenient
  • may not reflect gains or growth across time
  • exiting students may not benefit from feedback
  • existing instruments may not match to the mission and goals of departments
  • expensive
  • students may not be motivated to do their best work
  • when test occurs may not maximize true learning
  • administration may not be flexible
  • not student-centered
  • limited faculty ownership
  • verifying bad performance can be threatening to motivation
  • scores may be delayed in return, reducing the impact of feedback
  • there may not be a standardized test for the identified content
  • can facilitate problematic comparisons to other programs (e.g., comparisons may not take into account differential resources, student characteristics, etc.)
Capstone Experiences
  • fosters aura of importance that may motivate students throughout the curriculum
  • encourages departmental endorsement of culminating experience
  • promotes student responsibility for engaged course
  • supports program coherence for faculty and students
  • course content can be flexible
  • topical design of capstone can engage faculty in planning (e.g. seminar topics can be taught in special interest areas as long as the performance goals meet departmental expectations
  • high stakes performance can be impaired by performance anxiety
  • typically low enrollment course is expensive to provide seats for all seniors
  • graduation may depend on successful completion of capstone which can generate some anxiety for faculty and students
Portfolios
  • shows sophistication in student performance
  • illustrates longitudinal trends
  • highlights student strengths
  • identifyies student weaknesses for remediation, if timed properly
  • collection will be no better than the quality of collected instruments
  • time consuming and challenging to evaluate
  • space and ownership challenges making evaluation difficult
  • content will vary widely with students
  • time intensive to convert to meaningful data
Case and Longitudinal Studies
  • can provide rich detail
  • level of attention can build esteem
  • builds allegiance
  • transfer students may be omitted
  • expensive and time-consuming
  • small sample limits generalization
  • attribution of historical or cohort effects may taint participant reports
  • selection for tracking may influence outcome a change student experience

D. SELF-ASSESSMENT

Assessment Type Advantages Disadvantages
Self-Assessment
  • multiple modes and variable sophistication possible
  • quality of self-assessment related to quality of content knowledge
  • flexible in format; prompts provided or not
  • might ask about change over time
  • empowers students to practice self-evaluation
  • promotes transfer of accountability to other situations
  • student judgment may not be accurate
  • self-assessments are prone to evaluative biases
  • students have limited experience being held accountable to judge their own work
  • faculty may perceive this practice to set up more grade conflicts

E. COLLABORATIVE ASSESSMENTS

Assessment Type Advantages Disadvantages
Research Teams & Group Projects
  • student-centered designs promote engagement
  • provides opportunity to practice group skills, time management
  • promotes independent work at deeper level
  • breadth of assignments can address content coverage issue
  • simulates how professional activities/achievement transpire
  • produces synergy and excitement around project completion
  • creates a venue to synthesize content bases from multiple courses
  • students have limited training in group dynamics
  • erroneous ideas that are not caught and corrected spread across group members
  • challenging to faculty to judge when to redirect or rescue student groups in trouble
  • difficult to determine division of labor and effort
  • time-consuming

F. INTERVIEWS and SURVEYS

Assessment Type Advantages Disadvantages
Attitude Measurement General Analysis
  • easy to administer
  • cheap
  • easy to score
  • quick feedback
  • can be reliable but not valid
  • validity hinges on good design
  • may not be valid
  • demand characteristics may distort results
  • participants may not have good knowledge about their own attitudes
  • participants may demonstrate response bias or dishonesty
  • labor intensive to interpret

Satisfaction Surveys
(Alumni, Employers, Grad School Advisers, Parents, etc.)

  • fosters positive public relations because activity signals faculty concern for quality
  • targets of survey may be prompted to other positive actions (e.g., donations, hiring, recruitment of new students)
  • external judges may be more objective in their appraisal of student abilities, achievements
  • recurring insights may point to some problems that need remediation
  • provides important perspective on relevance of program to various occupations
  • tracking down and engaging targets may be problematic
  • low return rates compromise validity
  • some respondents may be motivated not to tell the truth (e.g., don't want to bear bad news, demand characteristics)
Exit Interviews
  • provides realistic picture
  • provides catharsis
  • provides in-depth, personal perspective on experience of major
  • can be embedded in existing courses to capture broad range of student experience
  • demonstrates overt department commitment to high quality
  • may promote long-term allegiance among graduating students
  • can generate reinforcing feedback to help departments sustain effectiveness
  • volunteers may have a negative or a positive agenda that may not be representative, producing a selection bias
  • time-consuming to coordinate and evaluate the results
  • students may not show up for discussion
  • negative discussion may influence formerly neutral students to redefine their experience negatively
  • completion challenge
  • participants may paint too rosy a picture partially due to timing
  • expensive
  • results can be influenced by the quality of the interviewer and protocol
Focus Groups
  • small discussion groups promote engagement
  • can be employed to provide feedback on a class, course, or program
  • participants can benefit directly from changes that result from their feedback
  • demonstrates overt department commitment to high quality
  • can generate reinforcing feedback to help departments sustain effectiveness
  • development of protocol can be involving for faculty
  • may tap unforeseen areas of concern
  • current students may feel some pressure to not be completely candid for fear of retribution
  • volunteers may have a negative or a positive agenda that may not be representative
  • time-consuming to coordinate and evaluate the results
  • students may not show up for discussion