Search the UMD Libraries website:
Thursday, October 2, 2014
McKeldin Open 24 hours. Ending Friday at 8pm

Art

11:00AM - 04:00PM
Architecture 12:00PM - 08:00PM
Chemistry
08:00AM - 10:00PM
EPSL 08:00AM - 11:00PM
Media Services

in Hornbake

08:00AM - 10:00PM
Special Collections

in Hornbake

10:00AM - 05:00PM
MSPAL 08:30AM - 11:00PM
Shady Grove See here for hours

Assessment Methods

Guidelines for Developing Assessment Methods (Taken From: http://www.unl.edu/svcaa/assessment/learningoutcomes_guidebook.pdf)

Methods for assessing outcomes should:

  • gather evidence that will provide the best representation of the learning that is expected
  • detail what evidence will be gathered, how that evidence will be gathered, and who is responsible for gathering that evidence
  • record and analyze data so there can be reflection on learning across all students
  • reflect the methods used for assessing learning in the curriculum and research approaches used in the discipline
  • be aligned with content and the outcomes of the curriculum
  • provide evidence of the extent to which learning was successful across students

 A. CLASSROOM/COURSE DATA

Assessment Type Advantages Disadvantages
Overall Analysis
  • maximizes faculty autonomy and investment in student learning
  • facilitates prompt feedback
  • can provide immediate feedback to faculty about teaching effectiveness
  • limited by pedagogical constraints of instructor
  • can produce unreliable evaluation results
  • results affected by instructor/departmental evaluation bias
  • generally can promote disconnected course experiences

Objective Tests
(multiple choice, true-false, fill-in-the-blank items)

  • displays good psychometric properties
  • facilitates rapid feedback through ease of scoring
  • develops norms
  • inexpensive
  • comprehensive
  • g
  • usually involves testing low-level knowledge
  • constructing high quality test questions difficult
Essay Tests
  • showcases deeper learning, higher order thought processes
  • requires transfer, integration of learning from other sources
  • can include applications or problem-based learning
  • develops writing skills and critical thinking
  • inexpensive and easy to administer
  • faster to construct than objective tests
  • may disadvantage students with poor writing or thinking skills
  • takes longer to grade and provide feedback
  • produces narrower sample of content knowledge
  • difficult to maintain consistency in evaluation across essays
Embedded Questions and Assignments
  • saves time since assignments will already be required for the course
  • overcomes faculty resistance due to reduced intrusion of external assessment activity
  • encourages faculty to discuss common course outcomes, goals, & objectives
  • promotes shared responsibility for agreeing where embedding should occur
  • can be time-consuming to coordinate effort
  • may be taxing to isolate key aspects of performance

Classroom Assessment Techniques
(1-minute papers, course focus groups, free-writing, etc)

  • convenience
  • provides immediate feedback to faculty about success
  • vividly demonstrates faculty commitment to student satisfaction
  • focus on teacher performance
  • should be combined with other methods for full picture of student learning
  • perceived to sacrifice content coverage for time required to assess
  • focus on retention, not long-term understanding

 

B. INDIVIDUAL PROJECTS/PERFORMANCE ASSESSMENT

Assessment Type Advantages Disadvantages

Oral Presentations
(debate, role play)

  • builds expertise in important communication area of oral expression
  • promotes importance of sharing knowledge
  • enhances oral skills
  • Q&A promotes thinking on your feet
  • assists professor to cover course content
  • may burden students with speech and language difficulties, speaking anxiety
  • may be hard to grade

Graphic Test and Displays
(concept maps, outlines)

  • provides experience in applying and organizing course concepts
  • assists in thinking through organization of information
  • additional grappling with the material enhances recall
  • appeals to visual learners
  • students have limited practice with displaying graphic skills
  • students may not have sufficient experiences in interpreting skills
  • technological sophistication will influence production quality
Posters
  • hold students accountable for independent project
  • reduces grading burden compared to writing projects
  • provides opportunity to integrate communication skills (e.g., writing, graphics, oral defense)
  • can incorporate team effort
  • expert judgment, peer review can be facilitated with criteria
  • may need to make special arrangements for space
  • students may invest money in project for one-shot exposure
  • lack of aesthetic sense may handicap poster effectiveness
  • numbers of posters to be judged can create quality pressures on grading

Structural/Situational Assessments
(guided leaning, in-baskets, critical situations, etc.)

  • provides realistic testing circumstance
  • reality engages and motivates students
  • promotes transfer of information, application
  • taps complex skills
  • difficult to construct and measure
  • locating designed instruments is challenging
  • prone to history/context/age cohort effects
  • students may rely on common sense under pressure rather than their knowledge from the course

C. SUMMATIVE PERFORMANCE ASSESSMENT 

Assessment Type Advantages Disadvantages
Overall Analysis
  • promotes coherence in curriculum planning
  • provides feedback loop to improve quality
  • some strategies can be adapted to student interests
  • some options are labor and/or cost intensive
  • students may not receive direct feedback regarding their performances, thus limiting their own gains from effort expended
  • departments may ignore available data in their planning
Standardized Tests
  • typically one shot assessment
  • facilitates comparisons over time
  • convenient
  • may not reflect gains or growth across time
  • exiting students may not benefit from feedback
  • existing instruments may not match to the mission and goals of departments
  • expensive
  • students may not be motivated to do their best work
  • when test occurs may not maximize true learning
  • administration may not be flexible
  • not student-centered
  • limited faculty ownership
  • verifying bad performance can be threatening to motivation
  • scores may be delayed in return, reducing the impact of feedback
  • there may not be a standardized test for the identified content
  • can facilitate problematic comparisons to other programs (e.g., comparisons may not take into account differential resources, student characteristics, etc.)
Capstone Experiences
  • fosters aura of importance that may motivate students throughout the curriculum
  • encourages departmental endorsement of culminating experience
  • promotes student responsibility for engaged course
  • supports program coherence for faculty and students
  • course content can be flexible
  • topical design of capstone can engage faculty in planning (e.g. seminar topics can be taught in special interest areas as long as the performance goals meet departmental expectations
  • high stakes performance can be impaired by performance anxiety
  • typically low enrollment course is expensive to provide seats for all seniors
  • graduation may depend on successful completion of capstone which can generate some anxiety for faculty and students
Portfolios
  • shows sophistication in student performance
  • illustrates longitudinal trends
  • highlights student strengths
  • identifyies student weaknesses for remediation, if timed properly
  • collection will be no better than the quality of collected instruments
  • time consuming and challenging to evaluate
  • space and ownership challenges making evaluation difficult
  • content will vary widely with students
  • time intensive to convert to meaningful data
Case and Longitudinal Studies
  • can provide rich detail
  • level of attention can build esteem
  • builds allegiance
  • transfer students may be omitted
  • expensive and time-consuming
  • small sample limits generalization
  • attribution of historical or cohort effects may taint participant reports
  • selection for tracking may influence outcome a change student experience

D. SELF-ASSESSMENT

Assessment Type Advantages Disadvantages
Self-Assessment
  • multiple modes and variable sophistication possible
  • quality of self-assessment related to quality of content knowledge
  • flexible in format; prompts provided or not
  • might ask about change over time
  • empowers students to practice self-evaluation
  • promotes transfer of accountability to other situations
  • student judgment may not be accurate
  • self-assessments are prone to evaluative biases
  • students have limited experience being held accountable to judge their own work
  • faculty may perceive this practice to set up more grade conflicts

E. COLLABORATIVE ASSESSMENTS

Assessment Type Advantages Disadvantages
Research Teams & Group Projects
  • student-centered designs promote engagement
  • provides opportunity to practice group skills, time management
  • promotes independent work at deeper level
  • breadth of assignments can address content coverage issue
  • simulates how professional activities/achievement transpire
  • produces synergy and excitement around project completion
  • creates a venue to synthesize content bases from multiple courses
  • students have limited training in group dynamics
  • erroneous ideas that are not caught and corrected spread across group members
  • challenging to faculty to judge when to redirect or rescue student groups in trouble
  • difficult to determine division of labor and effort
  • time-consuming

F. INTERVIEWS and SURVEYS

Assessment Type Advantages Disadvantages
Attitude Measurement General Analysis
  • easy to administer
  • cheap
  • easy to score
  • quick feedback
  • can be reliable but not valid
  • validity hinges on good design
  • may not be valid
  • demand characteristics may distort results
  • participants may not have good knowledge about their own attitudes
  • participants may demonstrate response bias or dishonesty
  • labor intensive to interpret

Satisfaction Surveys
(Alumni, Employers, Grad School Advisers, Parents, etc.)

  • fosters positive public relations because activity signals faculty concern for quality
  • targets of survey may be prompted to other positive actions (e.g., donations, hiring, recruitment of new students)
  • external judges may be more objective in their appraisal of student abilities, achievements
  • recurring insights may point to some problems that need remediation
  • provides important perspective on relevance of program to various occupations
  • tracking down and engaging targets may be problematic
  • low return rates compromise validity
  • some respondents may be motivated not to tell the truth (e.g., don't want to bear bad news, demand characteristics)
Exit Interviews
  • provides realistic picture
  • provides catharsis
  • provides in-depth, personal perspective on experience of major
  • can be embedded in existing courses to capture broad range of student experience
  • demonstrates overt department commitment to high quality
  • may promote long-term allegiance among graduating students
  • can generate reinforcing feedback to help departments sustain effectiveness
  • volunteers may have a negative or a positive agenda that may not be representative, producing a selection bias
  • time-consuming to coordinate and evaluate the results
  • students may not show up for discussion
  • negative discussion may influence formerly neutral students to redefine their experience negatively
  • completion challenge
  • participants may paint too rosy a picture partially due to timing
  • expensive
  • results can be influenced by the quality of the interviewer and protocol
Focus Groups
  • small discussion groups promote engagement
  • can be employed to provide feedback on a class, course, or program
  • participants can benefit directly from changes that result from their feedback
  • demonstrates overt department commitment to high quality
  • can generate reinforcing feedback to help departments sustain effectiveness
  • development of protocol can be involving for faculty
  • may tap unforeseen areas of concern
  • current students may feel some pressure to not be completely candid for fear of retribution
  • volunteers may have a negative or a positive agenda that may not be representative
  • time-consuming to coordinate and evaluate the results
  • students may not show up for discussion