Student Usability Test (June 2004)
Draft Summary Results from June 2004 Usability Testing
Who we tested: 4 undergraduates and 2 graduate students. All are student UM Libraries employees.
How we tested: We gave the participants 10 questions and showed them an on-screen image of draft #2 of the Libraries’ home page. Participants were not able to navigate through the site (since the page was an image rather than an HTML page), but we asked them to pretend the site was live and use the cursor to point to links they would use. We asked the participants to think out loud and explain the choices they made. We also asked some questions before and after the test.
We used Morae software and a Web camera to record the participants and their screen activity.
Following are the questions used in the test, an explanation of how users responded to the questions, and the Redesign Task Force’s comments and recommendations.
1. Does the library have the book Harry Potter and the Prisoner of Azkaban? How would you find out?
What users did: Three people said they would use either quick search or the catalog. Two people said they would use Quick Search. One person said s/he would use the Catalog. All said they would search by either “Harry Potter” or “Harry Potter and the Prisoner of Azkaban.”
Recommendations: None. The expectation is that they would have all found the appropriate result easily.
2. A librarian named Lily Griner came to your class to talk about library research. You want to reach her again; how would you find contact information for her?
What users did: Two people started with phone numbers. One person said s/he would start with either phone numbers or Library Administration. One person said s/he would start with Library Administration and go to Ask us! if that did not work. Two people said they would search the site.
Comments: While the expectation is that all users would have found the information, those who searched the site would not have taken the most efficient path (Searching the site for “Griner” turns up the staff directory listing toward the bottom of the first results page. Searching the site for “Lily Griner” turns up the McKeldin Service Plus Staff page with Griner’s contact information on the second page of results.)
Recommendations: Reconsider the labeling of the “Library Administration” link. Some users indicated that they would expect this link to lead to the Dean’s page or pages for upper level administration only.
3. A library instructor for one of your classes showed you a guide to research materials that might be helpful for your homework. You don’t remember where on the Web site the guide is located. How would you find it from the home page?
What users did: One person searched the site. One said s/he would search the site or use the site index. One person said s/he would use the site index. One person expected to find the answer under How do I…?
Two users weren’t sure and suggested starting points that would not have worked followed by one that would: One said “Faculty and Grad Students” or How do I…? The other said “DRUM??” or Search this site.
Recommendations: None.
4. You want to make a color photocopy of an article. How would you find out how much a color photocopy costs?
What users did: Two people started with How do I…? One said How do I…? or Ask us! or Visitors. One said site index or phone numbers. One said Site index or Ask us! One wasn’t sure and wondered about UM Digital Collections before saying maybe Library Administration or phone numbers.
Comments: Although it wasn’t obvious to all that they were selecting the appropriate link, most did select a link that would easily get them to the copy shop. Presumably, a live site would have quickly confirmed most were on the right track.
Recommendations: Add to site index under “C” (how one person said s/he would search); consider adding to the visitors’ page, since this does seem like a reasonable and sensible placement; update phone numbers page, which now has the old name “Photocopy Services”. Also update phone number page so all the listed departments are links. That way, those who go to phone numbers looking for general information will be able to first check the Web pages for those departments.
5. Your professor has asked you to write a paper on human cloning using a variety of sources. Where would you start your research?
What users did: Four people said they would start with Research Port. One wanted to Search this Site. One wasn’t sure and wondered about DRUM (seeing the word “research”) or Quick Search or e-journals or books.
Recommendations: Add more text to the search the site box, so it is not confused with a research search box: “Search for details about the Libraries”.
6. Does the library have the video To Kill a Mockingbird? How would you find out?
What users did: Two started with the Catalog. Two started with UM Digital Collections. On said s/he might start with UM’s 8 libraries to see which branch had multimedia or s/he might start with the Quick Search and enter the title. One saw the news item for a diversity video brown bag lunch and though by going to that page s/he might find links to videos (later this user saw UM Digital Collections and thought that would be the place to go to answer this question).
Recommendations: People are clearly misled by seeing the word “video” under UM Digital Collections. Perhaps the description should read “Selected images, videos….” Also, consider changing the last bullet point under catalog to read “videos & other materials owned by the Libraries.” This will give the hint of the wider forms of materials that might be included in that bullet.
7. It’s 100 degrees out and you don’t want to leave your room to go to the library, but you need help with your research. Can you get help from a librarian without leaving home? How?
What users did: Three people went to Ask us! Two people started with phone numbers with the idea that they would either call or e-mail for help. One person said s/he couldn’t get help from home (s/he later saw the Ask us! link and said that would do the trick).
Recommendations: Consider a different location for this link or a different visual presentation. It appears that half the users overlooked it, at least initially.
8. Your T.A. asked you to find newspaper articles using LexisNexis. Where would you start?
What users did: All six went to Research Port.
Comments: It appeared that some users expected to be able to click on the bullet points under Research Port (it looked like two wanted to click directly on databases). Will this be more clear when the users interact with an active page?
Recommendations: None.
9. You’re reading the Washington Post Opinion section and you see a reference to “that most fundamental of American values, pragmatism.” How would you look up the definition of “pragmatism” to make sure you know the precise definition?
What users did: Five users started with the Reference Shelf. One seemed to assume there would not be an online dictionary and said s/he would go to dictionary.com.
Comments: Some of the users weren…t sure where the Reference Shelf link would take them. Online dictionaries or just the location in the library to find dictionaries?
Recommendations: Consider whether there is a better label than “Reference Shelf.”
10. You’re going to do some research in the library, but want to get a cup of coffee before you start. How do you find out what time McKeldin’s Footnotes Café opens?
What users did: Five users started with hours. One said s/he’d look under phone numbers or Library Administration.
Comments: None of these users searched the site. If they had, they would not have reached very satisfactory results, since there is not a Footnotes page on our site.
Recommendations: Consider creating a Footnotes page with very general information.
Do you have any suggestions for improvements to this page?
- Everything looks cluttered. A lot of little words.
- There should be more emphasis on the reference shelf.
- Move digitized collections above the picture.
- I totally ignored this entire section [digitized collections]. For some reason this whole section did not catch my eye.
- I’m not sure if “More info for…” is for [campus or libraries].
- There’s a lot of things on it; I think that can be kind of confusing. Like I don’t know how helpful the Libraries’ New is….
- It’s rather nice. It looks like this is the home page…gives you a whole variety of things to look for.
- I think it’s fine.
Recommendations: Refine the design a bit more to reduce the “cluttered” effect. Change placement of digital collections so that it is not lost beneath the picture.
Is there anything that you would want to find on the Libraries’ home page that you don’t see here?
- “Other materials owned by the libraries” would include multimedia—maybe you should just say that.
- I find the photo distracting; it would be nice to have some promotional material for exhibits.
- Not really. This looks fine to me.
- Hours, but that might take up a lot of room. Maybe instead of Libraries’ news.
- ILL, circulation, different departments need to be obvious.
What do you like best about the page and what parts of the page are most useful to you?
- Search this site would be most useful. Like searching for cloning or books on cloning.
- I like the Quick Search box. If you’re not really sure…you can test it out.
- The questions [Ask us!] area…Sometimes you just need a person there.
- I’d be really interested to see how this Quick Search…works. What kind of information is brought back from that? It could be potentially very helpful for something quick.
- I like that the reference shelf is prominently displayed.
- The catalog and Research Port.
- The [quick] search. Some places don’t have it and you really have to start digging…I think the search is the best thing.
- Catalog and Research Port which makes it obvious what you’re looking for.
Recommendations: Add descriptive text under “Search this site” so it’s more clear what the search does.
Is there anything else you would like to add?
- I like the colors. I like how you have this set [bottom menu bar] and also this up here [yellow and black at top].
- What is this? Libraries’ news? Why is a piano recital Library news?
- A collage of different libraries in the background or a picture of campus. It’s just all white…kind of bland. It’s good with the [logo] and Testudo….
- I don’t like the gray. It’s just dull.
- Phone numbers: I’m not sure what numbers these are.
- No. I think it’s fine. I don’t know why Testudo is behind the reference desk, but I guess it is school spirit.
