Graduate Student Usability Test (August 2004)

Summary Results from August 2004 Usability Testing

Who we tested: Four incoming CLIS graduate students

How we tested: We gave the participants a list of 11 questions and showed them a live draft (#5) of the Libraries’ home page. We asked them participants to think out loud and explain the choices they made. We also asked some questions before and after the test. Each session lasted 20-40 minutes.

Does the library have the book Harry potter and the Prisoner of Azkaban? How would you find out?

All the students used the Catalog, searching on “title words” or “title beginning with” for Harry Potter or Azkaban. Students are able to find the catalog entry and availability. One student just entered “Azkaban” assuming the book would be found because it is an uncommon word.

A librarian named Paula Hayes came to your class to talk about library research. You want to reach her again; how would you find contact information for her?

1. Staff and Divisions, then to the introductory Directory page, tries Summary Phone List thinking it would have individual phone numbers. Then tries Directory by Division and scrolls until s/he find Paula Hayes’ name.
2. Phone numbers, then tries Staff and Divisions expecting to see a directory of staff. Returns to the home page, and searches on Paula Hayes in the search box. Student clicks on one of the results because it reads “Paula Hayes is available …” hoping it will have her contact information. Student finds contact information at the bottom of the page. But student feels s/he is missing where it says staff directory or contact the staff. Does one last check at Site index, but decides the page is way too big.
3. Staff and Divisions, but student does not see a list of staff members, or the link to the Directory. Returns to home page and clicks on Phone numbers, then the link to the Staff Directory.
4. Student scans page for staff directory, but does not see one, so s/he clicks on Site Index. Clicks on “S” and finds staff directory. Scans the initial introductory page, but finds the link to Directory by Staff, and locates Paula Hayes.

You’ve heard that the Libraries have guides for Information studies; can you find them?

1. Student isn’t sure what kind of guides are being asked for, thinks about databases and using the Catalog, but clicks on Site index. Student looks under “I” for Information Studies, finding nothing. Thinks it may be something a student will find in the Wasserman library, returns to Site index and clicks on Wasserman. There is a link to the guides from the Wasserman web page on the CLIS site.
2. “I don’t really understand the question. What kind of guides, a reference book or a person?” Student uses search box and types in “guides for information studies.” Leaves the search results although “Research Guides By Subject Discipline, UM Libraries” are the first two results. Goes to search the Catalog and searches on “guides for information studies,” but doesn’t really know what s/he is asking for.
3. Student thinks about using the Catalog, but clicks on Reference Shelf thinking that guides for information studies may be reference related. Student then tries Catalog, entering “information studies” into the search box., but gets an extensive results list. “I really don’t know how I’d find those then, it might be more helpful if that was included in the Reference Shelf stuff. It might be more intuitive to find it there. At this point I’d ask a librarian”
4. “I really don’t know what they’re asking me for.” Selects How do I…?, reads the first column, and notices Ask Us! at the bottom of the second column, but misses the link to Guides to Information Resources under the “learn about doing library research” section. Decides s/he doesn#&146;t want to use Ask Us! “Is it ok if I don’t know? The questionѺs assuming that I’ve heard of this and I really don’t know what they’re talking about, so I guess I’d go to someone else and ask them if they know what these guides are?” When directed back to using the web page to find the guides student replies, “I’d probably stop using them…I guess I could look up information studies in the Catalog? But, I don’'t know if it’s a catalog record or a file of some sort.”

While in the library, you want to make a color photocopy of an article. How much would that cost?

3 of the students use How do I…?, and while one checks Pay for Print first under the “copy and print” section all three choose the link to McKeldin Copy Shop and find the price list. One says s/he chooses How do I…?, “because it seems to be more of a frequently asked questions page.”

The other student clicks on UM’s 8 libraries, selects McKeldin, finds and selects McKeldin Copy Shop from list. Locates the price list on the Copy Shop page.

Find an article about Abraham Lincoln (you don’t have to find the full text).

1. Research Port, and tries using a biography database, BGMI. Student finds a few entries, but hasn’t discovered an article. Student tries history databases, selecting U.S. and Canada, but changes his/her mind. Returns to home page, felt s/he wasn’t finding anything that specifically said articles. Checks How do I…?, and under the find articles and article indexes selects Research Port. Feels more instruction is needed. Tries selecting both biography and history as databases. Then tries Quick Search with Abraham Lincoln, and finds an article.
2. Research Port, chooses history from databases list, selects U.S. and Canada sub-category. Selects the America History and Life database, searches for Abraham Lincoln as a keyword.
3. Research Port, “which looks different, but that’s ok.” Student clicks on Quick Search and enters “Abraham Lincoln” into search box. Finds an article from returned list.
4. Research Port, and scrolls for a history database. Chooses the U.S. and Canada sub-category. “I’m used to where you get the list and then you can click the ones you want to search, and you type in your term and it’ll search however many you want. And it looks like here I have to pick one database.” Chooses America History and Life, ends up at simple search, searches for Lincoln as a keyword, and locates an article.

Does the library have the move To Kill a Mockingbird?

All students use the Catalog. Two search for To Kill a Mockingbird under “title words” or “title beginning with.”

Two used Advanced Search entering “to kill a mockingbird” in title search box and changing format to videos/films. One first tries Kill a Mockingbird as title with video (VHS) format selected, and gets no results. Tries again selected format all videos/films, “which I guess I didn’t see before.” Finds catalog entry.

It’s 100 degrees out and you don’t want to leave your room to go to the library, but you need help with your research. Can you get help from a librarian without leaving home? How?

All students use Ask Us!

One student uses Ask Us!, “but if I had not seen that before I might go to How do I…” Student tries the Off-Campus Access to Library Resources link under “use the library from off campus,&$148; then sees the Ask Us! link on the How do I… page.

Your professor asked you to find newspaper articles using LexisNexis. How would you do that?

1. Looks for LexisNexis under Site Index. Then tries Research Port, the Database list, scrolls and clicks on “next” until s/he comes to LexisNexis.
2. Research Port, scrolls the database categories list first then tries the Database titles list, and selects L. Finds LexisNexis.
3. “I’m pretty sure that’s a database.” Clicks on Research Port, selects “L” from database title list. Finds LexisNexis
4. Clicks on Research Port, types LexisNexis in search box and finds link to database.

You’re reading the Washington Post Opinion section and you see a reference to “that most fundamental of American values, pragmatism.” Look up the definition of “pragmatism” to make sure you know the precise definition.

All students went to Reference Shelf and selected a dictionary.

You’re going to do some research in the library, but want to get a cup of coffee before you start. What time does McKeldin’s Café open?

One student uses the Site index. Student knows the name of the Café is Footnotes, and finds Footnotes Café on the page.

The other three use Hours, selecting McKeldin Library and finding the Footnotes Café hours. One of the students asks, “McKeldin has a café?”

Find the full text of the following article in the American Journal of Political Science: “Validating a Measure of National Political Legitimacy,” by John Fraser, vol. 18, no. 1 (Feb 1974), pp.117-134.

1. Research Port, tries title of Database, both “starts with” and “exact.” Checks for a political science category under Databases. Selects Politics and public policy, searches all sub-categories. Returns to home page, but re-enters Research Port. Selects E-Journals, and tries the alphabetical list, selects “A.” Continues to click “next” until s/he comes to American Journal of Political Science. Clicks on link for journal and reads “no electronic full text available” and decides to stop.
2. Research Port, E-Journals, selects “a”, but does not use next to further explore the results list. Student backs up to type in the name of the journal, and then clicks on the link for the journal. Receives the “no electronic full text available” message, and clicks on the link to the catalog. “If I were doing this from home I would start doing different ways, if it is held on an actual database and just not listed as full text, or I might just say I guess it’s not available.” “I don’t understand what this Item Available through UM Catalog message means, but I’m going to click it.” Catalog entry for the electronic resource lists places where it is available online, JSTOR for example. After clicking on a link the student feels the item is only available back to 2003. Student decides one could go to the shelf and finds the article, and finds the call number for the journal.
3. Research Port, first looks under database titles, “a.” Then selects E-Journals, clicks “a” from title list. Scrolls the first set of results, but then leaves Research Port. Goes to Catalog, searches American Journal of Political Science in “title beginning with.” Finds the catalog entry for the journal, and selects the URL for JSTOR, searches the journal with Fraser for author, and part of the title. Finds the article in the results list and finds the full text.
4. Research Port. “I know what confused me the first time I opened this page is it doesn’t have the three of the (choices/search boxes), you have to click on it first, and there’s all this white space.” Looks at locate under Databases, but goes back to titles. Types in the journal title, exact, but when it doesn’t come up student says, “oh wait, this isn’t a database, I can’t do that.” Then tries the political science databases, selects the American politics sub-category. Takes a look at Cross-Search but decides that’s not what s/he wants, goes back to select a database that fits the date of the article and offers full-text. “I don’t like this because I don’t know how to do what I’m used to doing. I’m used to getting a list where I can click as many as I can click, and type in the title and one of them will pull it up, but having to pick one database…I don’t like that. It looked like this Cross-Search thing just did the simple, the WorldCat, and the basic ones.” Student does try to use Academic Search Premier, but receives no results. Checks Simple vs Advanced Search. Goes back to databases. “I don’t like this scrolling either, I like the drop down menu that the old one had.” Re-selects Political science and the American politics sub-category. Tries CQ Public Affairs Collection with no luck. “Now I’m puzzled, because I’m not sure where to go.” Tries PCI Full Text, and locates the article, PCI leads the user to the JSTOR full text version. “That was frustrating, it took too long.”

Suggestions?

1. Feels the current home page has more choices with Databases and E-Journals. If I was unfamiliar with Research Port I wouldn’t know what to find there.
2. I know there are space constrictions on the bottom, but I find that when the links on the bottom are more explicit it’s easier…Like Information for Staff and Divisions, that might take up a lot of space, but it might also…Maybe they’re willing to sacrifice a couple of wrong clicks…Like Phone number of departments in the library
3. I’m not really sure why Reference Shelf doesn’t have a pretty button like Catalog and Research Port. It would make more sense to me to have it be consistent. I was also a little surprised to find that staff and divisions didn’t have a staff and division list like I found in phone numbers.
4. You can go straight to Catalog and Research Port and I think that’s helpful.

What’s missing from the page?

2. Nothing that I can think of. It seems like the things you would need information, and information on your account are here.
3. I think it’s fairly comprehensive.

What do you like best?

1. Site index and How do I…, it’s right at the top and catches your eye.
2. It’s attractive. The Catalog and Research Port are useful, but I haven’t used either extensively enough to know if they're easy to use.
3. Most useful would be the Catalog and Research Port. I would definitely use them more than I would use things like phone numbers, directions and hours, although those are useful as well. It’s nice that you can actually get to that with one click.
4. Catalog and Research Port are right in the middle. On the old page there were four things in the same font, and they’re right in the middle too, but I only used two of them. So, I think this makes it easier; they jump out more.

Comments on Research Port

1. Kept having to return to the top of the page to find the next/previous links.
   I was a little unsure of where to go. I had to try several things before I found what I really wanted. Maybe there could be descriptions at the top of what you would expect to find in that section.

3. I’ve used it a couple of times before, I’ve always found it a little bit confusing to actually get to things. When asked if this version was easier to use the student responded, “a little bit, yeah.” It was easier to get to the lists of the journals, although with the last question I didn’t know to look in JSTOR I was surprised that it didn&3146;t have a list of all the journals that are included in something like JSTOR. It would seem like it would be easier to get to them as opposed to having to go into the Catalog.
4. I couldn’t search more than one database at a time, or at least I couldn’t figure out how to do it. And, I’m not used to political science, so I’m not familiar with the journals at all.
   I would really like to have a link between the Catalog and Research Port instead of having to go back to the home page to go back and forth. I like how it [the page] starts in the middle and your eyes go outward when you’re looking at it.

General Comments.

1. Student is looking for Databases and E-Journals and initially feels like s/he does not see them on the home page.

3. It’s easy to find the Catalog and Research Port, which is could because before you had to click through a couple of pages to find Research Port. I like the tip of the week.
4. When I use the libraries’ page I always use Catalog and Research Port and it’s good that they’re smack in the middle, with the black around them. I know that I didn’t notice the bar across the bottom for a while, probably because it was at the bottom. One thing I know that’s always annoyed me on the regular home page is that you can’t go straight to the Catalog from Research Port and it would be nice if you could. The pictures [on the home page] are kind of cheesy. It’s like they’re there because someone was running around with a digital camera and just felt like taking pictures and they’re like oh these are cute let’s put them on the home page. I just don’t really see the point of them. It looks a little amateur, but the rest of it looks fine.

return to top