Student Feedback on Federated Search Use, Satisfaction, and Web Presence: Qualitative Findings of Focus Groups

The majority of articles published on federated searching in academic libraries are case studies. Many describe usability tests of federated searching in controlled settings. The tests typically involve asking participants to talk while conducting sample searches. Ponsford and vanDuinkerken describe results of “think out loud” sessions at Texas A&M University. Ponsford and vanDuinkerken conducted two rounds of testing, the first with forty-seven volunteers and the second with fifteen. Participants reported having found citations they would not have otherwise located with the native interfaces they regularly used. We would hear this same comment many times in our research. Pons-ford and vanDuinkerken found that some participants expected federated searching to function like Google, particularly in ranking results by relevance. Experienced searchers participating in the study expressed a desire for more complex searching options, including the ability to limit by format.⁶ The balance between simple and complex searches—between serving users wanting a few pertinent results and others seeking everything on their topic—is a theme throughout much of the federated search literature and one with which we have struggled at ISU.

Wrubel and Smith describe a similar “think aloud” study involving eighteen students from four campuses in the University System of Maryland. Some participants expressed frustration with the response time of the federated search engine compared with the response time of Internet search engines. As in the Texas A&M study, some participants expected results to be displayed in order of relevance with the resource type identified (e.g., article versus book).⁷

Several case studies have gone beyond recommendations for configuring federated search interfaces to include general feedback from research participants. In soliciting feedback at Carnegie Mellon University, George asked her six participants to complete a brief questionnaire after completing think-aloud sessions. Some participants had trouble understanding link names, icons, and system error messages.⁸ Belliston, How-land, and Roberts conducted an extensive usability study at three Brigham Young University campuses involving ninety-five undergraduate students. After completing sample searches using both the federated search engine and native interfaces, participants completed a four-question survey of their satisfaction with federated search. Participants were, on average, 17 percent more satisfied with results from their federated search than with their search of native interfaces, and 70 percent preferred using federated search. Findings of this quantitative study compare closely to findings of our qualitative research.⁹

Only a few studies have reported feedback from actual federated search users (as opposed to participants in controlled studies). All of these studies have used surveys to solicit feedback. Feedback has ranged from mixed to positive with qualifications. Tang, Hsieh-Yee, and Zhang surveyed twenty-two library science students who were users of the federated search engine at the Catholic University of America. Approximately seven in ten participants expressed a favorable yet qualified view of the tool. Many described the tool as “useful but complex and hard to figure out.” Students viewed the federated search tool as a way to locate full-text documents and expressed a desire to learn how federated search works. The authors conclude that federated search interface designers need to make system operations more transparent, a desire echoed by some of our research participants.¹⁰ Lampert and Dabbour surveyed a broader population of federated searchers at California State University at Northridge. The authors posted a link on the library website to an online survey consisting of sixteen close-ended and two open-ended questions. The majority of respondents were graduate students who had received formal library instruction and rated their research skills as good or very good. Of the eighty-eight respondents, 70 percent felt that native interfaces were very important to their research needs compared to 61 percent who felt that federated searching was very important to their work. Perhaps this difference reflects respondents’ greater familiarity with native interfaces at the time. Thirty-eight of the eighty-eight participants offered open-ended comments; 53 percent were positive, 21 percent were mixed, 8 percent were neutral, and 18 percent were negative. Also, 57 percent of respondents felt that training from a librarian was necessary to use the system effectively.¹¹

Few articles on federated searching in academic libraries have addressed the broader questions of where and how federated search engines should appear on library websites. Boock, Nichols, and Kristick mention a decision at Oregon State University to “push the envelope” by placing a customized search box on the library homepage with a link to information about the tool, perhaps wondering if users (or librarians?) would find that location appropriate.¹² Boock, Nichols, and Kristick did not probe users’ reaction to the issue in their survey. The most extensive discussion of how federated search engines appear on academic library websites is a 2007 Robbins and McCain article. The authors evaluated websites of fifty Association of Research Libraries (ARL) members offering federated searching. They looked specifically at how federated searching is explained to users and how users are guided in their searching. The resulting article is a snapshot of academic library practices. The article ends with recommendations for naming federated search engines, providing instruction and help, and publicizing the tool.¹³ While a valuable contribution to federated search research, it reflects the perspective of the librarian rather than the user. It is the user perspective that we sought in our research.

Method

The purpose of this study was to learn about patrons’ use of and satisfaction with Search It and especially to gather their ideas on how to incorporate Search It into the library website. To probe more deeply into users’ reactions to these issues, Milner Library’s federated search committee decided to conduct focus groups. Focus groups are ideal for gathering a range of viewpoints and generating ideas. While they provide valuable qualitative information, they do not provide quantitative data. Focus Groups by Krueger and Casey was the main guide for the planning, execution, and analysis of this study.¹⁴ Another useful resource was Von Seggern and Young’s 2003 article “The Focus Group Method in Libraries: Issues Relating to Process and Data Analysis.”¹⁵

The committee developed a series of questions (see appendix) that addressed the three main issues (use, satisfaction, and website integration) as well as a few wrap-up questions.

To encourage homogeneous groups and increase participants’ comfort, participation was limited to students. Participation was further limited to students who had used Search It previously, even just once, so participants would be able to provide informed feedback regarding their use and satisfaction.

Volunteers were recruited via four different methods. Flyers were posted throughout the library, and an announcement was posted on the library homepage. Because the study targeted students who had previously used Search It, brief announcements were also posted by the Quick Search boxes and on the Advanced Search page. At ISU, subject librarians can display messages in the university portal for students and faculty in their departments. Several librarians representing a variety of disciplines posted a brief announcement for the students in their departments. Focus group volunteers e-mailed one of the committee members, who selected the participants on a first-come, first-served basis depending on volunteer availability.

Pages: 1 2 3 4 5 6 7