Subject Searching Success: Transaction Logs, Patron Perceptions, and Implications for Library Instruction

Karen Antell and Jie Huang

Print version (Adobe Reader required)

Subject searching in the OPAC is the most problematic of all search types, causing far greater difficulty for patrons than keyword searching and known-item searching. This study combines two methodologies—transaction log analysis and user observation interviews—to examine the reasons for patrons’ failure to use subject searching effectively. The transaction log analysis shows that patrons rarely utilize correct and complete subject terms and that they retrieve zero results in almost half of their searches. Furthermore, the user observation interviews reveal that users generally are unaware of the many tools and services that librarians have created to assist them with subject searching, and that asking a librarian for help simply does not occur to them. Even when searchers do locate and employ subject terms, the subject terms appear not to help them very much: Analysis of observed searches reveals almost no correlation between finding a subject term and judging a subject search to be successful. The authors discuss the potential for further research on “just-in-time” instruction, online instruction, and “tagging” as possible strategies to improve patrons’ searching success.

The question is inescapable in librarians’ professional reading: will the rise of keyword searching spell the end of controlled vocabulary? A recent article in American Libraries coined the phrase “search fatigue” to describe the “feeling of frustration and dissatisfaction” that users suffer when they spend hours looking in databases for information that they know ought to be there, but that they cannot find.¹ According to the author, Jeffrey Beall, “The chief cause of search fatigue is a reliance on keyword searching” as opposed to controlled vocabulary searching.² In the same issue of American Libraries, ALA President Leslie Burger comments on Yahoo! Answers, a virtual reference service in which anyone can answer any question posed. As Burger notes, “There is no way to determine if [sic] the information is accurate, reliable, or authoritative, but people seem not to care. … These days, everyone can be an information expert.”³

Yet reference librarians are aware that patrons doing keyword searches in online catalogs do not find the best results. In fact, they frequently retrieve unhelpful result sets of zero, or they retrieve far too many results to be useful. Some of these patrons then consult librarians and are guided to subject terms and relevant materials. But others, surely, simply give up, concluding that the library catalog contains nothing relevant to their search. For academic librarians, patrons’ poor search skills are particularly worrisome because academic librarians are charged with developing students’ information literacy. According to the Association of College and Research Libraries’ (ACRL) information literacy standards, the “information literate student … selects controlled vocabulary specific to the discipline or information retrieval source.”⁴ In the case of the OPAC, this means that the information-literate student should be able to select appropriate subject terms. But anecdotal evidence from front-line reference librarians suggests that most students are unaware of the existence of subject terms, let alone capable of using them effectively. As librarians know, “the great advantage of metadata is that it compensates for all the weaknesses of keyword searching.”⁵ Therefore, teaching students about controlled vocabularies is an important job for academic libraries. Yet this kind of instruction is increasingly challenging in an environment in which keyword searching is so pervasive that “everyone can be an information expert”—or at least think that he or she is an expert.⁶

This study investigates patrons’ subject catalog searching behaviors at the University of Oklahoma Libraries. Two methods were used to gauge the success of subject searches: an analysis of the OPAC’s transaction logs and a series of observation interviews in which students were asked to perform a series of subject searches on the OPAC. The transaction log analysis enabled the authors to study a large number of subject searches and evaluate their success by asking questions such as:

• How many subject searches yielded zero results?
• How many subject searches yielded an unhelpfully large number of results?
• How many subject searches used correct and complete subject terms?

The observation interviews, on the other hand, allowed the authors to ask students qualitative questions about their searching, such as:

• Are you satisfied with these results?
• If not, how would you change your search strategy?
• Would you use these results, or would you look elsewhere for the information you need?

By combining these two methods, the authors were able to gather information that would be impossible to obtain by using either method alone.

Literature Review

Keyword searching is on the rise, thanks to the popularity of resources such as Google, Yahoo!, and Wikipedia. “Keyword searching is extremely popular and is essentially beginning the process of replacing metadata-enabled searching, such as online catalogs.”⁷ Some librarians are even questioning whether it is cost-effective to do subject cataloging at all, given that most patrons do not seem to use subject searching. As early as 1995, the Association for Library Collections and Technical Services addressed this issue in a program titled “Crisis in Subject Cataloging and Retrieval.”⁸ During this program, Arlene Taylor identified several elements of the coming crisis, including “an administrative push to cut back or eliminate subject cataloging … [due to] the availability of keyword searching, which many people think is sufficient.”⁹ These elements of “crisis” have only intensified in the intervening thirteen years, during which the availability of keyword searching resources has increased exponentially due to the ubiquitousness of the Internet and its many freely available search tools.

It is well documented in the research literature that patrons “do not understand the complexities of bibliographic structures” and that “users are normally more successful in conducting known item [author or title] searches than subject searches.”¹⁰ Many studies have employed OPAC transaction log analysis to examine the “success” of users’ subject searches.¹¹ Larson’s 1991 analysis shows a decline in the frequency of subject searching and a concomitant increase in known-item searching over the time period 1982–1988. More recently, Yu and Young also report a decline in the success of subject searching over the time period 2000–2002 and attribute this to the increasing prevalence of Web-based search engines and users’ expectations that OPACs will perform like Web-based search engines.¹²

Researchers apply various criteria to transaction log data to judge the success or failure of patrons’ subject searches. Although researchers generally agree that most searches retrieving zero results are unsuccessful, the upper limit varies tremendously. For Larson, a “successful” search retrieves between one and twenty records; for Hildreth, the upper limit is ninety, and for Yu and Young, the upper limit is one hundred.¹³ When success is defined in this “numeric” way, analysis of transaction logs is a simple way to determine the frequency of successful subject searching.

Pages: 1 2 3 4 5