Denise E. Agosto and Holly Anderton
Print version (Adobe Reader required)
This article presents a study of source citing in telephone reference service at the twenty-five largest public library systems in the United States and Canada. The results showed that in eighty-six out of the 125 total reference transactions analyzed (68.8 percent of the cases), respondents gave no sources for their answers. The article also discusses a number of additional issues uncovered during the study that are not related to source citing but that have important implications for reference services. The authors conclude that best reference practices are not being followed in many instances of public library telephone reference, and they close with a number of simple suggestions for improving telephone, face-to-face, and digital reference services.
One of the most basic rules for answering reference questions is to cite the source of all information given to users. Source citing is included as one measure of reference excellence in the Reference and User Services Association’s (RUSA’s) Guidelines for Behavioral Performance of Reference and Information Service Providers: the librarian “offers pointers, detailed search paths (including complete URLs), and names of resources used to find the answer, so that users can learn to answer similar questions on their own.”1 Failure to cite sources prevents users from being able to judge the authority of the information they receive and precludes them from being able to return to the sources later to find more information on their own. However, informal observations raised questions regarding how often public librarians cite their sources when answering reference questions via the telephone.
The authors investigated the question of source citation frequency by calling the twenty-five largest public library systems in the United States and Canada and asking a series of five ready reference questions, one question per telephone call, over a period of two months. Although the percentage of correct answers was much higher than previous researchers had generally found in their studies of telephone reference service, the frequency of source citing was surprisingly low.2
This article will present the results of the study and discuss implications for reference practice. It also will discuss additional issues uncovered during the study that are not related to source citing but have important implications for improving face-to-face, telephone, and digital reference services. These issues include librarians’ missing out on the chance to teach users about the nature of modern information; frustration with having to navigate the maze of telephone options employed by the automated messaging systems used by most of the libraries; librarians who did not seem to take the authors’ questions seriously; the infrequency with which respondents asked follow-up questions; respondents who did not identify themselves professionally (leaving the authors to wonder who they were); and librarians who exhibited an alarming lack of confidence in their own answers.
Literature Review
Telephones have played a role in public library work since the late nineteenth century, yet there have been relatively few research publications that have discussed studies of telephone reference service.3 As Kern wrote in a comparison of the history of telephone reference to the development of online chat reference, “The telephone is old technology and its use for library reference services is also long-standing. In many ways it lies forgotten in the literature, a ‘been there, doing that’ service that does not merit fresh reflection.”4
Telephone Reference Accuracy
Among the published research studies that do exist, many have focused on studying rates of reference accuracy. In now-classic studies, Crowley and Childers, and Myers and Jirjees used public and academic library telephone reference services to investigate the question of reference accuracy.5 These studies found roughly a 50 to 60 percent rate of accuracy, which Hernon and McClure famously labeled “The 55 Percent Rule.”6
A number of other studies also have found accuracy rates in the 50 to 60 percent range, such as Paskoff’s more recent study, for which she called fifty-one academic health and hospital libraries and asked a series of six “factual questions.”7 She found that 63.4 percent of the answers were correct, 25.2 percent of the answers were comprised of references to other information agencies, 3.6 percent of the answers were incorrect, and no answer was given in 7.8 percent of the cases.
However, Hubbertz argued that the consistency of “The 55 Percent Rule” is largely a function of the difficulty of the questions asked.8 He called attention to the common practice of eliminating questions shown to have extremely high or extremely low reference response rates, meaning that accuracy rate findings must consequently fall into this middle range. It does appear that most other studies of telephone reference accuracy that have found either significantly higher or significantly lower accuracy rates have used relatively “easy” or relatively “difficult” questions. As an example of a high accuracy rate study, Partin found an 81 percent correct response rate in rural public library telephone reference using relatively easy questions, such as “Can you tell me the address for the Wall Street Journal?” and “What is a nautical mile?”9 As an example of a low accuracy rate study, Dilevko and Dolan called twenty-one public libraries in Canada and asked ready reference questions “dealing with current topics of importance, as reported in major newspapers.”10 They found an accuracy rate of just 34.2 percent, including cases in which referrals to other agencies led to correct answers, and concluded that the extreme currency of the information requested led in part to the low accuracy rate. The questions they used were relatively difficult, such as “Could you give me some information as to whether it is legal to sell the manganese-based gasoline octane booster MMT in Canada?” and “Do you know of any publication that provides a list of abandoned communities (towns, villages, outposts) in Newfoundland?”11
Source Citing in Telephone Reference
There has been less interest in studying source citing than in studying telephone reference accuracy. Those studies that have examined citation practices have generally done so as a by-product of reference accuracy. For example, Partin found “the number of answers that were given without checking any source” to be “disturbing,” but did not provide extensive analysis of citation behaviors.12 Similarly, in their study of reference accuracy, Roger and Goodwin included “source cited or appropriate referral made” as part of the definition of “correctness,” yet they did not report rates of source citing as an independent measure.13
Telephone Reference As a Continuing Service
Despite the growth in popularity of other forms of virtual reference service, such as e-mail and chat, it appears that libraries are continuing to provide telephone reference. Allen and Smith analyzed telephone inquiries for a two-week period at the John C. Hodges Library at the University of Tennessee at Knoxville.14 They found that about one-third of the calls were reference queries, with the remainder being informational and directional queries. This is the same rate of reference versus informational queries that Brown had recorded nearly a decade earlier at the Chattanooga-Hamilton County Bicentennial Library, a medium-sized public library also located in Tennessee.15
More recently, Tenopir surveyed the directors of seventy academic libraries about “changes in their reference services over the last three years and how electronic resources have impacted them.”16 All seventy directors explained that their libraries continue to answer reference questions via the telephone, despite the growth of other forms of virtual reference. It is likely that public libraries also will continue to offer telephone reference on a wide-scale basis.
Study Method
Selecting the Libraries
The authors reasoned that most large public library systems were likely to have more active reference departments than most small public library systems, and that their study would be less of a burden on busier reference departments. Consequently, the twenty-five largest (in terms of populations served) public library systems in the United States and Canada (according to the 2005 World Book Almanac), were selected.17 These twenty-five libraries are listed in the appendix.
The first challenge was to select the phone numbers to call, so the twenty-five library systems’ Web sites were searched for numbers identified as “reference,” “questions,” or “information” lines. Finding an appropriate number often involved extensive searching throughout a large Web site, which proved to be a frustrating waste of time. Seven libraries had no number listed for “reference,” or “information,” or “questions,” so the authors called the main or central library general numbers for these libraries, or the general number for the first branch listed if there was no main or central library. It was especially frustrating to find that many libraries had “Ask a Librarian” pages that enabled users to ask chat or e-mail questions but did not include a number for telephone reference.
Selecting the Questions
The second challenge was to select the test questions. Bunge and Bopp defined the ready reference question as a question that “can be answered quickly by consulting only one or two reference tools.”18 The decision was made to focus on ready reference, as opposed to more involved reference queries, as “Ready reference questions constitute the majority of questions received at most reference desks in public and academic libraries,” and because ready reference questions are more suited to the telephone reference format.19 As a case in point, Kern reported that the University of Virginia’s telephone reference policy states “Normally, telephone [reference] service is appropriate for only factual or referral queries.”20
To use questions similar to the kinds of real questions that real patrons ask via telephone reference, students in a graduate library and information science (LIS) course were asked to provide some of the questions they had asked in the past using public library telephone reference. Five were selected from the resulting list of eight, limiting the selections to those that were true ready reference questions (easily answered using only one or two sources), and selecting questions that represented a range of topics as well as a range of difficulty, although all five seemed to be relatively easy to answer, reflective of the overall simplicity of ready reference questions offered by the students.
The questions were (listed in the order asked):
- Can you tell me when Valentine’s Day is?
- Who is the current governor/premier (of the state/province where the library is located)?
- What is the population of Montana?
- In which state is the Southern Poverty Law Center (SPLC) located?
- What is the French word for “chiropractor?”
Each week for five consecutive weeks at different times during the business day, the authors called the libraries and asked that week’s question. After receiving an answer, clarification was not requested; the respondents were simply thanked and the call was ended, even in cases in which the answer was known to be incomplete or incorrect. This was done in order to reduce respondent reactivity, assuming that a real questioner would not know the correct answer to his or her question.
Results
Aggregate Results
Of the aggregate 125 reference transactions, 117 (93.6 percent) of the answers provided were correct, 7 (5.6 percent) were incorrect, and the accuracy of 1 (0.8 percent) was unclear. These results represent a much higher frequency of correct answers than most previous researchers have found in their studies of telephone reference service, with previous accuracy rates hovering somewhere within the 50 to 70 percent range.211 It is likely that the simple nature of the questions led to this higher rate of accuracy; were more difficult questions posed (as with most previous researchers), the accuracy rate would probably have been significantly lower. As Hubbertz wrote, “Clearly, the overall score [reference question accuracy rate] is a function of the questions asked: easier questions and the scores go up, harder questions and the scores go down.”22 The questions that were used were real questions that had been asked by real patrons, so perhaps much of telephone reference work involves answering these types of simple questions. (A larger, more formalized study of the common types of telephone ready reference questions would be necessary to be sure.) In any case, this high accuracy rate indicates that, in terms of providing accurate answers, telephone reference can be a successful format for answering simple ready reference questions.
On the other hand, the frequency of source citing was disappointingly low, with complete citations provided just seven times (5.6 percent) out of the 125 total reference transactions, and with partial citations given an additional thirty-one times (24.8 percent), and an incorrect citation given once (0.8 percent). This means that in eighty-six (68.8 percent) of the 125 cases, no source was mentioned whatsoever. These figures are especially poor considering that the authors’ standards for deeming a citation “complete” were relatively low. A “complete citation” was defined as the minimum information required for users to be able to find the information again on their own:
- For a Web site–the complete URL (title and sponsor of the site not required).
- For a digital database–the database title and the title and year of the specific item (author, publisher, page number not required).
- For a print resource–the title and year (author, edition, page number, publisher, and place of publication not required).
Had the authors defined “complete citation” as including all of the elements necessary for inclusion in a formal bibliography, the results would have been much worse; a complete citation rate of 0.0 percent would have been found.