Print version (Adobe Reader required)

This exploratory study investigated the help-seeking preferences of library users at two large urban universities in Toronto. Reference desk and virtual reference users were compared in terms of their perceptions of the options now available for obtaining reference help. The premise for the study was based on the assumption that a reasonable exposure to newer reference services, such as chat and e-mail, had occurred, therefore allowing for an examination of emerging preferences for different types of services. Surveys were distributed to both reference desk and virtual reference users asking seven core questions exploring use and preference for reference services as well as habits and preferences for study location (in library, off campus, etc.). The results suggest that the reference desk continues to be the most popular method of getting help in the library, but virtual reference satisfies a niche for users who prefer to work outside the library. Those who use virtual reference tend to perceive their options for getting help differently from other users. Virtual reference users do not perceive virtual reference as a novelty or as a marginal service, but see it as a significant service option. In addition, the results show that virtual reference services may have a special appeal to graduate students since graduate students seem more likely to conduct their research outside the library. The study concludes with recommendations for planning and for future research.

“He not busy being born is busy dying.”

—Bob Dylan

Since the early 1990s, reference services have been in a period of decline and rebirth. The term “transitional” is often used to describe the service culture, the processes, and the technologies involved in providing reference assistance. Along with staggering increases in digital content, we have seen the emergence of a generation of new students who have grown up “native” in a technologically intense world, the rise of distance education and distributed course delivery, the development of the learning commons model; an eclectic student body made up of more nontraditional learners; and we have seen the decline in use of traditional reference services.¹ Prensky, in his essay “Digital Immigrants, Digital Natives,” argues that our era is a uniquely pressured one:

Today’s students have not just changed incrementally from those of the past, nor simply changed their slang, clothes, body adornments, or styles, as has happened between generations previously. A really big discontinuity has taken place. One might even call it a “singularity”—an event which changes things so fundamentally that there is absolutely no going back. This so-called “singularity” is the arrival and rapid dissemination of digital technology in the last decades of the 20th century.²

Reference librarians have considered and studied these wide-ranging transformations—the demographics, technological pressures, and the changing educational climate—and have experimented with and adopted new approaches to service. As Crowe noted, “While WHAT reference librarians do is not so different—we still assist users in finding and evaluating information, provide instruction in using resources, and select materials—HOW we do it, however, and the tools and resources we use have changed dramatically.”³

One of the most significant developments in reference services these past several years has been the emergence of real-time virtual reference (VR). VR for the purposes of this study is defined as a synchronous, real-time exchange “where patrons employ computers or other Internet technology to communicate with reference staff, without being physically present.”⁴ The terms “VR” and “chat reference services” will be used interchangeably. The emphasis in our study is on real-time or synchronous services (as opposed to other electronic services by asynchronous methods, such as e-mail reference).

VR services generally began to emerge in the early 2000s. The advent of VR was greeted by much of the library community with great optimism and a spirit of experimentation. More recently, the enthusiasm for chat services has been tempered by persistent technological challenges and questions regarding cost-effectiveness. At the same time, the community has matured to grapple with issues concerning the development of service standards and performance measures, as well as calls to adopt a comprehensive research agenda.⁵ One particularly noteworthy sentiment that struck a chord was voiced by Steve Coffman, one of the early and influential proponents of VR, when he questioned the ongoing relevance of synchronous Web-based services in late 2004.⁶ An important debate has emerged regarding the efficacy of VR compared to telephone, e-mail, and hybrid services.

We have been involved in a VR collaboration since 2001 among three Ontario universities. Catering to a student body of close to ninety thousand, we have witnessed the service grow in popularity. Despite the popularity of VR at our universities, questions regarding its cost-effectiveness persist. We continue to face questions about such issues as the length of time required to answer questions and the inconsistent performance of software and staffing models.

Like most North American academic libraries, we had experienced gradual declines in usage at our reference desks since the early 1990s. Because of the significant changes of the past decade, the decision to introduce VR services was, from our perspective, a logical, appropriate, and progressive response. A successful pilot phase segued into a relatively stable service. After four years of both e-mail and chat services, we felt the time was ripe to consider reference services in a comparative light and begin to explore, through surveys and other means, answers to the essential question, what is the best combination of services for our users when they need help? We were interested in “looking more closely at how users are dealing with their information problems and how they get help from reference librarians in technological environments.”⁷

Our study represents a preliminary exploration of the help-seeking preferences of a student body that increasingly prefers to be off-site using digital resources, away from the library, and for a variety of reasons is less likely to use the physical reference desk. Specifically, we were interested in exploring how users prefer to get help when on campus versus off campus, their preference for location when doing research, and the differences between reference desk users and users of our VR services. Basically, we were concerned with contributing to “informed decisions about space, services, and resources [by not only understanding] … the types of information that are being sought, but also the environment that the user is most comfortable [with]—either physical or virtual or some combination.”⁸

Literature Review

Since the early 2000s the burgeoning literature relating to VR has for the most part taken the form of case studies, commentary, and anecdotes.⁹ The empirically based literature is small but growing. A review of the empirical literature indicates that few studies have been conducted to tell us how changes in technology have informed or should inform reference services and, specifically, how these changes in technology have affected users’ help-seeking behaviors and preferences.

Massey-Burzio intimated this as far back as 1998 and called for more responsiveness “to actual user needs and preferences … [i]nstead of wasting time insisting on the value of delivering reference services the way we always have.”¹⁰ Ten years later, the study of user preferences within the reference service milieu has continued to receive little attention. Library users have ever-increasing amounts of digital content at their fingertips, and many studies show they prefer this format over print.¹¹ That much we know. However, the literature provides a limited view of what users prefer to do in this environment when they need assistance. There have been no connections made in studies between preference for format and preference for obtaining assistance.

Print version (Adobe Reader required)

Many library customers’ questions never reach designated service points such as circulation and reference desks. These questions may be addressed to personnel untrained in customer service such as student shelving staff in an academic library. This article presents data from a 2005 study investigating where and when shelvers received questions (and what types of questions they received) in Newman Library at Virginia Tech. Results showed that these students primarily received directional and item-location questions. Follow-up workshops helped shelvers improve their ability to accurately refer questions when needed, and to increase their accompaniment rate when answering customers’ queries.

For most of their existence, libraries have offered reference and information desks to answer their customers’ questions. For probably just as long, customers have had questions that did not reach these designated service points—either the customers never asked their questions, or they asked someone who was not a designated reference provider. As libraries consolidated service points, more spaces in the library became barren of designated spots for asking questions. Newman Library at Virginia Tech, like many research-sized libraries, has floors with no apparent place to ask for help. In the absence of service points, have customers found somebody else to ask?

In an effort to account for all questions asked in the library, the Newman Library shelving unit began asking its student workers in October 2003 to count each question they received. This count has shown that student shelvers, who received no customer service training, answered more than fifteen hundred questions in both the 2004–05 and 2005–06 academic years.

The authors of this study believed they needed to respond to this large number of questions. Before a response could be fashioned, though, the investigators needed to discover the details about the questions being asked. The investigators explored the types of questions asked of shelvers, and when and where those questions are asked. They also attempted to measure the effectiveness of shelvers in answering questions. Results of initial studies prompted the investigators to hold customer service workshops for students. Another round of data gathering followed to examine if the workshops had any effect.

Background

The University Libraries of Virginia Tech serve a population of approximately 22,000 undergraduate students, 6,000 graduate students, 3,000 faculty, 3,500 staff members, and are open to local and state residents. The library system includes one main building (Newman Library), three smaller branch libraries, and a remote high density storage building with a total collection exceeding two million volumes.

Newman Library consists of five public stacks floors spread over 200,000 square feet. Only two of the five floors, floors 1 and 4, offer service points. The first floor includes a reference/help desk in the building lobby and a desk for circulation/reserve functions. An additional reference/help desk is located on the fourth floor, close to an entrance from an adjacent building. Both reference desks are staffed during all operating hours of the building—7:30 a.m. to midnight Monday–Thursday, 7:30 a.m. to 8 p.m. Friday, 9 a.m. to 8 p.m. Saturday, and noon to midnight Sunday. Reference staff on the first floor can only accompany library customers to other parts of the building if two staff members are on the desk; the fourth floor reference staff person must remain at the desk. A photocopy service desk is also located on the fourth floor, although its primary function is to assist with customer copy needs. As indicated in table 1, floors 2, 3, and 5 offer no service points, but house significant parts of the Newman collection.

The Shelving Unit of Newman Library consists of three full-time employees and thirty-five to fifty-five part-time student employees, depending on the academic semester. The three full-time workers, long-term employees familiar with the collection and policies of the library, have received multiple customer-service training opportunities in prior years, so the investigators focused their study on student employees. Student shelvers include both undergraduate and graduate students, and both domestic and international students. Operating hours vary for the Shelving Unit, but usually run from 8 a.m. to 9 p.m. Monday through Thursday, 8 a.m. to 5 p.m. Friday, and noon to 9 p.m. Sunday.

Unrelated to the investigators’ study, the Virginia Tech Libraries began compiling data on the number of questions shelvers receive in October 2003 in an attempt to make sure that all questions asked in the library are counted. Shelvers place a tick mark on their shelving slips for each question received. The student shelver supervisor compiles and reports the totals monthly.

During the first full academic year of data collection (2004–05) the shelvers recorded 2,172 questions. In 2005–06, shelvers recorded 1,522 questions. The investigators believed these numbers to be significant amounts.

Literature Review

For as long as students have worked in academic libraries, publications have offered advice on how to train them. A 1995 issue of the Journal of Library Administration, titled “Libraries and Student Assistants: Critical Links,” focused exclusively on the topic, and Black’s introduction to the issue included this assessment: “Student workers are commonly the first individuals seen by the user and their interactions frequently form the basis for patron opinion of the library.”¹ White’s 1985 article provides a historical overview of the expanding role of the part-time student employee in the library. Increasingly, students have not only shelved and checked out books, but have also provided information services. White finds the beginning of this trend in the 1970s, a “decade of increased reliance upon student assistants for more responsible and demanding job performances.”²

Of particular interest to this research was the University of New Hampshire’s 1973 initiative to place reference aides in the stacks to provide assistance both in locating specific materials and in referring questions to appropriate service points.³ Chosen from undergraduates already working in the library, the students worked two-hour periods during hours of heaviest library use. The reference aids, with identifying badges, roamed the stacks and approached people to ask if they needed help. Over a period of ten weeks, the aids contacted 4,436 people and answered 2,411 questions. Although the students recorded questions in one of three categories—direction (questions that required a simple answer about the location of material), referred (those inquiries that required the help of the reference librarian), and search (simple reference questions that student aids could answer after a short search)—Tebbetts and Pritchard did not indicate the most frequent types of questions.

When the topic was student shelvers, authors focused on how to ensure the students are shelving materials properly. However, shelvers are also among the most visible library workers. Spending most of their time in public stacks in the library, shelvers are convenient and easily approachable for customers who have questions.

Swope and Katzer conducted a study at Syracuse University’s Carnegie Library in 1973 that explored whether library users had questions, and if they did, whether they would ask a librarian. Of 119 randomly selected users, forty-nine had questions, but only seventeen of those would ask a librarian. Most important to this research, “of the thirty-two ‘non-askers,’ twenty-three indicated that they would ask a fellow student for aid.”⁴ Gregory echoes the idea that students may be more comfortable asking questions of their peers. His 1995 article suggests that peer-to-peer interaction often facilitates communication, meaning student employees are frequently the library’s best hope for educating fellow students on use of the library.⁵

Print version (Adobe Reader required)

Subject searching in the OPAC is the most problematic of all search types, causing far greater difficulty for patrons than keyword searching and known-item searching. This study combines two methodologies—transaction log analysis and user observation interviews—to examine the reasons for patrons’ failure to use subject searching effectively. The transaction log analysis shows that patrons rarely utilize correct and complete subject terms and that they retrieve zero results in almost half of their searches. Furthermore, the user observation interviews reveal that users generally are unaware of the many tools and services that librarians have created to assist them with subject searching, and that asking a librarian for help simply does not occur to them. Even when searchers do locate and employ subject terms, the subject terms appear not to help them very much: Analysis of observed searches reveals almost no correlation between finding a subject term and judging a subject search to be successful. The authors discuss the potential for further research on “just-in-time” instruction, online instruction, and “tagging” as possible strategies to improve patrons’ searching success.

The question is inescapable in librarians’ professional reading: will the rise of keyword searching spell the end of controlled vocabulary? A recent article in American Libraries coined the phrase “search fatigue” to describe the “feeling of frustration and dissatisfaction” that users suffer when they spend hours looking in databases for information that they know ought to be there, but that they cannot find.¹ According to the author, Jeffrey Beall, “The chief cause of search fatigue is a reliance on keyword searching” as opposed to controlled vocabulary searching.² In the same issue of American Libraries, ALA President Leslie Burger comments on Yahoo! Answers, a virtual reference service in which anyone can answer any question posed. As Burger notes, “There is no way to determine if [sic] the information is accurate, reliable, or authoritative, but people seem not to care. … These days, everyone can be an information expert.”³

Yet reference librarians are aware that patrons doing keyword searches in online catalogs do not find the best results. In fact, they frequently retrieve unhelpful result sets of zero, or they retrieve far too many results to be useful. Some of these patrons then consult librarians and are guided to subject terms and relevant materials. But others, surely, simply give up, concluding that the library catalog contains nothing relevant to their search. For academic librarians, patrons’ poor search skills are particularly worrisome because academic librarians are charged with developing students’ information literacy. According to the Association of College and Research Libraries’ (ACRL) information literacy standards, the “information literate student … selects controlled vocabulary specific to the discipline or information retrieval source.”⁴ In the case of the OPAC, this means that the information-literate student should be able to select appropriate subject terms. But anecdotal evidence from front-line reference librarians suggests that most students are unaware of the existence of subject terms, let alone capable of using them effectively. As librarians know, “the great advantage of metadata is that it compensates for all the weaknesses of keyword searching.”⁵ Therefore, teaching students about controlled vocabularies is an important job for academic libraries. Yet this kind of instruction is increasingly challenging in an environment in which keyword searching is so pervasive that “everyone can be an information expert”—or at least think that he or she is an expert.⁶

This study investigates patrons’ subject catalog searching behaviors at the University of Oklahoma Libraries. Two methods were used to gauge the success of subject searches: an analysis of the OPAC’s transaction logs and a series of observation interviews in which students were asked to perform a series of subject searches on the OPAC. The transaction log analysis enabled the authors to study a large number of subject searches and evaluate their success by asking questions such as:

• How many subject searches yielded zero results?
• How many subject searches yielded an unhelpfully large number of results?
• How many subject searches used correct and complete subject terms?

The observation interviews, on the other hand, allowed the authors to ask students qualitative questions about their searching, such as:

• Are you satisfied with these results?
• If not, how would you change your search strategy?
• Would you use these results, or would you look elsewhere for the information you need?

By combining these two methods, the authors were able to gather information that would be impossible to obtain by using either method alone.

Literature Review

Keyword searching is on the rise, thanks to the popularity of resources such as Google, Yahoo!, and Wikipedia. “Keyword searching is extremely popular and is essentially beginning the process of replacing metadata-enabled searching, such as online catalogs.”⁷ Some librarians are even questioning whether it is cost-effective to do subject cataloging at all, given that most patrons do not seem to use subject searching. As early as 1995, the Association for Library Collections and Technical Services addressed this issue in a program titled “Crisis in Subject Cataloging and Retrieval.”⁸ During this program, Arlene Taylor identified several elements of the coming crisis, including “an administrative push to cut back or eliminate subject cataloging … [due to] the availability of keyword searching, which many people think is sufficient.”⁹ These elements of “crisis” have only intensified in the intervening thirteen years, during which the availability of keyword searching resources has increased exponentially due to the ubiquitousness of the Internet and its many freely available search tools.

It is well documented in the research literature that patrons “do not understand the complexities of bibliographic structures” and that “users are normally more successful in conducting known item [author or title] searches than subject searches.”¹⁰ Many studies have employed OPAC transaction log analysis to examine the “success” of users’ subject searches.¹¹ Larson’s 1991 analysis shows a decline in the frequency of subject searching and a concomitant increase in known-item searching over the time period 1982–1988. More recently, Yu and Young also report a decline in the success of subject searching over the time period 2000–2002 and attribute this to the increasing prevalence of Web-based search engines and users’ expectations that OPACs will perform like Web-based search engines.¹²

Researchers apply various criteria to transaction log data to judge the success or failure of patrons’ subject searches. Although researchers generally agree that most searches retrieving zero results are unsuccessful, the upper limit varies tremendously. For Larson, a “successful” search retrieves between one and twenty records; for Hildreth, the upper limit is ninety, and for Yu and Young, the upper limit is one hundred.¹³ When success is defined in this “numeric” way, analysis of transaction logs is a simple way to determine the frequency of successful subject searching.

Print version (Adobe Reader required)

This article describes a multiple-methods approach to examining and enhancing the quality of walk-in service points at a major university library. Selected methods included focus groups, benchmarking, surveys, transaction analysis, activity mapping, and secret shoppers. The results of the study generated many recommended enhancements, including the consolidation of service desks.

At Villanova University, two forces converged to initiate this continuous quality improvement project at Falvey Memorial Library. The first of these factors was an ongoing interest on the part of the library staff in maximizing services to patrons, both through technology and personal points of contact. The second was Villanova’s continuous quality improvement program, a well-established initiative introduced in 1993 and long spearheaded by the Office of Planning, Training, and Institutional Research (OPTIR). In recent years, several studies have been completed by and for the library dealing with the physical footprint of the library as well as the responsiveness of staff to the needs of their various patrons—students, faculty, staff, alumni, members of the surrounding community, and outside scholars. The present study located precisely the various points of patron service and delved deeper into their functions.

Villanova University and Falvey Memorial Library

Villanova University, a Roman Catholic institution based on the teachings of St. Augustine, is located in a pleasant suburban setting just twelve miles outside of Philadelphia. It is home to more than ten thousand students and more than five hundred faculty members. The university consists of four main colleges (College of Liberal Arts and Sciences, Villanova School of Business, College of Engineering, and College of Nursing) and a School of Law. Falvey Library has a staff of about sixty-five, including twenty librarians. The library offers patrons a place for social networking and studious collaboration with wireless network access, a twenty-four hour study lounge and coffee shop, group study rooms, laptop loans, and a rich complement of electronic and digital library resources in a medium-sized academic library environment.

A unique aspect of the Villanova Quality Improvement (VQI) model is its emphasis not only on the application of time-tested work process improvement tools, but on the integration of its efforts with the mission of the university. At Villanova, there are currently more than twenty-five VQI teams involving more than two hundred colleagues from practically all of the offices of the university. Some have a special university-wide goal such as the Incentives and Recognition Team or the Environmental Team, but the vast majority of teams are located within departments with the tri-fold charge of work process improvement, building community, and offering community service. VQI is coordinated by OPTIR. OPTIR staff are experienced in qualitative and quantitative research methodologies, and three have been certified in Lean Six Sigma, a popular system for indentifying and reducing waste and increasing the effectiveness of work functions.

In an effort to seek its own answer to “library as place” or, as Davenport advocates, “in a networked world … place as library,” Falvey seeks to steer a path beyond access versus ownership issues to provide a comfortable and collaborative base from which patrons can share ideas, conduct research, and receive assistance and instruction from within the library, within the colleges, and at a distance.¹ Reference and instruction services are becoming academic integration services with liaison information literacy teams; programming of library events has a programming coordinator; technical services is now resource(s) management; “access” means access to a host of services besides checking out and reserving books; and there is a growing need for even more skilled budget management and data-driven assessment coordination past standard monthly statistics.

Literature Review

Even several years ago, Ludwig and Starr wrote that library leaders were predicting “expanded roles for libraries as places where all members of the community can come together, as purposed and expanded learning centers, and as places for intellectual pursuits.”² Between now and 2025, “Libraries will continue to include areas designed for both group work and privacy. … Libraries will create a multi-functional desk that combines circulation and reference.”³ Church reminds us that locating services together is not merely a merging of services but an interweaving of cultures and that “what is most important is fluidity of ideas.”⁴ In their study, Harer and Cole concluded that a focus on students, faculty, and stakeholders is the most important aspect of ensuring quality in programs and services.⁵ Dempsey suggests that “the question we need to address is not the integration of library resources with each other; it is the integration of library services with the learning and research behaviors of users.”⁶ It is within this context that an improvement study was undertaken.

Through an experience similar to the one at the University of Arizona Library in the early 1990s, Falvey Memorial Library underwent a reorganization of staff and functions with a commitment to focus on the customer and to making more data-based decisions. To that end we chose to employ continuous quality improvement methodology, which, as described by Larson, “provides a system for gathering all of the data related to a process, analyzing it, and developing solutions based on customers’ requirements.”⁷ We decided to employ a wide variety of methods to gather the data we needed to begin the quality improvement process. The literature provides a few similar examples of a multi-pronged approach to collecting service-oriented data. Winkworth describes the Standard Conference of National and University Libraries (SCONUL) Benchmarking Pilot Project in the United Kingdom, which focused on “advice desks” and “counter services” and utilized site visits, mystery shoppers, staff questionnaires, and customer surveys.⁸ The Arapahoe Library District in Englewood, Colorado, employed secret shoppers, focus groups, and the quality walk in their year-long service evaluation.⁹ However, the Falvey study was particularly ambitious because it was composed of nine different data collection techniques completed in a six-month timeframe, some of which required collection over an extended period of time. This is where the VQI initiative became invaluably linked to our efforts.

The Present Study

In August 2005, OPTIR issued a special invitation to all departmental VQI teams reminding them that, through the VQI program, OPTIR staff would assist departments in defining and enhancing work processes. Interested departments were requested to contact OPTIR if they wished to pursue this opportunity. The leader of the VQI team shared this invitation with the library team and subsequently with the director of the library. All concurred that it would be helpful to invite OPTIR staff members to work with the library staff in studying and improving the physical layout of the library and the service points of contact.

A special study team was convened in January 2006. The team was composed of eight members of the library staff from key areas: Media Technologies, Library Access, Budget and Administrative Services, the information desk, Reference and Consulting, the periodicals/media desk, Assessment/Special Projects, and Central Library Administration.

Library personnel engaged in the process included librarians, technical staff, and paraprofessional support staff. Three members of the OPTIR team rounded out the study team, which was facilitated by OPTIR’s director of training and organizational development. On average, the team met every other week from January through June 2006; special homework assignments were often given to sub-teams and completed between meetings.