Developing a Model for Reference Research Statistics: Applying the “Warner Model” of Reference Question Classification to Streamline Research Services

Harry C. Meserve, Sandra E. Belanger, Joan Bowlby, and Lisa Rosenblum

Print version (Adobe Reader required)
The merger of an academic library with the main branch of a large city’s public library in 2003 required a new method for determining customer–patron transactions. The Warner model, previously reported in RUSQ in 2001, was adopted and used to investigate the possibilities for developing tiered reference, adjusting staffing levels, and improving service in a merged reference unit. The adopted model is recommended to other libraries that want to develop effective tools for analyzing reference services.

The new Dr. Martin Luther King Jr. (MLK) Library, which opened in August 2003, was a collaborative project of the City of San Jose and San Jose State University (SJSU). Two libraries, the University Library of SJSU and the MLK Library—the main branch of the San Jose Public Library (SJPL) system—merged to create a new entity. The six-year planning process has been documented on the library’s website and in a number of articles, covering the vision of one reference department, the principle of “economies of scale” as they apply to merged units, and creating a collaborative library from two different institutions.¹ In a broader context, other articles discuss the library as an example in the general discussion of the library as place and consider the implications of how different work and service cultures are brought together in a new institution.²

There are merged and unmerged units in the new library. The four merged units in the new library are Access Services (including Circulation), Information Technology, Technical Services, and Reference. Data is gathered in the same way by all public service points, including those that remained unmerged: the public library’s Youth Services, General Collections, and the California Room and the SJSU Special Collections unit.

A key element of the planning process dealt with the kind of statistics to be collected to evaluate the library. A comprehensive program of data collection commenced with the opening of the MLK Library in September 2003. The plan involved a number of library services and units, including circulation, collections, computer and study room booking, database usage, donations, financial reports, gate counts, interlibrary services, library services and instruction, website usage, and focus group reports. A consultant, Thomas Childers, was engaged in 2003 to undertake a user and cost analysis aimed at developing benchmarks and to administer two surveys over four years, one on library facilities and one on library services. A four-person assessment team under the direction of Jo Bell Whitlatch, associate dean of the University Library of SJSU, undertook the evaluation of service delivery, initiated or coordinated a number of service metric studies, and began a workload activity study in 2004. Only results from the assessment of reference and instructional services have been published to date.³

Literature Review

Many articles report the evaluation of reference services through studies of patron satisfaction, patron queuing, reference accuracy, information-seeking behavior, and patron perceptions of reference service.⁴ Methods ranging from observation, unobtrusive testing, time considerations, transaction logs, survey cards, and forms have been used to investigate reference interactions and staffing. ⁵ During the planning phase for the MLK Library, the SJSU and SJPL administrations and the Reference and User Services planning committee supported finding a method for evaluating service point activity to facilitate planning, determine levels of business, promote tiered reference models, and address staffing needs.⁶

Whitlatch uses state of the art reference question classification to analyze reference service despite the method’s limitations and the need for its categories to be mutually exclusive.⁷ A later survey of methods and forms conducted under the auspices of the Association of Research Libraries suggested that libraries could not agree on methods, models, frequency, or forms; however, the most important data elements were identified as date, type of question, time of day, and location in which the question was asked.⁸

Prior to the library merger, both the university and city libraries had been gathering the statistics required for the SJPL and the California State University systems. For the university library this meant sampling the number of reference transactions only one week out of each semester (in October and March). Initially, every question asked by a patron was simply noted by a slash. Later, academic reference staff and librarians moved to using the letter R for reference questions and the letter D for directional questions. Ciucki’s survey of libraries (fifty-seven academic and thirty-two public) on the types of data collected found that while academic libraries kept records by type of question, public libraries gathered written responses.⁹ The task for the combined library was to identify a method acceptable to both institutions.

The question assessment method selected for the MLK Library needed the capacity to support the joint tiered reference service, which was planned to use support staff and student assistants working in partnership with librarians. Courtois’s descriptive survey on the use of nonprofessionals in academic libraries is relevant here. Courtois found that 51 percent of responding libraries were using nonprofessionals more, identified four categories of reference queries, and ascertained that 37 percent limited nonprofessionals to answering questions in the first two categories (directional and instructional). The 63 percent of libraries that indicated that nonprofessionals answered questions in all categories also reported that nonprofessionals worked alone during evenings, weekends, and early mornings, when “there is little attempt to formally define the types of questions they should or should not answer.”¹⁰

Murfin and Bunge also studied the use of paraprofessionals in academic libraries using the Wisconsin–Ohio Reference Evaluation Program, which employs a two-part form completed by librarians and patrons. Their results show that 20 percent of libraries used paraprofessionals effectively but that 80 percent of libraries found that their use may lower patron success and satisfaction. Effectiveness can be improved when paraprofessionals are encouraged to consult others and are freed from time pressures.¹¹

The Warner Model

In the context of the MLK Library, Debra Warner’s new method for classifying reference questions looked like a promising option. Based on the experience of librarians at the East Carolina University Health Sciences Library, Warner’s schema categorized patron reference questions into four levels of increasing complexity, each level designating the kind of activities necessary to answer the question. The four levels were Level I (non-resource based), Level II (skill-based), Level III (strategy-based), and Level IV (consultation). For each level of question, a definition and series of questions was provided to help reference librarians and paraprofessionals recognize the level to which a particular question belonged.¹²

The Warner model showed the greatest promise for collecting useful data that could be used in the assignment of the appropriate mix of librarians and staff to in-person, telephone, e-mail, and live online reference services. Our article reports on the application of the Warner model in the MLK Library Reference unit and how activity count results were used to adjust staffing levels, change the service point configuration, and improve the tiered reference model. The Warner model was adopted as the most potentially useful method to count in a more sophisticated way all the questions asked in the building.

Reference Services

The impetus behind the merger of reference includes both a desire to achieve staff economies and a commitment to strengthen the mission to provide opportunities for lifelong learning. Planning for the merger of SJSU’s and the SJPL’s separate reference departments began in fall 2002 with the creation of planning teams charged with developing policies and procedures in specific function or service areas, including assessment, reference collection development, electronic resources, e-mail reference, information literacy, reference, information desk, telephone reference, training, and virtual reference. These cross-organizational teams also served as a means for the public and academic staffs to begin working together through a shared discussion list, a “library buddies” program, and team-oriented meetings. Conforming to the dictates of the Memorandum of Understanding between the SJPL and the University Library of SJSU, staffing of the reference desk was to be achieved through an equal division of staff hours.¹³ This was interpreted to mean that usually one academic and one public librarian would work at the desk together in a “seamless” way.

The idea of “seamless” service was that SJSU students or public customers would receive high-quality reference assistance regardless of the affiliation (academic or public) of the person assisting them. Patrons who needed more help than the desk staff could provide would be referred to a subject specialist. Formal staff training and informal and collegial consultation and mentoring increased the skill levels of all librarians. Generally speaking, the merged units in the new library (e.g., Circulation and Reference) work together with as little staff differentiation as possible in the delivery of customer service.

Upon opening in 2003, the reference desk was located at the north end of the second floor, a general information desk was on the first floor, and the SJPL’s general collection desk was on the third floor. The reference desk was configured to provide four workstations serving walk-in patrons. Distance services—telephone reference, e-mail, live online reference—were provided by librarians working in a separate room called the Reference Connection.

Data Collection

Data collection at the MLK Library began in September 2003, one month after the library opened. Each service point within the library collected data during survey weeks using the Service Point Activity Count forms (see appendix). The two forms, one for in-person and one for remote (e.g., telephone and e-mail), were used at all six public service points in the nine-floor facility: the information desk and the circulation desk (first floor), the reference desk (second floor), the general collections desk (third floor), the periodicals desk (fourth floor), and the Cultural Heritage Center desk (fifth floor). For one week in each month, hourly data was gathered. During the initial year of operation from September 2003 through June 2004, only the Reference unit distinguished the levels of questions as Levels I–IV.

Pages: 1 2 3 4