Maximizing Local and National Assessment for Evidence-Based Librarianship

Ellysa Stern Cahoy and Loanne Snavely, Guest Columnists

Print version (Adobe Reader required)
A few months ago I was riveted by an in-house presentation disseminating results from a 2008 survey of computing by Penn State University faculty, students, and staff. I couldn’t help but think that RUSQ readers would also find the Penn State data interesting, especially comparisons with national survey data. I invited one of the presenters (Loanne Snavely) to collaborate with Ellysa Stern Cahoy on an article that would share data from this survey as well as another locally executed survey. Both of these surveys represent a library partnership with other campus units. Additionally, I asked the authors to provide advice on how other libraries can replicate what Penn State has done in this area, and to demonstrate the importance of national and local technology-focused surveys for assessment.—Editor

Gaining a perspective on student and faculty opinions and abilities relevant to libraries and information technology (IT) is integral to and can locally inform planning for future services and resources. Yet, without ready access to campuswide survey instruments, how can academic librarians assess their users in this area? Building a culture of assessment can enrich evidence-based librarianship and provide a sound basis for decision makingand strategic planning.¹ LibQUAL+ and other standardized library assessment tools have provided a basis for under standing library user needs, and surveys of technology use are becoming more essential to library planning. With the continued merging of libraries and IT on college campuses, it makes sense to capitalize on and integrate within already existing IT user surveys. The constant and rapid shift in popular technologies mandates that librarians, faculty, IT departments, and all of higher education understand how students and faculty are using technology in connection with libraryresources. This knowledge assists librarians in developingtechnology-related resources, programming, collections, and services, keeping library programming vital and relevant.In only a few years, librarians have witnessed a shiftfrom e-mail to instant messaging to Facebook, from landline telephones to cell phones to the mobile Web. What are the national technology trends, and how do they play out on an individual campus? How do technology trends inform the development of new services and experimentation with emerging methods for serving users? This article provides strategies for using national surveys of library users and leveraging and maximizing partnerships for local library data collection and analysis. The Penn State University Libraries have locally executed two surveys of faculty and students in partnership with Penn State Information Technology Services and the Office of Student Affairs, Research and Assessment. Details of the Penn State studies are shared in this article, along with examples of national technology surveys useful in local benchmarking.

National Technology-Focused Surveys

In recent years, the academic library community has used LibQUAL+, a primary assessment tool, to manage user satisfaction with and effectiveness of library services.² Based on SERVQUAL, a survey instrument designed to measure service quality for businesses, LibQUAL+ was developed by the Association of Research Libraries in collaboration with Texas A&M faculty.³ Designed to identify gaps in library services, LibQUAL+ provides libraries with a standardized, Web-based survey to help librarians objectively evaluate services.⁴ The tool also carries the option of benchmarking results with other peer institutions (and LibQUAL+ participants). LibQUAL+ covers a broad range of library topics of interest, including information literacy outcomes, effectiveness of services, and library as place. Responses can be broken down by specific audiences, including discipline, age, sex, and academic status. While it is a highly powerful survey tool, it is one that is administered solely by the library without buy-in from other campus groups. Furthermore, LibQUAL+ may not provide the in-depth technology and library-use data needed to develop new and cutting edge services in today’s library environment.

Findings from national, technology-focused surveys can provide a powerful foundation for the structure and focus of locally based surveys. Consider the power of a statistic that can show not only your library users’ abilities or views, but also a comparison with national findings. Finding that a majority of your users access the library website gains greater significance if your users turn to the library webpage at a higher rate than users nationally. There are several large, recurring national studies of technology use that can be used to help provide a baseline of comparison between students at your college or university and students nationally. Perhaps the best current supplier of student technology and library-use statistics is the Online Computer Library Center (OCLC), which has produced what the company refers to as “landscape reports” since 2003.⁵ As its first effort in this realm, OCLC published The 2003 OCLC Environmental Scan: Pattern Recognition.⁶ This initial report sought to identify and detail current issues and future trends projected to affect libraries significantly. The report was focused to assist OCLC decision makers and the larger audience of librarians in strategic planning.⁷ Following up on the success of the Environmental Scan, OCLC produced Perceptions of Libraries and Information Resources in 2005. In the report’s introduction it was noted thatthere are no major recent empirical studies that look specifically and broadly at the role libraries and librarians play in the infosphere, from the point-of-view of the information consumer. How are libraries perceived by today’s information consumer? Do libraries still matter? On what level? Will library use likely increase or decrease in the future?⁸

The 2005 report explored the library brand—specifically, how users perceived this brand and the value of libraries in general. OCLC partnered with the corporate research firm Harris Interactive to survey library users in six countries: Australia, Canada, India, Singapore, the United Kingdom and the United States. Users were asked their relationship with, awareness of, and trust of library resources and services. The report provided an important, never-before-seen snapshot of the brand effect of libraries in the day-to-day lives of information consumers. The following are sample questions from Perceptions of Libraries and Information Services:

• How much has your personal library use changed over the last three to five years?
• Please indicate if you have used the following electronic information sources, even if you have used them only once. (The list of choices included search engine, library website, online databases, and online library question service.)
• Where do you typically begin your search for information on a particular topic?⁹

In 2006, OCLC published College Students’ Perceptions of Libraries and Information Resources, a subset of the 2005 report that focuses on a specific audience. Containing invaluable data for academic libraries, this report shares data on students’ general use and familiarity with libraries, usage of libraries (in-person and online), perceptions of the library brand, and student advice for libraries and librarians. Specifically, the report covers the responses of 396 college students, both graduate and undergraduate, from the 2005 report. Responses from fourteen- to seventeen-year-olds are also in the report to provide contrast with current college students and context on needs of potential future college students. Because it is a subset of the 2005 report, the same questions are used, but college student response trends and patterns are highlighted. In the introduction to the report, Cathy DeRosa, OCLC vice president for marketing and library services, writes, “As is the case with the full Perceptions report, the findings presented in this report do not surprise, they confirm.”¹⁰ The report highlights the significance of drawing out the responses from a specific population to gain different insights into certain user group needs.

OCLC took a different approach to their research in 2007 with the publication of Sharing, Privacy and Trust in our Networked World. This report looks only at a core set of issues facing libraries—the use of social spaces online and expectations for libraries in this realm. It explores user behaviors and preferences in online social spaces (including Facebook and MySpace), user attitudes toward sharing information online (including library websites and social spaces), user attitudes toward online privacy, and librarian views of social spaces and future possibilities for library services in this realm. This study expands to include users in Germany, France, and Japan. U.S. library directors were also queried as part of the study. The following are representative questions included in the study:

• What type(s) of online activities have you done or participated in during the last twelve months?
• Generally, do you think that your personal information on the Internet is kept more private than, less private than, or the same as it was two years ago?
• How likely would you be to participate in each of the following activities on a social networking or community site if built by your library? Choices included being notified of terms of interest to you, sharing ideas with library staff about services, and self-publishing creative work.¹¹

The Pew Internet and American Life Project has also been a source of statistical reports relevant to libraries. Known as a nonpartisan, “non-profit ‘fact tank,’”¹² the project regularly shares data findings on the effect of the different aspects of the Internet on a variety of audiences, including adolescents and adults. The Pew project primarily conducts phone surveys and relies on information from research partners. Recent Pew reports include Information Searches that Solve Problems: How People Use the Internet, Libraries and Government Agencies When They Need Help and Teens and Social Media.¹³ Each report surveys a large number (from more than nine hundred to several thousand) of U.S. residents on a specific topic related to the Web. While these surveys are not directly library related, they can provide data that highlights how users are responding to and integrating specific aspects of the Web into their daily lives.

Ithaka’s 2006 Studies of Key Stakeholders in the Digital Transformation in Higher Education also yields powerful data for libraries. The project details the aggregated responses from more than four thousand faculty, indicating preferences and opinions on the role of the library in higher education, the future of the library as a repository, faculty publishing, and the library as scholarly publisher. Libraries interested in their faculty’s feedback would be wise to build questions related to the Ithaka study and compare responses accordingly.¹⁴

Pages: 1 2 3 4