The premise for the study was based on the notion that a reasonable exposure to newer reference services like VR had occurred (both universities provide e-mail services). By issuing a survey, we could shed some light on emerging preferences for different types of service and whether there are different preferences depending on exposure to the VR service and preference for study location. The research reported here is considered exploratory and not intended to be representative.
A pop-up survey was provided to all VR users over a three-week period in November 2004 and appeared following the close of a session with a librarian; the same questions were issued in print to reference desk users over the same time period. Basically, the surveys were distributed until an acceptable amount had been filled out: approximately one hundred for each university’s reference desks and approximately one hundred in total from VR users. The final frequencies for each survey were reference desk, 242 and VR, 106. The survey (see appendix) asked seven core questions exploring use and preference for reference services and habits and preferences for study location (in library, off campus, etc.) as well as resources and chat software use.
The sample sizes for each service point were not proportionate to actual reference desk and VR statistics during this time period. During the same period our two universities jointly had 382 VR sessions and approximately 9,500 Reference Desk interactions.20 As it turned out, it was much easier to obtain feedback using a pop-up survey after a VR session, than to ask users in person at the reference desk. The return rate for our VR users during that period of time was approximately 28 percent. For the desk survey, the staff handed out surveys to users at the end of a reference interaction. Although all staff at the reference desks were aware of the study, they were not always able to hand them out consistently (this could have been for a number of reasons: e.g., staff would often forget or be too busy). Consequently, we cannot calculate the return rate of the reference desk surveys. Our purpose, however, was to obtain enough data to compare VR and reference desk users, not to sample the larger academic communities of our universities; and our main concern was to obtain enough data to ensure that smaller groups, like graduate students, would be adequately represented. The amount of data that would have resulted from a random sampling of our university populations would not have been adequate to describe our VR users. For example, Johnson sampled the entire university population, and of that sample only 3 percent had used the chat service.21
The intent was to be able to explore VR and desk users in terms of their preferences and behavior. While this type of sampling does not allow for statistical significance testing (a Chi square, for example), the sample size provided some latitude for analysis and speculation as well as the opportunity to suggest directions for further study. The focus of the study was primarily in the comparison of preferences of our two user groups. We compared the status (undergraduate, graduate students, staff, and so on) of these two user groups, but this study did not undertake a thorough analysis of other characteristics such as age, gender, and commuting distance.
Focus Groups
Four focus groups were conducted following the survey in the winter of 2005. They were carried out both in person and online. The purpose was to gain qualitative insight into some of the issues that arose from the survey results and to assist us in defining directions for future study. The online focus groups were made possible through the Meeting Room function of our Tutor.com VR software. Each focus group was given seven questions relating to help-seeking preferences, problems encountered when getting help, expectations of the chat reference service, and suggestions for improvement of the chat service. Seven volunteers for the in-person groups and four for the online groups took part in the focus groups. This anecdotal evidence will be noted in the discussion section.
Results
Surveys were completed by 348 library users. Of these, 106 (30 percent) were completed by VR users, and 242 (70 percent) were completed by reference desk users. The proportion by institution differed between the two types of surveys. Because York University’s VR service is busier than Ryerson’s, the majority of the VR surveys were completed by York users. Seventy-five percent were York users, 22 percent were Ryerson users, and the remaining 3 percent were unaffiliated with either institution. The proportion was more even for the surveys completed at a reference desk; 53 percent of these were York surveys and 47 percent were Ryerson surveys.
Respondents were asked what their current status was at the university (table 1). The response to this question differed considerably between the two surveys. In both cases the respondents were largely undergraduates (74.5 percent of the VR respondents, and 86.4 percent of the reference desk respondents). The real difference lay in the proportion of graduate students: 16 percent of the VR respondents were graduate students, compared to a mere 3.3 percent of the reference desk respondents. Very few VR respondents identified as continuing education students, but 6.2 percent of the reference desk respondents identified as such. The numbers of faculty and university staff were under 2 percent.
Reference desk and VR users were asked two questions on their preferences for obtaining research help. Ratings were on a scale of 1 to 5, where 1 was the least preferred, and 5 was the most preferred. Respondents were first asked about preference for getting help when in the library (table 2), and then asked for preferences when off campus (table 3). It was considered wise to divide the question into two separate questions, since options are clearly different depending on location. Among the response options, consulting information on the library website was included as well as searching Google or another Internet search engine. These options were included because it was felt that our users may increasingly seek answers to what would be reference questions by employing self-help strategies and consulting sources on the Internet (whether they are sites we have constructed ourselves or popular external sites). These two questions allowed for comparing preferences for obtaining help between the reference desk group and the VR group.
The traditional reference desk has the highest in-library preference rating for both groups (desk users rating = 4.4; VR users rating = 4.3). Surprisingly, our library websites seem to be the next highest rated option for getting research help (desk = 3.8; VR = 3.5). E-mail and telephone reference options were rated relatively low by both reference desk and VR users, although these services were rated slightly lower by the VR respondents. For the most part, the reference desk and the VR results were very similar in their ratings. The only rating in which there was a notable difference between the two groups of respondents was in the rating for the use of VR when seeking help from within the library. The VR respondents rated virtual reference 3.5 (roughly the same as the library website), whereas the desk respondents rated it only 2.2 (on par with the ratings of e-mail and telephone reference). It appears that virtual reference users see the service as a benefit in the library, and not just off campus.
When considering off-campus options for obtaining help with research, the reference desk respondents gave the highest preference ratings to the library websites (4.3) and Google or other Internet search engines (3.9). These self-help options were not rated as highly by the VR respondents. Instead, the VR group not surprisingly rated virtual reference highest (4.3). In fact, the VR group rated the VR service as high as an off-campus option as they rated the reference desk as an in-library option. The reference desk respondents did not rate VR very highly (2.6). In fact, the reference group respondents rated VR even lower than they rated e-mail and telephone reference services (which rated only 2.9 and 2.6, respectively). E-mail and telephone services also rated low in preference for the VR group (3.1 and 2.9, respectively). Again, the most dramatic difference between the two groups was in the preference ratings for virtual reference off-campus. The other interesting pattern was that the VR group gave lower ratings to self-help options (the library website and Internet search engines) than the reference desk group.