Each of the answers in our sample (n = 1557) were analyzed and coded in terms of how much the content differs in word count from that of the base answer form used. Differences in word count imply greater customization; instructional responses would be more customized than procedural responses. This suggests the use and analysis of descriptive statistics, especially mean and median scores of the customized content and the average level of customization in procedural responses compared to that in instructional responses. Further, a Mann-Whitney U test was performed to statistically compare the mean scores of customized content for instructional versus procedural. The theory developed here suggests that there would be statistically significant higher expected customization in instructional responses than in noninstructional responses, thus validating this operational definition.
Statistical Analysis
Figure 2 reports the descriptive statistics, including the range of difference of the word counts of instructional responses (n = 769) versus procedural responses (n = 788). A Mann-Whitney U test was conducted using SPSS 15 to evaluate the hypothesis that the answer forms would be customized more for instructionally related responses versus procedurally related responses (see figures 2, 3, and 4). The results of the test were in the expected direction and significant: z = -4.429, p < .000. Instructional responses had an average rank of 830.08, while procedural responses had an average rank of 729.16. Figure 5 shows the distributions of the word differences between the base response forms versus the actual response for the two groups.
Discussion and Conclusion
The statistical test therefore suggests that the librarians working on this Ask A Librarian service customize their user responses more when they intend to provide some level of instruction to the user. Also, procedurally oriented answer forms do not always need much in the way of customization. This latter point is very evident in the distribution of the word difference shown in figure 5.
However, there is a concern with the outlier data shown in figure 5. It could easily be assumed that these may be due to data entry error. But a review of the actual responses versus the base forms show that these data sets are accurate. In many of the instances, the customized responses were credited to one or two individuals who tended not to use the answer forms. Indeed, Doherty discovered that some librarians preferred to ignore the forms completely because of their perceived impersonal nature.25
Further, our statistical analysis alone does not suggest causation. While there is a statistically significant difference in the word count means between instructional and procedural answers, this does not necessarily correlate to the librarian’s instructional intent causing such a significant customization. However, returning to our theoretical frame outlined previously, the analysis is suggestively supportive of this causation. To be sure, a more interpretive study should be performed in which case-study or ethnographically informed methods could uncover the actual intent of the librarians.
One major concern of this study has been the usage of the generic answer form. It was used 181 times during the October–November 2004 period, 196 times during the same period in 2005, and fell to 118 in 2006. These numbers are consistent with internal studies that suggest this form is used about 25–30 percent of the time. In many instances, the content of the generic responses had to be examined to divine the intent of the librarian responder. A similar process was used here as was used previously to code the other entries as instructional or procedural.
Our data reveals that customization of canned responses occurs in this one online reference service. We suggest that there could be similar results in other such services examined by comparing the amount of customization that goes into the actual response sent to the user. Where possible, we encourage libraries to gather such data and thus begin to uncover some basic assumptions about their online reference services. Such studies would add to the empirical testing of the theories we have developed.
One assumption of the present service that this study revealed is the importance of customization. Anecdotal evidence suggests that librarians use the built-in ability to edit all responses to meet the query of the user as best as possible. Without the ability to conduct the traditional reference interview, the customized response becomes more essential as the librarian responder makes certain presumptions about the needs of the user and also tries to proactively anticipate potential unexpressed needs. Janes, supported by Doherty’s conclusions, implies that librarians in online reference are selectively applying the orthodox procedures of the reference interview.26 We believe our analysis shows that the librarians involved with this e-mail reference service have drawn on the limitations of the technology to critically engage in an examination of the reference interview in favor of a dialogical interaction.
In conclusion, our research shows that the academic librarians involved in this particular e-mail reference service intend to provide some form of instruction in just over half of the interactions examined here. It is this instructional intent on the part of the librarian that is deliberately designed to overcome the negative issues of transactional distance in the online learning environment. In other words, the librarian seeks to provide the user with an appropriate and relatively immediate response.
There are instances of significant customization of responses over the base standard answer form. As noted above, this suggests a strong sense of instructional interaction. The librarian has stepped outside of the structurally bound, standardized response to engage more directly with the user and to acknowledge the experience and intent the user brings to the interaction.
Acknowledgements
The authors gratefully acknowledge the assistance of Amber Lillie in completing our data entry. Further, we appreciate the assistance of Robert Horn in helping us to refine our statistical approach. Finally, we thank the referees and editor of RUSQ for feedback that significantly improved our final paper.
Martha Portree was a Librarian at Northern Arizona University (NAU) in Flagstaff. R. Sean Evans and Tina M. Adams are Librarians and John J. Doherty is an Instructional Designer at NAU. Submitted for review May 3, 2007; revised and accepted for publication August 30, 2007.