Follow-up, the fifth area of the revised guidelines, was observed in three types of behaviors. Interestingly, two of the three follow-up behaviors were found to be strong positive predictors of user satisfaction. According to our findings, user satisfaction was most strongly predicted by the simple behavior of asking follow-up questions, such as “Did you find what you needed?” “Does this completely answer your question?” or “Is there anything else I can help you with?” Certainly, this finding is not unprecedented. Gers and Seward contended that follow-up is “the single most important behavior because it has the potential for allowing one to remedy lapses in other desirable behaviors.”23 As Dewdney and Ross maintained, asking follow-up questions is a chance to “repair the interaction or to formulate a new, more promising search strategy,” but reference staff members tend to make referrals as a way of circumventing the reference interview.24 If such a case, avoidance certainly would not satisfy many users.
It should also be noted that two follow-up questions (“asking if the question was answered” and “asking to come back for further assistance”) were not significantly correlated with each other, and each behavior increased user satisfaction independently. This finding implies that these are two quite different questions; thus, librarians should make sure to ask both questions before closing their transaction.
One follow-up behavior, “providing referrals/alternatives,” did not increase user satisfaction in our study. This may be because users are rarely satisfied with “delayed” answers, but defer their assessment of satisfaction until they obtain the tangible answer they are seeking. This finding is consistent with the findings of Dewdney and Ross, who reported the ineffectiveness of unmonitored referrals at physical reference desks.25 This finding about referrals indicates that reference staff should make a referral only when they deem it necessary rather than using it as a good alternative to an answer.
Conclusion
By adopting the revised guidelines as a behavioral performance assessment tool, the present study revealed that many behaviors they prescribed are effective in enhancing user satisfaction with chat reference service. This study also attempted to explain why certain behaviors in the revised guidelines were not related to satisfaction, as discussed in the findings. Combined, the results of the present study offer many implications for reference service practitioners, for the researchers investigating various phenomena in virtual reference services, and for the developers at RUSA working on future revisions of the revised guidelines.
First, the results of this study could be of significant assistance to staff training in the chat reference service practice. Consistent with the findings of the studies conducted in the physical reference setting, the results of this study also indicate that follow-up behaviors should continue to be emphasized in staff training in the chat reference environment as a way to enhance both staff performance and user satisfaction.
The present study also identifies places for future adjustments of the revised guidelines. As pointed out earlier, some items in the behavioral guidelines were not readily observed as an explicit behavior, while other items were not located in the proper category across the five areas. These items need further clarification if the revised guidelines are to be adopted by reference librarians as a practical behavioral evaluation tool that can be used with few modifications. In addition, inclusion of specific behavioral examples for each item would be very useful, especially in the areas of interest and listening/inquiring. A future study could assess these areas utilizing further development of the guidelines.
Despite efforts to make this study as rigorous as possible, it is not without limitations and thus calls for further research. First, user satisfaction was examined as a way to investigate the effectiveness of chat reference services in the present study. Considering that user satisfaction is just one aspect of service effectiveness, researchers could employ other indicators of service effectiveness, such as answer accuracy or answer completeness. Another limitation is the fact that this study was conducted in a single public library system, although the observed librarians were from forty-nine library systems participating in a nationwide collaborative chat reference program. It also should be noted that the current study observed only 30.4 percent of the total 1,387 analyzable transactions. People who took the time to answer the online pop-up survey might not constitute an accurate representation of the entire user population of the chat reference service. Thus, our findings should be further confirmed by replicating this study in libraries of different types and sizes and by enhancing the sampling technique.
Finally, this study examined theusefulness of therevised guidelinesby observing the use of chat reference only. Nonetheless, the revised guidelines were developed to encompass various modes of virtual references, and thus similar studies should be conducted for other modes, such as e-mail and instant messenger, in order to develop further recommendations for therevised guidelines.
Despite the limitations mentioned above, the results of this study will encourage RUSA’s further promotion and perhaps enhancement of the revised guidelines. This was the first empirical study that investigated the revised guidelines’ value by assessing its actual items in the chat reference setting. By revealing the positive influences of the RUSA behaviors on user satisfaction, the present study demonstrated that therevised guidelines are effective behavioral standards for librarians’ real-time chat reference interviews. Thus, by incorporating the recommendations of this study, RUSA could further promote the revised guidelines as a practical reference staff training and assessment tool.
[...] Behavioral Performance on User Satisfaction in Chat Reference Services (pdf) (source: RUSQ via Stephen’s [...]