As presented in table 1, user satisfaction was statistically significantly different in six out of the ten RUSA behaviors at the significance level of a < .05. Satisfaction was statistically significantly higher when reference staff showed the following six behaviors than otherwise; that is to say, chat reference services were perceived to be more satisfying to the patrons when librarians:
- used the patron’s name during the reference interview;
- communicated more receptively and listened more carefully;
- searched with or for the patron;
- provided pointers;
- asked the patron whether the question was completely answered; and
- asked the patron to come back if they needed further assistance.
This result is consistent with the findings of the previous studies conducted in face-to-face reference setting, which reported the positive influence of the original guidelineson reference success.21 Thus, the findings of the present study indicate that performance of RUSA behaviors also are effective in the real-time chat reference setting.
Which RUSA Behaviors Are Predictors of User Satisfaction?
Which RUSA behaviors, if observed, could predict user satisfaction with chat reference? To answer this research question 3, a multivariable regression analysis was undertaken at the significance level of a < .05. The test result revealed that five of the ten RUSA behaviors were statistically significant predictors of user satisfaction. The strongest predictor of user satisfaction among all ten behaviors was “Answered?” (for example, asking patrons if their questions have been completely answered; regression coefficient ß = .181, p < .001), followed by “Offering Pointers” (for example, offering pointers, detailed search paths and URLs, and names of resources used to find the answer; regression coefficient ß = .124, p = .014), “Come back” (for example, encouraging the patrons to return if they have further questions; regression coefficient ß = .112, p = .019), “Searching” (for example, searching with or for patrons by explaining search strategies and sequences and reformulating searching strategies; regression coefficient ß = .112, p = .023), and “Listening” (for example, receptive and cordial listening; regression coefficient ß = .097, p = .044).
Discussion of Results
The present study’s purpose was to expand the understanding of effective reference staff behaviors, particularly focusing on whether chat reference can be more successful when librarians employ the behaviors recommended in the revised RUSA guidelines. From the content analysis of 422 chat reference transcripts, it was found that the selected ten RUSA behaviors appeared in a range between 28.7 percent and 63.7 percent of the total transactions. In six out of ten RUSA behaviors, user satisfaction was significantly higher when librarians demonstrated those behaviors during chat sessions than when they did not. Those behaviors were: (1) use of patrons’ names; (2) listening; (3) searching; (4) offering pointers; (5) asking if questions were answered; and (6) asking patrons to come back (see table 1). Furthermore, when examining the behavioral predictors of user satisfaction, five of the ten RUSA behaviors were found to be significant predictors of user satisfaction. They were: (1) asking whether the question was answered completely; (2) offering information sources; (3) asking patrons to come back when they need further assistance; (4) searching information sources with or for the patrons; and (5) listening to questions in a cordial and receptive manner. These findings indicate that behaviors recommended in the revised guidelines do, indeed, affect user satisfaction with chat reference. It confirms that the RUSA behaviors are as important in ensuring successful chat reference as they have been shown to be in the case of physical reference desk settings. The findings also suggest that the RUSA guidelines continue to constitute a pertinent assessment tool for chat reference service.
The results of this study suggest how each area in the revised guidelinescould be further developed. First, in approachability (Area 1), the coding process revealed that most behavior items in the revised guidelines were not observable through chat transcripts. Other than the scripted words indicating initial welcoming word contact (for example, “Hi, this is the reference librarian”), the transcripts did not have many expressions indicating approachability behaviors. Most items listed under approachability either relate to the pre-interview stage (such as informative signage) or lack explicit verbal indicators that help us code the transcripts for approachability as behaviors. Therefore, the finding that approachability has little influence on user satisfaction should be interpreted with caution. This problem in the coding process suggests that the revised guidelines need to incorporate clearer behavioral indicators of approachability in order to encourage reference staff to use more explicit welcoming gestures to the patrons and to make the revised guidelines a true behavioral assessment tool. The influence of approachability on user satisfaction should be reassessed with those new behavioral indicators in future studies.
A similar contention can be made with regard tointerest, the second area of the revised guidelines. Behavioral indicators of interest in text-based environments include librarians’ attention to the patron and maintaining or re-establishing word contact by sending written or prepared prompts and so forth. In the current study, interest was mostly captured in scripted words, such as “I’m reading your question. Just a moment please.” Observations of behaviors via transcripts demonstrated limitations in capturing librarians’ interest in answering questions. This limitation may have affected the lack of association of interest with user satisfaction in this study.
Listening/inquiring (Area 3) has been regarded among librarians as a core component of the successful reference interview.22 Consistent with previous findings, this study also found that user satisfaction was higher when librarians demonstrated listening behaviors than when they did not. Cordial and receptive listening also was found to be a significant positive predictor of user satisfaction. However, in this study, we found inquiring (or probing behavior) was not associated with user satisfaction. As a viable explanation for this finding, it is conjectured that these probing behaviors do not guarantee the actual answer that the service users are ultimately interested in receiving. It also is possible that probing behaviors may be more important in a face-to-face reference interview, whereas online reference users may, by contrast, only be interested in a specific answer, as the medium of the Web tends to attract users with the promise of an immediate answer.
Searching, the fourth area of the revised guidelines, was observed by two types of behaviors, “searching with or for patrons” and “offering pointers and information sources.” User satisfaction was higher when these two searching behaviors appeared in the transcripts. Both behaviors were revealed to be important positive predictors of user satisfaction. Indeed, this finding indicates that user satisfaction is directly associated with actual searching activities and tangible answers that can resolve patrons’ problems.
[...] Behavioral Performance on User Satisfaction in Chat Reference Services (pdf) (source: RUSQ via Stephen’s [...]