Results
User Satisfaction
User satisfaction has been one of the most frequently used outcome variables that measure reference service effectiveness.14 In the present study, user satisfaction was assessed through user responses to the following four questionnaire items: satisfaction with the answer, perceived staff quality, willingness to return to the service, and positivity of service experience. First,satisfaction with the answer was assessed by asking the question, “Were you satisfied with the answer you received to your reference question?” Figure 2 shows the distribution of the responses to this question.
Among the 417 respondents who answered this question, 65.2 percent reported that they were satisfied with the answer received, and 21.1 percent of the respondents were not sure whether they were satisfied or not. Finally, 12.6 percent of the respondents reported that they were not satisfied at all.
Perceived staff quality, the second indicator of user satisfaction, was assessed by the user response to the question, “The quality of the library staff service in answering this request was _________.” The results are presented in figure 3.
Among the 416 people who responded, 68.2 percent of the respondents answered that the librarians handling the reference questions were excellent. About 19.5 percent evaluated the librarians’ performance as good, and 11.3 percent as poor.
Willingness to use the service again, the third indicator of user satisfaction, was measured using the questionnaire item, “Will you use this service again?” Figure 4 presents the distribution of the responses.
Among the 417 people who responded to this question, 77.2 percent answered that they were very likely to use the service again, 19.0 percent of the respondents answered maybe, and only 3.8 percent said they would never use the service again.
Regarding positivity of service experience, among the total of 422 survey responses, 183 offered open-ended comments on the service. Figure 5 represents the distribution of responses.
As shown in figure 5, 62.9 percent of the respondents evaluated the experience positively, and 28.4 percent evaluated it negatively. About 10 percent of the respondents described it as either a mixed or neutral experience. Frequently mentioned expressions among the positive open-ended responses included “wonderful service,” “quick,” “helpful,” “innovative,” “cool,” “good use of public money,” and other comments indicating immediacy, convenience, and ease of use, including “human contact,” “anytime 24/7,” “likable,” “interesting,” and “time saver.” Frequently mentioned expressions among those reporting a negative experience include “hard to use interface design,” “didn’t answer the question,” “slow response time,” “virtual librarians should access account,” “virtual reference service coverage should be clearly indicated,” technical problems (such as disconnections), “waiting,” “service delay,” misunderstanding, hasty ending, listening skill lacking, and “poor service.” In addition to the patrons’ negative and positive experiences expressed via the responses, mixed comments also were observed. An example of a mixed comment is “the idea of the virtual service is great but I didn’t get much help.” Comments that were categorized as neutral included suggestions for additional features, unawareness of the service’s nationwide nature, additional information about the patrons’ information needs or their background, or reports about problems experienced after the session ended.
Finally, overall user satisfaction was computed by summing up the first three questionnaire items that measured different aspects of user satisfaction. This computation was necessary because the user satisfaction items in the survey questionnaire were measured on a simple three- or four-point ordinal scale (for example, “satisfied,” “not sure,” and “not satisfied” for “satisfaction with the answer” item). These ordinal level measurements are not suitable for undertaking the inferential statistical tests that are crucial to answer the research questions of the present study. To resolve this problem, the three ordinal level variables were summed to create a composite variable, which increases the variability of the measure. This data management procedure allowed the researchers to conduct necessary inferential statistical tests. Survey research literature indicates that a composite variable is generally more valid and reliable than a single question item because it increases variability of the measurement.15
[...] Behavioral Performance on User Satisfaction in Chat Reference Services (pdf) (source: RUSQ via Stephen’s [...]