Feature

The Digital Reference Electronic Warehouse Project: Creating the Infrastructure for Digital Reference Research through a Multidisciplinary Knowledge Base

Scott Nicholson and R. David Lankes

Print version (Adobe Reader required)
One of the valuable offerings of librarians in the digital age is the human intermediation of information needs. In physical libraries, these reference questions are answered, and few artifacts remain from the transaction; therefore, the knowledge created through the work of the librarian leaves with the patron. Due to the medium of communication, digital reference transactions capture the knowledge of information professionals. There are hundreds of digital reference services generating knowledge every day; however, the lack of a schema for archiving reference transactions from multiple services makes it difficult to create a fielded, searchable knowledge base. The development of such a schema would allow researchers to develop tools that practitioners can employ. In turn, this would create a collaborative environment for digital reference evaluation. The goal of this work is to outline the steps needed to develop this schema, present the results of a survey of digital reference services, explore some of the pitfalls in the process, and envision the future uses of this Digital Reference Electronic Warehouse.

The future, and some might even say the present, for the library professional is the digital library. Instead of waiting for the user to come to their information containers in a physical collection, librarians select high-quality materials for users to access through the Internet. It is relatively easy to put a collection of static files online, however, the library is more than just a collection of documents. A crucial part of a library is the human intermediary–the librarian. This intermediary connects the users to the information needed and can assist with advice about using the information retrieval systems and working with information.

However, many users turn to Web search tools for their information retrieval needs. While these tools provide the user with Web pages that match a word on the topic, the quality of the results is questionable. Most Web search tools are for-profit companies and bombard users with advert ising. In addition, search-engine optimizers work to place commercial sites at the top of lists; this has resulted in many searches leading to page after page of commercial results. This commercial information is appropriate for some information-seeking needs, and this is an area where the Web search tools excel. However, the search for noncommercial information can be frustrating. This is an opportunity for libraries.

There clearly is a need for intermediation with the location of material online. Users have turned to question-based search tools such as Ask.com with the hopes of finding such assistance; however, these tools perform no better than a general search tool. There is another type of Web search tool that can take a user’s question and match it to a set of results that are likely to be on topic with little advertising and no direct charge–a digital reference service. In fact, those teaching about Web search tools should always take the opportunity to present a digital reference service as a Web search tool with built-in intelligence.

Many libraries have started services where they allow users to submit questions via e-mail or Web forms. Librarians research the question and provide an answer and related documents to the user. Some libraries offer this service using a live-chat model, where the user is interacting with a librarian with little time elapsing between question and response. These services are usually free, although the user base may be limited to users who are affiliated with the library offering the services. Yahoo! has also entered this domain with Yahoo! Answers, which is a community-based reference service. Users earn points for good answers. Google ran a reference service for a few years called Google Answers, in which those asking questions set a payment for an answer, but Google shut this service down in 2006.

Some digital reference services, com monly known as AskA services, connect the user directly to an expert in the field instead of to a librarian. Services such as Ask Dr. Math and AskNSDL allow users to ask questions of experts in specific topics. This is a different model of the reference process, but the information contained in these transactions is valuable. Lankes presented a model that contrasted these two types of services in his research agenda for digital reference.¹

There are hundreds of these services around the world providing answers and resources in response to user needs. If collected into a knowledge base, it would be incredibly useful for researchers to explore the results of this process. Information-seeking research has been an active line of exploration for decades, and there are many theories developed from small samples that could be explored with this larger dataset. In addition, by examining the common works referred to in different types of questions, automatically generated directories of high-quality material could be created and shared. The goal of the Digital Reference Electronic Warehouse Project (DREW) is to create a large database of reference transactions so that researchers might better understand the process and then create tools for measurement and evaluation that managers of reference services could employ.

Relationship of DREW to Similar Projects

There are several different types of digital multidisciplinary knowledge bases currently available. Precursors to today’s knowledge bases are bibliographic databases such as ArticleFirst and database aggregators like DIALOG. As these tools have grown to include access to full-text resources, they have become true multidisciplinary knowledge bases. The difficulty in using these databases comes through the methods of retrieval. Searchers have to match the words used by the author when searching free-text fields such as the title, abstract, and text of the document. Conversely, searchers could attempt to match words selected by indexers such as subject headings. Users can get frustrated with these tools, as they tend to match either too few or too many articles.²

Ano ther type of multidisciplinary knowledge base available is the Web. Web search tools provide a portal to this knowledge base. Most current Web search tools allow the user to search large porti ons of the textual data available on a convenie nt ly acce ssed subset o f the Web. These search tools cannot access large portions of the Web known as the Invisible Web.³ In fact, one study claims that the well-known search tools index only about .03 percent of the Web.⁴

In addition, as these search tools index the words used on the page, the user has to search using the words used by the authors of the page. Due to the commercial nature of these tools, many Web authors use Search Engine Optimization (SEO) techniques to push their pages to the top of listings.⁵ If these two issues are combined–search tools only index a small portion of the Web, and some companies are changing their pages to aggressively hold the top positions in the rankings of search tools–then it is expected that the typical user who only explores the first page of rankings will become frustrated with the repetition of results.

One solution to these problems is human intermediation. Some search tools have integrated human intermediation through directory-based search tools; Yahoo!, for example, started as a directory-based search tool. These tools allow a user to discover a small subset of resources that were selected using some type of quality criteria through a hierarchical organization structure. Over time, search-tool companies have removed or reduced emphasis on these directory tools, promoting the full-text search tools in their stead.

There are some updated directory-based Web search tools that harness the power of human intermediation. The Open Directory and About.com use experts to select Web sites on a topic and provide users with a directory-based access method. For scholarly research, Infomine is a high-quality directory out of the United States, and BUBL is focused on academic Web-based information from the United Kingdom and Europe. The difficulty with these tools is similar to the problem with the bibliographic databases; searchers have to match either the terms selected by the authors of the pages or terms selected by the creator of the directory.

The setting for the current paper is in digital reference, which is human intermediation provided in direct response to a user’s query. Most of the time, the answer to a digital reference question contains text as well as links to Web pages, journal articles, and other information. Therefore, the answer will connect the same types of resources discussed in the previous few paragraphs. The transaction will also have some metadata, such as subject headings, attached to it by either the user or by a staff member during the digital reference process.

In addition, the resources selected by an expert during the digital reference process will be of high quality. By gathering answers from many different resources, directories of these quality materials can be automatically generated. And by appending commonly used query terms to the directory, the directory can be made more easily searchable. Therefore, the knowledge base created through the archiving of digital reference transactions will be more easily searchable, contain references to high-quality resources, and provide indirect access to the human intermediation process of librarians and experts from a multitude of backgrounds.

Other Digital Reference Archives

Most reference services maintain some type of archive. That archive may be accessible only to the administrators (and it may be a useful archive for those answering questions), or it may be available to users of the system. There are a few existing publicly accessible projects that archive digital reference queries. A number of projects, such as Ask-A-Scientist and Yahoo! Answers, allow anyone to search their internal archive of question and answer pairs. While this is useful, it lacks the richness available if the transactions are collected by multiple services.

One of the largest shared archives of reference transactions is QuestionPoint’s KnowledgeBase.⁶ The purpose of QuestionPoint’s KnowledgeBase is to provide reference librarians and their patrons with a repository for hard-to-find answers, answers to frequently asked questions, pathfinders and bibliographies on specific subjects, and the intellectual content resulting from aiding scholars in their research. Users of QuestionPoint can easily access the KnowledgeBase, and other libraries can provide access to the KnowledgeBase for their users by linking to QuestionPoint.OCLC also provides a Firefox extension for searching the KnowledgeBase at this web site. This is a notable project because it is a large-scale shared reference depository with more than 14,000 edited transactions as of January 2007. In addition, this knowledge base is growing.⁷ Transactions are selected in two ways: Any question submitted to the global network of reference librarians for an answer is considered, and individual libraries have the ability to select any local transaction and submit it to QuestionPoint for consideration. Once identified, the transactions are cleaned, removing all personal information about both the user and the librarian. The text of the question and answer are cleaned for clarity, free-text keywords are assigned, and classification headings are assigned from the top two levels of the Library of Congress Classification scheme. After ensuring that there are not similar transactions on the topic area, the transaction is placed in the knowledge base. At this time, a review date can be set to trigger a manual review of the information in the transaction to ensure it is up to date.

One goal of the DREW project is to maintain a relationship with other major reference archives such as QuestionPoint. Examining these similar projects allows us to determine the needs of DREW and learn from the exploration of others. Due to the time and resources invested by OCLC and the Library of Congress in the development of QuestionPoint’s KnowledgeBase, their process and policies can serve as a model to libraries creating a cleaned archive to aid patrons and librarians. DREW, being a project to provide data for researchers about the process, requires a different type of warehouse. The transactions will not be edited for content, although personally identifiable information will be removed. Transactions on the same topic are desired, as that will allow the discovery of trends and changes over time. One of the areas of exploration, to be discussed later, is automation of several of the cleaning processes such as assignment of subject headings.

Therefore, DREW will complement these archives and knowledge bases focused on aiding librarians and their users directly. In order to do this, one goal of DREW is to create a schema that is compatible with different existing knowledge-base projects. The challenge of this project is overcoming the complexity of many different services and user types. The landscape of digital reference is one of many types of services, librarians, and users interacting with a similar base of resources. There will be patterns across services, although teasing them out of the complex data is a challenge. The authors turn to complexity theory as the theoretical support for the success of this project.

Complexity Theory and DREW

To date, know ledge-base work in digital reference has been primarily a deductive process. That is, either a service makes every transaction searchable, or into an extensive transformation process of question selection, editing, and incorporation into a predetermined subject hierarchy. These deductive, and largely manual, processes have obvious scale problems. Further, these processes tend to be input-only syst ems in that they must be manually weeded of outdated information. Other issues in the deductive construction of knowledge bases are:

• Context Dependencies: Information in knowledge bases is very context- dependent. It is quite possible that the only application of the information in a digital reference transcript is to that given interchange between librarian and patron.
• Metadata Creation: Time, labor, and money are involved in creating metadata for transcripts and digital reference interchanges so that they may be later discovered and retrieved by end use rs. While some of this effort may be part of the reference process itself (for e xample, classifying a question for distribution in QuestionPoint), it may still require effort to confirm and refine this classification data for inclusion in a knowledge base.
• Chunking: It is well known that users will ask several questions in both real-time and asynchronous transactions. How those questions and answers are “broken apart” is often dependent on human intervention and a great deal of interpretation.
• Fact Shifting and Temporal Dependencies: Answers to reference questions are often time dependent. From the name of the U.S. president to the height of Mount Everest, answers to even simple questions change. These changes, while concrete, are often hard to track over time. This does not even take into account gray areas where an answer or fact to apply to a question is a matter of choice among equally good options.

Pages: 1 2 3 4 5 6

Comments (No comments)

There are no comments for this post so far.

Post a comment