The Credibility 2.0 Project funded by the MacArthur Foundation (September 2008 – August 2011) investigates what new sets of heuristics have emerged in the participatory Web environment (Web 2.0). The research activities involved in this project include:

1. Web-based online information activity diary survey
2. Interviews with participatory Web users
3. Experimental studies to collect behavioral data with respect to the application of credibility assessment heuristics in the processes of information seeking and content creation.

Given the changing Web environment, the proposed study extends existing credibility research in four ways — by:

1. Characterizing different types of Web users and identifying participatory Web users based on their
   participation in variety of activities including content creation and mediation
2. Examining the relationship between Users’ participation levels on the Web and their credibility assessment concerns and heuristics
3. Understanding credibility assessment of user-generated content contributions in the domain of news with respect to people’s selecting, posting, commenting, and other contributing behaviors on the Web
4. Making a direct comparison between the credibility judgments of active participatory Web users and those of general information seekers

Since the mid-1990s, Professor Soo Young Rieh has conducted a series of research projects about information credibility, information quality, and cognitive authority in various information seeking and use contexts. Her research on credibility has progressed with respect to the following three research goals:

1. To identify factors influencing people’s judgments of credibility
2. To characterize information seeking context and environment embedded in the process of credibility assessment
3. To enhance conceptual framework(s) of credibility assessment.

Goal 1: To identify factors influencing people’s judgments of credibility

Rieh’s first research project on information credibility started in 1997 as an exploratory study (Rieh & Belkin, 1998). Rieh was interested in whether scholars could apply the evaluation criteria that they used in traditional information resources (journals, books, etc.) to those on the Web that lacked quality control mechanisms. The study found that the subjects assessed information quality based on source credibility and authority on both the institutional and individual levels. The results of this study indicated that the range of evidence people used for ascribing source authority was much broader for the Web than that used for traditional printed sources. This is partly because the Web, a “new” medium, was not perceived as an authoritative resource by people. This study was the first to verify Patrick Wilson’s theory of cognitive authority, source credibility, and information quality. Rieh and Belkin’s (1998) work is also one of the earliest empirical studies to indicate that issues of information credibility are indeed important to people who search on the Web.

Building on the results from the exploratory study, Rieh designed an experimental study to identify the specific factors that influenced people’s judgments of quality and authority of information on the Web as well as the effects of those judgments on information searching. The factors were identified in terms of characteristics of information objects, characteristics of sources, knowledge, situation, ranking in search output, and general expectation. The results revealed that knowledge gained from first-hand experience was a primary factor influencing judgments, especially during the prediction stage of the search. Source characteristics, especially source reputation and source type, were also important factors influencing quality and authority judgments. Credibility judgments differed depending on the type of tasks (research, travel, medical, product). Two papers were published from this research: a journal article in the Journal of the American Society for Information Science and Technology (Rieh, 2002) and a conference paper in the Proceedings of the Annual Meeting of the ASIST (Rieh & Belkin, 2000).

Goal 2: To characterize information seeking context and environment embedded in the process of credibility assessment

Rieh has conducted an empirical study about credibility assessment in everyday life information seeking context. This research project differed from her previous studies on credibility in two ways: (1) it investigated credibility assessment across multiple information media and resources including humans, websites, libraries, books, newspapers, and other Internet-mediated communication sites; and (2) it included a variety of information activities in everyday life information seeking context at school, work, and home. The focus of this research was on examining the relationship between credibility judgments and information seeking strategies. The study also looked at how information seeking goals (e.g., academic achievement, problem solving, personal information needs, entertainment, and routines) were related to the extent of credibility concerns. Most subjects were aware of the potential problems of information credibility in the digital media and employed several information seeking strategies for dealing deal with it. For instance, they began their information seeking at a trusted place with which they had previous first-hand experience or they gained second-hand knowledge from their cognitive authorities. They also often used multiple information resources as a way of verifying information for their credibility assessments. The findings emphasized the importance of taking into account the broader contexts of information seeking for credibility research. Two papers came out from this study: A book chapter entitled College Students’ Credibility Judgments in the Information Seeking Process (Rieh & Hilligoss, 2008) in a book Digital Media, Youth, and Credibility, a volume of the John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning (MIT Press) and an article Developing a unifying framework of credibility assessment: Concept, heuristics, and interaction in context (Hilligoss & Rieh, 2008).

Goal 3: To enhance conceptual framework(s) of credibility assessment

Rieh (2002) adopted Robin M. Hogarth’s theory of two distinct kinds of human judgment − predictive judgment and evaluative judgment – to credibility judgments in Web searching. In predictive judgment, people make predictions about the information or source that reflects what they expect to happen. They turn to resources they have used before, have heard of directly from someone, or read of somewhere. In evaluative judgment, people make value judgments in which they express preferences about the information that they just encounter. When the evaluation of the information does not match their expectations made in the predictive judgment, people then start a new Web page or go back to a previous page. In fact, people make such judgments continuously on the Web until they find credible information.

This framework of predictive and evaluative judgments (Rieh, 2002) has been extended to the context of everyday life information seeking by adding the process of verification (Rieh & Hilligoss, 2008). When people have a number of different choices among information seeking paths and resources, predictive judgments guide them in deciding what actions to take. They begin the information seeking process where they think they are most likely to find the best information. Once people have accessed an information resource, they then make evaluative judgments. There are situations in which people need to verify or re-evaluate information after they have made evaluative judgments. People initially accept information, but are later prompted to doubt the credibility as a result of encountering contradictory information. Or people are uncertain about the credibility of information when first encountered and thus engage in the verification process. Rieh’s recent research findings suggest that credibility judgments are a continuous and iterative process from prediction and evaluation through verification rather than a dichotomous decision to believe or not believe the information.

The credibility framework of prediction, evaluation, and verification has further broadened into a multidisciplinary framework developed on the basis of exhaustive literature reviews. In collaboration with David Danielson, Rieh reviewed credibility literature across several disciplines including information science, communication, health science, management and information systems, and consumer behavior (Rieh & Danielson, 2007). In addition to the process of making judgments, four other critical perspectives of credibility assessment were identified: credibility constructs, orientation toward targets of credibility assessment, situational aspects, and evaluator background. This multidisciplinary framework of credibility has impacted several of my subsequent research projects by providing insights on the importance of taking multidisciplinary approaches across a range of information seeking goals, tasks, and contexts in which people use a variety of ICT tools and applications beyond the Web.

Based on empirical data from more than 245 information seeking episodes collected from ten-day information-activity diaries, Rieh and Hilligoss have developed a unifying framework of credibility assessment (Hilligoss & Rieh, 2008) in which credibility is characterized across various types of information resources, multiple media, and diverse information tasks including work and school tasks and personal interests. Using grounded theory analysis, four distinct levels of credibility assessment are identified: construct, heuristics, context, and interaction. The construct level identifies how an individual constructs, conceptualizes, or defines credibility. The heuristics level involves general rules of thumb used to make decisions regarding credibility. The context level involves contextual factors coming into play in a credibility judgment. The interaction level refers to credibility judgments based on specific source or content cues. This framework demonstrates that assessments made in interactions (i.e., cues) are in fact affected by judgments from the other three levels.

This is an exciting time for credibility researchers because of the dynamic and ever changing environment of digital revolution. In today’s participatory Web environment, people are increasingly engaging in a variety of information activities that involve content contribution on the Web by posting new content, copying content from other sources, and explicitly rating and tagging content. This new generation of Information and Communication Technology (ICT), popularly identified as Web 2.0, poses greater challenges than ever before for assessing the credibility of information on the Web because while people have greater access to information posted by individuals rather than by established organizations, more information tends to be disconnected from its origin or source. At the same time, Web 2.0 also provides unprecedented ways of assisting people to make more informed credibility judgments given that people can more easily obtain consumer evaluations.

Rieh’s Previous Credibility Research Publications

Hilligoss, B. & Rieh, S. Y. (2008). Developing a unifying framework of credibility assessment: Concept, heuristics, and interaction in context. Information Processing and Management, 44(4), 1467-1484.

Rieh, S. Y. & Hilligoss, B. (2008). College students’ credibility judgments in the information seeking process. In M. Metzger & A. Flanagin (Eds.), Digital media, youth, and credibility (pp. 49-72), MacArthur Foundation Series on Digital Media and Learning. Cambridge, MA: The MIT Press.

Rieh, S. Y. & Danielson, D. R. (2007). Credibility: A Multidisciplinary framework. In B. Cronin (Ed.), Annual Review of Information Science and Technology (Vol. 41, pp. 307-364). Medford, NJ: Information Today.

Rieh, S. Y. (2002) Judgment of information quality and cognitive authority in the web. Journal of the American Society for Information Science and Technology, 53, 145-161.

Rieh, S. Y. & Belkin, N. J. (2000). Interaction on the web: Scholars’ judgment of information quality and cognitive authority. In D. H. Kraft (Ed.), Proceedings of the 63rd Annual Meeting of the American Society for Information Science Vol. 37 (pp. 25-38). Medford, NJ: Information Today.

Rieh, S. Y. & Belkin, N. J. (1998). Understanding judgment of information quality and cognitive authority in the WWW. In C. M. Preston (Ed.), Proceedings of the 61st Annual Meeting of the American Society for Information Science Vol. 35 (pp. 279-289). Medford, NJ: Information Today.

• Web-based online information activity diary survey
• Interviews with participatory Web users
• Experimental studies to collect behavioral data with respect to the application of credibility assessment heuristics in the processes of information seeking and content creation.

Associate Professor, School of Information, University of Michigan

rieh at umich.edu | 734-647-8040 | http://www.si.umich.edu/rieh

Soo Young Rieh is an associate professor in the School of Information (SI) at the University of Michigan. Rieh’s research interests include credibility assessment, information quality and cognitive authority, information-seeking in everyday life context, mental effort in web searching, and institutional repositories. Rieh is the principal investigator of the Credibility Assessment in the Participatory Web Environment funded by the MacArthur Foundation from 2008-2011. This project investigates what kinds of heuristics Web users employ to make credibility judgments when using user-generated or user-mediated content. Rieh is also the principal investigator of the MIRACLE (Making Institutional Repositories a Collaborative Learning Environment ) project (with Professor Karen Markey and Associate Professor Elizabeth Yakel) funded by the Institute of Museum and Library Services (IMLS) from 2005-08. Previously Rieh held a position as a human factors research engineer at Excite@Home Search and Directory Group. She is a recipient of several awards, including the John Wiley & Sons Best JASIST Paper Award, the ASIST SIGUSE Best Information Behavior Conference Paper Award, and the Eugene Garfield-ALISE Doctoral Dissertation Award. She earned her PhD in Communication, Information, and Library Studies from Rutgers University.

Pursuing PhD, School of Information, University of Michigan

kimym at umich.edu

Yong-Mi Kim is a doctoral student in the School of Information. Her research interests include user-oriented information retrieval, and information seeking and sharing on sites built around user-generated content, such as online question-and-answer sites. She holds a B.S. degree in Computer Science from Virginia Tech, M.S. in Computer Science from the University of North Carolina at Chapel Hill and the MSI (Library & Information Services specialization) from the School of Information. Prior to SI, she was a software engineering consultant specializing in software process improvement.

Pursuing PhD, School of Information, University of Michigan

bstjean at umich.edu

Beth St. Jean is a doctoral student in the School of Information at the University of Michigan and a Graduate Student Research Assistant for the IMLS-funded MIRACLE (Making Institutional Repositories A Collaborative Learning Environment) project. She holds a Bachelor’s Degree in Mathematics from Smith College and a Master’s Degree in Information from the University of Michigan. Her research interests include consumer health information behavior, credibility assessment, information literacy, institutional repositories, and relevance.

Pursuing PhD, School of Information, University of Michigan

jiyeon at umich.edu

Ji-Yeon Yang is a doctoral student in the School of Information at the University of Michigan. She holds a Bachelor’s Degree in Political Science from Ewha Woman’s University and a Master’s Degree in Information from the University of Michigan. Her research interests focus on the credibility assessment of information in organizational work settings, including issues related to collaborative information retrieval and expertise location.