How the public, and scientists, perceive advancement of knowledge from conflicting study results
Journal Title: Judgment and Decision Making - Year 2019, Vol 14, Issue 6
Abstract
Science often advances through disagreement among scientists and the studies they produce. For members of the public, however, conflicting results from scientific studies may trigger a sense of uncertainty that in turn leads to a feeling that nothing new has been learned from those studies. In several scenario studies, participants read about pairs of highly similar scientific studies with results that either agreed or disagreed, and were asked, “When we take the results of these two studies together, do we now know more, less, or the same as we did before about (the study topic)?” We find that over half of participants do not feel that “we know more” as the result of the two new studies when the second study fails to replicate the first. When the two study results strongly conflict (e.g., one finds a positive and the other a negative association between two variables), a non-trivial proportion of participants actually say that “we know less” than we did before. Such a sentiment arguably violates normative principles of statistical and scientific inference positing that new study findings can never reduce our level of knowledge (and that only completely uninformative studies can leave our level of knowledge unchanged). Drawing attention to possible moderating variables, or to sample size considerations, did not influence people’s perceptions of knowledge advancement. Scientist members of the American Academy of Arts and Sciences, when presented with the same scenarios, were less inclined to say that nothing new is learned from conflicting study results.
Authors and Affiliations
Derek J. Koehler and Gordon Pennycook
Keywords
Related Articles
- EP ID EP678448
- DOI -
- Views 179
- Downloads 0
Reasons for cooperation and defection in real-world social dilemmas
Interventions to increase cooperation in social dilemmas depend on understanding decision makers’ motivations for cooperation or defection. We examined these in five real-world social dilemmas: situations where private i...
Easy does it: The role of fluency in cue weighting
We propose that people weight fluent, or easy to process, information more heavily than disfluent information when making judgments. Cue fluency was manipulated independent of objective cue validity in three studies, the...
Integrating theories of law obedience: How utility-theoretic factors, legitimacy, and lack of self-control influence decisions to commit low-level crimes
We conducted two studies using a sample of students (Experiment 1, N=84) and the general public (Experiment 2, N=412) to assess the relative and unique effects of factors suggested by three major theories of law obedienc...
Biases in casino betting: The hot hand and the gambler’s fallacy
We examine two departures of individual perceptions of randomness from probability theory: the hot hand and the gambler’s fallacy, and their respective opposites. This paper’s first contribution is to use data from the...
The impact of near-miss events on betting behavior: An examination of casino rapid roulette play
We examine how almost winning in roulette affects subsequent betting behavior. Our main finding is heterogeneity in gambler behavior with some gamblers less likely to bet on numbers that were near misses on the prior spi...