John et al. used multiple methods to assess the prevalence of questionable research practices among psychology researchers. They found a surprising high prevalence of such practices in their study.
They note that some questionable research practices are indeed, questionable, but that the researchers surveyed also found many of the practices to be unjustifiable:
As noted in the introduction, there is a large gray area of acceptable practices. Although falsifying data (Item 10 in our study) is never justified, the same cannot be said for all of the items on our survey; for example, failing to report all of a study’s dependent measures (Item 1) could be appropriate if two measures of the same construct show the same significant pattern of results but cannot be easily combined into one measure. Therefore, not all self-admissions represent scientific felonies, or even misdemeanors; some respondents provided perfectly defensible reasons for engaging in the behaviors. Yet other respondents provided justifications that, although self categorized as defensible, were contentious (e.g., dropping dependent measures inconsistent with the hypothesis because doing so enabled a more coherent story to be told and thus increased the likelihood of publication). It is worth noting, however, that in the follow-up survey—in which participants rated the behaviors regardless of personal engagement—the defensibility ratings were low. This suggests that the general sentiment is that these behaviors are unjustifiable.Even so, there are incentives in the research world to engage in questionable research practices:
The authors suggest that the prevalence of questionable research practices may help to explain the finding the many studies cannot be replicated:
I think I am on safe ground when I say that the problem of questionable research practices goes well beyond the discipline of psychology.