Qualitative Validity

Depending on their philosophical perspectives, some qualitative researchers reject the framework of validity that is commonly accepted in more quantitative research in the social sciences. They reject the basic realist assumption that there is a reality external to our perception of it. Consequently, it doesn’t make sense to be concerned with the “truth” or “falsity” of an observation with respect to an external reality (which is a primary concern of validity). These qualitative researchers argue for different standards for judging the quality of research.

For instance, Guba and Lincoln proposed four criteria for judging the soundness of qualitative research and explicitly offered these as an alternative to more traditional quantitatively-oriented criteria. They felt that their four criteria better reflected the underlying assumptions involved in much qualitative research. Their proposed criteria and the “analogous” quantitative criteria are listed in the table.

Credibility

The credibility criteria involves establishing that the results of qualitative research are credible or believable from the perspective of the participant in the research. Since from this perspective, the purpose of qualitative research is to describe or understand the phenomena of interest from the participant’s eyes, the participants are the only ones who can legitimately judge the credibility of the results.

Transferability

Transferability refers to the degree to which the results of qualitative research can be generalized or transferred to other contexts or settings. From a qualitative perspective transferability is primarily the responsibility of the one doing the generalizing. The qualitative researcher can enhance transferability by doing a thorough job of describing the research context and the assumptions that were central to the research. The person who wishes to “transfer” the results to a different context is then responsible for making the judgment of how sensible the transfer is.

Dependability

The traditional quantitative view of reliability is based on the assumption of replicability or repeatability. Essentially it is concerned with whether we would obtain the same results if we could observe the same thing twice. But we can’t actually measure the same thing twice – by definition if we are measuring twice, we are measuring two different things. In order to estimate reliability, quantitative researchers construct various hypothetical notions (e.g., true score theory) to try to get around this fact.

The idea of dependability, on the other hand, emphasizes the need for the researcher to account for the ever-changing context within which research occurs. The research is responsible for describing the changes that occur in the setting and how these changes affected the way the research approached the study.

Confirmability

Qualitative research tends to assume that each researcher brings a unique perspective to the study. Confirmability refers to the degree to which the results could be confirmed or corroborated by others. There are a number of strategies for enhancing confirmability. The researcher can document the procedures for checking and rechecking the data throughout the study. Another researcher can take a “devil’s advocate” role with respect to the results, and this process can be documented. The researcher can actively search for and describe and negative instances that contradict prior observations. And, after he study, one can conduct a data audit that examines the data collection and analysis procedures and makes judgements about the potential for bias or distortion.

There has been considerable debate among methodologists about the value and legitimacy of this alternative set of standards for judging qualitative research. On the one hand, many quantitative researchers see the alternative criteria as just a relabeling of the very successful quantitative criteria in order to accrue greater legitimacy for qualitative research. They suggest that a correct reading of the quantitative criteria would show that they are not limited to quantitative research alone and can be applied equally well to qualitative data. They argue that the alternative criteria represent a different philosophical perspective that is subjectivist rather than realist in nature. They claim that research inherently assumes that there is some reality that is being observed and can be observed with greater or less accuracy or validity. if you don’t make this assumption, they would contend, you simply are not engaged in research (although that doesn’t mean that what you are doing is not valuable or useful).

Perhaps there is some legitimacy to this counter argument. Certainly a broad reading of the traditional quantitative criteria might make them appropriate to the qualitative realm as well. But historically the traditional quantitative criteria have been described almost exclusively in terms of quantitative research. No one has yet done a thorough job of translating how the same criteria might apply in qualitative research contexts. For instance, the discussions of external validity have been dominated by the idea of statistical sampling as the basis for generalizing. And, considerations of reliability have traditionally been inextricably linked to the notion of true score theory.

But qualitative researchers do have a point about the irrelevance of traditional quantitative criteria. How could we judge the external validity of a qualitative study that does not use formalized sampling methods? And, how can we judge the reliability of qualitative data when there is no mechanism for estimating the true score? No one has adequately explained how the operational procedures used to assess validity and reliability in quantitative research can be translated into legitimate corresponding operations for qualitative research.

While alternative criteria may not in the end be necessary (and I personally hope that more work is done on broadening the “traditional” criteria so that they legitimately apply across the entire spectrum of research approaches), and they certainly can be confusing for students and newcomers to this discussion, these alternatives do serve to remind us that qualitative research cannot easily be considered only an extension of the quantitative paradigm into the realm of nonnumeric data.