ARTICLE 10
How Missing Data Can Change a Conclusion
An unanswered survey question is not the same as an answer of zero. It is also not always an insignificant gap. When information is missing, the important issue is not simply how many values remain. Analysts need to consider whether the absence of information is related to what they are trying to measure.
The U.S. Census Bureau distinguishes between a missing response from an entire eligible unit and missing answers to particular questions. It also explains that nonresponse can introduce bias when respondents differ from nonrespondents in relevant ways. A low response rate can increase concern, but the rate alone does not reveal the size or direction of the bias. Analysts may use adjustments or imputation, depending on the design and purpose of the analysis. Those procedures rely on information and assumptions; they do not turn unobserved answers into directly observed facts.
Imagine a hypothetical survey asking customers to rate a service visit. Customers whose appointments were canceled never receive the survey link. Among people who respond, most give favorable ratings. That result may accurately describe respondents’ experiences, but it would be risky to present it as a complete account of all scheduled customers. The missing experiences are connected to the process being evaluated. Simply sending additional reminders to the people who already received a link would not reach those excluded customers.
The practical question is therefore “Who or what might be absent, and why?” Sometimes the answer will reduce concern; sometimes it will change the interpretation of the entire result. Reporting missing information openly gives readers a way to judge the limits of a conclusion. It also creates an opportunity to improve the next collection process. Careful analysis includes the discipline to distinguish what the available evidence supports from what a more complete dataset might have shown.
Interactive questions appear when this published page runs JavaScript.
