ARTICLE 08
The Hidden Decisions Behind Every Dataset
A dataset often looks like a finished object: rows, columns, and values ready to analyze. Yet each part reflects decisions made before the analyst begins. Someone selected what to observe, how to record it, and which cases to include. Understanding those decisions is essential because a large collection of precise measurements can still answer the wrong question.
Data-quality guidance from the U.S. Environmental Protection Agency emphasizes the purpose of an analysis, the comparability of measurements, and the documentation of inclusion and exclusion decisions. These concerns extend beyond environmental data. Two columns with the same label may represent different procedures or time periods. A record removed as an error may be a genuine but unusual observation. Cleaning data therefore requires judgment, not just the elimination of inconvenient values. The reason for a change should remain available to the people who later interpret the results.
Consider a hypothetical analyst comparing how long customers wait for help at two branches. One branch starts its timer when a customer joins the line; the other starts when the customer reaches a service desk. Both columns are labeled “waiting time.” Averaging them would produce a number, but the apparent consistency would be misleading. Before comparing branches, the analyst would need comparable definitions or a clear account of the difference. More decimal places would not repair the underlying mismatch.
Documentation makes these hidden decisions visible. It can identify units, collection methods, missing values, and changes made during preparation. Such information does not guarantee that a dataset is suitable for every purpose. It allows another person to judge its suitability more intelligently. For data scientists, asking where a number came from is part of the analysis itself. A trustworthy result begins not with the most sophisticated calculation, but with an adequate understanding of what has actually been counted.
Interactive questions appear when this published page runs JavaScript.
