Statistics – Flawed Process – PPDAC
…
Identification
Persuasive statistics are objectively sound and they all follow a process. Ones that cut corners should be dismissed. Statistician David Spiegelhalter is a proponent of the statistical inquiry method of "Problem, Plan, Data, Analysis, and Conclusion" or PPDAC. You may recall your elementary school flavors of this in the scientific method (e.g., Hypothesis, Test, Results, Conclusion, Re-hypothesis). A sound statistical use process involves five steps. Many who bandy about numbers, conduct "surveys" or DIY studies must be tempered by proper process. Undermining the flaws in the statistical process is the work of peers in scientific literature and the imperative of any good business person or policymaker. Any questionable or imprecise method or execution of the process is grounds to undermine the purported objective study. Spiegelhalter's process and inquiries include not just knowing the steps but understanding them:
- Problem
- Was the problem defined properly?
- Framing of the question to answer (see, Positive and Negative Framing).
- Plan
- What and how to measure?
- How is the study designed?
- How is it recorded,
- How is data to be collected?
- Data
- How was data actually collected?
- How is data managed?
- How is data cleaned and validated?
- Analysis
- Proper sorting of data?
- Construction of tables, graphs, info-graphics - are they actually, culturally or psychologically misleading?
- Do patterns emerge?
- Can hypotheses be generated from the analysis?
- Conclusion
- Is interpretation sound?
- Conclusions?
- Any new ideas for further inquiry or unanswered questions?
- Are results communicated accurately?
Example
The following example shows errors in the media-filtered conclusion, interpretation, and presentment of data. It was widely reported that "those who eat cured meats( like bacon) increase their risk of colon cancer by 18%," which is a type of risk (see Relative Risk). The actual data is that in the general population, your risk is about 6%; for those who eat bacon sandwiches every day, the risk is 7%, an overall 18% increase, but still a relatively low risk for the general population. The scientists themselves need to be wary of trying to get a big-splash title.