Missing data: complete-case analysis, imputation and what each assumes
Every method for handling missing data makes an assumption about why it is missing. The assumption is usually untestable, which is exactly why it has to be stated.
Missing data is not a nuisance to be cleared before the real analysis. How it is handled is part of the analysis, and the choice can change the conclusion.
Three mechanisms
- Missing completely at random. Missingness is unrelated to anything, observed or not. Rare in practice.
- Missing at random. Missingness depends on observed variables, but not on the unobserved value itself. Older patients miss more visits, and you recorded age.
- Missing not at random. Missingness depends on the unobserved value. Patients doing badly stop attending, and how badly is exactly what you failed to record.
The names are unhelpful — "at random" does not mean random — but the distinction determines which methods are valid.
Complete-case analysis
Dropping incomplete records is valid under missing completely at random, and biased otherwise. It also discards information: with several variables each slightly incomplete, the complete cases can be a small and unrepresentative fraction of the sample.
It is defensible when missingness is genuinely trivial, and should be justified rather than defaulted to.
Multiple imputation
Missing values are filled in several times from a model that uses the observed data, producing several complete datasets. Each is analysed and the results combined by rules that carry the imputation uncertainty into the final standard errors.
Two points people get wrong.
The imputation model must include the outcome. Leaving it out biases associations towards the null, which feels safe and is not.
It assumes missing at random. Under missing not at random it is biased, though usually less so than complete-case analysis.
What not to do
Last observation carried forward assumes the patient stopped changing at the moment they stopped being observed, which is precisely the assumption most likely to be false in a deteriorating condition.
Mean imputation understates variance and distorts relationships. Both remain in use and neither is defensible in modern analysis.
Report and test
State how much data was missing, per variable, and what you think caused it. Describe the method and its assumption. Then run a sensitivity analysis under a plausible departure — a delta-adjusted or pattern-mixture approach — and report whether the conclusion survives.
A result that holds under a reasonable missing-not-at-random scenario is considerably stronger than one that has only been imputed.
Frequently asked questions
What is the difference between MCAR, MAR and MNAR?
Missing completely at random means missingness is unrelated to anything. Missing at random means it depends on observed variables but not on the unobserved value. Missing not at random means it depends on the value that is missing — the case standard methods cannot handle.
When is complete-case analysis acceptable?
Under missing completely at random, and when the amount of missing data is genuinely trivial. Otherwise it is biased and discards information, and the choice should be justified rather than defaulted to.
Should the outcome be included in the imputation model?
Yes. Omitting it biases associations towards the null, which feels conservative and is not. It is one of the most common errors in applied multiple imputation.