Heteroskedasticity: Unequal Residual Spread Threatens the Standard Error, Not the Coefficient
On this page
อ่านฉบับภาษาไทย (Thai version)
Abstract
Skedasticity is the width of the residuals in a regression: whether the gaps between observed and predicted values spread equally wide at every level of the predicted value. When the spread stays about the same, as when systolic blood pressure predicted from age is off by about 10 mmHg at every age, the errors are homoskedastic. When it widens, as when a hospital cost near 500,000 baht is off by about 200,000 baht but one near 5,000 baht by only 1,000, the residual plot fans out and the errors are heteroskedastic. Ordinary least squares assumes a constant error variance. Breaking that assumption does not by itself make a coefficient wrong. It makes the ordinary standard error wrong, and with it the 95% confidence interval, the P value and the hypothesis test: a slope of 2 can carry a standard error of 0.5 when 1.0 is right. This article shows how to recognise the pattern and why the answer is a heteroskedasticity-robust standard error.
A cost model whose residuals fan out
A hospital team fits a linear regression of the cost of an admission on the patient's age, length of stay and number of procedures. The coefficients make clinical sense. Then someone plots the residuals against the fitted values. For the cheap admissions the points hug zero; for the most expensive ones they spray far above and below it.
Two worries follow at once. Is the coefficient for length of stay now wrong? And can the team still trust its confidence interval and P value? Both answers start from one word: skedasticity.
Skedasticity: how wide the residuals spread
The word skedasticity is about the spread of the errors, or residuals, in a regression. Put as simply as possible, a regression model splits every outcome into two parts:
$$Y = \text{the model's predicted value} + \text{error}$$The predicted value is also called the fitted value. A residual is the distance between the real, observed value and the value the model predicted: observed minus predicted.
Skedasticity asks one question about those residuals:
Do the residuals spread equally wide at every level of the predicted value?
Homoskedasticity: about the same spread everywhere
Suppose a model predicts systolic blood pressure (SBP) from age. At age 20 its predictions are off by about ±10 mmHg. At age 40 they are off by about ±10 mmHg, and at age 70 by about ±10 mmHg again. The variance of the residuals is roughly constant.
This is called homoskedasticity. The picture to keep in mind is the residuals plotted against the fitted values: a band of about the same width all the way along, as in the left panel of the figure below.
| Age | Typical prediction error for SBP |
|---|---|
| 20 years | about ±10 mmHg |
| 40 years | about ±10 mmHg |
| 70 years | about ±10 mmHg |
Heteroskedasticity: the spread changes
Now suppose the model predicts hospital cost. For a patient whose care costs about 5,000 baht, the prediction may be off by only about ±1,000 baht. For a patient whose care costs about 500,000 baht, it may be off by about ±200,000 baht.
As the fitted value rises, the residuals spread wider and wider. On the residual plot they open out like a cone, a fan shape, as in the right panel below. This is called heteroskedasticity.
| Predicted hospital cost | Typical prediction error |
|---|---|
| about 5,000 baht | about ±1,000 baht |
| about 500,000 baht | about ±200,000 baht |
Where the idea is used: linear regression and OLS
The idea is used mainly in linear regression fitted by ordinary least squares (OLS), the usual fitting method, which chooses the coefficients that make the sum of squared residuals as small as possible. Suppose the analysis is
$$\text{Blood pressure} = \beta_0 + \beta_1(\text{age}) + \beta_2(\text{BMI}) + \varepsilon$$where BMI is body mass index and $\varepsilon$ is the error. OLS makes an important assumption about that error:
$$\mathrm{Var}(\varepsilon \mid X) = \sigma^2$$Here $X$ stands for the covariates, age and BMI, and $\sigma^2$ is one fixed variance. In words: as $X$ changes, the variance of the error should stay about the same.
If the variance is not constant, the errors are said to show heteroskedasticity.
Why it matters: the coefficient may be fine, the standard error may not
The most important point is this: heteroskedasticity does not mean that a coefficient is wrong straight away. In the blood pressure model, an estimate such as
$$\hat\beta_{\text{BMI}} = 2.5$$may still be a good estimate.
The problem is that the standard error may be computed wrongly. What is built on it can then be wrong too:
- the 95% confidence interval may be wrong;
- the P value may be wrong;
- the hypothesis test may be wrong.
An assumption about the variance of the errors
Put formally, skedasticity is about the assumption on the variance of the residuals, or errors, in a regression, above all in linear regression fitted by OLS. The assumption is
$$\mathrm{Var}(\varepsilon \mid X) = \sigma^2$$which says: as the value of $X$ changes, the variance of the error stays constant. That is homoskedasticity.
If $\mathrm{Var}(\varepsilon \mid X)$ changes with $X$, or with the fitted value, that is heteroskedasticity.
Why the assumption matters: OLS gives two separate things
OLS gives two separate things:
- the coefficient estimate, such as $\hat\beta_1$;
- the uncertainty of the coefficient: its standard error, 95% confidence interval and P value.
If the mean model is right and exogeneity holds (the errors average zero whatever the values of $X$, so the covariates carry no information about the error), the coefficient can still be estimated well even under heteroskedasticity.
But the ordinary OLS standard error will not be correct, because its formula relies on the error variance being constant. The main consequences are:
- the standard error is wrong;
- the 95% confidence interval is wrong;
- the P value is wrong;
- the hypothesis test may reach the wrong conclusion.
Hand example: a slope of 2 that looks more precise than it is
Hand example. Take the simplest model, with one covariate $X$:
$$Y = \beta_0 + \beta_1 X + \varepsilon$$-
The estimate
\[ \hat\beta_1 = 2 \]
Suppose the fitted slope is 2. Heteroskedasticity does not, by itself, change this number.
-
If the errors are homoskedastic
\[ SE = 0.5 \]
With a constant error variance, the ordinary OLS standard error is right: 0.5.
-
If the errors are heteroskedastic and the ordinary standard error is used
\[ SE_{\text{ordinary}} = 0.5 \quad \text{but it should be} \quad SE = 1.0 \]
With a fan-shaped spread, the ordinary formula may still give 0.5, when the standard error should really be 1.0.
-
What the 95% confidence interval then claims
\[ 2 \pm 1.96 \times 0.5 = 1.02 \text{ to } 2.98 \quad \text{versus} \quad 2 \pm 1.96 \times 1.0 = 0.04 \text{ to } 3.96 \]
Built on the ordinary standard error, the interval is half as wide as it should be, and the P value is smaller than it should be.
Result: The ordinary standard error says 0.5 when it should be 1.0, so the effect looks more precise than it really is.
The same pattern in a simulated cohort
The same thing shows up in a simulated cohort of 2,000 adults aged 30 to 80, whose systolic blood pressure scatters more with age: the random scatter added to each person's SBP has a standard deviation of 4 mmHg at age 30, 9 at 55 and 24 at 80. Residuals from a straight line of SBP on age fan out toward the older ages.
The straight-line regression of SBP on age gives a slope of 0.66 mmHg per year, close to the straight-line slope of 0.67 in the large simulated population the cohort was drawn from. Its ordinary standard error is 0.0215. Refitted with a heteroskedasticity-robust standard error, Stata's vce(robust), the slope is exactly the same and the standard error rises to 0.0243, 1.13 times the ordinary one.
As in the hand example, the coefficient stays put and only its uncertainty changes; here the ordinary standard error was too small. It is not too small in every dataset, which is one more reason not to rely on it when the spread is unequal.
Stata: SBP on age with the ordinary standard error
* Specification A: SBP on age (the high-leverage ages are the high-variance ages)
regress sbp age. * Specification A: SBP on age (the high-leverage ages are the high-variance ages)
. regress sbp age
Source | SS df MS Number of obs = 2,000
-------------+---------------------------------- F(1, 1998) = 951.44
Model | 189678.648 1 189678.648 Prob > F = 0.0000
Residual | 398321.189 1,998 199.359955 R-squared = 0.3226
-------------+---------------------------------- Adj R-squared = 0.3222
Total | 587999.837 1,999 294.146992 Root MSE = 14.119
------------------------------------------------------------------------------
sbp | Coefficient Std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------
age | .6644293 .0215406 30.85 0.000 .6221848 .7066737
_cons | 88.26957 1.232598 71.61 0.000 85.85226 90.68689
------------------------------------------------------------------------------Stata: the same fit with a heteroskedasticity-robust standard error
regress sbp age, vce(robust). regress sbp age, vce(robust)
Linear regression Number of obs = 2,000
F(1, 1998) = 750.63
Prob > F = 0.0000
R-squared = 0.3226
Root MSE = 14.119
------------------------------------------------------------------------------
| Robust
sbp | Coefficient std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------
age | .6644293 .0242514 27.40 0.000 .6168686 .7119899
_cons | 88.26957 1.203478 73.35 0.000 85.90937 90.62978
------------------------------------------------------------------------------vce(robust): the slope is unchanged, and its standard error rises from 0.0215 to 0.0243.Reading the pattern: the mean is fine, the variance is not
The pattern is often summed up in one line of a methods review or a statistical report:
The mean structure is appropriate, but the residual variance changes with the fitted values.
That line means the coefficient model is not wrong, but the variance assumption is wrong. So the coefficients stay, and only the way their uncertainty is computed changes:
Use heteroskedasticity-robust standard errors.
Common misreadings and their fixes
-
"The residuals fan out, so the coefficient is wrong."
Heteroskedasticity is about the variance of the errors, not their mean. If the mean model is right, a coefficient such as an estimated BMI effect of 2.5 can still be a good estimate.
Fix: Keep the coefficient and repair its uncertainty: report a heteroskedasticity-robust standard error.
-
"Heteroskedasticity means the regression line bends."
Whether the line curves is a question about the mean model. Heteroskedasticity asks a different question: whether the spread around the line is equally wide all the way along.
Fix: Look at the width of the band in the plot of residuals against fitted values, not at its shape.
-
"The coefficient looks fine, so the printed standard error is fine too."
The ordinary OLS standard error is built on a constant error variance. With a fan-shaped spread it can say 0.5 when 1.0 is right.
Fix: When the spread is unequal, read the robust standard error, and the confidence interval and P value built from it.
-
"Unequal spread means the whole model must be rebuilt."
When the mean structure is appropriate and only the residual variance changes with the fitted values, the mean equation is not the problem; the variance assumption is.
Fix: Keep the mean model and use heteroskedasticity-robust standard errors.
Glossary
- skedasticity
- The spread, or width, of the residuals around a regression's predicted values, and whether that width is the same at every predicted value.
- homoskedasticity
- Residuals that spread about equally wide at every level of the fitted value: a constant error variance.
- heteroskedasticity (ความแปรปรวนไม่คงที่)
- A residual variance that changes with the covariates or the fitted value, often seen as a fan shape in a residual plot.
- residual
- The observed value minus the value the model predicted.
- fitted value
- The value the regression model predicts for a person from their covariates.
- ordinary least squares (OLS)
- The usual way to fit a linear regression: choose the coefficients that make the sum of squared residuals as small as possible.
- exogeneity
- The errors average zero whatever the covariate values, so the covariates carry no information about the error.
- model-based standard error (standard error แบบอิงแบบจำลอง)
- The ordinary standard error from the model's own formula; in OLS it assumes one constant error variance.
- robust (sandwich) standard error (ค่าคลาดเคลื่อนมาตรฐานแบบแซนด์วิช)
- A standard error built from each person's own squared residual, so it does not rely on constant variance; the coefficient itself is unchanged.
References
- White H. A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica. 1980;48(4):817-838. doi:10.2307/1912934 https://doi.org/10.2307/1912934
Key takeaways
- Skedasticity is the width of the residuals: homo means the same variance at every fitted value, hetero means a different variance.
- Heteroskedasticity does not ask whether the regression line curves; it asks whether the spread around the line is equally wide throughout, and its signature is a fan in the plot of residuals against fitted values.
- OLS assumes a constant error variance; when that fails, the coefficient is not automatically wrong.
- What goes wrong is the ordinary standard error, and with it the 95% confidence interval, the P value and the test: a standard error of 0.5 where 1.0 is right makes an effect look more precise than it is.
- When the mean structure is appropriate but the residual variance changes with the fitted values, the variance assumption is wrong, not the mean model: use heteroskedasticity-robust standard errors.
Related in the wiki: [[risk-regression-models-epidemiology]] [[repeated-measures-modeling-guide]]