When IPTW Produces Extreme Weights: Diagnose Before You Estimate

อ่านฉบับภาษาไทย (Thai version)

Abstract

Extreme inverse probability of treatment weighting (IPTW) weights are not a new problem to be fixed but a diagnostic finding: once the weights are built, the weighted pseudo-population may turn out to be dominated by a few patients. Each patient is weighted by one over the probability of the treatment actually received, so a treated patient with a propensity score of 0.02 gets a weight of 50, and a control patient with a score of 0.99 gets a weight of 100. Such weights point back to poor overlap between the treatment groups, to near-violations of positivity, or to a propensity model that misses nonlinear terms or interactions. The right response is to investigate why: inspect the propensity-score distributions, the overlap, the largest weights and covariate balance, revise the model if needed, and recheck before any outcome is estimated. This article follows that workflow: weight first, diagnose second, estimate the outcome third.

---

🔍 Click to enlarge Click anywhere to close

Two weights that stop the analysis

A team is comparing two treatments for the same disease in routine hospital records. Treatment A is given mostly to patients with severe disease and Treatment B mostly to mild cases, so the analyst fits a propensity score (PS) model: a logistic regression of the probability of receiving Treatment A on age and severity. From it she builds inverse probability of treatment weights.

Before fitting any outcome model, she sorts the weights. Most sit near 1 or 2, but two stand out: a treated patient with a weight of 50 and a control patient with a weight of 100.

The quick path is to fit the outcome model anyway and read its P value. This article takes the other path. It treats the two weights as a finding, asks what they say about the comparison, and settles that before any treatment effect is estimated.

IPW, IPTW and where extreme weights fit

Inverse probability weighting (IPW) is the larger family: each patient is weighted by one over the probability of what actually happened to them, such as the treatment they received or their staying under observation. Inverse probability of treatment weighting (IPTW) is the member of that family that weights by treatment assignment. So extreme IPTW weights are not a new problem that turns up after some separate problem of extreme IPW has been solved. They are a diagnostic problem, found after the IPTW weights have been built.

The weights create a pseudo-population, a weighted version of the sample in which treatment no longer depends on the measured covariates. We use IPTW to solve a treatment-selection problem. Extreme weights tell us that, once the pseudo-population has been built, the weighting solution itself is not stable. The series part on the IPW family covers its other members.

What if IPTW produces extreme weights?

IPTW is designed to improve comparability between treatment groups. However, creating the weights is not the end of the analysis. After IPTW has been calculated, the weights themselves must be examined, and sometimes we find:

A small number of patients receive extremely large weights.

This creates a new problem. The original problem was:

Treatment groups were not comparable.

We attempted to address it using:

$$\boxed{\text{IPTW}}$$

But after weighting, we may discover:

The weighted pseudo-population is being dominated by a few observations.

Why do extreme IPTW weights occur?

Write $A = 1$ for Treatment A and $A = 0$ for the comparison treatment, $X$ for a patient's baseline characteristics and $e(X) = P(A = 1 \mid X)$ for the propensity score. For an IPTW analysis that targets the average treatment effect (ATE), the effect in the whole population the patients represent, the weights are:

for treated patients

$$w_i = \frac{1}{e(X_i)}$$

and for control patients

$$w_i = \frac{1}{1 - e(X_i)}$$

Two patients, two extreme weights

Take the two weights from the opening scene and work back to the propensity scores behind them.

1. A treated patient with $e(X) = 0.02$

   \[ w = \frac{1}{0.02} = 50 \]

   This patient had only a 2% estimated probability of receiving treatment, yet actually received treatment.

2. A control patient with $e(X) = 0.99$

   \[ w = \frac{1}{1 - 0.99} = 100 \]

   This single patient may contribute approximately 100 times as much as a patient with weight 1.

3. The rule behind both

   \[ \boxed{PS \approx 0 \text{ or } 1 \rightarrow \text{Extreme IPTW}} \]

   A weight becomes extreme when a patient received the treatment that was very unlikely for someone like them.

Result: A propensity score near 0 or 1 produces an extreme IPTW weight: 50 for the treated patient at 0.02, 100 for the control patient at 0.99.

Extreme weights point back to the overlap problem

Overlap asks: do patients with similar baseline characteristics actually exist in both treatment groups?

Suppose very severe patients almost always receive Treatment A:

$$P(A = 1 \mid \text{Severe}) = 0.98$$

while mild patients almost always receive Treatment B:

$$P(A = 1 \mid \text{Mild}) = 0.03$$

Although a propensity-score model can still generate probabilities for these patients, the observed data contain very little treatment contrast within these regions. Conceptually, the two groups' propensity scores look like this:

Propensity score

Control     ███████████
                         █

Treated                  ███████████

There is little common support: few propensity-score values at which both treated and control patients are found [2]. IPTW then attempts to reconstruct comparability by giving very large weights to the rare patients who received the unusual treatment.

The three questions to ask first

When extreme IPTW weights appear, do not proceed immediately to the outcome model. Investigate three related issues: overlap, positivity and the specification of the propensity model.

A. Overlap

Ask: are treated and control patients represented across similar regions of the propensity-score distribution?

Inspect the PS distributions of both treatment groups. If one group occupies regions where the other group has almost no observations, IPTW may become unstable.

B. Positivity

Positivity is the underlying causal requirement that patients with a particular set of measured characteristics must have some possibility of receiving either treatment [3]. Conceptually:

$$0 < P(A = 1 \mid X) < 1$$

If certain patient profiles virtually always receive one treatment, the data contain little information about the counterfactual treatment experience for those patients: what would have happened to them under the treatment they did not receive. In practical data,

$$P(A = 1 \mid X) \approx 0 \quad \text{or} \quad P(A = 1 \mid X) \approx 1$$

may therefore appear as extremely large IPTW weights.

C. Propensity-model specification

Extreme weights can also make us reconsider whether the treatment model adequately represents the treatment-assignment process. Suppose the original model is:

$$\operatorname{logit}[P(A = 1)] = \beta_0 + \beta_1 \text{Age} + \beta_2 \text{Severity}$$

but treatment decisions actually depend strongly on nonlinear severity or on interactions. For example,

$$\text{Severity}^2 \quad \text{or} \quad \text{Age} \times \text{Severity}$$

may be clinically important. A poorly specified propensity model may therefore produce inadequate balance or problematic weights [4]. The model should be reconsidered using clinical knowledge and weighting diagnostics, rather than relying only on statistical significance [1]. The balance-checking part of this series shows how to read covariate balance after such a revision.

Do not ask whether the outcome is significant yet

A common mistake is to proceed directly from IPTW to the treatment-effect estimate:

$$\text{IPTW} \rightarrow \text{Outcome model} \rightarrow P\text{ value}$$

This skips the most important intermediate step. The correct workflow is:

$$\boxed{\text{PS} \rightarrow \text{IPTW} \rightarrow \text{Weight diagnostics} \rightarrow \text{Balance} \rightarrow \text{Overlap} \rightarrow \text{Outcome analysis}}$$

If the weights are unstable, a statistically significant treatment estimate does not solve the underlying design problem. The credibility of the weighted comparison must be established before interpreting the treatment effect [1].

A useful clinical analogy

Imagine trying to compare Drug A and Drug B. Almost every patient with extremely severe disease receives Drug A. Almost every patient with mild disease receives Drug B. There are only two extremely severe patients who received Drug B.

IPTW may assign those two unusual Drug B patients enormous weights, because they are being asked to statistically represent many severe patients who, in the observed data, almost never received Drug B.

The correct response to extreme IPTW weights

The appropriate response is not:

$$\text{Extreme weights} \rightarrow \text{ignore them} \rightarrow \text{report treatment effect}$$

Instead:

$$\boxed{\text{Extreme weights} \rightarrow \text{investigate why}}$$

Then examine:

1. propensity-score distributions,
2. treatment-group overlap,
3. the largest individual weights,
4. covariate balance,
5. clinically important nonlinearities or interactions in the treatment model.

If the original propensity-score model is inadequate, revise the model and calculate the weights again. For example:

$$\text{Original PS} \rightarrow \text{Extreme weights}$$$$\downarrow$$$$\text{Reconsider model}$$$$\downarrow$$$$\text{New PS} \rightarrow \text{New IPTW}$$$$\downarrow$$$$\text{Recheck balance} + \text{overlap} + \text{weights}$$

Only after acceptable diagnostics should the analysis move to the treatment outcome.

The important conceptual shift

This produces an important causal-inference lesson. Initially, we may think:

$$\boxed{\text{Confounding problem} \rightarrow \text{IPTW solution}}$$

But the actual workflow is:

$$\boxed{\text{Confounding} \rightarrow \text{IPTW} \rightarrow \text{Check whether IPTW produced a credible comparison}}$$

Final memory box

Original problem: treatment groups differ at baseline.

Proposed solution: IPTW.

New diagnostic question: did IPTW create a stable, balanced, overlapping comparison?

If a few observations receive extremely large weights:

$$\boxed{\text{Extreme IPTW} \rightarrow \text{Check overlap, positivity and PS model}}$$

What comes next: when the weights stay extreme

Diagnosis comes first. If overlap, positivity and the propensity model have been examined, the model has been revised where it was inadequate, and some weights are still extreme, the next question is how to deal with them. That is a separate step with its own choices: unstabilised versus stabilised IPTW (weights rescaled by the overall share of patients on each treatment), truncation (capping the largest weights) or trimming (removing patients with extreme propensity scores), alternative weighting such as overlap weighting (weights that focus on patients for whom both treatments were plausible), and sensitivity analysis. Each of them changes something about the comparison, so each belongs after the diagnosis, never in place of it.

Common misreadings and their fixes

• "Extreme IPTW weights are a new problem, separate from IPW."

  IPW is the family and IPTW is its member that weights by treatment assignment. Extreme IPTW weights are what the diagnostics find after the IPTW weights have been built.

  Fix: Treat extreme weights as a diagnostic finding about the weighting solution you have just built.

• "Once the weights are created, confounding is handled."

  Creating the weights is not the end of the analysis; the weighted pseudo-population may be dominated by a few observations.

  Fix: IPTW must always be followed by weighting diagnostics.

• "The model gave every patient a probability, so the data support the comparison."

  A propensity model can produce probabilities such as 0.98 for severe and 0.03 for mild patients even where the data contain very little treatment contrast.

  Fix: Inspect the PS distributions of both treatment groups and ask whether they share common support.

• "Go from IPTW straight to the outcome model and its P value."

  This skips the most important intermediate step.

  Fix: Follow PS, then IPTW, then weight diagnostics, balance and overlap, and only then the outcome analysis.

• "A significant treatment effect shows the weighting worked."

  If the weights are unstable, a statistically significant estimate does not solve the underlying design problem.

  Fix: Establish the credibility of the weighted comparison before interpreting the treatment effect.

• "Extreme weights can be ignored as long as the effect is reported."

  Ignoring them and reporting the treatment effect leaves the reason for them unknown.

  Fix: Investigate why: examine the PS distributions, group overlap, the largest individual weights, covariate balance and clinically important nonlinearities or interactions in the treatment model.

• "Keep the propensity-model terms that are statistically significant."

  A model that omits clinically important terms such as Severity squared or Age by Severity can give inadequate balance or problematic weights, whatever its P values.

  Fix: Reconsider the model using clinical knowledge and weighting diagnostics rather than relying only on statistical significance.

Glossary

inverse probability weighting (IPW)

A family of methods that weight each patient by one over the probability of what actually happened to them, such as the treatment received or staying under observation.

IPTW

Inverse probability of treatment weighting: the member of the IPW family that weights by treatment assignment, 1/e(X) for treated and 1/(1 - e(X)) for control patients when the target is the ATE.

propensity score (PS)

The probability of receiving treatment given baseline characteristics, written e(X) = P(A = 1 | X).

ATE

The average treatment effect in the whole population the patients represent.

pseudo-population

The weighted version of the sample in which treatment no longer depends on the measured covariates.

weighting diagnostics

The checks run after the weights are built and before the outcome: propensity-score distributions, overlap, the largest weights and covariate balance.

overlap (common support)

The presence of patients with similar baseline characteristics, and so similar propensity scores, in both treatment groups.

positivity

The requirement that patients with any set of measured characteristics have some possibility of receiving either treatment, 0 < P(A = 1 | X) < 1.

covariate balance

How similar the distributions of the baseline covariates are between the weighted treatment groups.

References

1. Austin PC, Stuart EA. Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Stat Med. 2015;34(28):3661-3679. doi:10.1002/sim.6607 https://doi.org/10.1002/sim.6607
2. Crump RK, Hotz VJ, Imbens GW, Mitnik OA. Dealing with limited overlap in estimation of average treatment effects. Biometrika. 2009;96(1):187-199. doi:10.1093/biomet/asn055 https://doi.org/10.1093/biomet/asn055
3. Petersen ML, Porter KE, Gruber S, Wang Y, van der Laan MJ. Diagnosing and responding to violations in the positivity assumption. Stat Methods Med Res. 2012;21(1):31-54. doi:10.1177/0962280210386207 https://doi.org/10.1177/0962280210386207
4. Cole SR, Hernán MA. Constructing inverse probability weights for marginal structural models. Am J Epidemiol. 2008;168(6):656-664. doi:10.1093/aje/kwn164 https://doi.org/10.1093/aje/kwn164

Key takeaways

• Extreme IPTW weights are a diagnostic finding after the weights are built: the weighted pseudo-population is being dominated by a few observations, so the weighting solution itself is not stable.
• A propensity score near 0 or 1 makes a weight extreme: 1/0.02 = 50 for a treated patient and 1/(1 - 0.99) = 100 for a control patient.
• Extreme weights point back to poor overlap, and weighting cannot create genuine treatment overlap that is absent from the data.
• When extreme weights appear, check overlap, positivity and the propensity model, revise the model if needed, and recheck balance, overlap and weights before any outcome model.
• IPTW does not end the causal analysis. It starts the diagnostic phase: weight first, diagnose second, estimate the outcome third.

Related in the wiki: [[iptw-guide]] [[propensity-weighting-balance-check-and-model-revision]]