Design, Model and Estimand: What Each Study Design Can Actually Estimate

อ่านฉบับภาษาไทย (Thai version)

Abstract

A provincial hospital wants to cost a programme that helps older adults stop taking sleeping pills. Its economic model needs one input: how much the pill raises the risk of a fall that needs hospital care within a year. Three analyses of the same records answer 2.250, 2.111 and 2.000. All three are right, for different questions. This article follows one illustrative register of 10,000 adults aged 65 and over, 2,000 of whom take a sleeping pill. It first writes the question down in full. It then shows what complete follow-up gives, why counting time at risk turns the answer into an incidence rate ratio of 2.111, and why the same 1,200 people who fell give three ratios against three comparison groups. Regression models choose the scale too. A table shows what a trial, a cohort, each case-control plan and a one-day survey can estimate. The right input avoids 200 falls a year, not the 250 a misread ratio suggests.

🔍 Click to enlarge Click anywhere to close

One hospital, one question, three answers

A provincial hospital is costing a programme that helps older adults stop taking sleeping pills. Its records cover 10,000 adults aged 65 and over: 2,000 take a sleeping pill every night and 8,000 do not. In one year, 400 of the pill users and 800 of the non-users had a fall that needed hospital care. These are illustrative numbers for teaching, not Thai data.

The programme's economic model needs one input: how much does the pill change the risk of such a fall, the chance that a person has one within the year? The team finds three analyses of these same records. One reports 2.250, one 2.111 and one 2.000, and each calls its number the effect of the pill.

All three numbers are arithmetically right. They differ because each analysis answered a different question, set by how the study was designed and which model it used. This article shows which question each number answers, what each kind of study can estimate, and which number the economic model should take.

Write the question down first

Before any number, write the programme's question in full. It has five parts:

• Who: adults aged 65 and over in the province who now take a sleeping pill.
• Compared with what: the same people if they stopped the pill.
• Which outcome: a fall that needs hospital care.
• Over what time: one year.
• On which scale: the difference or the ratio of the risks, so the model can count falls.

A question written this way is called the estimand: the exact quantity a study sets out to estimate. The design and the statistical model come after it, because they decide which quantity the data can actually deliver.

Read first on this site: how the estimand differs from the estimator, the method, and the estimate, the number the method returns, is taught in The Three E’s of Quantitative Research: Estimand, Estimator, Estimate.

The question also says how to handle events along the way, called intercurrent events. Some of the 2,000 users stop the pill by themselves during the year. Counting their falls anyway asks what happens to people who take the pill today. Dropping them answers no question: people often stop because they feel unwell, and feeling unwell is linked to falling.

What the full records can tell

The hospital followed everyone for the whole year, so it can count directly. Among the 2,000 users, 400 fell: a risk of 20.00% in one year. Among the 8,000 non-users, 800 fell: a risk of 10.00%. A risk is the share of a group who have the event within a stated time.

Two comparisons follow. Subtract the risks and the pill adds 10.00 percentage points, 10 extra falls for every 100 users in a year; this is the risk difference. Divide them and the users' risk is 2.000 times the non-users' risk; this is the risk ratio.

The records allow a third comparison. Instead of dividing those who fell by everyone, divide them by those who did not fall. For users that is 400 to 1,600, or 0.250; for non-users it is 800 to 7,200, or 0.111. This number, those who fell for each one who did not, is called the odds.

Dividing the users' odds by the non-users' odds gives 2.250. An odds ratio compares the odds of an event between two groups, so it drifts further from 1 than the risk ratio when events are common. With 8 of 66 treated and 13 of 60 control patients having the event, it is 0.50 while the risk ratio is 0.56; the full lesson is in Risk Ratio, Odds Ratio, Risk Difference and NNT: One 2×2 Table, Four Answers.

Here the gap is wide, 2.250 against 2.000, because falls are common: 1 user in 5 fell. If falls were rare, say 2% of users and 1% of non-users in a year, the odds ratio would be 2.020, almost the risk ratio. So the odds ratio stands in for the risk ratio only when the outcome is rare in both groups.

A cohort or a trial that follows everyone can therefore give all of these: the two risks, their difference, their ratio and the odds ratio. The design does not tie a study to one measure; the question decides which one to report.

# One year of hospital records, sleeping-pill users and non-users.
people <- c(users = 2000, non_users = 8000)
falls <- c(users = 400, non_users = 800)
risk <- falls / people
odds <- falls / (people - falls)
cat("Risk of a fall, users and non-users (%):", sprintf("%.2f", 100 * risk), "\n")
cat("Risk difference (percentage points):", sprintf("%.2f", 100 * (risk[1] - risk[2])), "\n")
cat("Risk ratio:", sprintf("%.3f", risk[1] / risk[2]), "\n")
cat("Odds of a fall, users and non-users:", sprintf("%.3f", odds), "\n")
cat("Odds ratio:", sprintf("%.3f", odds[1] / odds[2]), "\n")
rare <- c(users = 0.02, non_users = 0.01)
rare_odds <- rare / (1 - rare)
cat("Odds ratio if the risks were 2% and 1%:", sprintf("%.3f", rare_odds[1] / rare_odds[2]), "\n")

Risk of a fall, users and non-users (%): 20.00 10.00
Risk difference (percentage points): 10.00
Risk ratio: 2.000
Odds of a fall, users and non-users: 0.250 0.111
Odds ratio: 2.250
Odds ratio if the risks were 2% and 1%: 2.020

Counting time as well as people

A user who fell in June stopped being at risk of a first fall in June. Counting her as a full year at risk overstates how long she could have had a first fall. Adding up, person by person, the time each one was still at risk gives the person-time (taught in Incidence, Prevalence and the Right Denominator: The Epidemiology Beneath Cost-of-Illness and Markov Models).

To keep the arithmetic simple, assume each fall happened at mid-year. The 400 users who fell give half a year each and the other 1,600 give a full year: 1,800 person-years. The non-users give 7,600 person-years in the same way.

Falls divided by person-time is the incidence rate: 22.2 falls per 100 person-years among users and 10.5 among non-users. One rate divided by the other is the incidence rate ratio (IRR), 2.111. It answers how much faster falls happen while people are still at risk, not what share of them fall within the year.

Checking only some records: where the controls come from

Read first on this site: how controls are chosen, and why the question, the controls and the model are three separate choices, is taught in Control Sampling in Stata: Question, Controls and Model Are Three Different Things and Nested Case-Control Studies: Efficient Cohort Sampling for Observational Research. Here we follow only what each choice does to the number the economic model receives.

Now suppose the hospital had not recorded who takes the pill. Checking 10,000 pharmacy histories by hand is slow and costly. A cheaper plan checks everyone who fell, all 1,200 of them, and only a sample of the others. Choosing people by whether they had the outcome is a case-control design: those who fell are the cases, and the people they are compared with are the controls [1].

Among the 1,200 cases, 400 took the pill and 800 did not, so their odds of being a user are 400 divided by 800 equals 0.500. The cases are the same in every analysis below. The only choice left is the group the controls are drawn from, and that choice decides the answer [2].

Pool 1 is the people who never fell. Controls come from those still free of a fall at the end of the year: 1,600 users and 7,200 non-users.

Pool 2 is the time at risk. Each time someone falls, a control is drawn from the people who have not yet fallen that day. People are then picked in proportion to their time at risk, so the pool mirrors the person-time: 1,800 to 7,600. This pool needs no assumption that falls are rare [2].

Pool 3 is everyone at the start. Controls come from the full list on the first day of the year, including people who later fell: 2,000 to 8,000.

The worked steps below divide by each pool in turn. Why does this work? Dividing the cases' odds by the pool's odds is the same as dividing the users who fell by the users in the pool, doing the same for non-users, and comparing the two. With people at the start in the pool, that is falls per person, a risk; with person-time, falls per person-year, an incidence rate; with people who never fell, falls per person who did not fall, the odds.

# The same records counted in time at risk, then three control pools.
people <- c(users = 2000, non_users = 8000)
falls <- c(users = 400, non_users = 800)
years <- people - falls / 2  # a fall at mid-year ends that person's time at risk
rate <- 100 * falls / years
cat("Person-years at risk:", years, "| falls per 100 person-years:", sprintf("%.1f", rate), "\n")
cat("Incidence rate ratio (IRR):", sprintf("%.3f", rate[1] / rate[2]), "\n")
case_odds <- falls[["users"]] / falls[["non_users"]]
cat("People who fell:", sum(falls), "| odds of being a user:", sprintf("%.3f", case_odds), "\n")
pools <- rbind(never_fell = people - falls, time_at_risk = years, everyone_at_start = people)
pool_odds <- pools[, "users"] / pools[, "non_users"]
pool_ratio <- case_odds / pool_odds
cat(sprintf("%s: pool odds %.3f, ratio %.3f\n", rownames(pools), pool_odds, pool_ratio), sep = "")

Person-years at risk: 1800 7600 | falls per 100 person-years: 22.2 10.5
Incidence rate ratio (IRR): 2.111
People who fell: 1200 | odds of being a user: 0.500
never_fell: pool odds 0.222, ratio 2.250
time_at_risk: pool odds 0.237, ratio 2.111
everyone_at_start: pool odds 0.250, ratio 2.000

The sampling plans have names

Each pool is a named sampling plan. Drawing controls from people still free of a fall at the end of the year is cumulative sampling, the classic case-control design. Only this plan needs falls to be rare before its odds ratio comes close to the risk ratio [2].

Drawing controls from those still at risk on the day of each fall is risk-set sampling; inside a register it is a nested case-control study. Drawing one random sample from the start list is a case-cohort design, and that one sample can serve several outcomes.

Both save effort when the exposure is costly to measure. In a real study a few controls are drawn per case, which adds chance error to the exact values above. The analysis must then follow how the controls were drawn, so the study has to keep who was sampled, from which pool and when.

A one-day survey answers a different question

Read first on this site: how a one-day survey can itself be sampled in several ways is taught in Cross-Sectional Analogue Designs: Population-Analogue, Cohort-Analogue & Case-Control-Analogue Sampling for Rare Exposure vs Rare Outcome.

A survey is quicker still. On one day, nurses visit the same 10,000 older adults and record whether each takes the pill and whether each is still recovering from a fall injury. Such a snapshot is a cross-sectional study: it counts prevalence, the share who have the condition at that moment, and it cannot show whether the pill or the injury came first (see Incidence, Prevalence and the Right Denominator: The Epidemiology Beneath Cost-of-Illness and Markov Models).

Suppose every injury lasts 3 months in both groups and the situation is stable. On survey day about 105 of the 2,000 users and 205 of the 8,000 non-users are still recovering: a prevalence of 5.26% and 2.56%. In a stable situation the odds of being found injured on survey day equal the incidence rate times the injury duration, 0.222 × 0.25 for users and 0.105 × 0.25 for non-users, which is how the 105 and 205 arise. Dividing the two shares gives the prevalence ratio, 2.053; dividing the two odds of being injured gives the prevalence odds ratio, 2.111 [3].

Prevalence mixes how often falls happen with how long injuries last. Only when the situation is stable, with as many new injuries starting each month as old ones healing, and injuries lasting equally long in both groups, does the prevalence odds ratio come close to the incidence rate ratio of 2.111. The prevalence ratio, 2.053, does not. So a survey helps plan today's services, but it cannot supply the yearly risk of a new fall that the programme needs.

# A one-day survey of the same register: every injury lasts 3 months, the situation is stable.
people <- c(users = 2000, non_users = 8000)
rate <- c(users = 400 / 1800, non_users = 800 / 7600)  # new falls per person-year at risk
duration <- 3 / 12  # years an injury lasts, the same in both groups
injured_odds <- rate * duration  # stable situation: odds of being injured = rate x duration
prevalence <- injured_odds / (1 + injured_odds)
cat("Incidence rate per person-year, users and non-users:", sprintf("%.3f", rate), "\n")
cat("Injury duration (years):", sprintf("%.2f", duration), "\n")
cat("Still recovering on survey day, users and non-users:", round(people * prevalence), "\n")
cat("Prevalence, users and non-users (%):", sprintf("%.2f", 100 * prevalence), "\n")
cat("Prevalence ratio (PR):", sprintf("%.3f", prevalence[1] / prevalence[2]), "\n")
cat("Prevalence odds ratio (POR):", sprintf("%.3f", injured_odds[1] / injured_odds[2]), "\n")

Incidence rate per person-year, users and non-users: 0.222 0.105
Injury duration (years): 0.25
Still recovering on survey day, users and non-users: 105 205
Prevalence, users and non-users (%): 5.26 2.56
Prevalence ratio (PR): 2.053
Prevalence odds ratio (POR): 2.111

The model sets the scale; its predictions give back risks

Read first on this site: how the form of a model picks its effect measure is taught in Link and Family: One Picks Your Effect Measure, the Other Your Standard Error. Why an odds ratio from a model moves when another patient characteristic is added, even with no confounding, is taught in Same Cohort, Two Odds Ratios: Non-Collapsibility Without Confounding.

The same choice returns when the hospital fits a regression model, a formula that links the chance of a fall to the pill and to other patient characteristics. Fitted to all 10,000 records, logistic regression models the odds and gives 2.250, the odds ratio. Poisson regression, a model for counts, can be fitted to each person's one fall or none; it gives 2.000, the risk ratio, once its standard errors are corrected for a yes/no outcome (robust variance) [4]. Fitted instead to falls per unit of time at risk, with each person's time at risk entered as an offset (a fixed adjustment the model does not estimate), it gives 2.111, the incidence rate ratio.

A Cox model works from the date of each fall and compares the speed of falling moment by moment among people who have not yet fallen. A hazard ratio compares how fast events happen among people still event-free, moment by moment; it is not a ratio of risks over a period. With control survival of 0.60 and a hazard ratio of 0.70, the five-year risks are 40.0% and 30.1%, a risk ratio of 0.75; the full lesson is in The Hazard Ratio Is Not a Risk Ratio: Proportional Hazards, Restricted Mean Survival Time and Model Inputs.

The register has no spread of fall dates, so we fit no Cox model.

So the model picks the scale of its coefficient. Its predictions, though, are risks, and they can be turned back into the numbers the economic model needs. Predict the risk of a fall for each of the 10,000 people as if all took the pill, then as if none did, and average each set: 20.00% and 10.00%. Their ratio is 2.000 and their difference 10.00 percentage points, from the same logistic model whose coefficient said 2.250.

This averaging is called marginal standardization [5]. With age or frailty in the model, each person gets their own pair of predictions, and the answer applies to the people you averaged over. For the programme those are the 2,000 users, not all 10,000. Here the model holds only the pill, so everyone has the same prediction and both choices give the same two risks.

# Three regression models fitted to the same 10,000 people, one row each.
pill <- rep(c(1, 0), c(2000, 8000))
fell <- c(rep(1:0, c(400, 1600)), rep(1:0, c(800, 7200)))
logistic <- glm(fell ~ pill, family = binomial)
per_person <- glm(fell ~ pill, family = poisson)
per_year <- glm(fell ~ pill + offset(log(1 - fell / 2)), family = poisson)  # mid-year falls
ratio <- function(model) sprintf("%.3f", exp(coef(model)[["pill"]]))
cat("Logistic", ratio(logistic), "| Poisson per person", ratio(per_person), "\n")
cat("Poisson with a person-time offset (IRR)", ratio(per_year), "\n")
risk_if <- function(p) mean(predict(logistic, data.frame(pill = rep(p, 10000)), type = "response"))
on_off <- c(on = risk_if(1), off = risk_if(0))
cat("Averaged predictions, all on / all off the pill (%):", sprintf("%.2f", 100 * on_off), "\n")
cat("Ratio of the two averages:", sprintf("%.3f", on_off[["on"]] / on_off[["off"]]), "\n")

Logistic 2.250 | Poisson per person 2.000
Poisson with a person-time offset (IRR) 2.111
Averaged predictions, all on / all off the pill (%): 20.00 10.00
Ratio of the two averages: 2.000

# The two recaps, traced: a small trial's 2x2 table and a five-year survival example.
risk <- c(treated = 8 / 66, control = 13 / 60)
odds <- risk / (1 - risk)
cat("Trial risk ratio:", sprintf("%.2f", risk[["treated"]] / risk[["control"]]), "\n")
cat("Trial odds ratio:", sprintf("%.2f", odds[["treated"]] / odds[["control"]]), "\n")
survival_control <- 0.60
hazard_ratio <- 0.70
cat("Control five-year survival:", sprintf("%.2f", survival_control), "\n")
cat("Hazard ratio:", sprintf("%.2f", hazard_ratio), "\n")
risk5 <- 1 - c(control = survival_control, treated = survival_control^hazard_ratio)
cat("Five-year risks, control and treated (%):", sprintf("%.1f", 100 * risk5), "\n")
cat("Five-year risk ratio:", sprintf("%.2f", risk5[["treated"]] / risk5[["control"]]), "\n")

Trial risk ratio: 0.56
Trial odds ratio: 0.50
Control five-year survival: 0.60
Hazard ratio: 0.70
Five-year risks, control and treated (%): 40.0 30.1
Five-year risk ratio: 0.75

What each design can estimate

The table below gathers the article on one page. Each row is a design and each column a measure: the risk difference (RD), the risk ratio (RR), the odds ratio (OR), the incidence rate ratio (IRR), the hazard ratio (HR), the prevalence ratio (PR) and the absolute risk in each group.

A randomised trial, where chance decides who takes the pill, and a cohort both follow everyone, so they can give every column except the prevalence ratio; the hazard ratio also needs the date of each fall. A one-day survey gives only the prevalence measures, so none of its numbers is the yearly risk the economic model needs.

Each case-control plan gives one ratio directly, and the control pool alone decides which column it lands in; the other cells in its row are conditional. With the sampled share means the study knows how many people each control stands for, which is what turns a ratio back into risks. If rare means the ratio approximates that column only when falls are uncommon. With dates means the date of each fall is kept.

What the economic model needs

The programme's model counts falls, so it needs a risk for each strategy, not a ratio alone [6]. If the 2,000 users kept taking the pill, about 400 would fall in a year. If they stopped, and if the pill causes the gap, their risk would drop to the non-users' 10.00% and 200 would fall. The programme would avoid 200 falls a year.

Now suppose the analyst takes the 2.250 from the first analysis and treats it as a risk ratio. Multiplying the non-users' 10.00% by 2.250 gives 22.50% on the pill instead of 20.00%. The model would then credit the programme with 250 falls avoided, 50 more than the records support, and each of those falls carries a hospital cost.

A case-control sample cannot give these risks on its own. In the register 12.00% of people fell, but in a sample with one control per faller 50.00% did, because the investigators chose that mix. Risks come back only if the study knows how many controls it drew from how many people, or if a local risk is supplied from elsewhere.

A ratio from one place is often applied to a local risk from another. That works only if the population, the outcome and the time horizon match, and if the ratio is on the scale the model uses. Every step above also assumes the pill causes the gap. Frail people may be the ones given sleeping pills, which is the subject of Chance, Bias and Confounding: Why an Association Is Not Yet a Treatment Effect.

# What the economic model receives for the 2,000 users.
users <- 2000
risk <- c(on = 400 / 2000, off = 800 / 8000)
odds_ratio <- (400 / 1600) / (800 / 7200)
cat("Falls a year among the users, on the pill / after stopping:", users * risk, "\n")
cat("Falls avoided, from the two risks:", users * (risk[["on"]] - risk[["off"]]), "\n")
wrong <- odds_ratio * risk[["off"]]  # the odds ratio applied as if it were a risk ratio
cat("Risk on the pill, odds ratio used as a risk ratio (%):", sprintf("%.2f", 100 * wrong), "\n")
cat("Falls avoided on that wrong risk:", users * (wrong - risk[["off"]]), "\n")
fell_share <- c(register = 1200 / 10000, one_control_each = 1200 / (1200 + 1200))
cat("Share who fell, register / sample (%):", sprintf("%.2f", 100 * fell_share), "\n")

Falls a year among the users, on the pill / after stopping: 400 200
Falls avoided, from the two risks: 200
Risk on the pill, odds ratio used as a risk ratio (%): 22.50
Falls avoided on that wrong risk: 250
Share who fell, register / sample (%): 12.00 50.00

The series, in learning order

1. Health Economics Series Guide: Read the Evidence Before the Costs
2. Incidence, Prevalence and the Right Denominator: The Epidemiology Beneath Cost-of-Illness and Markov Models
3. Design, Model and Estimand: What Each Study Design Can Actually Estimate (this article)
4. Risk Ratio, Odds Ratio, Risk Difference and NNT: One 2×2 Table, Four Answers
5. The Hazard Ratio Is Not a Risk Ratio: Proportional Hazards, Restricted Mean Survival Time and Model Inputs
6. Chance, Bias and Confounding: Why an Association Is Not Yet a Treatment Effect
7. Beyond the Evidence Pyramid: GRADE Certainty and Choosing Effectiveness Inputs for Economic Models
8. From Sensitivity to Cost per Case Detected: Predictive Values, Prevalence and the Economics of Screening
9. Whose Cost, in Which Year? Perspective, Opportunity Cost, Inflation and Discounting
10. QALY, DALY and the Utilities Behind Them: What Is Actually Being Measured
11. ICER, Dominance and the Cost-Effectiveness Plane: Reading the Thai Threshold of 160,000 Baht per QALY
12. Decision Trees, Markov Cohorts and Probabilistic Sensitivity Analysis: How a Model Turns Evidence into a Decision
13. Cost-Effective Is Not Affordable, and Efficient Is Not Fair: Budget Impact and Equity
14. Parameter Sourcing, Reference Cost Lists and Utility Catalogues When Local Data Are Sparse
15. Early Health Technology Assessment, Headroom Pricing and Benefit-Package Decisions
16. Communicating Health Economic Evidence: From Model Outputs to Policy Briefs

Related in the wiki: Incidence, Prevalence and the Right Denominator: The Epidemiology Beneath Cost-of-Illness and Markov Models · Risk Ratio, Odds Ratio, Risk Difference and NNT: One 2×2 Table, Four Answers · The Hazard Ratio Is Not a Risk Ratio: Proportional Hazards, Restricted Mean Survival Time and Model Inputs · Chance, Bias and Confounding: Why an Association Is Not Yet a Treatment Effect · Control Sampling in Stata: Question, Controls and Model Are Three Different Things · Nested Case-Control Studies: Efficient Cohort Sampling for Observational Research · The Three E’s of Quantitative Research: Estimand, Estimator, Estimate · Link and Family: One Picks Your Effect Measure, the Other Your Standard Error · Same Cohort, Two Odds Ratios: Non-Collapsibility Without Confounding · Cross-Sectional Analogue Designs: Population-Analogue, Cohort-Analogue & Case-Control-Analogue Sampling for Rare Exposure vs Rare Outcome