R for EHR
Menu
Course contents

Part IIChapter 13Supervised learning: the complete workflow

Regression: Predict a Number

11 min readhands-on

On this page
  1. 1. Regression versus classification
  2. 2. Start with a complete-case teaching example
  3. 3. Train the simplest useful baseline
  4. 4. Fit a linear model
  5. 5. Evaluate the model
  6. 6. Plot observed and predicted values
  7. 7. Is the dataset suitable?
  8. Try it
  9. Checkpoint

1. Regression versus classification

Regression predicts a number, such as a systolic blood pressure (SBP) or time to appointment. Classification predicts a category, such as a synthetic Yes/No label. Start with regression because you can visualize the idea of “fitting a line.”

We will use age and BMI to predict fabricated SBP. Because our dataset was simulated with related formulas, the model may detect an association; nothing learned here should be interpreted as a physiological relationship.

Example: the type of the outcome column tells you which kind of problem you have.

d <- readr::read_csv("data/synthetic_ehr.csv", show_col_types = FALSE)
c(sbp = class(d$sbp), diabetes = class(d$diabetes))

Output

        sbp    diabetes 
  "numeric" "character" 
head(d[c("age", "bmi", "sbp")], 3)

Output

# A tibble: 3 x 3
    age   bmi   sbp
  <dbl> <dbl> <dbl>
1    55  20.4   120
2    26  22.7   110
3    63  24.7   133

Reading the output: sbp is numeric, so predicting it is regression. diabetes is character (a category), so predicting it is classification. The three rows show the inputs (age, bmi) and the number we want to predict (sbp) side by side.

2. Start with a complete-case teaching example

To demonstrate the mechanics of a linear model, we will temporarily restrict this exercise to rows with observed SBP and BMI. Complete-case analysis is not universally appropriate. It can create selection bias. Chapter 12’s recipe is a safer starting place for missing predictors in our classification workflow.

library(tidymodels)
ehr <- readr::read_csv("data/synthetic_ehr.csv", show_col_types = FALSE)
numeric_data <- ehr |>
  filter(!is.na(sbp), !is.na(bmi)) |>
  select(age, bmi, sbp)
set.seed(500)
split <- initial_split(numeric_data, prop = 0.8)
train <- training(split)
test <- testing(split)
c(complete_rows = nrow(numeric_data), train_rows = nrow(train), test_rows = nrow(test))

Output

complete_rows    train_rows     test_rows 
          546           436           110 
nrow(ehr) - nrow(numeric_data)

Output

[1] 54

Reading the output: 54 rows were dropped because sbp or bmi was missing, leaving 546 complete rows. They are split 80/20 into 436 training rows and 110 test rows.

3. Train the simplest useful baseline

What if we ignored all predictors and always guessed the average training SBP? The result is a legitimate baseline.

baseline <- mean(train$sbp)
baseline_predictions <- rep(baseline, nrow(test))
baseline_rmse <- sqrt(mean((test$sbp - baseline_predictions)^2))
baseline_rmse

Output

[1] 11.36997

RMSE (root mean squared error) measures the typical scale of prediction errors, giving extra weight to large mistakes. Its units are the units of SBP: mmHg.

4. Fit a linear model

model <- lm(sbp ~ age + bmi, data = train)
summary(model)

Output

Call:
lm(formula = sbp ~ age + bmi, data = train)

Residuals:
     Min       1Q   Median       3Q      Max 
-10.6734  -6.9637  -0.2211   7.1324  10.6276 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 74.07774    1.76482   41.98   <2e-16 ***
age          0.53164    0.01777   29.91   <2e-16 ***
bmi          0.69944    0.04834   14.47   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 7.108 on 433 degrees of freedom
Multiple R-squared:  0.7164,    Adjusted R-squared:  0.7151 
F-statistic:   547 on 2 and 433 DF,  p-value: < 2.2e-16
pred <- predict(model, newdata = test)
head(data.frame(actual = test$sbp, predicted = pred))

Output

  actual predicted
1    120  117.5865
2    120  110.5487
3    145  136.8506
4    111  107.5543
5    123  133.1997
6    119  108.5194

Read the formula as: predict SBP using age and BMI. lm() estimates coefficients from training rows. It does not inherently establish causal effects. Beware of overinterpreting p-values or coefficients from data where the process and selection may be unknown.

5. Evaluate the model

rmse <- sqrt(mean((test$sbp - pred)^2))
mae <- mean(abs(test$sbp - pred))
c(baseline_rmse = baseline_rmse, model_rmse = rmse, model_mae = mae)

Output

baseline_rmse    model_rmse     model_mae 
    11.369968      6.932251      6.185452

MAE (mean absolute error) measures average absolute deviation in mmHg. Smaller errors are better, but whether an error is acceptable depends on the task and setting. A model that achieves lower test RMSE than baseline is an improvement on this synthetic split, not proof of clinical usefulness.

6. Plot observed and predicted values

ggplot(data.frame(observed = test$sbp, predicted = pred),
       aes(x = observed, y = predicted)) +
  geom_point() +
  geom_abline(slope = 1, intercept = 0, colour = "red") +
  labs(x = "Observed synthetic SBP (mmHg)",
       y = "Predicted SBP (mmHg)")
Scatter plot of observed synthetic systolic blood pressure against the model predicted value for held-out test rows, with a red diagonal line for perfect prediction. The dots rise along the line but scatter well above and below it, so the predictions are only roughly right.
Scatter plot of observed synthetic systolic blood pressure against the model predicted value for held-out test rows, with a red diagonal line for perfect prediction. The dots rise along the line but scatter well above and below it, so the predictions are only roughly right.

The red line represents a perfect prediction. Dots far from it are larger errors. Check whether error sizes differ by groups; average performance may hide important weaknesses.

7. Is the dataset suitable?

The file has one row (one SBP value) per fabricated patient, so train and test share no patients. With repeated real visits, you would group the split by patient. A model predicting SBP from measurements taken after it would need a time-based data check.

Example: one check on the file, and one on what happens at prediction time.

n_distinct(ehr$patient_id) == nrow(ehr)     # one row per patient?

Output

[1] TRUE
new_row <- data.frame(age = 60, bmi = NA_real_)
predict(model, newdata = new_row)

Output

 1 
NA

Reading the output: TRUE means every row is a different patient, so the random split cannot put the same person in both parts. The second result is NA: predict() returns NA for a patient whose BMI is missing, and that is the gap the recipe in chapter 12 fills.

Try it

  1. Fit a model predicting SBP using age alone. Compare test RMSE with the age + BMI version.
Show solution

Answer: The age-only model has a test RMSE of 8.39 mmHg, worse than 6.93 mmHg for age + BMI. Both beat the constant baseline of 11.37 mmHg.

Step by step

  1. Fit lm() with age as the only predictor, on the same train rows.
  2. Predict for test and compute RMSE the same way as in section 5.
age_only <- lm(sbp ~ age, data = train)
pred_age <- predict(age_only, newdata = test)
rmse_age <- sqrt(mean((test$sbp - pred_age)^2))
c(age_only = rmse_age, age_bmi = rmse, baseline = baseline_rmse)

Output

 age_only   age_bmi  baseline 
 8.393542  6.932251 11.369968

3. Read the three numbers in mmHg: smaller is better. Age alone cuts the error from 11.37 to 8.39, and adding BMI cuts it further to 6.93, so BMI helps on this split. 4. The comparison is on one test split, so a small difference between the two models may not repeat on another split.

  1. Calculate MAE for the constant baseline.
Show solution

Answer: The baseline MAE is 9.07 mmHg, larger than the age + BMI model’s 6.19 mmHg.

Step by step

  1. The baseline predicts the training mean for every test row, which is baseline.
  2. MAE is the mean of the absolute errors, abs(actual - predicted).
baseline_mae <- mean(abs(test$sbp - baseline))
c(baseline_mae = baseline_mae, model_mae = mae)

Output

baseline_mae    model_mae 
    9.065263     6.185452

3. The first number (9.07) is the average miss of the “always guess the average” rule, and the second (6.19) is the average miss of the age + BMI model. The model is about 3 mmHg closer on average.

  1. Explain the units of the RMSE.
Show solution

Answer: The RMSE is in mmHg, the same unit as SBP, and it is a typical size of the prediction error.

Step by step

  1. Each error is actual - predicted, in mmHg.
  2. Squaring makes the unit mmHg squared, and the square root at the end brings it back to mmHg. That is why RMSE can be read directly.
errors <- test$sbp - pred
head(errors)

Output

         1          2          3          4          5          6 
  2.413544   9.451267   8.149416   3.445709 -10.199701  10.480611 
c(rmse_mmHg = sqrt(mean(errors^2)))

Output

rmse_mmHg 
 6.932251

3. The first lines show individual errors in mmHg (for example the first test row is 2.4 mmHg above its prediction). The last number says the model is typically off by about 6.9 mmHg. Because of the squaring, large errors count more than small ones.

  1. Explain why a low RMSE does not prove the predictors cause SBP.
Show solution

Answer: RMSE only says how close the predictions are. A predictor can help predict an outcome because of a shared cause, a selection effect, or the simulator’s formula, without causing it.

Step by step

  1. Prediction works in either direction. If age predicts SBP, SBP also predicts age:
cor_age_sbp <- cor(train$age, train$sbp)
rmse_age_from_sbp <- sqrt(mean((test$age - predict(lm(age ~ sbp, data = train), newdata = test))^2))
c(correlation = cor_age_sbp, rmse_age_from_sbp_in_years = rmse_age_from_sbp)

Output

               correlation rmse_age_from_sbp_in_years 
                 0.7611245                 13.3429037

2. The correlation is 0.76 whichever variable we call the outcome, and the reverse model predicts age with a typical error of about 13.3 years. Blood pressure obviously does not cause a person’s age, yet it predicts it, so predictive success cannot show direction of cause. 3. Causal claims need a design that separates causes (for example a trial or a careful causal analysis), and here the data are also simulated.

  1. What happens if a test row has missing BMI? Why do we need a recipe for a general workflow?
Show solution

Answer: predict() from lm() returns NA for that row, so no prediction is made. A recipe applies a stored rule (such as the training median BMI) to fill the gap, so every new row gets a prediction.

Step by step

  1. Make a copy of three test rows and blank the BMI of the first.
test_missing <- test[1:3, ]
test_missing$bmi[1] <- NA
predict(model, newdata = test_missing)

Output

       1        2        3 
      NA 110.5487 136.8506

2. The first prediction is NA, while the other two work. predict() returns NA for that row instead of guessing a value. 3. In chapter 12 a recipe’s step_impute_median() learns the median BMI from train and fills the gap when new rows arrive, in the same way every time. Putting the recipe and the model in one workflow means this happens automatically for each new row.

Checkpoint

You can define regression, fit lm(), predict on held-out observations, compare to a simple baseline, and distinguish predictive associations from causal claims.