Beyond Conventional Supervised and Unsupervised Learning

Parts II to IV covered learning with a complete outcome column (supervised) and with none (unsupervised). This chapter opens Part V: kinds of learning that are neither, such as incomplete labels, images or notes, or decisions that depend on earlier actions. These methods are important to recognize, but they generally require more data, stronger validation, or specialized infrastructure. Beginners should understand their purpose and assumptions before using them.

1. Semi-supervised learning

Semi-supervised methods use a small number of labeled examples and many unlabeled examples. Some methods try to propagate labels among neighboring observations; others train on high-confidence pseudolabels, regularize model consistency, or learn a representation from unlabeled data.

Examples include self-training, label propagation/spreading, co-training, and consistency regularization.

Danger: An unknown label is not a confirmed No. Pseudolabels can reinforce mistakes. If labels are missing primarily because some clinic sites test less often, an algorithm may learn access to care rather than disease.

Example: Ten fictional records from two sites. NA means no test was done, so the label is unknown.

labels_df <- data.frame(
  site = rep(c("North", "South"), each = 5),
  label = c("Yes", "No", "No", "Yes", "No",   # North: tested often
            "No", NA, NA, NA, NA)             # South: rarely tested
)
table(labels_df$site, labels_df$label, useNA = "ifany")

        No Yes <NA>
  North  3   2    0
  South  1   0    4

tapply(is.na(labels_df$label), labels_df$site, mean)

North South 
  0.0   0.8 

North has all five labels and South has one. The NA rows are not “No”: four of the five South patients simply were not tested. Before spreading labels, ask why they are missing; here the answer is the site, so a model could learn “South” instead of disease.

2. Positive–unlabeled (PU) learning

This is a special form of partial-label learning: confirmed positives (P) and unknown/unlabeled (U) examples, with no trustworthy confirmed negatives. This section is a short overview; Chapter 28 works through a full simulated example of the distinction, with hidden true labels you can check against.

Approaches include bagging P/U classifiers, risk estimators using assumed class priors, reliable-negative discovery, and selection-model adjustments. Each has assumptions about how cases become labeled. P versus U metrics cannot automatically be read as sensitivity/specificity against true disease. A verified-label evaluation dataset is critical.

Example: Ten fictional patients. Five truly have the condition, but only two were ever confirmed.

truth   <- c(1, 1, 1, 1, 1, 0, 0, 0, 0, 0)   # real state, normally unknown
labeled <- c(1, 1, 0, 0, 0, 0, 0, 0, 0, 0)   # 1 = confirmed positive, 0 = unlabeled
table(labeled, truth)

       truth
labeled 0 1
      0 5 3
      1 0 2

The unlabeled group (labeled = 0) holds 8 people, and 3 of them truly have the condition. Treating U as “No” would call those 3 negatives, so a P-versus-U model measured against labeled would be rewarded for missing them.

3. Self-supervised and representation learning

A self-supervised model creates a teaching signal from input data itself: predict hidden words, reconstruct masked measurements, distinguish augmented views, or forecast withheld segments. It can then provide embeddings to a downstream supervised model.

Examples:

• Masked language modeling or next-token prediction for textual data.
• Masked-signal reconstruction for time-series recordings.
• Contrastive learning to make related views nearby in representation space.
• Autoencoders for nonlinear compression.

Learning embeddings on real clinical notes or images involves sensitive data. Only use approved sources and systems; self-supervised does not mean data governance is optional.

Example: The teaching signal is a hidden piece of the data itself. Hide one reading in a typed series and guess it from its neighbors, with no outside label.

sbp_series <- c(118, 122, 125, 121, 130)
hidden <- 3                                       # mask the third reading
guess <- mean(sbp_series[c(hidden - 1, hidden + 1)])
c(guess = guess, actual = sbp_series[hidden], error = guess - sbp_series[hidden])

 guess actual  error 
 121.5  125.0   -3.5 

The guess from the two neighbors is 121.5 and the hidden value was 125, an error of -3.5. A self-supervised model does this at scale: it learns to fill in masked pieces, and what it learns along the way (the embedding) is reused for a later task.

4. Weak supervision and active learning

Weak supervision creates imperfect labels from heuristic rules, existing codes, or multiple noisy labelers. It needs explicit assumptions about labeling functions, dependence, and label quality.

Active learning asks which new observations to send to a human labeler (e.g. a chart reviewer) to make annotation more informative. Selecting uncertain cases may bias the labeled set, so evaluation must be independent and appropriately sampled. Chart review requires authorization.

Example: A rule “SBP of 140 or more” acts as a weak label for the synthetic diabetes column (a deliberately poor rule). Then active learning picks the cases whose predicted probability is closest to 0.5 for review.

library(dplyr)
ehr <- readr::read_csv("data/synthetic_ehr.csv", show_col_types = FALSE)
weak_label <- ifelse(ehr$sbp >= 140, "Yes", "No")
table(weak_label, true_label = ehr$diabetes)

          true_label
weak_label  No Yes
       No  491  31
       Yes  35  23

probs <- c(p1 = .95, p2 = .52, p3 = .08, p4 = .47, p5 = .70)
names(sort(abs(probs - 0.5)))[1:2]

[1] "p2" "p4"

The weak rule gets 514 of the 580 rows with an SBP value right, but it marks 35 non-diabetic patients “Yes” and misses 31 of the 54 true cases, which is the label noise a weak-supervision method must model. In the active-learning step, p2 (0.52) and p4 (0.47) are the most uncertain, so those two would go to the chart reviewer first.

5. Transfer learning and domain adaptation

Transfer learning reuses a model trained on another task or dataset. Domain adaptation addresses a change in population or environment (different sites, dates, devices, or recording practices). A model trained elsewhere can transfer useful patterns, or transfer biases and hidden shortcuts.

Do not conclude that fine-tuning on a handful of local rows makes a model clinically validated. Report both source and target population limitations; evaluate at the destination.

Example: Before moving a model between sites, compare the populations. Each clinic site in the synthetic file is a “domain”.

ehr |>
  group_by(clinic_site) |>
  summarise(n = n(), mean_age = mean(age), mean_bmi = mean(bmi, na.rm = TRUE),
            diabetes_rate = mean(diabetes == "Yes"))

# A tibble: 3 x 5
  clinic_site     n mean_age mean_bmi diabetes_rate
  <chr>       <int>    <dbl>    <dbl>         <dbl>
1 Mobile        200     51.5     30.2          0.09
2 North         200     49.5     30.3          0.08
3 South         200     50.4     30.1          0.1 

The three sites have 200 patients each, mean age 49.5 to 51.5, mean BMI about 30, and a diabetes rate of 8% to 10%. Here the sites look alike, so a model moved between them faces little shift in these columns. With real sites, a large gap in a row like this is the first sign that the model needs to be checked at the destination.

6. Neural-network families

In R, common ecosystems include torch, keras3, tensorflow, reticulate, and tidymodels adapters. These are optional advanced dependencies, and installation may require separate Python/CUDA infrastructure. They are not installed by the beginner tutorial.

Example: The smallest member of the first row, an MLP, is available in the core nnet package. This one has 3 hidden units and learns from age and BMI (training data only, so no test claim).

complete <- ehr[complete.cases(ehr[, c("age", "bmi", "diabetes")]), ]
set.seed(29)
mlp <- nnet::nnet(factor(diabetes) ~ scale(age) + scale(bmi), data = complete,
                  size = 3, maxit = 200, trace = FALSE)
length(mlp$wts)

[1] 13

summary(as.vector(predict(mlp, type = "raw")))

     Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
7.958e-05 1.104e-02 3.330e-02 8.675e-02 1.147e-01 4.414e-01 

table(predicted = predict(mlp, type = "class"), actual = complete$diabetes)

         actual
predicted  No Yes
       No 516  49

The network has 13 weights (2 inputs times 3 hidden units, plus biases, plus the output layer). Its predicted probabilities of “Yes” run from about 0.0001 to 0.44, with a median of 0.03, so none crosses the usual 0.5 cutoff. Every prediction is therefore “No”, and the 516 of 565 correct (about 91%) is exactly the majority rate. This is the effect of class imbalance (only 49 patients have diabetes) together with a 0.5 cutoff, not a lack of signal: a logistic regression on the same age and BMI reaches a training ROC AUC of about 0.83, so the same columns carry information, and a lower cutoff or a ranking metric such as ROC AUC would show it.

7. Reinforcement learning (RL)

RL learns actions to maximize cumulative reward over time. Terms: state (what the agent sees), action (what it does), reward (feedback), and policy (choice rule). Examples include Q-learning, policy gradients, actor–critic, and offline RL.

Healthcare actions affect real people and future available data. Optimizing a simulated reward is not proof of safe treatment policy. Offline RL especially requires coverage of actions, causal assumptions, safe constraints, expert review, and ideally prospective testing. This book presents RL conceptually only.

Example: A simulated scheduling agent chooses how many follow-up slots to offer 10 fictional patients. The reward is minus the number of visits used (fewer visits is “better”).

policies <- data.frame(
  policy = c("book follow-up for all", "book for high-risk only", "book nobody"),
  visits_used = c(10, 4, 0),
  needed_followups_missed = c(0, 1, 5)
)
policies$reward <- -policies$visits_used
policies

                   policy visits_used needed_followups_missed reward
1  book follow-up for all          10                       0    -10
2 book for high-risk only           4                       1     -4
3             book nobody           0                       5      0

The policy with the best reward (0) is “book nobody”, which also misses 5 follow-ups that were needed. The agent did exactly what it was told to maximize; the reward was the problem.

8. Causal inference is different from prediction

A model can predict a diagnosis because a treatment occurred earlier in the record, but that does not mean treatment caused the condition. Predictive ML, causal inference, and randomized evidence answer different questions.

Tools include randomized trials, matching, inverse probability weighting, instrumental variables, difference-in-differences, regression discontinuity, and causal forests. These need causal assumptions and appropriate study designs; adding an ML algorithm does not remove confounding.

Example: In a simulation, sicker patients are more likely to be given a treatment, and the treatment truly helps (lowers a severity score by 2 points).

set.seed(8)
sick <- rnorm(500)                                    # baseline severity
treated <- rbinom(500, 1, plogis(2 * sick))           # sicker people get treated
outcome <- 5 + 3 * sick - 2 * treated + rnorm(500)    # true effect of treatment: -2
round(tapply(outcome, treated, mean), 2)

   0    1 
3.00 4.67 

round(coef(lm(outcome ~ treated))["treated"], 2)

treated 
   1.67 

round(coef(lm(outcome ~ treated + sick))["treated"], 2)

treated 
  -1.98 

The treated group has a higher average outcome (4.67) than the untreated group (3.00), and the simple regression says treatment adds 1.67, so treatment looks harmful. The reason is that treated people were sicker to begin with. Adding the baseline severity sick to the model gives -1.98, close to the true effect of -2. Prediction would use the raw pattern; a causal question needs the adjustment (and, in real data, the confounders must be measured).

9. Common EHR failure modes across all paradigms

1. Label error: missing codes used as negatives, noisy chart-derived labels.
2. Target leakage: post-diagnosis orders or medications offered as predictors.
3. Patient/site/time leakage: the same person or future observations cross splits.
4. Ascertainment and selection bias: testing/access differences determine who is documented.
5. Distribution shift: new clinic, new device, changed prevalence or protocol.
6. Calibration and fairness gaps: groups receive different probability reliability or error rates.
7. Privacy and operational risk: model memorizes records or exports forbidden details.

Example: Failure mode 1 in one line. A “diabetes code” column where blank means no code was entered becomes a label.

code <- c("E11", NA, NA, "E11", NA, "E11")
label_as_negative <- as.integer(!is.na(code))
table(label_as_negative)

label_as_negative
0 1 
3 3 

Three patients have the code, and three are labeled 0. Those three zeros mean “no code on file”, not “no diabetes”. Any of them might be undiagnosed or diagnosed elsewhere, which is the label error.

tapply(is.na(labels_df$label), labels_df$site, sum)

North South 
    0     4 

sum(truth[labeled == 0] == 1)

[1] 3

mean(truth[labeled == 0] == 1)

[1] 0.375

length(coef(glm(factor(diabetes) ~ age + bmi + sbp + pulse, data = ehr,
                family = binomial)))

[1] 5

12 * 12 * 768^2

[1] 84934656

policies[which.max(policies$reward), ]

       policy visits_used needed_followups_missed reward
3 book nobody           0                       5      0

policies$better_reward <- -policies$visits_used - 3 * policies$needed_followups_missed
policies[which.max(policies$better_reward), "policy"]

[1] "book for high-risk only"

raw <- unname(coef(lm(outcome ~ treated))["treated"])
adjusted <- unname(coef(lm(outcome ~ treated + sick))["treated"])
round(c(raw = raw, adjusted = adjusted), 2)

     raw adjusted 
    1.67    -1.98