Methods Map: Which Kind of Learning?

This chapter opens Part VI, which is about choosing and comparing methods. You have now met the main families: one complete binary-classification workflow in Part II, more supervised methods in Part III (chapters 19 to 22), unsupervised methods in Part IV (chapters 23 to 26), and other kinds of learning in Part V (chapters 27 and 28). The map below puts them side by side, so you can see how the families relate and which question each one answers. It is also a summary: if you skim it before Part III, it previews what is coming, and the chapter links tell you where each family is taught. No finite textbook can cover every method ever invented, so the map covers the major families, common variants, practical choices in R, and where more advanced tools belong. A beginner should master one reproducible workflow before comparing dozens of algorithms.

1. Tasks and algorithms are not the same

A task describes the type of answer. An algorithm describes how a model learns that answer.

Paradigm	Task	What the output means	Example using fictional EHR-like data
Supervised	Regression	A number	Estimated number of days until a fabricated follow-up
Supervised	Binary classification	One of two classes, often with probability	Synthetic Yes/No outcome
Supervised	Multiclass classification	Exactly one of three or more classes	One of three simulated referral categories
Supervised	Ordinal classification	Ordered categories	Fictional low/medium/high workflow priority
Supervised	Multilabel classification	Several labels can be true simultaneously	Multiple synthetic codes per visit
Supervised	Survival analysis	Time to an event with censoring	Time until follow-up, including incomplete observation
Supervised	Forecasting	Future observations	Next week’s aggregate visit counts
Unsupervised	Clustering	Groups by similarity	Similar measurement patterns, not diagnoses
Unsupervised	Dimensionality reduction	Smaller representation of features	PCA summary of vitals
Unsupervised	Anomaly detection	Unusual data patterns	Possible data-entry error to review
Unsupervised	Association mining	Co-occurrence rules	Frequent pairs of nonidentifying services
Semi-supervised / PU	Learning from partial labels	Predictions under explicit labeling assumptions	Some confirmed positives, many unknowns
Self-supervised	Representation learning	Useful features created by a pretext task	Embeddings from deidentified text, with authorized infrastructure
Reinforcement	Sequential decision policy	Actions optimized against rewards	Simulated staffing allocation, not unattended clinical actions

Classification is a supervised task, not a third independent category alongside supervised and unsupervised learning. Logistic regression, trees, forests, nearest neighbors, naive Bayes, support vector machines, and neural networks can all be used for classification. Some of those algorithms also support regression.

Example: the type of the column you want to predict usually points to the task. We check the type of each column of the course table.

ehr <- readr::read_csv("data/synthetic_ehr.csv", show_col_types = FALSE)
sapply(ehr[c("sbp", "diabetes", "visit_date")], function(col) class(col)[1])

Output

        sbp    diabetes  visit_date 
  "numeric" "character"      "Date"

sbp is numeric, so predicting it is regression. diabetes is a Yes/No text column, so predicting it is binary classification. visit_date is a date, which would point to counting visits over time and forecasting. The column type does not settle the task by itself, but it is the first thing to check.

2. Overview of major supervised algorithm families

This table summarizes the supervised families. Regression families are in chapter 19, classification families in chapter 20, how to evaluate them in chapter 21, and time-to-event and time series in chapter 22; neural networks are taught in chapter 27, section 6; the workflow underneath all of them is chapters 9 to 18.

Family	Typical methods	Strength	Important caveat
Linear / generalized linear	Linear, logistic, Poisson, negative binomial	Clear baseline, interpretable form	Relationships and link function assumptions
Regularized linear	Ridge, LASSO, elastic net	Controls complexity in high dimensions	Tuning and feature scaling
Neighbors	k-nearest neighbors (kNN)	Simple intuition, flexible	Distance scaling, memory, high dimensions
Trees and ensembles	CART, random forest, boosting, XGBoost, LightGBM, CatBoost	Nonlinearities and interactions	Leakage, tuning, calibration
Probabilistic / discriminant	Naive Bayes, LDA, QDA	Fast, useful structured assumptions	Strong distribution/independence assumptions
Margin / kernel	SVM / SVR	Flexible decision boundaries	Scaling, kernel, tuning, calibration
Neural networks	MLP, CNN, RNN, transformers	High-capacity representation learning	Data/compute requirements, reproducibility
Other regression	GAM, quantile, robust, mixed-effects	Addresses nonlinear, conditional, clustered data	Must match study design
Time-to-event	Cox, parametric survival, survival forests	Incorporates censoring	Event definitions and censoring assumptions

Example: two different algorithms from the table, solving the same classification task on the same rows. Both are given the Yes/No outcome and both return a probability.

cl <- ehr[!is.na(ehr$bmi) & !is.na(ehr$sbp), ]
cl$y <- as.integer(cl$diabetes == "Yes")
glm_fit  <- glm(y ~ age + bmi + sbp, family = binomial, data = cl)
tree_fit <- rpart::rpart(factor(y) ~ age + bmi + sbp, data = cl, method = "class")
round(cbind(logistic = predict(glm_fit, type = "response")[1:4],
            tree = predict(tree_fit, type = "prob")[1:4, "1"]), 2)

Output

  logistic tree
1     0.02 0.04
2     0.00 0.04
3     0.08 0.04
4     0.02 0.04

The two columns differ for the same four patients: the logistic model gives each patient their own probability (0.00 to 0.08), while the shallow tree gives all four the same 0.04 probably because they fall into the same leaf. The algorithms learn differently, but both answer the same question. Task first, algorithm second.

3. Overview of major unsupervised methods

This table summarizes the unsupervised families. K-means and PCA were introduced in chapter 23, the clustering families are in chapter 24, projection and manifold methods in chapter 25, and association and anomaly detection in chapter 26. Partial-label, self-supervised and reinforcement learning are in Part V, chapter 27 and chapter 28.

Family	Examples	Main question
Partitioning	K-means, K-medoids / PAM	What groups minimize within-group dissimilarity?
Hierarchical	Agglomerative, divisive	How do similarities nest at different resolutions?
Density-based	DBSCAN, HDBSCAN, OPTICS	Which dense groups and noise points occur?
Model-based	Gaussian mixtures, latent-class models	Could data arise from multiple probability components?
Graph / spectral	Spectral clustering, community detection	What structure appears from similarity relationships?
Projection	PCA, SVD, factor analysis, ICA, NMF	Can many variables be represented by fewer factors?
Manifold visualization	t-SNE, UMAP, Isomap	How can structure be displayed in 2D/3D?
Association	Apriori, FP-growth	Which items co-occur more often than expected?
Anomaly / novelty	Robust distances, isolation forest, one-class SVM, LOF	What looks unlike the rest?

Example: an unsupervised method gets the measurements and no outcome. Here K-means groups the rows by bmi and sbp, and only afterwards do we look at the diabetes column.

cc <- ehr[!is.na(ehr$bmi) & !is.na(ehr$sbp), ]
set.seed(21)
grp <- kmeans(scale(cc[, c("bmi", "sbp")]), centers = 2, nstart = 10)$cluster
table(cluster = grp, diabetes = cc$diabetes)

Output

       diabetes
cluster  No Yes
      1 262   9
      2 235  40

Both clusters contain patients with and without diabetes: 9 of 271 in cluster 1 and 40 of 275 in cluster 2. Cluster 2 has the higher share, but most of its members are still No. The groups describe similar measurements, not disease status.

4. Choose a task before choosing a method

Ask these questions in order:

Is the outcome known and valid? If not, supervised performance metrics may be misleading.
What does one row mean? Patient, visit, specimen, or time period?
What answer is needed? Number, category, time-to-event, group, rule, or anomaly?
What predictors are available at prediction time? Remove outcomes, future information, identifiers, and proxy leakage.
How will it be evaluated? Patient grouping, chronology, site shift, missingness, calibration, uncertainty, and a baseline.
Do we have enough independent observations? Complex algorithms cannot manufacture valid labels or sample size.
Can another member reproduce and interpret the result? Prefer an explainable baseline before a high-capacity model.

Example: questions 1 and 2 can be answered with a few lines of code. Is the outcome known, and what does one row mean?

c(rows = nrow(ehr), patients = dplyr::n_distinct(ehr$patient_id),
  sbp_missing = sum(is.na(ehr$sbp)), diabetes_missing = sum(is.na(ehr$diabetes)))

Output

            rows         patients      sbp_missing diabetes_missing 
             600              600               20                0

diabetes is never missing, so a classification outcome exists for every row. sbp is missing for 20 rows, and those rows cannot supervise a regression on SBP. The number of rows equals the number of patients (600), so one row is one patient; if patients had several rows, a split would have to keep each patient’s rows together (question 5).

5. Worked example: do not solve the wrong task

Imagine you have synthetic age, BMI, and pulse for 600 records.

“Predict an invented binary outcome” → supervised binary classification.
“Estimate fabricated SBP” → supervised regression.
“Find groups with similar measurements” → unsupervised clustering.
“Compress all vitals into two axes” → PCA.
“Detect implausible combinations needing verification” → anomaly detection, not labeling disease.
“Predict days until a fictional follow-up when some people have not yet returned” → survival analysis, not ordinary regression on observed returns alone.

To check how much data each request can use, count the rows that have what it needs:

c(classify_diabetes = sum(!is.na(ehr$diabetes)),
  regress_sbp = sum(!is.na(ehr$sbp)),
  cluster_age_bmi_pulse = sum(complete.cases(ehr[, c("age", "bmi", "pulse")])))

Output

    classify_diabetes           regress_sbp cluster_age_bmi_pulse 
                  600                   580                   535

The usable counts differ by task: the classification request can use all 600 rows, the SBP regression only the 580 rows with a measured SBP, and the clustering request only the 535 rows complete in all three measurements.

Try it

Classify the task: next month’s visit count, one of four referral categories, and groups of similar vital measurements.

Show solution

Answer: Next month’s visit count is forecasting (a supervised, time-ordered prediction of a number). One of four referral categories is multiclass classification. Groups of similar vital measurements are unsupervised clustering.

Step by step

For each request, ask what kind of answer is wanted: a number over future time, one category out of several, or groups with no known label.
A count of future visits is a number that depends on time, so it is forecasting and needs a validation split that respects time order.
Exactly one of four categories is multiclass classification (not multilabel, because only one can be true).
“Groups of similar measurements” has no target column to learn from, so it is unsupervised.

Name three algorithms that can classify Yes/No outcomes.

Show solution

Answer: Logistic regression, a decision tree and a random forest. Others that also work are kNN, SVM, naive Bayes, gradient boosting, LDA/QDA and neural networks.

Step by step

Look in the “Typical methods” column of the section 2 table for algorithms that can classify.
Check that three of them accept the same Yes/No outcome by fitting them in a few lines.

cl_small <- cl[1:300, ]
fits <- list(
  logistic = glm(y ~ age + bmi + sbp, family = binomial, data = cl_small),
  tree = rpart::rpart(factor(y) ~ age + bmi + sbp, data = cl_small, method = "class"),
  forest = ranger::ranger(factor(y) ~ age + bmi + sbp, data = cl_small))
names(fits)

Output

[1] "logistic" "tree"     "forest"

3. All three fitted without error on the same outcome. Their underlying assumptions differ, which the later chapters compare.

Explain why clustering cannot confirm that a patient has diabetes.

Show solution

Answer: A cluster is built from how similar the measurements are, with no reference diagnosis used. Similar vitals do not establish a disease.

Step by step

Reuse the clustering from section 3, which never saw the diabetes column.
Cross-tabulate the clusters with the recorded diabetes label.

round(prop.table(table(cluster = grp, diabetes = cc$diabetes), margin = 1), 2)

Output

       diabetes
cluster   No  Yes
      1 0.97 0.03
      2 0.85 0.15

3. Cluster 1 has 3% diabetes and cluster 2 has 15%, so the clusters differ in rate, but both contain patients with and without it (even cluster 2 is 85% No). Whether a person has diabetes is not decided by which cluster they fall in, so a cluster cannot confirm the diagnosis. Only a valid reference test can.

Describe one situation in which an algorithm may be powerful but inappropriate because the label is wrong.

Show solution

Answer: If the “positive” label only means that a code was recorded, and “negative” only means that nothing was recorded (as in Chapter 28), the algorithm learns who was documented, not who is ill.

Step by step

Name the label and ask what it really measures.
If the label records testing or billing habits, any algorithm, however powerful, learns those habits.
Its accuracy, AUC and other scores are then measured against the wrong target, so they can look good and still mislead.
The fix is to validate or replace the label, not to try a more powerful algorithm.

Find the family of CatBoost and of logistic regression in the tables above.

Show solution

Answer: CatBoost belongs to the trees and ensembles family (gradient-boosted trees). Logistic regression belongs to the linear / generalized linear family.

Step by step

Look in the “Typical methods” column of the section 2 table.
CatBoost is listed on the “Trees and ensembles” row, next to random forest and boosting.
Logistic regression is listed on the “Linear / generalized linear” row, because it is a generalized linear model with a logit link.

Checkpoint

You can distinguish paradigm → task → algorithm, choose a task from the desired output, and explain why validated labels and evaluation design matter more than picking the newest model.