What is machine learning?

How would you write a program that tells a dangerous tumour from a harmless one? You could sit down with a doctor and turn their experience into rules: "if the cells are larger than this, raise an alarm". Or you could show the computer hundreds of past cases with known outcomes and let it work out the rule by itself. The second approach is machine learning, and this course is about why it works, when it fails, and how to tell the difference.

Try this

• Move the threshold slider slowly from left to right and watch the accuracy on the training data. Stop where you think the rule works best.
• Press Learn and compare the threshold the algorithm picks with yours. Look at both numbers: accuracy on the training data and on the test data.
• Push the threshold slider all the way to one end. The rule now gives almost every tumour the same answer — yet the accuracy does not fall to zero. Work out why before reading on.
• After pressing Learn, check which score is higher, train or test, and decide which one you would trust with a real patient.

Now that you've tried it, let's see what was actually going on.

The idea in plain words

In ordinary programming a person writes the rule and the computer applies it. In machine learning a person supplies examples — inputs together with the correct answers — and an algorithm searches for a rule that reproduces those answers. The rule it finds is called a model. Once found, the model is used exactly like a hand-written rule: new input in, prediction out.

Think of learning to tell whether an avocado is ripe. Nobody hands you a formula. You press a few, cut them open, notice which ones were good, and after a dozen tries your thumb "knows". You never wrote the rule down, but you have one, and you built it from examples with known outcomes. A learning algorithm does the same thing, only with numbers and far more patience.

Why it works

The examples in this lesson come from a real medical dataset: 569 breast tumours, each described by measurements of cell nuclei seen under a microscope, and each labelled by a pathologist as malignant (212 cases) or benign (357 cases). To start, we use just one of those measurements and the simplest rule imaginable: "above a threshold → malignant, below it → benign".

Choosing the threshold by hand means guessing. Learning the threshold means trying every candidate value on the examples and keeping the one that makes the fewest mistakes. That is all "learning" is here: a search, guided by data, for the setting that works best on cases we already know. Every model you will meet later, from a straight line to a language model with billions of parameters, follows the same recipe — a family of possible rules, a way to score them, and a search for the best score.

There is one catch, and it is the most important idea in the whole field. A rule that fits the known cases well is not automatically good. What we care about is the cases we have not seen yet: the next patient. So before learning, we hide part of the data. The model learns from the training set and is then graded on the test set, which it never saw. The grade is usually accuracy, the share of cases the rule gets right:

accuracy = correct predictions / all predictions

A rule learned on the training set often scores a little lower on the test set. That gap is the honest price of meeting new data, and the test score is the one you would report to a doctor.

Common mistake

The classic beginner error is judging a model by how well it does on the data it learned from. On those cases the algorithm has had every chance to adjust itself, so the score flatters it. The second trap is forgetting the baseline. Most tumours in this dataset are benign, so a "rule" that always answers "benign" is already right in 357 of 569 cases, nearly two thirds. That is why even a rule that gives almost every tumour the same answer is nowhere near zero accuracy. Any learned model has to beat this trivial answer before it deserves attention, and in medicine it also has to make the right kind of mistakes — a theme we return to in lesson 6.

Machine learning, then, is not magic and not understanding. It is a disciplined search for a rule, followed by an honest exam on data the search never touched. In the next lesson we look more closely at the raw material — data, features and labels — and at why the way you split the data changes the score you get.

from sklearn.datasets import load_breast_cancer
from sklearn.dummy import DummyClassifier

X, y = load_breast_cancer(return_X_y=True, as_frame=True)
print("tumours:", len(y))
print("malignant:", (y == 0).sum(), "benign:", (y == 1).sum())  # in sklearn, 0 = malignant
always_benign = DummyClassifier(strategy="most_frequent").fit(X, y)
print(f"always 'benign': {always_benign.score(X, y):.1%}")

tumours: 569
malignant: 212 benign: 357
always 'benign': 62.7%