What is machine learning? Learning a rule from examples

A concrete example

Suppose you want to estimate a home's price. Ordinary programming could encode hand-written rules for size, location and age. A machine-learning system instead fits a model to past homes whose prices are known, then evaluates how well that learned relationship works on homes it did not train on.

The learned model is not a database of perfect answers. Its behavior depends on the data, target, assumptions and evaluation design.

The basic workflow

1. Frame

   Define the input, desired output, decision and cost of mistakes.

2. Collect and inspect

   Understand where data came from and what it leaves out.

3. Split and baseline

   Reserve evaluation data and establish a simple comparison.

4. Train

   Fit model parameters on the training examples.

5. Evaluate and investigate

   Use appropriate metrics, inspect errors and check whether results generalize.

6. Monitor

   After deployment, watch inputs, outcomes and changing conditions.

Three broad kinds

• Supervised learningLearn from examples with known target values, such as prices or labels.
• Unsupervised learningFind useful structure without a supplied target, such as clusters or lower-dimensional representations.
• Reinforcement learningLearn actions through feedback from interaction with an environment.

What the model actually learns

Training adjusts a model's parameters so its outputs fit examples according to an objective. The developer still chooses the target, available inputs, model family, objective and evaluation design. Those choices define what success means and which mistakes can remain hidden behind an average score.

Generalization is the central test: performance must hold on relevant examples that did not drive the parameter updates. That is why data is split, simple baselines are kept and errors are examined by meaningful groups. A system that remembers training examples or exploits a leak can look accurate without learning a relationship useful in the real task.

Terms beginners often confuse

• Feature and targetFeatures are the inputs available to the model; the target is the value or label it is trained to predict.
• Parameter and settingParameters are learned during training. Settings such as tree depth or learning rate are chosen around that process and tested carefully.
• Validation and test dataValidation informs model choices. A held-back test set estimates performance after those choices have been made.