How to make an AI program that learns, step by step in Python

An AI program that learns is a program whose rule comes from examples instead of from you. You give it labelled examples, start it with a rule made of adjustable numbers, and let a training loop nudge those numbers every time the program gets an example wrong. When it stops making mistakes, you test it on examples it has never seen. That loop of predict, compare and adjust is what "learning" means in machine learning, from the smallest classifier to a large language model.

Below you will build one in about 40 lines of plain Python, with no library needed for the learning itself, then do the same job with scikit-learn in four lines. The example predicts whether a student passes an exam from hours studied and hours slept. The data is invented for teaching, so the point is the method, not the finding.

A hand-written rule versus a rule that is learned

An ordinary program follows a rule a person wrote: "a student passes if they studied at least 5 hours". If the rule is wrong, the program stays wrong until someone rewrites it.

A program that learns keeps the rule's shape but leaves its numbers open: "pass if w1 × study + w2 × sleep + b is above zero". It sets w1, w2 and b, called the weights and bias, by looking at examples. Show it different examples and it ends up with a different rule, without anyone editing the code. This is the idea behind the perceptron, one of the first learning programs, and the same idea scales up to neural networks, which stack many of these weighted sums. What is a neural network? shows that next step.

The five steps of a learning program

1. Collect labelled examples. Each example is an input (study and sleep hours) and the right answer (passed or not). In machine learning terms these are samples, features and labels.
2. Choose a model with adjustable numbers. Here, a weighted sum and a threshold: a perceptron.
3. Train: predict, compare, adjust. Go through the examples. When a prediction is wrong, move the weights a little in the direction that would have made it right.
4. Test on examples the program never saw, and compare it with simple baselines. A program that only memorised its training examples is not useful.
5. Use it on a new case.

The complete program

Run this in a notebook or as a .py file. Everything above the scikit-learn section uses only Python itself. The last section needs scikit-learn (pip install scikit-learn).

# Invented teaching data: (hours studied, hours slept) -> passed (1) or not (0).
train = [
    ((6.0, 7.0), 1), ((1.0, 4.0), 0), ((5.0, 3.0), 0), ((4.5, 8.5), 1),
    ((7.0, 5.5), 1), ((2.0, 8.0), 0), ((3.5, 6.5), 0), ((6.5, 4.5), 1),
    ((5.0, 8.0), 1), ((3.0, 5.0), 0), ((4.0, 4.5), 0), ((5.5, 6.5), 1),
    ((8.0, 6.0), 1), ((2.5, 6.0), 0), ((1.5, 7.0), 0), ((7.5, 8.0), 1),
    ((4.5, 7.0), 1), ((5.5, 4.0), 0), ((6.0, 4.5), 1), ((4.0, 7.5), 0),
]
test = [((3.0, 7.5), 0), ((6.0, 5.0), 1), ((7.0, 7.0), 1),
        ((4.5, 9.0), 1), ((5.0, 2.5), 0), ((4.0, 6.0), 0)]

# 1. A hand-written rule: a person chose it, nothing was learned.
def hand_rule(x):
    return 1 if x[0] >= 5 else 0

# 2. A program that learns its own rule from the examples (a perceptron).
mean = [sum(x[i] for x, _ in train) / len(train) for i in range(2)]
weights, bias = [0.0, 0.0], 0.0

def predict(x):
    centred = [x[0] - mean[0], x[1] - mean[1]]
    score = weights[0] * centred[0] + weights[1] * centred[1] + bias
    return 1 if score > 0 else 0

def accuracy(rows, model):
    return sum(model(x) == y for x, y in rows) / len(rows)

print(f"before training: train accuracy {accuracy(train, predict):.2f}")
for epoch in range(1, 101):
    mistakes = 0
    for x, y in train:
        error = y - predict(x)  # -1, 0 or +1
        if error:
            mistakes += 1
            weights[0] += 0.1 * error * (x[0] - mean[0])
            weights[1] += 0.1 * error * (x[1] - mean[1])
            bias += 0.1 * error
    print(f"epoch {epoch}: {mistakes} mistakes, train accuracy {accuracy(train, predict):.2f}")
    if mistakes == 0:
        break

print("learned weights:", [round(w, 2) for w in weights], "bias:", round(bias, 2))
majority = max((0, 1), key=lambda c: sum(y == c for _, y in train))
print(f"test accuracy: learned {accuracy(test, predict):.2f}, "
      f"hand rule {accuracy(test, hand_rule):.2f}, "
      f"always {majority} {accuracy(test, lambda x: majority):.2f}")
print("new student, 6.5 h study and 6 h sleep:", predict((6.5, 6.0)))

# 3. The same job with a library: scale the inputs, fit, score.
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
model = make_pipeline(StandardScaler(), LogisticRegression())
model.fit([x for x, _ in train], [y for _, y in train])
print(f"scikit-learn test accuracy: {model.score([x for x, _ in test], [y for _, y in test]):.2f}")

Checked output:

before training: train accuracy 0.50
epoch 1: 4 mistakes, train accuracy 0.80
epoch 2: 5 mistakes, train accuracy 0.85
epoch 3: 2 mistakes, train accuracy 0.90
epoch 4: 2 mistakes, train accuracy 0.90
epoch 5: 3 mistakes, train accuracy 0.95
epoch 6: 2 mistakes, train accuracy 1.00
epoch 7: 0 mistakes, train accuracy 1.00
learned weights: [0.7, 0.3] bias: 0.0
test accuracy: learned 1.00, hand rule 0.67, always 0 0.50
new student, 6.5 h study and 6 h sleep: 1
scikit-learn test accuracy: 1.00

How the program learns, line by line

predict computes a score from the two inputs and answers 1 (pass) when the score is above zero. Before training, every weight is zero, so every score is zero and the program answers "fail" for everyone. That is why it starts at 0.50 accuracy: half the training students passed.

The training loop is the learning. For each example, error = y - predict(x) is 0 when the prediction is right, +1 when the program said fail but the student passed, and −1 for the opposite mistake. On a mistake, each weight moves by 0.1 × error × input. If the program wrongly failed a student who studied a lot, the study weight goes up, so similar students score higher next time. The bias moves the same way. The 0.1 is the learning rate: how big each correction is.

The inputs are centred first: the program subtracts the average study and sleep hours, so an input says "more or less than usual". Without this, the bias has to carry the whole offset alone: on this data, training takes 107 passes instead of 7. Scaling inputs before training is a standard step for most models, and the scikit-learn version does it with StandardScaler.

Each full pass through the training examples is an epoch. The loop stops after the first epoch with no mistakes, because a perceptron only changes its weights when it is wrong.

Read the output

The mistake count does not fall smoothly. Fixing one example can break another, so epoch 2 has more mistakes than epoch 1. Accuracy, measured after each epoch, still climbs from 0.50 to 1.00, and by epoch 7 every training example is right.

The learned weights are 0.7 for study and 0.3 for sleep. The invented data follows a rule that counts study twice as much as sleep, and the program found a similar balance from the examples alone. Nobody told it that sleep matters.

The test line is the one that matters. On six students the program never saw, the learned rule gets all six right. The hand-written "studied at least 5 hours" rule gets four right: it fails a student who studied 4.5 hours and slept 9, and passes one who studied 5 hours on 2.5 hours of sleep. Always answering "fail" gets three right. Six test examples is far too few to trust in a real project, but it shows the habit: always compare a learned model with a simple baseline on held-out data. The overfitting, underfitting and the test set lesson explains why the test examples must stay unseen during training.

Do the same with scikit-learn

The last four lines do the whole job with a library. make_pipeline(StandardScaler(), LogisticRegression()) scales the inputs, then fits a logistic regression, a close relative of the perceptron that outputs a probability and learns by gradient descent-style optimisation instead of mistake-by-mistake corrections. fit trains it and score reports test accuracy, here also 1.00.

In practice you will use a library, because it handles scaling, stopping, regularisation and many model types for you. Writing the loop once yourself is still worth it: fit stops being magic, and you can tell what went wrong when a model does not learn. The estimator pattern lesson shows how to wrap your own model in the same fit and predict interface.

What this program can and cannot do

• It can only draw a straight boundary. A perceptron separates passes from fails with one straight line. If the true pattern is curved, or the classic XOR problem, no setting of its weights will be right. Neural networks fix this by combining many such units.
• It learns only what the examples show. If every training student slept at least 4 hours, the program has no evidence about students who slept 2. More varied data matters more than a cleverer model.
• It stops at the first perfect pass. A perceptron finds a boundary that separates the training data, not necessarily the best one. Logistic regression and support vector machines choose more carefully.
• It never stops if the data cannot be separated. Real data has noise: two students with the same hours can get different results. That is why the loop has a limit of 100 epochs.

Where to go next

To turn this into a real skill, learn the pieces in order: Python, then data tools, the maths behind the weights and the statistics behind testing. How to learn AI from scratch sets that out as a 12-week plan. If you want to build a program that talks instead of one that classifies, build a chatbot in Python starts from rules and adds learning.

In the Machine Learning course, the perceptron lesson covers this exact algorithm, with the geometry of why the update works. Check what you have learned with the machine learning basics interview questions. Lesson parts and key terms are free to read, and a free account adds the quizzes and saves your progress.

Common questions

Can I make an AI program that learns without a library?

Yes. The perceptron above uses only Python's built-in features. Libraries such as scikit-learn and PyTorch become worth it when you need many model types, large datasets, or neural networks trained on a GPU.

Which programming language is best for making an AI?

Python. Almost every machine learning library, tutorial and research codebase uses it, and it is readable enough for a first program. Python for machine learning covers how much you need.

How much data does an AI program need to learn?

It depends on how complicated the pattern is. Twenty examples are enough for this two-input toy. Real tasks with many inputs need hundreds to millions, and the honest answer comes from a learning curve: train on more and more data and watch the test score.

Is this how ChatGPT learns?

The principle is the same: make a prediction, measure the error, adjust the weights to reduce it. A large language model predicts the next piece of text instead of pass or fail, has billions of weights instead of three, and adjusts them by gradient descent over a very large amount of text. Deep learning vs machine learning explains how the two relate.

References

scikit-learn: Perceptron — The perceptron as a linear model updated only on mistakes.

scikit-learn: StandardScaler — Centring and scaling features before training.

scikit-learn: logistic regression — The probabilistic linear classifier used in the library version.