How to build a chatbot in Python: rules, retrieval, then an LLM

To build a chatbot in Python, first decide how it chooses an answer. There are three ways. A rule-based chatbot matches patterns you write, such as the word "open", to fixed replies. A retrieval chatbot compares each message with example questions and returns the answer of the closest one. A generative chatbot asks a large language model (LLM) to write the reply, ideally from facts that retrieval found. Start with rules, add retrieval when people phrase things in ways you did not predict, and add an LLM only when you need open-ended replies.

Below you will build the first two in about 75 lines of Python, combine them, and test all three versions on the same messages. Everything runs on your own computer with scikit-learn, with no API key. The shop and its answers are invented for teaching.

Three ways a chatbot can answer

An intent is what the person wants, such as opening hours or a puncture repair. Many practical chatbots, including some built on LLMs, decide the intent first and only then produce a reply.

Build a rule-based and a retrieval chatbot

Install scikit-learn with pip install scikit-learn, then run:

import re
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

# A fictional bike repair shop. Every answer below is invented for teaching.
ANSWERS = {
    "hours": "We are open 9am to 6pm, Tuesday to Saturday.",
    "puncture": "A puncture repair takes about 30 minutes while you wait.",
    "service": "A full service takes one working day. Book online or call us.",
    "location": "We are at 12 Mill Lane, next to the station.",
    "greeting": "Hello! Ask me about opening hours, repairs, servicing or where we are.",
}

# 1. Rule-based: hand-written patterns decide the intent.
RULES = [
    (r"\b(hi|hello|hey)\b", "greeting"),
    (r"\b(open|hours|close)\b", "hours"),
    (r"\b(puncture|flat)\b", "puncture"),
    (r"\b(service|tune)\b", "service"),
    (r"\b(where|address)\b", "location"),
]

def rule_bot(message):
    for pattern, intent in RULES:
        if re.search(pattern, message.lower()):
            return intent
    return None

# 2. Retrieval: find the most similar example question, if it is similar enough.
EXAMPLES = [
    ("what time do you open", "hours"), ("when are you open on saturday", "hours"),
    ("what are your opening hours", "hours"),
    ("can you fix a puncture", "puncture"), ("my tyre is flat", "puncture"),
    ("how long does a tube repair take", "puncture"),
    ("how much is a full service", "service"), ("can i book my bike in for a service", "service"),
    ("my gears are slipping and the brakes squeak", "service"),
    ("where is the shop", "location"), ("what is your address", "location"),
    ("how do i find you", "location"),
    ("hello", "greeting"), ("hi there", "greeting"),
]
vectorizer = TfidfVectorizer(ngram_range=(1, 2), stop_words="english")
example_vectors = vectorizer.fit_transform([text for text, _ in EXAMPLES])

def retrieval_bot(message, threshold=0.3):
    scores = cosine_similarity(vectorizer.transform([message.lower()]), example_vectors)[0]
    best = scores.argmax()
    return (EXAMPLES[best][1] if scores[best] >= threshold else None), scores[best]

def combined_bot(message):
    # Rules first for the exact cases, retrieval for everything else.
    return rule_bot(message) or retrieval_bot(message)[0]

def reply(intent):
    return ANSWERS[intent] if intent else "Sorry, I don't know that one. Please call the shop."

tests = [
    "Hi!",
    "Are you open on Sunday?",
    "I've got a flat tyre",
    "The back wheel tube burst on my commute, how long to repair?",
    "The gears keep slipping",
    "How do I get to your shop?",
    "Where are you?",
    "Do you sell electric scooters?",
]
print(f"{'message':36} {'rules':9} {'retrieval':16} combined")
for message in tests:
    found, score = retrieval_bot(message)
    print(f"{message[:35]:36} {rule_bot(message) or '-':9} "
          f"{(found or '-') + f' ({score:.2f})':16} {combined_bot(message) or '-'}")

print()
for message in ["The gears keep slipping", "Do you sell electric scooters?"]:
    print("you:", message)
    print("bot:", reply(combined_bot(message)))

Checked output:

message                              rules     retrieval        combined
Hi!                                  greeting  greeting (1.00)  greeting
Are you open on Sunday?              hours     hours (0.52)     hours
I've got a flat tyre                 puncture  puncture (0.82)  puncture
The back wheel tube burst on my com  -         puncture (0.65)  puncture
The gears keep slipping              -         service (0.65)   service
How do I get to your shop?           -         location (1.00)  location
Where are you?                       location  - (0.00)         location
Do you sell electric scooters?       -         - (0.00)         -

you: The gears keep slipping
bot: A full service takes one working day. Book online or call us.
you: Do you sell electric scooters?
bot: Sorry, I don't know that one. Please call the shop.

How the rule-based bot works

RULES is a list of regular expressions, each tied to an intent. \b marks a word boundary, so open matches "open" but not "opener". The bot lowercases the message, tries each pattern in order and returns the first intent that matches. If nothing matches it returns None, and reply turns that into a polite "I don't know".

Rules are exact and cheap, and they never invent anything. Their weakness shows in the table: the rule bot misses "the back wheel tube burst", "the gears keep slipping" and "how do I get to your shop", because none of those use a word that a rule was waiting for. Covering them means writing more rules, and rules soon start to conflict. The strings and dictionaries lessons cover the Python this part needs.

How the retrieval bot works

Instead of patterns, the retrieval bot has EXAMPLES: a few ways people ask each question. TfidfVectorizer turns every example into a vector of word weights, where words that are rare across the examples count more than common ones. ngram_range=(1, 2) adds word pairs such as "flat tyre". A new message is turned into a vector the same way, and cosine_similarity scores how close it is to every example, from 0 (no shared words) to 1 (the same words). The turning text into numbers lesson explains these vectors, and cosine similarity vs Euclidean distance explains the score.

The threshold is the most important line. If the best similarity is below 0.3, the bot says it does not know instead of returning the nearest answer. Without that, a chatbot always answers something, and a confident wrong answer is worse than "please call the shop".

stop_words="english" drops words such as "do", "you" and "are" before comparing. Without it, "Do you sell electric scooters?" matched "when are you open on saturday" with a similarity of 0.55, above the threshold, and the bot gave the opening hours. The shared words "do" and "you" said nothing about meaning.

Removing those words has a cost, which the table also shows: "Where are you?" is made entirely of stop words, so nothing is left to compare, the similarity is 0.00 and retrieval gives up. The rule bot answers it, because "where" is one of its patterns.

Combine the two

combined_bot tries the rules first and falls back to retrieval: rule_bot(message) or retrieval_bot(message)[0]. In the table, the combined bot answers all seven questions the shop can answer and declines the scooter question. That pattern, exact rules for the cases you are sure of and a scored fallback for the rest, is a common design for chatbots in real use.

To chat with it in a terminal, replace everything from tests = [ to the end with:

while (message := input("you: ")) != "bye": print("bot:", reply(combined_bot(message)))

Type bye to stop.

Make it better before adding an LLM

1. Collect real messages. Log what people ask, especially the ones that got "I don't know", and add them as new examples.
2. Test it like a model. Keep a separate list of messages with the right intent, never used as examples, and measure how many the bot gets right and how often it wrongly refuses. Change the threshold only by checking it on that list.
3. Add more answers only when you can keep them true. A chatbot that gives wrong opening hours does more harm than one that says "call us".

Add an LLM without letting it invent answers

A generative chatbot sends the message to a language model and returns its reply. Used alone, an LLM will answer questions about your shop with confident text it has no source for. The safer design is retrieval-augmented generation (RAG):

1. Use retrieval, like the bot above, to find the relevant answers or documents.
2. Send the model the question and only those passages, with an instruction to answer from them and to say so when they do not contain the answer.
3. Check the reply before showing it, and keep the "I don't know" path for low retrieval scores.

Build a local RAG pipeline with checked source citations does this end to end with an open model on your own computer. To let the bot take actions, such as booking a service, see LLM tool calling with validated arguments. Validate structured LLM outputs covers checking replies, and prompt injection in a retrieval pipeline covers keeping retrieved text from giving the model instructions.

Where to go next

The Generative AI course syllabus covers language models, retrieval-augmented generation, tools and agents, and evaluating LLM systems; its lessons are being published. The skills underneath come first: Python, then text as numbers in the Machine Learning course. For a program that learns its own rules from examples, read how to make an AI program that learns.

Common questions

Do I need machine learning to build a chatbot?

No. The rule-based bot uses only Python's re module. Machine learning helps when people word the same question in many ways: the retrieval bot uses TF-IDF vectors from scikit-learn, and an LLM goes further.

Can I build a chatbot in Python without an API key?

Yes. Everything in this article runs locally. For generated replies without a paid API, you can run an open-weights model on your own machine, as the local RAG article does.

How do I stop a chatbot from making things up?

Give it a way to say "I don't know": a similarity threshold for retrieval, and for an LLM, an instruction to answer only from the passages you retrieved, checked before the reply is shown.

Which Python library is best for a chatbot?

For rules, the standard library's re. For retrieval, scikit-learn's TfidfVectorizer, or sentence embeddings when wording varies a lot. For generated replies, the client library of whichever model you use. Learn the pattern first; the libraries change faster than the design.

References

scikit-learn: TfidfVectorizer — Word and word-pair weighting, stop words and n-grams.

scikit-learn: cosine_similarity — Similarity between message and example vectors.

Python: re module — Regular expressions and word boundaries.