Build a content-based recommendation system and test its top results

A content-based recommender ranks items using their attributes or descriptions. It can recommend related items without user interaction history, but its usefulness depends on the representation and the definition of relevance. Similar words are evidence of content overlap, not proof that a person will want an item.

We will build a small original catalogue of learning notes, recommend notes related to one seed item and calculate precision and recall at a fixed review budget. The relevance labels are an authored teaching fixture. They are not feedback from SchoolWhool learners.

Separate catalogue fitting from query evaluation

The catalogue is the set of items available to recommend. Fit TF-IDF on its descriptions, then transform the seed item in the same representation. Using all available catalogue descriptions is appropriate for this declared content-indexing task; using future user clicks as features would be a different information boundary.

We exclude the seed item itself. Otherwise its perfect self-similarity would occupy a recommendation slot and inflate an apparently useful result. Ties use an explicit item-ID ordering so the checked ranking does not depend on an unspecified sorting accident.

The held-out relevance set is not supplied to the ranking function. It is used only after ranking. For a real development process, use separate queries or users for tuning representation choices and final evaluation.

Build and score the miniature recommender

import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity
catalogue = {
 'seed': 'python pandas table cleaning missing values',
 'clean': 'pandas table cleaning missing values duplicates',
 'join': 'pandas table merge join keys',
 'array': 'numpy array broadcasting shapes',
 'probability': 'probability conditional events distributions',
 'plot': 'table chart plot visualization',
 'network': 'neural network gradients torch',
 'files': 'python files csv reading',
}
ids = list(catalogue)
vectorizer = TfidfVectorizer()
matrix = vectorizer.fit_transform(catalogue.values())
scores = cosine_similarity(matrix[ids.index('seed')], matrix).ravel()
ranked = sorted((item for item in ids if item != 'seed'),
                key=lambda item: (-scores[ids.index(item)], item))
k = 3
relevant = {'clean', 'join', 'files'}  # evaluation labels, not ranking input
selected = ranked[:k]
hits = len(set(selected) & relevant)
print('top items:', selected)
print('scores:', [round(float(scores[ids.index(item)]), 3) for item in selected])
print(f'precision@{k}={hits/k:.3f} recall@{k}={hits/len(relevant):.3f}')
unknown = vectorizer.transform(['volcanic mineral crystal'])
print('out-of-vocabulary nonzero features:', unknown.nnz)
assert 'seed' not in selected
assert len(selected) == len(set(selected)) == k

Checked output:

top items: ['clean', 'join', 'files']
scores: [0.781, 0.242, 0.189]
precision@3=1.000 recall@3=1.000
out-of-vocabulary nonzero features: 0

Diagnose a plausible but incomplete ranking

The ranked items share vocabulary with the seed description. A short description can underrepresent a useful item, and common domain terms may dominate when the catalogue is tiny. Inspect the actual recommended IDs and scores before trusting a summary metric.

Precision at three asks what fraction of three recommendations belongs to the declared relevant set. Recall asks what fraction of that set was found. These numbers depend on the completeness and quality of relevance labels. Treating every unjudged item as irrelevant can misrepresent a large catalogue.

The out-of-vocabulary query has no nonzero features. Its cosine scores cannot support a meaningful content ranking. Add an explicit fallback such as a request for a different query or an editorially selected list; do not present an arbitrary tie ordering as personalized relevance.

State what this system cannot infer

The recommender has no learner preferences, item difficulty, prerequisite graph or history. It cannot infer that a beginner is ready for an advanced note. Adding those constraints changes the ranking task and should be evaluated accordingly.

Popularity can be a comparison baseline when appropriate interaction data exists. An offline content score alone does not measure satisfaction, learning progress or novelty. Evaluate those outcomes with suitable consented data and an experiment matching the product decision.

Extend the catalogue

Add a relevant item whose description uses synonyms rather than exact words, and inspect the failure. Compare word n-grams with an embedding representation using a separate development query set. Keep the final evaluation queries untouched while choosing preprocessing and k.

The Machine Learning course connects recommendation with representation and evaluation. Use cosine versus Euclidean similarity for the underlying geometry, and RAG retrieval evaluation for a related ranking task where relevance means finding answer evidence.

References

scikit-learn text feature extraction — TF-IDF representations, tokenization and vocabulary behavior.

Cosine similarity API — Normalized dot products and the similarity calculation.