Images, language and games
Machine learning for LLM
Machine learning for LLM work covers fine-tuning language models, grounding their answers in retrieved documents and measuring whether outputs are correct. Typical projects are document question answering with citations, classification by prompting and structured extraction from reports.
How machine learning is used to build and adapt LLMs
Grounding answers in retrieved documents
Retrieval-augmented generation finds relevant passages with embedding search and gives them to the model, so answers can cite the source. It keeps answers tied to your documents without retraining the model.
Real exampleRetrieval-Augmented GenerationThe original RAG paper combined a pretrained sequence-to-sequence model with a dense retriever over Wikipedia for knowledge-intensive tasks.
Efficient fine-tuning
Instead of updating every weight, adapters train a small number of extra parameters, which makes fine-tuning possible on modest hardware. This is used to teach a model a format, tone or task.
Real exampleLoRALoRA freezes the pretrained weights and trains small low-rank matrices inside each transformer layer, greatly reducing the number of trainable parameters.
Instruction tuning and human feedback
Models are fine-tuned on example instructions and responses, then on human preference comparisons, so they follow requests more helpfully. This step is what turns a raw language model into an assistant.
Real exampleInstructGPT (OpenAI)OpenAI fine-tuned GPT-3 with human demonstrations and reinforcement learning from human feedback to follow instructions.
Measuring model quality
Evaluation runs a model across many tasks and metrics, such as accuracy, calibration and robustness, before it is trusted. Teams also build small evaluation sets for their own use case.
Real exampleHELM (Stanford CRFM)Stanford's Holistic Evaluation of Language Models evaluates language models across many scenarios and metrics in a standard way.
Code generation
Language models trained on source code complete functions from a description, and are judged by whether the code passes unit tests. This powers coding assistants.
Real exampleCodex and HumanEvalOpenAI introduced Codex, a GPT model fine-tuned on public code, and the HumanEval benchmark that checks generated functions against unit tests.
Can you learn machine learning for LLMs?
Yes. Using and adapting LLMs needs solid Python and an understanding of how models are evaluated, while training them from scratch is a research and infrastructure speciality. Learn Python, machine learning evaluation and neural networks, then transformers, retrieval and fine-tuning.
- Python and APIsCall model APIs or run local models, handle JSON and batch requests, and log every prompt and output.Learn it in Python
- Evaluation habitsTrain and test splits, metrics and error analysis matter as much for LLMs as for any model.Learn it in Machine Learning
- Neural networks and transformersUnderstand embeddings, attention and next-token prediction so model behaviour makes sense.Learn it in Deep Learning
- Retrieval, prompting and fine-tuningBuild a retrieval pipeline, compare prompts systematically, and fine-tune a small model with LoRA.Learn it in Generative AI
Moving into LLM engineering
If you already work in the field
If you are a software engineer or domain expert, you already know the documents, users and failure costs that an LLM application must handle. Learn retrieval and evaluation, then build a small question answering tool over documents you know, with a test set of real questions.
If you come from data science or ML
If you already train models, the new pieces are prompting, retrieval, tokenisation, context limits and evaluating open-ended outputs. Learn to build evaluation sets and to measure groundedness, not just fluency.
Roles that do this workMachine learning engineer, LLMAI engineerApplied scientist, generative AIResearch engineer
Tools you will use
| Tool | What you use it for |
|---|---|
| Hugging Face Transformers | Load open language models for inference and fine-tune them on your own task data. |
| Ollama | Run open language models locally for prototyping without sending documents to an outside service. |
| LlamaIndex | Index documents, retrieve relevant passages and build question-answering pipelines over them. |
| FAISS | Search large collections of embedding vectors quickly for retrieval. |
| Ragas | Score retrieval-augmented answers for faithfulness to sources and relevance to the question. |
Public data to practise on
- SQuAD 2.0: Questions on Wikipedia paragraphs, including unanswerable ones, for testing answer extraction and abstention.
- HotpotQA: Questions that need facts from two Wikipedia articles, with supporting facts marked for checking evidence.
- Natural Questions: Real search questions answered from Wikipedia pages, for open-domain question answering.
Your first project
Download the SQuAD 2.0 dev set, answer questions with a small open model given paragraphs retrieved by embedding search, and compare with the model answering without context using F1 and the share of unanswerable questions it correctly declines.
Common questions
Should I fine-tune an LLM or use retrieval?
Use retrieval when the model needs facts from your documents that change over time. Fine-tune when you need a consistent format, style or narrow task behaviour, and often combine both.
Can I fine-tune an LLM on a single GPU?
Small and medium open models can be fine-tuned on one GPU using adapter methods such as LoRA, or QLoRA with quantised weights. Full fine-tuning of large models needs much more memory.
Do I need to understand transformers to build with LLMs?
You can build simple tools without it, but understanding attention, tokens and context windows helps you debug failures and choose models sensibly.
How do you test whether an LLM answer is correct?
Build a set of questions with known answers or source passages, then score outputs automatically where possible and review a sample by hand.
Courses for these skills
- Generative AI: LLMs, RAG and AgentsStudy retrieval-augmented generation, retrieval evaluation and testing LLM outputs.Syllabus available · lessons open as produced
- Data Tools: NumPy, pandas, Matplotlib and scikit-learnPrepare document metadata and keep asset identifiers and document versions traceable.23 published lessons · all planned lessons open
- Practical Machine Learning in PythonBuild classification baselines and evaluate rare categories.51 published lessons · more lessons in production
- Deep Learning with PyTorchStudy learned sequence representations.Syllabus available · lessons open as produced
New to programming or maths? Start with Python, or follow the machine learning path in order. Published lessons are free to read; compare Free and Pro.