Generative AI tutorials and worked examples

Measure formatted inputs with the model's own tokenizer.

• Tokenization and context windows: budget the formatted conversation

  Count tokens with a pinned open-model tokenizer, inspect chat-template overhead and reserve room for generated output.

  tutorial · 4 min read

Define tasks, frozen questions and component-level scoring.

• Build an LLM evaluation dataset that separates retrieval from answers

  Define answerable and unsupported questions, expected evidence, a scoring rubric and deterministic failure fixtures for a closed factual task.

  tutorial · 4 min read

Compare parameter updates on a declared task and budget.

• LoRA vs full fine-tuning: measure a low-rank update on a declared task

  Compare full and rank-one parameter updates on the same synthetic regression task, with checked parameter counts and held-out reconstruction error.

  comparison · 4 min read

Connect forward noising and reverse inference through checked distributions.

• Diffusion models step by step with a one-dimensional Gaussian

  Calculate the forward noising distribution and an exact reverse conditional on a declared Gaussian, separating an analytic oracle from a learned denoiser.

  tutorial · 4 min read

Check whether the system finds the evidence required to answer.

• Build a local RAG pipeline with checked source citations

  Retrieve from original public teaching notes, generate with a pinned local model and publish only a source-matching quotation or an explicit abstention.

  project · 5 min read

• RAG retrieval evaluation: calculate recall, precision and reciprocal rank

  Build an offline evidence-retrieval test set, compare rankings with explicit relevance judgments and separate retrieval success from answer correctness.

  tutorial · 4 min read

Validate outputs and handle failures at application boundaries.

• LLM tool calling with validated arguments and a bounded dispatcher

  Treat tool proposals as untrusted input, validate a read-only unit conversion and reject unknown tools, fields and nonfinite values.

  tutorial · 4 min read

• Prompt injection in a retrieval pipeline: keep documents from granting permissions

  Test a malicious retrieved instruction with deterministic proposal fixtures and enforce a read-only application capability boundary.

  tutorial · 4 min read

• Validate structured LLM outputs in Python: syntax, schema and evidence

  Test an extraction boundary with malformed JSON, wrong types and schema-valid mistakes, using an offline validator and explicit source facts.

  tutorial · 4 min read