AI engineering

Topics, in order

1. Mark "Prompting: clear instructions, examples, structured output" as done

   5.1 Prompting: clear instructions, examples, structured output

   • Read

     Prompt engineering overview

     Claude docs

     Worth it: The official guide, with concrete before-and-after prompts for clarity, examples, structure and long documents. Practical and short.

   • Read

     AI Engineering, Ch 5 (prompt engineering)

     Chip Huyen · book you own

2. Mark "Structured outputs and function calling" as done

   5.2 Structured outputs and function calling

   • Read

     Tool use with Claude

     Claude docs

   • Read

     Creating your first schema

     json-schema.org

   • Read

     AI Engineering, Ch 2 (sampling and structured outputs)

     Chip Huyen · book you own

3. Mark "Embeddings and semantic search" as done

   5.3 Embeddings and semantic search

   • Watch

     Cosine Similarity, Clearly Explained

     StatQuest (Josh Starmer)

   • Read

     Embeddings: What they are and why they matter

     Simon Willison

     Worth it: A clear, practical tour of embeddings with real examples of related-content search, clustering and code. The best single read on using them.

   • Read

     Semantic Search

     Sentence Transformers docs

4. Mark "Vector databases and approximate nearest-neighbour search" as done

   5.4 Vector databases and approximate nearest-neighbour search

   • Read

     Hierarchical Navigable Small Worlds (HNSW)

     Pinecone Learn · illustrated

     Worth it: Clear diagrams of the layered graph and the greedy search, plus how the tuning knobs change speed and recall.

   • Code

     Faiss: getting started

     Meta AI · GitHub wiki

   • Paper

     Efficient and robust approximate nearest neighbor search using HNSW graphs (2016)

     Malkov and Yashunin · arXiv · optional

5. Mark "RAG: chunking, retrieval and re-ranking" as done

   5.5 RAG: chunking, retrieval and re-ranking

   • Read

     Introducing Contextual Retrieval

     Anthropic

     Worth it: Shows, with measured results, why chunks lose context and how adding context plus hybrid search and re-ranking cuts retrieval failures. Directly useful for building RAG.

   • Read

     Patterns for Building LLM-based Systems & Products

     Eugene Yan

   • Read

     AI Engineering, Ch 6 (RAG and agents)

     Chip Huyen · book you own

6. Mark "Context engineering: long context and memory" as done

   5.6 Context engineering: long context and memory

   • Read

     Effective context engineering for AI agents

     Anthropic

     Worth it: A practical guide to treating context as a limited budget: what to keep, compact, retrieve just in time, and hand to sub-agents. Written for building real agents.

   • Read

     AI Engineering, Ch 5 (context length and context efficiency)

     Chip Huyen · book you own

   • Paper

     Lost in the Middle: How Language Models Use Long Contexts (2023)

     Liu et al. · arXiv · optional

7. Mark "Evaluation: test sets, LLM-as-judge, error analysis" as done

   5.7 Evaluation: test sets, LLM-as-judge, error analysis

   • Read

     Your AI Product Needs Evals

     Hamel Husain

     Worth it: The most practical guide to evaluating LLM products: unit-test-style checks, LLM judges, and reading traces, from real client work.

   • Read

     Creating a LLM-as-a-Judge That Drives Business Results

     Hamel Husain

   • Read

     AI Engineering, Ch 3–4 (evaluation)

     Chip Huyen · book you own

8. Mark "Agents and tool use" as done

   5.8 Agents and tool use

   • Read

     Building effective agents

     Anthropic

     Worth it: Clear diagrams of the main patterns, from simple workflows to full agents, with advice on when each is worth it. The best starting point for building agents.

   • Read

     LLM Powered Autonomous Agents

     Lilian Weng

   • Read

     AI Engineering, Ch 6 (the agents half)

     Chip Huyen · book you own

9. Mark "MCP: connecting models to tools and data" as done

   5.9 MCP: connecting models to tools and data

   • Read

     Introduction to the Model Context Protocol

     modelcontextprotocol.io

     Worth it: The official docs, with architecture diagrams and quick-starts for building a server and a client. Short, clear, and exactly what you need to build your own.

   • Read

     Introducing the Model Context Protocol

     Anthropic

   • Code

     MCP Python SDK

     GitHub

10. Mark "Choosing between prompting, RAG and fine-tuning" as done

    5.10 Choosing between prompting, RAG and fine-tuning

    • Read

      AI Engineering, Ch 7 (finetuning: when to finetune)

      Chip Huyen · book you own

    • Read

      Patterns for Building LLM-based Systems & Products

      Eugene Yan

11. Mark "Dataset engineering for fine-tuning" as done

    5.11 Dataset engineering for fine-tuning

    • Read

      AI Engineering, Ch 8 (dataset engineering)

      Chip Huyen · book you own

    • Paper

      LIMA: Less Is More for Alignment (2023)

      Zhou et al. · arXiv · optional

    • Paper

      Self-Instruct: aligning language models with self-generated instructions (2022)

      Wang et al. · arXiv · optional

12. Mark "Serving: APIs, Docker, latency and cost" as done

    5.12 Serving: APIs, Docker, latency and cost

    • Read

      FastAPI tutorial

      fastapi.tiangolo.com

    • Read

      Get started with Docker

      docs.docker.com

    • Read

      Made With ML: serving and deployment lessons

      Goku Mohandas · free

13. Mark "Caching, model routing and cost control" as done

    5.13 Caching, model routing and cost control

    • Read

      Prompt caching

      Claude docs

    • Read

      AI Engineering, Ch 10 (AI engineering architecture: caches, routers and gateways)

      Chip Huyen · book you own

    • Read

      AI Engineering, Ch 9 (inference optimisation)

      Chip Huyen · book you own

    • Paper

      RouteLLM: Learning to Route LLMs with Preference Data (2024)

      Ong et al. · arXiv · optional

14. Mark "Observability: tracing and debugging LLM apps" as done

    5.14 Observability: tracing and debugging LLM apps

    • Read

      Your AI Product Needs Evals (the section on logging traces)

      Hamel Husain

    • Read

      Observability primer

      OpenTelemetry docs

    • Read

      AI Engineering, Ch 10 (monitoring and observability)

      Chip Huyen · book you own

15. Mark "Safety: guardrails and prompt injection" as done

    5.15 Safety: guardrails and prompt injection

    • Read

      Prompt injection (series)

      Simon Willison

      Worth it: From the person who named the attack: real exploits against real products, and an honest account of why it is so hard to fix. The essential read on LLM security.

    • Read

      OWASP Top 10 for LLM Applications

      OWASP

16. Mark "ML system design: pipelines, training vs serving, drift" as done

    5.16 ML system design: pipelines, training vs serving, drift

    • Read

      Data Distribution Shifts and Monitoring

      Chip Huyen

      Worth it: The clearest explanation of the kinds of drift, how to detect them and what to monitor, from the author of the standard book on ML systems.

    • Read

      CS329S: Machine Learning Systems Design (notes)

      Stanford

    • Read

      Designing Machine Learning Systems (whole book)

      Chip Huyen · paid