Chapter 1: What AI engineering actually is

AI engineering is a distinct discipline. It is not data science. It is not machine learning research. It is the practice of building software systems powered by large language models - systems that retrieve knowledge, reason over it, use tools, and interact with users. The AI engineer does not train foundation models from scratch. They take existing models and build reliable, useful systems around them. This chapter gives you the accurate map of the field before you write any code.

Learning objectives

After this chapter you will be able to:

Define AI engineering and distinguish it from data science, ML research, and traditional software engineering
Describe the core building blocks of an AI system: models, prompts, retrieval, tools, and evaluation
Understand the terminology you will need throughout this handbook
Identify what AI engineers actually build day to day

Why this matters

Every newcomer to AI engineering faces the same problem: the field is young, the terminology is unstable, and the path from “I know Python” to “I can build an AI system” is not well mapped. You can waste months reading ML theory that does not help you build anything, or you can copy-paste API tutorials that produce a demo and nothing more. This chapter draws the map so you spend your time on the skills that matter.

What AI engineers build

An AI engineer builds software systems where an LLM is a core component, not an accessory. The system takes input from a user or another system, processes it through a pipeline that may involve retrieval, prompting, tool execution, and validation, and produces output that is correct enough to ship.

Here are the systems AI engineers build:

Retrieval-augmented generation systems. A user asks a question. The system searches a knowledge base, retrieves the relevant documents, feeds them to an LLM along with the question, and returns a grounded answer with citations. You will build one from scratch in Chapter 6.

Fine-tuned models for specific tasks. You take an open-source model (like Llama 4) and train it further on your own dataset so it performs better on your specific task. The model learns your domain’s vocabulary, your preferred output format, or your classification taxonomy. Chapter 7 walks through fine-tuning with LoRA.

Autonomous agents. An LLM in a loop with tools. The agent decides what to do next based on what it observes. It might search the web, query a database, run code, or call an API - and then use the results to decide the next action. Chapter 8 covers agents from scratch.

Agentic pipelines. Multiple agents or LLM calls chained together where the output of one feeds into the next. A research agent gathers information, a writing agent synthesizes it, and a review agent checks the result. Chapter 9 covers composing these pipelines.

Evaluation harnesses. Code that measures whether your AI system is producing correct output. Test sets, automated metrics, LLM-as-judge. The harness tells you when a prompt change improved the system and when it broke something. Chapter 10 is a full chapter on this because it is the skill that separates prototypes from production systems.

These five categories cover the vast majority of what AI engineers build. If you can build each one, you have the core skills.

The AI engineering toolkit

The tools an AI engineer uses daily. You do not need to know them all now. You will learn each one in the chapter where it is relevant.

Models and APIs. You call LLMs through APIs. OpenAI (GPT-5.6), Anthropic (Claude 5), and Google (Gemini) are the major hosted providers. You can also run open-source models locally through Ollama or Hugging Face. A model is a service you call, not a library you import. The API takes text in and returns text out. Everything else is engineering around that.

Prompt engineering. The text you send to the model is a prompt. Writing prompts that produce consistent, correct, parseable output is a skill - and a testable, versionable engineering practice. Chapter 5 treats prompts as software artifacts.

Vector databases and embeddings. To search documents semantically, you convert text to vectors (embeddings) and store them in a vector database. The database finds vectors that are close to your query vector, returning the most relevant documents. ChromaDB, pgvector, Pinecone, Qdrant - they all do the same thing. Chapter 6 covers this in depth.

Fine-tuning frameworks. Hugging Face’s PEFT library, specifically LoRA (Low-Rank Adaptation), lets you fine-tune large models on consumer hardware. You freeze the original model weights and train a small set of adapter weights. Chapter 7 walks through the full pipeline.

Agent and pipeline frameworks. LangChain and LlamaIndex provide abstractions for building agents and pipelines. They handle the loop, the tool definitions, and the state management. You will learn to build agents without frameworks first (Chapter 8) so you understand what the frameworks are doing, then use frameworks for production pipelines (Chapter 9).

Evaluation tools. You need to measure whether your system works. Accuracy, faithfulness, relevance. You build test sets, run automated metrics, and calibrate LLM judges. Chapter 10 covers building an evaluation harness from scratch.

What you will not need

A clear non-goals section saves you months of studying the wrong things. This handbook does not cover and does not require:

Linear algebra, calculus, or probability theory. You do not need to compute gradients or understand backpropagation. You are building with models, not building models.
Training foundation models from scratch. You will fine-tune existing models (Chapter 7), not train GPT-4 from scratch. That requires clusters of GPUs and teams of researchers.
Computer vision, speech recognition, or reinforcement learning. This handbook is about language models. The AI engineering principles transfer, but the specific tools and techniques do not.
GPU cluster administration. You will run models on a single GPU (or CPU) locally or through hosted APIs. You will not configure CUDA drivers or manage Kubernetes GPU nodes.

If your goal is to become an ML researcher, this handbook is not the right starting point. If your goal is to build working AI systems and ship them, it is.

The AI engineering workflow

The workflow an AI engineer follows, from idea to production:

Define the task. What should the system do? Extract data from invoices. Answer questions from documentation. Generate summaries of pull requests. Be specific.
Choose the approach. Do you need RAG? Fine-tuning? A simple prompt? An agent? The decision depends on your data, your accuracy requirements, and your latency budget. Chapters 5-9 each map to a different approach.
Build a baseline. Get a minimal version working. A single API call with a hardcoded prompt. Measure it. Does it work at all? If not, the approach needs to change before you invest in complexity.
Evaluate. Run the baseline against a test set. What is the accuracy? What kinds of inputs does it fail on? The evaluation tells you where to focus your improvement effort.
Iterate. Improve the prompt. Add retrieval. Fine-tune. Add guardrails. Each change is measured against the evaluation suite. Only changes that improve the score stay.
Deploy. Containerize the system. Write a thin API around it. Set up monitoring and cost tracking. Ship it.
Observe and maintain. Watch production logs. Track costs. When the model’s behavior changes (and it will), the evaluation harness catches the regression.

This workflow is the spine of the handbook. Every chapter teaches one phase in depth.

Key takeaways

AI engineering is building software systems around LLMs, not training LLMs from scratch or doing data science.
The five core systems: RAG, fine-tuned models, agents, agentic pipelines, and evaluation harnesses.
You do not need ML theory, calculus, or GPU clusters. You need Python, API calls, and engineering judgment.
The workflow is: define, approach, baseline, evaluate, iterate, deploy, observe. This loops through every chapter.

Common pitfalls

Confusing AI engineering with ML research. If you are reading papers about transformer variants and calculating perplexity scores, you are doing ML research, not AI engineering. Engineering is about building working systems.

Learning theory before practice. You can build a RAG system before you understand how attention works. Build something first. The theory makes more sense when you have seen the system work.

Using AI coding tools as a substitute for understanding. This handbook teaches you to build AI systems. It does not teach you to use AI to write code. If you cannot build a RAG system without Cursor, you do not understand RAG. Learn the engineering first.

Checkpoint exercise

Write down three AI-powered features you could build for a product you use or work on. For each one, identify which of the five approaches applies: RAG, fine-tuning, simple prompting, agents, or a pipeline. Do not code anything yet. Just identify the pattern. This exercise builds the judgment to match problems to approaches, which is the most important skill in AI engineering.