Chapter 2: Setting up your AI engineering environment

Before you can build AI systems, you need the tools. This chapter gets you from zero to a working AI engineering environment: Python, API access to hosted models, local models via Ollama, and Hugging Face for open-source models and datasets. By the end, you will run your first model inference both through an API and locally on your own machine.

Learning objectives

After this chapter you will be able to:

• Set up a Python environment with the key AI engineering libraries
• Obtain and configure API keys for OpenAI and Anthropic
• Run open-source models locally using Ollama
• Access models and datasets through Hugging Face
• Write and run your first inference script against both hosted and local models

Why this matters

The environment setup is the first thing that stops beginners. API keys are not obvious. Model downloads are large. Library versions matter. Spending an hour getting this right now prevents hours of debugging cryptic errors later. Every chapter in this handbook assumes you have this environment working.

Python environment setup

Create a dedicated virtual environment for your AI engineering work. This keeps dependencies isolated and prevents version conflicts with other projects.

# Create a project directory
mkdir ai-engineering && cd ai-engineering

# Create and activate a virtual environment
python3.12 -m venv .venv
source .venv/bin/activate  # On macOS/Linux
# .venv\Scripts\activate   # On Windows

# Upgrade pip
pip install --upgrade pip

Install the core libraries you will use across every chapter:

pip install openai anthropic google-genai
pip install jupyter jupyterlab
pip install chromadb sentence-transformers
pip install transformers datasets peft accelerate
pip install langchain llama-index
pip install pydantic python-dotenv tiktoken
pip install fastapi uvicorn  # For deployment in Chapter 11

If you have an Apple Silicon Mac, install the Metal-accelerated version of PyTorch:

pip install torch torchvision torchaudio

PyTorch will automatically use MPS (Metal Performance Shaders) on Apple Silicon, which is important for running and fine-tuning models locally in later chapters. If you are on an NVIDIA GPU, install the CUDA version of PyTorch following the instructions at pytorch.org.

API access to hosted models

The three major LLM providers you will use throughout this handbook.

OpenAI

1. Go to platform.openai.com and create an account.
2. Navigate to API keys and create a new key. Copy it immediately - you will not see it again.
3. Store the key in a .env file in your project root:

OPENAI_API_KEY=sk-your-key-here

Never commit this file. Add .env to your .gitignore.

Load the key in Python:

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

Anthropic (Claude)

1. Go to console.anthropic.com and create an account.
2. Generate an API key from the API Keys page.
3. Add it to your .env:

ANTHROPIC_API_KEY=sk-ant-your-key-here

from anthropic import Anthropic

anthropic_client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

Google Gemini

1. Go to aistudio.google.com and create an account.
2. Generate an API key.
3. Add it to .env:

GEMINI_API_KEY=your-key-here

from google import genai

gemini_client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])

API key security

A few rules that prevent the most common security incidents:

• Keys go in .env. Never in source code.
• .env goes in .gitignore. Verify with git status before committing.
• If you ever commit a key, rotate it immediately. The provider’s dashboard has a button for this.
• Most providers let you set usage limits and IP restrictions. Set these before using keys in production.

Running models locally with Ollama

Hosted APIs are convenient but cost money per call. For development, experimentation, and learning, running models locally on your own machine is free and gives you full control.

Ollama makes this trivial:

# Install (macOS)
brew install ollama

# Install (Linux)
curl -fsSL https://ollama.com/install.sh | sh

# Pull a model - this downloads several GB
ollama pull llama4

# Run it interactively
ollama run llama4

Ollama exposes an OpenAI-compatible API at http://localhost:11434/v1. You can call it with the same OpenAI Python client you use for GPT-5.6:

from openai import OpenAI

local_client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"  # Ollama does not require authentication
)

response = local_client.chat.completions.create(
    model="llama4",
    messages=[{"role": "user", "content": "Explain what a token is in one paragraph."}]
)

print(response.choices[0].message.content)

Models worth pulling for development:

If you have limited RAM or an older machine, start with gemma2:2b. It runs on nearly anything and is fast enough for development iteration.

Hugging Face access

Hugging Face is the central repository for open-source models, datasets, and fine-tuning tools. You will use it extensively in Chapters 7 and 10.

1. Go to huggingface.co and create an account.
2. Generate an access token from Settings → Access Tokens.
3. Add it to .env:

HUGGINGFACE_TOKEN=hf_your-token-here

Log in from Python:

from huggingface_hub import login

login(token=os.environ["HUGGINGFACE_TOKEN"])

You do not strictly need a token to download public models and datasets, but you will hit rate limits without one. A free account is sufficient for everything in this handbook.

Your first inference

Verify everything works by running a single inference call against each provider:

import os
from dotenv import load_dotenv
from openai import OpenAI
from anthropic import Anthropic

load_dotenv()

# OpenAI
openai_client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
response = openai_client.chat.completions.create(
    model="gpt-5.6-luna",
    temperature=0,
    messages=[{"role": "user", "content": "Say hello in one word."}]
)
print(f"OpenAI: {response.choices[0].message.content}")

# Anthropic
anthropic_client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
response = anthropic_client.messages.create(
    model="claude-haiku-4-5",
    temperature=0,
    max_tokens=10,
    messages=[{"role": "user", "content": "Say hello in one word."}]
)
print(f"Anthropic: {response.content[0].text}")

# Local via Ollama
local_client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"
)
response = local_client.chat.completions.create(
    model="llama4",
    temperature=0,
    messages=[{"role": "user", "content": "Say hello in one word."}]
)
print(f"Local: {response.choices[0].message.content}")

If all three print a greeting, your environment is ready. If any fail, check that the API key is loaded correctly, the Ollama server is running (ollama serve), and the model is pulled (ollama list).

Jupyter for exploration

Jupyter notebooks are the standard environment for AI engineering exploration. You iterate faster in a notebook than in scripts because you can re-run individual cells without restarting the whole program.

Start Jupyter:

jupyter lab

Create a new notebook. The first cell should always load your environment:

import os
from dotenv import load_dotenv
from openai import OpenAI
from anthropic import Anthropic

load_dotenv()
openai_client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

From here you can experiment interactively. The pattern for the rest of this handbook: explore in Jupyter, then extract the working code into scripts for the final project.

Key takeaways

• A dedicated virtual environment keeps dependencies isolated and reproducible.
• API keys go in .env. .env goes in .gitignore. Never in source code.
• Ollama runs models locally for free. Start with gemma2:2b for low-spec machines or llama4 if you have 16 GB RAM.
• The three major providers (OpenAI, Anthropic, Gemini) all use the same pattern: client, model name, messages, temperature.
• Hugging Face is your gateway to open-source models and datasets. Create a free account.
• Verify your environment works before moving to Chapter 3: run an inference call against OpenAI, Anthropic, and a local model.

Common pitfalls

Forgetting to activate the virtual environment. If pip install says “externally-managed-environment” (macOS) or installs to the wrong Python, you forgot source .venv/bin/activate.

Committing API keys. Check git status before every commit. If .env shows up, stop. Add it to .gitignore and rotate the key.

Pulling the wrong model with Ollama. Run ollama list to see what you have downloaded. Run ollama pull <model> to add a new one. The first inference after pulling a model takes longer because the model loads into memory.

Not setting temperature 0 for testing. At non-zero temperature, the model gives different answers each time. This makes debugging confusing. Always use temperature 0 when verifying your setup.

Checkpoint exercise

Run the three-provider inference script from this chapter. Note the latency of each call (use time or measure with Python’s time module). Which is fastest? Which output is best? Now modify the prompt: ask each model to explain what a vector embedding is in two sentences. Compare the answers. Are they all correct? Are they saying the same thing in different words or genuinely different things? This exercise calibrates your expectations for model behavior before building real systems.