AI Engineering: What AI Assistants Are (and Aren't)

What you will learn

The difference between AI models and AI products, how tokens and context windows work, and — most importantly — when not to use an AI assistant.

Models vs products

An AI model (like DeepSeek V4 Flash, GPT-4o, Claude) is the underlying neural network that predicts the next token. An AI product (ChatGPT, Claude.ai, Hermes Agent, this Code Tutor) wraps a model with a user interface, context management, system prompts, and safety guardrails.

You don't "use a model directly" in most cases — you use an API or a product that exposes it. Understanding this distinction helps you debug problems: is the model bad, or is the product prompting it badly?

Tokens and context windows

Token: ~0.75 words (in English)
Context window: the maximum tokens the model can see at once
128K tokens ≈ 96,000 words ≈ a 300-page book

Tokens are limited — the model processes everything in the window, so longer inputs cost more and may lose focus on the most important parts (the "lost-in-the-middle" problem).

When NOT to use AI

AI assistants are terrible at:

• Exact arithmetic — they approximate. Use a calculator.
• Consistent formatting — they may change output style each time. Use templates.
• Deterministic logic — they are non-deterministic by nature. Use code.
• Private data without safeguards — never paste customer PII into a public AI product.
• High-stakes decisions — AI can confidently produce wrong answers (hallucination).

Common mistakes

• Treating AI output as fact — always verify, especially for code, math, and citations.
• Expecting consistency — the same prompt can give different answers. Temperature controls randomness.
• Ignoring context limits — if your document is 150K tokens and the model's window is 128K, the tail gets truncated.
• Thinking more context = better — relevant context improves answers; irrelevant context creates noise.

Quick check below!