AI Jargon, in Plain English

For people who use AI but have no plans to become ML engineers.

---

1. What the model is actually doing#

AI model. The component that makes predictions. Give it text, an image, or audio and it produces an output based on patterns learned during training. One distinction clears up a surprising amount of confusion: ChatGPT is a product, GPT is a model family. A product wraps one or more models in search, memory, tools, security rules, an interface, and its own data, which is why swapping the model underneath doesn't always mean rebuilding the product.#

LLM (Large Language Model). A model trained on enormous amounts of text, which is why it can summarize, classify, extract, write code, or answer questions. You'll hear that it "predicts the next word," and that's close enough for an intro but wrong in two ways: models work in tokens rather than words, and calling the whole thing "autocomplete" undersells how much structure it picks up in training. A better mental model is that the LLM keeps guessing which token fits best next, given everything so far. A goal that simple produces behavior that is not simple at all, but it does not include checking a fact database before speaking.#

Probabilistic. Generation is a roll of loaded dice. The model computes a distribution over possible next tokens and its decoding settings decide how it picks among them. That's why testing an AI feature feels nothing like testing a form or a calculator; working four times tells you very little about the next thousand.#

Temperature. A setting, where the model supports it, that controls how much variation you allow. Lower pushes toward predictable choices, higher lets it wander. It does not control accuracy. Temperature zero won't turn an LLM into a database, and hallucinations happen at zero too.#

Training vs. inference. Training is when the model learns its internal parameters from data; inference is when you use the trained model. Asking an AI to review a contract is inference. Your company almost certainly isn't training a foundation model every time someone uploads a PDF. Most AI products live entirely on the inference side, taking an existing model and building software around it.#

Parameters. When someone says "70 billion parameters," they mean the internal values learned during training. For product decisions the number is close to useless on its own, since a bigger model may win on one task and lose to a smaller one on another while costing more and responding slower. Test on your actual task.#

Tokens. Models process text in tokens, which are chunks of language rather than tidy words. You care because tokens determine how much fits in a request and, with most providers, what the request costs. Your instructions consume tokens; so do the conversation history, the retrieved documents, and the answer coming back. Multiply a long prompt by a few thousand requests a day and it shows up in the cloud bill.#

Input and output tokens. Input is what you send, output is what comes back, and providers often price them differently, so "$X per million tokens" is where the cost calculation starts rather than where it ends. The question I'd rather see in a model review is what one successfully completed user task costs. A cheap model that needs retries, extra tool calls, or a second model to check its work can cost more in practice than one with a scarier headline price.#

Context window. How much the model can receive in one request. I think of it as a desk: instructions, history, retrieved documents, tool results, and the current question all have to fit on it. A bigger window is a bigger desk, and a bigger desk doesn't mean the model reads everything on it. Liu et al. showed in "Lost in the Middle" (2023) that information buried in the middle of a long context is retrieved less reliably than information near the start or end. Behavior varies by model, but "it fits" and "the model will use it" remain different claims, and dumping every document you own into the prompt is clutter rather than context engineering.#

---

2. What you give the model matters as much as the model#

Prompt. The instructions and information you hand the model. "Summarize this PDF" is a prompt. So is the production version, which might carry business rules, user state, examples, retrieved documents, format requirements, and restrictions, all before the user's question arrives. Inside a product, "prompt" means something quite different from what someone types into ChatGPT.#

System prompt. Rules set by the application rather than the end user, usually with higher priority. Suppose a customer asks "Can I get a refund?" Behind the scenes the product has already told the model: use the supplied refund policy, don't invent policy details, and if the policy doesn't answer the question, say so. The user supplies the request; the product supplies most of the operating context.#

Prompt engineering. Designing those instructions so the model handles a task more reliably. Sometimes that's clarifying the task, sometimes showing examples, defining what to do when information is missing, or specifying the exact format the software downstream expects. Most of the time the biggest improvement is writing clearer instructions.#

Zero-shot and few-shot. Zero-shot means asking without examples; few-shot means showing a handful first. If "high-priority customer complaint" has a very particular meaning inside your company, five labeled examples will teach it faster than another page of prose.#

Context engineering. Broader than prompt writing. It asks what information the model should see for this particular job, and the answer might include instructions, the user's current state, conversation history, retrieved documents, previous tool results, saved memory, or examples. A great model handed outdated or conflicting context will fail, and when it does, that isn't a model problem.#

Structured output. People can read paragraphs. Software mostly can't. Structured output means asking the model to return a predictable format, so instead of "Maya seems like a high-priority lead from Acme" your application gets name: Maya, company: Acme, priority: high, and the next system in line can act on it.#

Schema. The definition of that structure and its allowed types. Maybe priority can only be low, medium, or high; maybe deal_size must be a number. A schema is the form the model has to fill in.#

Validation. A schema isn't a reason to trust whatever comes back. Validation checks whether the result follows the rules your software expects, so when the model returns "deal_size": "probably huge" where you required a number, the system catches it before that value reaches the next step.#

---

3. Why AI sounds right while being wrong#

Hallucination. Unsupported information presented as if it were valid. Picture an assistant asked about your cancellation policy that doesn't have the policy and writes a believable one anyway. Don't blame temperature. The underlying issue is that language models are built to produce plausible text, not to serve as truth databases.#

Grounding. Giving the model something to base its answer on. Instead of asking "what's our cancellation policy?" the system supplies the actual policy and asks the model to answer from it. Grounding doesn't guarantee correctness, but the model now has evidence to work with, which brings us to the acronym that sounds far more complicated than it is.#

RAG (Retrieval-Augmented Generation). Find useful information first, give it to the model, then ask the model to answer. When someone asks an internal HR assistant how much parental leave they get, the model doesn't need the handbook baked into its training; a RAG system searches the handbook, pulls the relevant section into context, and asks the model to answer from that. One correction worth making loudly: RAG does not mean a vector database. Vector search is one way to retrieve. Keyword search, SQL, APIs, web search, and graph queries are others, and combinations are common.#

Fine-tuning. Taking an existing model and training it further so its behavior changes. This is different from RAG, and the difference costs money when confused. If your problem is "the AI doesn't know this morning's updated policy," you have an information problem, not a fine-tuning problem. The short version: prompting tells the model what to do, RAG gives it the information it needs for this request, fine-tuning changes what the model has learned. My default is to assume a fine-tuning proposal is really a retrieval or prompting problem until someone shows me an eval that says otherwise, because that's how it usually turns out.#

---

4. Data and search: the part people skip until the answer is wrong#

Knowledge base. The collection of information your system can search: documentation, manuals, support transcripts, CRM records, contracts, policies. It's separate from the LLM, and you can update one without touching the other.#

Ingestion. Getting that information into a form the system can work with. Your documents live in Notion, Google Drive, a website, and two databases; something has to collect, clean, update, and prepare them. Nobody demos this part, and it decides more than the model does.#

Chunking. A 200-page document is too blunt a unit for search, so you divide it into smaller pieces that can be retrieved on their own. Too large and you send the model piles of unrelated text; too small and you separate sentences that only make sense together. There's no universal right size, so teams tune it per dataset.#

Retrieval. The step that decides what the model sees. When a RAG answer is wrong, the reflex is to rewrite the prompt or swap models. I'd check what retrieval returned before touching either, because the model may have answered perfectly reasonably from the wrong evidence, and no prompt fixes that.#

Keyword search vs. semantic search. Keyword search matches words; it's good when exact language matters (a product code, a legal phrase, an error message) and bad when two sentences mean the same thing with different wording. Semantic search matches meaning, so "Can I join the course late?" and "Is enrollment allowed after classes begin?" land near each other despite sharing almost no vocabulary.#

Embedding. A numerical representation of an input that preserves useful relationships, so similar texts end up near each other in that space. A separate embedding model usually produces these, and the LLM that writes the final answer can be a completely different model. Worth remembering when someone says "the model" as if there's only one.#

Vector database. A system that stores and searches those numerical representations efficiently. Often useful, never required for RAG. If your existing search infrastructure meets your retrieval needs, you don't need one.#

Hybrid search and reranking. Hybrid search combines methods, usually semantic plus keyword, because each catches cases the other misses; a SKU wants exact matching while a loosely worded question wants similarity. Reranking takes the candidate list search produces and judges relevance more carefully before sending the best few to the model.#

---

5. When AI can do things, not only write things#

API. How software talks to software. Your product sends text to a provider and gets a response back through one, and it's also how an AI application reaches your CRM, calendar, payment system, or internal database.#

Tool and tool calling. A tool is an action the model can request: search the web, check a calendar, query a database, create a ticket. The model doesn't gain access to your systems on its own; developers decide which tools exist and what each can do. Tool calling (sometimes "function calling") is the mechanism. Ask "what meetings do I have tomorrow?" and the model, which contains no calendar, decides it needs the calendar tool, passes the arguments, receives the result, and uses it. The model chooses; your software executes.#

Workflow. A sequence the developer decided in advance. User uploads contract, extract clauses, compare against policy, generate review, send to a human. The AI handles individual steps but the route is fixed, which is exactly what makes it testable.#

Agent. The model gets discretion over what happens next. A research system that decides whether to search, which source to inspect, whether it has enough evidence, and whether it needs another tool call before answering is more agentic than a fixed pipeline. Calling an LLM twice doesn't make something an agent. The practical test is who decides the next step, the software flow or the model.#

Agentic loop. Model decides, tool runs, result returns, model decides again, for a number of steps nobody knows in advance. That flexibility costs you more model calls, higher latency, harder debugging, and more places to fail.#

Multi-agent system. Separate model-controlled actors with different roles, say a researcher, a coder, and a reviewer talking to each other. It makes sense when the work truly splits into independent responsibilities. It also multiplies every coordination problem you already had, and in my experience the diagram with five agents on it is usually drawn before anyone has tried one agent with a good workflow around it. Start with the simplest workflow that works and add autonomy only when it solves a problem you can name.#

MCP (Model Context Protocol). A shared way for AI applications and external systems to expose tools and context, so every pairing doesn't invent its own integration format. It's connection-layer plumbing; it doesn't make a model smarter. And it moves fast: the July 28, 2026 revision made the protocol stateless and removed the old initialization handshake, which broke compatibility with earlier servers. "Supports MCP" tells you a vendor has an integration. It doesn't tell you which version.#

Memory. An application stores information somewhere, then hands relevant pieces back to the model later: history, preferences, prior actions, facts from earlier sessions. When someone says a product "has memory," ask what gets remembered, where, for how long, and whether the user can delete it.#

---

6. Choosing a model: stop asking which one is best#

There's no universally best model. The only useful version of the question is best for which task, under which constraints.

Hosted model. Someone else runs the infrastructure and you access it through an API. You skip operating it yourself and inherit their pricing, rate limits, data policies, reliability, and feature roadmap.#

Open-weight model. The trained weights are available under a license, so you can run it yourself or through another provider. Open weight and open source aren't the same thing; read the license. Running your own model buys control and costs you deployment, GPUs, scaling, updates, monitoring, and security. Open weights don't guarantee privacy either, since a hosted version of an open model still routes your data through someone else's systems.#

Model routing. Sending different jobs to different models: a cheap fast one for simple classification, a heavier one for hard research. Usually smarter than forcing one model to handle everything. The router itself needs testing, though, because a bad one saves money by sending hard tasks to a model that fails them.#

Caching. Reusing work you've already paid for. If the same long instructions, document context, or answer can safely be reused, caching cuts cost and latency. The implementation details differ between providers.#

The real tradeoff. People describe model choice as a triangle of quality, latency, and cost. I'd add a fourth corner: control. You may want the strongest result, and users won't wait twelve seconds for it, and finance won't sign off on the per-task cost, and security won't approve where the data goes. Benchmark your actual task, decide which compromise the product can live with, and measure cost per successful task rather than cost per token.#

---

7. Evals: how you stop judging AI by vibes#

A demo might look impressive and still turn out to be a poor product. The question is whether the system works across the cases that matter.

Eval. Short for evaluation: a repeatable test of some part of the system. You might test whether answers are correct, whether retrieval finds the right evidence, whether the model follows a policy, or whether an agent picks the right tool. If a team changes a prompt and reports that this one feels better, no evaluation has happened.#

Golden set. Test cases with known or reviewed expected outcomes. For a support assistant, that's representative customer questions paired with the correct source documents and acceptable answers. Now every change runs against the same cases instead of whichever example someone happens to be staring at.#

Retrieval eval. RAG systems have two separate questions: did we retrieve the right information, and given that information, did the model answer well? A retrieval eval isolates the first. Keeping them apart saves enormous wasted effort.#

Regression test. A regression is when a change breaks something that used to work; a new prompt fixes contract extraction and quietly ruins invoice extraction. Regression testing reruns known cases after every change, and "every change" includes model swaps, prompt edits, retrieval changes, updated tool descriptions, and agent logic.#

LLM-as-a-judge. When quality has no clean exact answer (was the summary faithful? did the response actually answer the question?), one option is having another model score the output against defined criteria. It helps. It also has its own biases and failure modes, so I wouldn't trust a judge's scores until I'd checked them against a sample of human-graded cases and seen them agree.#

Human eval. Some work still needs people, usually reviewing outputs against a rubric. Slower than automation and worth it when decisions are subjective, mistakes are expensive, or you don't yet trust the automated judge.#

---

8. Production: the part after the demo works#

Guardrail. A constraint or check around model behavior: validating input, blocking restricted content, limiting tool permissions, capping spend, rejecting malformed output, or requiring a person to approve an action. Guardrails reduce specific risks. They don't make an unpredictable system safe.#

Human-in-the-loop. A person stays part of the decision. An agent drafts the refund and someone approves it before it sends; a recruiting tool organizes applications and a recruiter decides. Match the amount of oversight to the cost of getting the action wrong.#

Trace. A record of what happened during one execution. Instead of "user asked question, got bad answer," you see the retrieved documents, model calls, tool choices, arguments, outputs, timing, and where it went sideways. Without traces you're guessing.#

Observability. Your ability to understand how the system behaves once real users touch it: traces, errors, latency, model usage, tool failures, cost, feedback, quality signals. Evals tell you how the system performs on your test cases. Observability tells you what people are actually doing to it.#

Timeout, retry, fallback. APIs fail, tools return junk, models hang. A timeout says how long you'll wait; a retry means trying again under defined conditions; a fallback gives you another route when the preferred one dies, whether that's another model, a simpler workflow, or a human queue.#

Rate limit. How many requests a service accepts over a period. A feature that works for ten testers can fall over when several thousand requests arrive together.#

---

9. Data privacy: "we use AI" tells you almost nothing#

If the system touches customer data, contracts, employee records, source code, health or financial information, or internal strategy, model quality is not the only conversation. You need to know where the data goes.

Data retention. How long a provider keeps your prompts, files, responses, and logs. It varies by product, plan, endpoint, and contract. Ask for the actual policy.#

Training use. A separate question from storage. A provider may retain data for abuse monitoring without using it to train models. OpenAI, for instance, excludes API and business-product data from training by default while consumer ChatGPT depends on the user's settings (checked September 2026). That's one provider's current policy, not an industry rule.#

Zero Data Retention (ZDR). The exact promise depends on the provider, but the general idea is that eligible requests aren't retained after processing. OpenAI announced ZDR for eligible API customers in August 2026, describing it as not retaining prompts or responses once a request completes. Don't stop at the acronym: check which endpoints are covered, what the exceptions are, and whether your whole pipeline follows the same policy, because your model provider can promise ZDR while another tool in your workflow logs everything.#

Data residency. Where the data is physically stored or processed. It matters for contracts, regulated industries, and local law. Review privacy at the system level, not just at the model.#

---

The translation table#

Questions to ask#

When someone proposes an architecture, a tool, or a model change:

1. What exactly is failing today: the model, the context, retrieval, a tool, or the workflow?
2. Do we need an agent, or would a fixed workflow be easier to control?
3. Which eval will tell us whether this change worked?
4. What are cost and latency per successful task?
5. Where does user data go, how long is it kept, and who can access it?

You don't have to build the system to ask these.

---

Further reading#

These were my technical source material and they're the right next step if you want more depth.

• DataTalksClub, LLM Zoomcamp (Agentic RAG lessons). Hands-on coverage of retrieval, search, prompts, tool use, agents, and the agentic loop.
• Shaw Talebi, Practical LLM series. A bridge between the plain-English concepts here and the mechanics of prompting, APIs, embeddings, and fine-tuning.