Prompting

A prompt is the instruction set given to a model at call time. The same model, on the same input, produces materially different output depending on how the request is written, which makes the prompt a piece of production logic rather than a piece of copy.

Treat it accordingly: version it, review it, and evaluate changes to it. A prompt edited directly in a dashboard is an untested deploy.

---

Anatomy of a Prompt

A production prompt has a predictable structure. The order matters, both for recall and for prompt caching.

1. Role and task        "You are a contracts analyst. Extract the termination clause."
2. Constraints          "Use only the supplied document. Do not infer missing values."
3. Output schema        "Return JSON: {clause, page, confidence}."
4. Examples             one or two input/output pairs
5. Retrieved context    the document chunks
6. The question         the specific ask, last

Items 1 to 4 are the system prompt: stable across calls, cacheable, and the part you version. Items 5 and 6 are the user turn, assembled per request.

---

Techniques That Change Output Quality

Few-shot examples do more work than any other technique in extraction and classification pipelines. Pick examples that cover the awkward cases - the empty field, the ambiguous date, the multi-value answer - rather than the clean ones.

---

System Prompt Versus User Prompt

That last row is the one with security consequences. Anything reaching the user turn - including retrieved document text and tool results - is untrusted and can contain instructions. A model has no built-in notion that content in its context is data rather than a command. Separate the two roles, state in the system prompt that supplied content is reference material only, and validate output regardless. See Guardrails & Safety for the enforcement points.

---

Structured Output

Asking for JSON in the prompt text produces valid JSON most of the time, and "most of the time" is a parsing bug at volume. Use provider-side schema enforcement where it exists.

{
  "type": "object",
  "properties": {
    "supplier_name": { "type": "string" },
    "invoice_total": { "type": "number" },
    "currency":      { "type": "string", "enum": ["GBP", "USD", "EUR"] },
    "confidence":    { "type": "number", "minimum": 0, "maximum": 1 }
  },
  "required": ["supplier_name", "invoice_total", "currency", "confidence"]
}

Two rules that prevent most downstream failures:

• Give the model an explicit way to say "not present". Without a null path it will invent a plausible value to satisfy the schema.
• Ask for a confidence field and route low-confidence results to review rather than treating all output as equivalent.

---

Prompts as Production Assets

The minimum viable discipline:

• Prompts live in source control, not in a UI text box.
• Every prompt change runs against a labelled evaluation set before it ships. Twenty representative cases is enough to catch a regression that eyeballing three examples will not.
• Log the prompt version alongside every response so a quality change can be traced to a specific edit.
• Improving one case while breaking four is the normal failure mode of prompt editing, and it is invisible without an evaluation set.

---

Key Takeaways

• Most prompt quality problems are underspecification, not model limitation. Be specific before reaching for a bigger model.
• Few-shot examples are the cheapest reliable way to fix format and edge-case handling. Choose awkward examples, not clean ones.
• Order the prompt static-first: role, constraints, schema, examples, then context, then the question last.
• Retrieved content and tool results are untrusted input. The model cannot tell data from instructions on its own.
• Enforce output schemas provider-side, give the model a null path, and ask for confidence so low-certainty results can be routed to review.
• A prompt change is a code change. Version it and gate it on an evaluation set.