Production Prompt Engineering: Systematic Design and Version Control

Hardcoding prompts into your code is like baking environment variables directly into source. It’s convenient at first, but it will inevitably cause problems in production.

Advanced Prompting Techniques

Chain-of-Thought (CoT)

For complex problems, “show your reasoning” is far more accurate than “just give the answer.”

Python

# Basic prompt (low accuracy)
basic = "Classify the risk clauses in the following contract as HIGH/MEDIUM/LOW."

# CoT prompt (high accuracy)
cot = """Analyze the following contract.

Analysis procedure:
1. Break the contract down clause by clause
2. List potential risk factors in each clause
3. Assess likelihood and impact
4. Finally classify as HIGH/MEDIUM/LOW and explain your reasoning

[Contract content]
{contract_text}

Follow the procedure above and analyze step by step:"""

Few-shot Prompting

Python

few_shot = """Classify the customer inquiry.

Examples:
Input: "I can't make a payment"
Output: {{"category": "billing", "priority": "high", "sentiment": "frustrated"}}

Input: "I'd like to know how to use this"
Output: {{"category": "inquiry", "priority": "low", "sentiment": "neutral"}}

Now classify:
Input: "{user_message}"
Output:"""

Few-shot example principles: include edge cases, 3–5 examples is about right, and extract them from real failure cases.

XML Structuring (Claude-specific)

Python

structured = """<system>
You are a code review expert.
Review criteria: security vulnerabilities, performance issues, readability
</system>

<context>
Language: Python, PEP 8 compliance, type hints required
</context>

<code_to_review>
{code}
</code_to_review>

<output_format>
Return JSON only:
{{"issues": [{{"type": "security|performance|readability", "line": number, "description": "description", "suggestion": "fix"}}],
  "overall_score": 1-10}}
</output_format>"""

Prompt Version Control

CODE

prompts/
├── customer_support/
│   ├── classify_v1.txt      # keep old version
│   ├── classify_v2.txt      # current production
│   └── classify_v3_test.txt # experiment in progress
└── manifest.yaml

YAML

# manifest.yaml
prompts:
  customer_support_classify:
    production: v2
    canary: v3_test
    canary_percent: 5

Python

import yaml, random
from pathlib import Path

class PromptRegistry:
    def __init__(self, prompts_dir="prompts"):
        self.base = Path(prompts_dir)
        with open(self.base / "manifest.yaml") as f:
            self.manifest = yaml.safe_load(f)

    def get(self, name, use_canary=False):
        config = self.manifest["prompts"][name]
        if use_canary and "canary" in config:
            version = (config["canary"]
                       if random.random() < config.get("canary_percent", 0) / 100
                       else config["production"])
        else:
            version = config["production"]

        category = name.rsplit("_", 1)[0].replace("_", "/")
        return (self.base / f"{category}_{version}.txt").read_text(encoding="utf-8")

Prompt A/B Testing

Python

import hashlib
from dataclasses import dataclass

@dataclass
class PromptExperiment:
    name: str
    control_prompt: str
    variant_prompt: str
    traffic_split: float = 0.5

class PromptABTester:
    def __init__(self, experiment):
        self.exp = experiment

    def get_prompt(self, user_id):
        hash_val = int(hashlib.md5(f"{self.exp.name}:{user_id}".encode()).hexdigest(), 16)
        bucket = (hash_val % 100) / 100
        if bucket < self.exp.traffic_split:
            return self.exp.control_prompt, "control"
        return self.exp.variant_prompt, "variant"

    def log_result(self, user_id, variant, metric):
        print(f"exp={self.exp.name} user={user_id} variant={variant} metric={metric}")

Defending Against Prompt Injection

Python

def sanitize_user_input(user_input):
    dangerous = [
 "ignore previous instructions", " when when",
        "system prompt", "</system>", "<system>"
    ]
    cleaned = user_input
    for pattern in dangerous:
        cleaned = cleaned.replace(pattern, "[FILTERED]")
    return cleaned[:2000]

Managing prompts like code makes change tracking, rollbacks, A/B testing, and team collaboration possible. This single practice significantly improves the stability of an LLM service.

When Quality Drops After a Prompt Change: Cause, Reproduction, Prevention

Even with versioning, regressions that hit only certain request types are often discovered after deployment.

Causes

Missing template variable: the new version adds {customer_tier} but calling code doesn't pass it, so an empty string goes in
Model alias update: the prompt is unchanged, but the provider moved the alias to a new model
Conflicting instructions: a new rule contradicts existing few-shot examples

Reproduce: run a regression suite Keep representative inputs and expected conditions in a file and compare both versions on the same inputs. With promptfoo, for example:

YAML

# promptfooconfig.yaml
prompts: [file://prompts/support_v12.txt, file://prompts/support_v13.txt]
providers: [openai:gpt-4o-mini]
tests:
  - vars: {question: "My refund window has passed. Can I get an exception?", customer_tier: "gold"}
    assert:
      - type: contains
        value: "refund"
      - type: not-contains
        value: "{customer_tier}"

Bash

npx promptfoo@latest eval && npx promptfoo@latest view

Fix

Throw an exception at render time if any unsubstituted {...} pattern remains.
Pin models by snapshot name, and log prompt version and model version together on every request.

Prevention

Run the regression suite on every PR and keep adding failures found in production as test cases.

#Prompt engineering #LLM #Chain-of-Thought

✦ ✦ ✦

Sources and verification

Sources and checks vary by article. Only articles with a recorded check date, scope and sources below were checked, and only within that scope. For other articles, verify against the current docs for your version before applying anything, and email us about errors. Editorial policy · email us

This article has no recorded human check (date and scope) yet.

Published · May 20, 2026

How we write and review →Editorial policy →

Was this helpful?

Comments

Be the first to comment.