Prompt Engineering — Master Guide

The Right Mental Model for Prompt Engineering

Prompt engineering is often taught as a bag of tricks. The better mental model: you are writing a job description for a brilliant, literal-minded contractor who has read almost everything but remembers nothing between assignments. Clarity, context, and constraints are more valuable than clever phrasing.

🎯 PM Perspective

Prompts are product. They determine quality, tone, safety, and behavior. A prompt change is a product change — it needs testing, versioning, and review. Investing in prompt quality and systematic iteration is one of the highest-ROI activities in an AI product team.

⚙️ Engineer Perspective

A prompt is a specification. The LLM is the executor. Ambiguous specs produce inconsistent output. The discipline is the same as API design: be explicit about inputs, expected outputs, error cases, and constraints. Test systematically, not by feel.

🧠 The Five Dimensions of a Good Prompt

1. Context

Who is the model acting as? What situation is it in? What does it know? Without context, the model defaults to generic behavior.

2. Task

What exactly should it do? One clear task per prompt. Vague tasks produce vague outputs. Use active verbs: classify, summarize, extract, generate, evaluate.

3. Format

What should the output look like? JSON, markdown, bullet list, a single word? Models will guess if you don't specify — and often guess wrong.

4. Constraints

What should it NOT do? Length limits, tone restrictions, things to avoid, what to do when uncertain. Negative constraints are as important as positive instructions.

5. Examples

One good example is worth 100 words of description. Show the model what good output looks like. Show what bad output looks like too.

Bonus: Persona

Who is the model? "You are a senior software engineer at a fintech company" activates different knowledge clusters than "you are a helpful assistant."

📊 Impact vs Effort — Which Techniques Move the Needle

Technique	Quality Lift	Effort	When to Apply
Clear task description	Very High	Low	Always — foundation of everything
Output format specification	High	Low	Whenever you need structured or consistent output
1-3 examples (few-shot)	High	Medium	Classification, formatting tasks, style matching
Chain-of-thought instruction	High	Low	Math, logic, multi-step reasoning, analysis
Persona definition	Medium	Low	Domain expertise, tone, user-facing products
Negative examples	Medium	Medium	When model keeps making a specific mistake
XML/delimited structure	Medium	Low	Long prompts with multiple sections
Self-consistency (multiple runs)	Medium	High	High-stakes reasoning where accuracy matters most

🔬 Prompt Anatomy — Dissecting a Production Prompt

A production prompt is not a question. It's a structured specification with distinct sections. Understanding each section and its role lets you build prompts systematically rather than by intuition.

🏗️ Full Production Prompt Structure

## SECTION 1: PERSONA + ROLE (activates relevant knowledge) You are a senior product manager at a B2B SaaS company with 10 years of experience writing PRDs, user stories, and competitive analyses. ## SECTION 2: TASK (specific, active verb, one clear goal) Analyze the following customer feedback and: 1. Classify each piece of feedback into one of: Bug, Feature Request, UX Issue, Positive 2. Extract the core pain point in one sentence 3. Suggest the most impactful product change that addresses the feedback ## SECTION 3: CONTEXT / INPUT (delimited clearly) <feedback> {{CUSTOMER_FEEDBACK}} </feedback> ## SECTION 4: CONSTRAINTS (what NOT to do) - Do not infer intent beyond what is explicitly stated - If classification is ambiguous, choose the most specific category - Suggested changes must be implementable in under 2 weeks of engineering time - Do not suggest changes that require a new pricing tier ## SECTION 5: OUTPUT FORMAT (exact structure, leave nothing to chance) Respond in this exact JSON format: { "classification": "Bug | Feature Request | UX Issue | Positive", "pain_point": "one sentence, max 20 words", "suggested_change": "actionable description, max 50 words", "confidence": "high | medium | low" } Return ONLY valid JSON. No explanation, no markdown fences.

💡

Use XML-style delimiters (<feedback>, <context>, <examples>) to separate sections clearly. Claude in particular handles XML delimiters extremely well. It prevents the model from confusing instructions with data.

📍 Prompt Position Matters

Position	Content	Why Here
System prompt (top)	Persona, role, global constraints, output format	Highest priority; sets the model's entire operating context
Beginning of user prompt	Core task, most important instructions	Models are biased toward early content; critical info first
Middle	Examples, context, background data	Referenced when needed; doesn't compete with task instruction
End of user prompt	The actual data/content to process	Recency bias helps the model focus on the immediate input
After data (final line)	Re-state the specific instruction or output format	Recency bias: last thing seen before generating is the output reminder

⚠️

The "lost in the middle" problem: Models pay more attention to content at the beginning and end of long prompts. Critical instructions buried in the middle of a 5000-token prompt are often missed. Put the most important instructions first AND last if needed.

📚 Few-Shot Prompting — Show, Don't Just Tell

One well-chosen example is worth a paragraph of instruction. Few-shot prompting shows the model exactly what you want through input-output demonstrations rather than abstract description.

Zero-Shot vs Few-Shot — When Each Wins

⚠️ Zero-Shot (for classification)

Classify this customer email as: Urgent, Normal, or Low priority. Email: "Hey, just wondering when my order might ship?" Classification:

Problem: Model doesn't know what "Urgent" means in YOUR context. Will guess. Inconsistent across runs.

✅ Few-Shot (same task)

Classify customer emails: Urgent, Normal, or Low. Email: "Our entire team can't log in. Production is down." Classification: Urgent Email: "Can you update my billing address?" Classification: Normal Email: "Hey, just wondering when my order might ship?" Classification:

Examples calibrate what "Urgent" means. Model learns from YOUR definitions, not generic ones.

📋 Few-Shot Best Practices

✓
3-5 examples is the sweet spot for most tasks. More examples rarely add much but add tokens and cost.
✓
Cover edge cases not just easy cases. Include examples of ambiguous inputs and how to handle them.
✓
Match distribution to your actual inputs. If 20% of inputs are edge cases, include edge case examples.
✓
Show negative examples for what NOT to do, especially if the model keeps making a specific mistake.

✓
Consistent format across all examples — if one uses JSON, they all should.
✓
Label your examples with XML tags: <example><input>...<output>...
✓
Order matters slightly — put the most representative examples last (recency bias).
✓
Real examples outperform synthetic ones — use actual production inputs when possible.

💡 Dynamic Few-Shot — Retrieving Examples at Runtime

Static examples in the prompt work for stable tasks. For variable tasks, retrieve the most relevant examples from an example store based on the current input — similar to RAG but for demonstrations.

# Dynamic few-shot: retrieve examples similar to the current input def get_dynamic_examples(input_text: str, n: int = 3) -> list: # Embed the input input_emb = embed(input_text) # Find similar examples from your labeled example store similar = example_store.search(input_emb, k=n) return similar def build_few_shot_prompt(input_text: str) -> str: examples = get_dynamic_examples(input_text) example_block = "\n\n".join([ f"Input: {ex.input}\nOutput: {ex.output}" for ex in examples ]) return f"Here are examples:\n\n{example_block}\n\nNow classify:\nInput: {input_text}\nOutput:" # Result: examples are always semantically similar to the current task

💭 Chain-of-Thought — Making the Model Show Its Work

Chain-of-thought (CoT) prompting instructs the model to reason step-by-step before giving a final answer. For tasks requiring logic, math, analysis, or multi-step inference, CoT significantly improves accuracy.

Why CoT Works

LLMs generate tokens left-to-right. Without CoT, the model must "compress" all its reasoning into the answer token directly — error-prone for complex tasks. CoT externalizes the reasoning process, giving the model "scratch space" to work through the problem before committing to an answer.

❌ Without CoT — Direct Answer

Q: A store offers 15% off on orders over $100. A customer buys $85 of items and adds a $20 item. What is the final price? A: $89.25

Result: Wrong. Model jumped to answer without verifying the $100 threshold logic. ($85+$20=$105, qualifies for 15% off, so $105×0.85=$89.25 is actually right here — but models make mistakes on variations without CoT.)

✅ With CoT — Step-by-Step

Q: [same question] Let me work through this step by step. Step 1: Total order = $85 + $20 = $105 Step 2: Does $105 qualify for discount? Yes ($105 > $100) Step 3: Discount = $105 × 15% = $15.75 Step 4: Final price = $105 - $15.75 = $89.25 A: $89.25

Each step is verifiable. Errors surface visibly instead of hiding in a single wrong answer.

📋 CoT Trigger Phrases — Pick One That Fits Your Context

Zero-Shot CoT

"Think through this step by step." "Let's work through this carefully." "Reason step by step before answering." "Think before you respond."

Simplest. Just add to any prompt. Works surprisingly well.

Structured CoT

"Use this format: Analysis: [your reasoning] Answer: [final answer]" "First state your reasoning, then give the conclusion."

Ensures reasoning is separated from answer for parsing.

Scratchpad CoT

"Use <thinking> tags for your reasoning. Then give your final answer outside the tags."

Reasoning hidden from user but still happens. Claude native feature.

🎯 CoT Decision Guide

Task Type	Use CoT?	Reason
Math / calculations	Always	Direct answers on arithmetic are error-prone without step-by-step
Multi-step logical reasoning	Always	Steps expose where logic breaks down
Code debugging	Always	Trace through execution before diagnosing
Classification (simple)	Rarely	Adds latency and tokens; direct answer is fine for clear categories
Information extraction	Rarely	Extract is mechanical; CoT not needed
Judgment calls / analysis	Yes	Shows the factors considered; improves quality and trust
Latency-critical tasks	Skip	CoT adds 2-5x output tokens; use direct prompting

📐 Structured Output — Getting Consistent, Parseable Responses

Unstructured prose is fine for conversations. For anything that feeds a downstream system (database, API, UI component), you need structured output. Getting reliable JSON, XML, or typed responses from LLMs is a learnable skill.

🎯 Techniques for Reliable Structured Output

Technique 1: Schema in the prompt
Show the exact JSON schema. Include field names, types, and allowed values. The more specific, the more reliable.

# Bad: vague format instruction "Return the result as JSON" # Good: exact schema "Return ONLY this JSON, no other text: { \"sentiment\": \"positive|negative|neutral\", \"score\": 0.0-1.0, \"key_phrases\": [\"string\", ...], \"reason\": \"max 20 words\" }"

Technique 2: Prefill the response (Claude-specific)
Start the assistant turn with {" — the model will continue the JSON from where you started.

# In Claude API, prefill the response messages = [ {"role": "user", "content": "Analyze this: ..."}, {"role": "assistant", "content": '{"'} ] # Model is "forced" to continue valid JSON # Because it already started it

Technique 3: JSON Mode / Tool Use
Modern models support native JSON mode or structured tool call schemas that enforce valid JSON at the API level — more reliable than prompt-based enforcement.

# OpenAI JSON mode response = client.chat.completions.create( model="gpt-4o", response_format={"type": "json_object"}, messages=[...] ) # Claude tool use for structured output tools = [{ "name": "output_result", "input_schema": { "type": "object", "properties": {"sentiment": {"type": "string"}} } }]

Technique 4: Validation + Retry
Always validate the output. If it fails schema validation, send it back with a correction prompt. Most failures correct on retry.

import json from jsonschema import validate def get_structured_output(prompt, schema, max_retries=2): for attempt in range(max_retries): raw = llm(prompt) try: data = json.loads(raw) validate(data, schema) return data except Exception as e: prompt += f"\n\nYour output was invalid: {e}\nPlease fix and return valid JSON only." raise ValueError("Max retries exceeded")

⚙️ System Prompts — The Persistent Foundation

The system prompt is evaluated once per session and sets the global operating context for the model. Getting system prompts right is the highest-leverage prompt engineering activity for product teams.

📋 System Prompt Template (Production)

## Identity and Role You are Aria, an AI assistant for Acme Corp's enterprise software platform. You help users troubleshoot issues, navigate features, and get the most out of the product. You do NOT discuss competitor products or company financials. ## Tone and Style - Professional but warm. Use "you" not "the user." - Concise by default. Offer to expand if the topic is complex. - When uncertain, say so. Never make up product features. ## Knowledge Boundaries You know: Acme platform features, common troubleshooting steps, pricing tiers. You don't know: Individual account data (direct to support for that). If asked about things outside your knowledge, say: "I'd need to check on that — let me connect you with our support team at support@acme.com" ## Behavior Rules - Never claim to be human if sincerely asked - Always offer to escalate to a human agent for billing disputes - Do not provide legal or financial advice - If the user seems frustrated, acknowledge first before solving ## Output Format - For step-by-step instructions: numbered list - For comparisons: table format - For definitions: bold the term, then explain - Default response length: 2-4 sentences unless complexity demands more

🔐 System Prompt Security Considerations

⚠️

System prompts are not secret. A determined user can often extract system prompt contents through persistent questioning. Design system prompts assuming they will be read — put nothing in a system prompt you wouldn't put in public documentation.

Risk	What Happens	Mitigation
Prompt extraction	User asks "what's your system prompt?" and model reveals it	Instruct model not to reveal verbatim; it can confirm it has instructions
Jailbreak	User convinces model to ignore system prompt instructions	Robust instructions, model-level safety filters, monitoring for anomalous outputs
Injection via user input	User crafts input that looks like a system prompt override	Use XML delimiters to separate system instructions from user content
Sensitive data in system prompt	API keys, internal URLs, proprietary logic exposed	Never put secrets in prompts; use environment variables and tool calls

🔄 Evaluation-Driven Prompt Iteration

Prompt engineering without evaluation is guesswork. The teams that build the best prompts are the ones that treat prompt development like software development: version control, test suites, and metrics-driven iteration.

🔄 The Prompt Development Lifecycle

1️⃣

Define

What does "good" look like? Write 5-10 test cases with expected outputs before writing the prompt.

2️⃣

Draft

Write v1 prompt. Use the anatomy framework: role, task, format, constraints, examples.

3️⃣

Evaluate

Run all test cases. Score each output against expected. Identify failure patterns.

4️⃣

Iterate

Fix specific failures. One change at a time. Re-run full suite after each change.

5️⃣

Version

Store prompts in version control. Tag with eval scores. Never replace without regression testing.

📊 Prompt Versioning — Treat Prompts Like Code

# prompts/classify_feedback.py PROMPT_VERSIONS = { "v1.0": { "prompt": "Classify this feedback...", "eval_score": 0.72, "date": "2025-01-15", "notes": "Baseline — struggles with ambiguous inputs" }, "v1.1": { "prompt": "Classify this feedback... [added 3 examples]", "eval_score": 0.84, "date": "2025-01-22", "notes": "Added few-shot; improved ambiguous cases" }, "v2.0": { # current production "prompt": "Classify this feedback... [full structured prompt]", "eval_score": 0.91, "date": "2025-02-01", "notes": "Added chain-of-thought; improved edge cases" } } # Never modify a prompt in production without running the full eval suite first

🚫 Prompt Anti-Patterns — What to Avoid

These patterns appear in prompts constantly. Each one either produces unreliable outputs, makes the model's job harder, or creates security risks. Recognizing them is as important as knowing the positive techniques.

❌ Anti-Pattern 1: Ambiguous Instructions

❌ Bad

"Summarize this article briefly."

What's "briefly"? 1 sentence? 1 paragraph? 5 bullet points? The model will guess — inconsistently.

✅ Good

"Summarize this article in exactly 3 bullet points, each under 20 words. Focus on the main argument, key evidence, and conclusion."

Explicit format, length, and focus. Same output structure every time.

❌ Anti-Pattern 2: Negation Without Alternatives

❌ Bad

"Don't use jargon. Don't be too long. Don't be too formal."

Negation without alternatives forces the model to guess what you DO want. Often inconsistent.

✅ Good

"Write at an 8th grade reading level. Keep responses under 100 words. Use a friendly, conversational tone — as if explaining to a smart friend."

Positive description of what you want, not just what to avoid.

❌ Anti-Pattern 3: Hallucination-Inviting Prompts

❌ Bad

"What are the statistics on customer satisfaction for our product?"

Model has no data about YOUR product. Will hallucinate numbers. This is the most dangerous anti-pattern in production.

✅ Good

"Based ONLY on the data below, summarize customer satisfaction trends. If the data doesn't contain enough information to answer, say 'insufficient data.' <data>{{CUSTOMER_DATA}}</data>"

Ground the model in provided data. Explicit instruction to admit uncertainty.

The Full Anti-Pattern List

Anti-Pattern	Problem	Fix
"Be creative" without constraints	Outputs are wildly variable in tone, length, style	Define what creative means: genre, tone, length, audience
Asking multiple questions in one prompt	Model answers some, skips others, or conflates them	One prompt, one task. Use multiple calls for multiple questions.
Assuming model knows your context	Model doesn't know your company, users, or domain	Provide context explicitly in every prompt
Relying on ChatGPT-style "magic" phrases	"Take a deep breath" worked on specific models; not transferable	Use explicit, model-agnostic instructions
Never testing with adversarial inputs	Prompt works on easy cases, breaks on edge cases	Include edge cases in your eval suite from the start
Modifying production prompts without eval	"Small change" breaks other cases you didn't test	Run full eval suite before any production prompt change

❌ Weak Prompt (before)

✅ Strong Prompt (after)

Task / Question to test all three techniques on:

✍️ Prompt Engineering — Master Guide

🧠 The Five Dimensions of a Good Prompt

📊 Impact vs Effort — Which Techniques Move the Needle

🏗️ Full Production Prompt Structure

📍 Prompt Position Matters

Zero-Shot vs Few-Shot — When Each Wins

📋 Few-Shot Best Practices

💡 Dynamic Few-Shot — Retrieving Examples at Runtime

Why CoT Works

📋 CoT Trigger Phrases — Pick One That Fits Your Context

🎯 CoT Decision Guide

🎯 Techniques for Reliable Structured Output

📋 System Prompt Template (Production)

🔐 System Prompt Security Considerations

🔄 The Prompt Development Lifecycle

📊 Prompt Versioning — Treat Prompts Like Code

❌ Anti-Pattern 1: Ambiguous Instructions

❌ Anti-Pattern 2: Negation Without Alternatives

❌ Anti-Pattern 3: Hallucination-Inviting Prompts

The Full Anti-Pattern List

Lab 1 — Before/After Prompt Comparison

🔍 What Made the Difference?

Lab 2 — Technique Comparison (Zero-Shot vs CoT vs Few-Shot)

⚡ Zero-Shot

💭 Chain-of-Thought

📚 Few-Shot

📊 Technique Verdict for This Task