Prompt Structure for Production Ready Agentic System
Lately, I have picked up a new hobby: reading prompts.
Whenever I want to understand a new agentic project, whether it is an open-source tool or an internal system here at work, I usually start with Claude Code. My flow is pretty simple:
- Run
git clone. - Ask Claude Code: “I want to understand the architecture of this project. Could you please explain it to me in detail?”
- Get the high-level overview, and then ask follow-up questions to dig into the details.
This works perfectly. But recently, I found an even better way to truly understand how an agentic system works: reading its Prompts directory.
People have a lot of opinions on how to build production-ready AI systems. But honestly, if you cannot figure out what a system does just by reading its prompts, it probably is not built right. To me, if you can get all the answers from the prompts, then you are looking at a true agentic system.
Building good infrastructure/architecture is important, but writing a good prompt is just as crucial. When you spend time getting the prompt right, the LLM gets a clear picture of what to do, which means it makes far fewer random guesses.
This got me thinking: What exactly makes a good prompt? What patterns should we use to write them for production-ready agentic systems?
To find out, I spent the last two weeks reading through blogs from OpenAI and Anthropic. I also dug into real-world prompts from open-source repositories and enterprise systems.
I found some really interesting patterns, and here is what I learned.
OpenAI has shared a Prompt Structure that can be used for any type of agent:
The OpenAI Baseline Prompt Structure
Role: [1-2 sentences defining the model's function, context, and job]
# Personality
[tone, demeanor, and collaboration style]
# Goal
[user-visible outcome]
# Success criteria
[what must be true before the final answer]
# Constraints
[policy, safety, business, evidence, and side-effect limits]
# Output
[sections, length, and tone]
# Stop rules
[when to retry, fallback, abstain, ask, or stop]
While OpenAI’s structure is a great baseline, enterprise systems require more specific details depending on the task. I divide agentic prompts into 4 categories:
- Router: I call this the deciding node that figures out how the other nodes should be called and in what order.
- Specialist Prompt: This is a node created to do a specific task or act as a fallback. Every node I create follows this pattern.
- Prompt for Tools: (Covered in a future post)
- Orchestrator: (Covered in a future post)
Note: I will use the example of a job search agentic system to help you understand what the different category prompts look like.
Router Prompt Template
If your system uses a Router node to plan which specialized nodes to call, this prompt is incredibly important. If your prompt does not figure out the perfect plan, or the LLM guesses the wrong plan, the user will likely get the wrong response.
Below is the structure I use when writing the prompt for the router:
Identity
Available Workflows
Decision Logic
Critical Rules
Output Format
Key Principles
I will explain exactly why each section is needed and show you the format.
1. Identity
This is the first thing the LLM reads. It tells the node who it is and, more importantly, what it is not allowed to do. The word “ONLY” is placed here on purpose. Without it, a router might start answering the user’s questions because it seems helpful. “ONLY” creates a hard boundary.
The role of the router is just to plan the order. If it starts helping with content, it is broken.
You are a routing assistant for the Job Search Agent.
Your ONLY job is to analyze the user's message and
return the correct workflow name.
2. Available Workflows
If you do not clearly define where the LLM can send a user, it will often pick the first option that looks okay. This section tells the LLM everything it needs to know about each option. Four things matter here:
- When to use: the ideal situation.
- When NOT to use: the confusing edges where two workflows could both apply.
- Keywords: helpful words for the LLM to decide if this workflow fits.
- Example phrases: real examples that help the LLM match patterns to what users actually say.
The “When NOT to use” is where most people stop too early. Without it, the LLM will just pick the first match. Every boundary needs to be clear from both sides.
## Available Workflows
### search_jobs
**When to use**: User wants to find job listings based on role, location, or any search criteria.
**Do NOT use if**: A specific job is already confirmed in context — use apply_job instead.
**Keywords**:
- “find jobs", "looking for a role",
- “jobs in", "show me openings"
**Example phrases**:
- "Find me backend engineer jobs in Berlin"
- "What product manager roles are available?"
- "Show me remote data science jobs"
---
### apply_job
**When to use**: User wants to apply to a specific job they have already identified.
**Do NOT use if**: No specific job is confirmed in context. Use search_jobs first to identify one.
**Keywords**:
- “apply",
- “submit my application",
- "send my CV to"
**Example phrases**:
- "I want to apply for this role"
- "Submit my application for the senior engineer job"
3. Decision Logic
This section is needed because if a user’s input looks like it belongs to two different workflows, the LLM might just guess or try to mix them together. A priority list gives the LLM a clear, ranked order to follow so it never has to make a random guess.
## Decision Logic
### Step 1: Out-of-scope check (always first)
Is the message clearly unrelated to jobs, resumes,
or career topics?
- YES → route to general_query immediately. Stop.
- NO → continue to Step 2.
### Step 2: Route by priority
| Priority | Condition | Route to |
|-------------|------------------------------------------|--------------------|
| 1 (highest) | Applying to a confirmed specific job | apply_job |
| 2 | Tracking existing applications | track_applications |
| 3 | Jobs matched to user's profile | match_jobs |
| 4 | Searching jobs by criteria | search_jobs |
| 5 | Resume feedback requested | analyze_resume |
| 6 | Unclear or general intent | general_query |
4. Critical Rules
This section handles inputs that look similar on the surface but mean completely different things. I keep it separate from the decision logic just to keep the priority table clean and easy to read.
Every entry in this section represents a real mistake the LLM made in the past that had to be fixed. I use the ✅/❌ pair format on purpose. If you only tell an LLM what not to do, it does not know what to do instead. The correct action must always be shown right next to the wrong one.
## Critical Rules
✅ "What jobs match my background?" → match_jobs
❌ NOT search_jobs — user is asking for profile-based
matching, not criteria-based search
✅ "Apply to the first result" (job shown in context)
→ apply_job
❌ NOT search_jobs again — a job is already confirmed
✅ "I want to apply for a data engineer role"
(no specific job confirmed yet)
→ search_jobs first, then apply_job
❌ NOT apply_job directly — no job has been identified yet
✅ "How is my resume for this job?" → analyze_resume
❌ NOT match_jobs — user wants feedback, not job matches
5. Output Format
This section tells the LLM exactly how to send back its answer, like as a JSON array or plain text. Showing examples of both correct and incorrect formats is super helpful.
Without explicit “do not do this” examples, LLMs will default to bad habits, like adding extra text before the answer or returning a string instead of an array.
## Output Format
Return a JSON array of workflow names in execution order.
✅ Correct:
["search_jobs"]
["search_jobs", "apply_job"]
❌ Incorrect:
"search_jobs" ← not an array
I recommend search_jobs ← no extra text before or after
["search_jobs", null] ← no nulls in the array
Return ONLY the JSON array. No explanation. No extra text.
6. Key Principles
This is a numbered summary of the most important rules, written plainly as a final self-check. It does not introduce anything new.
In a long prompt, the LLM can forget rules stated early on. This section brings those critical rules back into focus right before the LLM generates its output to remind it what to do.
## Key Principles
1. **Route, never respond**: This node produces a workflow
name only — never user-facing content.
2. **When in doubt, use general_query**: Never leave
the user without a path forward.
3. **Job confirmed = apply_job**: If a specific job exists
in context, do not send the user back to search.
4. **Always an array**: Even single-step intents return
a one-element array — never a plain string.
5. **Priority order is strict**: When two workflows could
both apply, the higher priority always wins.
Specialist Node Prompt Template
Below is the structure I use when writing the Prompt for a specialist node:
Identity
Your Role
Assumption at Entry
Available Tools
Workflow
Output Format
Error Handling
Key Principles
1. Identity
Just like the Router template, this is a single sentence telling the node who it is. However, here is the difference: a specialist does not use the word “ONLY” because it has a broader job. Instead, this sentence clearly states who the node is, what it does, and exactly when to stop and hand things over to the next step.
You are the Job Search Specialist for the Job Search Agent.
Your job is to search for relevant job listings based on
the user's criteria and present a curated shortlist.
2. Your Role
This is a simple list of what success looks like for this node. It tells the node what its job is at a high level. The last bullet point is the most important. It clearly states where this node stops. Without this, the LLM will naturally just keep going into the next step.
## Your Role
- Understand what the user is looking for in a role
- Search job listings using available tools
- Rank and curate results by relevance to the user's profile
- Present a shortlist of 3–5 best matches with enough
detail for the user to decide
- You do NOT handle applications — that is done by
the Apply Job node
3. Assumption at Entry
This section states what information needs to be present before this node can run correctly. Specialist nodes get information handed to them from previous steps. This section checks for that information right away so the process doesn’t break later on.
Two things belong here:
- What must already exist.
- What to do if it does not exist (the failure response).
## Assumption at Entry
This node is invoked when the user wants to search for jobs.
**What must be true:**
- The user has expressed a search intent (role, skill,
location, or general job interest)
**If criteria cannot be inferred from the conversation:**
Do NOT call any tool. Ask one clarifying question first:
"What kind of role are you looking for, and do you
have a preferred location or work arrangement?"
Do NOT proceed until at least one criterion is known.
4. Available Tools
This section lists every tool this node can use, but focuses strictly on how to use it for this specific task.
It might seem repetitive to write down the tool details here when the LLM already gets the basic tool descriptions straight from your code. However, the tool descriptions are usually very general. By writing the tool rules in this prompt, you give the LLM the exact context of how this specific node should use the tool.
Three things must be present for each tool:
- What it returns: so the LLM knows how to read the result.
- When to call it: so the LLM does not call it when it shouldn’t.
- When NOT to call it: this explicitly tells the LLM what past mistakes to avoid.
If the data is already there from an earlier step, the tool should not be called again. That rule is worth writing down clearly.
## Available Tools
- `search_job_listings` — Searches job listings by criteria.
- Input: `role` (str), `location` (str, optional),
`remote` (bool, optional), `salary_min` (int, optional)
- Output: array of job objects, each with `jobId`,
`title`, `company`, `location`, `salary_range`,
`description`
- When to call: once at least one search criterion
is known
- ⚠️ Never pass a job title as `jobId` — jobId is a
system identifier, not a display name
- `get_user_profile` — Fetches the user's skills,
experience, and preferences.
- Output: `{"skills": [...], "experience_years": int,
"preferred_location": str}`
- When to call: only when the request is vague and
profile context would improve ranking
- Memory-first rule: if profile data already exists
in context from a prior turn, read from there —
do not call this tool again
5. Workflow
This is the step-by-step recipe the node follows to complete its job. A workflow forces the LLM to do things one step at a time. Without it, an LLM might try to use multiple tools at once or guess the answer before checking the facts.
A good workflow includes:
- Step 0 as a guard: a check before any tool is called, catching problems early.
- Named paths: when the process splits, giving each path a name stops the LLM from mixing them up.
- Inline error handling: each step clearly states what to do if it fails, right there in the step.
## Workflow
### Step 0: Guard — verify criteria before any tool call
If no role, skill, or location can be inferred from
the conversation, ask one clarifying question.
Do not proceed to Step 1 until at least one
criterion is known.
### Step 1: Determine search strategy
**Path A — Specific request** (user named a role,
skill, or location):
Proceed directly to Step 2 using the stated criteria.
**Path B — Vague request** (user said "find me a job"
or "something in tech"):
Call `get_user_profile` first to enrich the search,
then proceed to Step 2.
### Step 2: Search for jobs
Call `search_job_listings` with the available criteria.
- If results return empty:
Inform the user no listings were found.
Suggest broadening criteria (remove location filter
or try a related title).
Do not proceed to Step 3.
- If results return listings:
Proceed to Step 3.
### Step 3: Rank and curate
From returned listings select 3–5 best matches by:
1. Relevance to stated role or skill
2. Match to user profile if fetched in Step 1
3. Recency of posting
### Step 4: Build and return the response
Output the curated shortlist using the response
format below.
6. Output Format
Just like the Router, this shows exactly how this node should format its final answer. I use this section for the exact same reason: to provide a correct example and show incorrect examples to prevent bad habits.
## Output Format
Your entire response must be a single valid JSON object.
No extra text before or after.
{
"answer": {
"case": "jobSearchResults",
"text": "Found 4 backend engineer roles in Berlin matching your profile.",
"actions": [
{ "name": "Apply to a job" },
{ "name": "Search again" }
],
"data": {
"jobs": [
{
"jobId": "j_4521",
"title": "Senior Backend Engineer",
"company": "Acme Corp",
"location": "Berlin, Germany",
"salary_range": "€80k–€100k",
"summary": "Backend role focused on distributed systems and Go."
}
]
}
},
"reasoning": {
"reason": "Searched for backend engineer roles in Berlin, ranked by relevance to stated Go experience.",
"disclaimer": "Job listings are retrieved live and may change at any time."
},
"tool_calls": ["Searched job listings", "Retrieved user profile"]
}
Field rules:
- text: one sentence summarising what was found. No job details.
- summary: maximum 2 sentences per job. Never the full description.
- jobId: internal use only — never render as visible text to the user.
7. Error Handling
This section is only needed when the agent handles any API interaction. Nodes that just read information do not need it.
Each error code gets its own response because each one means something different to the user. A 403 means they don’t have permission, while a 404 means the item wasn’t found. Generic error messages are confusing, and the LLM should tell the user exactly what went wrong. You can also return the JSON response with the fields your workflows require.
## Error Handling
### Pre-flight checks (before calling apply_job API)
Check these conditions before making the API call:
- No job confirmed in context → do not call API.
Ask the user to select a job first.
- User profile incomplete → do not call API.
Ask the user to complete their profile before applying.
### ✅ Success
Application submitted. Confirm to the user with
the job title and company name.
### ❌ 400 — Invalid application data
Inform the user the application could not be submitted
due to a validation issue.
Describe the specific issue from the error response.
Do NOT suggest reapplying without fixing the issue first.
### ❌ 403 — Permission denied
Inform the user they do not have permission to apply
for this role.
Suggest contacting their administrator.
### ❌ 404 — Job no longer available
Inform the user the job listing was not found.
Suggest searching for similar roles.
8. Key Principles
Just like the Router, this is a numbered summary of the most critical rules. Since I already covered why this matters earlier, I use it here for the exact same reason: to give the LLM a quick, final reminder of what to do right before it generates its answer.
## Key Principles
1. **Criteria before tools**: Never call search tools
until at least one criterion is known.
2. **Curate, never dump**: Show 3–5 best matches —
never return the full raw results list.
3. **Memory-first**: Check context for existing profile
data before calling get_user_profile again.
4. **jobId is internal**: Never render a jobId as
visible text to the user.
5. **Scope boundary**: This node presents jobs.
It does not submit applications.
6. **Handle failures well**: If search returns empty,
explain clearly and suggest broadening the criteria.
You might be wondering why there are so many sections and why some of them sound a bit similar. Remember, LLMs are eager to please based on the user’s input. If there is any mismatch or a lack of direct, explicit guidance, the LLM will often panic and just guess, which can completely change the desired output. So, through all these sections, I am simply trying to tell the LLM exactly “what to do” and “what not to do” in different situations.