PlainAgent / Guides / Why the demo works and production doesn't

Why the Demo Works and Production Doesn't

Your first AI agent that actually works. By Andrius Macikas · October 2026 · no code needed.

Abstract. The demo works; real use doesn't. Most first agents fail for the same simple reason: they try to do too much in one run, and every extra step is another chance to go wrong. This guide shows the fix, which needs no smarter model: one narrow job, three short steps, a separate checker, and a test set before anything goes live. The running example throughout is a real starter recipe: invoice PDFs from email into a spreadsheet.

By the end you'll know

• How to pick a job an agent can do
• How to test if it is narrow enough
• Why fewer steps win
• How to build a three-step version
• Who checks the agent's work
• What to tick before go-live

Why first agents fail

A first agent is usually a chain: read the email, open the PDF, pull out the fields, check for duplicates, write the row. If each step is right 9 times in 10, the chance that every step is right drops fast, because the odds multiply.

Simple arithmetic (0.9ⁿ and 0.99ⁿ), not a study. Real chains are often worse, because one bad step feeds the next.

Shorter chains beat clever prompts. Cut steps before you add features.

Pick one narrow job

Choose something you do by hand several times a week, with a clear right answer, where a mistake is cheap. Write it as one sentence: when X happens, the agent does Y, and the result goes to Z.

The scoping test. Could a new temp do this job from your one sentence, without asking a question? If not, it's too big or too vague. Split it, or write more detail into the spec.

Example. "When an email with an invoice PDF arrives, pull out supplier, number, date and amounts, and add them as a new row in my bookkeeping sheet."

Three narrow steps

Build the smallest version that works end to end. Make the one action safe: something you can check or undo. Nothing gets sent or paid.

1. Trigger. New email with a PDF that looks like an invoice.
2. AI step. Read the PDF text; return the fields as JSON, null if not printed.
3. Safe action. Skip known invoice numbers; add one row to the sheet.

Scanned PDFs need an OCR step first. A fixed workflow like this isn't a lesser agent; it's the reliable choice when the job allows it.

A separate checker

The agent drafts; something else approves. The thing that makes a decision shouldn't approve its own work.

At first the checker is you: read each new row before you rely on it.

Add a second, dumb rule where you can. For invoices: a sheet formula that flags totals that don't add up, and a duplicate check on invoice number.

Remove approval only for parts that have been consistently right.

Naive vs safe first agent

Before you switch it on

• Ran all 20–30 real examples and read every result
• Tested missing info, needs-a-person and must-never-do cases
• After every fix, re-ran the whole set, not just the failing case
• The action is safe: a row, label or draft; nothing sent or paid
• A person approves results, and "talk to a person" always works
• Every run is logged; you'll read the logs daily for week one
• It's on for a small slice only, such as one inbox label
• A monthly spending limit is set with your model provider

The full path, step by step

Step 1: Pick one small job

What: Choose something you do by hand several times a week that has a clear right answer and where a mistake is cheap.

Why: Agents fail on vague, open-ended work. A narrow job with a right answer is easy to test, and a cheap mistake means you can learn safely.

Write it as one sentence: when X happens, the agent does Y, and the result goes to Z.

Running example: "When an email with an invoice PDF arrives, the agent pulls out the supplier, number, date and amounts, and the result goes into a new row in my bookkeeping sheet."

Other good first jobs: labelling support emails, or answering FAQs from one fact sheet.

Common mistakes

• "An agent that runs my inbox" is several jobs at once. Pick one.
• Picking something where an error costs real money or upsets a customer. Save that for agent number three.

Ideas: 21 free agent templates and two starter recipes.

Step 2: Pick a tool and set up accounts

What: Choose the kind of builder, then get access to what it needs.

Which tool:

• If people will chat with it, use a chatbot builder. Custom GPTs or Claude Projects work for internal use. Voiceflow or Botpress work for a website widget.
• If it reacts to emails, forms, files or a schedule, use an automation tool like n8n, Zapier or Make.

Don't overthink it. The spec you write in step 3 works in any of them.

Accounts you usually need (three things):

1. The builder or automation tool.
2. Access to a model. That means a ChatGPT or Claude plan for chatbot builders, or an API key from a provider such as OpenAI, Anthropic or Google for automation tools.
3. Access to the app it works in (your inbox, sheet or help desk).

Running example: Invoices arrive by email, so this is an automation job. You need an email account the tool can read, an automation tool (n8n, Zapier or Make), a model API key for the AI step, and a spreadsheet with one column per field.

Tip: API usage is billed by the provider and prices change. Check current pricing and set a monthly spending limit now, before you build anything.

Tool comparison: tools for building AI agents. Choosing the model for the AI step? See the writing and chat assistant ranking on TaskRank, our sister site.

Step 3: Write the spec

What: Before building anything, write down exactly what the agent does. Fill in the planner: the job, trigger, inputs, knowledge, allowed actions, never-do rules, handoff and done condition. You get a spec sheet and a system prompt.

Why: Most broken agents are really unclear instructions. A spec forces the decisions up front, and it moves with you if you switch tools later.

Running example (short version):

• Job: extract invoice fields into the sheet.
• Trigger: a new email with a PDF attachment that looks like an invoice.
• Inputs: the PDF, turned into text. Scanned images need an OCR step.
• Allowed action: add one row to the bookkeeping sheet.
• Never do: guess a value that isn't printed (use null instead), or pay or reply to anything.
• Handoff: you check the rows by hand.
• Done when: the row is added, or the invoice number is already in the sheet and gets skipped.

Collect your test set now: gather 20 to 30 real examples of the input. Here, that means 20 to 30 real invoice emails, including messy ones. You'll need them in step 5.

Common mistake: leaving out the never-do rules. They matter more than the do rules.

Open the planner · The spec behind this example · Writing the system prompt

Step 4: Connect trigger, model and one tool

What: Build the smallest version that works: a trigger, one AI step with your system prompt, and one action.

Why: Every extra step is another thing that can break. Get one path working end to end, then grow it.

Make the action safe: a label, a draft, a row in a sheet. These are actions you can undo or check. Nothing gets sent or paid.

Ask for structured output: when the next step reads the model's answer, ask for JSON with fixed field names, not free text.

Running example, high level:

1. Trigger on new emails that have a PDF attachment and look like invoices.
2. Turn the PDF into text (with OCR for scans).
3. Ask the model for the fields as JSON, with null for anything not printed.
4. Skip invoice numbers already in the sheet, then add the row.

Common mistake: adding a second tool or a "smart" branch before the first path works.

Step 5: Test, then go live safely

Test with a checklist. Run every example from your test set and read every result. Include:

• normal cases,
• cases with missing information,
• cases that should go to a person,
• cases asking for something it must never do.

When something fails, fix the instructions or the knowledge, then re-run the whole set, not just the failing case. Fixes often break something else.

Running example: feed in your invoices, including a scanned one, one with no invoice number and a duplicate. Check that missing fields come back as null rather than a guess, and that duplicates are skipped. Add a formula to the sheet that flags totals that don't add up.

Go live in a small slice. Turn it on for one inbox label, one web page or one teammate first. Then:

• keep a person approving actions,
• make sure "talk to a person" always works (for chatbots),
• log every run, and read the logs daily for the first week.

Only remove approval for parts that have been consistently right, and keep re-testing whenever you change the prompt.

Running example: point it at one "Invoices" label only. Keep checking every row by hand for the first week before you trust it.

Common mistake: "it worked on three examples, ship it." Three examples isn't a test.

Conclusion

You don't need a better model to get a working first agent. Narrow the job, keep it to three steps, give it a checker, and test before it goes live. The full seven-step path, the planner and 21 free templates are at the links below.

• Start here: the full 7-step path
• 21 free templates
• Planner
• Starter recipes

First published as an X Article by @AndriusMacikas on 16 October 2026. If it helped, reply there and tell me what you built.

Related guides

• How to build an AI agent, step by step
• AI agent examples
• What are AI agents?