One Agent, Two Runs: One Large Model for All Steps, or One Chosen per Step

With most agents, every step goes to a single model. Often that’s the largest one you can pay for. But each step can get its own model instead. Little models cover the simple stuff. The larger ones take the tough jobs. You don’t have to guess which setup wins. Put one agent together. Send the same inputs through both versions. Then check the cost, the speed, and what comes out.
Why the choice exists at all
Pricing varies by model. OpenAI’s API price list and Anthropic’s price list charge you per token, and the number changes with the model you hit. You also pay more for what comes out than what goes in. Each step of an agent hits a model at least once, sometimes more. So whichever rate you choose repeats across the full job.
The vendors themselves lean this way. Anthropic’s building effective agents guide walks through routing. Routine, common questions head to a small, inexpensive model, and difficult or odd ones head to a stronger model. That guide also says agentic systems often accept more latency and cost in return for better task performance, and that complexity belongs in a system only once you can show it improves the outcome. OpenAI’s model selection guide tells you to run the same inputs through each option and keep “the lightest setting that meets your quality bar.”
Can choosing a different model for each step bring your bill down without making the output worse? You won’t know till you test it. Here’s how.
Pick a test agent with distinct steps
Find a job that naturally breaks into stages, and not all of them are equally hard. Here’s a good one: gather sources on a subject, turn them into notes, then write up a short brief. That’s 5 steps.
- Go online, find sources, and hang on to what you get back.
- Pull the main figures and key points from each one.
- Use the notes to write the brief.
- Make sure each line in the brief matches the sources you kept.
- Tidy it up and send it over for approval.
Some steps do nothing but read and sort. The rest call for judgment and careful writing. That blend is what makes the test worth running. Don’t assign models to steps from a hunch. The two runs will make it clear.
Set up run A and run B
Run A puts your biggest model on every step. Run B chooses a model for each step. Keep it simple at first. Hand step 2 and step 5, the reading and sorting ones, to a smaller model. Leave drafting and checking with the big model. For every step, jot down why you picked that model. You’ll want it later.
Keep the rest unchanged. Each step keeps the same prompt in both runs. Tools and settings stay the same. Same sources, most of all. When each run searches the web on its own, the pages won’t match. Then a difference between the runs could trace back to which pages each search turned up, rather than to the model you chose. Run the gathering step a single time and store the results. Both runs get that stored set.
Don’t test with just one input. A single subject won’t teach you much. Answers shift between runs. Pick several topics that people on your team genuinely need briefs on. Give each setup multiple passes on every topic.
Write the pass mark down before run A starts. Once you’ve read a few briefs, it’s easy to put the bar wherever your favorite setup lands.
Keep one comparison sheet
Get the sheet in place before you start. Every step in every run gets a row. Columns:
| Step | Model | Tokens in | Tokens out | Tool calls | Retries | Time | Cost |
Then keep a second, leaner sheet. Each finished brief gets a row. Record the run that produced it, the note from the reviewer, and how much time fixing took. Those two sheets are all you need to make the call.
AI agent costs: count them per step
For each step, write down how many tokens go in and how many come out. Multiply those numbers by the rate that model lists. Then add whatever the tools charge. Anthropic’s pricing, for instance, bills web search on top of token costs. Keep a running total for each step and for the whole pass. Now you can see what’s actually eating your budget. Count the repeats too. A step can look cheap. Fail it twice and it stops being cheap.
AI Dust has a write up on the hidden costs of AI agents that says the number you want is what each successful task ends up costing, not what a single call costs. Use that here. Add up everything you spent on a setup. Then split it across the briefs that made it through review.
Log every retry on the step that caused it. A small model that needs 3 tries can end up costing more than a big one that gets it right the first time.
Speed: time the whole run
Time every step, and the whole job from kickoff to finish. Add up how many attempts you need and any minutes lost while a tool churns. Vendors will tell you their compact models are fast. Test it for yourself. A person waiting on the brief doesn’t care which part was slow. They feel the full stretch. For a run that fires overnight, speed hardly counts.
Quality: set the bar before you look
Work out what counts as a solid brief before you look at anything the tool hands back. OpenAI’s guide backs this up. Lock in what passes before you put any two runs side by side. For this task, that bar could be: every claim ties back to a source you’ve saved, nothing on the required list got skipped, any figure you see has to come from one of those sources, and the wording needs to fit whoever’s reading it.
Then go in blind. Pull the labels off. Mix the pile up. Ask a reviewer to mark every brief as keep, fix, or reject. They shouldn’t be able to tell which run made which. Write down how long the fixes take, because fixing time costs you even when nobody sends a bill.
Read the result one step at a time
Lay run A next to run B. Say B kept the quality up and cost you less. Then B is the one to keep. But if the quality dropped, don’t ditch the whole thing. Track down the exact step that broke. Return the big model to that spot only. Feed it the same inputs again. Compare again. The answer could be a blend: the big model where judgment shows in the brief, small models for the rest.
If A beats B on every measure you care about, that’s a good outcome too. That’s the whole point of running the test. You want the call to come from your own numbers, not from a demo.
Where the budgets and sign offs go
Don’t let either setup near a schedule until you’ve put caps in place. The hidden costs article is clear: spend, retries, and tool calls all need a ceiling before you scale. Set them during this test, since you now have numbers to work from.
Work out the most you’ll spend on any single run using your own data. Give it some room above that number. The moment a run touches the limit, it should halt and report which step it reached. If one part of the run costs way more than the others, cap that part by itself. Do the same with tool calls and retries for each step. A step that loops forever shouldn’t be able to burn through everything else.
Put a human approval at every point where an error could escape. For this agent, it’s once the check is done and before anything goes out. Before the brief lands with anyone who depends on it, someone reads it. Add another approval just for how you’ve mixed the models. Change the model on one step and you’ve changed the agent. Feed your test inputs through again and look at what comes out before you put that mix into production.
A cheaper mix still runs the check step. That step is where a small model’s wrong number gets caught. And if the large model only gets one step, that model belongs on the check.
Run it again when the lineup changes
Models change and costs move. Anthropic’s pricing page already carries a separate legacy section alongside today’s lineup. Whenever a fresh release appears or a rate moves, put the same test set through again. Hold on to the same inputs, pass marks and scoring sheet. You don’t have to set anything up again.
Disclosure: I founded ChatFuse. It’s a workspace where every agent step can use its own model. Nothing goes out unless you approve it, or it fits a standing permission you checked earlier. The method above doesn’t need it. That test works anywhere you can pick a different model on every step, even if you’re calling the API yourself.
Frequently asked questions
It can, because vendors charge per token and the rate depends on the model. But retries and fixing time count too. Run both setups on the same inputs and compare what each brief that passes review cost you.
The ones that read and sort, like pulling figures out of sources or tidying a finished draft. Keep drafting and checking on the larger model until your own numbers say otherwise.
More than one. Answers change from run to run, so use several real topics and run each setup on every topic a few times.
Add up everything a setup spent, tool charges and retries included, and divide it by the briefs that made it through review. Cost per call hides the failures.
When a new model comes out or a price changes. Keep the same inputs, the same pass marks and the same sheet, so the new numbers line up with the old ones.
Related posts

Buy the tool or build your own? A practical build or buy framework for businesses choosing AI
A practical guide to help businesses decide whether to buy AI tools, build custom systems, or combine both for better results at scale.
Read more
Google’s New AI Detector Just Went Public and It’s Not What Anyone Was Expecting
Google’s SynthID Detector is now free for everyone, but it’s not the universal AI content detector most people are expecting it to be.
Read more
Best AI Logo Generators in 2026: How AI Is Changing the Way Brands Create Logos
AI logo generators are evolving from tools that simply help users create logos into platforms that help people explore and build brands.
Read more