The Cheapest AI Model Wasn’t the Cheapest Worker

Clients ask me versions of the same questions all the time:

“Should we use an open-source model?”

“Can’t we just install AI on an office computer and stop paying subscription fees?”

“Why pay Claude rates when DeepSeek costs almost nothing?”

Strictly speaking, most of what people mean by “open source” here is open-weight. But the more important mistake is economic: we are comparing the unit price of intelligence before defining the unit of work.

We would never evaluate labour that way. “Why hire a $250,000 operator when a $50,000 employee exists?” is meaningless until you know how much time, supervision, coordination and rework each one consumes. Asking why you cannot simply get a cheap freelancer is the same argument taken to its logical extreme.

Token price is closer to a wage rate than a cost per outcome.

So I ran a simple experiment. First, this is how the four models looked on the rate card:

All figures are USD. DeepSeek and GLM were accessed through OpenRouter using published provider pricing. Luna and Sonnet use their published first-party API pricing.

On the rate card, DeepSeek and GLM look like the obvious bargains. Claude looks extravagant.

Then I gave them the job.

The job

I gave four model + agent harness combinations exactly the same task.

There were 15 emails across 12 conversations in a real Gmail inbox.

The worker had to:

• read the inbox without changing anything;
• summarise the emails and make suggested actions for each;
• create a Google Doc with the analysis;
• find a specifically named Google Drive folder;
• save the finished document there.

Each worker had the same basic operating environment:

• Model: the underlying intelligence (LLM), such as Claude, GPT, DeepSeek or GLM.
• Agent harness: the software around the model that lets it plan work, use tools, maintain context and decide when it is finished.
• Cloud sandbox: an isolated temporary computer where the agent actually performs the job. It costs money for as long as it is running.
• Business-tool connections: secure API access to Gmail, Google Drive and Google Docs. Every tool execution also has a real marginal cost.

The four combinations were:

So this wasn’t a pure model benchmark.

It was a comparison of four complete AI workers trying to achieve the same business outcome.

What actually happened

Here is what each worker consumed while performing exactly the same job.

The providers report caching differently. To make the comparison fair, the table now shows total input/context processed. Claude’s 450,187 total includes 384,349 cached tokens, 65,822 tokens written to cache and 16 ordinary input tokens. Its API cost remains $0.2866 because those cached and cache-write tokens were already included in the original cost calculation.

• Claude finished in 75 seconds and used 3 business-tool calls.
• Luna used one additional business-tool call and finished in under four minutes.
• DeepSeek ran for more than 15 minutes.
• GLM processed 4.22 million cumulative input tokens and made 56 business-tool calls.

The price per million tokens was clearly not telling us the whole story.

What did the job actually cost?

For the non-model costs, I used:

• Cloud sandbox: $0.000046 per second.
• Business-tool execution: $0.0003 per call.

Then I combined those costs with the actual token usage above.

If I ranked them purely by the cost of the model:

• DeepSeek - $0.0093
• Luna - $0.0291
• GLM - $0.1017
• Claude - $0.2866

But once they actually had to perform the work:

• Luna - $0.0405
• DeepSeek - $0.0578
• GLM - $0.1421
• Claude - $0.2909

DeepSeek was the cheapest model. It was not the cheapest worker.

And the reason becomes obvious when you examine what each worker actually did.

Claude Sonnet 5.5: 3 calls

Claude’s execution was remarkably simple. It effectively did:

• Fetch the Gmail inbox once.
• Find the destination folder once.
• Create the finished document there.

The first Gmail call returned enough information for Claude to understand and summarise all 15 emails. It did not reread every email individually, create the document somewhere else and move it afterwards, or enter an elaborate checking loop.

Total: 3 business-tool calls, 75 seconds, all 15 emails represented.

Operationally, this was by far the most efficient and sophisticated execution.

GPT-6 Luna: 4 calls, with better synthesis

Luna needed one additional business-tool call, but it also did something useful with the information.

It recognised that 15 emails did not mean 15 separate conversations. Several emails belonged to the same discussion, so Luna identified the 12 actual conversations and summarised them accordingly.

The result was arguably the most useful executive summary of the four.

Total: 4 business-tool calls, 3 minutes 42 seconds, 15 emails understood as 12 conversations, $0.0405 total variable cost.

That made Luna the cheapest completed outcome for this job.

DeepSeek V4 Flash: 23 calls

DeepSeek started with an enormous price advantage. Its actual model inference bill was less than one cent.

But its execution was much more complicated. Its business-system operations included:

• 3 inbox fetches;
• 15 individual message fetches;
• document creation;
• 2 file searches;
• metadata retrieval;
• file movement.

Total: 23 business-tool calls.

It also encountered an execution problem, followed by one connection-discovery operation that took roughly five minutes.

Although the intelligence itself cost less than a cent, the sandbox ran for more than 15 minutes. The sandbox alone cost $0.0417 and tool calls added another $0.0069.

The final job cost became $0.0578 - about 43% more than Luna.

GLM 5.3 Flash: 56 calls, and still incomplete

GLM produced the most extreme execution pattern.

It fetched the inbox once, then made 49 separate fetch-message-by-ID calls for an inbox containing only 15 messages.

After that it also:

• created the document;
• searched for a file;
• searched for the folder;
• retrieved metadata;
• moved the file twice.

Total: 56 business-tool calls.

The repeated work drove its model usage to 4.22 million cumulative input tokens. Despite GLM having cheaper token pricing than Luna, its actual model bill reached $0.1017.

Then add 8 minutes 32 seconds of cloud compute and 56 business-tool calls.

Total: $0.1421 - more than 3.5 times Luna’s total cost.

But cost wasn’t the biggest problem. The run reported 15 messages across 12 threads. When I inspected the finished Google Doc, it contained only 11 conversation summaries. One conversation was missing.

So the worker that made the most tool calls, processed the most input context and performed the most repeated retrieval still returned an incomplete result.

And that introduces another cost that isn’t even in the table: rework.

What I learned

1. Start with the business outcome, not the model

There is no meaningful “best model” in isolation. The right level of intelligence depends on the work.

If the job is genuinely deterministic grunt work - red files into Bucket 1, blue into Bucket 2, green into Bucket 3 - a much cheaper model may be entirely sufficient.

If the work requires complex logic, judgement, correlation across multiple sources, pattern recognition, prioritisation or ambiguity resolution, stronger model intelligence can be worth paying for.

Most business work sits somewhere between those two extremes. This inbox task did.

Three workers completed it fully. GLM got most of the way there but missed one of the 12 conversations. So the real question is not simply which model “won”. It is what the business can tolerate.

• Can one missed item be tolerated, or is that a failure?
• Does the job need to finish in 75 seconds, 15 minutes or overnight?
• Will a human review every result, only exceptions, or nothing at all?
• How observable does the process need to be?
• What happens, commercially or operationally, when the agent gets something wrong?

The acceptable model depends on the standard of the outcome, not the model leaderboard.

2. Dissect the task before you automate it

The biggest efficiency gain may come before model selection.

Break the work down first:

• Which steps are deterministic?
• Which steps require judgement?
• What information is actually required?
• What can be retrieved once instead of repeatedly?
• What are the exception conditions?
• What proves the job is complete?

Claude showed that this particular job could be completed with three business-system operations. That is useful even if Claude is not the model you ultimately deploy, because it tells you what an efficient process can look like.

Design the minimum reliable sequence of work first. Then test which model can execute it.

3. Match the intelligence to the step

The best production architecture may not use one model for everything.

• Use very cheap models for deterministic extraction, sorting and classification.
• Use stronger models where reasoning, synthesis, correlation, pattern recognition or judgement materially changes the result.
• Escalate difficult or ambiguous cases instead of paying premium inference for every step.

Do not pay for maximum intelligence everywhere. But do not underpower the parts of the process where intelligence actually changes the business outcome.

4. Optimise the whole system, not the token price

Inference price is only one component of the economics. The real cost of the job also includes compute time, business-tool calls, retries, failures, human review and rework.

That is why a cheaper model can become the more expensive worker, and why a premium model can sometimes be economically rational even when its token price looks outrageous.

The unit that matters is cost per correctly completed business outcome.

The conclusion

The real lesson from this experiment is not that Luna beat DeepSeek, or that Claude is better than GLM.

It is that AI adoption is fundamentally an operating-design problem.

The sequence should be:

• define the business outcome;
• dissect the task into the work that actually needs to happen;
• define accuracy, speed, observability and error tolerance;
• design the minimum reliable workflow;
• then find the best way to deliver that outcome economically.

Too often we start at the other end. We choose a model because it is cheap, powerful, open-weight or fashionable, then build the process around the technology.

The biggest efficiency gains will often come from better work design, not cheaper tokens.

So do not start with: “Which model is cheapest?”

Start with: “What outcome must this process deliver, what is the minimum reliable sequence of work to get there, and where does more intelligence materially improve the result?”

Then buy only as much intelligence as the job actually needs.