AI Costs at Scale: The Margin Question Every Investor Should Ask

Here's a pattern we keep seeing in AI startups: beautiful growth, healthy-looking margins, and then a quiet problem that only shows up when the biggest customers really start using the product.

Software used to cost almost nothing to serve. AI doesn't. Every answer, summary and agent step has a real price. If a company prices like software and pays like a service, growth makes things worse, not better.

Where AI costs come from

• Model calls. Charged by the amount of text going in and out.
• Long contexts. Feeding large documents to a model on every request is expensive.
• Agents. One user request can trigger many model calls behind the scenes.
• Retries and checks. Quality controls often mean calling a model twice.

The three numbers to ask for

1. Cost to serve one active customer per month.
2. That cost at ten times today's usage. Heavy users often cost far more than average users.
3. Gross margin after AI costs, not before.

What healthy companies do

• Route by difficulty. Use smaller, cheaper models for simple tasks and large models only when needed.
• Cache common answers instead of generating them every time.
• Keep a second provider to protect against price changes.
• Price for usage where usage drives cost.

A quick example

A support automation startup charges $500 per month per customer and spends $60 on model costs. That looks like an 88% margin on AI. But its largest customers send ten times more requests. If pricing doesn't scale with usage, those customers cost $600 a month to serve and become unprofitable.

Check it before the term sheet

AI Due Diligence estimates AI costs and model dependency for any startup in about 10 minutes, so you can ask founders for the right numbers early.