The AI Revolution: A Double-Edged Sword for Margins

There’s no doubt that AI is on a path to revolutionizing every industry on the face of the planet (and Mars if you ask Elon). But the goal of business isn’t just to make things better – it’s ultimately to make money. So, while AI offers the promise of unprecedented capabilities, it comes with a financial cost that could dramatically reduce the enterprise values assigned to many AI-centric-businesses.

Inference: The Cost Hiding In Plain Sight

Let’s start by highlighting the well-known fact that efficient software businesses are loved by Investors due to their ability to scale infinitely with minimal marginal costs. Once developed, software can be replicated and distributed at near-zero cost, allowing companies to enjoy substantial profit margins as they grow. This model has propelled firms like Microsoft and Adobe to financial heights that manufacturing-based businesses can only dream of.

However, while many Investors want to think of AI as “software” and are currently valuing AI startups using multiples that even the best software companies would dream of, AI-driven businesses have to face a different economic reality. While they share some characteristics with traditional software companies, they also incur significant ongoing costs related to their core operations. The primary culprit is the cost of inference – the process of using trained AI models to generate outputs based on user supplied inputs.

Inference costs are non-trivial and scale with usage. Unlike static software that runs on a user’s device, AI models require substantial cloud computing resources for each interaction (energy, GPU chips, cooling units, real estate, etc). This creates a situation more akin to a service business than to a pure software play. Companies must continually pay for the resources needed to run their AI models, a cost that grows in proportion to their user base and usage intensity. And these costs aren’t trivial.

Consider the contrast between Google’s search business and an AI-centric company like OpenAI.

Google’s search business, while computationally intensive, benefits from efficient indexing and retrieval systems built over decades. The marginal cost of completing a search query, while not zero, is relatively low. The cost of indexing and retrieval (the basis for search) is a fraction of the cost of inference (the basis for OpenAI’s core product).

This fundamental difference in operational structure suggests that AI companies like OpenAI may never achieve the profit margins seen in pure software businesses or Google’s search business.

Understanding and pricing for the economics of AI are further complicated by the rapid pace of technological advancement. While compute costs are generally decreasing, AI models are simultaneously growing in size and complexity.

This arms race between efficiency gains and model sophistication makes it challenging to predict longer-term cost structures. Additionally, improvements in AI quality often come at the expense of increased computation, such as allowing for recursive processing or longer “thinking” times. These enhancements, while valuable, further impact the cost of inference.

It’s crucial for businesses leveraging AI to internalize these economic realities. The cost of inference must eventually be factored into product pricing and business models. Companies need to view AI capabilities not as a one-time development cost, but as an ongoing operational expense that directly impacts their bottom line.

The Good News: Labor Is More Expensive Than Inference

While the cost of AI inference isn’t cheap, there are positive benefits that can positively impact profit margins, most notably the impact of AI on the amount of labor needed to run a business. For many companies, AI solutions will eventually replace costly human labor. Traditional human-driven tasks often incur significant expenses that are measured as a “cost per minute,” with many tasks requiring minutes or even hours of “human compute” to complete.

In contrast, trained AI models can often perform similar tasks by delivering inference based solutions at a fraction of the cost of “human compute”, drastically reducing the time and cost associated with these solutions.

This labor arbitrage between human and AI compute can lead to substantial net savings, even after accounting for the costs of AI inference.

For instance, in customer service, an AI chatbot might handle hundreds of inquiries simultaneously at a fraction of the cost of a human call center.

In legal document review, AI can process thousands of pages in minutes, a task that would take human lawyers days or weeks.

Even in creative fields like content generation or data analysis, AI is proving to be not just faster but often more cost-effective than human alternatives.

Thus, while AI inference costs do impact margins, the elimination or reduction of human labor costs in many sectors could result in a net positive effect on a company’s bottom line. This dynamic underscores the complex economic calculus that businesses must navigate in the AI era, balancing the costs of technology against the substantial savings and efficiencies it can provide.

The Bottom Line

So while AI will be a major driving force behind the companies that dominate their industries a decade from now, it also introduces new challenges to the economics of AI-centric products and services. The cost of inference represents a fundamental shift in the business model of technology companies and this cost cannot be ignored.

As the AI landscape evolves, successful companies will be those that can balance the power of AI with efficient operations and innovative pricing strategies.

The future may not hold the near-infinite margins of software’s past, but it promises a new era of value creation that will unseat many incumbents and crown new winners. It just needs to be internalized that the profit margins of the winners many more closely resemble those of cloud service providers and other resource-intensive businesses rather than those of the software companies that are so loved in today’s market by Investors because of their stunningly high margins.