You're probably using 10x more AI than you need to.

In 2019, two researchers at MIT's Computer Science and Artificial Intelligence Laboratory published a paper that most enterprise AI buyers have never read. They should.

Jonathan Frankle and Michael Carbin demonstrated something that runs counter to how most organizations think about AI: you don't need the whole network to get the same result. In fact, the whole network might be slowing you down.

Standard neural networks are trained with hundreds of millions (sometimes billions) of parameters. Conventional wisdom held that you needed all of them to achieve the model's accuracy. Pruning techniques had shown you could remove parameters after training without significant performance loss. But nobody had cracked how to identify, in advance, which parts of a network were actually doing the work.

Frankle and Carbin found the answer. They called it the Lottery Ticket Hypothesis.

Inside every large, randomly-initialized neural network, there exist smaller subnetworks ("winning tickets") whose initial weights are configured in a way that makes them exceptionally capable of learning. When these subnetworks are trained in isolation, they reach test accuracy comparable to the full network. They train faster. They perform better. And they do it at 10–20% of the size of the original architecture.

The dense network isn't necessary. It's a lottery. And most of the tickets are losing ones.

The Lottery Ticket Hypothesis was published for feed-forward networks trained on benchmark datasets. But the principle it established, that most of a model's compute is doing very little useful work, has carried forward into the architecture of modern AI deployment in ways that have direct commercial consequences.

The enterprise AI cost problem is not abstract. Organizations running large language models at scale are paying for every token processed by every parameter in a model that is, in many cases, dramatically oversized for the task at hand. A frontier model queried to extract a structured field from a document is like hiring a principal engineer to file your expenses. The capability is there. The cost is unjustifiable.

What Frankle and Carbin's work points toward, and what the subsequent decade of research in model compression, distillation, and sparse architectures has confirmed, is that the right-sized model for most enterprise tasks is significantly smaller than the one most organizations are currently paying for.

The practical application of this research for enterprise AI is routing. Not every query needs a frontier model. Not every task requires GPT-4, Claude Opus, or Gemini Ultra. In fact, for the majority of structured enterprise workflows (data extraction, classification, summarization of known document types, validation against fixed schemas) a smaller, fine-tuned, locally-deployed model will match the frontier model's accuracy at a fraction of the inference cost.

The architecture looks like this: a routing layer sits in front of your model stack. Incoming queries are classified by complexity and task type. Simple, well-defined tasks route to a local or smaller model. Complex, judgment-intensive tasks escalate to a frontier model. The frontier model is used only when it is genuinely needed.

Done well, this architecture can reduce compute spend by 70–90% on the right workflows, without any measurable degradation in output quality. That is not a theoretical estimate. It is what organizations deploying blended architectures are already achieving.

The implications for enterprise AI procurement and architecture are significant and largely ignored in the vendor conversations most organizations are having.

First, the default is wrong. Most AI implementations default to the largest, most capable model available, because it is the easiest choice, because it requires no architectural judgment, and because the vendor has no incentive to suggest otherwise. Paying per token on a frontier model is a good business for the model provider. It is a poor one for the customer.

Second, task analysis precedes model selection. The first architectural question for any enterprise AI system should not be "which model?" It should be "what is this task actually requiring, and what is the minimum model capability that satisfies that requirement?" That question requires engineering judgment. It cannot be answered by a sales team.

Third, local deployment is undervalued. For tasks where data sensitivity, latency, or cost predictability is a requirement, locally-deployed smaller models, running on your own infrastructure, not a third-party API, are increasingly viable. The Lottery Ticket Hypothesis is one of the intellectual foundations for why: small, well-configured networks can match large ones on specific, bounded tasks.

Fourth, the savings compound. An organization running 10 million inference calls per month on a frontier model at full cost, versus a routed architecture that sends 80% of those calls to a local model, is not making a marginal efficiency improvement. It is changing the commercial viability of AI entirely.

We focus closely on TCO because our clients' CFOs are right to demand it. AI that costs more than it saves is not a technology investment, it's an experiment being paid for by the business.

The Lottery Ticket Hypothesis is one of the reasons we build blended architectures: combining traditional ML, fine-tuned smaller models, and frontier model access in a routing layer designed around the actual task distribution of each client's workflows. We don't recommend the biggest model. We identify the right model for each task, and we build the architecture that gets each query to the right place automatically.

That is how you get to 70–90% compute reduction without sacrificing accuracy. That is how you make AI commercially viable, not as a pilot, not as an experiment, but as infrastructure that earns its place in a business permanently.

The lottery ticket was always inside the network. Most organizations are still paying for the whole ticket book.

Source: Frankle, Jonathan and Michael Carbin. "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks." 7th International Conference on Learning Representations, May 2019. MIT Computer Science and Artificial Intelligence Laboratory.