A study finds that the software environment surrounding language models affects efficiency by up to 100 times and explains more cost and performance variations than the underlying architecture.
Progress in artificial intelligence is often attributed to model training, reinforcement learning, and inference efficiency. However, recent research indicates that "scaffolding" — the software layer and context provided to a model at runtime — has a substantial and frequently underestimated impact. The study, conducted by Hans Gundlach, Zachary Brown, Jayson Lynch, and Neil Thompson, analyzes data from the Holistic Agent Leaderboard (HAL) and concludes that this structuring can alter a model's inference efficiency by up to 100 times.
The data reveals that scaffolding, also referred to as a wrapper or harness, is the program responsible for turning an AI model into an autonomous agent. The research identified that performance and cost variations between different systems are explained more by scaffolding layers than by the underlying language models themselves. In practical terms, the impact of this layer can be so decisive that, when evaluating a system, asking which scaffold was used is almost as relevant as asking which base model is in operation.
A distinctive characteristic of this technology is its inconsistency in results. Unlike many innovations in machine learning that offer generalized gains, the same scaffold can produce opposite effects depending on the model and the task. The research points out that while some models show significant performance improvements with a given software structure, others may suffer efficiency drops or be hindered by the exact same configuration.
These complex interactions between model and scaffold carry direct implications for the economics of AI agents and for how systems are evaluated. According to the study's authors, reliance on highly optimized software layers tailored to each model can act as a technical barrier. They speculate that this dynamic could act as a driver for increased market concentration in the artificial intelligence sector, favoring companies capable of developing and integrating these structures more effectively.
Scaffolding is the software layer, also known as a wrapper or harness, that provides context to an AI model at runtime. It is the program responsible for turning a language model into an autonomous agent.
A study analyzing the Holistic Agent Leaderboard (HAL) found that scaffolding can alter a model's inference efficiency by up to 100 times. It explains more cost and performance variations between systems than the underlying language models themselves.
Scaffolding requires highly optimized software layers tailored to each specific model. This creates a technical barrier that favors companies with the resources to develop and integrate these structures effectively, potentially driving market concentration in the AI sector.