Zillow's Engineering Chief: You Cannot Measure AI ROI Without a Baseline
Zillow SVP of Engineering Toby Roberts says companies that skip pre-deployment measurement cannot trust their AI returns. At VB Transform, he and Glean's CEO explained why context layers matter more than raw data.
Zillow's customer journey does not happen in a single session. Users move between mobile apps, loan officers, and real estate agents over months, sometimes years. At the VB Transform conference, Toby Roberts, Senior Vice President of Engineering at Zillow, argued that no single chatbot can serve that kind of fragmented experience. Roberts and Arvind Jain, chief executive of Glean, made a joint case for why context architecture, not raw data volume, is the defining challenge in enterprise AI.
Most organizations start their AI programs by focusing on data: lineage, governance, permissions. Zillow did the same. Roberts acknowledged that organizing data was, in his words, the straightforward part. The harder problem was building a persistent context layer, one capable of tracking where a customer stands in their home-buying journey across every digital and physical touchpoint, without losing continuity between handoffs.
To solve that, Zillow did not route all user interactions through a single general-purpose model. The company built an internal harness that draws on years of in-house machine learning and deploys smaller, task-specific models where they perform better. For internal workflows, Zillow runs thousands of Glean agents handling routine operational tasks. Jain explained that centralizing these integrations prevents teams across marketing, finance, and product from building separate, redundant connections to the same underlying databases, recovering engineering time in the process.
Cost control follows naturally from centralization. Jain outlined two techniques Zillow relies on: model routing and precomputed context. Model routing sends simpler queries to lighter, cheaper models rather than defaulting to premium frontier options every time. Precomputed context stops agents from burning unnecessary tokens reconstructing basic customer history on every call. Jain said routing queries through a structured gateway can cut token consumption by up to 50 percent, while also reducing the latency that comes with larger models assembling context from scratch.
The most transferable lesson from Zillow's approach is deceptively simple: measure before you build. Roberts said Zillow could report a 40 percent increase in developer code output with confidence precisely because the company had been tracking productivity metrics for years before any AI tools were introduced. Without that baseline, he argued, ROI claims become guesswork. Companies that skip that step have no reliable way to separate genuine gains from noise.
What This Means for African Tech
The discipline Zillow applies to measurement and context architecture speaks directly to conditions in African tech markets. With venture funding tighter and investor scrutiny higher, founders cannot afford AI deployments that look impressive in demos but fail to move auditable numbers. Establishing operational baselines before rolling out productivity tools is not a luxury; it is the only way to demonstrate that an investment worked. Zillow's 40 percent figure carried weight because it was grounded in prior data, not estimated after the fact.
The cost-reduction techniques are equally relevant. Cloud computing and data transfer remain expensive across much of the continent, and many African developers are building for users on constrained data budgets or inconsistent networks. Smaller, specialized models running through a centralized gateway offer a practical path to responsive, context-aware services without passing high API costs onto end users. A centralized context layer also makes it easier to integrate local data, regional languages, informal address formats, and other variables that generic foreign models handle poorly, without querying expensive external servers repeatedly.
The architecture Zillow built is not a shortcut; it is the kind of deliberate infrastructure that makes AI deployments defensible when someone asks for the numbers.
Source: VentureBeat
Written by
StacyAI-assisted news curation. Every story is reviewed by our editors before publication.
