At VB Transform 2026, Zillow's engineering chief said AI ROI numbers only hold up if you measure before you build
Our take
Zillow's recent experience, as shared at VB Transform 2026, underscores a critical, often overlooked, truth about implementing AI within complex enterprise systems: context is king. The company’s journey highlights that simply amassing data, a common initial focus for many organizations, isn’t sufficient. As Zillow discovered, the real challenge lies in building systems that retain and utilize context across disparate touchpoints within a customer’s journey. This resonates strongly with the discussions around agentic AI, which require not just powerful models, but a robust understanding of the user's history and current needs – a capability that a single, isolated chatbot fundamentally lacks. For those seeking to explore the mechanics of building high-performance agentic programming, a valuable starting point can be found in A Beginner’s Guide to Setting Up Claude Code for High Performance Agentic Programming. The underlying principle here is that AI’s true power is unlocked when it can weave together fragmented interactions into a cohesive narrative.
The emphasis on measuring AI ROI *before* implementation is a particularly insightful takeaway. Zillow’s ability to attribute a 40% increase in shipped code to AI adoption stems directly from establishing baseline DORA metrics years prior. This proactive approach demonstrates a mature understanding of how to evaluate AI’s impact, moving beyond retrospective analysis to a data-driven assessment of effectiveness. It’s a powerful reminder that any AI initiative must be grounded in measurable objectives, and that success isn’t defined by the technology itself, but by its contribution to tangible business outcomes. This contrasts starkly with the often-hyped, but rarely substantiated, claims of "revolutionary" AI solutions that frequently fail to deliver on their promises. The conversation also touches on the complexities of open-weight models, a point explored in China's K3 Model Reveals the Problem With Open Weights, illustrating that simply having access to a model isn’t a guarantee of success—robust infrastructure and thoughtful integration are equally vital.
What Zillow and Glean's approach demonstrates is a pragmatic, task-specific architecture, leaning into smaller, fine-tuned models rather than relying on monolithic, general-purpose ones. This strategy, coupled with Glean’s centralized integration hub, provides a compelling framework for managing AI costs and complexity within an enterprise. The focus on precomputed context and model routing to minimize token consumption is particularly noteworthy, highlighting a shift towards optimizing AI spend rather than simply chasing increased capability. This resonates with a growing awareness that sustainable AI adoption requires a keen eye on resource utilization and a willingness to prioritize efficiency. Furthermore, acknowledging the challenge of ensuring permission inheritance for regulated data, even with a context-aware platform, underscores the necessity of layered security measures and ongoing compliance checks—a crucial consideration for any organization handling sensitive information. [Deep learning tackles single-cell analysis – A survey of deep learning for scRNA-seq analysis [R]](/post/deep-learning-tackles-single-cell-analysis-a-survey-of-deep-cmrt6doaf03k7djxxw3a1m0iq) provides a different but relevant example of the need for careful architectural choices when deploying complex AI solutions.
Ultimately, Zillow's experience reinforces the idea that AI isn’t a standalone technology but rather a component of a broader ecosystem. The true value lies in seamlessly integrating AI into existing workflows, providing context, and optimizing resource utilization. As enterprises continue to grapple with the complexities of agentic AI, the key takeaway is clear: focus on building robust, context-aware architectures, establish clear measurement baselines, and prioritize efficiency – not just innovation for its own sake. The question now becomes: how will organizations proactively address the "hidden costs" of duplicated integration work, and will centralized platforms like Glean become the norm for managing increasingly complex AI deployments across various departments?
Zillow, the real estate technology company, doesn't get one conversation with its customers. They move from a phone screen to a loan officer to a real estate agent, sometimes over months or years, and expect the context to follow them. A single chatbot could never carry that thread.
At VB Transform 2026, Zillow SVP of Engineering Toby Roberts and Glean co-founder and CEO Arvind Jain described how they built AI architecture meant to carry context across that entire journey — and why context, not raw data, turned out to be the harder problem to solve. Zillow's products touch roughly 80% of U.S. real estate transactions each year, and the company has been using AI long before ChatGPT existed.
"We pretty quickly identified that we were going to need a persistent context layer that was going to meet our customers and the professionals wherever they were," Roberts said.
Data was never the hard part
Roberts said Zillow's AI effort started where most enterprise AI efforts start, with the data itself.
"We started with a large push around making sure our data did have the right foundation," Roberts said. That meant a data mesh approach, clear data lineage and a governance structure with permissions and identity attached to the data itself.
None of that turned out to be the hard problem. The hard problem was building something that remembered where a customer was in their journey and carried that forward, no matter which surface they showed up on next.
"This context layer has to live to be able to support you where you are at any given point in your journey," Roberts said. Zillow chose to own that layer itself rather than depend on a single external chat interface, a decision Roberts said the team reached quickly once it looked at the shape of a real transaction rather than a single conversation.
Why Zillow built its own architecture, and where Glean fits into it
Zillow built its own harness rather than route customers through a single model API. The team drew on 20 years of machine learning history behind products like Zestimate, leaning into smaller, task-specific fine-tuned models instead of one general-purpose model.
Internally, that harness runs alongside Glean. Roberts said Zillow now has thousands of Glean agents in production, handling repetitive tasks with tens of thousands of executions across the company. Glean's pitch, per Jain, is centralizing that integration work once, through the Glean MCP gateway, rather than letting finance, legal and marketing each rebuild their own connections to the same systems.
That centralization is also a cost lever. Jain pointed to two mechanisms: model routing, which sends most tasks to smaller, cheaper models instead of defaulting to frontier models, and precomputed context, which avoids an agent burning tokens assembling its own context from scratch.
"Claude is also very slow because the first part of assembling that context actually takes forever," Jain said. Routing that request through Glean instead, he said, can cut token consumption by as much as half.
What Zillow and Glean's approach means for enterprises
Across data, cost and permissions, the session offered a few practical takeaways for enterprises building agentic AI on their own systems.
Build the measurement baseline before the AI push, not after. Roberts said Zillow's ability to credibly attribute a 40% increase in shipped code to AI adoption rests on a DORA metrics baseline the team put in place years earlier, not on the AI rollout itself.
Centralize context once instead of letting every team rebuild it. Jain's core argument for Glean's platform is that duplicated integration work across finance, legal and marketing teams is a hidden cost most enterprises haven't accounted for.
Don't assume permission inheritance is enough for regulated data. Even with a permissions-aware context platform in place, Zillow layered hard rules and a standing compliance check on top for its most sensitive categories, rather than trusting the architecture to handle it automatically.
Treat context as a cost lever, not just a capability. Model routing and precomputed context were the two mechanisms Jain pointed to for cutting AI spend, both aimed at reducing wasted token consumption rather than adding new capability.
"Models by themselves are not enough to bring automation with AI inside your enterprise," Jain said. "You do have to connect it with your enterprise context."
Read on the original site
Open the publisher's page for the full experience