Agents think in milliseconds, legacy infrastructure doesn't. LinkedIn, Walmart and Zendesk shared how they closed the gap at VB Transform 2026
Our take
The recent VB Transform 2026 conference illuminated a crucial bottleneck hindering the widespread adoption of AI agents: legacy infrastructure. It’s a point often missed in the breathless hype surrounding large language models, but as LinkedIn, Walmart, and Zendesk revealed, the real challenge isn't the models themselves, but the systems built to support human workflows struggling to keep pace with agentic speed. This isn't about model performance; it’s about architectural limitations. As Cloud Native Infrastructure Emerges as the Foundation for Trustworthy Agentic AI highlights, a shift towards cloud-native infrastructure is becoming increasingly essential for supporting the demands of AI agents, and this panel underscored that urgency. The core issue, repeatedly emphasized, is that our existing IT foundations, designed for relatively infrequent human interaction, are simply inadequate for the relentless, millisecond-driven pace of AI agents.
The specific hurdles each company faced—Kubernetes' slow container provisioning at LinkedIn, internal "citizen developer" chaos at Walmart, and data pipeline limitations at Zendesk—paint a clear picture. What’s particularly insightful is the realization that innovation can quickly outstrip infrastructure. Walmart’s experience, in particular, demonstrates the potential for rapid, decentralized experimentation, but also the subsequent need for governance and standardization. This mirrors a wider trend within organizations: empowering individual teams to leverage AI, while simultaneously maintaining control and ensuring consistency. The emphasis on robust evaluations, as championed by Ghoche, further reinforces the need for a structured approach. Using Classical ML to Empower AI Agents reminds us that building on existing ML foundations, rather than solely relying on new LLMs, can be a pragmatic and effective strategy for managing this complexity.
Ultimately, the lessons from these industry leaders converge on a few key principles. First, anticipate infrastructure needs *before* scaling agents – don’t wait for bottlenecks to appear. Second, build for modularity and independence; avoid vendor lock-in and ensure flexibility in model selection. LinkedIn's AI gateway and memory subsystem are excellent examples of how to achieve this. Third, empower users with agent harnesses, but couple that empowerment with robust monitoring and governance. This isn't about stifling innovation, but rather about channeling it effectively. The shared advice to invest in evaluations early and often is crucial, forcing teams to define success and iteratively improve performance. These approaches represent a significant shift away from simply throwing more compute power at the problem; it demands a fundamental rethinking of how we design and manage our IT ecosystems.
Looking ahead, the question isn't *if* AI agents will transform the enterprise, but *how* quickly we can adapt our infrastructure to support them. The discussions at VB Transform 2026 suggest that the focus is shifting from model architecture to operational excellence – a welcome development. As QCon AI Boston: Production AI Moves Beyond Prompts to Platforms, Harnesses, and Evals noted, the field is moving beyond simple prompting and towards building robust platforms and governance structures. Will organizations prioritize these foundational investments, or will they continue to chase the latest model advancements while neglecting the underlying infrastructure that will ultimately determine their success?
Legacy infrastructure, not the models themselves, is what's actually slowing AI agents down. That was the shared conclusion of three infrastructure leaders — from LinkedIn, Walmart, and Zendesk — at VB Transform 2026.
The panel brought together Animesh Singh, senior director of AI platform and infrastructure at LinkedIn, Desiree Gosby, SVP of corporate technology services and technology strategy at Walmart, and Sami Ghoche, VP of applied AI at Zendesk, each describing what actually broke when they moved agents from pilot to production. Each arrived at the same conclusion from a different starting point: None of the bottlenecks they hit were model problems.
What tied their answers together was a shared premise: most enterprise infrastructure was built for how humans work, not for how agents work. The gap between those two speeds is where the real engineering happened.
Gosby put it plainly when asked what she'd learned scaling agents inside Walmart's own workforce. The goal, she said, is to make sure "engineering doesn't once again become the bottleneck for what it is we're trying to do."
Where the bottleneck actually was
Each company hit a different version of the same wall: infrastructure designed for how people work doesn't hold up once agents are doing the work instead.
At LinkedIn, the first bottleneck wasn't a model, it was Kubernetes, which assumes containers spin up on demand, a process that takes seconds. Singh said that's too slow for agents. The fix was moving from on-demand provisioning to pre-provisioned pools of containers that swap agentic workloads in and out in real time.
A second, harder problem surfaced once LinkedIn let agents control their own orchestration. A five-point evaluation system looked clean, but hallucination kept showing up anyway. Singh said the issue was structural, an LLM evaluating another LLM's output shares the same failure mode as the thing it's evaluating.
"We built our own harness, our own control flow, and pushed the LLMs to the leaf instead of them orchestrating the loop," Singh said. Roughly 80% of the workflow is now scripted, deterministic code, with LLMs used only where reasoning is required, and each step's evidence is committed to disk before the system moves on.
Walmart's bottleneck came from success. An agent harness put directly into employees' hands went viral internally, and what Gosby called "citizen developers" began building their own agents to solve problems that once required a formal engineering roadmap. The upside was real innovation. The downside was duplication, dozens of overlapping agents with no coordination. The fix wasn't reining in the harness, it was building governance to spot duplication, promote the best version of an agent, and get it into production without engineering becoming a chokepoint.
Zendesk hit its bottleneck from the data side. Ghoche, who joined through Zendesk's acquisition of Forethought, which closed in March 2026, described sitting on what he called a public figure of 20 billion customer conversations in Zendesk's repository. The instinct is to hand that history to a large language model with a big context window and let it generate the agents a business needs. Ghoche said that doesn't work. "You can't really do that, so instead you have to really invest in the underlying data pipelines and all the data infrastructure that comes with that," he said.
The role of open source
On open source, all three leaders landed on a similar instinct: own what you can, and lean on frontier labs only where they still have a clear edge.
Ghoche said his own view is that most enterprises would prefer to own their models and infrastructure wherever that's possible, and that reasoning is what drives Zendesk's own approach. The exception is frontier reasoning work, where the labs still lead, though he said that slice of use cases is shrinking relative to everything else enterprises now do with AI.
LinkedIn's answer was to build two subsystems specifically for independence. The first is what the company calls an AI gateway, a single interface that every outbound call to a model runs through regardless of provider. The second component is a memory subsystem built to hold context independent of any model provider.
"Every single outbound call going to an LLM, whether it's on a public cloud or on-prem in our own data centers, follows the same semantics, the same API calls. We can quickly switch between different providers," Singh said.
Walmart built its own internal gateway to stay vendor agnostic across three workload types: fully deterministic workflows, planner-and-reasoner workflows for open-ended tasks, and a hybrid of the two. Compliance-heavy work stays deterministic by design; governance, security and evaluation run through the gateway regardless of which model is on the other end. Gosby said the choice between a frontier model and an open-weight model comes down to whichever is most effective for the specific workload, not a fixed policy.
Advice for the modernization journey
Three pieces of advice came up directly, each tied to the wall a leader had already hit.
Invest in evals before anything else. Ghoche called it the thing common to every use case, internal or customer facing.
"The thing that's common to all of these is evals. It'll force you to break the problem down, and once you have a robust set of evals, you can move a lot faster," he said,
Own your agent harness from day one. Gosby's advice was to put the AI harness directly in employees' hands early, paired with the infrastructure to monitor what it produces.
"It will unlock a huge amount of innovation," she said.
Build for model and context independence. Ensuring flexibility is critical for success.
"Build for independence, whether it's a frontier model of today versus an open source model of tomorrow," Singh said. "Keep that context within your enterprise so that you can reuse it when you ship the model or the harness tomorrow," Singh said.
Read on the original site
Open the publisher's page for the full experience