- Beyond RAG: Agentic AI demands a new hybrid knowledge layer
The vector database category is undergoing a shift in response to the needs of agentic AI.
The retrieval-augmented generation (RAG)-to-vector database pipeline doesn't cut it anymore; agentic AI requires a different approach that incorporates context. VentureBeat's Q1 2026 Pulse survey underscores this trend: Every standalone vector database is losing adoption share, while hybrid retrieval intent has tripled to 33.3%, the fastest-growing strategic position in the dataset.
Vector database pioneer Pinecone recognizes this and is pivoting to meet the specific needs of agentic AI.
The company today announced Nexus, which it positions as a knowledge engine rather than an improvement on retrieval. Nexus introduces a context compiler that converts raw enterprise data into persistent, task-specific knowledge artifacts before agents query them, and a composable retriever that serves those artifacts with field-level citations and deterministic conflict resolution.
Alongside Nexus, Pinecone is releasing KnowQL, a declarative query language that gives agents a vocabulary to specify output shape, confidence requirements, and latency budgets. In Pinecone's own internal benchmark, one financial analysis task that previously consumed 2.8 million tokens was completed by Nexus with just 4,000. This represents a 98% reduction, although the company has not yet validated it in customer production deployments. Nexus is in early access starting today.
"RAG was built for human users," Pinecone CEO Ash Ashutosh told VentureBeat. "Nexus was built for agentic users, because their language is very different. The responses they expect are very different. The task that an agent is assigned to do is very different from what a chatbot is supposed to do."
Why RAG was never built for what agents actually do
RAG encompasses one query, one response, and a person in the loop to interpret the result. But agents work differently. They are assigned tasks, not questions — and completing these requires assembling context from multiple sources, resolving conflicts, tracking what has already been retrieved, and deciding what to query next.
The distinction matters. A RAG pipeline retrieves documents and hands them to a model at inference time. Each agent session starts cold, with no compiled understanding of the enterprise data estate — which tables relate to which, which sources are authoritative for which questions, and which formats an agent downstream will actually be able to consume. Every session re-discovers that from scratch.
"At the heart of all this stuff was a very simple problem," Ashutosh said. "You're asking agents — machines — to work on systems and data that was designed for humans."
Pinecone estimates that 85% of agent compute effort goes to the re-discovery cycle rather than task completion. The downstream effects compound: unpredictable latency, runaway token costs, and non-deterministic results. Run the same task twice against the same data, and an agent may return different answers with no record of which sources drove either result. For enterprises where auditability is a compliance requirement, that is a structural disqualifier, not a tuning problem.
What Nexus is and how it works
Nexus moves reasoning work from inference time to compilation time. In a conventional RAG pipeline, the reasoning required to interpret, contextualize, and structure knowledge happens at the moment an agent queries — every session, every time, burning tokens on work that could have been done in advance. But Nexus reasons just once during a compilation stage that runs before any agent query, then stores the result as a reusable knowledge artifact. The agent receives structured, task-ready context rather than raw documents to interpret on the fly.
The architecture Pinecone is shipping has three distinct components, each addressing a different layer of the agent retrieval problem.
Context compiler. Nexus takes raw source data and a task specification and builds specialized knowledge artifacts — structured, task-optimized representations that agents consume directly without interpretation overhead. The same underlying data estate produces different artifacts for different agents: a sales agent gets deal context synthesized from CRM and call records, a finance agent gets revenue context linking contracts to billing schedules. Artifacts are persistent and reused across agent sessions, not regenerated at inference time.
Composable retriever. Compiled artifacts are served at query time with typed fields, per-field citations with confidence levels, and deterministic conflict resolution. Output is shaped to match the agent's specified format rather than returned as raw text for the agent to re-parse.
KnowQL. Pinecone describes this as the first declarative query language designed for agents rather than humans. Six primitives — intent, filter, provenance, output shape, confidence, and budget — allow agents to specify structured responses and source grounding and latency envelopes in a single interface. Ashutosh compared the structural gap that KnowQL fills to what SQL did for relational databases: Before a standard interface existed, every application built its own data access layer from scratch.
The relationship between Nexus and Pinecone's underlying vector database is additive. The context compiler produces knowledge artifacts that are indexed and stored in the vector database; the compilation layer shapes and serves knowledge; the vector layer handles storage, retrieval speed, and scale.
"The vectors are still stored and managed by the Pinecone vector database," Ashutosh said.
What analysts make of the architectural claim
Moving reasoning upstream from inference to a compilation stage is not a novel concept — ontologies, data catalogs, and semantic layers have pursued versions of it for years. What has changed is the ability to do this at scale without dedicated engineering teams for every domain. That is the specific argument Nexus is making, and it is where analysts see the genuine advance.
Stephanie Walter, practice leader for AI stack at HyperFRAME Research, told VentureBeat that Nexus is directionally important because it shifts knowledge work from runtime chaos to pre-compiled structure. She stressed, however, that it is an evolution of RAG architecture, not a complete reinvention.
"The real innovation isn't the idea itself, but the productization of knowledge compilation as a first-class infrastructure layer," Walter said. "If Pinecone can operationalize that reliably, it becomes meaningful infrastructure, not just another RAG tuning trick."
The technical mechanism behind that claim is what Gartner distinguished VP analyst Arun Chandrasekaran called the meaningful architectural distinction.
"Unlike traditional RAG, which relies on pure semantic search at runtime, architectural compilation embeds structural logic into the metadata layer, which can boost time to response and provide better reasoning," Chandrasekaran told VentureBeat. "This is an important leap from simple retrieval to enhanced reasoning, allowing agents to navigate enterprise schemas and acquire better memory for contextualization."
The competitive landscape
Multiple vendors acknowledge that a vector database and traditional RAG are not enough for agentic AI.
Microsoft has extended its FabricIQ technology to provide semantic context for agentic AI. Google recently announced its Agentic Data Cloud as an approach to help solve the same issues. There are also standalone contextual memory technologies, like hindsight, that provide yet another option for users.
But analysts are less focused on the feature comparison than on what buyers should actually be evaluating.
"The agentic AI stack is fragmenting into dozens of features, but enterprise buyers shouldn't chase features," Walter said. "They should chase control: cost control, governance control, and security control."
Most enterprise failures in agentic AI, she argued, will not be technical. They will be operational — tied to cost overruns, governance gaps, and security discipline.
The capability bar goes beyond retrieval speed.
"The true differentiator is deterministic grounding," Chandrasekaran said, pointing to techniques like knowledge graphs that ensure agents understand structural relationships within enterprise data rather than returning surface-level matches. Interoperability is a related consideration: Standards like model context protocol (MCP) matter for connecting agents to legacy data sources without creating new dependencies.
What this means for enterprises
RAG and standalone vector databases were built for a different era. Agentic workloads are exposing the limits of both.
The retrieval cost problem is architectural
Teams running complex agentic workloads on conventional RAG pipelines are burning tokens at inference time on work that could be done in advance — interpreting, contextualizing, and structuring knowledge, every session, from scratch. That is a design problem. Tuning the retrieval layer will not fix it. The question for data engineering teams is whether their current stack is structurally capable of pre-compiling knowledge for specific agent tasks, or whether it was built for a human user who never needed that capability.
Governance is what separates a pilot from a production deployment
The capabilities that determine whether agentic AI gets approved for enterprise use are not performance metrics.
"The real enterprise value proposition isn't just faster retrieval, but governed knowledge pipelines," Walter said. "Those are the capabilities that turn agentic AI from an experiment into something finance and risk teams will actually approve."
The budget has shifted
VentureBeat's Q1 Pulse data shows that retrieval optimization investment rose to 28.9% in March, overtaking evaluation spending for the first time in the quarter. Enterprises have finished measuring their retrieval problems. They are now spending to fix them.
"The future of agentic AI won't be decided by who has the longest context window," Walter said. "It will be decided by who can operationalize trusted knowledge at scale without blowing up cost or governance."
- Enterprise RAG shifts focus from adding retrieval layers to rebuilding them
Something shifted in enterprise RAG in Q1 2026. VB Pulse data spanning January through March tells a consistent story: the market stopped adding retrieval layers and started fixing the ones it already has. Call it the retrieval rebuild.
The survey covered three consecutive monthly waves from organizations with 100 or more employees, with between 45 and 58 qualified respondents per month across platform adoption, buyer intent, architecture outlook and evaluation criteria. The data should be treated as directional.
Enterprise intent to adopt hybrid retrieval tripled from 10.3% to 33.3% in a single quarter — even as 22% of qualified enterprise respondents reported having no production RAG systems at all. For data engineers and enterprise architects building agentic AI infrastructure, the data reveals a market in active transition: the RAG architecture most enterprises built to scale is not the one they expect to run by year-end.
Hybrid retrieval has become the consensus enterprise strategy. Unlike single-method RAG pipelines that rely on vector similarity alone, hybrid retrieval combines dense embeddings with sparse keyword search and reranking layers, trading simplicity for the retrieval accuracy and access control that production agentic workloads require.
The standalone vector database category is under pressure. Weaviate, Milvus, Pinecone and Qdrant each lost adoption share across the quarter in the VB Pulse data. Custom stacks and provider-native retrieval are absorbing their displaced share.
A growing minority of enterprises are stepping back from RAG altogether — a signal that the market's maturity narrative has meaningful exceptions.
Organizations that went wide on RAG in 2025 are hitting the same failure point: the architecture built for document retrieval does not hold at agentic scale.
Enterprises that scaled RAG fast are now paying to rebuild it
The two largest intent movements in Q1 are directly connected — enterprises confronting retrieval quality problems at scale, and hybrid retrieval emerging as the consensus answer.
Investment priorities shifted in parallel. Evaluation and relevance testing led budget intent in January at 32.8% and fell to 15.6% by March. Retrieval optimization moved in the opposite direction, from 19.0% to 28.9% — overtaking evaluation as the top growth investment area for the first time.
Steven Dickens, vice president and practice lead at HyperFRAME Research, described the operational burden enterprise data teams are facing in a VentureBeat interview in March on Oracle's agentic AI data stack. "Data teams are exhausted by fragmentation fatigue," Dickens said. "Managing a separate vector store, graph database and relational system just to power one agent is a DevOps nightmare."
That fatigue shows directly in the platform data. The custom stack rise to 35.6% is not a rejection of managed retrieval — many organizations run both. It is a consolidation response from engineering teams that have hit the limits of assembling too many components.
Not every enterprise has made it that far. The VB Pulse data includes a signal that complicates the market's overall growth narrative: 22.2% of qualified respondents reported no production RAG by March, up from 8.6% in January. The report attributes this cohort to organizations that have "not yet committed to any retrieval infrastructure, or have paused programs" — concentrated in Healthcare, Education and Government, the same sectors showing the highest rates of flat budgets.
Standalone vector databases are losing the adoption argument but winning the reliability one
Recent reporting by VentureBeat illustrates why the dedicated retrieval layer still matters in production.
Two enterprises building on Qdrant show why purpose-built vector infrastructure still wins in production.
&AI builds patent litigation infrastructure and runs semantic search across hundreds of millions of documents. Grounding every result in a real source document is not optional — patent attorneys will not act on AI-generated text. That requirement makes the architectural choice clear.
"The agent is the interface," Herbie Turner, &AI's founder and CTO, told VentureBeat in March. "The vector database is the ground truth."
GlassDollar, a startup that helps Siemens and Mahle evaluate startups, runs an agentic retrieval pattern across a corpus approaching 10 million indexed documents. A single user prompt fans out into multiple parallel queries, each retrieving candidates from a different angle before results are combined and re-ranked. That query volume and precision requirement is what drove the choice of purpose-built vector infrastructure.
"We measure success by recall," Kamen Kanev, GlassDollar's head of product, told VentureBeat in March. "If the best companies aren't in the results, nothing else matters. The user loses trust."
The VB Pulse data shows that framing — retrieval as ground truth rather than feature — is gaining traction across the broader enterprise market, even as standalone vector database adoption declines.
Why enterprises say they need a dedicated vector layer shifted significantly across Q1. In January the top reasons were access control complexity (20.7%) and retrieval precision (19.0%). By March, operational reliability at scale had surged to 31.1% — more than doubling and overtaking everything else. Enterprises are no longer keeping vector infrastructure primarily for precision. They are keeping it because it is the part of the stack they can rely on when query volumes scale.
How enterprises are redefining what good retrieval means
How enterprises judge their retrieval systems shifted notably across Q1 — and the direction of that shift points to a market getting more sophisticated about what good retrieval actually means.
In January, response correctness dominated evaluation criteria at 67.2% — far above anything else. By March, response correctness (53.3%), retrieval accuracy (53.3%) and answer relevance (53.3%) had converged exactly. Getting the right answer is no longer enough if it came from the wrong document or missed the context of the question.
Answer relevance was the only criterion that rose across the quarter, gaining five percentage points. It is also the hardest to measure — whether the retrieved context is actually the right context for that specific question requires purpose-built evaluation infrastructure, not just pass-or-fail correctness checks. Its rise signals that a meaningful share of enterprise buyers have moved past basic RAG testing entirely.
The market's verdict: RAG isn't dead. The original architecture is
The "RAG is dead" narrative had real momentum heading into 2026. It rested on two claims. The first: that long-context windows — models capable of processing hundreds of thousands of tokens in a single prompt — would make dedicated retrieval unnecessary. The second: that agentic memory systems, which store what an agent learns across sessions rather than retrieving it fresh each time, would absorb the knowledge access problem entirely.
The VB Pulse data is the enterprise market's answer to the first claim. The long-context-as-dominant-architecture position collapsed from 15.5% in January to 3.5% in February before partially recovering to 6.7% in March. January's sample was heavily weighted toward Technology and Software respondents — the segment most exposed to long-context model announcements in late 2025. As the sample diversified, the position evaporated.
On the memory question, Jonathan Frankle, chief AI scientist at Databricks, framed the architecture clearly in a March interview with VentureBeat: a vector database with millions of entries sits at the base of the agentic memory stack, too large to fit in context. The LLM context window sits at the top. Between them, new caching and compression layers are emerging — but none of them replace the retrieval layer at the base. New agentic memory systems like Hindsight, developed by Vectorize, and observational memory approaches like those in the Mastra framework address session continuity and agent context over time — a different problem than high-recall search across millions of changing enterprise documents.
The most consequential signal: the share of respondents not expecting large-scale RAG deployments by year-end grew from 3.4% to 15.6% — nearly 5x. That is not a verdict against retrieval. It is a verdict against the retrieval architecture most enterprises built first.
The retrieval rebuild is not optional
The retrieval rebuild is the cost of scaling RAG without first deciding what architecture could actually support it.
If your organization is among the 43.1% that entered Q1 planning to expand RAG into more workflows, the VB Pulse data suggests that plan has already changed for many of your peers — and may need to change for you. Hybrid retrieval is the consensus destination. Custom stack growth to 35.6% reflects teams building retrieval infrastructure around requirements that off-the-shelf products do not fully address.
RAG is not dead. The architecture most enterprises used to implement it is. The data suggests the rebuild is not a future decision. For 33% of enterprises, the rebuild is already the stated priority.
- Two paths emerge for taming AI agents in production.
The era of enterprises stitching together prompt chains and shadow agents is nearing its end as more options for orchestrating complex multi-agent systems emerge. As organizations move AI agents into production, the question remains: "how will we manage them?"
Google and Amazon Web Services offer fundamentally different answers, illustrating a split in the AI stack. Google’s approach is to run agentic management on the system layer, while AWS’s harness method sets up in the execution layer.
The debate on how to manage and control gained new energy this past month as competing companies released or updated their agent builder platforms—Anthropic with the new Claude Managed Agents and OpenAI with enhancements to the Agents SDK—giving developer teams options for managing agents.
AWS with new capabilities added to Bedrock AgentCore is optimizing for velocity—relying on harnesses to bring agents to product faster—while still offering identity and tool management.
Meanwhile, Google’s Gemini Enterprise adopts a governance-focused approach using a Kubernetes-style control plane. Each method offers a glimpse into how agents move from short-burst task helpers to longer-running entities within a workflow.
Upgrades and umbrellas
To understand where each company stands, here’s what’s actually new.
Google released a new version of Gemini Enterprise, bringing its enterprise AI agent offerings—Gemini Enterprise Platform and Gemini Enterprise Application—under one umbrella.
The company has rebranded Vertex AI as Gemini Enterprise Platform, though it insists that, aside from the name change and new features, it’s still fundamentally the same interface.
“We want to provide a platform and a front door for companies to have access to all the AI systems and tools that Google provides,” Maryam Gholami, senior director, product management for Gemini Enterprise, told VentureBeat in an interview. “The way you can think about it is that the Gemini Enterprise Application is built on top of the Gemini Enterprise Agent Platform, and the security and governance tools are all provided for free as part of Gemini Enterprise Application subscription.”
On the other hand, AWS added a new managed agent harness to Bedrock Agentcore. The company said in a press release shared with VentureBeat that the harness “replaces upfront build with a config-based starting point powered by Strands Agents, AWS’s open source agent framework.”
Users define what the agent does, the model it uses and the tools it calls, and AgentCore does the work to stitch all of that together to run the agent.
Agents are now becoming systems
The shift toward stateful, long-running autonomous agents has forced a rethink of how AI systems behave. As agents move from short-lived tasks to long-running workflows, a new class of failure is emerging: state drift.
As agents continue operating, they accumulate state—memory, too, responses and evolving context. Over time, that state becomes outdated. Data sources change, or tools can return conflicting responses. But the agent becomes more vulnerable to inconsistencies and becomes less truthful.
Agent reliability becomes a systems problem, and managing that drift may need more than faster execution; it may require visibility and control.
It’s this failure point that platforms like Gemini Enterprise and AgentCore try to prevent.
Though this shift is already happening, Gholami admitted that customers will dictate how they want to run and control any long-running agent.
“We are going to learn a lot from customers where they would be using long-running agents, where they just assign a task to these autonomous agents to just go ahead and do,” Gholami said. “Of course, there are tricks and balances to get right and the agent may come back and ask for more input.”
The new AI stack
What’s becoming increasingly clear is that the AI stack is separating into distinct layers, solving different problems.
AWS and, to a certain extent, Anthropic and OpenAI, optimize for faster deployment. Claude Managed Agents abstracts much of the backend work for standing up an agent, while the Agents SDK now includes support for sandboxes and a ready-made harness. These approaches aim to lower the barrier to getting agents up and running.
Google offers a centralized control panel to manage identity, enforce policies and monitor long-running behaviors.
Enterprises likely need both.
As some practitioners see it, their businesses have to have a serious conversation on how much risk they are willing to take.
“The main takeaway for enterprise technology leaders considering these technologies at the moment may be formulated this way: while the agent harness vs. runtime question is often perceived as build vs. buy, this is primarily a matter of risk management. If you can afford to run your agents through a third-party runtime because they do not affect your revenue streams, that is okay. On the contrary, in the context of more critical processes, the latter option will be the only one to consider from a business perspective,” Rafael Sarim Oezdemir, head of growth at EZContacts, told VentureBeat in an email.
Iterating quickly lets teams experiment and discover what agents can do, while centralized control adds a layer of trust. What enterprises need is to ensure they are not locked into systems designed purely for a single way of executing agents.
