natural language processing

The next chapter of AI agents demands production-ready reliability.

As enterprises embrace AI agents, they face a critical reliability challenge that underscores the need for robust infrastructure.

4 min readVentureBeat
The next chapter of AI agents demands production-ready reliability.

As enterprise AI agents move into production, organizations are increasingly facing the reliability problem that has come to define the next phase of AI integration. Many teams are realizing that the performance of large language models (LLMs) alone does not ensure success in real-world applications. Long-running AI workflows must not only survive crashes but also preserve state, recover from failures, and coordinate seamlessly across APIs and enterprise systems. This challenge echoes themes from other recent developments in technology, such as how Pinterest cut AI costs 90% by gutting a frontier model's vision layer, which showcases the necessity of optimizing existing systems rather than perpetually pushing forward with untested innovations.

Preeti Somal, Senior VP of Engineering at Temporal Technologies, highlights a crucial realization among enterprises: the need to revisit first-generation AI agent implementations. The initial focus on rapid deployment has left many organizations grappling with foundational issues, much like the early days of cloud adoption when businesses rushed to migrate workloads without adequate redesigns. This lack of foresight can lead to significant operational pitfalls, where teams are forced to rebuild agents from the ground up after experiencing failures due to inadequate architecture. As organizations begin to understand that AI is not merely a plug-and-play solution, but requires a robust infrastructure, the design of AI systems must evolve to prioritize workflow orchestration, observability, governance, and recovery.

The implications of this shift extend beyond technical specifications; they touch on the economic realities facing enterprises today. As AI becomes a strategic priority, leaders must evaluate the return on investment (ROI) associated with these systems. Costs can spiral when workflows fail, requiring reruns of entire processes, thereby driving up inference expenses and impacting customer experiences. The idea of a "deterministic spine," as articulated by Somal, provides a framework for understanding how orchestration software can support the reliability of probabilistic models, ensuring consistent execution even when faced with interruptions. This perspective is crucial as enterprises navigate the complexities of integrating AI into their existing workflows.

Looking ahead, the need for governance will become even more pronounced. As organizations seek to build standardized frameworks that balance flexibility with necessary controls, the focus will shift from merely adopting AI solutions to creating sustainable, long-term systems that enhance productivity. As seen in the healthcare example with Abridge, where workflows are complex and multifaceted, successful AI agents must be able to maintain continuity over time and withstand interruptions. This raises a significant question for enterprises: how will they ensure that their AI systems are not only innovative but also resilient and economically viable?

As organizations embark on this journey, the importance of collaboration with experts in workflow orchestration will only grow. The challenges presented by agentic AI are not merely technical hurdles; they are opportunities for enterprises to reimagine their data management practices and improve overall operational efficiency. The trend toward revisiting and refining first-generation implementations underscores a pivotal moment in the evolution of enterprise AI, encouraging organizations to build a foundation that will support the transformative potential of AI in the future. The journey is just beginning, and the successful enterprises will be those that not only adopt new technologies but also construct the robust systems that enable them to thrive.

From VentureBeat

As enterprise AI agents move into production, organizations are confronting a growing reliability problem. Many teams are discovering that LLM performance alone does not determine whether agents succeed in production. Long-running AI workflows must survive crashes, preserve state, recover from failures, manage inference costs, and coordinate across APIs, tools, and enterprise systems.

After a first wave focused on rapid deployment, organizations now need to revisit those first-generation implementations, and redesign early agent architectures around workflow orchestration, observability, governance, and recovery, said Preeti Somal, Senior VP Engineering at Temporal Technologies, during the latest AI Impact Series event in New York.

Read the original at VentureBeat