Human in the Loop

Human-in-the-Loop Without Killing Throughput

Traditional Human-in-the-Loop (HITL) processes often create a bottleneck, slowing down AI agent throughput. Our approach redefines HITL, intelligently routing human attention only where it’s genuinely needed, preserving efficiency. We detail how we shifted from reviewing every agent action to a targeted system, dramatically improving both accuracy and speed. Explore the strategies that unlock scalable, high-quality AI oversight. For deeper insights into the broader AI landscape, see "Open-weight AI companies are the Valley’s hottest acquisition targets.”

Cloudflare Cuts Astro Github Issues by 85% with AI Agents

Cloudflare significantly enhanced developer productivity by leveraging AI agents to manage GitHub issues, achieving an 85% reduction in processing time. This innovative application of agentic AI within GitHub Actions streamlines issue triage, automating workflows and accelerating software engineering cycles. Utilizing Cloudflare Workers and Flue, the system incorporates a “human-in-the-loop” approach, ensuring quality while maximizing efficiency.

AI coding agents are blowing through budgets — Replit, Kilo Code, and Symbotic explain how they're managing it

The rise of AI coding agents presents a compelling evolution for development teams, though it's also sparking crucial conversations around budget management and responsible implementation. Leaders at Replit, Kilo Code, and Symbotic are navigating this shift, recognizing that while agents excel in greenfield projects, human oversight remains vital for complex brownfield environments. Kilo Code, for example, now supports over 500 models, demonstrating a move towards flexible, multi-model architectures—a strategy increasingly critical for optimizing both performance and cost.

Looking for the right pipeline to convert academic textbook figures into interactive/editable assets [R]

The agent evaluation gap: Enterprise AI organizations have a reality-alignment problem, not a coverage problem — and most are shipping to production anyway

Enterprise AI organizations face a critical reality-alignment problem: an “evaluation gap” where increasing agent autonomy outpaces trust in the evaluations meant to govern it. A recent VentureBeat Pulse Research survey of 157 enterprises reveals that half have already deployed an agent that passed internal evaluations but then failed a customer. Despite this, two-thirds are moving toward fully automated deployments—highlighting a concerning disconnect. This research underscores the urgent need for evaluations that accurately reflect real-world outcomes, not just passing scores.

The agent evaluation gap: Enterprise AI organizations have a reality-alignment problem, not a coverage problem — and most are shipping to production anyway

Enterprise AI organizations face a critical reality-alignment problem: an “evaluation gap” where increasing agent autonomy outpaces trust in the evaluations meant to govern it. A recent VentureBeat Pulse Research survey of 157 enterprises reveals that half have already deployed an agent that passed internal evaluations but subsequently failed a customer. Only 5% fully trust automated evaluation, citing a key weakness – evaluations often don't reflect real-world outcomes. Despite this, two-thirds are moving toward fully automated deployments, highlighting a pressing need for more reliable assurance.

Google's Genkit Ships Agents API with Detached Turns and Human-in-the-Loop for TypeScript and Go

Google’s Genkit Agents API, now in preview for TypeScript and Go, represents a significant advancement in AI agent development. This framework streamlines complex workflows by packaging message history, tool loops, streaming, and state persistence into a unified `chat()` interface. Notably, detached turns enable continuous agent operation even after client disconnection. Furthermore, interruptible tools introduce a crucial human-in-the-loop capability, ensuring control and security through anti-forgery validation when resuming tasks.