AI security

Navigating AI security and agent verification in production systems

Production AI systems don't just need to be smart, they need to be secure.

3 min readInfoQ
Navigating AI security and agent verification in production systems

The industry is right to push AI agents into production, but it is moving faster than its own safeguards. Two new InfoQ certification cohorts on AI security and agent verification acknowledge a reality many teams are only now confronting: deploying an AI system that interacts with live data or writes code into an existing codebase requires a fundamentally different discipline than training a model in isolation. This is not an abstract concern. It is the difference between a tool that augments your team and one that introduces risk you cannot measure.

The security cohort addresses decisions that are easy to postpone and expensive to get wrong, data lineage, access controls, audit trails, while the verification cohort tackles the harder problem of confirming that an agent's output is correct in a production environment. These are not separate concerns. They are two sides of the same operational question: can you trust what the system produces? A recent case study in which coding agents refactored 300,000 lines of C for roughly $4,000 showed what is possible when the codebase is well-understood and the task is bounded. 300K lines refactored for $4,000: what a C codebase taught AI agents illustrates the promise, but it also implies a precondition that most teams have not met: a codebase clean enough that an agent's changes can be validated quickly. Without that, agent verification becomes a bottleneck, not an enabler.

The practical takeaway is straightforward. If your team is planning to deploy AI agents that modify code, you need a verification strategy before you need an agent. That means test suites that actually catch regressions, not just suites that pass. Beyond Green: Understanding True Test Suite Efficacy makes the point that a green build can mean nothing if the tests do not exercise the right behaviors. An agent that passes a weak test suite is not verified; it is merely unchecked. The certification cohorts are correct to center verification, but verification only works when the target system is testable.

The security side is less technical and more structural. Production AI systems introduce new attack surfaces, prompt injection, data exfiltration through model outputs, unauthorized access via agent permissions, that traditional security reviews miss. Teams that treat AI security as an extension of application security will miss the gaps. The certification content is a signal that the industry is beginning to formalize what safe operation looks like, but formalization without adoption changes nothing. The open question is whether most organizations will invest in these practices before an incident forces the issue. Watch which teams treat certification as a baseline rather than an accolade. That is where the real progress will happen.

From InfoQ

A look at two InfoQ online certification cohorts covering security and privacy decisions in production AI systems and the verification needed when coding agents work in existing codebases.

Read the original at InfoQ