Software engineers' new job isn't writing code — it's designing the boundaries AI agents can't break
Our take
The shift described in this article – the diminishing role of syntax-level coding and the rise of AI agents in software development – isn't just a technological curiosity; it’s a fundamental re-evaluation of the software engineer’s role. The ease with which agents can now generate initial implementations of complex systems, as evidenced by tools like Cursor and Claude Code, signals a profound change in the bottleneck of data platform development. We’ve seen this trend accelerate rapidly, and it’s forcing us to confront a core question: what remains for the human engineer when the AI can write the code? It’s a question that echoes throughout the industry, as highlighted in related discussions like "Your LLM Can Return Perfect JSON and Still Be Wrong"[/post/your-llm-can-return-perfect-json-and-still-be-wrong-cmthjp2by0wdzmi9zo07km0n4], which underscores the importance of understanding the broader context and data quality even when outputs appear technically correct, and "Identity and permissions aren’t enough to govern AI agent behavior"[/post/identity-and-permissions-aren-t-enough-to-govern-ai-agent-be-cmthjowtc0wdhmi9zza8kim5r], emphasizing the necessity of more robust governance strategies beyond basic access controls as these agents gain autonomy.
The analogy to thermodynamics, specifically the concept of entropy, is particularly insightful. The article rightly points out that agents, left unchecked, accumulate “operational entropy” – stale assumptions, unresolved dependencies, and conflicting details that lead them away from a correct outcome. This isn't a flaw in the agents themselves, but a consequence of the inherent complexity of enterprise systems. The real value, then, isn’t in the agent’s ability to generate code, but in the engineer’s ability to design the boundaries and feedback loops that prevent that entropy from spiraling out of control. This shift moves the focus from *creating* logic to *curating* it – ensuring that the agent’s efforts align with the desired outcome and that errors are detectable and recoverable. The three-body problem analogy further illustrates this point; the inherent instability of complex systems demands constant vigilance and the establishment of clear constraints.
The implications for the future of software engineering are significant. The days of engineers spending the bulk of their time writing boilerplate code are likely numbered. Instead, their expertise will be increasingly focused on defining semantic data contracts, designing robust APIs, and building deterministic state machines – the very structures that allow agents to operate effectively and reliably. This doesn’t diminish the role of the engineer; it elevates it. It requires a deeper understanding of system architecture, data governance, and the interplay between different components. The ability to anticipate potential failure points and design systems that are resilient to change will become paramount. We're seeing this need for more robust infrastructure highlighted in discussions about optimizing LLM inference, as explored in "Speed Up LLM Inference with DSpark Speculative Decoding"[/post/speed-up-llm-inference-with-dspark-speculative-decoding-cmthjnpki0w9pmi9zt4016tdq], demonstrating the necessity of efficient and well-defined systems to harness the power of AI effectively.
Ultimately, this transformation represents a maturation of the software development process. We’re moving beyond the era of individual brilliance and entering an age of collaborative intelligence, where humans and AI work together to build and maintain complex systems. The challenge now lies in fostering a culture of experimentation and learning, where engineers are empowered to design and implement these crucial boundaries and feedback loops. The question worth watching isn't whether AI will replace software engineers, but rather, how we can redefine the role to leverage the power of AI while maintaining control and ensuring the integrity of our data and systems.
If you look at the commit histories of modern data platforms, something profound has shifted over the last two years. The friction of writing syntax has collapsed. With Cursor, Claude Code, and agentic workflows now living inside our Docker containers and IDEs, generating the first implementation of a distributed streaming pipeline or a complex API integration is no longer the central bottleneck.
Agents can navigate repositories, write test coverage, inspect stack traces, and propose refactors. Describe a Kafka-to-Iceberg sink mapping in plain English, and an agent can produce a credible starting point before the engineer has opened every relevant file.
That changes the question for software engineers.
If the agent is becoming the primary author of local system logic, what exactly is left for the engineer to do? Are we headed toward an industry of reviewers rubber-stamping an endless stream of plausible pull requests? Or has the work shifted away from constructing logic and toward something more abstract?
To answer that, it helps to borrow a lens from thermodynamics, which gives us a language for directed work, feedback, loss, and the boundaries that keep a complex system coherent.
The agent as a heat engine
When we strip away the anthropomorphic illusion of AI, what remains is a computational engine. It takes direction and turns it into action.
An LLM sitting in a data center has immense capacity, but it does no useful work until it is given intent. A prompt, a business requirement, a system instruction, or a failing test gives the agent a direction. It turns that direction into code, tool calls, queries, tests, and changes to a running system.
Every engine has losses. Every agent loop does, too.
Anyone who has left an agent running against a difficult repository has seen this. It starts with a clear task. Then it follows a stale assumption, fixes a symptom rather than a cause, treats an old migration as current behavior, and begins to accumulate its own history. A few tool calls later, the context contains enough plausible but conflicting detail that the next step is less certain than the first.
Call this operational entropy: the buildup of stale assumptions, branching context, and unresolved dependencies inside a loop that is still trying to move forward.
A human interruption helps because it introduces new information. So does a failing test, a precise data contract, a deterministic tool, or an evaluation that tells the agent exactly what it got wrong. Without that signal, an agent can keep generating output while drifting further from a correct outcome.
Agents clearly generate motion. The real question is whether the system around them turns that motion into useful work.
The infinite monkey and the accelerating search space
The infinite monkey theorem gives us a useful picture of what follows: repeated attempts, finite constraints, and feedback.
The theorem says that a monkey hitting keys at random for an infinite amount of time will almost surely type the complete works of Shakespeare. Modern agents are much smarter monkeys. They have compilers, tools, repositories, test suites, and feedback loops. Their work is not random — the feedback steers the next attempt — but the dynamic is familiar: propose, execute, observe, correct, and try again.
In a bounded task, that loop is remarkably effective.
Give an agent a known input schema, a known target schema, a small codebase, and tests that catch the relevant failures. It can inspect the code, make a change, run the tests, absorb the result, and try again. The definition of done is visible. The search space is narrow. The loop has a chance to converge.
But enterprise systems rarely offer that kind of stillness. A real-time pricing engine may depend on mutable operational state, third-party APIs, late-arriving events, regional policy, and business rules that exist partly in code and partly in someone’s head. A data lakehouse can be physically consistent and semantically wrong. A pipeline can pass its tests and still produce numbers that finance does not recognize.
The environment is changing while the monkey is typing.
The three-body problem of enterprise logic
This is why the three-body problem is such a useful image for enterprise software.
With two bodies — a planet and a star — you can predict the motion with a clean mathematical description. Add a third body and the problem becomes much harder to solve. There is no general closed-form solution, and some configurations exhibit chaotic behavior. Small changes in one place can produce very different trajectories elsewhere.
Modern data platforms have the same shape. Clickstream data changes with product behavior. Operational databases mutate under customer activity. APIs impose rate limits and change versions. Schemas evolve. Security policies shift. Legacy systems carry rules that no one has written down because they have been buried in exception handling for years.
Each system exerts pressure on the others. A change in one place alters the meaning or behavior of another. What begins as a local feature request starts to pull on the entire system.
Consider a hypothetical: an agent is asked to add a customer_tier field to a revenue model. It finds a field called status in the operational database, maps it into the transformation, and passes the existing type and nullability tests. The code is clean. The pipeline is green. The answer is still wrong.
A semantic data contract says that customer_tier is derived from trailing twelve-month spend, has an assigned business owner, and cannot be populated from account status. The contract rejects the change before it reaches the dashboard. The engineer's contribution wasn't the transformation — it was the boundary that made the agent's mistake visible, specific, and recoverable.
The new mandate: Designing equilibrium
The software engineer’s job is no longer to write every piece of micro-logic. Agents will increasingly do that work, often faster. The new mandate — designing equilibrium — is to create the conditions in which generated logic can be trusted.
When a business requirement changes faster than an agent can absorb feedback, the engineer has to build containment fields. Strict semantic layers, immutable event logs, data contracts, idempotent APIs, and deterministic state machines are not just good platform hygiene. They reduce the number of assumptions an agent has to make at once.
They turn a coupled problem into a bounded domain with clear inputs, explicit rules, and reliable feedback.
Once that domain exists, the agent becomes genuinely powerful. It can write the transformation, execute the tests, repair the failures, and ship the change without needing to infer the unwritten history behind every table and service.
The value of software engineering doesn't disappear as code generation gets cheaper — it becomes more visible, which is the shift that actually matters.
Autonomous systems will increasingly generate software. But the contracts, feedback loops, and boundaries that determine whether that software succeeds or spirals into chaos will still be designed by software engineers.
Ananth Packkildurai is a data engineering leader, writer, and author of Data Engineering Weekly, sharing insights on modern data platforms, large-scale pipelines, and AI-driven architectures.
Read on the original site
Open the publisher's page for the full experience