The term "real-time" is doing too much work in this space, and that imprecision is becoming a real problem for anyone trying to evaluate the technology. When a vendor says "live video inference," they might mean a model that generates a few frames quickly, or they might mean a system that continuously transforms an incoming stream with latency measured in milliseconds. Those are not the same engineering challenge, and pretending they are only muddies the waters for buyers who need to make informed decisions.
The core distinction comes down to architecture and constraints. Fast video generation is a batch problem with a short deadline. You have an input, you generate an output, and you optimize for speed within that discrete task. Continuous inference is a different beast entirely. The model has to process a live stream, make decisions frame by frame, and do so within a latency budget that leaves no room for retries or buffering. The architecture that works for one simply will not work for the other. The field has not converged on a shared definition, and that lack of convergence is not a minor semantic quibble. It has practical consequences for procurement, for research direction, and for setting realistic performance expectations.
For users, this means you cannot rely on marketing language to tell you what you are actually getting. You need to ask pointed questions about latency budgets, about whether the system is processing frames sequentially or in parallel, and about what happens when the input stream changes in unexpected ways. The harder version of this problem, the one where a model is genuinely responding to a live feed in real time, demands a different set of trade-offs. It demands specialized infrastructure and a willingness to accept that some tasks simply cannot be done at interactive speeds yet. That is not a failure of the technology; it is a reflection of the physics involved.
The organizations that are actually doing the harder version of this problem are the ones that talk about their constraints openly. They are not hiding behind a label. They are discussing their frame rates, their pipeline designs, and their fallback strategies for when the model cannot keep up. That transparency is the standard the rest of the industry should be held to. Until then, treat "real-time" as a starting point for a technical conversation, not a conclusion. Ask the hard questions, and demand answers that reflect the actual complexity of the task.