The CISO playbook for generative AI has been built around a single assumption: that the sensitive part of the interaction happens when data leaves the building. That assumption is quietly dissolving, and the security industry is only starting to feel the floor shift. Local inference is not a niche workaround anymore. It is a mainstream capability running on the same laptops your engineers already carry, and it operates without a single packet crossing the firewall. If your governance model still starts and ends with the browser, you are not managing Shadow AI 2.0. You are auditing the wrong layer of the stack.
Here is what changes in practical terms. When an employee runs a model locally, your DLP, your CASB, and your proxy logs all go dark. There is no outbound call to inspect, no cloud audit trail to pull, no sanctioned gateway to route around. The activity looks like nothing happened, because from the network's perspective, nothing did. But on the device, something significant just occurred: a model ingested proprietary code, financial data, or customer information, and returned a response shaped by weights you did not vet, a license you did not review, and a supply chain you did not verify. The risk profile shifts from exfiltration to something more insidious. You are no longer worried only about data walking out the door. You are worried about what happens to that data while it never leaves the room.
The integrity problem is the one that should keep you up at night. A developer pulls down a community-tuned coding model because it benchmarks well, runs it offline on sensitive code, and commits the output. The result compiles, passes tests, and looks reasonable. But the model was never vetted for your environment, and the changes it suggests may introduce weak validation, unsafe defaults, or dependencies your policy explicitly forbids. When you later discover the vulnerability, you will investigate the symptom. You will not see the cause, because there is no record that AI influenced the code path at all. That is not a hypothetical edge case. That is a direct consequence of letting ungoverned inference run on the endpoint. The same logic applies to licensing. A model with a non-commercial license used in production code is a legal time bomb that detonates during M&A diligence or a customer security review. And because there is no inventory of what ran where, you cannot even begin to assess the exposure.
The path forward is not to block local inference. That would be both futile and counterproductive, because the demand is real and the productivity gains are tangible. The answer is to treat model weights like software artifacts and the endpoint like the new perimeter. That means scanning for high-fidelity indicators like large .gguf files or local listeners on port 11434, using EDR and MDM policies to control which runtimes can execute, and building a curated internal model hub with pinned versions, verified licenses, and hashes. It also means updating your acceptable use policy to explicitly cover downloading and running model artifacts on corporate devices, because "cloud services" no longer describes where the work happens. The teams that get this right will not just avoid the next audit finding. They will give their developers a paved road that makes the safe path the easy path. The ones that do not will keep staring at network logs while the real action happens on the silicon sitting right in front of them. The perimeter has moved. Your controls need to follow.