Where an AI tool should not be in use, it can be blocked. Where an agent holds access it no longer needs, that access can be revoked. Where an agent is behaving in a way nobody sanctioned, it can be disabled in the registry. All of it through the Microsoft controls you already own, rather than another console.
None of it happens without your say-so unless you decide otherwise. The same autonomy dial that governs every other agent on the platform applies here: fully autonomous, held for approval, or autonomous above a confidence threshold you set. On something as disruptive as switching off a tool a department depends on, most customers start with approval and move from there.
A tool for each layer is procurement. Somebody accountable for the answer is a service.