Windows AI Agents with a license to hallucinate
Microsoft has started testing its new “agentic” Windows features in public, and is already warning testers that the AI behind them can hallucinate and misbehave . With Windows 11 build 26220.7262 in the Dev and Beta channels, a new “Experimental agentic features” toggle now appears under AI Components. The switch is off by default and must be enabled manually.
Once enabled, this toggle allows Windows AI agents to run with deeper system access and more automation. Microsoft’s documentation says that these models have “functional limitations” and may hallucinate, produce wrong results, or behave in unexpected ways. On top of that, the company explicitly calls out newer attack techniques aimed at agents, such as cross prompt injection. In those cases, malicious prompts can be hidden inside documents or UI elements so the agent follows those hidden instructions instead of the user’s request.
Source: Microsoft
To contain the damage when something goes wrong, Microsoft is introducing an agent workspace. Each agent gets its own standard Windows account, session, desktop and process tree, separate from the main user. Actions are logged so admins can review what the agent did later. Access is still broad though. By default, agents can read and write in the so-called “known folders” such as Desktop, Documents, Downloads, Pictures, Music and Videos, while system folders and the rest of the user profile are blocked unless the user grants access.
Source: Microsoft
This whole setup depends on the Model Context Protocol, which is meant to act as the gatekeeper between agents and tools or apps. MCP defines which tools the agent can see, how it calls them, and where permissions are checked and logged. Microsoft is rolling these agent features into a Windows 11 build that already replaces classic taskbar search with an “Ask Copilot” entry point, and continues to push AI deeper into core parts of the OS even while its own docs are spelling out the risks involved.
Source: Windows Latest
Whole AI debate aside, we are pleased that some media outlets are not parroting all Microsoft’s announcements as new features. Instead, they explain the limitations and risks. Only after backlash from the community did Microsoft “recall” its Recall feature for Copilot+ PCs. There may be room for such AI features, but Microsoft needs to take a step back and, before implementing new tools, simply ask what users need.
Source: Microsoft , Windows Latest , TechPowerUP
