Microsoft has rolled out a new feature called "Computer Use" for its Copilot Studio platform. This platform lets businesses and regular users create AI agents to automate tasks on websites and desktop apps. Copilot Studio launched in November 2023, allowing users to build and customize AI agents without coding. The AI agents can already handle tasks like answering questions, autofilling data, managing tasks, and working with Microsoft 365.
The "Computer Use" feature lets AI agents interact with websites and desktop applications just like humans do. They can click buttons, select menu items, and enter data into fields on a screen. This means the AI agents can perform various tasks. Users don’t need to connect the AI to other systems or apps via API to get things done. The "Computer Use" feature makes it possible to integrate AI into tasks without needing an API connection.
One of the key benefits of "Computer Use" is its ability to detect changes in UI. So, if a button is moved on a website or app, or the screen design is changed, the AI agent will adapt. It can find the button and click it to submit a form, ensuring that tasks are completed smoothly.
Microsoft had previously introduced a feature called "Actions" in its Copilot for general users. This feature allows Copilot to perform tasks like booking restaurant tables, purchasing event tickets, buying concert tickets or movie tickets, and shopping online. However, "Actions" seems to be limited to websites and apps partnered with Microsoft. In contrast, "Computer Use" can work with any website or app, giving it much broader applicability. This means users can use it to book a table or order flowers on any website.
How it Works
The "Computer Use" feature is part of Microsoft’s efforts to make AI more accessible and useful for everyday tasks. By allowing AI agents to interact with various applications and websites, Microsoft is opening up new possibilities for automation and efficiency.
For more information, check out the original source on The Verge.