Initially, Copilot was considered an AI component attached to Bing, but it has since evolved into a comprehensive AI that now exists in almost every Microsoft product. Copilot has received a series of improvements that gradually make it competitive with ChatGPT and Google’s Gemini in functionality, and also allow Bing to catch up with Google.
Microsoft adds major expansion to Copilot features to help it catch up with other chatbots
Microsoft announces major expansion of Copilot features.The biggest change is the introduction of a new feature called “Action,” which enables AI assistants to perform specific online tasks for users based on prompts from conversational instructions. According to Microsoft, this system primarily handles tasks in the background while users are dealing with other matters. These tasks may include booking restaurants, purchasing event tickets, or ordering takeout items.
Action is somewhat like the most expensive tier of ChatGPT’s Operator, except it only applies to a limited, curated list of websites. For this type of functionality, a restricted list is probably a better approach for now. Amazon also announced its “Buy for You” shopping agent, which similarly works with a curated list of sites. Training AI to understand a specific list of websites is much easier than fully trusting it to handle everything right away. Copilot Action’s initial partners include travel and booking services such as Booking.com, Expedia, Kayak, Tripadvisor, Skyscanner, Viater, Vrbo, and Priceline. At launch, it also supports restaurant reservations through OpenTable and ordering flowers via 1-800-Flowers.com.
Microsoft said the feature will eventually work with various websites, and mentioned it could handle transportation bookings and may integrate with ride-hailing services. But for now, it’s only available to specific select partners. While Copilot may be fully tested to operate on these particular sites, you should still be careful about the personal information it enters, in case something goes wrong.
Microsoft has also expanded the coverage of its visual analysis feature, Copilot Vision. Previously it was only available in the Edge browser, but Vision is now integrated into the Windows operating system and the Copilot mobile app (available for both iOS and Android). On mobile devices, Copilot Vision can analyze live images from the device’s camera, enabling it to provide more assistance, such as plant care tips, home decor suggestions, and more. For Windows users, you simply share your screen with Vision and let the AI analyze what’s displayed on it. This can support use cases like getting help with a specific application or helping you analyze the photos and web pages you’re viewing. The Windows version of Vision will first roll out to Windows Insider preview members, then come in a broader release.
Microsoft is also adding its own response to Google’s AI mode by introducing “Copilot Search” in Bing, which essentially replaces search results with pure AI-generated results, similar to searching in Google’s AI mode. When users perform a search, Copilot Search can generate summaries of key information, provide direct answers, or display information in a structured layout, depending on the nature of the query. Microsoft emphasized that these AI-generated responses will highlight information sources, including inline links within the text and a list of URLs used, making it easier for users to verify information and navigate to the original content with ease.
Except for the Windows version’s Vision feature, which still needs further refinement and testing, the other items should be rolling out to everyone soon.
Source: KOCPC Chinese