The controversy surrounding Windows 11's newly-announced Recall continues. The UK data protection watchdog has already confirmed that it's "making inquiries" with Microsoftwhile billionaire SpaceX CEO Elon Musk has warned millions of followers on X to ditch the feature— all of this despite Recall not actually being available on any Windows 11 PCs yet.
Recall is one of several Artificial Intelligence (AI) features coming exclusively to Copilot+ PCs. These devices, which include the all-new Surface Laptop 7 from Microsoft, require a dedicated Neural Processing Unit (NPU) to handle AI tasks. Samsung, Dell, and Asus amongst others have all pledged to launch Copilot+ PCs.
But ahead of the launch of the first batch of Copilot+ PCs on June 18, cybersecurity expert Kevin Beaumont has flagged a number of potential flaws in the Recall feature. Given that Beaumont used to work for Microsoft, he's well-placed to comment on this upcoming AI feature.
Recall is the headline feature of this slate of Copilot+ PCs, which lets you scroll back through everything that has happened on your Windows 11 machine and jump back in time with a click. Everything is searchable as AI has trawled through the images and text on-device
MICROSOFT PRESS OFFICEAfter a week of testing, Kevin Beaumont discovered that Recall stores data in a database in plain text. Without encryption, that could make it trivial for a hacker to extract data about everything you've been doing on your PC.
“Every few seconds, screenshots are taken. These are automatically OCR’d by Azure AI, running on your device, and written into an SQLite database in the user’s folder,” Kevin Beaumont explained in a lengthy blog post criticising the feature.
“This database file has a record of everything you’ve ever viewed on your PC in plain text.”
OCR — Optical Character Recognition — is the practice of digitising pictures of typed, handwritten, or printed text. Microsoft will leverage AI to transcribe the text from webpages, Word documents, PDFs, handwritten notes, and everything else displayed on-screen on your PC so that everything is instantly searchable.
For example, if you know you were looking at flights to Spain in the last month — you could search for the destination to find the exact webpage. With a single click, Windows 11 will summon the document, picture, video or webpage to pick up where you left off. You can also scroll back in time through the screenshots, which are captured hundreds of times every hour and can be stored for months at a time.
Yusuf Mehdi, Chief Marketing Officer at Microsoft, described how Recall works a company blog: "We set out to solve one of the most frustrating problems we encounter daily – finding something we know we have seen before on our PC. Today, we must remember what file folder it was stored in, what website it was on, or scroll through hundreds of emails trying to find it. Now with Recall, you can access virtually what you have seen or done on your PC in a way that feels like having photographic memory."
But cybersecurity expert Kevin Beaumont has branded the feature as a "disaster", warning that "stealing everything you’ve ever typed or viewed on your own Windows PC is now possible with two lines of code."
As a proof of concept, Beaumont claims to have "automated exfiltration, and made a website where you can upload a database and instantly search it" so anyone can trawl through the complete history of everything seen on-screen by the Recall feature.
Microsoft told media outlets a hacker cannot exfiltrate Copilot+ Recall activity remotely.
— Kevin Beaumont (@GossiTheDog) May 30, 2024
Reality: how do you think hackers will exfiltrate this plain text database of everything the user has ever viewed on their PC? Very easily, I have it automated.
HT detective pic.twitter.com/Njv2C9myxQ
"I am deliberately holding back technical details until Microsoft ship the feature as I want to give them time to do something. I actually have a whole bunch of things to show and think the wider cyber community will have so much fun with this when generally available.. but I also think that’s really sad, as real world harm will ensue," the ex-Microsoft employee has written in the detailed blog post about the flaws.
Recall is enabled by default on all Copilot+ PCs, although it can be switched off in the settings at a later date. By default, it won't capture screenshots whenever you're using a private browsing mode, like Incognito Mode in Google Chrome, Microsoft has reassured PC owners. You can also disable the feature on a per-app basis.
Kevin Beaumont has advised Microsoft not to release Recall with the first batch of Copilot+ PCs this month as it risks damaging customers' trust in the desktop operating system and feature. He writes: "In my opinion — they should recall Recall and rework it to be the feature it deserves to be, delivered at a later date. They also need to review the internal decision making that led to this situation, as this kind of thing should not happen.
"Earlier this month, Microsoft’s CEO emailed all their staff saying“If you’re faced with the tradeoff between security and another priority, your answer is clear: Do security.” We will find out if he was serious about that email.
"They need to eat some humble pie and just take the hit now, or risk customer trust in their Copilot and security brands. Frankly, few if any customers are going to cry about Recall not being immediate available — but they areabsolutely going to be seriously concerned if Microsoft’s reaction is to do nothing, ship the product, slightly tinker or try to wordsmith around the problem in the media."
Recall is one of a number of features that will only appear on Windows 11 devices designated as Copilot+ PCs, which feature a chipset with an NPU to handle all Artificial Intelligence tasks
MICROSOFT PRESS OFFICE
Several other privacy campaigners have also reacted strongly to the announcement of Recall.
"This could be a privacy nightmare," said Dr Kris Shrishak, an adviser on AI and privacy, told the BBC. "The mere fact that screenshots will be taken during use of the device could have a chilling effect on people."
Jen Caltrider, who serves as Program Director for the *Privacy Not Included team at Mozilla, has warned that Recall would allow anyone with access to your laptop or desktop PC to a treasure-trove of personal information since it contains a categorised list of recent activity.
Caltrider cautioned: "[This includes] law enforcement court orders, or even from Microsoft if they change their mind about keeping all this content local and not using it for targeted advertising or training their AIs down the line."
Microsoft says Recall will not censor or wipe information from the screenshots that it takes, even when passwords or bank account details are visible on-screen.
In a blog post about the new functionality, which will arrive with the new Surface Laptop next month, the US company wrote: "Recall leverages your personal semantic index, built and stored entirely on your device. Your snapshots are yours; they stay locally on your PC. You can delete individual snapshots, adjust and delete ranges of time in Settings, or pause at any point right from the icon in the System Tray on your Taskbar.
"You can also filter apps and websites from ever being saved. You are always in control with privacy you can trust."
Recall and a number of other AI features will be exclusive to Windows 11 running on so-called Copilot+ PCs, including the newly-announced Surface Laptop 7th Edition, pictured above MICROSOFT PRESS OFFICE But Jake Moore, global cybersecurity adviser at software security firm ESET, said the creation and storage of more private data through the feature could be an enticing prospect for cyber criminals.
“Enabling a feature which has the ability to capture screen data not only offers even more data to the company behind the software but also opens up another avenue for criminals to attack,” he said.
LATEST DEVELOPMENTS
“Whilst this feature is not on by default, users should be mindful of allowing any content to be analysed by AI algorithms for a better experience.
“Although it may produce better results, there is a balance that must be kept regarding functionality versus privacy and so users must remain aware of the potential risks should any sensitive data ever become compromised. Creating and storing more private data seems unnecessary when cyber criminals continually look for any given vulnerability to exploit.”
