If an organization runs a survey in 2024 on whether it should get into AI, then they’ve already bodged an LLM into the system and they’re seeing if they can get away with it. Proton Mail is a priva…
we appear to be the first to write up the outrage coherently too. much thanks to the illustrious @self
also I keep meaning to push on this and getting distracted:
only for business users, who have asked for it
fuck no, this breaks the security model for every proton user. one of the key assumptions of Proton’s end to end encrypted model is that the plaintext of a messsge never touches Proton’s servers, on both ends of the conversation. now if a proton business/visionary (and they keep fucking forgetting they forced their visionary accounts into having this horseshit) user sends me a message or a reply, there’s a chance the plaintext on their end was exposed to Proton’s servers, and as the receiver I can’t control or even detect the conditions that cause the plaintext leak (is the sender a proton business/visionary account? did they use the cloud version of the LLM? what text did it operate on?)
fucking unworkable. I’m not even a cryptographer, but this is the most obvious plaintext leak I’ve ever seen in a cryptography product.
and now, my swing at a secure version of this feature:
if I receive a message whose content was sourced from the cloud LLM (ie the user activated the feature at any point while writing), instead of pulling the content of the message, protonmail displays a warning that the content of the message was exposed to their servers, and I’m given buttons to either display the message, or delete it and block the sender. if I delete the message and block the sender, protonmail itself sends a message back to the message’s author proving that I deleted the message unopened.
I’m not kidding, that’s the only secure version of this. that’s the version a privacy-oriented company would have implemented, if they really had to do any of this at all (they didn’t)
also the other one, where this feature gets lacklustre uptake but not enough to kill it, and then it just gets sorta shoved into a side panel, and then every so often it’s turned on by default again because someone updated the config/prefs code or some other banal-but-instantly-effective reason (presuming it’s not even intentionally turned on again by adding new default-on settings for “different” uses-that-to-build features)
“but that’s insanely paranoid, nobody would take a risk like that into account” shout the big Proton fans doing security kayfabe. “are you fucking lost”, I shout back
also I keep meaning to push on this and getting distracted:
fuck no, this breaks the security model for every proton user. one of the key assumptions of Proton’s end to end encrypted model is that the plaintext of a messsge never touches Proton’s servers, on both ends of the conversation. now if a proton business/visionary (and they keep fucking forgetting they forced their visionary accounts into having this horseshit) user sends me a message or a reply, there’s a chance the plaintext on their end was exposed to Proton’s servers, and as the receiver I can’t control or even detect the conditions that cause the plaintext leak (is the sender a proton business/visionary account? did they use the cloud version of the LLM? what text did it operate on?)
fucking unworkable. I’m not even a cryptographer, but this is the most obvious plaintext leak I’ve ever seen in a cryptography product.
and now, my swing at a secure version of this feature:
if I receive a message whose content was sourced from the cloud LLM (ie the user activated the feature at any point while writing), instead of pulling the content of the message, protonmail displays a warning that the content of the message was exposed to their servers, and I’m given buttons to either display the message, or delete it and block the sender. if I delete the message and block the sender, protonmail itself sends a message back to the message’s author proving that I deleted the message unopened.
I’m not kidding, that’s the only secure version of this. that’s the version a privacy-oriented company would have implemented, if they really had to do any of this at all (they didn’t)
also the other one, where this feature gets lacklustre uptake but not enough to kill it, and then it just gets sorta shoved into a side panel, and then every so often it’s turned on by default again because someone updated the config/prefs code or some other banal-but-instantly-effective reason (presuming it’s not even intentionally turned on again by adding new default-on settings for “different” uses-that-to-build features)
“but that’s insanely paranoid, nobody would take a risk like that into account” shout the big Proton fans doing security kayfabe. “are you fucking lost”, I shout back