Excerpts: My Note I knew they might report people for harmful image generation. I am slightly surprised that they do so automatically for chats with LLMs. I thought that if they do it for chats, why wouldn’t search companies do it for searches too. It seems that searches don’t provide as much contex […]
Hi Wrecker, your practical conclusion is sound, although I would sharpen two details.
First, this was
not apparently an entirely automatic report to police. According to the arrest-report account, automated safety systems flagged the statements, after which a human review team examined them and contacted law enforcement. In other words: automatic detection and escalation, followed by a human disclosure decision. The reported second statement about obtaining a gun would also have supplied considerably more context than one isolated phrase. These remain allegations underlying a charge, not a conviction. (SWFL/WINK report)
Second, calling a chatbot a “diary” does not grant it diary privilege. Anthropic says conversations are encrypted in transit and at rest, but also says designated Trust & Safety staff may access them when review is required to enforce its Usage Policy. That is conventional cloud encryption,
not end-to-end encryption where the provider lacks the key. (Anthropic’s explanation)
There are several distinct disclosure routes:
- the provider identifies what it considers an imminent danger and makes an emergency disclosure;
- police send the provider an emergency request;
- authorities obtain compulsory legal process such as a subpoena, court order or warrant;
- particular illegal-content categories may carry separate statutory reporting obligations.
Search and cloud-storage companies are not exempt. Google’s published policy says it may provide information when it reasonably believes that doing so could prevent death or serious physical harm, including cases involving bomb threats or school shootings. It also expressly identifies email,
documents and photos as content that US authorities can obtain with a search warrant. That does not demonstrate that Google proactively reports ordinary searches or scans every Drive document for threats, but it does mean that “Google has not publicized such a case” should not be mistaken for “Google cannot or will not disclose it.” (Google’s policy)
Chats do provide more context than search queries and therefore may be easier to assess—but they also generate richer, account-linked records. A search query is not protected merely because it is short and ambiguous.
I would only soften “is no secret at all.” Non-E2EE cloud content is not automatically public; encryption still protects it from network interception and some forms of theft. It is simply
not confidential from the service provider, its authorized reviewers or valid legal demands. For genuinely private notes, an encrypted local application or a properly designed E2EE service is preferable, while remembering that metadata, backups, compromised endpoints and anyone with whom the content is shared remain outside the magic circle.
Sources
