Online inference with large language models carries serious privacy risks that many users underestimate. Prompts—often containing passwords, emails, private thoughts, medical data, intimate details, and business data—are sent in plain text and processed by third-party servers. These prompts can be seen not only by the major providers but also by some smaller platforms, which may lack proper security or even relay your data through multiple services. The user has no clear control or visibility over where their input goes or who accesses it.
Despite this, people regularly hand over sensitive information without hesitation. The reason is simple: the AI works well, and the benefits are immediate. The risks—like data breaches or misuse—are abstract, or hard to trace. This trade-off, convenience over caution, is common in digital behavior.
Companies often claim not to use your data for training, but these assurances are hard to verify. Terms of service can include loopholes, policies can change. Past examples like email scanning or the 23andMe saga, or many others selling genetic data show that even well-known platforms can repurpose user input in ways people never expected.
Regulation offers little protection, including the GDPR and AI Act. Privacy laws are outdated or not properly enforced. I won't even speak of politics.
But there's an alternative.
Running AI models locally offers a safer alternative. Your data never leaves your device. No third party can access it. While this approach requires significant computing power (powerful laptop or a desktop, say, with 64GB RAM+) and some technical skill. For most users, it’s not yet practical, though these models gradually arrive also to smartphones, or even web browsers.
Until local AI becomes easier to use, online inference will remain the norm—at a cost to privacy that’s far greater than most realize.
From company security to national security
Lastly, some data just cannot be sent to some remote servers for contractual, regulatory or national security purposes. Local LLMs are a perfect match for such processing. Such data never leaves the organisation data center.
