Skip to main content
AI risk

Risk type: Data Leakage

Sensitive information in AI prompts

Prompt leaks start when people paste private data into AI. Your plan and settings decide what the vendor keeps. Some data never belongs in a prompt.

OWASP lists sensitive information disclosure as LLM02. Consumer ChatGPT, Claude, and Gemini can use chats for training. Each offers a setting to stop that. Business plans exclude training by default. Endpoint DLP can block risky pastes on managed devices. It only catches what its rules describe. Never paste passwords, keys, or customer records.

Last updated

2026-10-30

What each AI vendor does with your prompts

Summarized from each vendor’s help and privacy pages. OpenAI rows use archived copies from October 27 and 30. Other sources checked October 30, 2026.
Tool and planUsed for trainingWhat else to knowMain control
ChatGPT Free, Plus, and ProYes, unless you opt out.Unsaved Temporary Chats are not used for training. OpenAI may keep them up to 30 days for safety.Turn off “Improve the model for everyone.”
ChatGPT Business, Enterprise, Edu, and APINot by default.Business plans need a separate agreement to opt in. API owners can enable data sharing.Workspace and API organization settings.
Claude Free, Pro, and MaxWhen Model Improvement is on.Kept up to 5 years when on. Incognito chats are never used. Feedback can be used.Model Improvement in Privacy Settings.
Claude Team, Enterprise, and APINot by default.Commercial terms apply, not consumer settings.Admin and contract settings.
Gemini Apps, personal accountsYes, with Keep Activity on.Reviewers read some chats. Reviewed chats are kept up to 3 years.Turn off Keep Activity. Chats still stay 72 hours.
Gemini in Google WorkspaceNot outside your domain without permission.No human review without permission. Gemini sees only content the user can access.Workspace admin settings and sharing permissions.
Microsoft 365 CopilotNot for foundation models.Copilot surfaces only data the user can already view.SharePoint permissions and sensitivity labels.

Summarized from each vendor’s help and privacy pages. OpenAI rows use archived copies from October 27 and 30. Other sources checked October 30, 2026.

Affected tools

  • ChatGPT
  • Claude
  • Google Gemini
  • Microsoft 365 Copilot
  • AI browser extensions

How it usually happens in the browser

  • Engineers paste tokens and config snippets to debug errors.
  • Support teams paste tickets, logs, and screenshots for summaries.
  • Finance teams paste invoices and bank details to extract fields.
  • People upload contracts and CSV exports to browser AI tools.
  • Prompts and answers get copied across tabs and tools.
  • AI extensions add input boxes your DLP may not cover.

What traditional defenses miss

  • Email and storage DLP does not see prompt boxes.
  • Network tools rarely see prompt text inside encrypted sessions.
  • Logs and stack traces hide secrets in normal-looking text.
  • Written policies cannot keep pace with daily AI use.
  • Personal devices sit outside endpoint DLP.

Mitigation checklist

  • Write a never-paste list with real examples.
  • Approve AI tools whose business terms exclude training.
  • Block or warn on pastes of keys, secrets, and PII.
  • Set role-based rules for engineering, finance, and support.
  • Redact logs and error payloads before anyone pastes them.
  • Track blocked patterns, top AI tools, and exceptions.

Where browser isolation fits

  • Shield runs the page in a browser that is not on your computer, destroyed when you close the tab.
  • Shield moves page execution off your device. Downloads and copied data cross that boundary.
  • Prompt rules belong in your AI tool and DLP settings.
  • Use Shield for untrusted links that AI answers surface.
Add to Chrome

Limits

  • Vendor terms change, so recheck sources before policy decisions.
  • Contracts and admin settings can override public help pages.
  • DLP coverage depends on your devices, browsers, and rules.
  • We reviewed vendor documentation and did not test the tools.

FAQs.

What counts as sensitive information in an AI prompt?
OWASP lists personal data, financial details, and health records. It adds confidential business data, credentials, and legal documents. Treat source code and customer records the same way.
How do I stop ChatGPT from training on my chats?
Turn off “Improve the model for everyone” in Data controls. Or use a Temporary Chat and do not save it. Feedback you send can still train models. Business, Enterprise, and Edu plans are excluded by default.
Do Claude and Gemini train on what you paste?
Claude does when Model Improvement is on. Incognito chats are never used. Gemini does when Keep Activity is on. Work and school accounts follow separate terms.
Does browser DLP stop AI data leaks?
Partly, and only where your policies reach. Endpoint DLP can warn on or block risky pastes. Microsoft Purview checks sensitive info types at paste time. Rules miss data they do not describe.
Is AI DLP different from classic DLP?
The goal is the same: keep sensitive data in bounds. AI adds new paths: prompt boxes, uploads, and AI extensions. Classic programs often started with email and storage.
What should we block first?
Start with credentials, API keys, and tokens. Add regulated data and customer personal data. These are high impact and easy to describe as patterns.
How do teams enforce AI data rules without banning AI?
Approve one AI tool with business terms. Restrict unapproved AI sites and extensions. Add paste rules for the riskiest data. Review exceptions every month.

References

  1. 01
    OWASP LLM02:2025 Sensitive Information DisclosureOWASP Gen AI Security Project
  2. 02
    How your data is used to improve model performanceOpenAI Help Center
  3. 03
    Data controls in ChatGPTOpenAI Help Center
  4. 04
    Is my data used for model training? (consumer plans)Anthropic Privacy Center
  5. 05
    How long do you store my data?Anthropic Privacy Center
  6. 06
    Is my data used for model training? (commercial products)Anthropic Privacy Center
  7. 07
    Gemini Apps Privacy HubGoogle
  8. 08
    Generative AI in Google Workspace Privacy HubGoogle Workspace
  9. 09
    Data, Privacy, and Security for Microsoft CopilotMicrosoft Learn
  10. 10
    Help prevent leakage of sensitive content by restricting paste actions into browsersMicrosoft Learn

Keep exploring

Try it on the next page you do not trust

Legba is a Chrome extension with two modes. Ghost gives you a private browser route. Shield opens a page in an isolated browser off your device.

Add to Chrome

Free for 30 days. No card required.

Ghost. A private route for your browser. Shield. An isolated browser, off your device. Choose the mode. Close when finished.