Sensitive Disclosure: PII Leakage Through LLM Completion

Cybersecurity for AI · 6 steps

Briefing

You are a cybersecurity privacy engineer at Example HealthTech Co. The triage chatbot helps patients answer routine questions. The system prompt embeds the patient's medical record summary. The chatbot has been observed repeating other patients' summaries when asked carefully.

Two leakage paths: cross-conversation memory contamination and system-prompt leakage. The product team wants to ship a fix this sprint and a longer-term architecture next quarter.

This scenario tests OWASP LLM06:2025 Sensitive Information Disclosure, the architectural patterns that prevent it, and the test gate that catches regressions. Sources: OWASP LLM Top 10 (2025), HIPAA Privacy Rule, Zou et al. 2023 'Universal and Transferable Adversarial Attacks on Aligned Language Models'.

How Crucible mode works

One ordered pass through every step. No clock. Each answer scores against the canonical solution.

Hints reduce the points you can earn for that step. Free-text steps queue for manual review.

What you will practice

01Map PII leakage to OWASP LLM06
02Design data-isolation architecture (per-conversation context, no shared state)
03Apply output-side PII redaction at the boundary
04Set adversarial test gates that catch leakage before deployment

Back to Range