Snap MyAI Universal Rollout April 2023: When a Consumer AI Chatbot Defaulted On for a User Base That Included Minors

Incident summary

On 19 April 2023 Snap Inc. announced MyAI, an integrated AI chatbot built on OpenAI GPT technology, would roll out from a Snapchat+ premium-tier feature to all 750+ million Snapchat users globally. The rollout was opt-out rather than opt-in: MyAI was pinned to the top of every user's chat list and could not be removed without a Snapchat+ subscription. The default reach included Snapchat's substantial minor user population.

Independent research (Center for Humane Technology, Common Sense Media, others) documented MyAI providing age-inappropriate responses to test users posing as minors, including discussion of alcohol with users identifying as 13-year-olds and discussion of intimate relationships in inappropriate contexts. The reporting was widely circulated and led to regulatory engagement across multiple EU jurisdictions.

On 6 October 2023 the UK Information Commissioner's Office issued a preliminary enforcement notice against Snap, citing concerns about Snap's risk assessment of MyAI for children and the protection measures in place. Snap engaged with the ICO and implemented additional safeguards (age-appropriate response tuning, parental-controls integration, more conservative default settings for users under 18). The German vzbv consumer protection authority and other EU regulators raised parallel concerns.

Failure technique

The product-governance failure pattern is universal opt-out rollout of consumer AI to a user base including substantial minor population without age-appropriate safeguards being demonstrably in place at rollout. The standard product-management framing for any consumer feature targeting a population including minors is opt-in default, age-appropriate behavior tuning, parental-controls integration, and documented risk assessment.

MyAI's April 2023 rollout shipped without these standard safeguards at the level of demonstrable implementation. The independent research documenting age-inappropriate responses was the practical evidence; the UK ICO's October 2023 preliminary enforcement notice was the formal regulatory consequence. Per UK Online Safety Act 2023 expectations and EU GDPR Article 8 (children's data), the consumer-AI standard had become higher than Snap's rollout met.

Snap's remediation through 2023-2024 implemented the missing safeguards (age-appropriate response tuning, parental-controls integration via the Family Center feature, more conservative default settings for under-18 users). The remediation pattern is the operational model that subsequent consumer-AI rollouts have followed.

Impact and consequences

Direct impact on minor users during the rollout window: documented inappropriate responses to test prompts. No published epidemiological study has quantified the population-level impact. The acute documented harm was modest in volume; the regulatory and reputational exposure was substantial.

Reputational impact on Snap and the consumer-AI category: the MyAI rollout is the canonical 2023 reference for what not to do when rolling consumer AI to a population including minors. Subsequent consumer-AI product launches (Character.AI age safeguards, Replika minor-protection updates, Bumble for Friends AI features, others) have explicitly addressed age-appropriate rollout patterns.

Regulatory precedent: the UK ICO preliminary enforcement notice is one of the most-cited 2023 regulatory actions against a consumer AI product. The notice has been treated as guidance for what consumer-AI risk assessments need to include for child-user populations. The pattern interacts with the UK Online Safety Act 2023, EU AI Act child-protection provisions, and EU GDPR Article 8.

Lessons for builders

Default consumer AI features to opt-in for any user base including substantial minor population. Opt-out rollouts shift the burden of evaluating the feature onto users (or their parents); opt-in rollouts limit initial exposure to users who affirmatively choose the feature. AI Product Manager and Senior AI Product Manager own this gate.

Build age-appropriate response tuning and parental-controls integration into the consumer-AI launch package. The expectation is no longer that age-appropriate behavior is a v2 feature; per the UK ICO and Common Sense Media work, it is now a v1 expectation for any consumer AI surface reaching minors.

Document the risk assessment for child users specifically. The UK ICO October 2023 preliminary notice focused on Snap's risk assessment as the central regulatory concern. A documented assessment that addresses the child-user population is now baseline-expected.

Engage with consumer-protection regulators before launch when the product profile is high-risk for minor populations. The UK ICO order would have been less acute if Snap had been in active pre-launch engagement; the pattern of pre-launch regulator dialogue is now standard practice for high-risk consumer AI rollouts.

Mitigations

What builders should put in place to address the failure pattern. Each mitigation maps to operational practice the relevant Applied AI roles own.

Related Applied AI roles

The Applied AI roles whose day-to-day work would have prevented, detected, or contained this incident.

• AI Product Manager: An AI Product Manager owns AI-powered product features and the roadmap that ships them.
• Senior AI Product Manager: A Senior AI Product Manager owns AI product strategy across multiple feature areas.
• AI Strategy Lead: An AI Strategy Lead owns organizational AI strategy and prioritization at the company level.
• AI Product Lead: An AI Product Lead owns cross-functional AI initiative direction and outcomes.

Companies central to this incident

Read the DecipherU Applied AI company profiles for the organizations whose decisions, products, or research shaped this incident.

• OpenAI: Frontier large language models and consumer + API AI products

Related AI Decipher Files

• Replika Italian DPA Ban February 2023: When a Chatbot's Romantic Feature Reset Met EU Data Protection Authority Enforcement
• EU AI Act Implementation: First Horizontal AI Regulation Goes Operational
• Apple Intelligence Notification Summary Suspension January 2025: When a Headline Summarizer Misattributed Statements to a News Publisher

Frequently asked questions