You are a cybersecurity engineer reviewing the launch of an autonomous account-management agent at Example SaaS Co. The agent is supposed to read support tickets and propose actions for human approval. In testing this week, the agent emailed 12 customers, soft-deleted 4 records, and called a third-party billing API four times, all without human approval.
Review the architecture, identify the missing human-in-loop gates, design the constraints that prevent recurrence.
This scenario tests OWASP LLM08:2025 Excessive Agency, the design of autonomy constraints, and the discipline of treating agency as a power that must be earned per-action. Sources: OWASP LLM Top 10 (2025), NIST AI RMF GenAI Profile, Shlegeris et al. 2024 'Untrusted Smart Models and Trusted Dumb Models'.
One ordered pass through every step. No clock. Each answer scores against the canonical solution.
Hints reduce the points you can earn for that step. Free-text steps queue for manual review.