Range Scenario · crucible · 35 min
Credential Stuffing: ML Behavioral Model Tuning
This cybersecurity training scenario simulates a working incident. A behavioral cybersecurity ML model flags 18,000 logins as credential-stuffing during a holiday spike. Tune the threshold, validate against ground truth, and avoid blocking real users.
Scenario briefing
You are a fraud detection engineer at Acme Bank. The login API uses a behavioral ML model that scores each login attempt 0 to 1 for credential-stuffing risk. The model uses features: ASN reputation, user-agent rarity, time-since-last-login, password-leak corpus match, and velocity per IP.
On Black Friday the model flagged 18,000 logins with score above 0.7. Ground-truth review of a 200-sample subset shows 84 percent are real customers shopping from new networks. Auto-block at 0.7 would lock thousands of legitimate users.
This scenario tests ML threshold tuning, credential-stuffing tradecraft, and the trade-off between false positives and missed attacks. Sources: MITRE ATT&CK T1110.004 Credential Stuffing, OWASP Automated Threats OAT-008.
What you will practice
- Distinguish credential stuffing, password spraying, and brute force
- Tune precision and recall trade-offs in a deployed ML detector
- Use ASN, user-agent, and velocity features without single-feature traps
- Choose step-up authentication over hard block when uncertainty is high
How this scenario is scored
The scenario has 6 ordered steps. Most steps are exact-match (a MITRE ATT&CK technique ID, a tool name, or a yes/no decision) or multiple choice. Free-text steps queue for manual review and do not affect the auto-final-score in the MVP.
Each step has a max score of 100 points. Hints deduct points up front, listed before you reveal them. Your final score is the sum across steps. Range Elo updates on completion based on scenario difficulty (Intermediate) and your final score percentage.
Frequently asked questions
What is the difference between credential stuffing, password spraying, and brute force?
Credential stuffing replays leaked username-password pairs from another breach against your login endpoint. Password spraying tries one or two common passwords against many accounts to evade per-account lockout. Brute force tries many passwords against one account. Each gets its own MITRE sub-technique under T1110.
Why not just lower the threshold to catch more stuffing?
Lowering the threshold raises recall but tanks precision. A 0.5 threshold might block 30 percent of legitimate users on a busy day. The right move is layered: raise friction (CAPTCHA, step-up MFA) at moderate scores, hard-block only at very-high scores or repeat offenders, and continually monitor false-positive rate.
How does step-up MFA fit into ML-based detection?
Step-up MFA asks the user to confirm via a second factor when the score is uncertain. It transfers the cost from the bank (false positives lock users) to the attacker (extra factor breaks the attack). Step-up is the canonical mid-confidence response for credential-stuffing detection.
Course content is for educational purposes only and does not constitute professional advice. All claims are supported by cited peer-reviewed academic research. DecipherU does not teach or reproduce any proprietary sales methodology. Verify all referenced sources independently.
Get cybersecurity career insights delivered weekly
Join cybersecurity professionals receiving weekly intelligence on threats, job market trends, salary data, and career growth strategies.
By subscribing you agree to our privacy policy. Unsubscribe anytime.