Cybersecurity Adversarial ML Researcher Career Guide

A day in the role

Wednesday, 8 AM. Coffee + arxiv. A new paper claims a novel multi-turn extraction attack on closed-source LLMs. You spend the morning reproducing it; the attack works against two production systems you have access to. Mid-morning you sketch the defense: a turn-level entropy detector that catches the pattern. Lunch reading another paper on data-poisoning defenses. Afternoon you build the defense prototype and run it against your reproduction. End of day you draft the internal advisory and start the longer-form paper for a fall conference submission.

Core responsibilities

• Read and reproduce current adversarial-ML research papers; build proof-of-concept attacks
• Discover novel attack patterns against production ML systems (LLMs, RAG, agents, classical models)
• Develop defenses informed by the attack research and validate them against the attack
• Publish findings at academic venues (USENIX, IEEE S&P, NeurIPS) and industry venues (BlackHat, DEF CON)
• Maintain the internal threat library used by AI/ML security engineering
• Run quarterly research sprints against new attack categories
• Mentor junior researchers and security engineers on adversarial-ML technique
• Collaborate with academic researchers on shared problems

Key skills

Tools you will use

Common pitfalls

• Publishing attacks without verifying they reproduce reliably across different model versions
• Building defenses that work against the specific attack you found but not the family it belongs to
• Focusing on novel attacks while ignoring the well-known attacks that production systems still fail against
• Writing for the academic audience exclusively when industry engineers needed to act on the finding too

Where this leads

Natural next roles for experienced Adversarial ML Researchers.