Cybersecurity Adversarial ML Researcher Interview Questions & Preparation Guide

What they evaluate

Foundational attack knowledge

Strong answer framework

FGSM (Goodfellow, Shlens, Szegedy 2014) generates adversarial examples by taking one step in the direction of the sign of the gradient of loss with respect to the input, scaled by epsilon. It demonstrated that adversarial examples are linear-time computable, ubiquitous, and transferable across models. The paper popularized the gradient-based attack family that includes BIM, PGD, and Carlini-Wagner. Critically, it shifted the field from anecdotal failures to a systematic adversarial robustness research program.

Common mistake

Knowing the formula but missing the historical and conceptual significance.

What they evaluate

Attack methodology depth

Strong answer framework

PGD (Madry et al. 2018) iteratively applies FGSM-style steps, projecting back onto the epsilon-ball after each step. With random initialization across the ball, PGD approximates the worst-case first-order adversary. Madry conjectured (and empirically supported) that adversarial training against PGD provides robustness against any first-order attacker. PGD is the standard benchmark; if your defense fails against PGD, it likely fails against stronger attacks.

Common mistake

Treating PGD as just iterative FGSM without understanding the saddle-point formulation.

What they evaluate

Evaluation rigor

Strong answer framework

Use the AutoAttack ensemble (Croce and Hein 2020) for parameter-free evaluation: APGD-CE, APGD-DLR, FAB, Square Attack. Test across multiple epsilon values and norms (L-infinity, L2, L0). Verify gradient computations are not obfuscated; check for gradient masking using BPDA (Athalye et al. 2018). Use white-box, black-box, and transfer attacks. Report attack hyperparameters fully. Avoid the common pitfall of evaluating only against a single attack with fixed hyperparameters.

Common mistake

Reporting a single FGSM accuracy number and claiming robustness.

What they evaluate

Theoretical depth

Strong answer framework

Certifiable robustness provides a mathematical guarantee that no adversarial example exists within an epsilon-ball around a given input. Methods include interval bound propagation, randomized smoothing (Cohen et al. 2019), and Lipschitz-bounded networks. Empirical robustness measures resistance to known attacks but offers no guarantee against future ones. Certifiable methods often have lower clean accuracy and tighter applicability. The two are complementary: certifiable for safety-critical pixels, empirical for broader coverage.

Common mistake

Treating empirical robustness as equivalent to a guarantee.

What they evaluate

Research practice

Strong answer framework

Use a real example. Describe the open question, hypothesis, methodology, baselines compared against, and findings. Discuss reviewer feedback during the publication process and how the work evolved. Reflect on what the work contributed to the field and what the next open question is. If unpublished, describe internal research with the same structure. Senior researchers articulate research narratives clearly.

Common mistake

Listing project outcomes without articulating the research question or methodology.

What they evaluate

Threat model rigor

Strong answer framework

Specify: adversary capability (white-box, gray-box, black-box, query budget), perturbation budget (norm, epsilon, semantic constraints), goal (untargeted, targeted, evasion, poisoning), and adversary knowledge (model architecture, training data, defense). State assumptions explicitly; many real-world attacks violate the assumptions of academic threat models. Reference the NIST AI 100-2 (Adversarial Machine Learning) taxonomy as a reference vocabulary.

Common mistake

Designing attacks under unrealistic threat models that do not transfer to deployment scenarios.

What they evaluate

Defense methodology depth

Strong answer framework

Randomized smoothing (Cohen, Rosenfeld, Kolter 2019) wraps a base classifier in Gaussian noise sampling: at inference, predict the majority class across many noisy samples. The smoothed classifier has provable robust radius proportional to noise level and the gap between top and runner-up class probabilities. Trade-off: more noise gives larger certified radius but lower clean accuracy. Used in Cohen et al. as the first scalable certified defense.

Common mistake

Confusing noise-based defense with adversarial training; randomized smoothing has formal guarantees.

What they evaluate

Modern LLM attack landscape

Strong answer framework

Discrete token space changes the attack surface; gradient-based attacks need adaptation (GCG attack, Zou et al. 2023). Jailbreaking exploits training distribution gaps (DAN-style, AutoDAN). Indirect prompt injection delivers attacks through retrieved content. Embedding-space attacks bypass token-level defenses. Multi-turn attacks build context to elicit harmful outputs. Compared to vision: discrete inputs, semantic rather than pixel-level perturbations, longer reasoning chains, and instruction-following as a vulnerability vector.

Common mistake

Applying vision-attack mental models without adapting for tokenization and instruction following.

What they evaluate

Defense evaluation rigor

Strong answer framework

Athalye, Carlini, Wagner 2018 showed that many published defenses worked by causing gradient computation to fail (gradient masking) rather than removing adversarial examples. The defenses fell to BPDA (Backward Pass Differentiable Approximation) and EOT (Expectation Over Transformations) attacks. Diagnose with: does PGD with many random restarts and many iterations still fail? Do transfer attacks succeed? Does BPDA work? If yes to any, you have obfuscated gradients, not robustness.

Common mistake

Publishing a defense that defeats a single weak attack and claiming robustness.

What they evaluate

Research practice

Strong answer framework

Open-source code with explicit dependency versions. Provide trained model checkpoints. Specify random seeds, hyperparameters, and hardware. Use standard benchmarks (CIFAR-10, ImageNet, HarmBench for LLMs). Run AutoAttack rather than custom attacks for evaluation. Provide adversarial example inputs as artifacts. Encourage independent reproduction; the RobustBench leaderboard requires it. Document compute costs honestly.

Common mistake

Reporting numbers without the artifacts needed for independent verification.

What they evaluate

Research ethics

Strong answer framework

For attacks against deployed systems: notify affected parties under coordinated disclosure timelines (typically 90 days, longer for systemic issues). For attacks against frontier models: engage with frontier lab safety teams privately before public release. Avoid releasing attack code that enables material harm without mitigation. For academic publications, follow venue guidelines (USENIX Security and IEEE S&P have specific responsible disclosure expectations). Publish defenses alongside attacks where possible.

Common mistake

Releasing attack tooling without coordinating with affected systems.

What they evaluate

Critical perspective

Strong answer framework

Adversarial examples are real but not universally critical. In closed-set image classification with controlled inputs, they may be edge cases. In safety-critical domains (autonomous vehicles, medical imaging, security classifiers), they are direct risks. In LLMs, jailbreaking is operationally relevant for any deployed system. The honest answer recognizes both academic interest and bounded operational risk; over-claiming dilutes the field.

Common mistake

Either dismissing adversarial robustness entirely or claiming it is the most pressing security concern.

What they evaluate

Career strategy

Strong answer framework

Both inform each other. Strong attacks expose defense weaknesses; strong defenses constrain attacks. Many researchers oscillate. Choose based on the open question you find most pressing, the available collaborators, and the venues you target. Frontier labs hire heavily for both. Independent research often leans attack-side because publication is faster. Defense-side research benefits from longer-form collaboration with deployment teams.

Common mistake

Picking a side based on perceived prestige rather than the research question.

What they evaluate

Benchmark literacy

Strong answer framework

RobustBench (Croce et al.) standardized adversarial robustness reporting on CIFAR-10 and ImageNet, raising rigor across the community. HarmBench (Mazeika et al.) does similar work for LLM jailbreaking. Both have limits: benchmarks become contested over time, and improvements may overfit. Use them as one signal among many. Pair with novel evaluations relevant to your target deployment context. Track methodology debates around how benchmarks measure what they claim.

Common mistake

Treating leaderboard ranking as the sole measure of research quality.

What they evaluate

Strategic thinking

Strong answer framework

Examples: interpretability of adversarial features, adversarial robustness in foundation models at scale, certifiable defenses for LLMs (currently sparse), evaluation of agents under multi-step adversarial pressure, adversarial robustness in scientific ML (climate models, drug discovery), and the relationship between robustness and generalization. Pick a real open problem and articulate why it is open and what progress would look like.

Common mistake

Naming a vague problem without explaining what current research has and has not addressed.