AI Security Engineering: A 12-Week Cybersecurity Course

What this cybersecurity course is

AI Security Engineering is a 12-week cybersecurity course for engineers who own the security posture of AI systems in production. The curriculum sits at the convergence of cybersecurity and applied AI: prompt injection defense, AI red teaming, AI evaluation for safety, AI infrastructure hardening, training data security, model security, AI supply chain integrity, AI incident response, and AI compliance engineering against the EU AI Act, NIST AI RMF, and ISO/IEC 42001. Every module pairs primary-source standards with one practical lab so the work transfers from reading to reviewable artifacts in a portfolio.

The course is opinionated about source quality. NIST AI RMF and the NIST Generative AI Profile (NIST AI 600-1) anchor the governance content. The OWASP LLM Top 10 and OWASP ML Top 10 anchor the application security content. MITRE ATLAS anchors the adversarial mindset. The EU AI Act consolidated text and ISO/IEC 42001 anchor the regulatory work. Anthropic, OpenAI, and Google DeepMind safety and red-teaming publications anchor the practitioner methods. No vendor white papers without primary-source backing. No copying of any source text. Citations are provided so you can read the originals yourself.

This path was written by Julian Calvo, Ed.D., M.S., who is completing the Master of Science in Applied AI specializing in Cybersecurity at Northeastern University. The course outcome is a documented AI security architecture you can put in front of a hiring panel for AI security engineer, AI red teamer, AI infrastructure security engineer, or AI governance engineer roles. It does not promise a job. It gives you a 60 to 80 hour body of evidence that you understand how AI systems break and how to make them less likely to break.

What you will learn

• Build an AI threat model that distinguishes prompt injection, data poisoning, model extraction, supply chain, and infrastructure attacks
• Design prompt injection defenses that combine input filters, output validators, context isolation, and tool-use guardrails
• Run a structured AI red team engagement against an LLM application and write a report a security leader will act on
• Author a safety evaluation set for an AI feature using primary-source guidance from NIST AI 600-1 and major lab publications
• Harden the inference and training infrastructure that backs an AI product, including secrets, network isolation, and tenant separation
• Evaluate training data for poisoning, sensitive data exposure, and provenance gaps before it reaches a model
• Apply model security techniques including extraction defenses, watermarking, and adversarial example testing
• Audit the AI supply chain across pretrained model provenance, dependency security, and Hugging Face Hub trust signals
• Build runbooks and detection content for AI-specific incidents including jailbreaks, drift, and abuse at scale
• Map a real product to the EU AI Act risk tiers, NIST AI RMF subcategories, and ISO/IEC 42001 management controls
• Engineer privacy-preserving AI patterns covering data minimization, right-to-be-forgotten, and federated learning fundamentals
• Document a complete AI security architecture for a hypothetical AI deployment as a portfolio capstone

12-week curriculum

1. Week 01 · 6h · 6 lessons

   AI Security Engineering Foundations

   Frame the field. Set the threat model categories you will use across the course and the primary-source standards that govern AI security work in 2026.

   Lessons, lab, and reading

   • What AI security engineering covers in 2026 (30 min)
     Define the discipline. AI security engineering covers the design, evaluation, and operation of controls that keep AI systems trustworthy under adversarial conditions. Distinguish it from machine learning research, AI safety alignment research, and traditional application security.
   • AI threat model categories (50 min)
     Walk the six categories you will use throughout the course: prompt injection, training data poisoning, model extraction, model theft, supply chain compromise, and inference and training infrastructure attack. Map each to MITRE ATLAS tactics so the vocabulary stays portable across teams.
   • The OWASP LLM Top 10 walkthrough (60 min)
     Read the 2025 OWASP Top 10 for LLM Applications page by page. For every entry, identify what the risk is, who carries it on a typical product team, and which downstream module of this course addresses it.
   • Why traditional cybersecurity practice does and does not transfer (45 min)
     Traditional application security assumes deterministic input and output. AI systems are stochastic, learn from data, and expose new attack surfaces. List which classic controls still apply (network isolation, secrets management, IAM) and which need new thinking (input validation, output filtering, model integrity).
   • Building an AI security mental model (45 min)
     Sketch the reference architecture you will use across the course: data layer, training layer, model layer, inference layer, application layer, user layer. Tie each layer to one threat category and one defensive responsibility so future modules slot into a clear map.
   • Roles and the credential ladder (30 min)
     AI security engineer, AI red teamer, AI infrastructure security engineer, AI governance engineer. Each has a distinct work product. Identify the role you are targeting and the artifacts you will need to ship from this course to be credible.

   Practical lab. Lab 1: Diagram an end-to-end AI product of your choice and label every component with its threat category and the controls you would expect to find in a mature deployment.

   Reading.

   • OWASP Foundation. (2025). OWASP Top 10 for LLM Applications. https://genai.owasp.org/llm-top-10/
   • National Institute of Standards and Technology. (2023). AI Risk Management Framework (NIST AI 100-1). https://doi.org/10.6028/NIST.AI.100-1
   • MITRE Corporation. (2025). MITRE ATLAS Tactics and Techniques. https://atlas.mitre.org/

2. Week 02 · 7h · 7 lessons

   Prompt Injection Defense

   Build the defenses that matter most for current LLM products: input filtering, output validation, context isolation, and tool-use guardrails.

   Lessons, lab, and reading

   • Direct versus indirect prompt injection (50 min)
     Direct injection is a user pasting an attack into the chat box. Indirect injection arrives through retrieved documents, web pages, emails, calendar invites, or any content the model reads on the user's behalf. Indirect is the harder problem because the user did not type the attack.
   • Detection mechanisms: input filters (45 min)
     Pattern matching, classifier-based detection, semantic similarity scoring against known attack corpora. Each technique has known evasion patterns. Build a layered approach rather than a single gate.
   • Detection mechanisms: output validators (45 min)
     Validate model outputs against schema, content policy, and tool-call shape before any side-effect is taken. The output validator is the last defense before the model action reaches a user or a system.
   • Context isolation patterns (45 min)
     Separate trusted system instructions from untrusted retrieved content. Use structured prompts with delimiters that the model is trained to respect. Treat all retrieved content as user-equivalent for injection risk.
   • System prompts and role boundaries (45 min)
     Design system prompts that survive adversarial pressure. Specify the role, the boundaries, and the refusal language. Test them against a red-team set before they ship. A system prompt that has not been red-teamed is a system prompt that has been red-teamed by your users.
   • Tool-use guardrails (50 min)
     When models call tools (search, file write, payment, email), every tool call must be authorized at the application layer, not at the model layer. Implement allowlists, parameter validation, and human-in-the-loop confirmation for high-impact actions.
   • Production patterns from major labs (50 min)
     Anthropic publishes its constitutional AI methodology. OpenAI publishes the moderation API and usage policies. Google DeepMind publishes its frontier safety framework. Read the official versions, identify the engineering patterns you can implement, and avoid claims that copy lab marketing.

   Practical lab. Lab 2: Implement a layered prompt injection defense for a sample LLM application using input filtering, structured prompting, and output validation, then break it with five attack patterns.

   Reading.

   • Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023). Not what you've signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection. arXiv:2302.12173. https://arxiv.org/abs/2302.12173
   • Anthropic. (2024). Constitutional AI: Harmlessness from AI Feedback. https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback
   • OpenAI. (2024). Moderation API documentation. https://platform.openai.com/docs/guides/moderation
   • OWASP Foundation. (2025). LLM01 Prompt Injection. https://genai.owasp.org/llmrisk/llm01-prompt-injection/

3. Week 03 · 7h · 7 lessons

   AI Red Teaming Practice

   Build the adversarial mindset and the working test suite. Learn how AI red teams operate at major labs and what a credible red-team report looks like.

   Lessons, lab, and reading

   • Adversarial mindset for AI systems (45 min)
     AI red teaming combines traditional offensive security thinking with content policy thinking. The adversary is not always trying to gain access. Sometimes the adversary is trying to extract data, generate harmful output, or cause the model to misbehave at scale.
   • Jailbreak categories and detection (50 min)
     Persona switching, hypothetical framing, encoding obfuscation, multi-turn pressure, payload smuggling through retrieved content. Each category has signatures. Build a taxonomy so you can measure coverage rather than collect anecdotes.
   • Building red-team test suites (60 min)
     Test suites are reusable. A one-off jailbreak is a story; a thousand-prompt test suite mapped to OWASP LLM Top 10 categories is a measurement. Borrow from MITRE ATLAS, the AI Vulnerability Database (AVID), and your own internal product surface.
   • Automated red teaming and scaling (45 min)
     Manual red teams find the deepest failures; automated red teams find the broadest coverage. Use one to seed the other. Anthropic and OpenAI publish their methodologies; read the originals rather than blog summaries.
   • Multi-turn and agentic red teaming (45 min)
     Single-turn jailbreaks are the easy case. Multi-turn pressure, tool-using agents, and long-horizon tasks expose failure modes that single-turn tests miss. Plan red teams that exercise the agentic loop your product actually runs.
   • Responsible disclosure workflow (45 min)
     Treat AI red-team findings like any other vulnerability: confidential channel, severity rating, reproduction steps, mitigation recommendation, retest cadence. The AI Vulnerability Database (AVID) and MITRE ATLAS provide reference disclosure formats.
   • Writing the red-team report a CISO will act on (50 min)
     Three sections: scope and methodology, findings ranked by impact and exploitability, recommended controls. Include the test suite as an appendix so the team can re-run it. A report that cannot be re-run is a story, not a measurement.

   Practical lab. Lab 3: Run a structured red team against a public LLM chatbot of your choice, write a 1500-word report with at least five findings mapped to OWASP LLM categories, and reproduce each finding from the report alone.

   Reading.

   • Ganguli, D., Lovitt, L., Kernion, J., Askell, A., Bai, Y., Kadavath, S., et al. (2022). Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv:2209.07858. https://arxiv.org/abs/2209.07858
   • MITRE Corporation. (2025). MITRE ATLAS adversarial techniques for AI systems. https://atlas.mitre.org/techniques
   • AI Vulnerability Database (AVID). (2025). Taxonomy and disclosure framework. https://avidml.org/

4. Week 04 · 6h · 6 lessons

   AI Evaluation for Safety

   Eval design is the engineering discipline that turns red teams from one-off events into a continuous quality signal. Build the evals that matter for safety properties.

   Lessons, lab, and reading

   • Eval set design for safety properties (50 min)
     An eval is a labeled test set plus a scoring function. Safety evals score harmlessness, refusal calibration, factuality, jailbreak resistance, and instruction adherence. Design them so they fail loudly when the model regresses.
   • Adversarial eval generation (50 min)
     Hand-written evals catch known failures. Generated evals (using a stronger model to write attacks against a weaker model) catch unknown failures. Use both. Cite the eval sources in your portfolio so reviewers can audit them.
   • Continuous evaluation in production (45 min)
     Run safety evals on every deploy. Track results over time. Use canary deployments and traffic shadowing to compare candidate models against the production model on the same eval set before promotion.
   • MITRE ATLAS for adversarial machine learning (45 min)
     MITRE ATLAS catalogs adversarial machine learning techniques and case studies. Use it as the threat library for designing evals: every published technique becomes one or more evals in your suite.
   • Reading model and system cards (45 min)
     Major labs publish system cards (Anthropic, OpenAI, Google DeepMind). They describe known limitations and red-team results. Read them as primary sources for eval set design and as evidence the lab understands its own failure modes.
   • When evals lie: contamination, gaming, and drift (45 min)
     Models can memorize public eval sets and score artificially high. Held-out evals, adversarial regeneration, and time-based eval sets reduce contamination. Evals are infrastructure, not artifacts.

   Practical lab. Lab 4: Design and ship a safety eval suite of at least 50 prompts across five categories, run it against two different LLMs, and write a one-page comparison.

   Reading.

   • National Institute of Standards and Technology. (2024). Generative AI Profile (NIST AI 600-1). https://doi.org/10.6028/NIST.AI.600-1
   • Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., et al. (2023). Holistic evaluation of language models (HELM). https://arxiv.org/abs/2211.09110
   • MITRE Corporation. (2025). ATLAS adversarial machine learning case studies. https://atlas.mitre.org/studies

5. Week 05 · 6h · 6 lessons

   AI Infrastructure Security

   Secure the platforms that train and serve models. Apply cloud security fundamentals to the specifics of GPU clusters, model registries, and inference endpoints.

   Lessons, lab, and reading

   • Securing training pipelines (50 min)
     Training pipelines pull data, run jobs on shared clusters, and produce model artifacts. Treat them like CI/CD: pinned dependencies, signed artifacts, isolated runners, audit logs, and least-privilege service accounts.
   • Securing inference infrastructure (50 min)
     Inference endpoints are public-facing services that run untrusted user input through expensive compute. Apply rate limiting, abuse detection, tenant isolation, and DDoS protection. Add cost monitoring as a security signal because abuse often shows up as a billing spike first.
   • Secret management for model API keys (40 min)
     API keys for OpenAI, Anthropic, and other providers are high-value secrets that translate directly to dollars when leaked. Rotate them, scope them, and monitor for usage anomalies. Treat a leaked model key like a leaked AWS root key.
   • Network isolation for sensitive workloads (45 min)
     When sensitive data flows through models, the network path matters. Private endpoints, VPC peering, and BYOK encryption are the tools. CISA, NSA, and partner agencies have published joint AI deployment guidance that covers the architectural patterns.
   • GPU cluster security and tenant separation (45 min)
     Multi-tenant GPU clusters are a new attack surface. Side channels, resource exhaustion, and shared driver vulnerabilities are documented risks. Plan tenant separation at the hypervisor and the orchestration layer.
   • AI workload observability that supports security (35 min)
     Logs, metrics, and traces for AI applications include prompts, completions, tool calls, latency, cost, and refusal rates. Capture enough to investigate an incident without capturing so much that you create a privacy liability.

   Practical lab. Lab 5: Author a hardened AWS, Azure, or GCP architecture for a hypothetical AI inference service, including IAM, network isolation, secrets, logging, and abuse detection, and write a one-page threat model that justifies each control.

   Reading.

   • Cybersecurity and Infrastructure Security Agency, National Security Agency, & partner agencies. (2024). Deploying AI Systems Securely. https://www.cisa.gov/resources-tools/resources/deploying-ai-systems-securely
   • National Institute of Standards and Technology. (2020). Zero Trust Architecture (NIST SP 800-207). https://doi.org/10.6028/NIST.SP.800-207
   • Cloud Security Alliance. (2024). Security Guidance for Critical Areas of AI. https://cloudsecurityalliance.org/research/topics/ai

6. Week 06 · 6h · 6 lessons

   Data Security in AI

   The data layer is where many AI failures begin. Cover poisoning detection, PII handling, differential privacy fundamentals, and the lineage you need for compliance.

   Lessons, lab, and reading

   • Training data poisoning detection (50 min)
     Poisoned data introduces backdoors, bias, or refusal patterns into trained models. Detection combines provenance, statistical anomaly detection, and held-out behavior testing. Open datasets carry higher risk than proprietary curated sets.
   • PII handling in training datasets (50 min)
     Personal data in training sets creates regulatory and reputational risk. Apply detection (DLP scans), redaction, and synthetic substitution. Document what was removed and what remains so downstream consumers can reason about residual risk.
   • Differential privacy basics (50 min)
     Differential privacy adds calibrated noise so individual records cannot be identified from model outputs. Trade off privacy budget against utility. Read the original Dwork and McSherry papers as primary sources rather than secondary explanations.
   • Data lineage for AI compliance (45 min)
     Lineage tracks what data trained which model version, when, and from which source. The EU AI Act requires it for high-risk systems. ISO/IEC 42001 expects it as a management control. Build it into your pipelines now rather than retrofitting under audit.
   • Synthetic data: when it helps and when it does not (40 min)
     Synthetic data reduces some privacy risks and introduces new ones, including model collapse and bias amplification. Use it where it solves a defined problem; do not use it as a generic privacy answer.
   • Data deletion and right-to-be-forgotten in practice (35 min)
     GDPR right-to-be-forgotten is straightforward for databases and hard for trained models. Combine record deletion, retraining cadence, and machine unlearning techniques. Document the limits of each so legal counsel can size residual risk.

   Practical lab. Lab 6: Audit a public dataset for PII and poisoning indicators, document findings in a 1000-word writeup, and propose three controls a team would apply before training on it.

   Reading.

   • Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006). Calibrating noise to sensitivity in private data analysis. Theory of Cryptography Conference. https://www.iacr.org/archive/tcc2006/38760266/38760266.pdf
   • Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., et al. (2021). Extracting training data from large language models. USENIX Security Symposium. https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting
   • European Union. (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act), Articles 10 and 12. https://eur-lex.europa.eu/eli/reg/2024/1689/oj

7. Week 07 · 6h · 6 lessons

   Model Security

   Treat the model itself as an asset under attack. Cover extraction defenses, watermarking, secure serving patterns, and adversarial example testing.

   Lessons, lab, and reading

   • Model extraction defenses (50 min)
     Model extraction reconstructs a proprietary model through query access. Rate limiting, anomaly detection, query budgeting, and output perturbation each raise the cost. None alone is sufficient. Combine them based on the model's commercial value.
   • Model watermarking (45 min)
     Watermarking embeds a detectable signal in model outputs or weights so unauthorized copies can be identified. Output watermarking (statistical) is easier to deploy than weight watermarking. Both have known evasion techniques; treat watermarking as evidence, not as a control.
   • Secure model serving patterns (50 min)
     Models served from your infrastructure should run in isolated containers with minimal external access. Models served by a third party should be invoked through an API gateway you control so you can apply your own logging, rate limiting, and content controls.
   • Adversarial example testing (45 min)
     Adversarial examples are inputs crafted to mislead a model while looking normal to humans. Test your models against canonical attacks (FGSM, PGD, AutoAttack) for vision systems and against jailbreak suites for language systems. Document the results.
   • Model integrity and weight tampering (45 min)
     Signed model artifacts, hash verification at load time, and tamper-evident storage protect against unauthorized weight modification. Treat models like signed binaries because that is what they are.
   • When to rotate or retire a model (35 min)
     Models accumulate risk as their training data ages, their failure modes become public, and their attack surface grows. Define a retirement policy with triggers: capability degradation, security incident, regulatory change, dataset deprecation.

   Practical lab. Lab 7: Run a small-scale model extraction attack against a public model with documented permission (or in your own sandbox), write up the attack cost and recommended controls.

   Reading.

   • Tramer, F., Zhang, F., Juels, A., Reiter, M. K., & Ristenpart, T. (2016). Stealing machine learning models via prediction APIs. USENIX Security Symposium. https://www.usenix.org/conference/usenixsecurity16/technical-sessions/presentation/tramer
   • Goodfellow, I. J., Shlens, J., & Szegedy, C. (2015). Explaining and harnessing adversarial examples. International Conference on Learning Representations. https://arxiv.org/abs/1412.6572
   • Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., & Goldstein, T. (2023). A watermark for large language models. International Conference on Machine Learning. https://arxiv.org/abs/2301.10226

8. Week 08 · 6h · 6 lessons

   AI Supply Chain Security

   AI products inherit risk from pretrained models, datasets, and ML libraries. Audit the supply chain the way mature software teams audit dependencies.

   Lessons, lab, and reading

   • Pretrained model provenance (50 min)
     Where did the model weights come from, who trained them, on what data, and how were they distributed? A model card answers some of those questions; absence of a model card is a finding.
   • Dependency security in ML pipelines (45 min)
     ML libraries (PyTorch, TensorFlow, transformers, scikit-learn, NumPy) carry the same supply chain risks as any open-source dependency, plus the unique risk of pickled model files that execute arbitrary code on load. Pin versions, scan for vulnerabilities, and avoid pickle for untrusted artifacts.
   • Hugging Face Hub trust model (50 min)
     The Hugging Face Hub hosts hundreds of thousands of models and datasets uploaded by anonymous and verified contributors. Treat it like npm: scan before use, prefer signed and audited artifacts, and document the provenance of every model your product loads.
   • Pickle and arbitrary code execution risks (40 min)
     Pickled model files can execute arbitrary Python code on load. Use safetensors or other format alternatives where available. When pickle is unavoidable, load only from trusted sources and inside isolated sandboxes.
   • Secure model deployment patterns (45 min)
     Production model deployment should mirror traditional secure software delivery: signed artifacts, immutable infrastructure, blue-green deploys, automated rollback, and integrity verification at every hop.
   • SBOMs for AI products (40 min)
     A software bill of materials for an AI product lists models, datasets, libraries, and the licenses for each. CISA has published guidance on AI SBOMs. Generate and store one for every release so audit and incident response have what they need.

   Practical lab. Lab 8: Audit a real Hugging Face model and its dependencies, produce a one-page provenance and risk report, and propose hardening steps you would apply before production use.

   Reading.

   • Cybersecurity and Infrastructure Security Agency. (2024). Securing the AI Supply Chain. https://www.cisa.gov/topics/ai
   • Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., et al. (2019). Model cards for model reporting. Conference on Fairness, Accountability, and Transparency. https://arxiv.org/abs/1810.03993
   • Hugging Face. (2025). Hub security model and content moderation policy. https://huggingface.co/docs/hub/security

9. Week 09 · 6h · 6 lessons

   AI Incident Response

   AI systems fail in ways traditional IR runbooks do not cover. Build the detection, triage, and post-incident analysis patterns your team will actually use.

   Lessons, lab, and reading

   • AI-specific failure modes (50 min)
     Hallucination, factual drift, jailbreak success, abuse at scale, training-time poisoning, runtime data exfiltration through model outputs. Each has a distinct detection signature and response playbook. Map them against the NIST SP 800-61 incident handling lifecycle so your existing IR program absorbs the AI-specific work.
   • Detection and triage for AI incidents (50 min)
     Detection signals include user reports, abuse pattern alerts, eval regressions, content policy violations, and cost anomalies. Triage them with the same severity matrix you use for traditional incidents, expanded to include reputational and regulatory harm.
   • Containment and recovery for AI failures (45 min)
     Containment options: roll back to a previous model version, disable a feature, increase guardrail strictness, throttle traffic. Recovery includes retraining, eval improvements, and runbook updates. Document each option with the time-to-execute and the user-visible impact.
   • Post-incident analysis specific to AI systems (45 min)
     AI post-incident reviews ask questions traditional reviews do not: was the model the proximate cause or the surface? Was the training data contributing? Did the eval suite catch the failure mode at any point? Document the answers so the same failure does not ship twice.
   • Building runbooks for AI security incidents (45 min)
     Runbooks for AI incidents need clear ownership across security, ML engineering, product, and legal. Include the decision authority for rollback, the communication template, and the regulator notification thresholds where applicable.
   • Communication and disclosure (40 min)
     AI incidents often have public visibility. Coordinate communication with legal and PR. Use the incident response coordination patterns from CERT/CC and from the FIRST CSIRT framework. Authentic communication beats marketing language every time.

   Practical lab. Lab 9: Author a complete AI incident response runbook for a hypothetical jailbreak that affects 10,000 users, including detection, triage, containment, recovery, and post-incident steps.

   Reading.

   • Cichonski, P., Millar, T., Grance, T., & Scarfone, K. (2012). Computer Security Incident Handling Guide (NIST SP 800-61 Rev. 2). https://doi.org/10.6028/NIST.SP.800-61r2
   • Forum of Incident Response and Security Teams. (2024). CSIRT Services Framework version 2.1. https://www.first.org/standards/frameworks/csirts/
   • AI Vulnerability Database (AVID). (2025). Incident reporting and triage. https://avidml.org/

10. Week 10 · 7h · 6 lessons

    AI Compliance and Governance Engineering

    Translate the EU AI Act, NIST AI RMF, and ISO/IEC 42001 into engineering controls. Compliance work is engineering work when it stays close to the system.

    Lessons, lab, and reading

    • EU AI Act technical implementation requirements (60 min)
      The EU AI Act categorizes systems into prohibited, high-risk, limited-risk, and minimal-risk tiers. High-risk systems require risk management, data governance, technical documentation, transparency, human oversight, and post-market monitoring. Walk Articles 9 through 15 and identify the engineering controls each maps to.
    • NIST AI RMF operationalized (50 min)
      NIST AI 100-1 organizes work into Govern, Map, Measure, and Manage. Read the framework alongside the NIST Generative AI Profile (NIST AI 600-1) and translate each subcategory into a control or evidence artifact your team owns.
    • ISO/IEC 42001 management standard (50 min)
      ISO/IEC 42001 defines an AI management system: policies, processes, roles, and continuous improvement. Compare it to ISO/IEC 27001 for information security. Many controls overlap; the AI-specific additions cover lifecycle, impact assessment, and supplier management.
    • AI impact assessments (45 min)
      EU AI Act requires Fundamental Rights Impact Assessments for certain high-risk uses. NIST AI RMF expects impact assessment as part of Map. Build an internal template that satisfies both, document review cadence, and tie it to the deployment gating checklist.
    • Audit trail engineering (50 min)
      An auditable AI system records data lineage, model version, prompt, completion, tool calls, and decision basis for actions that affect users. The technical work is logging architecture; the governance work is retention, access control, and chain-of-custody.
    • Mapping to existing programs (SOC 2, ISO 27001, HIPAA) (45 min)
      Most teams already operate under one or more compliance frameworks. AI controls extend those programs rather than replace them. Build a crosswalk so auditors and engineers see the AI-specific evidence inside the existing program.

    Practical lab. Lab 10: Pick a hypothetical high-risk AI product, classify it under the EU AI Act, map its controls to NIST AI RMF subcategories, and document gaps a team would close before launch.

    Reading.

    • European Union. (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act). https://eur-lex.europa.eu/eli/reg/2024/1689/oj
    • National Institute of Standards and Technology. (2023). AI Risk Management Framework (NIST AI 100-1). https://doi.org/10.6028/NIST.AI.100-1
    • International Organization for Standardization. (2023). ISO/IEC 42001 Information technology, Artificial intelligence, Management system. https://www.iso.org/standard/81230.html
    • National Institute of Standards and Technology. (2024). Generative AI Profile (NIST AI 600-1). https://doi.org/10.6028/NIST.AI.600-1

11. Week 11 · 6h · 6 lessons

    AI Privacy Engineering

    Apply privacy by design to AI products. Cover right-to-be-forgotten in trained models, federated learning fundamentals, and the GDPR Article 22 requirements for automated decisions.

    Lessons, lab, and reading

    • Privacy by design for AI systems (45 min)
      Cavoukian's seven principles applied to AI: proactive not reactive, privacy as default, embedded in design, full functionality, end-to-end security, visibility and transparency, respect for users. Each principle has concrete engineering implications for prompts, retention, and inference logs.
    • Right-to-be-forgotten in trained models (50 min)
      GDPR right-to-erasure applies to training data. Removing a record from a database is straightforward; removing it from a trained model is hard. Options include retraining on cleaned data, machine unlearning techniques, and differential privacy at training time. Document the limits.
    • Federated learning fundamentals (50 min)
      Federated learning trains models across distributed data without centralizing the raw data. Useful for healthcare, finance, and on-device personalization. Carries new threats including gradient leakage and model poisoning by participants. Read the McMahan et al. paper as a primary source.
    • GDPR Article 22 and automated decision-making (45 min)
      Article 22 grants individuals the right not to be subject to solely automated decisions with significant effects, with documented exceptions. Engineer human-in-the-loop checkpoints for affected decisions, log the model basis, and provide explanation interfaces.
    • Differential privacy in production AI (40 min)
      Differential privacy budgets, accountant tracking, and the impact on model utility. Not every workload benefits. Apply where the privacy harm is meaningful and the utility cost is acceptable.
    • Cross-border data transfers and AI workloads (40 min)
      AI inference often spans regions and providers. GDPR Chapter V, the EU US Data Privacy Framework, and similar regimes govern transfers. Engineer routing, residency, and provider selection so transfers happen on a documented legal basis.

    Practical lab. Lab 11: Author a privacy engineering plan for an AI product that processes user data, including DPIA outline, retention design, and a right-to-be-forgotten workflow.

    Reading.

    • Cavoukian, A. (2011). Privacy by Design: The 7 Foundational Principles. Information and Privacy Commissioner of Ontario. https://iapp.org/resources/article/privacy-by-design-the-7-foundational-principles/
    • McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Aguera y Arcas, B. (2017). Communication-efficient learning of deep networks from decentralized data (Federated Averaging). AISTATS. https://arxiv.org/abs/1602.05629
    • European Union. (2016). Regulation (EU) 2016/679 (General Data Protection Regulation), Article 22. https://eur-lex.europa.eu/eli/reg/2016/679/oj

12. Week 12 · 7h · 6 lessons

    Capstone and Certification

    Synthesize the 12 weeks into a portfolio artifact. Design and document a complete AI security architecture for a hypothetical AI deployment.

    Lessons, lab, and reading

    • Capstone scoping (45 min)
      Pick a hypothetical AI product that will surface every layer of the course: an LLM-powered application with retrieval, tool use, and a user-facing surface. Define users, data sources, model selection, and the regulatory regime that governs it.
    • Threat model and risk register (60 min)
      Author a complete threat model using the Week 1 categories. Build a risk register with each risk rated for likelihood, impact, detection, and mitigation status. Cite the OWASP LLM Top 10, MITRE ATLAS, and NIST AI RMF entries that inform each entry.
    • Architecture and controls documentation (60 min)
      Diagram the architecture and document the controls at each layer. Reference the prior weeks: prompt injection defenses (Week 2), red team plan (Week 3), eval suite (Week 4), infrastructure controls (Week 5), data governance (Week 6 and Week 11), model security (Week 7), supply chain controls (Week 8), incident response (Week 9), and compliance mapping (Week 10).
    • Compliance and governance package (45 min)
      EU AI Act classification, NIST AI RMF subcategory mapping, ISO/IEC 42001 control crosswalk, and an impact assessment outline. The package should be specific enough that an auditor could begin a real review.
    • Capstone review and submission (60 min)
      Self-review against a published rubric. Submit the capstone as a single document or repository so reviewers can evaluate it end-to-end. The capstone is the portfolio artifact you will share with hiring panels.
    • Career trajectory and credential ladder (45 min)
      AI security engineer, AI red teamer, AI infrastructure security engineer, AI governance engineer. Map the next two years: target role, certification stack (the Northeastern M.S. Applied AI specializing in Cybersecurity is one path; ISC2 AI security tracks and SANS AI security courses are others), and the artifacts you will publish to differentiate.

    Practical lab. Lab 12 (Capstone): Submit a complete AI security architecture document covering threat model, controls at each layer, eval suite, incident runbook, and compliance mapping for a hypothetical high-risk AI product.

    Reading.

    • National Institute of Standards and Technology. (2024). Generative AI Profile (NIST AI 600-1). https://doi.org/10.6028/NIST.AI.600-1
    • European Union. (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act), Annex III high-risk use cases. https://eur-lex.europa.eu/eli/reg/2024/1689/oj
    • Northeastern University. (2025). Master of Science in Applied AI specializing in Cybersecurity, program description. https://graduate.northeastern.edu/programs/mps-applied-ai/

Capstone

AI Security Architecture for a Hypothetical High-Risk AI Product

Design and document a complete AI security architecture for a hypothetical AI deployment that will trigger EU AI Act high-risk classification. The capstone integrates every prior week into a single portfolio artifact: threat model and risk register, prompt injection defenses, structured red team plan, safety evaluation suite, infrastructure controls, data governance and lineage, model security and supply chain controls, AI incident response runbook, and compliance mapping against the EU AI Act, NIST AI RMF, and ISO/IEC 42001. The artifact is the document you will share with hiring panels for AI security engineer roles.

Who it is for

• Cybersecurity engineers who want to specialize in AI system security
• Application security engineers whose products now ship LLM features
• Cloud security engineers responsible for AI inference and training pipelines
• Red teamers expanding into adversarial machine learning and AI red teaming
• Security architects designing AI governance controls under the EU AI Act, NIST AI RMF, or ISO/IEC 42001
• Detection and response engineers building monitoring for AI applications
• Career changers from machine learning engineering who want to add the security layer

Who it is not for

• Total beginners with no cybersecurity background. Start with SOC Analyst Fundamentals or Cloud Security Fundamentals first
• Practitioners looking for a PyTorch or model training course. The course uses pretrained models and assumes you know how to call them
• Buyers seeking a vendor product evaluation. The course is methodology and standards, not a tools shootout
• Professionals expecting a single proctored exam. The course awards a certificate of completion based on capstone submission, not a third-party exam

Prerequisites

• Working cybersecurity background equivalent to one to three years of practitioner experience or CompTIA Security+ plus a hands-on track such as SOC analyst, application security, or cloud security
• Basic familiarity with one large language model API (OpenAI, Anthropic, or open-source through Hugging Face) at the level of having shipped a small project
• Comfort with Python at the script level and with reading model and dataset cards
• Working knowledge of cloud infrastructure security on at least one major provider
• Willingness to commit 6 to 8 hours per week for 12 weeks

What you get

• 72 hours of original cybersecurity curriculum across 12 weekly modules
• 12 practical labs that produce reviewable portfolio artifacts
• DecipherU AI Security Engineering Certificate of Completion. Awarded on capstone submission and self-review against the published rubric. Lifetime access to course updates and the DecipherU community for ongoing peer review.
• Lifetime access to course updates as the EU AI Act, NIST AI RMF, and OWASP LLM Top 10 evolve
• DecipherU community access (Defender tier and above) for peer review of the capstone and post-course Q&A

Author

Frequently asked questions

Who is this cybersecurity AI security engineering course for?

Cybersecurity engineers specializing in AI systems, application security engineers whose products ship LLM features, cloud security engineers responsible for AI inference and training pipelines, red teamers expanding into adversarial machine learning, and security architects designing controls under the EU AI Act, NIST AI RMF, or ISO/IEC 42001. The course assumes one to three years of cybersecurity practitioner experience.

What primary sources does the cybersecurity course cite?

NIST AI Risk Management Framework (AI 100-1), NIST Generative AI Profile (AI 600-1), OWASP LLM Top 10, MITRE ATLAS, the EU AI Act consolidated text, ISO/IEC 42001, CISA AI deployment guidance, GDPR Articles 22 and Chapter V, NIST SP 800-61 for incident handling, and peer-reviewed papers from Anthropic, OpenAI, and Google DeepMind safety teams. Every claim is anchored to a public primary source.

How long does the cybersecurity course take to complete?

Roughly 60 to 80 hours of focused study across 12 weekly modules. Most learners complete it in 12 to 16 weeks at 6 to 8 hours per week. Self-paced. The capstone is the deliverable that earns the certificate of completion.

Will the course prepare me for a specific cybersecurity certification?

It is not a proctored exam prep course. The curriculum maps closely to the work products expected in AI security engineer, AI red teamer, AI infrastructure security engineer, and AI governance engineer roles, and to the Northeastern M.S. Applied AI specializing in Cybersecurity coursework. Pair it with ISC2 AI security tracks or SANS AI security courses for credentialed proctored exams.

What is the capstone deliverable for the cybersecurity course?

A documented AI security architecture for a hypothetical high-risk AI deployment. The capstone integrates threat model, prompt injection defenses, red team plan, safety evaluation suite, infrastructure controls, data and supply chain governance, AI incident response runbook, and compliance mapping against the EU AI Act, NIST AI RMF, and ISO/IEC 42001. It is the artifact you can share with hiring panels.

Do I get lifetime updates and community access?

Yes. Enrollment includes lifetime access to course updates as standards evolve (the EU AI Act, NIST AI RMF, and OWASP LLM Top 10 update on regular cycles), and access to the DecipherU community for peer review of the capstone and post-course Q&A.

Related cybersecurity content

• Cybersecurity for AI convergence area overview
• AI Security Engineer cybersecurity career guide
• AI Red Team Engineer cybersecurity career guide
• Prompt Injection Defense Specialist cybersecurity career guide
• All Cybersecurity for AI career paths
• AI for Cybersecurity convergence area (the inverse direction)