AI Engineering Mastery bibliography

peer-reviewed journal articles · 4

1. Chen, L., Zaharia, M., and Zou, J. (2024). FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance. Transactions on Machine Learning Research. https://openreview.net/forum?id=cSimKw5p6R
2. Gou, J., Yu, B., Maybank, S. J., and Tao, D. (2021). Knowledge Distillation: A Survey. International Journal of Computer Vision, 129(6), 1789-1819. doi:10.1007/s11263-021-01453-z
3. Liang, P., Bommasani, R., Lee, T., et al. (2023). Holistic Evaluation of Language Models. Transactions on Machine Learning Research. https://openreview.net/forum?id=iO4LZibEqW
4. Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., Zhao, W. X., Wei, Z., and Wen, J.-R. (2024). A Survey on Large Language Model based Autonomous Agents. Frontiers of Computer Science, 18(6). doi:10.1007/s11704-024-40231-1

peer-reviewed conference papers · 18

1. Bronson, N., Aghayev, A., Charapko, A., and Zhu, T. (2021). Metastable Failures in Distributed Systems. Proceedings of the Workshop on Hot Topics in Operating Systems (HotOS '21), 221-227. doi:10.1145/3458336.3465286
2. Cormack, G. V., Clarke, C. L. A., and Buettcher, S. (2009). Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods. ACM SIGIR Conference. https://doi.org/10.1145/1571941.1572114
3. Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. (2023). QLoRA: Efficient Finetuning of Quantized LLMs. Advances in Neural Information Processing Systems 36 (NeurIPS 2023). https://proceedings.neurips.cc/paper_files/paper/2023/hash/1feb87871436031bdc0f2beaa62a049b-Abstract-Conference.html
4. Gao, L., Ma, X., Lin, J., and Callan, J. (2023). Precise Zero-Shot Dense Retrieval without Relevance Labels. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1762-1777. doi:10.18653/v1/2023.acl-long.99
5. Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., and Fritz, M. (2023). Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec '23), 79-90. doi:10.1145/3605764.3623985
6. Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2022). LoRA: Low-Rank Adaptation of Large Language Models. International Conference on Learning Representations (ICLR 2022). https://iclr.cc/virtual/2022/poster/6319
7. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., and Kiela, D. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems 33 (NeurIPS 2020). https://proceedings.neurips.cc/paper_files/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html
8. Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., and Gebru, T. (2019). Model Cards for Model Reporting. Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19), 220-229. doi:10.1145/3287560.3287596
9. Muennighoff, N., Tazi, N., Magne, L., and Reimers, N. (2023). MTEB: Massive Text Embedding Benchmark. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2014-2037. doi:10.18653/v1/2023.eacl-main.148
10. Ouyang, L., Wu, J., Jiang, X., et al. (2022). Training Language Models to Follow Instructions with Human Feedback. Advances in Neural Information Processing Systems 35 (NeurIPS 2022). https://proceedings.neurips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html
11. Qi, X., Huang, K., Panda, A., Henderson, P., Wang, M., and Mittal, P. (2024). Visual Adversarial Examples Jailbreak Aligned Large Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 38(19), 21527-21536. doi:10.1609/aaai.v38i19.30150
12. Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., and Sutskever, I. (2023). Robust Speech Recognition via Large-Scale Weak Supervision (Whisper). Proceedings of the 40th International Conference on Machine Learning (ICML 2023), PMLR 202, 28492-28518. https://proceedings.mlr.press/v202/radford23a.html
13. Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., and Finn, C. (2023). Direct Preference Optimization: Your Language Model is Secretly a Reward Model. Advances in Neural Information Processing Systems 36 (NeurIPS 2023). https://proceedings.neurips.cc/paper_files/paper/2023/hash/a85b405ed65c6477a4fe8302b5e06ce7-Abstract-Conference.html
14. Reimers, N. and Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 3982-3992. doi:10.18653/v1/D19-1410
15. Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., and Hajishirzi, H. (2023). Self-Instruct: Aligning Language Models with Self-Generated Instructions. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 13484-13508. doi:10.18653/v1/2023.acl-long.754
16. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. International Conference on Learning Representations (ICLR 2023). https://iclr.cc/virtual/2023/poster/11003
17. Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., Zhang, H., Gonzalez, J. E., and Stoica, I. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. Advances in Neural Information Processing Systems 36 (NeurIPS 2023), Datasets and Benchmarks Track. https://proceedings.neurips.cc/paper_files/paper/2023/hash/91f18a1287b398d378ef22505bf41832-Abstract-Datasets_and_Benchmarks.html
18. Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., Zhang, S., Ghosh, G., Lewis, M., Zettlemoyer, L., and Levy, O. (2023). LIMA: Less Is More for Alignment. Advances in Neural Information Processing Systems 36 (NeurIPS 2023). https://proceedings.neurips.cc/paper_files/paper/2023/hash/ac662d74829e4407ce1d126477f4a03a-Abstract-Conference.html

books or book chapters · 1

1. Nygard, M. (2018). Release It! Design and Deploy Production-Ready Software (2nd ed.). Pragmatic Bookshelf. https://pragprog.com/titles/mnee2/release-it-second-edition/

preprints · 3

1. Bagdasaryan, E., Hsieh, T.-Y., Nassi, B., and Shmatikov, V. (2023). Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs. arXiv preprint. https://arxiv.org/abs/2307.10490
2. Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., and Wang, H. (2024). Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv preprint. https://arxiv.org/abs/2312.10997
3. Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv preprint (presented at the NIPS 2014 Deep Learning Workshop; no proceedings volume or review process confirmed). https://arxiv.org/abs/1503.02531

government or legal documents · 6

1. National Institute of Standards and Technology (2023). Artificial Intelligence Risk Management Framework (NIST AI 100-1). U.S. Department of Commerce. https://doi.org/10.6028/NIST.AI.100-1
2. National Institute of Standards and Technology (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). U.S. Department of Commerce. https://doi.org/10.6028/NIST.AI.100-1
3. National Institute of Standards and Technology (2024). Artificial Intelligence Risk Management Framework: Generative AI Profile (NIST AI 600-1). U.S. Department of Commerce. https://doi.org/10.6028/NIST.AI.600-1
4. National Institute of Standards and Technology (2024). Generative AI Profile (NIST AI 600-1): Measure Function. U.S. Department of Commerce. https://doi.org/10.6028/NIST.AI.600-1
5. National Institute of Standards and Technology (2024). AI RMF Generative AI Profile (NIST AI 600-1). U.S. Department of Commerce. https://doi.org/10.6028/NIST.AI.600-1
6. National Institute of Standards and Technology (2024). AI RMF Generative AI Profile: Measure Function. U.S. Department of Commerce. https://doi.org/10.6028/NIST.AI.600-1

standards or frameworks · 10

1. Cloud Security Alliance (2024). AI Controls Matrix. Cloud Security Alliance. https://cloudsecurityalliance.org/research/working-groups/ai-controls
2. MITRE Corporation (2024). ATLAS: Adversarial Threat Landscape for Artificial-Intelligence Systems. MITRE. https://atlas.mitre.org/
3. MITRE Corporation (2024). ATLAS Tactics: ML Supply Chain Compromise and AI Model Inference. MITRE. https://atlas.mitre.org/tactics/
4. MITRE Corporation (2024). ATLAS Adversarial Threat Landscape for AI Systems. MITRE. https://atlas.mitre.org/
5. MITRE Corporation (2024). ATLAS Matrices. MITRE. https://atlas.mitre.org/matrices/
6. OWASP Foundation (2024). OWASP Top 10 for Large Language Model Applications. OWASP. https://owasp.org/www-project-top-10-for-large-language-model-applications/Cited in 2 modules
7. OWASP Foundation (2024). LLM01: Prompt Injection. OWASP. https://owasp.org/www-project-top-10-for-large-language-model-applications/
8. OWASP Foundation (2024). LLM06: Sensitive Information Disclosure. OWASP. https://owasp.org/www-project-top-10-for-large-language-model-applications/
9. OWASP Foundation (2024). LLM08: Excessive Agency. OWASP. https://owasp.org/www-project-top-10-for-large-language-model-applications/
10. OpenTelemetry (2024). Semantic Conventions for GenAI. Cloud Native Computing Foundation. https://opentelemetry.io/docs/specs/semconv/gen-ai/

vendor documentation or web pages · 40

1. Meta AI (2024). Llama Model Card. Meta Platforms. https://llama.meta.com/
2. Anthropic (2024). Building with Claude: Engineering Documentation. Anthropic. https://docs.anthropic.com/
3. Anthropic (2024). Models Overview. Anthropic. https://docs.anthropic.com/en/docs/about-claude/models
4. Anthropic (2024). Prompt Engineering Overview. Anthropic. https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview
5. Anthropic (2024). Tool Use with Claude. Anthropic. https://docs.anthropic.com/en/docs/build-with-claude/tool-use
6. Anthropic (2024). Prompt Caching. Anthropic. https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching
7. Anthropic (2024). Be Clear, Direct, and Detailed. Anthropic. https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/be-clear-and-direct
8. Anthropic (2024). Introducing Contextual Retrieval. Anthropic. https://www.anthropic.com/news/contextual-retrieval
9. Anthropic (2024). Citations. Anthropic. https://docs.anthropic.com/en/docs/build-with-claude/citations
10. Anthropic (2024). Evaluation Tool. Anthropic. https://docs.anthropic.com/en/docs/test-and-evaluate/eval-tool
11. Anthropic (2024). Building Effective Agents. Anthropic. https://www.anthropic.com/research/building-effective-agentsCited in 2 modules
12. Anthropic (2024). When to Fine-Tune Claude. Anthropic. https://docs.anthropic.com/en/docs/build-with-claude/fine-tuning
13. Anthropic (2024). Vision. Anthropic. https://docs.anthropic.com/en/docs/build-with-claude/vision
14. Anthropic (2024). Streaming Messages. Anthropic. https://docs.anthropic.com/en/api/messages-streaming
15. Anthropic (2024). Pricing. Anthropic. https://www.anthropic.com/pricing
16. Anthropic (2024). Message Batches API. Anthropic. https://docs.anthropic.com/en/docs/build-with-claude/batch-processing
17. Anthropic (2024). Mitigating Jailbreaks and Prompt Injections. Anthropic. https://docs.anthropic.com/en/docs/test-and-evaluate/strengthen-guardrails/mitigate-jailbreaks
18. Anthropic (2024). Claude Model Card. Anthropic. https://www.anthropic.com/system-cards
19. Apple (2024). Apple Intelligence Foundation Language Models. Apple. https://machinelearning.apple.com/research/apple-intelligence-foundation-language-models
20. Google Cloud (2024). Document AI. Google. https://cloud.google.com/document-ai/docs
21. Cloudflare (2024). AI Gateway Documentation. Cloudflare. https://developers.cloudflare.com/ai-gateway/
22. Cohere (2024). Rerank Documentation. Cohere. https://docs.cohere.com/docs/rerank
23. DecipherU (2026). AI Engineering Mastery Capstone Rubric. DecipherU. https://decipheru.com/courses/ai-engineering-mastery
24. Google DeepMind (2024). Gemini API: Models. Google. https://ai.google.dev/gemini-api/docs/models/gemini
25. Hodgson, P. (2017). Feature Toggles (aka Feature Flags). martinfowler.com. https://martinfowler.com/articles/feature-toggles.html
26. Microsoft (2024). PyRIT: Python Risk Identification Tool for generative AI. Microsoft. https://github.com/Azure/PyRIT
27. OpenAI (2024). OpenAI Platform Documentation. OpenAI. https://platform.openai.com/docs
28. OpenAI (2024). Prompt Engineering Guide. OpenAI. https://platform.openai.com/docs/guides/prompt-engineering
29. OpenAI (2024). Structured Outputs. OpenAI. https://platform.openai.com/docs/guides/structured-outputs
30. OpenAI (2024). Prompt Caching. OpenAI. https://platform.openai.com/docs/guides/prompt-caching
31. OpenAI (2024). Embeddings Guide. OpenAI. https://platform.openai.com/docs/guides/embeddings
32. OpenAI (2024). Fine-Tuning Guide. OpenAI. https://platform.openai.com/docs/guides/fine-tuning
33. OpenAI (2024). Vision. OpenAI. https://platform.openai.com/docs/guides/vision
34. OpenAI (2024). Realtime API. OpenAI. https://platform.openai.com/docs/guides/realtime
35. OpenAI (2024). Pricing. OpenAI. https://openai.com/api/pricing/
36. OpenAI (2024). Batch API. OpenAI. https://platform.openai.com/docs/guides/batch
37. OpenAI (2024). Moderation Guide. OpenAI. https://platform.openai.com/docs/guides/moderation
38. Pinecone (2024). Pinecone Documentation. Pinecone Systems. https://docs.pinecone.io/
39. Qdrant (2024). Qdrant Documentation. Qdrant. https://qdrant.tech/documentation/
40. Vercel (2024). Vercel AI Gateway. Vercel. https://vercel.com/docs/ai-gateway