Taxonomy of Failure Modes in Agentic AI Systems

This technical report presents a detailed analysis of failure modes in agentic AI systems, conducted by Microsoft's AI Red Team (AIRT). The report begins by defining agentic AI systems and outlines the importance of understanding their functionality and common patterns. It identifies various safety and security failure modes unique to these systems, particularly in multi-agent contexts, and discusses the implications of these failure modes on system performance and user safety. The report also covers mitigations and design considerations that can be employed to reduce risks associated with these failures. A case study on a memory poisoning attack on an AI email assistant is included, providing insights into the mechanisms of such attacks and the challenges faced in mitigating them. The report concludes with a taxonomy detailing both novel and existing failure modes, emphasizing the need for robust testing and security measures in the development of agentic AI systems.