Summary
This case study details the AI penetration testing conducted by the Tevora Threat team on a new chatbot developed by a SaaS provider of financial services solutions. The objective of the testing was to identify potential vulnerabilities in the chatbot, which utilizes Generative AI (GenAI) to assist customers in accessing confidential data. The Tevora team employed a combination of technical skills and social engineering tactics to simulate threat actor behavior, aiming to trick the AI into revealing sensitive information. The testing involved assessing the chatbot against ten specific threats related to large language models (LLMs). During this process, one significant vulnerability was discovered, allowing the client to rectify the issue before the chatbot's full market launch. The study raises critical questions regarding the security of AI applications, including the risks of data exposure and unintended consequences of AI misuse.
