Iterate.ai Case Study on AMD Ryzen AI PRO Processors

This case study details the collaboration between Iterate.ai and AMD to optimize the Generate AI platform for enterprise use. The document outlines the challenges enterprises face in using generative AI, particularly regarding data security and rising token costs associated with public large language models (LLMs). Iterate.ai's solution leverages AMD Ryzen AI PRO processors to run private LLMs locally, ensuring data sovereignty and reducing cloud costs. The study presents the performance metrics achieved, including the ability to run 32 billion parameter models with a 32K context window at approximately 60-80 tokens per second. It also discusses the integration of AMD Lemonade Server to streamline local LLM development, enhancing efficiency and reducing resource requirements. The case study emphasizes the importance of security and cost predictability in AI deployments, highlighting how the partnership with AMD enables organizations to maintain control over sensitive data while optimizing performance.