Summary
This document is a technical report detailing the AI Computing Broker (ACB) developed by Fujitsu, which aims to address the challenge of underutilized GPU resources in enterprise AI infrastructures. The report outlines how ACB dynamically allocates GPUs based on workload activity, significantly enhancing GPU utilization and reducing idle time. It presents the architecture of ACB, which includes runtime-aware GPU scheduling middleware that integrates seamlessly with existing AI applications without requiring code changes. The report also discusses the efficiency gains achieved through ACB, including up to 45% increased GPU utilization for complex AI workloads. Furthermore, it highlights various use cases, demonstrating the impact of ACB on AI model training throughput and memory oversubscription. The document emphasizes the importance of optimizing GPU resources to maximize return on investment in AI infrastructure, particularly as the AI infrastructure market continues to grow rapidly. ACB is positioned as a solution to streamline GPU resource allocation and enhance overall AI development speed.
