Automatically Surfacing Opportunities for Improvements in Internet-Scale Applications

This technical report articulates a vision for automatically surfacing opportunities for improvements in Internet-scale applications, addressing the challenges of observability data. It outlines the limitations of current commercial tools in effectively identifying business performance enhancement opportunities across diverse user behaviors. The report presents a systematic approach to derive insights from telemetry data, emphasizing the need for a shift from predefined static attributes to on-demand data slicing. Key challenges identified include hypothesis generation across a vast attribute space, scalable computation of derived attributes, and the development of an opportunity-finding mechanism utilizing machine learning. A proof-of-concept prototype is described, which integrates hypothesis generation and an efficient attributes computation engine aimed at revealing previously overlooked improvement opportunities. This groundwork aims to enable continuous performance enhancements in digital services by optimizing user engagement and overall business outcomes.