ZOO: A Simulated Environment for Web Agent Evaluation

This technical report presents ZOO, a simulated web environment designed for evaluating web agents, which are AI systems capable of executing complex online workflows. The report outlines the challenges associated with testing these agents in real-world scenarios, including security risks and the need for reproducibility. ZOO addresses these issues by providing a controlled environment that mimics the open web while allowing for state observability and deterministic resets. The document details the architecture and design principles of ZOO, emphasizing its capability to support interconnected applications and realistic workflows. It also describes the setup and operational usage of ZOO, including a demonstration of an email-based indirect prompt injection attack to illustrate its effectiveness in security testing. The report concludes with a discussion on the feasibility and limitations of ZOO, suggesting its potential as a foundation for standardized benchmarks in web agent security evaluation.