Public Safety Analytics Data Platform Case Study

This case study details the challenges faced by a public safety analytics company in building a data analytics platform for law enforcement agencies. The company encountered significant barriers due to legacy systems that were often 10-20 years old, which hampered data access and analysis. The document outlines the requirements for a solution that would normalize data across departments, enhance searchability, and maintain security. The company selected Dremio for its speed, flexibility, accessibility, security, and scalability. The implementation involved using Dremio on Kubernetes with Amazon S3 as a data lake, employing Apache Iceberg format for data management. The architecture included Python-based ETL processes, Airflow for orchestration, and Elasticsearch for search capabilities. The results showed a dramatic increase in efficiency, allowing the company to scale from one to over twenty customers while processing four terabytes of data daily. The case study concludes with future plans for enhancing the platform's capabilities and integrating additional data systems.