Summary
This document is a case study detailing the implementation of a centralized big data platform within a logistics company. The company, which operates with a significant volume of freight and infrastructure data, faced challenges due to the existence of multiple business intelligence platforms that resulted in high integration costs and complexity. To address these issues, a pilot project was initiated to demonstrate the advantages of a single data platform. The project aimed to establish a group-wide concept for handling big data challenges, including the development of an on-premises big data lake and the piloting of an initial use case. The implementation involved the use of industry-standard Lambda architecture and various technologies such as Logstash, Kafka, and Hive for data processing and analysis. The case study outlines the challenges faced, the solutions developed, and the benefits realized, including a high level of acceptance for the new platform and the potential for further development in a production environment.
