Reference Architecture Models for Big Data and Analytics

This white paper outlines the reference architecture for Big Data and Analytics, emphasizing the importance of a well-planned data platform to avoid costly corrections post-implementation. It discusses the challenges faced when best practices are overlooked during the data preparation stage and the necessity of addressing non-functional requirements and data governance in data architecture. The document presents a checklist of components essential for architecting an enterprise data platform, covering core elements like data ingestion, processing, and storage, as well as additional areas such as metadata management and data quality. The paper also defines key terms related to data architecture, including Systems of Record and Systems of Truth, and describes the enterprise city map concept, which inventories applications and services within an organization. Furthermore, it provides an overview of various reference architectures, including Lambda and Kappa architectures, and emphasizes the need for a guiding plan to design a comprehensive data platform architecture.