Real-time Data Replication Solution for Azure SQL

This document is a technical report detailing the development of a real-time data replication solution for a prescription benefit facilitator. The client faced challenges in designing and implementing a cost-effective solution for data replication between an on-premises DB2 LUW source and an Azure SQL target. XTIVIA was engaged to create a custom data replication solution utilizing Debezium, an open-source change data capture platform, and the Kafka Connect framework. The report outlines the installation, configuration, and testing processes involved in setting up Kafka Connect, integrating it with Azure Event Hubs, and executing Azure Functions for data consumption. It also discusses various change data capture options and their respective advantages and disadvantages. The document concludes with the business results achieved, including a configurable real-time data replication solution and enhanced knowledge of change data capture methods for the client.