This whitepaper discusses the integration of Large Language Models (LLMs) with knowledge graphs and data catalogs to enhance the effectiveness of generative AI. It outlines the limitations of LLMs, including issues of accuracy, explainability, and governance, which can hinder organizations from fully trusting AI-generated results. The document emphasizes the importance of a data catalog built on a knowledge graph as a solution to these challenges. By providing a structured and governed platform for data, organizations can improve the relevance and correctness of LLM outputs. The paper details how this integration can lead to increased accuracy, clearer explainability, and governed responses, thereby enabling organizations to leverage AI more effectively. The conclusion reinforces the necessity of adopting a data catalog approach to ensure successful AI implementation and to mitigate risks associated with generative AI technologies.