Synthetic Trace Generation for Modeling Operations

This technical report presents a conceptual framework aimed at generating synthetic traces of modeling operations using large language models (LLMs) within the context of model-driven engineering (MDE). The report outlines the challenges faced in producing accurate software models, particularly the need for extensive training data, which is often limited due to privacy concerns and the complexity of real-world systems. The proposed framework integrates modeling event logs and intelligent modeling assistants to facilitate the generation of relevant modeling operations. The authors evaluate the capabilities of various LLMs, including GPT-4, in generating realistic modeling operations and assess their effectiveness in training intelligent modeling assistants. The findings indicate that while LLMs can produce useful synthetic data, human-generated traces still yield higher accuracy. The report also discusses the implications of these findings for industrial applications and the potential for LLMs to support the modeling process in scenarios where traditional data is unavailable.