BPMN Assistant: An LLM-Based Approach to Business Process Modeling
Problem Statement
Business process modeling with BPMN normally requires specialist tooling and expertise, and LLM-based natural-language modeling usually has the model emit BPMN XML directly. XML is verbose, so it is slow and expensive to generate. It is also fragile: complex modifications that force the model to regenerate or patch large documents often introduce syntax or structural errors. This limits interactive, conversational process modeling and is an instance of the broader problem of LLMs editing large structured artifacts.
Key Novelty
- A purpose-built JSON intermediate representation for BPMN that abstracts away XML verbosity while keeping the semantics needed for executable process models.
- An editing paradigm built on atomic operations exposed via function calling, so the model issues small targeted edits instead of regenerating the whole document.
- A head-to-head comparison of JSON-IR versus direct XML manipulation across several current LLMs (GPT-5.1, Claude 4.5 Sonnet, DeepSeek V3), measuring editing success, semantic fidelity via process-mining conformance checking, latency and token usage.
Evaluation Highlights
- Editing tasks: the JSON-based approach achieved higher or equivalent success rates than direct XML across all evaluated models.
- Conformance checking: average F1 was 0.72 for JSON versus 0.69 for XML. However, the frontier models (GPT-5.1, Claude 4.5 Sonnet) achieved higher precision with direct XML generation.
- Efficiency: generation latency fell by about 43% and output tokens by more than 75%, at the cost of larger input context.
Signal Assessment
Methodology
- Design a JSON-based intermediate representation of BPMN process elements (e.g., tasks, gateways, events, flows) that is more compact and LLM-friendly than BPMN XML.
- Define a set of atomic editing operations over the representation and expose them to the LLM through function calling, enabling incremental, targeted modifications from natural-language requests.
- Convert the JSON representation into standard BPMN for rendering and execution. This step is implied by the need to produce BPMN diagrams; the abstract does not describe it.
- Build a baseline in which the same LLMs create and edit BPMN by generating or manipulating XML directly.
- Evaluate both approaches across several LLMs on creation and editing tasks, measuring task success rate, conformance-checking precision/recall/F1 against reference process behavior, latency, and input/output token counts.
System Components
A compact, structured encoding of BPMN process elements and their relationships that replaces verbose XML as the format the LLM reasons over.
Tool definitions for fine-grained operations on the process model, letting the LLM make localized edits instead of regenerating the full diagram.
Interchangeable models (GPT-5.1, Claude 4.5 Sonnet, DeepSeek V3, among others) that interpret natural-language requests and produce JSON structures or function calls.
Translates the JSON representation into standard BPMN for visualization and execution. Its existence is implied by the tool producing BPMN diagrams; the abstract does not detail it.
A process-mining-based assessment that tests whether generated models preserve the intended executable behavior, reported as precision, recall and F1.
Results
| Metric/Benchmark | Baseline | This Paper | Delta |
|---|---|---|---|
| Editing task success rate | Direct XML manipulation (lower or equal) | JSON IR + function calling (higher or equal across all models) | Improvement or parity on every evaluated model |
| Conformance checking F1 (average across models) | 0.69 (XML) | 0.72 (JSON) | +0.03 |
| Conformance precision (GPT-5.1, Claude 4.5 Sonnet) | Higher with direct XML | Lower with JSON | Regression for frontier models |
| Generation latency | Direct XML (reference) | About 43% lower | ~-43% |
| Output token count | Direct XML (reference) | More than 75% fewer tokens | >-75% |
| Input context size | Direct XML (smaller) | JSON approach (larger, due to schema/function definitions) | Increase (trade-off) |
Key Takeaways
- When LLMs must edit large structured artifacts (diagrams, configs, documents), expose a compact domain-specific representation with atomic tool calls rather than asking for full-document regeneration. This improves edit reliability and cuts output tokens sharply.
- Trading more input context for far fewer output tokens is usually worthwhile for interactive tools. Output tokens dominate latency and are typically priced higher, which explains the ~43% speedup despite larger prompts.
- The benefit of an intermediate representation depends on the model: the strongest frontier models can still achieve better precision in native formats like XML. Benchmark both paths per model, or use a hybrid of native-format generation for creation and IR-based edits for modifications.
- Evaluate generated structured outputs on semantic or behavioral fidelity (e.g., conformance checking), not just syntactic validity. The format choice affects precision and recall differently even when outputs parse correctly.
Abstract
This paper presents BPMN Assistant, a tool that leverages Large Language Models for natural language-based creation and editing of BPMN diagrams. While direct XML generation is common, it is verbose, slow, and prone to syntax errors during complex modifications. We introduce a specialized JSON-based intermediate representation designed to facilitate atomic editing operations through function calling. We evaluate our approach against direct XML manipulation using a suite of state-of-the-art models, including GPT-5.1, Claude 4.5 Sonnet, and DeepSeek V3. Results demonstrate that the JSON-based approach significantly outperforms direct XML in editing tasks, achieving higher or equivalent success rates across all evaluated models. Conformance checking evaluation confirms that generated models preserve executable semantics, with JSON achieving an average F1 score of 0.72 compared to 0.69 for XML, though frontier models like GPT-5.1 and Claude 4.5 Sonnet demonstrated superior precision with direct XML generation. Furthermore, despite requiring more input context, our approach reduces generation latency by approximately 43% and output token count by over 75%, offering a more reliable and responsive solution for interactive process modeling.