← Back to Paper Radar

BPMN Assistant: An LLM-Based Approach to Business Process Modeling

Josip Tomo Licardo, Nikola Tanković, D. Etinger
Applied Sciences | 2025
Having LLMs edit BPMN diagrams through atomic function calls on a compact JSON intermediate representation is more reliable and much faster than having them generate or rewrite raw BPMN XML. The gains are largest on editing tasks.

Problem Statement

Business process modeling with BPMN normally requires specialist tooling and expertise, and LLM-based natural-language modeling usually has the model emit BPMN XML directly. XML is verbose, so it is slow and expensive to generate. It is also fragile: complex modifications that force the model to regenerate or patch large documents often introduce syntax or structural errors. This limits interactive, conversational process modeling and is an instance of the broader problem of LLMs editing large structured artifacts.

Key Novelty

  • A purpose-built JSON intermediate representation for BPMN that abstracts away XML verbosity while keeping the semantics needed for executable process models.
  • An editing paradigm built on atomic operations exposed via function calling, so the model issues small targeted edits instead of regenerating the whole document.
  • A head-to-head comparison of JSON-IR versus direct XML manipulation across several current LLMs (GPT-5.1, Claude 4.5 Sonnet, DeepSeek V3), measuring editing success, semantic fidelity via process-mining conformance checking, latency and token usage.

Evaluation Highlights

  • Editing tasks: the JSON-based approach achieved higher or equivalent success rates than direct XML across all evaluated models.
  • Conformance checking: average F1 was 0.72 for JSON versus 0.69 for XML. However, the frontier models (GPT-5.1, Claude 4.5 Sonnet) achieved higher precision with direct XML generation.
  • Efficiency: generation latency fell by about 43% and output tokens by more than 75%, at the cost of larger input context.

Signal Assessment

4/10 This is a solid, practically useful applied contribution that validates a known design pattern (compact intermediate representation plus tool-based atomic edits) in the BPMN domain, with a clear efficiency win. The semantic-quality gains are modest (+0.03 F1), and frontier models sometimes do better with native XML, so it is not a significant methodological advance.

Methodology

  1. Design a JSON-based intermediate representation of BPMN process elements (e.g., tasks, gateways, events, flows) that is more compact and LLM-friendly than BPMN XML.
  2. Define a set of atomic editing operations over the representation and expose them to the LLM through function calling, enabling incremental, targeted modifications from natural-language requests.
  3. Convert the JSON representation into standard BPMN for rendering and execution. This step is implied by the need to produce BPMN diagrams; the abstract does not describe it.
  4. Build a baseline in which the same LLMs create and edit BPMN by generating or manipulating XML directly.
  5. Evaluate both approaches across several LLMs on creation and editing tasks, measuring task success rate, conformance-checking precision/recall/F1 against reference process behavior, latency, and input/output token counts.

System Components

JSON Intermediate Representation

A compact, structured encoding of BPMN process elements and their relationships that replaces verbose XML as the format the LLM reasons over.

Atomic Editing Functions (Function Calling Interface)

Tool definitions for fine-grained operations on the process model, letting the LLM make localized edits instead of regenerating the full diagram.

LLM Backend

Interchangeable models (GPT-5.1, Claude 4.5 Sonnet, DeepSeek V3, among others) that interpret natural-language requests and produce JSON structures or function calls.

BPMN Rendering/Conversion Layer

Translates the JSON representation into standard BPMN for visualization and execution. Its existence is implied by the tool producing BPMN diagrams; the abstract does not detail it.

Conformance Checking Evaluation

A process-mining-based assessment that tests whether generated models preserve the intended executable behavior, reported as precision, recall and F1.

Results

Metric/Benchmark Baseline This Paper Delta
Editing task success rate Direct XML manipulation (lower or equal) JSON IR + function calling (higher or equal across all models) Improvement or parity on every evaluated model
Conformance checking F1 (average across models) 0.69 (XML) 0.72 (JSON) +0.03
Conformance precision (GPT-5.1, Claude 4.5 Sonnet) Higher with direct XML Lower with JSON Regression for frontier models
Generation latency Direct XML (reference) About 43% lower ~-43%
Output token count Direct XML (reference) More than 75% fewer tokens >-75%
Input context size Direct XML (smaller) JSON approach (larger, due to schema/function definitions) Increase (trade-off)

Key Takeaways

  • When LLMs must edit large structured artifacts (diagrams, configs, documents), expose a compact domain-specific representation with atomic tool calls rather than asking for full-document regeneration. This improves edit reliability and cuts output tokens sharply.
  • Trading more input context for far fewer output tokens is usually worthwhile for interactive tools. Output tokens dominate latency and are typically priced higher, which explains the ~43% speedup despite larger prompts.
  • The benefit of an intermediate representation depends on the model: the strongest frontier models can still achieve better precision in native formats like XML. Benchmark both paths per model, or use a hybrid of native-format generation for creation and IR-based edits for modifications.
  • Evaluate generated structured outputs on semantic or behavioral fidelity (e.g., conformance checking), not just syntactic validity. The format choice affects precision and recall differently even when outputs parse correctly.

Abstract

This paper presents BPMN Assistant, a tool that leverages Large Language Models for natural language-based creation and editing of BPMN diagrams. While direct XML generation is common, it is verbose, slow, and prone to syntax errors during complex modifications. We introduce a specialized JSON-based intermediate representation designed to facilitate atomic editing operations through function calling. We evaluate our approach against direct XML manipulation using a suite of state-of-the-art models, including GPT-5.1, Claude 4.5 Sonnet, and DeepSeek V3. Results demonstrate that the JSON-based approach significantly outperforms direct XML in editing tasks, achieving higher or equivalent success rates across all evaluated models. Conformance checking evaluation confirms that generated models preserve executable semantics, with JSON achieving an average F1 score of 0.72 compared to 0.69 for XML, though frontier models like GPT-5.1 and Claude 4.5 Sonnet demonstrated superior precision with direct XML generation. Furthermore, despite requiring more input context, our approach reduces generation latency by approximately 43% and output token count by over 75%, offering a more reliable and responsive solution for interactive process modeling.

Generated automatically from available metadata and abstract on 2026-11-09.