AI and LLMs: Making Business Process Design Talk the Talk

AI and LLMs: Making Business Process Design Talk the Talk

Ever tried to explain a complex business process – how a customer order flows from clicking 'buy' to getting a delivery notification – to someone who isn't directly involved? It's tricky! Businesses often use detailed diagrams, called process models, to map these steps out. This helps them work more efficiently, reduce errors, and improve communication.

But here's a challenge: creating and updating these diagrams often requires specialized skills in modeling languages like BPMN (Business Process Model and Notation). This creates a communication gap between the "domain experts" (the people who actually do the work and understand the process best) and the "process modelers" (the ones skilled in drawing the diagrams). Constantly translating the domain experts' knowledge into technical diagrams can be a slow and burdensome task, especially when processes need frequent updates due to changes in the business world.

Imagine if you could just talk to a computer system, tell it how your process works or how you want to change it, and it would automatically create or update the diagram for you. This is the idea behind conversational process modeling (CPM).

Talking to Your Process Model: The Power of LLMs

Recent advancements in artificial intelligence, particularly with Large Language Models (LLMs), are making this idea more feasible. These powerful AI models can understand and generate human-like text, opening up the possibility of interacting with business process management systems using natural language.

This research explores a specific area of CPM called conversational process model redesign (CPD). The goal is to see if LLMs can help domain experts easily modify existing process models through iterative conversations. Think of it as having an AI assistant that understands your requests to change a process diagram.

How Does Conversational Redesign Work with AI?

The proposed CPD approach takes a process model and a redesign request from a user in natural language. Instead of the LLM just guessing how to make the change, the system uses a structured, multi-step approach based on established "process change patterns" from existing research.

Here's the simplified breakdown:

1. Identify the Pattern: The AI (the LLM) first tries to figure out which standard "change pattern" the user's request corresponds to. Change patterns are like predefined ways to modify a process model, such as inserting a new step, deleting a step, or adding a loop. They simplify complex changes into understandable actions.
2. Derive the Meaning: If a pattern is identified, the LLM then clarifies the specific details (the "meaning") of the change based on the user's wording. For example, if the pattern is "insert task," the meaning would specify which task to insert and where.
3. Apply the Change: Finally, the AI system applies the derived meaning (the specific, parameterized change pattern) to the existing process model to create the redesigned version.

This multi-step process, leveraging the LLM's understanding of language and predefined patterns, aims to make changes explainable and reproducible. The researchers also identified and proposed several new patterns specifically needed for interacting with process models through conversation, like splitting a single task into multiple tasks or merging several tasks into one.

Testing the AI: What Did They Find?

To see how well this approach works and how users interact with it, the researchers conducted an extensive evaluation. They asked 64 people with varying modeling skills to describe how they would transform an initial process model into a target model using natural language, as if talking to an AI chatbot. The researchers then tested these user requests with different LLMs (specifically, gpt-4o, gemini-1.5-pro, and mistral-large-latest) to see if the AI could correctly understand, identify, and apply the intended changes.

The results offered valuable insights into the potential and challenges of using artificial intelligence for this task.

Successes:

• Some change patterns were successfully implemented by the LLMs based on user requests in a significant number of cases, demonstrating the feasibility of CPD. This included some of the newly proposed patterns as well as existing ones.

Challenges and Failures:

• User Wording: A big reason for failure was user wording. Users sometimes struggled to describe the desired changes clearly or completely, making it hard for the LLM to identify the pattern or derive the specific meaning. For instance, users might use vague terms or describe complex changes in a way that didn't map cleanly to a single pattern. This indicates that users might need support or guidance from the AI system to formulate clearer requests.
• LLM Interpretation: Even when a pattern was identified and meaning derived, the LLMs didn't always apply the changes correctly. Sometimes the AI misidentified the pattern based on the wording, or simply failed to implement the correct change, especially with more complex patterns. This suggests issues with the LLM's understanding or the way the prompts were designed.
• Pattern Ambiguity: In some cases, the user's wording could be interpreted as multiple different patterns, or the definitions of the patterns themselves weren't clear enough for the AI to consistently choose the right one. This highlights the need to refine pattern definitions for conversational contexts.

Interestingly, the study also revealed common user behaviors like asking to delete everything and start over, or requesting to undo a previous change. These aren't standard process change patterns but suggest interaction patterns the AI system should support.

While some LLMs performed better than others (gemini and gpt generally had higher success rates than mistral and followed instructions more closely), the overall trends in why things failed were similar across the different AI models.

The Future: Better AI, Better Processes

This research demonstrates that using AI, specifically LLMs, for conversational process model redesign is possible and holds great potential for making process modeling more accessible to domain experts, helping to bridge that persistent communication gap.

However, it also highlights that there are clear areas for improvement. Future work needs to focus on:

• Developing ways for the AI agent to help users provide clearer and more complete requests.
• Improving the LLMs' ability to correctly interpret and apply changes, possibly by combining AI with more traditional, deterministic methods for applying the identified changes.
• Clarifying the definitions of change patterns to reduce ambiguity for both users and the AI.

By addressing these challenges, artificial intelligence can become a powerful tool, enabling domain experts to directly shape and improve their business processes through simple conversation, leading to more accurate models and increased efficiency.