Reduce Digital Transformation failures with augmented communication between Business and IT

Integration of Business Process Diagrams with Catalogues for Business Processes and Information Structures

Digital Transformation Projects can be implemented in less time, of higher quality and with reduced risk of failing, by empowering Subject Matter Experts and Business users with a platform to specify Business Logic without assistance from IT- and Data Teams. In this article you will learn the basic functional requirements of such a platform.

>84% of all Digital Transformation projects are failing (forbes 2022), and the main reason can be traced back to miscommunication between the Business and IT. Subject Matter Experts (SME) and Business Users (BU) do not have a standard procedure in place to specify requirements towards IT- and Data Teams (ITDT), which therefore becomes vague and ambiguous. After the Digital Solution is developed, ITDT in return becomes owners of Business Logic they do not truly understand.  

Miscommunication can be removed if SME and BU have access to a platform where they can organize Business Logic and specify their requirements to IT solutions without assistance from ITDT. A successful Business Logic Platform requires integration of three main components:

1.      A visual interface for creating Business Process Diagrams.

2.      A Catalogue to capture Business Process details.

3.      A Catalogue to capture Information details.

Examples throughout this article are created using EICORE, a Business Logic Platform being developed using the described principles. The visual interface for the Business Process Diagrams is based on an open standard, whereas the Catalogues to capture the details for Business Processes and its associated information is developed by EI-Group. Success is, however, primarily tied to the principles and methodology, so a similar outcome can also be obtained with alternative components.

Empower Subject Matter Experts and Business Users to create specifications of requirements prior to any implementation.

Miscommunication between Business and IT is the main driver for failed Digitalization projects. The root cause can be further narrowed down to a very unstructured way of communicating from the Business side, which nurtures misunderstandings. ITDT are often given vague or ambiguous requirement specifications, which they must interpret and transform into structured requirements in their own tools not accessible for SME and BU. If/when they seek approval for their attempt to translate it into meaningful requirements, it is highly likely the Business will not understand the implications of accepting the proposed solution. Before any development is initiated, a high risk therefore exists that Business and IT are misaligned, and the wrong solution will be developed. To prevent this situation, SME and BU must have access to a platform they can operate for specification of their Business Logic requirements. The platform must have a proper level of version control and ownership registration to support a dynamic evolution in Business Logic and requirements.

Business Users and Subject Matter Experts use Business Processes and associated information input/output to describe what they do, which information is required, and what information they produce. The key to integration with the IT world of requirements lies in a user interface intuitively operated by Business Users with the ability to distribute requirements to IT at the level of detail they expect to receive from the Business.

Visualizing Business Processes with BPMN

The Business Process Model and Notation standard (BPMN) is a good candidate for a visual and intuitive interface that will allow Business users to specify their own work as well as requirements towards IT at a conceptual level. EICORE has implemented this standard for creating Business Process diagrams and interacting with the catalogues for additional details, see Fig. 1. At its core, it contains:

1.      an Event object for the start and end of a Business Process

2.      a Business Process Task object

3.      Information Input and Output objects for each Business Process Task

Integrating Business Process Diagrams with Catalogues for Business Processes and Information Structures

The information provided from the Business Process Diagram (Fig. 1) is not sufficient to provide all the specification inputs that ITDT expect from SME and BU. This level of detail can be captured by integrating with a catalogue for Business Processes and Information Structures, respectively. A benefit by powering Business Process Diagrams with catalogues, is the ability to assign ownership, reuse entities and thereby enable lineage and Impact analysis across all Business Process specifications.

Business Process Catalogue

A catalogue of Business Processes contains a list of all the Business Processes and all individual Business Process Tasks. Because a Business Process Task specifies what is being done at a conceptual level, Business Process Task Implementations are used to capture different variations of how the Task is performed. A specific Task can e.g. be executed manually, or by one or another IT Application as depicted in Fig. 2.

When the Business Process Diagram is integrated with a Catalogue, the user can select from a list of available Business Process Tasks, and afterwards also select the preferred implementation variant. All is accessible and captured visually in the Diagram.

Information Structure Catalogue

A catalogue for Information contains five different aspects of the information details usually provided by Subject Matter Experts and Business Users. The five aspects are:

1.      Semantic Model

2.      Technical Information Structure

3.      Information entry Guardrails

4.      Information Content

5.      Rulesets and Data Quality Rules

All information aspects are specified at the logical level, which means they are format agnostic and only focus on the logical content. It is important that the user freely can choose which aspects to enter at any given time, as some aspects may not be of relevance, or just not available when the other aspects are filled out (see Fig. 3).

The five aspects will be presented in further detail below.

Semantic Model

The Semantic Model consists of a collection of Business Terms that together describe the information content for a specific Business Process Task Implementation. The Business Terms are available from a Business Glossary for Organization- or Domain-wide consistency. An example Semantic Model for a file with measurement data will be the collection of Business Terms representing each measurement parameter in the file.

Technical Information Structure

The Technical Information Structure is a specification of the information schema containing the same technical names available in a physical data format. The schema can be further subdivided into Blocks- and Attributes of Information. An example for a file with measured data is the header names, as represented within the physical file, for each block of information and the header names for each attribute within the blocks. Business Terms from the Semantic Model can be linked to the technical structure and provide additional and detailed Business Insights for the users of the Information.

Information entry Guardrails

When Information content is created by a user, Guardrails can assist with entering the correct information. Typical Guardrails are:

·        Prompting the user with a descriptive explanation for what to enter

·        Assisting content entry with:

o   Limit entry to selection from Lists of defined options

o   Dedicated forms for entry of different Data Types (e.g. boolean, numeric, date, string, Weblinks)

Information Content

Information Content can be added by Business users, primarily to provide example content to support the detailed Information specification. The entry of content can be supported by the guardrails whereas the storage of the content is limited by the defined Information/Data Types from the Technical Information Structure specification.

Rulesets and Data Quality Rules

Rules for specifying data quality, or to specify how information shall be interpreted by a Business Process Task, can be assigned to the Technical Information Structure. The rules are gathered in Rulesets. Rules can be conceptual and only consist of a descriptive name, or more complex with variables for the user to fill out while applying the rule. An example can be data quality rules as “value >/</= [user input] “.

Transformation of Information

Subject Matter Experts and Business Users specify transformation of information in the context of a Business Process Task. This is different compared to IT- and Data Teams that focus on the direct transformation of data from one state to another. A Subject Matter Expert defines a Business Process Task and only subsequently the consumed and produced information. The transformation of information from one state to another is therefore an indirect outcome when specified by Subject Matter Experts. In Fig. 4 the user has selected a Business Process Task with information input- and output. In the right-hand pane, the user has direct access to the details of the information transformation because is linked to the Information Catalogue that contains the information transformation details.

The specification of information for transformation can also be at different levels of detail. At the conceptual level, only a descriptive name for the type of information is specified. At the logical level, the user may only specify the semantic model (relevant Business Terms) or a more detailed specification using the relevant elements from the Technical Information Structure.

Knowledge Graph for Conceptual Specification and Catalogue Organization

Catalogues need to be organized for search, especially when they scale. The user must be able to find what they entered. A knowledge graph is the de facto standard for preparing elements for search, e.g. google is powered by a knowledge graph.

When Business users have initial discussions about a Business Process and sketch it out, it is often at a conceptual level for both the Business Process Tasks and the associated Information input- and output. Conceptual descriptions can be used for organizing the detailed specifications of both Information and Business Process Tasks, and support users when they search for the same elements in the Catalogues at a later stage. A Knowledge Graph with embedded hierarchies is optimal for managing the conceptual naming of the Business Process Tasks and the associated Information. In Fig. 5, the user can tie the conceptual level of the Diagram Objects (Business Process Task and Information) to a knowledge Graph from the right-hand pane. Note, in the example from EICORE in Fig. 6, the conceptual names are still managed by a set of hierarchies in the backend, as the knowledge graph is not yet fully implemented.

Reporting and Analysis

When Business Processes and Information is specified as presented above, the user can automatically generate many different valuable outputs such as specification reporting and linage-/impact analysis.

Reporting in any Format

SME and BU often prefer specification reports in Excel, Word or Pdf formats, as it is easier for them to consume. An option for ad-hoc browsing of the Information details, from within the Business Process diagram, is another benefit that will improve the immediate understanding of the information.

For ITDT, the format requirements of the same specifications are different, they prefer to consume the information through an API or querying the catalogues directly. A significant part of the specification content can be directly used for supporting generation of Data Contracts, to ensure Business and IT are aligned from the very beginning.

Explore Dependencies with Lineage- and Impact Analysis

Linage- and Impact Analysis becomes easy to support because all elements for Business Processes and Information are stored in the Catalogues and organized with the Knowledge Graph. The user can see the lineage and impact at the conceptual level for both Business Processes and Information. For Information, the user can drill deeper and extract lineage and impact analysis at more detailed levels such as the Attribute level.

Benefits by Integrating Business Process Maps with Information Catalogues

When SME and BU use a Business Logic platform for communication with ITDT, it gives several highly valuable benefits as listed below:

Improved Quality: IT solutions developed based on a Business Logic platform operated by Business Users, have documented >15 times higher probability for delivering correct outputs.

Faster Delivery at reduced costs: Business Processes and associated Information can fast and easily be defined by Subject Matter Experts and Business Users without any assistance from IT-/Data Teams.

Documentation, Execution and Validation in sync: Specifications and Documentation are automatically created out of the same core of information, which prevents documentation being out of sync. It can be exported to any format that fits the users and systems, which allows for close integration with execution and validation.