In another article we have described the digital maturity model. A sure sign that an organization is moving from level 3 ("connected") to higher levels is that they have the ambition to make their processes digital. This has been hyped since 1990, see Wikipedia, but up to recently it has never really paid back as expected.
We think that a company that has reached level 3 in an orderly fashion has two of the core technology preconditions needed to easily implement digital processes: Events that can trigger the start of the processes and a business API to data and functionality that can be used by the activities in the processes.
There are two more things that are required: Some kind of "engine" that can run the processes and a new capability, that can send a task to someone, make sure that they do it and report back to the process when the task has been fulfilled.
The rest of this article will dive into the why and what of digital processes.
Why should the business make their processes digital?
Most organizations execute processes daily, whether they are aware of it or not. To make sure that the processes are executed and efficient they use training, written instructions, and checklists to help their employees carry out the processes. Examples of such processes are "on-board a new employee" and "pick, pack and deliver the ordered products to a customer".
An example: To plan for the support over the summer, a manager may need to know when a group of employees would like to have their vacation. A company on digital maturity level 3 would typically do that as a number of manual activities with support from various IT systems in most activities. This is what it could look like:
- The manager gets a list of all the employees and their e-mail addresses from the HR system
- The manager sends an email to all of them that instructs them to update their vacation plans and reply back before a deadline.
- Repeat until all employees have responded or the deadline has been reached:
- The manager checks their inbox and notices that an employee has replied
- The manager checks the employee off the list
- If the manager thinks it is time, they send an e-mail to remind the rest
- Done
Each of the individual activities of the process are synchronous, i.e. they can be completed while the manager waits. The process as a whole is asynchronous, i.e. there can be longer or shorter breaks between the activities. A supporting system may be temporarily unavailable or the manager can choose to do another task and plan to continue the process the next day. This means that the manager will have to keep the "state" of the process in their mind. This is a very fragile way of running a process. If the manager becomes ill for instance, they need to inform someone else of how the process is designed and which "state" it currently is in so that person can complete the process.
A company on digital maturity level 4 or higher would have an IT system that makes sure that the process is run in an efficient manner. This is what the process could look like:
- The manager requests a system to start the "update absence" process, entering the deadline
- The system gets a list of all the employees and their e-mail addresses from the HR system
- The system sends an email to each individual employee with instructions and a link to where they can report their plans
- Repeat the following until all employees have responded or the deadline has been reached
- Wait a couple of days
- The system sends reminders to those who haven't reported their plans
- The system sends an e-mail with the result to the manager
With this implementation of the process, the manager no longer needs to "run" or "manage" the process and the process will work even if the manager is temporarily or permanently replaced and it would require no or very little instructions to the new person.
A digital company wants to make many of their processes digital. Especially processes that fulfill one or more of these characteristics:
- Executed often and we want repeatability or efficiency, i.e. don't be dependent on different interpretations of the process
- Executed seldom but still important, i.e. hard to remember how to execute them
- Executed by employees that are new to their job, i.e. less training and experience required
- More than one department involved, i.e. the whole chain must work, and nothing must fall through the cracks
What are the technology challenges?
The majority of existing programming languages are focused on solving synchronous problems, i.e. operations that can be finished while the client is waiting. All information needed to solve the problem must either be provided upfront before executing the operation or available through synchronous calls. If the information is incomplete or any of the resources needed are unavailable, the operation fails and the client will have to try again.
Some examples of synchronous operations that are typically provided by a computer system are: "Add one item to my shopping basket", "Get the number of items in stock for a certain product" and "Save updated personal details".
As a consequence of computers mostly dealing with synchronous operations, most of the asynchronous parts and keeping the state of the current process are handled manually by employees. The employees know that they first must do one activity in one IT system, maybe wait (seconds or days) for something to happen, and then do another activity in the same or another IT system for the process to progress.
Hibernate and resume
If we want to make a process digital, we need to have a way to develop programs that can handle long-running processes, i.e. programs that spend most of the time waiting. The major challenge with writing such programs is that when they come to a point where it needs to wait longer than sub seconds, e.g. for days. At this point, we would like the program to hibernate, i.e. to save its state and stop using resources. When the wait eventually is over, we would like the program to restore its state and somehow continue at the point where it hibernated, do another action, and maybe hibernate again.
In a programming language that doesn't handle long-running processes out of the box, the programmer needs tools for hibernating, i.e. saving the state of a process and resuming the program at the point where we left off.
Process versions
Suppose there are a number of incomplete instances for a process, i.e. executions that are currently hibernated and waiting for something before they can resume. If we now modify the process, new instances will run fine with the modified process, but what about the hibernated ones? When they are resumed they may not fit into the modified process and therefore fail.
To avoid this, we need a way to keep old process versions until all the instances that were started with those versions have been completed, while new instances are executed using the latest process version.
Manual interaction
If a process activity requires that someone, perhaps a customer or employee, does something manually then we need to alert them about this, remind them to do it and maybe escalate the matter if the task is not completed in time. It would also be nice for persons involved in the process to see all their outstanding tasks and maybe also a task history.
From our point of view, this is such an important part of being a digital company of level 4 that you should consider this as an important capability, maybe your "Manual task fulfillment" capability.
Error handling
When a program runs into an exception in synchronous programming, the programmer often just aborts the operations with an error message, expecting the client to try again later. With long-running processes, we want to avoid this. We need ways to let an employee solve the problem and then retry the failing activity again.
In synchronous programming some things can be done as a transaction, i.e. if one step fails all the previous activities can be rolled back so when we abort the operation and the transaction, no changes have been. With long-running processes, we need to have other measures such as compensating activities that undoes the changes made by earlier activities.
Inspect running processes
There must be ways for a business person or IT maintenance personnel to inspect the running processes to see if they are behaving as expected and to handle exceptions where the process gets into a bad state where it can't continue without manual intervention.