This article presents a new architecture for organizations that want to be agile and mature digitally. This architecture meets the requirements laid out in the article: New Architecture Requirements.
Instead of treating integrations as a necessary evil, the new architecture embraces them. Indeed, integrations are the very foundation of the architecture.
This new architecture can be implemented in small steps without affecting any existing backend applications. This article will present the foundation for the architecture. Followed by a discussion of how it forms a digital business platform for the organization.
Core principles, concepts and patterns
The new architecture is based on well known principles, concepts and patterns from software development and integration: loose coupling, contract-first, domain-driven design, distributed integration, request-response, publish-subscribe, etc.
It is how we combine them together that makes the architecture new.
Loose coupling
A loosely coupled system is one in which each of its components has little or no knowledge of the definitions of other components.
Architects generally agree that loose coupling, see Wikipedia, is preferred to tight coupling. One of the great advantages is that other components are not affected if one component is modified.
In traditional architecture, every new integration introduces tight coupling between the involved IT systems. If one of the systems is modified, there is a risk that the other system is affected. This has a strong negative effect on the IT landscape which becomes more difficult and expensive to modify for each integration. Loose coupling between separate IT systems is not only recommended, it is mandatory.
Here are some examples that we consider tight coupling between systems and therefore forbidden in the new architecture:
- knowing the entity or attribute names in another system
- knowing the id's of objects or other value sets in another system
- knowing the format of data for another system (such as date format, currency encoding, etc.)
- knowing the end-point of another system
With true loose coupling, the individual IT systems are unaware of each other and each system can be modified or replaced with minimal disturbance to the other IT systems.
Contract-first
Contract-first means you design the way your services communicate with each other before you design the services themselves.
In traditional architecture, it is the individual IT systems that decides what their integration contracts looks like. It is then up to the consuming parties to adapt. In the new architecture, the contract is specified by the business. It is the individual IT systems that must implement it as specified.
When a system needs to use functionality in another component, it only has to know about the contract. It does not need to know any details about the component.
Distributed integration
Distributed integration means that the responsibility for integration is placed on the individual services, not the client or middleware (e.g. an integration platform).
Every system that is made available through integration must fulfill one or more contracts. If a system doesn't fulfill a contract natively, then there must be an adapter that does. It is the responsibility of the adapter to deal with the system's proprietary integration.
This means that the responsibility for dealing with a system's proprietary integration is put on the people who are familiar with the system. It is no longer the responsibility of the party that wants to access the system.
One huge advantage of this turnaround is that if a system modifies it's proprietary integration method, it will not affect the consuming parties at all.
It is up to the developers to modify that specific adapter to deal with those changes.
Domain-driven design
In domain-driven design, the structure and language of software code (such as class names, class methods, class variables) match the business domain.
The structure and language of the contracts (entity names, operations and attributes) should also match the business domain. So, in the end, the structure of the code matches the business domain which, in turn, matches the contract structure.
We offer a business specification language, Biznish, that provides exactly what is required here. When the organization defines a business capability in Biznish, it is defined with operations, entities and attributes. It has a one-to-one relationship with the API contract.
This one-to-one relationship makes the API very stable and easy to maintain. If there are any modifications to the business capability, those changes can be easily mapped to the corresponding API contract.
Request-response
In computer science, request–response [...] is one of the basic methods computers use to communicate with each other in a network. The first computer sends a request for some data and the second responds to the request.
This pattern is used whenever a client needs to create, read, update or delete data in another system. Traditionally, request-response is implemented in a purely synchronous fashion; i.e. the client waits for the response.
In the new architecture asynchronous requests must also be supported. In asynchronous requests, the request is accepted by the server but, while waiting for a response, the client is free to do other things. The server processes the request when it has the time to do so. This could be almost instantly, or hours, even days, later. When the server has completed the request, the response is sent to the client.
Publish-subscribe pattern
In software architecture, publish–subscribe is a messaging pattern where message senders, called publishers, are unaware of the message receivers, called subscribers. The publisher tags each message with a category and all the subscribers to that category receive the message.
Publish-subscribe is the second and final communication pattern that is used in the new architecture.
The publisher and subscriber don't need to know about each other. Adding and removing subscribers can be done easily without informing the publisher. This is the definition of loosely coupled. Since there is no firm connection between publisher and subscriber, this pattern is inherently asynchronous.
The digital business platform
A digital business platform supports the seamless interaction of people, software, and machines in an intelligent manner.
The new architecture is well-suited to building a digital business platform. It is also a natural platform for helping an organization reach digitally maturity.
Unified API
A unified API is an abstraction layer that easily handles communication from many different APIs and backend data models. You can think of a Unified API as an airport hub that maintains relationships with airlines and helps route passengers to their destination of choice.
Implemented business capability contracts are represented by individual APIs. There is one contract for each capability. The contracts are made easily accessible to clients in one unified API.
The unified API should use the same patterns and rules for each capability, such as:
- API technology
- API guidelines (version handling, exception handling, etc.)
- End user and client authentication
- Logging
The most common way to structure an API is to base it on entities such as Customer, Invoice, Product, etc. However, we have found that this does not scale well as the number of operations grow.
The unified API works more efficiently, if instead of being based on entities, it is structured on capabilities. One capability consists of a number of entities with operations. The same entity may reappear in several capabilities. But it will have a different set of operations. This gives a more natural grouping of entities and operations, making it much easier to find the correct endpoint.
Capability providers
A capability provider is someone or something that provides or "implements" a specific capability.
For a system to provide a capability, it must fully comply with a capability contract. To achieve this, a technical adapter is often needed.
Note: Apart from being just a capability provider, a system might also need to adhere to other rules. These are defined on a per system basis, i.e. not on a capability basis,
Some examples :
- Master data synchronization. If a system stores master data that is also stored in other systems, it will need to adhere to a master data synchronization contract.
- Analytics data. If a system has internal data that is not exposed through the capability contracts, part of that data may need to be made available for analytical purposes.
- Rule compliance. Rules that the system needs to adhere to that are not covered by the capability contracts. With GDPR, for example, each system needs to provide a contract for deleting a person, for getting all the information stored about a person, etc. This is not per capability, but per system.
Composite applications
In computing, a composite application is a software application built by combining multiple existing functions into a new application.
When an organization has published its capabilities, the next natural step is to access data and functionality in composite applications.
There are different types of composite applications. Let's look at two of them:
- role based applications
- workflow applications
Role based applications
The current use of proprietary interfaces for individual applications is contra productive to digital maturity and should be limited to a minimum. End-users should be provided with composite end-user applications. These are what we call role-based applications.
These applications suggest what tasks to do and guide the end user through all of the task steps. They provide needed information to help the user make well informed decisions. This guidance allows the user to complete the task without requiring the user to learn all of the individual IT systems.
With the new architecture, horizontal applications are easy and cheap to create. They utilize the business capability API so it is natural to extend this kind of applications to multiple target groups, employees, business partners and customers.
Role based applications should cover around 80% of the tasks that you do within a company. Vertical applications are still useful for ad-hoc, unusual or complicated maneuvers executed by well- trained and experienced users.
Workflow applications
One reason that business capabilities are generally stable is that only describe what the business needs. A workflow describes how that need is provided. It is, therefore, inherently prone to adjustments and complete make-overs.
Workflow applications are a type of composite applications that controls and executes workflows by combining operations from a unified API.
A digital business platform must deal with how the workflows are manifested. Processes that are internal to individual systems can remain as they are. Some other processes may be in horizontal applications and it is best they stay there. These workflows are very volatile and may span multiple systems. They are best implemented somewhere else in your business.
Technology capabilities
In traditional architecture, a system needs to deal with a number of difficult technology challenges by itself. But many of these scenarios are better solved centrally. The new architecture is ideal for this. In these solutions, the new architecture presents APIs for business capabilities as well as APIs for selected technology capabilities. This makes the creation of new micro services much easier.
The actual technology capabilities you implement in your digital platform depends on the needs of your organization. A few examples of useful micro services would be end-user authentication, master data synchronization and workflow orchestration.
With the new architecture, an organization can create an ecosystem of business and technology capabilities that work together seamlessly as a single organism.