Evolutionary Architectures: multi-dimensional evolvability
As described in the last article, “Evolutionary Architectures: Designing for Change”, organisations require evolvability in their software architectures to maintain agility while building their business and product. This relationship often goes astray, making organisations follow build & rebuild cycles. Seven-year itches with their technology systems that push them into radical re-platforming projects every several years.
With evolutionary architecture design, we try to prevent that itch by not only developing our features in an agile fashion – one where we try to find the best alignment with our users by responding quickly to changing user requirements and desires – but also taking this principle into account when building our software architectures.
As such, evolutionary architecture design can be defined as aligning our architecture with our business and product in the best way possible and embedding the ability to respond quickly to changing requirements.
Fitness functions
In Neal Ford, Rebecca Parsons and Patrick Kua’s book “Building Evolutionary Architectures”, they describe an evolutionary architecture as one which “supports guided, incremental change across multiple dimensions”.
In the book, they describe the focus of most architects and developers as too narrow in terms of dimensionality. Most architects strictly focus on the to-be-used frameworks, dependencies and ways of integration while often forgetting the necessary dimensions of data architecture, security, scalability and testability of a system. As a way of adequately monitoring these dimensions throughout the lifecycle of a software system, the book proposes to define a set of “fitness functions” that guard specific necessary attributes of a system. Protecting attributes like performance, scalability, and security during the evolution of a software system.
The value of defining fitness functions is enormous in fast-paced environments. It allows the definition of non-functional requirements for a software system which (often) can be tested through the deployment pipelines of a system. Fitness functions provide a way of objectively assessing and guarding the integrity of specific characteristics of your software architecture which are not directly protected by a value stream. They provide measurable factors that protect important properties during software growth but not necessarily giving direction to that growth.
For example, we could define a part of a system as one that will be exposed to high amounts of load. Usually, aggressive caching would be a good way of increasing the performance of a system. The part of the system is, however, handling user authentication. Using a distributed cache would improve performance but significantly reduce the security element of such a system (since user information would be available in layers of your infrastructure, which could easily be accessed).
While performance is an easy-to-monitor fitness factor, making it a strictly primary goal could hurt other fitness functions (like security in this case).
A well-defined composition of fitness functions helps to balance all critical aspects of your system while a system grows. However, there is one characteristic of a system that not only needs guidance and guarding during development; it’s the trait that is the absolute necessity of an evolvable system: modularity.
Vertically-layered architectures
Earlier in this series, we defined a traditional software architecture focusing on building features around an initially and – often immutable – designated “core”.
Usually, this design results from too much up-front design and a lack of serious re-adjustment of foundational technology beyond its initiation. This architectural design is, however, still recommended (often for vendor-lock-in) in the extension methods of ERPs, CRMs and e-commerce solutions. Instead of building functionality separate from the solution, the alteration and addition of new functionality are done through a custom scripting language or business rule engine.
A way to increase separation beyond this single-tier model is to define a multi-tier architecture which allows additional modularity and agility through layering. Instead of defining the “core” in software architecture as a single-dimensional element to which features can be added, a method is taken to separate the architecture vertically. This approach divides, for example, data management, business logic, request processing and user interfaces into specific system parts, each preferably communicating with well-defined contracts.
This idea of vertical separation within a software architecture is found in most development frameworks, separating the “model” of the application from the “view” and the coordinating “controller” in between. A more elaborate rendition of this pattern is seen in most “modern” application architectures, where the front-end of an application is fully separated from a back-end system and potential data systems.
Building systems in this manner adds a lot of flexibility beyond architectures which don’t have this separation. It allows single-dimensional evolvability of each layer when the contracts or APIs are well-defined between the layers. In practice, it still requires serious design effort to shield the complexity of a layer from the contracts it defines. When not correctly implemented, changing a layer often requires adjustment in the upper layers.
When, for instance, a SQL database is used as a core data platform, SQL statements should only be formed in a layer with the sole responsibility of data processing and mutation. If those queries are exposed in higher layers of the application, a change of storage engine will often result in significant parts of the application having to be rebuilt.
Additionally, multiple business departments or domains are often represented within a single solution. A drastic change to a department’s operation will affect all vertical layers within the application and the potential logic of other departments if a form of horizontal separation isn’t applied.
Correct implementation of a vertical model is necessary for any evolvable software architecture. Ensuring architectural patterns are in place that force and accurately guard separation between the layers is vital—using, for example, well-defined REST APIs that form around data models instead of database queries.
In “Domain Driven Design”, a concept is described as the ultimate of this separation pattern: the “Anti Corruption Layer”. This layer between different sub-systems is designed in such a way that it forms an adapter between two parts which don’t and shouldn’t share the same semantics. For example:
A well-defined REST API that ensures specifics of language or framework choice are shielded from the consumer of an API.
A separate layer between your business logic and database engine, often called the repository pattern, that ensures no direct SQL statements are being used in upper layers, potentially making it easier to redesign parts of your storage infrastructure.
But as stated before, modularity must go beyond vertical layering to successfully evolve a system in its lifetime. A way which doesn’t necessarily allow the evolvability of used technology but a direction that will enable the evolvability of the business logic within your technology.
Domain-focused modularity
In most modern organisations, processes are fully embedded and run in their software architectures. Because of this, a change in business direction immediately requires adjustment in their software systems and vice-versa. The principle also finds its proof in Conway’s Law:
“Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organisations.”
Considering the law’s dual, we can’t escape from properly aligning business and technology if we want to operate expectedly.
While vertically layering an application can ensure the evolvability of used technology, many re-platforming projects are often caused by entanglement and inflexibility generated within those layers. This is because vertical separation doesn’t necessarily provide the evolvability of business logic within those layers.
When our purpose is to build software architectures that allow evolvability for an organisation over multiple dimensions, we also have to embed the same evolvability in the product or business parts within our software architecture.
A design pattern focusing on modularity and isolation on a business domain level is one of the first principles of “Domain Driven Design”. DDD is a way of designing, building and evolving software systems matching business domains in the best way possible. Using DDD as a guideline within our software architecture allows us to separate and isolate business parts and logic from our other modules. In the same way as using formally defined contracts to communicate between the vertical layers of our system, its design patterns allow us to apply the same principles horizontally.
The next step in evolvable systems finds its basis in business-domain-driven modularity, allowing code to grow and evolve in separate business-connected contexts. In its origin, the pattern ensures better collaboration between the business and technology teams because of its increased focus and better alignment. However, its ways of isolation also ensure that parts of logic can grow separately and be changed or replaced without directly impacting other business domains.
When properly combined with vertical layering, it allows the multi-dimensional evolvability of both technologies, business logic, and its structure over the lifecycle of a software system. The principles work in every software architecture, be it an extension on top of a bought solution, a monolithic software architecture or one formed around separate (micro)services. They, however, define the success in evolvability in each of these structures and all follow an identical set of design patterns.
These design patterns should focus on properly guarding isolation to ensure the ability of incremental and parallel deployment of features without breaking others. But most importantly, they should concentrate on limiting coupling to such an extent that it supports parts of an organisation in evolving independently.
—
This article continues a series about evolutionary architectures, describing the common design and implementation patterns in the next editions.