Inleiding
Modern applications are expected to be available at all times. Users should be able to access a service whenever they want, regardless of whether the application is being updated or a server has crashed. For some applications, such as banking systems, even a short interruption can have serious consequences.
Over the years, several techniques have been developed to improve the availability of applications. By running multiple instances of a service and distributing the workload between them, an application can remain available when one of its instances fails. This is commonly referred to as high availability.
However, high availability does not necessarily mean that an application can be updated without downtime.
When deploying a new version of an application, the existing instances need to be replaced. Techniques such as blue-green deployments make it possible to run the old and new versions alongside each other. Once the new version is ready, traffic can be redirected without interrupting the service.
For stateless applications, this is relatively straightforward. Unfortunately, most applications are not entirely stateless. They depend on databases to persist their data, and changes to the application often require changes to the database schema.
This introduces an interesting problem: how can we update a database schema while both the old and new application versions are running?
During my master’s research at the University of Twente, in collaboration with ING, I investigated this problem. More specifically, I researched how the expand-contract pattern could be applied to PostgreSQL databases in highly available, distributed systems.
The problem with database schema migrations
Let’s consider a simple application that stores information about its users. The database contains a table with the following structure:
CREATE TABLE users ( id BIGINT PRIMARY KEY, name VARCHAR(255) NOT NULL);At some point, we decide to separate the user’s name into a first name and last name. This requires us to change the database schema.
A straightforward migration would look something like this:
ALTER TABLE users DROP COLUMN name;
ALTER TABLE users ADD COLUMN first_name VARCHAR(255);
ALTER TABLE users ADD COLUMN last_name VARCHAR(255);There are several problems with this approach.
First, dropping the original column would result in data loss unless the data had already been migrated. More importantly, the old application version still expects the name column to exist.
As soon as the migration is applied, requests handled by the old application may fail.
We could solve this by temporarily stopping the application, applying the migration and deploying the new version. However, this introduces planned downtime.
Another option would be to deploy the new application before changing the database schema. Unfortunately, that introduces the opposite problem: the new application expects columns that do not exist yet.
The problem is not necessarily the migration itself. The problem is that the application and database schema are tightly coupled.
To achieve zero-downtime deployments, we need a way to temporarily support both versions of the application.
Blue-green deployments
Blue-green deployment is a technique in which two versions of an application run alongside each other.
The blue environment represents the currently deployed application, while the green environment contains the new version.
Initially, all traffic is directed towards the blue environment. Once the green environment has been deployed and verified, the load balancer redirects traffic to the new version.
flowchart TB
accTitle: Blue-green deployment before switching traffic
accDescr: The load balancer sends live traffic to Blue version 1. Green version 2 is on standby. Both versions depend on the same database.
LB["Load balancer"] -->|Live traffic| Blue["Blue · v1 · active"]
LB -.->|After verification| Green["Green · v2 · standby"]
Blue --> DB[("Shared database")]
Green --> DBThe advantage of this approach is that we do not have to stop the application to deploy a new version. In addition, if something goes wrong, traffic can potentially be redirected to the previous version.
However, both application versions commonly depend on the same database.
If the database schema is incompatible with either version, blue-green deployment alone is insufficient.
This is where the expand-contract pattern becomes useful.
The expand-contract pattern
The expand-contract pattern separates a database migration into multiple steps. Rather than immediately replacing the existing schema, we temporarily extend it to support both application versions.
The pattern consists of two main phases:
Expand: Extend the database schema so that both the old and new application versions can operate.
Contract: Remove the obsolete database structures after the old application version is no longer in use.
Between these phases, the database is in what we call a mixed-state. This means that it supports two different schema versions simultaneously.
flowchart LR
accTitle: The expand-contract schema lifecycle
accDescr: Expand the original name-only schema with first_name and last_name, synchronize and backfill data, then deploy version 2 and validate it. Contract only after retiring version 1 and closing its rollback window.
Original["Original schema<br/>name"] -->|Expand| Mixed["Mixed-state schema<br/>name + first_name + last_name"]
Mixed -->|Synchronize and backfill| Ready["Deploy v2<br/>shift traffic and validate"]
Ready -->|Retire v1 and close rollback window| Final["Contract<br/>first_name + last_name"]Let’s return to our earlier example.
-
Expand the schema
Instead of removing the
namecolumn, we introduce the new columns while keeping the existing one.Foto door ALTER TABLE usersADD COLUMN first_name VARCHAR(255);ALTER TABLE usersADD COLUMN last_name VARCHAR(255);The old application can continue using the
namecolumn, while the new application can start usingfirst_nameandlast_name.However, there is still an important issue.
If the old application updates the
namecolumn, how do we ensure that the new columns are updated as well?Similarly, if the new application changes a user’s first name, the old application should still receive the correct value.
We therefore need to synchronize the data between both representations.
Depending on the migration, this could be achieved through database triggers, compatibility views or application-level synchronization. Each approach has its own advantages and disadvantages.
For example, a PostgreSQL trigger could synchronize the columns when a record is updated. The implementation must take care to avoid conflicting writes and ensure that the synchronization behaves correctly in both directions.
Once synchronization is in place, existing records can be migrated to the new representation.
An important consideration is that this migration should not unnecessarily block the database. For large tables, migrating all records in a single transaction may introduce significant overhead.
Instead, the data can be migrated in smaller batches.
-
Deploy the new application
After expanding the schema, we can deploy the new application version.
Both application versions can now operate against the same database because the schema supports both representations.
flowchart TB accTitle: Two application versions sharing an expanded schema accDescr: During the traffic transition, Blue version 1 uses the name column and Green version 2 uses first_name and last_name. Both use the same mixed-state database, with synchronization keeping the representations consistent. LB["Load balancer"] -->|During transition| Blue["Blue · v1"] LB -->|During transition| Green["Green · v2"] Blue -->|Reads and writes name| DB[("Mixed-state database<br/>name<br/>first_name<br/>last_name")] Green -->|Reads and writes first_name / last_name| DB DB --- Sync["Temporary synchronization<br/>keeps both representations consistent"]At this point, traffic can gradually be redirected from the old application to the new one.
The database remains compatible with both versions throughout the deployment.
This is particularly useful in distributed systems, where multiple application instances may be running simultaneously and cannot necessarily be updated at exactly the same moment.
-
Contract the schema
Once all application instances have been updated, the old schema is no longer required.
We can now remove the obsolete column and any temporary synchronization mechanisms.
Foto door ALTER TABLE usersDROP COLUMN name;The database has reached its intended schema.
The expand-contract pattern therefore makes schema evolution a coordinated deployment process rather than a single database operation.
Applying expand-contract to distributed databases
So far, we have assumed that the application uses a single database instance.
In practice, highly available applications often use database replication to reduce the risk of downtime.
A PostgreSQL database might consist of a primary instance and one or more replicas. Changes made to the primary database are propagated to the replicas, allowing the system to recover from failures and, depending on the architecture, distribute read workloads.
However, replication introduces additional complexity.
When changing the database schema, we need to consider how those changes are propagated and how they affect the consistency and availability of the replicated database.
There are two common replication approaches worth distinguishing.
Asynchronous replication
With asynchronous replication, changes are committed on the primary database before they have necessarily been applied to the replicas.
This generally reduces the latency of write operations, since the primary does not have to wait for every replica.
The disadvantage is that replicas may temporarily contain outdated information. If the primary fails before a change has been replicated, recently committed data may be lost during failover.
Synchronous replication
With synchronous replication, a transaction waits for confirmation from the configured synchronous replicas before the commit is acknowledged.
This provides stronger durability guarantees, depending on the replication configuration, but introduces additional latency.
The performance of the database therefore depends not only on the queries being executed, but also on the communication between database instances.
When applying schema migrations, these differences become important.
A migration that performs well on a single database instance may behave differently when the database is replicated.
In addition, operations that require locks or migrate large amounts of data may affect the availability of the entire system.
My research: implementing expand-contract for PostgreSQL
During my master’s thesis, I built upon earlier research by Jorryt Dijkstra, who investigated the expand-contract pattern for Oracle databases using Liquibase.
My research focused on extending this approach to PostgreSQL and evaluating its behaviour in a highly available environment.
To simplify the deployment of database migrations, I developed a Liquibase plugin called liquibase-zd.
Liquibase is a database versioning tool that allows developers to define and execute database changes through version-controlled changelogs.
The plugin extended Liquibase with functionality for executing database changes through the expand-contract pattern.
The implementation included support for several migration operations, such as renaming columns and tables, changing column data types and migrating data in batches.
It also included mechanisms for switching between the expand and contract phases and retrieving metadata required to perform the migrations.
One of the goals was to reduce the complexity developers would otherwise face when manually implementing these migrations.
Instead of requiring every developer to design the complete expand-contract procedure themselves, the plugin could handle parts of that process.
Testing in Kubernetes
Implementing the migration mechanism was only part of the research.
It was equally important to determine whether the approach would remain usable under realistic workloads.
For this purpose, I configured a PostgreSQL environment in Kubernetes and used HammerDB to generate database workloads.
The evaluation focused on both functional correctness and performance.
Functional correctness was important because the database must preserve its data throughout the migration. Supporting two schema versions is not useful if records become inconsistent or are lost.
For performance, I measured two characteristics:
Latency describes how long it takes to execute a database transaction.
Throughput describes how many transactions the database can process within a given period.
These measurements allowed me to investigate how schema migrations affected a running database, including environments using synchronous and asynchronous replication.
I also evaluated the behaviour of migrations while the database was in a mixed-state and investigated the effect of batch migration operations.
The research demonstrated that the expand-contract pattern could be applied to PostgreSQL to support zero-downtime schema migrations under the evaluated conditions.
This does not mean that every possible database migration is automatically safe or has no performance impact. The migration strategy, database workload, replication configuration and specific schema operation all remain important.
Architectural considerations
Although expand-contract provides a solution to schema compatibility, it also introduces additional complexity.
- Old and new application versions can coexist during deployment.
- Traffic can move gradually while the schema supports both versions.
- Keeping the old representation preserves a rollback option during the mixed-state.
There are several considerations I believe are important when applying this pattern in practice.
Backward compatibility should be intentional
A database schema should not be changed independently of the applications that depend on it.
During a deployment, different application versions may coexist. The database must support the operations required by each version until the migration is complete.
This means that schema compatibility should be considered during application design, rather than only when a deployment fails.
Data consistency is more important than schema compatibility alone
Adding a new column is relatively simple. Keeping two representations of the same data consistent is more difficult.
If both application versions can update the same data, synchronization must be designed carefully.
For more complicated migrations, it may be preferable to restrict which application version can write certain fields or to introduce a controlled transition between representations.
Large migrations should be performed incrementally
Migrating millions of records in a single transaction can introduce unnecessary database locks, transaction log growth and performance degradation.
Batch migrations reduce the amount of work performed at once and make it easier to monitor the migration.
However, batch size introduces a trade-off. Smaller batches reduce the impact of individual transactions but may increase the total migration time.
Rollback requires planning
Blue-green deployments make it relatively easy to redirect traffic to an earlier application version.
Database migrations are more complicated.
Once data has been transformed or old columns have been removed, reversing a migration may not be straightforward.
For this reason, I would generally avoid executing destructive schema changes as part of the same deployment that introduces a new application version.
The contract phase should be treated as a separate operation, executed only after the new application has been validated.
Zero downtime does not eliminate operational risk
Even a carefully designed migration may affect performance.
Schema locks, long-running transactions, replication lag and resource consumption can all introduce problems.
Monitoring is therefore essential. In addition to application availability, I would monitor database latency, throughput, lock contention and replication health throughout the migration.
Conclusion
Zero-downtime deployments are relatively straightforward when an application does not maintain state. However, most real-world applications depend on databases, which introduces additional challenges.
A database schema cannot always be updated independently of the application because different application versions may require different representations of the same data.
The expand-contract pattern addresses this problem by introducing a temporary mixed-state in which both schema versions are supported.
Combined with blue-green deployments, this makes it possible to gradually update an application and its database without requiring a maintenance window.
During my master’s research, I implemented this approach for PostgreSQL through a Liquibase plugin and evaluated its behaviour in a highly available Kubernetes environment.
One of the most important lessons I took from this research is that zero-downtime deployments are not simply a matter of infrastructure or deployment tooling. They require architectural decisions that account for compatibility, data consistency and the lifecycle of an application.
A well-designed deployment strategy should therefore consider the application and its database as a single evolving system.
Further reading
- Zero Downtime Schema Migrations in Highly Available Databases — Master’s thesis, University of Twente (2022)
- Liquibase — Database change management
- PostgreSQL — Documentation
This article is based on my master’s research at the University of Twente, conducted in collaboration with ING. The SQL examples are simplified illustrations of the expand-contract pattern and are not a complete production migration implementation.
