Database schema migrations often represent the highest risk factor in a CI/CD pipeline. When you deploy a new version of an application that requires a modified database structure, a synchronized "big bang" release usually leads to downtime or, worse, java.sql.SQLException errors during the transition. If the database updates before the application code is ready to handle the new schema, the current live version of your app crashes. If the app updates first, it fails because the expected columns or tables do not exist yet. This circular dependency is the primary cause of deployment anxiety in high-scale environments.
To solve this, you must decouple database changes from application logic. By adopting the "Expand and Contract" pattern (also known as Parallel Change), you can ensure that your database remains compatible with at least two versions of your application simultaneously. This strategy allows for zero-downtime deployments, canary releases, and safe rollbacks without losing data integrity.
TL;DR — Use Flyway to implement migrations in three distinct phases: first, expand the schema by adding new elements while keeping old ones; second, update application logic to write to both locations; and third, contract the schema by removing the deprecated elements once the new version is stable.
The Expand and Contract Concept
💡 Analogy: Think of it like replacing a bridge while traffic is still moving. You don't blow up the old bridge and then start building the new one. Instead, you build a new set of lanes next to the existing ones (Expand). You gradually shift traffic to the new lanes (Transition). Once all cars are safely on the new lanes, you dismantle the old bridge (Contract).
In the context of Flyway and SQL databases, backward compatibility means that Version N of your database must support both Version N and Version N-1 of your application code. This is essential for blue/green deployments where two different versions of the code are running against the same database at the same time. If your migration script contains a DROP COLUMN or RENAME COLUMN command, you are performing a breaking change that violates this principle.
Backward compatibility is not just about avoiding errors; it is about maintaining the ability to roll back. If you deploy a new version of your Java or Node.js service and it contains a critical bug, you should be able to instantly revert to the previous container image. If your database migration already deleted the columns that the previous image required, your rollback will fail, leaving your system in a broken state. The Expand and Contract pattern provides a safety net by ensuring the data structures required by the old code remain present and valid until they are no longer needed.
When to Adopt Backward Compatible Migrations
You should use this architectural pattern whenever you cannot afford a maintenance window. For startups or internal tools with low traffic, a 5-minute maintenance window might be acceptable. However, for SaaS platforms, fintech applications, or any distributed system using Kubernetes for rolling updates, backward compatibility is a hard requirement. In these environments, Kubernetes might keep the old Pods running for several minutes while the new ones spin up, meaning the database must satisfy both simultaneously.
Another critical scenario is when you utilize "Canary Releases." In a canary setup, you route 5% of your traffic to the new version of the app while 95% remains on the old version. Both versions share the same database. If the database is not backward compatible, that 95% of your users will experience immediate failures. By measuring metrics like Mean Time to Recovery (MTTR) and deployment frequency, you will find that although the expand/contract pattern takes more steps, it drastically reduces the risk of high-severity incidents during the release cycle.
Architecture of Parallel Schema Evolution
The architecture relies on a multi-stage data flow. Rather than changing a column type or name directly, you create a new column and synchronize data between the two. This ensures that the application can read from either source during the transition period.
[Phase 1: Expand] Database: Table A (col_old, col_new) App v1: Reads/Writes col_old App v2: Reads/Writes col_old AND col_new (Double Write) [Phase 2: Transition] Database: Table A (col_old, col_new) App v2: Reads col_new, Writes both [Phase 3: Contract] Database: Table A (col_new) App v2: Reads/Writes col_new
This flow ensures that data is never lost and that no version of the application is ever left without the fields it expects. While this increases the number of Flyway migration scripts, it moves the complexity from the "unpredictable runtime" to the "predictable development" phase. Using Flyway 10.x or later, you can easily manage these versioned scripts within your existing project structure, ensuring that each environment (dev, staging, prod) follows the same disciplined path.
Implementation Steps with Flyway
Step 1: The Expand Migration
In this first step, you create the new structure. If you are renaming a column, you add the new column but do not delete the old one. If you are changing a data type, you create a column with the new type. Ensure the new column is nullable or has a default value so that the old application (which doesn't know about it) doesn't trigger database constraints.
-- V1__Expand_schema_add_new_column.sql
-- Goal: Add new_phone column while keeping old_phone for backward compatibility
ALTER TABLE users ADD COLUMN phone_number_v2 VARCHAR(20);
Step 2: Application Double-Writing
Update your application code. The application should now be configured to write data to both old_phone and phone_number_v2. This ensures that the new column starts populating with fresh data. During this phase, the application should still rely on old_phone as the "source of truth" for reading, as the new column does not yet contain historical data.
Step 3: Backfilling Historical Data
Now you must migrate existing data from the old column to the new column. For large tables, do this in batches to avoid locking the database for extended periods. This can be done via a Flyway script if the dataset is small, or via a background job/task for millions of rows.
-- V2__Backfill_new_phone_column.sql
-- Run in batches for high-traffic tables
UPDATE users
SET phone_number_v2 = old_phone
WHERE phone_number_v2 IS NULL
AND old_phone IS NOT NULL;
Step 4: The Contract Migration
Once you have verified that the new version of the application is stable and that all data has been backfilled, you can remove the old column. This should ideally happen in a subsequent release cycle to ensure you don't need to roll back to a version that still uses the old column.
-- V3__Contract_schema_remove_old_column.sql
-- Final cleanup: old application versions are no longer running
ALTER TABLE users DROP COLUMN old_phone;
Trade-offs and Decision Criteria
While backward compatibility is the gold standard for reliability, it introduces overhead. You are writing more code, managing more migrations, and temporarily consuming more storage. It is important to weigh these costs against the potential impact of downtime. Below is a comparison to help you decide when to apply this pattern rigorously.
| Criteria | Expand & Contract | Direct Migration |
|---|---|---|
| Downtime | Zero | Required (Maintenance Window) |
| Complexity | High (Multiple releases) | Low (One release) |
| Rollback Safety | Excellent | Poor (Requires DB restore) |
| Storage Impact | Temporary Increase | Neutral |
⚠️ Common Mistake: Forgetting to remove the "Contract" script from the same release as the "Expand" script. If you put both `ADD COLUMN` and `DROP COLUMN` in the same Flyway version or release, you haven't achieved backward compatibility; you've just made the migration more complex without any of the safety benefits.
Pro-Tips for Production Flyway Usage
When working with Flyway in a professional environment, consistency is key. Always use a strict naming convention for your migration files. A pattern like VYYYYMMDDHHMMSS__description.sql prevents versioning conflicts when multiple developers are working on different features simultaneously. This is especially important when managing the long-lived "Contract" phases of the expand-contract pattern.
Furthermore, ensure that your migrations are idempotent where possible, though Flyway handles the execution state. More importantly, always test your "Expand" migration against a copy of production data. This allows you to measure how long an ALTER TABLE takes. On tables with hundreds of millions of rows, adding a column—even a nullable one—can lock the table depending on your database engine (e.g., older versions of PostgreSQL or MySQL). Using tools like pt-online-schema-change or the native ALGORITHM=INPLACE in MySQL can help keep these migrations non-blocking.
📌 Key Takeaways
- Never
DROPorRENAMEin a single deployment; always use the Expand and Contract pattern. - Backward compatibility is mandatory for rolling updates and blue/green deployments.
- Use "Double Writing" in the application layer to keep old and new columns in sync during transitions.
- Backfill data in batches to prevent database performance degradation.
- Delete deprecated schema elements only after the new version is confirmed stable in production.
Frequently Asked Questions
Q. How do you handle breaking database changes in Flyway?
A. You handle breaking changes by splitting them into multiple non-breaking steps. Instead of modifying an existing object, you create a new one, migrate the data, and only remove the old object in a future deployment cycle. This ensures the database always supports the currently running application code.
Q. What is the expand and contract pattern in database migrations?
A. It is a phased migration strategy. The "Expand" phase adds new schema elements (columns/tables). The "Transition" phase syncs data and updates app logic. The "Contract" phase removes the now-obsolete schema elements. This allows two versions of an application to share a single database without errors.
Q. Can Flyway migrations be rolled back automatically?
A. While Flyway Teams edition supports "Undo" migrations, automatic rollbacks are dangerous in production. The Expand and Contract pattern is superior because it makes "rolling back" the application code safe without needing to revert the database schema at all, as the schema remains compatible with the previous code version.
For more information on managing database schema evolution, refer to the official Flyway documentation and explore the Parallel Change pattern described by Martin Fowler. By integrating these architectural principles into your CI/CD pipeline, you can eliminate the primary source of deployment failures and achieve true zero-downtime releases.
Post a Comment