Most monolith splits fail at the database

A monolith split usually begins as a complaint rather than a design. Deploys queue behind each other. Two teams keep breaking each other’s code. The test suite takes forty minutes and everybody ignores it. Three separate complaints arrive as one request, and the answer comes back as services.

Splitting fixes two of them, each under a condition, and does nothing for the third. The one that fails later is the database. Two services over a single schema keep every lock, every migration ordering problem and every shared outage they had before, and add a network hop on top.

What splitting a monolith into services actually fixes

Splitting a monolith fixes deploy coupling and team coupling, each under a condition, and it does nothing at all for a slow test suite. The table below gives each complaint, its cause, and what a boundary does about it.

Two services owned by one team are distribution without autonomy: same people, same calendar, two releases.

The third row deserves arithmetic. Ferrystone is invented — a payments ledger this site uses whenever retries, idempotency or timeouts come up, so the same fiction carries across articles instead of a fresh anonymous company each time. Nothing below was measured on anybody’s hardware; the figures are picked so you can redo every division yourself. Ferrystone’s suite is 4,000 tests at a mean of 600ms, which is 2,400 seconds: the forty minutes above. Split the code in two and you get two suites of 1,200 seconds plus the contract and end-to-end layers you didn’t have before, so the unit compute is the same, the total goes up, and each team waits twenty minutes. Eight workers under pytest-xdist or Jest take the original 2,400 seconds down to 300, but only where each worker owns its own database, because workers truncating the same tables queue behind the same locks. Check the flag names: the Jest CLI reference gives --maxWorkers, capital W and all, and pytest-xdist gives -n. Derate it fourfold — a thousand tests at 600ms is 600 seconds — and splitting the repository still buys that suite nothing.

Why a shared database leaves you with a distributed monolith

A shared database leaves you with a distributed monolith because the coupling lives in the data rather than in the process boundary. Splitting the code puts a network call in the middle of the same contention. Lock waits don’t go away, and a transaction that spans a network call holds its rows for longer than the one it replaced. Multiplying two services at 99.9% to get 99.8% assumes the two failures are independent, and one shared database takes both down in the same second.

Connection pools bite first. Ferrystone’s four services at a pool of 20 each is 80 connections against one max_connections, where the monolith used 20. The PostgreSQL connection settings documentation puts the default at typically 100 and says the parameter “can only be set at server start”, so the cure for running out is a restart somebody has to schedule.

On one of our own builds we left a shared users table in place and meant to split it the following quarter. A year later it was still shared, every change needed both services deployed in a set order, and nothing had ever shipped on its own — the whole point of the work.

What a service split costs: one schema per service

A real service split costs one schema per service with no cross-schema reads, and that boundary only holds where database grants enforce it rather than code review. Take back the defaults first with revoke all on schema public from public. On PostgreSQL 15 and later a fresh database no longer hands create on public to everybody, so what you take back there is usage; on a database upgraded from 14 you take back both. The schema documentation is precise about which versions need the statement: “In PostgreSQL 15 and later, the default configuration supports this usage pattern. In prior versions, or when using a database that has been upgraded from a prior version, you will need to remove the public CREATE privilege from the public schema.”

Then the default privileges, with the role spelled out: alter default privileges for role migrator in schema svc grant select, insert, update, delete on tables to svc_role. The ALTER DEFAULT PRIVILEGES page covers objects created by the role you name or by the role issuing the statement, and it does not reach objects that already exist. Issue it under your own login while migrations run as migrator and it enforces nothing whatsoever.

Name the wrong grantee there and the next migration quietly gives the other service’s role select on your tables. Postgres never hands a new table to everybody. By default, as the same schema page puts it, “users cannot access any objects in schemas they do not own”. People do that themselves, one convenience grant at a time.

How an outbox table and a relay replace the transaction you lost

An outbox replaces the transaction you lost by making the message part of the write: your own tables and one outbox row commit together, and a relay publishes that row afterwards.

create table outbox (
  id           bigint generated always as identity primary key,
  aggregate_id text  not null,
  event_type   text  not null,
  payload      jsonb not null,
  published_at timestamptz
);
create index outbox_unpublished on outbox (id) where published_at is null;

aggregate_id is the partition key that preserves ordering per entity. The partial index tracks the backlog rather than the table, so the relay’s query stays flat as published rows pile up.

Now the part that bites. Sequence values are handed out before commit, so a row with a lower id can commit after a higher one. Well down the CREATE SEQUENCE page, past the point where most people stop reading, the small print says that nextval and setval calls “are never rolled back” and that values “might be generated out of sequence when all the sessions are considered”. Keep a high-water mark in your relay and it will read straight past a late-committing row and never publish it. Claim rows with for update skip locked instead, which the SELECT documentation describes as a way “to avoid lock contention with multiple consumers accessing a queue-like table”, then publish, then set published_at in the same transaction.

Delivery is at-least-once, so the consumer does the other half. The relay carries outbox.id as the message id, and the consumer inserts that id into a processed_messages table under a unique constraint, in the same transaction as its own write, so a duplicate rolls everything back. Replay a day of events a second time and diff the result.

Events are also what replaces the joins. A query that crossed the new boundary becomes a composed call at the edge, an N+1 of network calls for a list of two hundred rows unless the owning service offers a batch read over the ids; or a read model in the consuming service, fed by these same events; or evidence that the two tables belong together. And the real test is the admin screen joining seven tables and the monthly finance export. Both belong in a read model, and the question they raise isn’t technical: how stale is acceptable? A relay polling once a second can’t be fresher than its own interval, which is a second or two in the ordinary case and minutes once a backlog forms. That is arithmetic off the poll interval rather than a measurement.

Which service should you extract from the monolith first?

Extract outbound messaging first: email, SMS, push notifications, outgoing webhooks. It is the cheapest place to learn idempotency because it has none by nature. Send twice and the recipient gets two emails, and the side effect is already outside the system where no transaction can reach it. “Have we already sent this?” is a question that service has to answer anyway, so its dedupe table earns its keep on the first day. Build it and you get three things at once: the outbox, the relay, and one idempotent consumer.

Don’t start with users or auth. That table is the most-joined in the schema and sits in the path of every request, where a mistake is a total outage. Verifying a signed token is a library call and a cached fetch of the issuer’s JSON Web Key Set, which RFC 7517 defines as a JSON object with a “keys” member.

Moving one route at a time, and when to stay put

Move one route at a time behind a reversible flag, and shadow the read paths only: call both, serve the old answer, log the difference. A write path can’t be shadowed: the side effects happen twice, so write ownership moves in one step or not at all. One writer, always. Expand and contract works inside one schema, where both shapes share a transaction; across two schemas what moves is write ownership, either by having the monolith call the new service or by replicating with change data capture until cut-over. Dual writes are a reconciliation job you have scheduled for yourself.

Work this way because projects get abandoned halfway through, and this shape still runs when they do.

The recommendation that costs us work is to stop here. Four engineers, one pipeline, nobody on call for a broker: a modular monolith gets most of the benefit at a fraction of the price. Enforce module imports in CI with import-linter, ArchUnit or ESLint’s no-restricted-imports, and give each module its own database role with usage on its own schema, so a cross-module query fails on a developer’s machine. That last one only bites where each module connects as its own role — with one shared application login the grants are decoration.

If you can’t name the schema each new service would own, you’re not ready to split. You’re ready to draw the boundaries in the code you already have.