System Design + Concurrency

How would you safely transfer money between accounts?

A safe money transfer system must handle concurrency, consistency, deadlocks, retries, duplicate requests, and distributed system failures while ensuring money is never lost or duplicated.

System DesignConcurrencyTransactionsDistributed SystemsConsistencyBackend

The Real Problem

This question gets asked a lot. Transferring money sounds simple:

java
from.balance -= amount;
to.balance += amount;

But then we need to run several hundreds or thousands of these transfers concurrently. And some of these may involve the same account. In one transfer, account may be credited to account 100. And in another, money may be debited from the same account. Once multiple threads, retries, failures, or distributed systems are involved, the problem becomes much harder. Financial systems have to guarantee such operations. You don't want extra money debited from your account (or charged to your credit card) even by mistake.

The core requirement is:

Money must never be lost, duplicated, or partially transferred.

The Concurrency Problem

Imagine two transfers happening simultaneously:

java
Thread A:
Transfer $100 from Account X to Account Y

Thread B:
Transfer $50 from Account X to Account Z

Without coordination, both threads may read the same balance and overwrite each other's updates.

Unsafe

Both threads read balance = 1000
Both modify independently
One update may overwrite another

Safe

Coordinate access
One transfer updates at a time
Correct balances preserved

Naive Locking Solution

java
synchronized(fromAccount) {
    synchronized(toAccount) {

        from.withdraw(amount);
        to.deposit(amount);
    }
}

This protects the critical section, but introduces another major problem:

Deadlocks.

The Deadlock Problem

Imagine:

java
Thread A:
locks Account X
waits for Account Y

Thread B:
locks Account Y
waits for Account X

Both threads wait forever.

Thread A: X → waiting for Y
Thread B: Y → waiting for X

The Classic Interview Solution

Lock accounts in a consistent order, always locking the smaller account id first.

java
Account first =
    from.id < to.id ? from : to;

Account second =
    from.id < to.id ? to : from;

synchronized(first) {
    synchronized(second) {

        from.withdraw(amount);
        to.deposit(amount);
    }
}

This avoids circular waiting because every thread acquires locks in the same global order.

Consistent lock ordering is one of the most important concurrency interview patterns.

Why This Still Is Not Enough

The JVM locking solution only solves part of the problem.

Real systems also face:

  • duplicate requests
  • network retries
  • partial failures
  • multiple application servers
  • database crashes
  • distributed transactions

If two different servers process the same transfer request, JVM locks on one server do not protect the other server.

The Idempotency Problem

Imagine a client retries the same transfer request because it timed out waiting for a response.

java
POST /transfer

Idempotency-Key: abc123

The original transfer may actually have succeeded already.

Without idempotency handling, the retry could transfer the money a second time.

Correct distributed systems often require both locking and idempotency protection.

Database Transactions

In real financial systems, correctness is usually enforced primarily through database transactions.

java
BEGIN TRANSACTION

UPDATE accounts
SET balance = balance - 100
WHERE id = 1;

UPDATE accounts
SET balance = balance + 100
WHERE id = 2;

COMMIT;

The database can then provide:

  • atomicity
  • isolation
  • rollback on failure
  • durability

Important Production Insight

In-memory locking is often appropriate for interview discussion and single-JVM coordination.

Real financial systems typically rely heavily on transactional databases, idempotency keys, durable ledgers, and carefully designed failure handling.

The Interview-Friendly Explanation

A safe money transfer system must coordinate concurrent updates while preventing lost updates, deadlocks, duplicate transfers, and partial failures. A common JVM-level approach is consistent lock ordering to avoid deadlocks. In distributed systems, correctness also requires database transactions, idempotency protection, and careful failure handling.

Common Interview Follow-Ups

Why can deadlocks happen?

Two threads may acquire locks in opposite order and wait forever for each other.

How does consistent lock ordering help?

If every thread acquires locks in the same global order, circular waiting cannot occur.

Why are JVM locks insufficient in distributed systems?

Locks only coordinate threads inside the same JVM. Multiple servers may still process the same accounts concurrently.

Why are idempotency keys important?

Retries can accidentally execute the same transfer multiple times. Idempotency keys help detect duplicate requests safely.

Would a database transaction alone solve everything?

Transactions solve many consistency problems, but distributed retries, messaging, and external side effects may still require idempotency and additional coordination.

Final Takeaway

The money transfer problem is really a correctness-under-concurrency problem. The interesting part is not subtracting and adding numbers. The interesting part is guaranteeing consistency when multiple threads, servers, retries, and failures exist simultaneously.