In a take-home assessment, I wrote a withdraw endpoint. To stop duplicate requests, it first checked Redis for the idempotency key, and if the key wasn't there, it saved the key and processed the withdrawal.
It looked correct. It passed every test I wrote. And it had a bug that can pay the same request twice.
The bug has a name: check-then-act. Once you learn to see it, you find it everywhere: bonuses, coupons, refunds, follows, stock counts, daily limits.
The pattern
Check-then-act is any code that reads something, makes a decision, and then writes based on that decision, as separate steps:
const user = await User.findById(userId);
if (!user.bonusClaimed) { // check
await wallet.credit(userId, 500); // act
user.bonusClaimed = true;
await user.save();
}
Read it slowly and it's fine. The problem is that two requests don't run slowly, one after the other. They run at the same time:
Request A Request B
--------- ---------
read bonusClaimed → false
read bonusClaimed → false
credit 500
credit 500 ← paid twice
set bonusClaimed = true
set bonusClaimed = true
Both requests checked before either one acted. Both saw "not claimed yet". Both paid.
This is a race condition: the result depends on the timing of requests you don't control. A double-click, a mobile app retrying on a slow network, or a script sending two requests 50 milliseconds apart is enough.
Why your tests don't catch it
Most tests send one request, wait for the answer, then send the next one. Sequential requests never overlap, so the check always sees the result of the previous act. The code is only wrong when requests overlap, and normal tests never make them overlap.
The fix is never "add another check". Another check has the same gap. The fix is to make the check and the act a single step that the database does atomically.
This bug comes in three shapes, and each shape has its own fix.
Shape 1: changing a state that can change only once
Examples: claiming a one-time bonus, cancelling an order that might be shipping at the same moment, paying out a bet that might be cashed out at the same moment.
Fix: a conditional update. Put the check inside the WHERE clause:
const result = await client.query(
UPDATE users
SET bonus_claimed = true
WHERE id = $1 AND bonus_claimed = false,
[userId]
);
if (result.rowCount === 0) {
// someone else already claimed it: do nothing
return;
}
// only one request ever gets here
The database runs this update one at a time for that row. The first request changes bonus_claimed to true and updates one row. The second request waits for the first, then finds no row where bonus_claimed = false, and updates zero rows. The number of updated rows tells you who won.
The same idea works for any state machine. For example, an order can be cancelled or shipped, but not both:
UPDATE orders SET status = 'cancelled'
WHERE id = $1 AND status = 'pending';
And the shipping worker uses SET status = 'shipped' WHERE id = $1 AND status = 'pending'. Whichever runs first wins; the other one updates zero rows and knows it lost.
Do the credit in the same transaction as the update, so you never end up with the flag set but no money moved, or the other way around:
await client.query('BEGIN');
const claim = await client.query(
UPDATE users SET bonus_claimed = true
WHERE id = $1 AND bonus_claimed = false,
[userId]
);
if (claim.rowCount === 1) {
await client.query(
'UPDATE wallets SET balance = balance + 500 WHERE user_id = $1',
[userId]
);
}
await client.query('COMMIT');
In MongoDB the idea is identical. Put the condition in the filter:
const res = await User.updateOne(
{ _id: userId, bonusClaimed: false },
{ $set: { bonusClaimed: true } }
);
if (res.modifiedCount === 0) return; // already claimed
Shape 2: creating something that must exist only once
Examples: an idempotency key, "follow this user", "redeem this coupon once per customer".
This was my assessment bug. The check-then-act version looks like this:
const exists = await Follow.findOne({ userId, targetId });
if (!exists) {
await Follow.create({ userId, targetId });
}
Two requests both find nothing and both create. Now there are two identical rows.
Fix: a unique constraint, and let the insert itself be the check.
CREATE UNIQUE INDEX follows_user_target ON follows (user_id, target_id);
try {
await client.query(
'INSERT INTO follows (user_id, target_id) VALUES ($1, $2)',
[userId, targetId]
);
} catch (err) {
if (err.code !== '23505') throw err; // 23505 = unique_violation
// already following: treat it as success
}
Or, in PostgreSQL, without the exception:
INSERT INTO follows (user_id, target_id) VALUES ($1, $2)
ON CONFLICT (user_id, target_id) DO NOTHING;
Now it doesn't matter how many requests arrive at once. The database guarantees only one row exists, because the uniqueness check and the write happen inside the same operation.
For my withdraw endpoint, the fix was the same idea: insert the idempotency key into a table with a unique constraint inside the same transaction as the withdrawal. If the insert conflicts, it's a duplicate request, and I return the stored result instead of withdrawing again. The insert is the check.
In MongoDB, a unique index does the same job, and a duplicate insert fails with error code 11000.
Shape 3: a limit across many rows
Examples: "a customer can spend at most 5,000 per day", "a seller can list at most 100 products".
This one is trickier. The rule isn't about one row, it's about a sum or a count of many rows:
const { rows } = await client.query(
SELECT COALESCE(SUM(amount), 0) AS total
FROM payments
WHERE user_id = $1 AND created_at >= current_date,
[userId]
);
if (Number(rows[0].total) + amount <= 5000) {
await client.query('INSERT INTO payments ...');
}
Two payments of 3,000 arrive together. Both see a total of 0. Both insert. The customer spent 6,000.
You can't lock "today's payments" easily, because at the start of the day there are no rows to lock. So lock a row that always exists, exactly one per customer: the user's own row.
await client.query('BEGIN');
await client.query('SELECT id FROM users WHERE id = $1 FOR UPDATE', [userId]);
const { rows } = await client.query(
SELECT COALESCE(SUM(amount), 0) AS total
FROM payments
WHERE user_id = $1 AND created_at >= current_date,
[userId]
);
if (Number(rows[0].total) + amount > 5000) {
await client.query('ROLLBACK');
throw new Error('Daily limit reached');
}
await client.query('INSERT INTO payments (user_id, amount) VALUES ($1, $2)', [userId, amount]);
await client.query('COMMIT');
FOR UPDATE makes the second request wait at that line until the first one commits. Then it reads the new total, 3,000, and correctly rejects the second payment. Payments from different customers lock different rows, so they don't slow each other down.
(Note Number(...): the pg driver returns SUM of a numeric or bigint column as a string. "0" + 3000 is "03000" in JavaScript, which is its own kind of bug.)
How to spot it in code review
Look for this shape:
A read (
findOne,findById,SELECT,GETfrom Redis)An if based on what was read
A write (
create,save,UPDATE,INSERT,SET)
...with nothing between them that stops a second request: no conditional WHERE, no unique constraint, no row lock.
When you find it, ask one question: "What happens if two requests run this at the same moment?" Then pick the fix by shape:
to prove the fix works
Don't trust it; test it with real concurrency. Fire many identical requests at once and count the results:
test('bonus is credited only once under concurrency', async () => {
const results = await Promise.all(
Array.from({ length: 20 }, () => claimBonus(userId))
);
const wallet = await getWallet(userId);
expect(wallet.balance).toBe(500);
});
This needs a real database, not a mock, because the whole point is how the database handles concurrent writes.
I ran 20 concurrent requests against PostgreSQL 16, three times, for the naive and the fixed versions of the examples above:
20 concurrent requestsNaive check-then-actFixedOne-time bonus of 500balance 10,000 (paid 20 times)500Daily limit of 5,000, payments of 3,000total 42,000 to 60,0003,000Follow, must exist once1 row
One surprise: my first run showed no bug at all in the naive code. The connection pool was still opening connections, so the requests were queued and effectively ran one by one. Only after warming up the pool, opening all the connections before the test, did the requests truly overlap and the bug appear. If your concurrency test passes on buggy code, check that your requests really run at the same time.
The safety net
Even with the right fix, money code deserves a second line of defence:
A unique constraint on the ledger, for example on
(type, reference_id), so the same payout can't be written twice even if a bug slips through.A regular reconciliation job that checks every balance equals the sum of its ledger entries, and alerts when it doesn't.
The one-line version
If your code says "if it hasn't happened yet, make it happen", make sure the database checks and acts in one step: a conditional update, a unique constraint, or a lock.
The assessment taught me this the hard way: my code was correct for one request at a time, and wrong for the way real users actually send requests.