Automated decision systems don't fail loudly. They fail in the gap between the rules the business believes it enforces and the states the architecture actually permits. A limit that lives in a policy document, a spreadsheet, or an analyst's head is not a limit. It's a suggestion the system is free to ignore the moment load, latency, or an unanticipated input says otherwise.
The uncomfortable claim is this: if a forbidden decision is technically reachable, it will eventually be reached. Not might — will. Scale doesn't introduce new rules. It removes the human reflexes that were quietly compensating for the rules that were never encoded. The manual review that caught the bad approval at a thousand decisions a day is not slow at a million decisions a day. It is simply absent.
Consider what that means on a credit platform that clears more than 200 million decisions a day. There is no human in the loop — not as a fallback, not as an exception path, not anywhere. At that volume, the only thing standing between the business and a decision it never authorized is whether the limit was built as an invariant the architecture cannot cross, or left as a check the system performs right up until the first edge case slips past it.
This is the difference between a forbidden state and a discouraged one. A forbidden state is unreachable by construction — the system can no more enter it than a balance can drop below a floor that doesn't exist in the schema. A discouraged state is reachable, and at 200 million decisions a day, "reachable" means "happening continuously, somewhere in the portfolio, right now."
So the question every operator of an automated decision system should be able to answer is brutally simple: which of your business limits are architecture, and which are hope? The first kind holds under any load. The second kind holds until the load finds the gap — and then it was never a limit at all, just a story you told yourself about what the system would never do.