In short
The most capable model is the wrong first question for Indian bank case work. The bank needs work it can explain to the RBI and to internal audit afterwards.
Across roughly 10,000 runs covering 150 distinct outcome scenarios, breaking work into defined steps reached 93% coverage and an 82% success rate. An open-ended approach on the best available model reached 79% and 71%. Defined steps on a cheaper model still beat it, at 91% and 80%.
Capability and accountability are not the same purchase
Ask a vendor how their system handles a difficult case and the answer is usually about the model. Newer, larger, better at reasoning.
Ask what happens when it gets one wrong, and the answer gets vaguer.
For a bank that has to explain its actions to the RBI, or to its own internal audit, the second question decides the purchase. A system that is right slightly more often but cannot tell you why is worth less than one that is right slightly less often and can.
This is not a philosophical preference. It is what an RBI systems audit asks for, and what an internal auditor writes up when it is missing.
The measured difference between the two approaches
Patched published a benchmark in 2026 comparing the two designs directly across roughly 10,000 runs and 150 distinct outcome scenarios.
| Approach | Coverage | Success rate |
|---|---|---|
| Open-ended, best model | 79% | 71% |
| Defined steps, best model | 93% | 82% |
| Defined steps, cheaper model | 91% | 80% |
The third row is the one worth sitting with. The step-based approach on a cheaper model beat the open-ended one on the best model available, on both coverage and success rate.
The design is doing more work than the model is. That is useful to know before anyone starts arguing about which model to license - or budgeting for the expensive one.
It uses AI inside controlled workflows, not as a free-form black box.
What explainable has to mean in practice
Explainable is used loosely in sales conversations. Here is the version that survives an audit.
When a case comes out wrong, you should be able to say which part of the work went wrong. Not that the system made an error, but the specific point at which it went off course, and what it was looking at when it did.
That is only possible if the work has parts. An open-ended process produces a single opaque outcome, and no amount of logging after the fact turns that into an explanation.
The test is concrete: hand an auditor one failed case and see whether anyone can name the step.
Where AI belongs, and where it does not
The line between interpretation and decision matters most to anyone in a risk function, and it gets blurred constantly.
AI is genuinely good at interpretation. Reading a complaint, working out what a notice is asking for, noticing that a KYC record and a transaction record refer to the same person. That is hard for conventional software and the improvement here is real.
Decisions and submissions are a different matter. Those should execute against explicit rules, validations and approval controls that the bank set and can inspect.
That is the difference between a system that made a judgement you cannot reconstruct, and one that applied a rule you wrote.
The audit trail is part of the product
In one Indian banking deployment, audit trail completeness and accuracy runs at 98%+. Every action records what was checked, what was submitted, what was skipped, and why.
The part about what was skipped is the one people forget to ask for. A system that logs its successes and stays quiet about the steps it did not perform is giving you a highlight reel.
Regulation is moving the same way. The Ministry of Home Affairs SOP of 2 January 2026 asks a bank that cannot complete a restoration direction to record its reasons in the Money Restoration Module. A reason has to exist before it can be recorded.
CaseLEA deployments in India run RBI SAR compliant and SOC 2 certified, hosted in Mumbai on GCP or AWS, with no transaction or user data stored on our server and data accessed read-only.
Five questions to ask before you buy
- When a case fails, can you tell me where? If the answer is a log file rather than a step, there is nothing to investigate.
- What does it do when it is not confident? A system that produces an answer for every case is describing a risk, not a feature.
- Who wrote the rules it applies? If the answer is the vendor, you have outsourced a decision you remain accountable for.
- What can it do without a person? The honest answer is specific and narrow, and set by your policy rather than their software.
- Show me the numbers on the hard cases. A blended rate carried by the easy ones tells you nothing about your backlog.
None of those questions is about the model, which is rather the point. They are also the five CaseLEA gets asked most often by Indian banks, and the five worth putting to whoever you end up buying from.
Why the cheaper model result matters commercially
The benchmark has a budget consequence most buyers miss.
If defined steps on a cheaper model reach 91% coverage and 80% success, against 79% and 71% for an open-ended approach on the best model, then the premium model is not what you are paying for. The structure is.
That changes the conversation with a vendor whose pricing is anchored to model cost. It also changes your risk position, because a system tied to one frontier model inherits that model's availability, pricing and deprecation schedule.
Model-agnostic matters here for a practical reason rather than a philosophical one. If the work is structured, the model underneath can be swapped when a better or cheaper one appears, and the bank keeps its rules either way.
The question an auditor will actually open with
Not how the model works. They will ask you to produce one case and explain it.
So the test worth running internally is the same one: take a case that went wrong last quarter and try to name the step. If your answer is that the system got it wrong, you have a problem that predates the audit.
A structured design gives you a step to name. That is the entire practical difference, and it is worth more than a few points of accuracy on a vendor's slide.
Sources
- Patched, published benchmark, 2026: 10,000 runs across 150 outcome scenarios, 79%/71% open-ended against 93%/82% and 91%/80% for defined steps.
- Patched data points, 2026: audit trail completeness and accuracy 98%+.
- CaseLEA deployment architecture documentation, 2026: RBI SAR and SOC 2, Mumbai region.
- Ministry of Home Affairs, Standard Operating Procedure for NCRP-CFCFRMS, Custody, Restoration of Money and Grievance Redressal, 2 January 2026: reasons for non-compliance recorded in the Money Restoration Module.
