Why OCR nails totals but fumbles merchant names
When a scan comes back wrong, it is worth knowing which of two separate steps let you down, because the fix is different in each case.
Recognition turns pixels into characters. Extraction decides which of those characters is the total, which is the date, and which is the shop’s name. They fail independently, and they fail in recognisable patterns.
Recognition failures look like garbage text
If the raw text is full of nonsense, that is a photography or paper-quality problem. Retaking the photo genuinely helps. Classic symptoms:
- Digits substituted for each other:
0andO,1andl,5andS,8andB - Whole lines missing where glare or a fold sat
- Character soup in place of a logo
The digit confusions are worth knowing about because they are silently plausible. 1O5.OO is obviously broken, but a misread 3 as 8 produces a number that looks entirely reasonable and is wrong by fifty. Amount fields deserve a second look for exactly this reason, and a good parser applies targeted repairs inside numeric tokens where it is safe to do so.
Extraction failures look like plausible text in the wrong box
Here the raw text is fine and the field assignment is wrong. Different fix: just correct the field. Retaking the photo will not help, because the photo was never the problem.
Totals are the easy case
Receipts label their totals, and the label vocabulary is small and predictable: total, grand total, amount due, balance due, amount payable. Position helps too, because the total is almost always near the bottom. Two independent signals agreeing is why totals land right so often.
The traps are decoy labels. Total Items: 3 and Total Savings 45.00 both contain the word “total” and neither is what you paid. So does Cash Tendered 500.00, which is what you handed over, not what the purchase cost. A parser has to exclude those explicitly.
A stronger trick is arithmetic: if subtotal plus tax equals the total, three readings corroborate each other and confidence in all three goes up. If they contradict, at least one is wrong and all three deserve suspicion.
Dates are easy except when they are not
Most date formats are unambiguous. 2026-04-03 and 3 Apr 2026 can only mean one thing, and 25/12/2026 settles itself because there is no twenty-fifth month.
03/04/2026 is the genuine problem. It is 3 April across most of the world and 4 March in the United States, and nothing in the receipt tells you which convention the printer used. The only available fallback is your locale, which is a guess, and a good scanner marks the field as uncertain rather than pretending otherwise.
Merchant names are hard
There is no label. The name is identified purely by being near the top and looking like a name, which is a weak signal competing with:
- Logos that OCR reads as random characters before reaching the actual name
TAX INVOICEorCUSTOMER COPYprinted above the business name- Addresses, phone numbers and tax registration numbers in the same header block
- Franchise branding that differs from the trading name on the receipt
Expect to correct this field more often than any other. It is also the field where the correction costs you least, since you know perfectly well where you were.
Line items are the hardest
Item rows defeat extraction in ways that are difficult to solve with rules:
- Long product names wrap onto a second line, splitting one item into two
- Modifier rows (
no ice,extra shot) look like items with no price - Multi-column layouts put quantity, unit rate and amount on one line, and telling them apart depends on column alignment that OCR does not reliably preserve
- Discount and offer lines are negative amounts that look like items
Treat extracted items as a starting draft. For most expense tracking, the total and the category are what you actually need.
What this means in practice
Look at the raw text when something is wrong. Garbage text means retake the photo. Clean text in the wrong field means just fix the field and move on. Knowing the difference saves you from retaking a photo that was fine to begin with.